introduce script to detect 3P backends

ref #2376

CHANGELOG.md

@@ -2,43 +2,7 @@
 
 ## master (unreleased)
 
-### New Features
-
-### Breaking Changes
-
-### New Rules (0)
-
--
-
-### Bug Fixes
-
-### capa explorer IDA Pro plugin
-
-### Development
-
-### Raw diffs
-- [capa v7.3.0...master](https://github.com/mandiant/capa/compare/v7.3.0...master)
-- [capa-rules v7.3.0...master](https://github.com/mandiant/capa-rules/compare/v7.3.0...master)
-
-## v7.3.0
-This release comes with the following three major enhancements:
-
-### 1. Support for VMRay sandbox analysis archives
-Unlock powerful malware analysis with capa's new [VMRay sandbox](https://www.vmray.com/) integration! Simply provide a VMRay analysis archive, and capa will automatically extract and match capabilities to streamline your workflow. This is the second support for the analysis of dynamic analysis results after [CAPE](https://www.mandiant.com/resources/blog/dynamic-capa-executable-behavior-cape-sandbox).
-
-### 2. Support for BinExport files generated by Ghidra
-[BinExport](https://github.com/google/binexport) files store disassembled data into a Protocol Buffer format. capa now supports the analysis of BinExport files generated by Ghidra. Using Ghidra and the BinExport file format users can now analyze ARM (AARCH64) ELF files targeting Android.
-
-### 3. Introducing the capa rules website
-You can now browse capa's default rule set at https://mandiant.github.io/capa/rules. In modern terminals the CLI capa tool hyperlinks to resources on the web, including entries on the capa rules website.
-Furthermore, https://mandiant.github.io/capa provides a landing page for the capa tool project.
-
-### Additional updates
-- [capa Explorer Web](https://mandiant.github.io/capa/explorer/) received several enhancements and bug fixes.
-- Support for the IDA Pro 9.0 IDAPython API while keeping compatibility to older IDA Pro versions
-- Six rules have been added and two rules have been updated
-
-Thanks to @r-sm2024 for their contribution in https://github.com/mandiant/capa/pull/2155 and their further work. And of course a big thanks to the community for reporting issues, participating in discussions, and supporting the capa tool and capa rules.
+Unlock powerful malware analysis with capa's new [VMRay sandbox](https://www.vmray.com/) integration! Simply provide a VMRay analysis archive, and capa will automatically extract and match capabilties, streamlining your workflow.
 
 ### New Features
 - regenerate ruleset cache automatically on source change (only in dev mode) #2133 @s-ff
@@ -47,10 +11,13 @@ Thanks to @r-sm2024 for their contribution in https://github.com/mandiant/capa/p
 - add .justfile @williballenthin #2325
 - dynamic: add support for VMRay dynamic sandbox traces #2208 @mike-hunhoff @r-sm2024 @mr-tz
 - cli: use modern terminal features to hyperlink to the rules website #2337 @williballenthin
+- update IDAPython to IDA Pro 9.0 @mr-tz
 - support analyzing BinExport2 files generated by Ghidra #1950 @williballenthin @mehunhoff @mr-tz
 - add support for Android OS #1950 @williballenthin @mehunhoff @mr-tz
 - add support for aarch64 architecture via BinExport2 backend #1950 @williballenthin @mehunhoff @mr-tz
 
+### Breaking Changes
+
 ### New Rules (6)
 
 - linking/static/minhook/linked-against-minhook jakub.jozwiak@mandiant.com
@@ -59,6 +26,7 @@ Thanks to @r-sm2024 for their contribution in https://github.com/mandiant/capa/p
 - host-interaction/network/traffic/filter/enumerate-network-filters-via-wfp-api jakub.jozwiak@mandiant.com
 - nursery/check-thread-suspend-count-exceeded ervinocampo@google.com
 - nursery/create-thread-bypassing-process-freeze ervinocampo@google.com
+-
 
 ### Bug Fixes
 
@@ -66,14 +34,15 @@ Thanks to @r-sm2024 for their contribution in https://github.com/mandiant/capa/p
 - fix code path reference in linter @williballenthin #2350
 
 ### capa explorer IDA Pro plugin
-- update IDAPython to IDA Pro 9.0 @mr-tz
-- fix byte search IDA Pro 7.5 compatibility @mr-tz #2371
+- fix byte search IDA Pro 7.5 compatability @mr-tz #2371
+
+### Development
 
 ### Raw diffs
-- [capa v7.2.0...v7.3.0](https://github.com/mandiant/capa/compare/v7.2.0...v7.3.0)
-- [capa-rules v7.2.0...v7.3.0](https://github.com/mandiant/capa-rules/compare/v7.2.0...v7.3.0)
+- [capa v7.2.0...master](https://github.com/mandiant/capa/compare/v7.2.0...master)
+- [capa-rules v7.2.0...master](https://github.com/mandiant/capa-rules/compare/v7.2.0...master)
 
-## v7.2.0
+### v7.2.0
 capa v7.2.0 introduces a first version of capa explorer web: a web-based user interface to inspect capa results using your browser. Users can inspect capa result JSON documents in an online web instance or a standalone HTML page for offline usage. capa explorer supports interactive exploring of capa results to make it easier to understand them. Users can filter, sort, and see the details of all identified capabilities. capa explorer web was worked on by @s-ff as part of a [GSoC project](https://summerofcode.withgoogle.com/programs/2024/projects/cR3hjbsq), and it is available at https://mandiant.github.io/capa/explorer/#/.
 
 This release also adds a feature extractor for output from the DRAKVUF sandbox. Now, analysts can pass the resulting `drakmon.log` file to capa and extract capabilities from the artifacts captured by the sandbox. This feature extractor will also be added to the DRAKVUF sandbox as a post-processing script, and it was worked on by @yelhamer as part of a [GSoC project](https://summerofcode.withgoogle.com/programs/2024/projects/fCnBGuEC).

capa/features/extractors/binja/find_binja_api.py

@@ -20,9 +20,9 @@ from importlib import util
 spec = util.find_spec('binaryninja')
 if spec is not None:
     if len(spec.submodule_search_locations) > 0:
-            path = Path(spec.submodule_search_locations[0])
-            # encode the path with utf8 then convert to hex, make sure it can be read and restored properly
-            print(str(path.parent).encode('utf8').hex())
+        path = Path(spec.submodule_search_locations[0])
+        # encode the path with utf8 then convert to hex, make sure it can be read and restored properly
+        print(str(path.parent).encode('utf8').hex())
 """
 
 

capa/version.py

@@ -5,7 +5,7 @@
 # Unless required by applicable law or agreed to in writing, software distributed under the License
 #  is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
 # See the License for the specific language governing permissions and limitations under the License.
-__version__ = "7.3.0"
+__version__ = "7.2.0"
 
 
 def get_major_version():

pyproject.toml

@@ -183,6 +183,7 @@ known_first_party = [
     "binaryninja",
     "flirt",
     "ghidra",
+    "ida",
     "ida_ida",
     "ida_bytes",
     "ida_entry",

scripts/detect-backends.py

@@ -0,0 +1,292 @@
+import os
+import sys
+import json
+import logging
+import importlib.util
+from typing import Optional
+from pathlib import Path
+
+import rich
+import rich.table
+
+logger = logging.getLogger(__name__)
+
+
+def get_desktop_entry(name: str) -> Optional[Path]:
+    """
+    Find the path for the given XDG Desktop Entry name.
+
+    Like:
+
+        >> get_desktop_entry("com.vector35.binaryninja.desktop")
+        Path("~/.local/share/applications/com.vector35.binaryninja.desktop")
+    """
+    assert sys.platform in ("linux", "linux2")
+    assert name.endswith(".desktop")
+
+    default_data_dirs = f"/usr/share/applications:{Path.home()}/.local/share"
+    data_dirs = os.environ.get("XDG_DATA_DIRS", default_data_dirs)
+    for data_dir in data_dirs.split(":"):
+        applications = Path(data_dir) / "applications"
+        for application in applications.glob("*.desktop"):
+            if application.name == name:
+                return application
+
+    return None
+
+
+def get_binaryninja_path(desktop_entry: Path) -> Optional[Path]:
+    # from: Exec=/home/wballenthin/software/binaryninja/binaryninja %u
+    # to:        /home/wballenthin/software/binaryninja/
+    for line in desktop_entry.read_text(encoding="utf-8").splitlines():
+        if not line.startswith("Exec="):
+            continue
+
+        if not line.endswith("binaryninja %u"):
+            continue
+
+        binaryninja_path = Path(line[len("Exec=") : -len("binaryninja %u")])
+        if not binaryninja_path.exists():
+            return None
+
+        return binaryninja_path
+
+    return None
+
+
+def find_binaryninja() -> Optional[Path]:
+    if sys.platform == "linux" or sys.platform == "linux2":
+        # ok
+        logger.debug("detected OS: linux")
+    elif sys.platform == "darwin":
+        raise NotImplementedError(f"unsupported platform: {sys.platform}")
+    elif sys.platform == "win32":
+        raise NotImplementedError(f"unsupported platform: {sys.platform}")
+    else:
+        raise NotImplementedError(f"unsupported platform: {sys.platform}")
+
+    desktop_entry = get_desktop_entry("com.vector35.binaryninja.desktop")
+    if not desktop_entry:
+        return None
+    logger.debug("found Binary Ninja application: %s", desktop_entry)
+
+    binaryninja_path = get_binaryninja_path(desktop_entry)
+    if not binaryninja_path:
+        return None
+    logger.debug("found Binary Ninja installation: %s", binaryninja_path)
+
+    module_path = binaryninja_path / "python"
+    if not module_path.exists():
+        return None
+
+    if not (module_path / "binaryninja" / "__init__.py").exists():
+        return None
+
+    return module_path
+
+
+def is_binaryninja_installed() -> bool:
+    """Is the binaryninja module ready to import?"""
+    try:
+        return importlib.util.find_spec("binaryninja") is not None
+    except ModuleNotFoundError:
+        return False
+
+
+def has_binaryninja() -> bool:
+    if is_binaryninja_installed():
+        logger.debug("found installed Binary Ninja API")
+        return True
+
+    logger.debug("Binary Ninja API not installed, searching...")
+
+    binaryninja_path = find_binaryninja()
+    if not binaryninja_path:
+        logger.debug("failed to find Binary Ninja installation")
+
+    logger.debug("found Binary Ninja API: %s", binaryninja_path)
+    return binaryninja_path is not None
+
+
+def load_binaryninja() -> bool:
+    try:
+        import binaryninja
+
+        return True
+    except ImportError:
+        binaryninja_path = find_binaryninja()
+        if not binaryninja_path:
+            return False
+
+        sys.path.append(binaryninja_path.absolute().as_posix())
+        try:
+            import binaryninja  # noqa: F401 unused import
+
+            return True
+        except ImportError:
+            return False
+
+
+def is_vivisect_installed() -> bool:
+    try:
+        return importlib.util.find_spec("vivisect") is not None
+    except ModuleNotFoundError:
+        return False
+
+
+def load_vivisect() -> bool:
+    try:
+        import vivisect  # noqa: F401 unused import
+
+        return True
+    except ImportError:
+        return False
+
+
+def is_idalib_installed() -> bool:
+    try:
+        return importlib.util.find_spec("ida") is not None
+    except ModuleNotFoundError:
+        return False
+
+
+def get_idalib_user_config_path() -> Optional[Path]:
+    """Get the path to the user's config file based on platform following IDA's user directories."""
+    # derived from `py-activate-idalib.py` from IDA v9.0 Beta 4
+
+    if sys.platform == "win32":
+        # On Windows, use the %APPDATA%\Hex-Rays\IDA Pro directory
+        config_dir = Path(os.getenv("APPDATA")) / "Hex-Rays" / "IDA Pro"
+    else:
+        # On macOS and Linux, use ~/.idapro
+        config_dir = Path.home() / ".idapro"
+
+    # Return the full path to the config file (now in JSON format)
+    user_config_path = config_dir / "ida-config.json"
+    if not user_config_path.exists():
+        return None
+    return user_config_path
+
+
+def find_idalib() -> Optional[Path]:
+    config_path = get_idalib_user_config_path()
+    if not config_path:
+        return None
+
+    config = json.loads(config_path.read_text(encoding="utf-8"))
+
+    try:
+        ida_install_dir = Path(config["Paths"]["ida-install-dir"])
+    except KeyError:
+        return None
+
+    if not ida_install_dir.exists():
+        return None
+
+    libname = {
+        "win32": "idalib.dll",
+        "linux": "libidalib.so",
+        "linux2": "libidalib.so",
+        "darwin": "libidalib.dylib",
+    }[sys.platform]
+
+    if not (ida_install_dir / "ida.hlp").is_file():
+        return None
+
+    if not (ida_install_dir / libname).is_file():
+        return None
+
+    idalib_path = ida_install_dir / "idalib" / "python"
+    if not idalib_path.exists():
+        return None
+
+    if not (idalib_path / "ida" / "__init__.py").is_file():
+        return None
+
+    return idalib_path
+
+
+def has_idalib() -> bool:
+    if is_idalib_installed():
+        logger.debug("found installed IDA idalib API")
+        return True
+
+    logger.debug("IDA idalib API not installed, searching...")
+
+    idalib_path = find_idalib()
+    if not idalib_path:
+        logger.debug("failed to find IDA idalib installation")
+
+    logger.debug("found IDA idalib API: %s", idalib_path)
+    return idalib_path is not None
+
+
+def load_idalib() -> bool:
+    try:
+        import ida
+
+        return True
+    except ImportError:
+        idalib_path = find_idalib()
+        if not idalib_path:
+            return False
+
+        sys.path.append(idalib_path.absolute().as_posix())
+        try:
+            import ida  # noqa: F401 unused import
+
+            return True
+        except ImportError:
+            return False
+
+
+def main():
+    logging.basicConfig(level=logging.INFO)
+
+    table = rich.table.Table()
+    table.add_column("backend")
+    table.add_column("already installed?")
+    table.add_column("found?")
+    table.add_column("loads?")
+
+    if True:
+        row = ["vivisect"]
+        if is_vivisect_installed():
+            row.append("True")
+            row.append("-")
+        else:
+            row.append("False")
+            row.append("False")
+
+        row.append(str(load_vivisect()))
+        table.add_row(*row)
+
+    if True:
+        row = ["Binary Ninja"]
+        if is_binaryninja_installed():
+            row.append("True")
+            row.append("-")
+        else:
+            row.append("False")
+            row.append(str(find_binaryninja() is not None))
+
+        row.append(str(load_binaryninja()))
+        table.add_row(*row)
+
+    if True:
+        row = ["IDA idalib"]
+        if is_idalib_installed():
+            row.append("True")
+            row.append("-")
+        else:
+            row.append("False")
+            row.append(str(find_idalib() is not None))
+
+        row.append(str(load_idalib()))
+        table.add_row(*row)
+
+    rich.print(table)
+
+
+if __name__ == "__main__":
+    main()