Commit Graph

5036 Commits

Author SHA1 Message Date
Mike Hunhoff
21887d1ec6 vmray: merge upstream 2024-06-18 15:43:19 -06:00
Mike Hunhoff
85a85e99bf vmray: emit recorded artifacts as strings 2024-06-18 15:38:44 -06:00
Mike Hunhoff
d26a806647 vmray: update scripts/show-features.py to emit process name from extractor 2024-06-18 14:59:29 -06:00
Mike Hunhoff
e5fa800ffb vmray: emit empty thread features 2024-06-18 14:45:08 -06:00
Mike Hunhoff
b3ebf80d9b vmray: emit process name 2024-06-18 14:41:47 -06:00
Mike Hunhoff
8f32b7fc65 vmray: emit process handles 2024-06-18 14:32:11 -06:00
Mike Hunhoff
f3d69529b0 vmray: invoke VMRay feature extractor from capa.main 2024-06-18 13:27:40 -06:00
ygasparis
1975b6455c extract import / export symbols from stripped elf binaries (#2142) 2024-06-18 12:38:02 -06:00
Mike Hunhoff
51656fe825 vmray: merge upstream 2024-06-18 10:53:32 -06:00
Capa Bot
1360e08389 Sync capa-testfiles submodule 2024-06-18 11:00:26 +00:00
dependabot[bot]
40061b3c42 build(deps): bump viv-utils from 0.7.9 to 0.7.11 (#2150) 2024-06-18 06:36:10 +02:00
dependabot[bot]
45fca7adea build(deps): bump python-flirt from 0.8.6 to 0.8.10 (#2151) 2024-06-18 06:35:50 +02:00
Mike Hunhoff
654804878f vmray: clean up global_.py debug output 2024-06-14 09:34:59 -06:00
Mike Hunhoff
8b913e0544 vmray: extract global features for PE files 2024-06-14 09:32:02 -06:00
Moritz
482686ab81 Merge pull request #2147 from mandiant/release/v710
bump to v7.1.0
v7.1.0
2024-06-14 12:56:46 +02:00
mr-tz
67f8c4d28c bump to v7.1.0 2024-06-14 09:06:04 +00:00
Capa Bot
3f151a342b Sync capa rules submodule 2024-06-14 09:02:02 +00:00
Mike Hunhoff
00cb7924e1 vmray: clean up pydantic models and add sample hash extraction 2024-06-13 17:02:50 -06:00
Mike Hunhoff
7e079d4d35 vmray: restrict analysis to PE files 2024-06-13 16:52:25 -06:00
Mike Hunhoff
346a0693ad vmray: clean up VMRayAnalysis 2024-06-13 16:48:12 -06:00
Mike Hunhoff
8d3f032434 vmray: clean up pydantic models and implement base address extraction 2024-06-13 16:43:23 -06:00
Mike Hunhoff
7d0ac71353 vmray: cleanup pydantic models and implement file section extraction 2024-06-13 16:31:12 -06:00
Mike Hunhoff
970b184651 vmray: add stubs for file imports 2024-06-13 14:20:11 -06:00
Mike Hunhoff
ca02b4ac7c vmray: expand extractor to emit file export features 2024-06-13 14:12:41 -06:00
Mike Hunhoff
a797405648 vmray: add example models for summary_v2.json 2024-06-13 12:54:59 -06:00
mr-tz
a9dafe283c example using pydantic-xml to parse flog.xml 2024-06-13 16:37:45 +00:00
dependabot[bot]
e87e8484b6 build(deps): bump ruff from 0.4.7 to 0.4.8 (#2139)
Bumps [ruff](https://github.com/astral-sh/ruff) from 0.4.7 to 0.4.8.
- [Release notes](https://github.com/astral-sh/ruff/releases)
- [Changelog](https://github.com/astral-sh/ruff/blob/main/CHANGELOG.md)
- [Commits](https://github.com/astral-sh/ruff/compare/v0.4.7...v0.4.8)

---
updated-dependencies:
- dependency-name: ruff
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Willi Ballenthin <wballenthin@google.com>
2024-06-13 13:24:33 +02:00
Willi Ballenthin
8726de0d65 ELF: Detect OS from Go binaries (#1987)
* elf: read segment memory size

* elf: add routine to read mapped memory

* elf: better detect OS for binaries compiled by Go

* elf: guess OS from Go source filenames

* changelog

* elf: mypy

* merge

* elf: add OS detection based on vDSO strings

* elf: document VTGrep searches

* elf: describe further technique to identify Go binaries

* elf: search for `.go.buildinfo` section via @yelhamer

* black

* elf: detect Alpine Linux ident

* elf: log interest symtab entries

* tests: add test for OS detection by Go buildinfo

* loader: handle missing viv modules

* pre-commit: run deptry before tests (which are slow)

* loader: describe removing viv symbolic switch solver

* pyproject: add PyGithub for deptry

* black
2024-06-13 13:23:47 +02:00
Moritz
7d1512a3de Merge pull request #2146 from mandiant/fix/2145
fix black and mypy
2024-06-13 11:49:18 +02:00
Capa Bot
73d76d7aba Sync capa-testfiles submodule 2024-06-13 09:30:44 +00:00
mr-tz
1febb224d1 add scripts dependency group 2024-06-13 07:50:58 +00:00
Moritz
e3ea60d354 Apply suggestions from code review
Co-authored-by: Willi Ballenthin <wballenthin@google.com>
2024-06-13 09:36:12 +02:00
mr-tz
93cd1dcedd add scripts to install step 2024-06-12 15:24:10 +00:00
mr-tz
7b0270980d add capa2sarif dependencies 2024-06-12 15:19:24 +00:00
mr-tz
cce7774705 add scripts section 2024-06-12 15:17:31 +00:00
mr-tz
9ec9a6f439 fix mypy issues 2024-06-12 09:32:03 +00:00
mr-tz
97a3fba2c9 fix black 2024-06-12 09:24:16 +00:00
Capa Bot
893352756f Sync capa rules submodule 2024-06-11 18:11:24 +00:00
malwarefrank
0cc06aa83d dnfile 0.15.0 changed API (#2037)
* dnfile 0.15.0 changed API

* deduplicate str() calls and isort fixes

* revert accidental change to imports ordering

* add table variable annotation

---------

Co-authored-by: Moritz <mr-tz@users.noreply.github.com>
Co-authored-by: mr-tz <moritz.raabe@mandiant.com>
2024-06-11 11:46:09 -06:00
dependabot[bot]
1888d0e7e3 build(deps): bump setuptools from 69.5.1 to 70.0.0 (#2135)
Bumps [setuptools](https://github.com/pypa/setuptools) from 69.5.1 to 70.0.0.
- [Release notes](https://github.com/pypa/setuptools/releases)
- [Changelog](https://github.com/pypa/setuptools/blob/main/NEWS.rst)
- [Commits](https://github.com/pypa/setuptools/compare/v69.5.1...v70.0.0)

---
updated-dependencies:
- dependency-name: setuptools
  dependency-type: direct:production
  update-type: version-update:semver-major
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2024-06-11 15:03:56 +02:00
ReWithMe
52e24e560b FEAT(capa2sarif) Add SARIF conversion script from json output (#2093)
* feat(capa2sarif): add new sarif conversion script converting json output to sarif schema, update dependencies, and update changelog

* fix(capa2sarif): removing copy and paste transcription errors

* fix(capa2sarif): remove dependencies from pyproject toml to guarded import statements

* chore(capa2sarif): adding node in readme specifying dependency and applied auto formatter for styling

* style(capa2sarif): applied import sorting and fixed typo in invocations function

* test(capa2sarif): adding simple test for capa to sarif conversion script using existing result document

* style(capa2sarif): fixing typo in version string in usage

* style(capa2sarif): isort failing due to reordering of typehint imports

* style(capa2sarif): fixing import order as isort on local machine was not updating code

---------

Co-authored-by: ReversingWithMe <ryanv@rewith.me>
Co-authored-by: Willi Ballenthin <wballenthin@google.com>
2024-06-11 15:01:26 +02:00
dependabot[bot]
c97d2d7244 build(deps): bump pyinstaller from 6.7.0 to 6.8.0 (#2138)
Bumps [pyinstaller](https://github.com/pyinstaller/pyinstaller) from 6.7.0 to 6.8.0.
- [Release notes](https://github.com/pyinstaller/pyinstaller/releases)
- [Changelog](https://github.com/pyinstaller/pyinstaller/blob/develop/doc/CHANGES.rst)
- [Commits](https://github.com/pyinstaller/pyinstaller/compare/v6.7.0...v6.8.0)

---
updated-dependencies:
- dependency-name: pyinstaller
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2024-06-11 14:36:58 +02:00
Willi Ballenthin
833ec47170 relax pyproject dependency versions and introduce requirements.txt (#2132)
* relax pyproject dependency versions and introduce requirements.txt

closes #2053
closes #2079

* pyproject: document dev/build profile dependency policies

* changelog

* doc: installation: describe requirements.txt usage

* pyproject: don't use dnfile 0.15 yet

---------

Co-authored-by: Moritz <mr-tz@users.noreply.github.com>
2024-06-11 14:29:34 +02:00
Willi Ballenthin
07ae30875c features: add aarch64 arch (#2144)
* features: add aarch64 arch
2024-06-11 09:36:04 +02:00
Willi Ballenthin
76a4a5899f test_scripts: avoid unsupported logic combinations 2024-06-07 05:54:49 +02:00
Willi Ballenthin
4d81b7ab98 rules: add references to existing issues 2024-06-07 05:54:49 +02:00
Willi Ballenthin
b068890fa6 rules: match: optimize rule matching by better indexing rule by features
Implement the "tighten rule pre-selection" algorithm described here:
https://github.com/mandiant/capa/issues/2063#issuecomment-2100498720

In summary:

> Rather than indexing all features from all rules,
> we should pick and index the minimal set (ideally, one) of
> features from each rule that must be present for the rule to match.
> When we have multiple candidates, pick the feature that is
> probably most uncommon and therefore "selective".

This seems to work pretty well. Total evaluations when running against
mimikatz drop from 19M to 1.1M (wow!) and capa seems to match around
3x more functions per second (wow wow).

When doing large scale runs, capa is about 25% faster when using the
vivisect backend (analysis heavy) or 3x faster when using the
upcoming BinExport2 backend (minimal analysis).
2024-06-07 05:54:49 +02:00
dependabot[bot]
d10d2820b2 build(deps): bump types-requests from 2.32.0.20240523 to 2.32.0.20240602
Bumps [types-requests](https://github.com/python/typeshed) from 2.32.0.20240523 to 2.32.0.20240602.
- [Commits](https://github.com/python/typeshed/commits)

---
updated-dependencies:
- dependency-name: types-requests
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
2024-06-06 10:43:08 +02:00
Capa Bot
5239e40beb Sync capa-testfiles submodule 2024-06-05 12:15:41 +00:00
Capa Bot
bce8f7b5e5 Sync capa rules submodule 2024-06-05 09:40:58 +00:00