Commit Graph

4994 Commits

Author SHA1 Message Date
Mike Hunhoff
fb72e5e8fd fix lints 2024-06-10 14:49:03 -06:00
Willi Ballenthin
89ebfe6b0c features: add BinExport2 declarations 2024-06-10 14:48:36 -06:00
Willi Ballenthin
76a4a5899f test_scripts: avoid unsupported logic combinations 2024-06-07 05:54:49 +02:00
Willi Ballenthin
4d81b7ab98 rules: add references to existing issues 2024-06-07 05:54:49 +02:00
Willi Ballenthin
b068890fa6 rules: match: optimize rule matching by better indexing rule by features
Implement the "tighten rule pre-selection" algorithm described here:
https://github.com/mandiant/capa/issues/2063#issuecomment-2100498720

In summary:

> Rather than indexing all features from all rules,
> we should pick and index the minimal set (ideally, one) of
> features from each rule that must be present for the rule to match.
> When we have multiple candidates, pick the feature that is
> probably most uncommon and therefore "selective".

This seems to work pretty well. Total evaluations when running against
mimikatz drop from 19M to 1.1M (wow!) and capa seems to match around
3x more functions per second (wow wow).

When doing large scale runs, capa is about 25% faster when using the
vivisect backend (analysis heavy) or 3x faster when using the
upcoming BinExport2 backend (minimal analysis).
2024-06-07 05:54:49 +02:00
dependabot[bot]
d10d2820b2 build(deps): bump types-requests from 2.32.0.20240523 to 2.32.0.20240602
Bumps [types-requests](https://github.com/python/typeshed) from 2.32.0.20240523 to 2.32.0.20240602.
- [Commits](https://github.com/python/typeshed/commits)

---
updated-dependencies:
- dependency-name: types-requests
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
2024-06-06 10:43:08 +02:00
Capa Bot
5239e40beb Sync capa-testfiles submodule 2024-06-05 12:15:41 +00:00
Capa Bot
bce8f7b5e5 Sync capa rules submodule 2024-06-05 09:40:58 +00:00
Capa Bot
0cf9365816 Sync capa-testfiles submodule 2024-06-05 08:49:12 +00:00
Fariss
30d23c4d97 render maec/* fields (#2087)
* Render maec/* fields

* add test for render_maec

---------

Co-authored-by: Soufiane Fariss <soufiane.fariss@um5s.net.ma>
Co-authored-by: Moritz <mr-tz@users.noreply.github.com>
2024-06-05 10:31:13 +02:00
Capa Bot
b3ed42f5f9 Sync capa-testfiles submodule 2024-06-04 21:25:58 +00:00
Fariss
508a09ef25 include rule caching in PyInstaller build process (#2097)
* include rule caching in PyInstaller build process

The following commit introduces a new function that caches the capa
rule set, so that users don't have to manually run ./scripts/cache-
ruleset.py, before running pyinstaller.

* ci: omit Cache rule set step from build.yml workflow

* refactor: move cache generation to cache.py

* mkdir cache directory when it does not exist

---------

Co-authored-by: Soufiane Fariss <soufiane.fariss@um5s.net.ma>
Co-authored-by: Moritz <mr-tz@users.noreply.github.com>
2024-06-04 18:47:41 +02:00
Capa Bot
e517d7dd77 Sync capa rules submodule 2024-06-04 10:35:46 +00:00
Moritz
142b84f9c5 Merge pull request #2118 from mandiant/dependabot/pip/deptry-0.16.1
build(deps): bump deptry from 0.14 to 0.16.1
2024-06-04 12:33:51 +02:00
dependabot[bot]
72607c6ae5 build(deps): bump ruff from 0.4.5 to 0.4.7
Bumps [ruff](https://github.com/astral-sh/ruff) from 0.4.5 to 0.4.7.
- [Release notes](https://github.com/astral-sh/ruff/releases)
- [Changelog](https://github.com/astral-sh/ruff/blob/main/CHANGELOG.md)
- [Commits](https://github.com/astral-sh/ruff/compare/v0.4.5...v0.4.7)

---
updated-dependencies:
- dependency-name: ruff
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
2024-06-03 17:56:43 +02:00
dependabot[bot]
2fd01835dc build(deps): bump rich from 13.4.2 to 13.7.1
Bumps [rich](https://github.com/Textualize/rich) from 13.4.2 to 13.7.1.
- [Release notes](https://github.com/Textualize/rich/releases)
- [Changelog](https://github.com/Textualize/rich/blob/master/CHANGELOG.md)
- [Commits](https://github.com/Textualize/rich/compare/v13.4.2...v13.7.1)

---
updated-dependencies:
- dependency-name: rich
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
2024-06-03 17:55:55 +02:00
dependabot[bot]
80600f59c7 build(deps): bump deptry from 0.14 to 0.16.1
Bumps [deptry](https://github.com/fpgmaas/deptry) from 0.14 to 0.16.1.
- [Release notes](https://github.com/fpgmaas/deptry/releases)
- [Changelog](https://github.com/fpgmaas/deptry/blob/main/CHANGELOG.md)
- [Commits](https://github.com/fpgmaas/deptry/compare/0.14.0...0.16.1)

---
updated-dependencies:
- dependency-name: deptry
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
2024-06-03 14:07:40 +00:00
Capa Bot
1ec1185850 Sync capa-testfiles submodule 2024-06-02 14:32:18 +00:00
Moritz
22e12928a6 Merge pull request #2114 from RainRat/master
fix typos
2024-06-02 16:23:29 +02:00
RainRat
8ad74ddbb6 fix typos 2024-06-01 11:48:19 -07:00
Capa Bot
2c1d5592ca Sync capa rules submodule 2024-06-01 10:23:18 +00:00
Capa Bot
267f5e99b7 Sync capa-testfiles submodule 2024-06-01 10:19:40 +00:00
Capa Bot
6b77c50ae8 Sync capa rules submodule 2024-05-31 20:25:51 +00:00
Capa Bot
8a0a24f269 Sync capa rules submodule 2024-05-31 17:24:45 +00:00
Capa Bot
4f2494dc59 Sync capa-testfiles submodule 2024-05-31 09:35:22 +00:00
Fariss
2e5da3e2bd Add deptry support (#2085)
* Add deptry support

This commit resolves #1497.

Note: known_first_party refers to modules that are supposed to be
local, i.e. idaapi, ghidra, java, binaryninja, ... etc.

* adjust running stages for deptry hook

* adjust deptry exclusions, and humanize dependency

---------

Co-authored-by: Soufiane Fariss <soufiane.fariss@um5s.net.ma>
2024-05-31 09:43:10 +02:00
Moritz
0ac21f036c update to Ubuntu 22.04 for Binary Ninja tests 2024-05-29 14:21:02 +02:00
Moritz
4ecf3a1793 Merge pull request #2090 from mandiant/dependabot/pip/protobuf-5.27.0
build(deps): bump protobuf from 5.26.1 to 5.27.0
2024-05-29 10:21:38 +02:00
Moritz
b14db68819 Merge pull request #2091 from mandiant/dependabot/pip/types-requests-2.32.0.20240523
build(deps): bump types-requests from 2.31.0.20240406 to 2.32.0.20240523
2024-05-29 10:21:25 +02:00
Moritz
54106d60ae Merge pull request #2092 from mandiant/dependabot/pip/pyinstaller-6.7.0
build(deps): bump pyinstaller from 6.6.0 to 6.7.0
2024-05-29 10:21:14 +02:00
Capa Bot
0622f45208 Sync capa-testfiles submodule 2024-05-28 13:44:27 +00:00
Moritz
adb9de8d4b Merge pull request #2089 from mandiant/dependabot/pip/ruff-0.4.5
build(deps): bump ruff from 0.4.4 to 0.4.5
2024-05-28 13:18:33 +02:00
dependabot[bot]
48dd64beba build(deps): bump protobuf from 5.26.1 to 5.27.0
Bumps [protobuf](https://github.com/protocolbuffers/protobuf) from 5.26.1 to 5.27.0.
- [Release notes](https://github.com/protocolbuffers/protobuf/releases)
- [Changelog](https://github.com/protocolbuffers/protobuf/blob/main/protobuf_release.bzl)
- [Commits](https://github.com/protocolbuffers/protobuf/compare/v5.26.1...v5.27.0)

---
updated-dependencies:
- dependency-name: protobuf
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
2024-05-28 07:59:24 +00:00
Fariss
abaabae164 Replace halo spinner with rich (#2086)
* Replace halo spinner with rich

* remove Halo dependency

* Omit halo from mypy.ini

---------

Co-authored-by: Soufiane Fariss <soufiane.fariss@um5s.net.ma>
2024-05-28 09:58:32 +02:00
dependabot[bot]
8316a74ca2 build(deps): bump pyinstaller from 6.6.0 to 6.7.0
Bumps [pyinstaller](https://github.com/pyinstaller/pyinstaller) from 6.6.0 to 6.7.0.
- [Release notes](https://github.com/pyinstaller/pyinstaller/releases)
- [Changelog](https://github.com/pyinstaller/pyinstaller/blob/develop/doc/CHANGES.rst)
- [Commits](https://github.com/pyinstaller/pyinstaller/compare/v6.6.0...v6.7.0)

---
updated-dependencies:
- dependency-name: pyinstaller
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
2024-05-27 14:33:10 +00:00
dependabot[bot]
1dd2af7048 build(deps): bump types-requests from 2.31.0.20240406 to 2.32.0.20240523
Bumps [types-requests](https://github.com/python/typeshed) from 2.31.0.20240406 to 2.32.0.20240523.
- [Commits](https://github.com/python/typeshed/commits)

---
updated-dependencies:
- dependency-name: types-requests
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
2024-05-27 14:32:57 +00:00
dependabot[bot]
bbc4e5cd97 build(deps): bump ruff from 0.4.4 to 0.4.5
Bumps [ruff](https://github.com/astral-sh/ruff) from 0.4.4 to 0.4.5.
- [Release notes](https://github.com/astral-sh/ruff/releases)
- [Changelog](https://github.com/astral-sh/ruff/blob/main/CHANGELOG.md)
- [Commits](https://github.com/astral-sh/ruff/compare/v0.4.4...v0.4.5)

---
updated-dependencies:
- dependency-name: ruff
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
2024-05-27 14:32:37 +00:00
Capa Bot
7da3ef89ca Sync capa rules submodule 2024-05-23 11:37:39 +00:00
Moritz
44e319a604 Merge pull request #2081 from mandiant/dependabot/pip/mypy-protobuf-3.6.0
build(deps): bump mypy-protobuf from 3.5.0 to 3.6.0
2024-05-22 14:09:26 +02:00
Moritz
21c346d0c2 Merge pull request #2082 from mandiant/dependabot/pip/types-requests-2.31.0.20240406
build(deps): bump types-requests from 2.31.0.20240311 to 2.31.0.20240406
2024-05-22 14:09:17 +02:00
Capa Bot
f9953d1e99 Sync capa rules submodule 2024-05-21 07:58:30 +00:00
dependabot[bot]
9bce98b0ae build(deps): bump types-requests from 2.31.0.20240311 to 2.31.0.20240406
Bumps [types-requests](https://github.com/python/typeshed) from 2.31.0.20240311 to 2.31.0.20240406.
- [Commits](https://github.com/python/typeshed/commits)

---
updated-dependencies:
- dependency-name: types-requests
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
2024-05-20 14:14:50 +00:00
dependabot[bot]
7f39a5b1d6 build(deps): bump mypy-protobuf from 3.5.0 to 3.6.0
Bumps [mypy-protobuf](https://github.com/nipunn1313/mypy-protobuf) from 3.5.0 to 3.6.0.
- [Changelog](https://github.com/nipunn1313/mypy-protobuf/blob/main/CHANGELOG.md)
- [Commits](https://github.com/nipunn1313/mypy-protobuf/compare/v3.5.0...v3.6.0)

---
updated-dependencies:
- dependency-name: mypy-protobuf
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
2024-05-20 14:14:47 +00:00
Moritz
e9cc193dd4 Merge pull request #2077 from mandiant/dependabot/pip/tqdm-4.66.4
build(deps): bump tqdm from 4.66.3 to 4.66.4
2024-05-16 14:15:15 +02:00
Moritz
5482021c75 Merge pull request #2076 from mandiant/dependabot/pip/flake8-bugbear-24.4.26
build(deps): bump flake8-bugbear from 24.2.6 to 24.4.26
2024-05-16 14:14:58 +02:00
Moritz
5507991575 Merge pull request #2078 from mandiant/dependabot/pip/ruff-0.4.4
build(deps): bump ruff from 0.3.5 to 0.4.4
2024-05-16 14:13:01 +02:00
dependabot[bot]
65114ec2d7 build(deps): bump ruff from 0.3.5 to 0.4.4
Bumps [ruff](https://github.com/astral-sh/ruff) from 0.3.5 to 0.4.4.
- [Release notes](https://github.com/astral-sh/ruff/releases)
- [Changelog](https://github.com/astral-sh/ruff/blob/main/CHANGELOG.md)
- [Commits](https://github.com/astral-sh/ruff/compare/v0.3.5...v0.4.4)

---
updated-dependencies:
- dependency-name: ruff
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
2024-05-13 14:54:11 +00:00
dependabot[bot]
e4ae052f48 build(deps): bump tqdm from 4.66.3 to 4.66.4
Bumps [tqdm](https://github.com/tqdm/tqdm) from 4.66.3 to 4.66.4.
- [Release notes](https://github.com/tqdm/tqdm/releases)
- [Commits](https://github.com/tqdm/tqdm/compare/v4.66.3...v4.66.4)

---
updated-dependencies:
- dependency-name: tqdm
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
2024-05-13 14:53:54 +00:00
dependabot[bot]
3ae8183a4a build(deps): bump flake8-bugbear from 24.2.6 to 24.4.26
Bumps [flake8-bugbear](https://github.com/PyCQA/flake8-bugbear) from 24.2.6 to 24.4.26.
- [Release notes](https://github.com/PyCQA/flake8-bugbear/releases)
- [Commits](https://github.com/PyCQA/flake8-bugbear/compare/24.2.6...24.4.26)

---
updated-dependencies:
- dependency-name: flake8-bugbear
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
2024-05-13 14:53:49 +00:00
Willi Ballenthin
b59df659c9 pep8 2024-05-08 16:20:10 +02:00