Commit Graph

1102 Commits

Author SHA1 Message Date
Ana Maria Martinez Gomez
3831f1c104 extractors: Do not use generate_api_features
`generate_api_features` was merged with the implementation of
`generate_import_features` and replaced by `generate_symbol`:
2b2656c2a3
 Use the new function in the miasm backend implementation.
2021-02-05 15:41:13 +01:00
Ana Maria Martinez Gomez
dc828e82b3 extractors: add required loc_db
Since the following PR, miasm requires LocationDB in the object's
constructor instead of creating a new LocationDB:
https://github.com/cea-sec/miasm/pull/1274

This was not the case at the point I started the miasm backend
implementation. Adapt the code to work with this change, which also
means interacting with miasm in a better way.
2021-02-05 15:41:04 +01:00
Ana María Martínez Gómez
2e98ba990c tests: enable tests for miasm
Everything is red :( Some tests are failing due to the not yet
implemented features. In addition, it looks like miasm has problems
disassembling some of the used files.
2021-02-03 15:07:31 +01:00
Ana María Martínez Gómez
d008fef23f extractors: enable miasm in Python3
Do not make miasm the default until we have ensured everything works as
it should.
2021-02-03 15:07:31 +01:00
Ana María Martínez Gómez
fe458c387a extractors: use block and feature offset function
`f` and `bb` in miasm are not an integer. Introduce `block_offset()` and
`feature_offset()` in the extractors and use them in main to solve this.

Related to https://github.com/cea-sec/miasm/pull/1277
2021-02-03 12:50:56 +01:00
Ana María Martínez Gómez
3e52c7de23 features: store mnemomics lower case
miasm extracts mnemonic capitalized while other backends do it
lowercase. To ensure capa works with all of them, use lower case in the
Mnemomic constructor.
2021-02-03 12:50:56 +01:00
Ana María Martínez Gómez
2d1e7946e3 extractors: Implement extract_insn_mnemonic_features
Extract insn mnemonic features in miasm.
2021-02-03 12:50:56 +01:00
Ana María Martínez Gómez
f2fe173ef3 extractors: Implement extract_insn_api_features
Extract insn API features in miasm.
2021-02-03 12:50:56 +01:00
Ana María Martínez Gómez
b2fc52d390 extractors: implement miasm insn features template
Add a template for insn features. These features needs some work and
there are many of them, so I'll introduce them independently in their
own commit.
2021-02-03 12:50:56 +01:00
Ana María Martínez Gómez
5ba4629c3c extractors: implement miasm function features
Add function features.
2021-02-03 12:50:56 +01:00
Ana María Martínez Gómez
4fc9c77791 extractors: implement miasm basic block features
Add basic block features.
2021-02-03 12:50:55 +01:00
Ana María Martínez Gómez
31ba9ee1b3 extractors: Implement get_basic_blocks in miasm
Implement `get_basic_blocks` in `MiasmFeatureExtractor`.
2021-02-03 12:50:55 +01:00
Ana María Martínez Gómez
b4a808ac76 extractors: Implement get_functions in miasm
Implement `get_functions` in `MiasmFeatureExtractor`. It is a proof of
concept, which just considers all loc_keys targets of calls a function.
This is enough to test feature extraction against the functions. A final
version should include other function recognition techniques and be
ported to miasm.
2021-02-03 12:50:55 +01:00
Ana María Martínez Gómez
0f030115d1 extractors: Implement cfg in miasm
Implement `_build_cfg()` in `MiasmFeatureExtractor`.

Co-authored-by: William Ballenthin <william.ballenthin@fireeye.com>
2021-02-03 12:50:55 +01:00
Ana María Martínez Gómez
42573d8df2 extractors: implement miasm file features
Begin to implement miasm backend. Add file features.

This implementation needs:
- https://github.com/cea-sec/miasm/pull/1273

Co-authored-by: William Ballenthin <william.ballenthin@fireeye.com>
2021-02-03 12:50:51 +01:00
Moritz
073c2b5754 Merge pull request #412 from fireeye/ida/meta-add-baseaddr
add imagebase to IDA meta data
2021-02-02 16:48:22 +01:00
mike-hunhoff
ef41d74b82 Merge pull request #411 from fireeye/fix/410
fixes #410
2021-02-02 08:38:23 -07:00
Moritz Raabe
84b3f38810 add imagebase to IDA meta data 2021-02-02 13:54:46 +01:00
mike-hunhoff
2288f38a11 Update capa/main.py
Co-authored-by: Willi Ballenthin <willi.ballenthin@gmail.com>
2021-02-01 12:45:36 -07:00
mike-hunhoff
dbc4e06657 Update capa/main.py
Co-authored-by: Willi Ballenthin <willi.ballenthin@gmail.com>
2021-02-01 12:45:29 -07:00
Michael Hunhoff
2433777a76 fixes #410 2021-02-01 11:43:24 -07:00
Moritz
bb7001f5f2 Merge pull request #409 from fireeye/fix/extract-bytes
improve bytes feature extraction
2021-02-01 17:38:40 +01:00
Moritz Raabe
9b5aaa40de improve bytes feature extraction 2021-02-01 17:17:22 +01:00
Capa Bot
96d74f48f4 Sync capa rules submodule 2021-02-01 11:55:33 +00:00
Capa Bot
f07af25a6a Sync capa rules submodule 2021-01-28 16:52:21 +00:00
Willi Ballenthin
14e65c4601 Merge pull request #401 from fireeye/linter-format
Lint rule formatting and improved rule dump
2021-01-28 09:18:20 -07:00
Capa Bot
b5c2fb0259 Sync capa rules submodule 2021-01-28 16:06:09 +00:00
Capa Bot
92d98db7bb Sync capa-testfiles submodule 2021-01-28 15:25:17 +00:00
Moritz
e6f7ef604a Merge pull request #404 from fireeye/bugfix/403
fixing #403
2021-01-28 11:17:39 +01:00
Moritz Raabe
0eb8d3e47c fix time debug output 2021-01-28 11:09:25 +01:00
Moritz Raabe
072e30498b adjust negative hex numbers in to_yaml 2021-01-28 10:54:17 +01:00
Moritz Raabe
d6e73577af dont change quotes when dumping 2021-01-28 10:54:17 +01:00
Moritz Raabe
a81f98be8e manual adjust negative numbers 2021-01-28 10:54:17 +01:00
Moritz Raabe
0980e35c29 simplify string comparison 2021-01-28 10:54:17 +01:00
Moritz Raabe
336c2a3aff add option to only check reformat status 2021-01-28 10:54:17 +01:00
Moritz Raabe
e3055bc740 check rule format consistency 2021-01-28 10:54:17 +01:00
Capa Bot
9406e3dbfb Sync capa rules submodule 2021-01-28 09:52:43 +00:00
Moritz
5307b7e1b1 Merge pull request #408 from fireeye/fix/lint-lib-path
adjust expected lib path and log time
2021-01-28 10:28:30 +01:00
Moritz Raabe
f18a8f5b31 adjust expected lib path and log time 2021-01-28 10:18:03 +01:00
Moritz
cfe99c4b72 Merge pull request #407 from fireeye/fix/lint-logging
disable extractor progress
2021-01-28 09:25:07 +01:00
Moritz Raabe
0d439c0f55 disable extractor progress 2021-01-28 09:22:15 +01:00
Moritz
6288a96a8b Merge pull request #406 from fireeye/ci/disable-python36
Disable Python 3.6 tests
2021-01-28 08:35:42 +01:00
Moritz
819b6f6ccf Merge pull request #402 from fireeye/lib-rules-subscoped
potential fix for #398
2021-01-28 08:35:28 +01:00
Moritz Raabe
4bc06aa8cd closes #405 2021-01-28 08:23:15 +01:00
Moritz Raabe
7b64425c24 update doc and test case 2021-01-28 08:18:23 +01:00
Michael Hunhoff
44c9d6a22b fixing #403 2021-01-27 18:29:53 -07:00
Moritz Raabe
c750447d62 potential fix for #398 2021-01-27 17:59:56 +01:00
Willi Ballenthin
059ec8f3f2 Merge pull request #400 from fireeye/ci/enable-py39-2
bump smda, enable Python 3.9
2021-01-22 07:18:54 -07:00
Moritz Raabe
2c5508febd bump smda, enable Python 3.9 2021-01-22 10:00:25 +01:00
Capa Bot
905fff041b Sync capa rules submodule 2021-01-21 21:32:42 +00:00