Commit Graph

5110 Commits

Author SHA1 Message Date
William Ballenthin c2a4629c62 scripts: add cli arguments to specify signatures 2021-03-04 15:04:33 -07:00
William Ballenthin c0f4fe6867 merge master 2021-03-04 14:59:17 -07:00
William Ballenthin f2c95568bd main: add FLIRT signature matching configuration 2021-03-04 14:52:22 -07:00
William Ballenthin 358aab85e7 viv: move FLIRT matching into viv-utils 2021-03-04 14:51:40 -07:00
Ana María Martínez Gómez 829274cd5e Merge pull request #421 from Ana06/viv-py3 2021-03-03 21:40:08 +01:00
Ana Maria Martinez Gomez c522f5094a Use -j option in test_backend_option
Use `-j` option in `test_backend_option` to check the extractor and that
rules have been extracted. This way we don't need to check if a concrete
rule matches, but only that at least a rule matches.
2021-03-03 18:33:20 +01:00
Ana Maria Martinez Gomez 29b6772721 Test backend option
As `get_extractor` returns only vivisect now, `test_main` is not run for
smda. Test that capa works with all backends. It doesn't test that the
backend is actually called.
2021-03-03 17:36:51 +01:00
Ana Maria Martinez Gomez 695b5b50ab Remove va not None check
Instead of checking if `va` is `None in `get_section()` we should avoid
calling this function with `None`. This have been fixed in the following
PR, so this is not longer needed:
https://github.com/fireeye/capa/pull/442
2021-03-03 17:36:51 +01:00
Ana Maria Martinez Gomez 42af7b2d8b Use default backend instead of None
Set the `backend` variable to the default backend by default instead to
`None`. The `backend` variable is needed in Python 2 as `args.backend`
is only set in Python 3. Although the value of the backend variable is
ignored in Python 2, so that the default value is not used.

Co-authored-by: William Ballenthin <william.ballenthin@fireeye.com>
2021-03-03 17:36:51 +01:00
Ana Maria Martinez Gomez 079a9b5204 Remove backend option from Python 2
Do only provide the backend option in Python 3, as there is only one
backend in Python 2. This way we keep the help text simpler.
2021-03-03 17:36:51 +01:00
Ana Maria Martinez Gomez e5048fd3ac Add missing va parameter to SegmentationViolation
The `envi.SegmentationViolation()` was missing the `va` required
parameter. This has started failing now, because calling
`vw.getSegment(0x4BA190)` for the `tests/data/mimikatz.exe_` produces
different results in Python 2 and Python 3. It returns `None` in Python
3 while the output in Python 2 is:
`(4939776, 16840, '.data', 'mimikatz')`

I have reported the issue to vivisect:
https://github.com/vivisect/vivisect/issues/370
2021-03-03 17:36:51 +01:00
Ana Maria Martinez Gomez 18eaea95fa Fix TypeError exception in Python3
`va` can be None and this causes Python 3 to raise a TypeError
exception. This is caused by the following breaking change in Python3:
> The ordering comparison operators (<, <=, >=, >) raise a TypeError
> exception when the operands don’t have a meaningful natural ordering.

This didn't failed in the previously tried vivisect version (master from
one week ago and not the release). This may have been caused by a bug in
vivisect that has been fixed.
2021-03-03 17:36:51 +01:00
Ana Maria Martinez Gomez a4a0a56448 Vivisect 1.0.0 released
Vivisect 1.0.0 (Python 3) has been released, so we do not need to link
to my GitHub branch anymore.

https://pypi.org/project/vivisect
2021-03-03 17:36:50 +01:00
Ana Maria Martinez Gomez 40ed2f39a4 Make backend a required parameter in get_extractor
Make the `backend` argument required in the `get_extractor` internal
routine. Specify a backend in the scripts which call this function. Add
a CLI backend option in capa/features/freeze.py as well.
2021-03-03 17:36:50 +01:00
Ana Maria Martinez Gomez 2859b037aa Use constants for backend option
Use constants instead of string literals for the backend option.
2021-03-03 17:36:50 +01:00
Ana Maria Martinez Gomez bbb7878e0a Enable tests for vivisect in Python3
Now we support vivisect as backend in Python3. We should test it.
2021-03-03 17:36:50 +01:00
Ana Maria Martinez Gomez fc438866ec Add option to select the backend in Py3
Now we have two working backends in Python3! Add an option to select
which one to use. With this code, vivisect is the default backend, but
this is really easy to change. We could do some analysis to see if smda
performances better than vivisect once the vivisect implementation.
2021-03-03 17:36:50 +01:00
Ana Maria Martinez Gomez 2da2f498a2 Add script to compare vivisect Python 2 vs 3
Compare the performance of vivisect Python 2 vs 3 by counting the number
of feature of each type extracted for every binary in `tests/data`.
Render the ones that perform bad (under a threshold - 98) and the total
performance. Render also the running time per binary for both Python 2 and 3.

From this result, it seems that vivisect behaves properly with Python3.
2021-03-03 17:36:50 +01:00
Ana Maria Martinez Gomez 29dffffe1b Python3 support for vivisect
Vivisect has moved to Python3. Allow to run vivisect with Python3 in
capa.

I am using the following version of vivisect (which includes fixes for
some bugs I have found and some open PRs in vivisect):
https://github.com/Ana06/vivisect/tree/py-3
2021-03-03 17:36:49 +01:00
Capa Bot 1ecaad5413 Sync capa rules submodule 2021-03-02 15:06:24 +00:00
Willi Ballenthin cd56d672c0 Merge pull request #442 from fireeye/williballenthin-patch-2
viv: ignore empty branch targets
2021-03-01 08:43:26 -07:00
Willi Ballenthin 68aed3c190 insn: better document when branch va may be none 2021-02-28 23:03:08 -07:00
William Ballenthin f16ecd837e viv: flirt: add more documentation 2021-02-26 05:02:10 -07:00
Willi Ballenthin 68fcc03d5c viv: ignore empty branch targets
but what does this really mean? why would `getBranches` return `None`?

closes #441
2021-02-25 13:34:59 -07:00
William Ballenthin bfcae0e754 extractors: viv: match flirt signatures [wip] 2021-02-25 12:21:27 -07:00
William Ballenthin 1b2c8880ee capa: extractors: sketch API extension to support function id 2021-02-25 12:20:29 -07:00
Ana Maria Martinez Gomez fa7d58d01a Add missing va parameter to SegmentationViolation
The `envi.SegmentationViolation()` was missing the `va` required
parameter. This has started failing now, because calling
`vw.getSegment(0x4BA190)` for the `tests/data/mimikatz.exe_` produces
different results in Python 2 and Python 3. It returns `None` in Python
3 while the output in Python 2 is:
`(4939776, 16840, '.data', 'mimikatz')`

I have reported the issue to vivisect:
https://github.com/vivisect/vivisect/issues/370
2021-02-25 11:20:45 +01:00
Ana Maria Martinez Gomez ec558f377a Fix TypeError exception in Python3
`va` can be None and this causes Python 3 to raise a TypeError
exception. This is caused by the following breaking change in Python3:
> The ordering comparison operators (<, <=, >=, >) raise a TypeError
> exception when the operands don’t have a meaningful natural ordering.

This didn't failed in the previously tried vivisect version (master from
one week ago and not the release). This may have been caused by a bug in
vivisect that has been fixed.
2021-02-25 10:15:49 +01:00
Ana Maria Martinez Gomez 186eba7197 Vivisect 1.0.0 released
Vivisect 1.0.0 (Python 3) has been released, so we do not need to link
to my GitHub branch anymore.

https://pypi.org/project/vivisect
2021-02-25 10:05:04 +01:00
Ana Maria Martinez Gomez d28ba3c628 Make backend a required parameter in get_extractor
Make the `backend` argument required in the `get_extractor` internal
routine. Specify a backend in the scripts which call this function. Add
a CLI backend option in capa/features/freeze.py as well.
2021-02-25 10:04:19 +01:00
Ana Maria Martinez Gomez a026cb84d1 Use constants for backend option
Use constants instead of string literals for the backend option.
2021-02-25 09:35:40 +01:00
Ana Maria Martinez Gomez 3acc3eeabd Enable tests for vivisect in Python3
Now we support vivisect as backend in Python3. We should test it.
2021-02-25 09:35:40 +01:00
Ana Maria Martinez Gomez a92d2af7f8 Add option to select the backend in Py3
Now we have two working backends in Python3! Add an option to select
which one to use. With this code, vivisect is the default backend, but
this is really easy to change. We could do some analysis to see if smda
performances better than vivisect once the vivisect implementation.
2021-02-25 09:35:40 +01:00
Ana Maria Martinez Gomez adcb683458 Add script to compare vivisect Python 2 vs 3
Compare the performance of vivisect Python 2 vs 3 by counting the number
of feature of each type extracted for every binary in `tests/data`.
Render the ones that perform bad (under a threshold - 98) and the total
performance. Render also the running time per binary for both Python 2 and 3.

From this result, it seems that vivisect behaves properly with Python3.
2021-02-25 09:35:40 +01:00
Capa Bot 939b29bf60 Sync capa rules submodule 2021-02-24 23:00:34 +00:00
Ana Maria Martinez Gomez e4925613b3 Python3 support for vivisect
Vivisect has moved to Python3. Allow to run vivisect with Python3 in
capa.

I am using the following version of vivisect (which includes fixes for
some bugs I have found and some open PRs in vivisect):
https://github.com/Ana06/vivisect/tree/py-3
2021-02-24 17:55:39 +01:00
Capa Bot 2f6a6e4628 Sync capa rules submodule 2021-02-24 08:07:52 +00:00
Capa Bot 7938ea34d0 Sync capa rules submodule 2021-02-24 08:06:30 +00:00
Capa Bot ed94e36f7a Sync capa rules submodule 2021-02-24 00:12:19 +00:00
mike-hunhoff 1c3a8df136 Merge pull request #439 from fireeye/explorer/rulegen-support-file-scope
adding file scope support to rule generator IDA plugin
2021-02-23 11:50:54 -07:00
Michael Hunhoff 9f254b22ee adding file scope support to rule generator IDA plugin 2021-02-23 11:10:34 -07:00
Capa Bot 753f8ce84e Sync capa rules submodule 2021-02-23 17:33:38 +00:00
Capa Bot acf3b549de Sync capa rules submodule 2021-02-23 15:29:20 +00:00
Capa Bot 669f6dcf98 Sync capa rules submodule 2021-02-23 15:23:19 +00:00
Capa Bot e4f7c4aab1 Sync capa rules submodule 2021-02-23 15:22:43 +00:00
Moritz 5836d55e21 Merge pull request #438 from fireeye/explorer/show-results-by-function
explorer: adding option to show results by function
2021-02-22 18:23:44 +01:00
Michael Hunhoff e17bf1a1f4 explorer: adding option to show results by function 2021-02-22 08:16:18 -07:00
Willi Ballenthin acb253ae9c Merge pull request #437 from fireeye/scripts/show-capabilities
update to support running in IDA w/ Python 3
2021-02-19 17:02:53 -07:00
Michael Hunhoff cc0aaa301f update to support running in IDA w/ Python 3 2021-02-19 14:28:20 -07:00
mike-hunhoff 4256316045 Merge pull request #436 from fireeye/fix/ida/unmapped-data-ref
check for unmapped addresses when resolving data references
2021-02-19 12:58:16 -07:00