Commit Graph

119 Commits

Author SHA1 Message Date
Ana Maria Martinez Gomez
dfb7cf4888 py3: set and document env
Document how to use env now that we are Python3 only. Adapt
`scripts/ci.sh`.
2021-04-07 18:20:07 +02:00
Michael Hunhoff
e51c79c241 adding lint for incorrect rule string format, refined rendering for strings 2021-03-23 15:55:48 -06:00
Michael Hunhoff
7bf8c6e3a1 merging upstream 2021-03-22 10:33:36 -06:00
Moritz Raabe
8719a23de4 dos2unix 2021-03-19 09:40:44 +01:00
Moritz Raabe
7e0b5236af better deal with CRLF/LF issues 2021-03-19 09:40:43 +01:00
Moritz Raabe
c7798b3254 ensure LF end of line 2021-03-19 09:40:43 +01:00
William Ballenthin
063e1229bc pep8 2021-03-05 11:10:12 -07:00
William Ballenthin
eacd70329a merge from master, sorry 2021-03-05 11:06:40 -07:00
William Ballenthin
3a1d5d068c scripts: use common argument handler
closes #449
2021-03-05 10:58:40 -07:00
William Ballenthin
bdea61f93b scripts: remove old migration script 2021-03-05 10:57:14 -07:00
Ana Maria Martinez Gomez
40ed2f39a4 Make backend a required parameter in get_extractor
Make the `backend` argument required in the `get_extractor` internal
routine. Specify a backend in the scripts which call this function. Add
a CLI backend option in capa/features/freeze.py as well.
2021-03-03 17:36:50 +01:00
Ana Maria Martinez Gomez
2da2f498a2 Add script to compare vivisect Python 2 vs 3
Compare the performance of vivisect Python 2 vs 3 by counting the number
of feature of each type extracted for every binary in `tests/data`.
Render the ones that perform bad (under a threshold - 98) and the total
performance. Render also the running time per binary for both Python 2 and 3.

From this result, it seems that vivisect behaves properly with Python3.
2021-03-03 17:36:50 +01:00
Michael Hunhoff
cc0aaa301f update to support running in IDA w/ Python 3 2021-02-19 14:28:20 -07:00
Moritz
7783543153 Merge pull request #429 from fireeye/scripts/multiple-backends-show-features
mirror show-capabilities-by-function to enable multiple backends
2021-02-18 09:33:36 +01:00
Michael Hunhoff
47b3ef29be removing viv dep from show-capabilities-by-function.py 2021-02-17 14:49:52 -07:00
Michael Hunhoff
1eb615f97c mirror show-capabilities-by-function to enable multiple backends 2021-02-17 14:40:33 -07:00
Michael Hunhoff
2d34458d10 linter: adding ntoskrnl, ntdll overlap lint 2021-02-17 13:29:36 -07:00
Moritz Raabe
0eb8d3e47c fix time debug output 2021-01-28 11:09:25 +01:00
Moritz Raabe
072e30498b adjust negative hex numbers in to_yaml 2021-01-28 10:54:17 +01:00
Moritz Raabe
a81f98be8e manual adjust negative numbers 2021-01-28 10:54:17 +01:00
Moritz Raabe
0980e35c29 simplify string comparison 2021-01-28 10:54:17 +01:00
Moritz Raabe
336c2a3aff add option to only check reformat status 2021-01-28 10:54:17 +01:00
Moritz Raabe
e3055bc740 check rule format consistency 2021-01-28 10:54:17 +01:00
Moritz Raabe
f18a8f5b31 adjust expected lib path and log time 2021-01-28 10:18:03 +01:00
Moritz Raabe
0d439c0f55 disable extractor progress 2021-01-28 09:22:15 +01:00
Moritz
8788e0a9c9 Merge pull request #388 from fireeye/ci/linter-update
lint with tags
2021-01-05 16:37:21 +01:00
Moritz Raabe
b1c1cb4b9b lint with --tag 2021-01-05 16:16:35 +01:00
DoomedRaven
e158e3f13c remove type hint to make CI happy 2020-12-08 21:46:39 +01:00
DoomedRaven
b1bbded23c black -l 120 . 2020-12-08 21:39:50 +01:00
DoomedRaven
b77d9d3738 isort --profile black --length-sort --line-width 120 capa_as_library.py 2020-12-08 21:34:42 +01:00
DoomedRaven
d0b2421752 isort capa_as_library.py 2020-12-08 20:53:26 +01:00
DoomedRaven
96b65a7c60 add example how to render it as library
```
>>> from capa_as_library import capa_details
>>> details = capa_details("/opt/CAPEv2/storage/analyses/83/binary", "dictionary")
>>> from pprint import pprint as pp
>>> pp(details)
{'ATTCK': {'DEFENSE EVASION': ['Obfuscated Files or Information [T1027]',
                               'Virtualization/Sandbox Evasion::System Checks '
                               '[T1497.001]'],
           'EXECUTION': ['Shared Modules [T1129]']},
 'CAPABILITY': {'anti-analysis/anti-vm/vm-detection': ['execute anti-VM '
                                                       'instructions (3 '
                                                       'matches)'],
                'anti-analysis/obfuscation/string/stackstring': ['contain '
                                                                 'obfuscated '
                                                                 'stackstrings'],
                'data-manipulation/encryption/rc4': ['encrypt data using RC4 '
                                                     'PRGA'],
                'executable/pe/section/rsrc': ['contain a resource (.rsrc) '
                                               'section'],
                'host-interaction/cli': ['accept command line arguments'],
                'host-interaction/environment-variable': ['query environment '
                                                          'variable'],
                'host-interaction/file-system/read': ['read .ini file',
                                                      'read file'],
                'host-interaction/file-system/write': ['write file (3 '
                                                       'matches)'],
                'host-interaction/process': ['get thread local storage value '
                                             '(3 matches)',
                                             'set thread local storage value '
                                             '(2 matches)'],
                'host-interaction/process/terminate': ['terminate process (3 '
                                                       'matches)'],
                'host-interaction/thread/terminate': ['terminate thread'],
                'linking/runtime-linking': ['link function at runtime (7 '
                                            'matches)',
                                            'link many functions at runtime'],
                'load-code/pe': ['parse PE header (3 matches)']},
 'MBC': {'ANTI-BEHAVIORAL ANALYSIS': ['Virtual Machine Detection::Instruction '
                                      'Testing [B0009.029]'],
         'ANTI-STATIC ANALYSIS': ['Disassembler Evasion::Argument Obfuscation '
                                  '[B0012.001]'],
         'CRYPTOGRAPHY': ['Encrypt Data::RC4 [C0027.009]',
                          'Generate Pseudo-random Sequence::RC4 PRGA '
                          '[C0021.004]']},
 'md5': 'ad56c384476a81faef9aebd60b2f4623',
 'path': '/opt/CAPEv2/storage/analyses/83/binary',
 'sha1': 'aa027d89f5d3f991ad3e14ffb681616a77621836',
 'sha256': '16995e059eb47de0b58a95ce2c3d863d964a7a16064d4298cee9db1de266e68d'}
>>>
```
2020-12-08 20:00:24 +01:00
doomedraven
5920552649 small improvements 2020-12-01 20:31:56 +01:00
Willi Ballenthin
eac7e2b749 capa_as_library: style and comments 2020-12-01 06:54:55 -07:00
doomedraven
51ad526cfc Simple example how to use capa as library
Just quick example how to use capa as library, to save time to someone, reading code and scripts
2020-12-01 11:20:49 +01:00
Moritz Raabe
76b46d7957 ensure function is defined in vivisect (or do so)
and show features in IDA
2020-10-20 15:09:07 +02:00
William Ballenthin
314757a235 scripts: add script demonstrating bulk processing
closes #307
2020-09-23 09:13:49 -06:00
Christian Clauss
5f184b278f Undefined name: import ida_funcs for lines 48, 52, 57 2020-09-07 09:48:42 +02:00
Moritz Raabe
34e7991081 black 20.8b1 updates 2020-08-27 11:26:28 +02:00
Moritz Raabe
3e20f0fc71 dos2unix 2020-08-27 11:25:43 +02:00
Ana María Martínez Gómez
f9abbbe9ba hooks: change from post- to pre-commit
Avoid erroneous commits. The pre-commit can be skipped with the
`--no-verify` option, which is not available with the post-commit.
Note that `pre-commit` doesn't run when rebase while post-commit did.

This allows to have a single script which is run by both hooks. This
scripts can also be used independently to run the CI setup locally.
2020-07-29 20:25:03 +02:00
Ana María Martínez Gómez
8ff9e339f5 hooks: improve comment in setup-hooks.sh
Previously the `setup-hooks.sh` took care of more things. Consequently
the comments needed to be separated. This is not needed anymore and it
looks weird.
2020-07-29 20:16:58 +02:00
Ana María Martínez Gómez
6d00ae26ae hooks: do not repeat #!/usr/bin/env bash
The comment claims that:
> doesn't matter if `#!/usr/bin/env bash` gets repeated later on in a hooks file

However it is a convetion that there is only one comment like that and
the `setup-hooks.sh` script already takes care of adding it if the file
didn't exist.

Alternatively, we could leave the extra comment and remove the related
code in `setup-hooks.sh`. Currently two comments are added even if the
file didn't exist.
2020-07-29 20:16:56 +02:00
Ana María Martínez Gómez
22fd52ccb9 hooks: do not render git stash output
The `-q` options hides most of the output, but not everything.
2020-07-29 20:16:55 +02:00
Ana María Martínez Gómez
70dda980e8 hooks: update style checkers in hooks
Run `isort` and `black` in the hooks. Run them with the `--check` option
to not modify files which could be committed by accident.

Note that the changes in `.gitignore` are not needed as `.log` is
already excluded in the Django's section. We add those file to this
section in case `.log` is removed in the future.
2020-07-29 20:16:52 +02:00
William Ballenthin
b6c9540469 pep8 2020-07-25 10:29:21 -06:00
William Ballenthin
605ee00f0a scripts: bn: remove extra character 2020-07-25 10:28:11 -06:00
William Ballenthin
bf4d12e5b6 Merge branch 'master' into fix-197 2020-07-25 10:24:25 -06:00
Willi Ballenthin
14463de5e7 Merge pull request #207 from psifertex/master
adding BN script
2020-07-27 09:19:31 -06:00
Jordan Wiens
f547ca0fae updates for pull 205 2020-07-25 22:45:51 -04:00