* web: rules: redirect from various rule names to canonical rule URL
closes#2319
Update index.html
Co-authored-by: Moritz <mr-tz@users.noreply.github.com>
* cli: link to rule names to capa rules website
* just: make `just lint` run all steps, not fail on first error
---------
Co-authored-by: Moritz <mr-tz@users.noreply.github.com>
* web: index: add gif of capa running
* index: add screencast of running capa
produced via:
```
asciinema capa.cast
./capa Practical\ Malware\ Analysis\ Lab\ 01-01.dll_
<ctrl-d>
agg --no-loop --theme solarized-light capa.cast capa.gif
```
* web: index: start to sketch out style
* web: landing page
* web: merge rules website
* web: rules: update bootstrap and integrate rules
* web: rules: use pygments to syntax highlight rules
Use the Pygments syntax-highlighting library to parse
and render the YAML rule content. This way we don't have
to manually traverse the rule nodes and emit lists; instead,
we rely on the fact that YAML is pretty easy for humans
to read and let them consume it directly, with some text
formatting to help hint at the types/structure.
* web: rules: use capa to load rule content
capa (the library) has routines for deserializing the YAML
content into structured objects, which means we can use tools
like mypy to find bugs. So, prefer to use those routines instead
of parsing YAML ourselves.
* web: rules: linters
Run and fix the issues identified by the following linters:
- isort
- black
- ruff
- mypy
* web: rules: add some links to rule page
Add links to the following external resources:
- GitHub rule source in capa-rules repo
- VirusTotal search for matching samples
* web: rules: accept ?q= parameter for initial search
Update the rules landing page to accept a HTTP
query parameter named "q" that specifies an initial
search term to to pass to pagefind. This enables
external pages link to rule searches.
* web: rules: add link to namespace search
* web: rules: use consistent header
Import header from root capa landing page.
* web: rules: add umami script
* web: add initial whats new section, TODOs
* web: rules: remove old images
* changelog
* CI: remove temporary branch push event triggers
* Delete web/rules/public/css/bootstrap-4.5.2.min.css
* Delete web/rules/public/js/bootstrap-4.5.2.min.js
* Delete web/public/img/capa.cast
* Rename readme.md to README.md
* web: rules: add scripts to pre-commit configs
* web: rules: add scripts to pre-commit configs
* lints
* ci: add temporary branch push trigger to get incremental builds
* web: rules: assert start_dir must exist
* ci: web: rules: deep checkout so we can get rule history
* web: rules: check output of subprocess
* web: rules: factor out common CSS
* web: rules: fix header links
* web: rules: only index rule content, not surrounding text
* ci: web: remote temporary branch push trigger
Add support to extract dynamically resolved APIs stored in global
variables that have been renamed (for example using the `renimp.idc`
script included with IDA Pro).
* elf: read segment memory size
* elf: add routine to read mapped memory
* elf: better detect OS for binaries compiled by Go
* elf: guess OS from Go source filenames
* changelog
* elf: mypy
* merge
* elf: add OS detection based on vDSO strings
* elf: document VTGrep searches
* elf: describe further technique to identify Go binaries
* elf: search for `.go.buildinfo` section via @yelhamer
* black
* elf: detect Alpine Linux ident
* elf: log interest symtab entries
* tests: add test for OS detection by Go buildinfo
* loader: handle missing viv modules
* pre-commit: run deptry before tests (which are slow)
* loader: describe removing viv symbolic switch solver
* pyproject: add PyGithub for deptry
* black
* feat(capa2sarif): add new sarif conversion script converting json output to sarif schema, update dependencies, and update changelog
* fix(capa2sarif): removing copy and paste transcription errors
* fix(capa2sarif): remove dependencies from pyproject toml to guarded import statements
* chore(capa2sarif): adding node in readme specifying dependency and applied auto formatter for styling
* style(capa2sarif): applied import sorting and fixed typo in invocations function
* test(capa2sarif): adding simple test for capa to sarif conversion script using existing result document
* style(capa2sarif): fixing typo in version string in usage
* style(capa2sarif): isort failing due to reordering of typehint imports
* style(capa2sarif): fixing import order as isort on local machine was not updating code
---------
Co-authored-by: ReversingWithMe <ryanv@rewith.me>
Co-authored-by: Willi Ballenthin <wballenthin@google.com>
Implement the "tighten rule pre-selection" algorithm described here:
https://github.com/mandiant/capa/issues/2063#issuecomment-2100498720
In summary:
> Rather than indexing all features from all rules,
> we should pick and index the minimal set (ideally, one) of
> features from each rule that must be present for the rule to match.
> When we have multiple candidates, pick the feature that is
> probably most uncommon and therefore "selective".
This seems to work pretty well. Total evaluations when running against
mimikatz drop from 19M to 1.1M (wow!) and capa seems to match around
3x more functions per second (wow wow).
When doing large scale runs, capa is about 25% faster when using the
vivisect backend (analysis heavy) or 3x faster when using the
upcoming BinExport2 backend (minimal analysis).
* include rule caching in PyInstaller build process
The following commit introduces a new function that caches the capa
rule set, so that users don't have to manually run ./scripts/cache-
ruleset.py, before running pyinstaller.
* ci: omit Cache rule set step from build.yml workflow
* refactor: move cache generation to cache.py
* mkdir cache directory when it does not exist
---------
Co-authored-by: Soufiane Fariss <soufiane.fariss@um5s.net.ma>
Co-authored-by: Moritz <mr-tz@users.noreply.github.com>
* Add deptry support
This commit resolves#1497.
Note: known_first_party refers to modules that are supposed to be
local, i.e. idaapi, ghidra, java, binaryninja, ... etc.
* adjust running stages for deptry hook
* adjust deptry exclusions, and humanize dependency
---------
Co-authored-by: Soufiane Fariss <soufiane.fariss@um5s.net.ma>
