add landing page and rules website (#2310)

* web: index: add gif of capa running

* index: add screencast of running capa

produced via:

```
asciinema capa.cast
./capa Practical\ Malware\ Analysis\ Lab\ 01-01.dll_
<ctrl-d>
agg --no-loop --theme solarized-light capa.cast capa.gif
```

* web: index: start to sketch out style

* web: landing page

* web: merge rules website

* web: rules: update bootstrap and integrate rules

* web: rules: use pygments to syntax highlight rules

Use the Pygments syntax-highlighting library to parse
and render the YAML rule content. This way we don't have
to manually traverse the rule nodes and emit lists; instead,
we rely on the fact that YAML is pretty easy for humans
to read and let them consume it directly, with some text 
formatting to help hint at the types/structure.

* web: rules: use capa to load rule content

capa (the library) has routines for deserializing the YAML
content into structured objects, which means we can use tools
like mypy to find bugs. So, prefer to use those routines instead
of parsing YAML ourselves.

* web: rules: linters

Run and fix the issues identified by the following linters:

  - isort
  - black
  - ruff
  - mypy

* web: rules: add some links to rule page

Add links to the following external resources:

  - GitHub rule source in capa-rules repo
  - VirusTotal search for matching samples

* web: rules: accept ?q= parameter for initial search

Update the rules landing page to accept a HTTP
query parameter named "q" that specifies an initial 
search term to to pass to pagefind. This enables
external pages link to rule searches.

* web: rules: add link to namespace search

* web: rules: use consistent header

Import header from root capa landing page.

* web: rules: add umami script

* web: add initial whats new section, TODOs

* web: rules: remove old images

* changelog

* CI: remove temporary branch push event triggers

* Delete web/rules/public/css/bootstrap-4.5.2.min.css

* Delete web/rules/public/js/bootstrap-4.5.2.min.js

* Delete web/public/img/capa.cast

* Rename readme.md to README.md

* web: rules: add scripts to pre-commit configs

* web: rules: add scripts to pre-commit configs

* lints

* ci: add temporary branch push trigger to get incremental builds

* web: rules: assert start_dir must exist

* ci: web: rules: deep checkout so we can get rule history

* web: rules: check output of subprocess

* web: rules: factor out common CSS

* web: rules: fix header links

* web: rules: only index rule content, not surrounding text

* ci: web: remote temporary branch push trigger
This commit is contained in:
Willi Ballenthin
2024-08-22 09:42:40 +02:00
committed by GitHub
parent c3f24c2f48
commit a33f67b48e
33 changed files with 1379 additions and 37 deletions

View File

@@ -2,7 +2,7 @@ name: deploy web to GitHub Pages
on:
push:
branches: [ master, "wb/webui-actions-1" ]
branches: [ master ]
paths:
- 'web/**'
@@ -22,6 +22,7 @@ concurrency:
jobs:
build-landing-page:
name: Build landing page
runs-on: ubuntu-latest
steps:
- name: Checkout
@@ -32,6 +33,7 @@ jobs:
path: './web/public'
build-explorer:
name: Build capa explorer web
runs-on: ubuntu-latest
steps:
- name: Checkout
@@ -63,12 +65,51 @@ jobs:
name: explorer
path: './web/explorer/dist'
build-rules:
name: Build rules site
runs-on: ubuntu-latest
steps:
- name: Check out the repository
uses: actions/checkout@v4
with:
submodules: 'recursive'
# full depth so that capa-rules has a full history
# and we can construct a timeline of rule updates.
fetch-depth: 0
- name: Set up Python
uses: actions/setup-python@0a5c61591373683505ea898e09a3ea4f39ef2b9c # v5.0.0
with:
python-version: '3.12'
- uses: extractions/setup-just@v2
- name: Install pagefind
uses: supplypike/setup-bin@v4
with:
uri: "https://github.com/CloudCannon/pagefind/releases/download/v1.1.0/pagefind-v1.1.0-x86_64-unknown-linux-musl.tar.gz"
name: "pagefind"
version: "1.1.0"
- name: Install dependencies
working-directory: ./web/rules
run: pip install -r requirements.txt
- name: Build the website
working-directory: ./web/rules
run: just build
- name: Index the website
working-directory: ./web/rules
run: pagefind --site "public"
# upload the build website to artifacts
# so that we can download and inspect, if desired.
- uses: actions/upload-artifact@v4
with:
name: rules
path: './web/rules/public'
deploy:
name: Deploy site to GitHub Pages
environment:
name: github-pages
url: ${{ steps.deployment.outputs.page_url }}
runs-on: ubuntu-latest
needs: [build-landing-page, build-explorer]
needs: [build-landing-page, build-explorer, build-rules]
steps:
- uses: actions/download-artifact@v4
with:
@@ -78,6 +119,10 @@ jobs:
with:
name: explorer
path: './public/explorer'
- uses: actions/download-artifact@v4
with:
name: rules
path: './public/rules'
- name: Setup Pages
uses: actions/configure-pages@v4
- name: Upload artifact

1
.gitignore vendored
View File

@@ -126,3 +126,4 @@ Pipfile.lock
.github/binja/binaryninja
.github/binja/download_headless.py
.github/binja/BinaryNinja-headless.zip
justfile

View File

@@ -38,6 +38,7 @@ repos:
- "capa/"
- "scripts/"
- "tests/"
- "web/rules/scripts/"
always_run: true
pass_filenames: false
@@ -55,6 +56,7 @@ repos:
- "capa/"
- "scripts/"
- "tests/"
- "web/rules/scripts/"
always_run: true
pass_filenames: false
@@ -72,6 +74,7 @@ repos:
- "capa/"
- "scripts/"
- "tests/"
- "web/rules/scripts/"
always_run: true
pass_filenames: false
@@ -90,6 +93,7 @@ repos:
- "capa/"
- "scripts/"
- "tests/"
- "web/rules/scripts/"
always_run: true
pass_filenames: false
@@ -107,6 +111,7 @@ repos:
- "capa/"
- "scripts/"
- "tests/"
- "web/rules/scripts/"
always_run: true
pass_filenames: false

View File

@@ -4,6 +4,9 @@
### New Features
- add landing page https://mandiant.github.io/capa/ @williballenthin #2310
- add rules website https://mandiant.github.io/capa/rules @DeeyaSingh #2310
### Breaking Changes
### New Rules (0)

View File

@@ -172,7 +172,8 @@ scripts = [
[tool.deptry]
extend_exclude = [
"sigs",
"tests"
"tests",
"web",
]
# dependencies marked as first party, to inform deptry that they are local

View File

@@ -84,8 +84,7 @@ def main() -> int:
args = _parse_args()
try:
with Path(args.capa_output).open() as capa_output:
json_data = json.load(capa_output)
json_data = json.loads(Path(args.capa_output).read_text(encoding="utf-8"))
except ValueError:
logger.error("Input data was not valid JSON, input should be a capa json output file.")
return -1

1
web/public/.gitignore vendored Normal file
View File

@@ -0,0 +1 @@
rules/

File diff suppressed because one or more lines are too long

Binary file not shown.

After

Width:  |  Height:  |  Size: 154 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 108 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 73 KiB

BIN
web/public/img/capa.gif Normal file

Binary file not shown.

After

Width:  |  Height:  |  Size: 240 KiB

BIN
web/public/img/logo.png Normal file

Binary file not shown.

After

Width:  |  Height:  |  Size: 9.8 KiB

View File

@@ -1,50 +1,399 @@
<!DOCTYPE html>
<html>
<!doctype html>
<html lang="en" data-bs-theme="auto">
<head>
<meta charset="UTF-8" />
<meta charset="utf-8">
<meta name="viewport" content="width=device-width, initial-scale=1">
<meta name="description" content="">
<title>capa - extract capabilities from executable files</title>
<link rel="canonical" href="https://mandiant.github.io/capa/">
<link href="./css/bootstrap-5.3.3.min.css" rel="stylesheet" integrity="sha384-QWTKZyjpPEjISv5WaRU9OFeRpok6YctnYmDr5pNlyT2bRjXh0JMhjY6hW+ALEwIH">
<script src="./js/bootstrap-5.3.3.bundle.min.js" integrity="sha384-YvpcrYf0tY3lHB60NNkmXc5s9fDVZLESaAA55NDzOxhy9GkcIdslK1eN7N6jIeHz"></script>
<link rel="stylesheet" href="./rules/pagefind/pagefind-ui.css">
<link rel="stylesheet" href="./rules/pagefind/pagefind-modular-ui.css">
<script src="./rules/pagefind/pagefind-ui.js"></script>
<script defer src="https://cloud.umami.is/script.js" data-website-id="0bb8ff9e-fbcc-4ee2-9f9f-b337a2e8cc7f"></script>
<meta name="theme-color" content="#712cf9">
<link rel="icon" href="img/icon.ico" />
<meta name="viewport" content="width=device-width, initial-scale=1.0" />
<title>capa</title>
<style>
/*
Josh's Custom CSS Reset
https://www.joshwcomeau.com/css/custom-css-reset/
*/
*, *::before, *::after {
box-sizing: border-box;
:root {
/* from the icon */
--capa-blue: #2593d7;
--capa-blue-darker: #1d74aa;
--bs-primary: var(--capa-blue);
--bs-primary-rgb: var(--capa-blue);
}
* {
margin: 0;
a:not(.btn) {
color: var(--capa-blue);
text-decoration: none;
}
a:not(.btn):hover {
text-decoration: underline;
}
.btn-primary {
--bs-btn-bg: var(--capa-blue);
--bs-btn-border-color: var(--capa-blue);
}
.btn:hover {
background-color: var(--capa-blue-darker);
border-color: var(--capa-blue-darker);
}
.text-justify {
text-align: justify;
}
body {
line-height: 1.5;
-webkit-font-smoothing: antialiased;
overflow-x: clip;
}
img, picture, video, canvas, svg {
display: block;
max-width: 100%;
/* a link that looks like inline text, until you hover and see a colorful underline */
.understated-link {
color: var(--bs-body-color) !important;
text-decoration: none;
text-decoration-color: var(--capa-blue);
}
input, button, textarea, select {
font: inherit;
.understated-link:hover {
text-decoration: underline;
text-decoration-color: var(--capa-blue) !important;
}
p, h1, h2, h3, h4, h5, h6 {
overflow-wrap: break-word;
}
#root, #__next {
isolation: isolate;
.b-divider {
width: 100%;
height: 3rem;
background-color: rgba(0, 0, 0, .1);
border: solid rgba(0, 0, 0, .15);
border-width: 1px 0;
box-shadow: inset 0 .5em 1.5em rgba(0, 0, 0, .1), inset 0 .125em .5em rgba(0, 0, 0, .15);
}
</style>
<script defer src="https://cloud.umami.is/script.js" data-website-id="0bb8ff9e-fbcc-4ee2-9f9f-b337a2e8cc7f"></script>
</head>
<body>
<div style="height: 100%; display: flex; align-items: center; justify-content: center;">
<div>
<!-- this is centered -->
<main>
<header
class="d-flex flex-wrap justify-content-center py-1 mb-4 border-bottom fixed-top"
style="background-color: rgba(255,255,255,0.95);
box-shadow: 0 0.5rem 1rem rgba(0,0,0,0.05),inset 0 -1px 0 rgba(0,0,0,0.15);"
>
<a href="/" class="d-flex align-items-center mb-3 mb-md-0 me-md-auto">
<img src="./img/logo.png" height=48 />
</a>
<ul class="nav nav-pills">
<li class="nav-item d-flex align-items-center"><a href="#rules" class="nav-link text-dark">Rules</a></li>
<!-- TODO(williballenthin): create this section (currently doesn't exist and URL doesn't go anywhere) -->
<li class="nav-item d-flex align-items-center"><a href="#examples" class="nav-link text-dark">Examples</a></li>
<li class="nav-item d-flex align-items-center"><a href="#download" class="nav-link text-dark">Download</a></li>
</ul>
</header>
<div class="container my-5 mt-3" style="margin-top: 4rem !important;">
<div class="row p-4 pb-0 pe-lg-0 pt-lg-5 align-items-center">
<div class="col-lg-7 p-3 p-lg-5 pt-lg-3">
<h1 class="display-4 fw-bold lh-1 text-body-emphasis">
Extract capabilities from executable files
</h1>
<p class="lead text-justify">
capa is the FLARE team's
<a href="https://github.com/mandiant/capa" class="understated-link">free and open-source tool</a>
to identify capabilities in executable files.
<!-- TODO(williballenthin): add links to each one of these use cases -->
Triage unknown files, guide reverse engineering, and hunt across a corpus for novel malware.
Refer to capa's rule set as an encyclopedia of techniques used in real-world attacks,
pivoting across <a href="https://attack.mitre.org/" class="understated-link">MITRE ATT&CK</a>
and <a href="https://github.com/MBCProject/mbc-markdown" class="understated-link">Malware Behavior Catalog (MBC)</a>
references.
</p>
<div class="d-grid gap-2 d-md-flex justify-content-md-start mb-4 mb-lg-3">
<a href="#download" type="button" class="btn btn-primary bs-primary btn-lg px-4 me-md-2 fw-bold">Download</button>
<a href="./rules/" type="button" class="btn btn-outline-secondary btn-lg px-4">Browse Rules</a>
</div>
</div>
<div class="col-lg-4 offset-lg-1 p-0">
<img
class="rounded-lg-3 shadow-lg"
style="border: 4px solid #f8f9fb;"
src="./img/capa-default-pma0101.png"
alt="default capa output"
width="720">
</div>
</div>
</div>
<div class="b-divider"></div>
<div id="rules" class="px-4 mt-5 text-center border-bottom">
<h1 class="display-4 fw-bold text-body-emphasis">Rules Crafted by Experts</h1>
<div class="col-lg-6 mx-auto text-start">
<p class="lead mb-4 text-justify">
capa recognizes behaviors by matching rules crafted by expert reverse engineers.
<p class="mb-4 text-justify">
Rules describe logical combinations of features familiar to human analysts.
Things like:
</p>
<ul class="my-0 py-0 mx-3">
<li>API calls, like <code>CreateRemoteThread</code>,</li>
<li>integer constants, like <code>0x100000001b3 = FNV prime</code>,</li>
<li>string references, like <code>"ZIG_DEBUG_COLOR"</code>.</li>
</ul>
<p class="my-4 text-justify">
capa looks for these features within instructions, basic blocks, and functions, having already disassembled the input file.
In contrast to YARA, which primarily searches for sequences of bytes, capa rules describe features at the code-level.
</p>
<div class="d-grid gap-2 d-sm-flex justify-content-sm-center mb-4">
<a href="./rules/" type="button" class="btn btn-primary btn-lg px-4 me-sm-3">Browse Rules</a>
<a href="https://github.com/mandiant/capa-rules/blob/master/doc/format.md" type="button" class="btn btn-outline-secondary btn-lg px-4">Learn the Syntax</a>
</div>
<div id="search" class="mb-4"></div>
</div>
<div class="overflow-hidden" style="max-height: 38vh;">
<div class="container px-5">
<img
src="./img/capa-rule-create-socket.png"
class="img-fluid border shadow-lg mb-4"
alt="capa rule source"
width="700"
height="500"
loading="lazy"
/>
</div>
</div>
</div>
<!-- TODO(williballenthin): need section for examples -->
<!-- such as: -->
<!-- - default output -->
<!-- - verbose output -->
<!-- - capa Explorer IDA Plugin -->
<!-- - capa Explorer Ghidra Plugin -->
<!-- - capa Explorer web application -->
<!-- - JSON output -->
<!-- - VT integration -->
<!-- we'll want screenshots and/or styled text inline -->
<!-- maybe also a bunch (all) of result documents from PMA or testfiles that people can browse -->
<div class="b-divider"></div>
<!-- TODO(williballenthin): auto-generate this section from release notes and git history -->
<div class="container col-xxl-8 px-4 py-5">
<div class="row flex-lg-row-reverse align-items-center g-5">
<h1>What's New</h1>
<h3 class="mt-3">Rule Updates</h3>
<ul class="mt-2 ps-5">
<!-- TODO(williballenthin): add date -->
<li>
added:
<a href="./rules/overwrite-dll-text-section-to-remove-hooks.html">
overwrite DLL .text section to remove hooks
</a>
</li>
<li>
added:
<a href="./rules/attach-bpf-to-socket-on-linux.html">
attach BPF to socket on Linux
</a>
</li>
</ul>
<h3 class="mt-3">Tool Updates</h3>
<h5 class="mt-2">v7.2.0</h5>
<!-- TODO(williballenthin): add date -->
<p class="mt-0">
<a href="https://github.com/mandiant/capa/releases/tag/v7.2.0">capa v7.2.0</a>
introduces a first version of capa explorer web: a web-based user interface to inspect capa results using your browser.
capa explorer web was worked on by @s-ff as part of a GSoC project, and it is available at https://mandiant.github.io/capa/explorer/.
This release also adds a feature extractor for output from the DRAKVUF sandbox. Now, analysts can pass the resulting drakmon.log file to capa and extract capabilities from the artifacts captured by the sandbox.
</p>
<p class="mt-0">New features:</p>
<ul class="mt-0 ps-5">
<li>webui: explore capa analysis results in a web-based UI online and offline</li>
<li>support analyzing DRAKVUF traces</li>
<li>IDA extractor: extract names from dynamically resolved APIs stored in renamed global variables</li>
<li>cli: add the ability to select which specific functions or processes to analyze</li>
</ul>
</div>
</div>
<div class="b-divider"></div>
<!-- TODO(williballenthin): this whole section's text is terrible, aside from the headers maybe -->
<div class="container px-4 py-5">
<div class="row row-cols-1 row-cols-md-2 align-items-md-center g-5">
<div class="col d-flex flex-column align-items-start gap-2">
<h2 class="fw-bold text-body-emphasis">
Integrates with Popular Reverse Engineering Tools
</h2>
<p class="text-body-secondary">
<!-- TODO(williballenthin): add links for each of these tools -->
capa uses the results of static or dynamic analysis to find the capabilities of programs.
The tool can rely on many different analysis backends, including IDA, Ghidra, Binary Ninja, CAPE, DRAKVUF, and VMRay.
The capa Explorer plugins provide interfaces for understanding the behaviors in programs, directly within reverse engineering tools.
</p>
</div>
<div class="col">
<div class="row row-cols-1 row-cols-sm-2 g-4">
<div class="col d-flex flex-column gap-2">
<h4 class="fw-semibold mb-0 text-body-emphasis">
IDA Pro
</h4>
<p class="text-body-secondary">
<!-- TODO(williballenthin): add link to find out more -->
Use the capa Explorer IDA Plugin to guide your reverse engineering, zeroing in on the interesting functions by behavior.
</p>
</div>
<div class="col d-flex flex-column gap-2">
<h4 class="fw-semibold mb-0 text-body-emphasis">
Ghidra
</h4>
<p class="text-body-secondary">
<!-- TODO(williballenthin): add link to find out more -->
Invoke Ghidra in headless mode to collect features for capa, or use the capa Explorer Ghidra plugin to understand key functions.
</p>
</div>
<div class="col d-flex flex-column gap-2">
<h4 class="fw-semibold mb-0 text-body-emphasis">
Binary Ninja
</h4>
<p class="text-body-secondary">
<!-- TODO(williballenthin): add link to find out more -->
Use Binary Ninja as the disassembler backend, relying on its state-of-the-art code analysis to recover capabilities.
</p>
</div>
<div class="col d-flex flex-column gap-2">
<h4 class="fw-semibold mb-0 text-body-emphasis">
CAPE
</h4>
<p class="text-body-secondary">
<!-- TODO(williballenthin): add link to find out more -->
Analyze the API trace captured by CAPE as it detonates malware, summarizing the behaviors seen across thousands of function calls.
</p>
</div>
</div>
</div>
</div>
</div>
<!-- TODO(williballenthin): need section on the capa Explorer web application -->
<div class="b-divider"></div>
<div id="download" class="px-4 pt-5 text-center border-bottom">
<h1 class="display-4 fw-bold text-body-emphasis">
Download capa
</h1>
<div class="col-lg-8 mx-auto">
<p class="lead">
We distribute capa as a standalone executable and as a Python library.
</p>
<div class="row flex-lg-row align-items-center g-5 py-2 text-start">
<div class="col-10 col-sm-8 col-lg-6">
<p class="mb-4 text-justify">
The standalone executable comes ready-to-go with 890+ embedded rules.
Use this when analyzing files from the terminal on Windows, Linux, and macOS.
There's a <a href="https://github.com/mandiant/capa/raw/master/doc/capa_quickstart.pdf">quickstart guide here</a> (PDF).
</p>
<p class="text-center">
<a href="https://github.com/mandiant/capa/releases/latest/" type="button" class="btn btn-primary me-md-2 mb-3 fw-bold">
Download standalone
</a>
</p>
<p class="text-justify">
capa is already included with malware analysis distributions like
<a href="https://github.com/mandiant/flare-vm" class="understated-link">FLARE-VM</a>
and <a href="https://remnux.org/" class="understated-link">REmnux</a>.
</p>
</div>
<div class="col-lg-6">
<p class="text-justify">
The Python library makes it easy to integrate capa into other projects,
like our plugins for IDA Pro and Ghidra.
Follow the <a href="https://github.com/mandiant/capa/blob/master/doc/installation.md">
instructions here
</a>,
which starts with installation it from PyPI like this:
</p>
<pre class="py-3 text-center" style="border: 2px solid #eeeeee;"><code>pip install flare-capa</code></pre>
<p class="text-justify">
<a href="https://github.com/mandiant/capa/blob/master/scripts/bulk-process.py">Here's an example</a>
of using capa as a library to process many files in parallel.
</p>
</div>
</div>
</div>
</div>
<div class="bg-dark text-secondary px-4 pt-5 text-center">
<div class="py-5">
<img src="./img/icon.png" />
<br />
<a href="./explorer/">capa Explorer Web<a>
<h3 class="display-5 fw-bold text-white">
capa
</h3>
<div class="col-lg-6 mx-auto">
<p class="fs-5 my-4">
The FLARE team's open-source tool to identify capabilities in executable files.
</p>
<div class="d-grid gap-4 d-sm-flex justify-content-sm-center mb-4">
<div><a href="#rules" class="fw-bold text-white text-decoration-none">Rules</a></div>
<div><a href="#examples" class="fw-bold text-white text-decoration-none">Examples</a></div>
<div><a href="#download" class="fw-bold text-white text-decoration-none">Download</a></div>
</div>
<p>
Copyright © 2024 Mandiant, Inc. All Rights Reserved. <br />
Licensed under the Apache License, Version 2.0.
</p>
</div>
</div>
</div>
</main>
<script>
window.addEventListener('DOMContentLoaded', (event) => {
new PagefindUI({
element: "#search",
showSubResults: true,
showEmptyFilters: false,
excerptLength: 15,
translations: {
placeholder: "Search rules",
},
});
});
</script>
</body>
</html>

File diff suppressed because one or more lines are too long

12
web/rules/.gitignore vendored Normal file
View File

@@ -0,0 +1,12 @@
_site
.sass-cache
.jekyll-cache
.jekyll-metadata
vendor
.direnv/
.env/
.envrc
file_modification_dates.txt
public/*.html
public/pagefind/
public/index.html

16
web/rules/README.md Normal file
View File

@@ -0,0 +1,16 @@
# capa rules documentation website
## requirements
- [just](https://github.com/casey/just)
- [pagefind](https://pagefind.app/)
- `pip install -r requirements`
## building
```
just clean
just build
````
then `just serve` and visit http://127.0.0.1:8000/ or (upload `./public` somewhere).

30
web/rules/justfile Normal file
View File

@@ -0,0 +1,30 @@
modified-dates:
python scripts/modified-dates.py ../../rules/ ./file_modification_dates.txt
build-rules:
mkdir -p ./public/rules/
python scripts/build_rules.py ../../rules/ ./file_modification_dates.txt ./public/
build-root:
python scripts/build_root.py ../../rules/ ./file_modification_dates.txt ./public/
index-website: build-rules build-root
pagefind --site "public"
build: modified-dates build-rules build-root index-website
clean:
rm -f file_modification_dates.txt
rm -f public/index.html
rm -rf public/*.html
rm -rf public/pagefind
serve:
python -m http.server --b localhost --directory ./public

File diff suppressed because one or more lines are too long

View File

@@ -0,0 +1,213 @@
:root {
--pagefind-ui-scale: 0.8;
--pagefind-ui-primary: #034AD8;
--pagefind-ui-fade: #707070;
--pagefind-ui-text: #393939;
--pagefind-ui-background: #ffffff;
--pagefind-ui-border: #eeeeee;
--pagefind-ui-tag: #eeeeee;
--pagefind-ui-border-width: 2px;
--pagefind-ui-border-radius: 8px;
--pagefind-ui-image-border-radius: 8px;
--pagefind-ui-image-box-ratio: 3 / 2;
--pagefind-ui-font: system, -apple-system, ".SFNSText-Regular",
"San Francisco", "Roboto", "Segoe UI", "Helvetica Neue",
"Lucida Grande", sans-serif;
}
[data-pfmod-hidden] {
display: none !important;
}
[data-pfmod-suppressed] {
opacity: 0 !important;
pointer-events: none !important;
}
[data-pfmod-sr-hidden] {
-webkit-clip: rect(0 0 0 0) !important;
clip: rect(0 0 0 0) !important;
-webkit-clip-path: inset(100%) !important;
clip-path: inset(100%) !important;
height: 1px !important;
overflow: hidden !important;
overflow: clip !important;
position: absolute !important;
white-space: nowrap !important;
width: 1px !important;
}
[data-pfmod-loading] {
color: var(--pagefind-ui-text);
background-color: var(--pagefind-ui-text);
border-radius: var(--pagefind-ui-border-radius);
opacity: 0.1;
pointer-events: none;
}
/* Input */
.pagefind-modular-input-wrapper {
position: relative;
}
.pagefind-modular-input-wrapper::before {
background-color: var(--pagefind-ui-text);
width: calc(18px * var(--pagefind-ui-scale));
height: calc(18px * var(--pagefind-ui-scale));
top: calc(23px * var(--pagefind-ui-scale));
left: calc(20px * var(--pagefind-ui-scale));
content: "";
position: absolute;
display: block;
opacity: 0.7;
-webkit-mask-image: url("data:image/svg+xml,%3Csvg width='18' height='18' viewBox='0 0 18 18' fill='none' xmlns='http://www.w3.org/2000/svg'%3E%3Cpath d='M12.7549 11.255H11.9649L11.6849 10.985C12.6649 9.845 13.2549 8.365 13.2549 6.755C13.2549 3.165 10.3449 0.255005 6.75488 0.255005C3.16488 0.255005 0.254883 3.165 0.254883 6.755C0.254883 10.345 3.16488 13.255 6.75488 13.255C8.36488 13.255 9.84488 12.665 10.9849 11.685L11.2549 11.965V12.755L16.2549 17.745L17.7449 16.255L12.7549 11.255ZM6.75488 11.255C4.26488 11.255 2.25488 9.245 2.25488 6.755C2.25488 4.26501 4.26488 2.255 6.75488 2.255C9.24488 2.255 11.2549 4.26501 11.2549 6.755C11.2549 9.245 9.24488 11.255 6.75488 11.255Z' fill='%23000000'/%3E%3C/svg%3E%0A");
mask-image: url("data:image/svg+xml,%3Csvg width='18' height='18' viewBox='0 0 18 18' fill='none' xmlns='http://www.w3.org/2000/svg'%3E%3Cpath d='M12.7549 11.255H11.9649L11.6849 10.985C12.6649 9.845 13.2549 8.365 13.2549 6.755C13.2549 3.165 10.3449 0.255005 6.75488 0.255005C3.16488 0.255005 0.254883 3.165 0.254883 6.755C0.254883 10.345 3.16488 13.255 6.75488 13.255C8.36488 13.255 9.84488 12.665 10.9849 11.685L11.2549 11.965V12.755L16.2549 17.745L17.7449 16.255L12.7549 11.255ZM6.75488 11.255C4.26488 11.255 2.25488 9.245 2.25488 6.755C2.25488 4.26501 4.26488 2.255 6.75488 2.255C9.24488 2.255 11.2549 4.26501 11.2549 6.755C11.2549 9.245 9.24488 11.255 6.75488 11.255Z' fill='%23000000'/%3E%3C/svg%3E%0A");
-webkit-mask-size: 100%;
mask-size: 100%;
z-index: 9;
pointer-events: none;
}
.pagefind-modular-input {
height: calc(64px * var(--pagefind-ui-scale));
padding: 0 calc(70px * var(--pagefind-ui-scale)) 0 calc(54px * var(--pagefind-ui-scale));
background-color: var(--pagefind-ui-background);
border: var(--pagefind-ui-border-width) solid var(--pagefind-ui-border);
border-radius: var(--pagefind-ui-border-radius);
font-size: calc(21px * var(--pagefind-ui-scale));
position: relative;
appearance: none;
-webkit-appearance: none;
display: flex;
width: 100%;
box-sizing: border-box;
font-weight: 700;
}
.pagefind-modular-input::placeholder {
opacity: 0.2;
}
.pagefind-modular-input-clear {
position: absolute;
top: calc(2px * var(--pagefind-ui-scale));
right: calc(2px * var(--pagefind-ui-scale));
height: calc(60px * var(--pagefind-ui-scale));
border-radius: var(--pagefind-ui-border-radius);
padding: 0 calc(15px * var(--pagefind-ui-scale)) 0 calc(2px * var(--pagefind-ui-scale));
color: var(--pagefind-ui-text);
font-size: calc(14px * var(--pagefind-ui-scale));
cursor: pointer;
background-color: var(--pagefind-ui-background);
border: none;
appearance: none;
}
/* ResultList */
.pagefind-modular-list-result {
list-style-type: none;
display: flex;
align-items: flex-start;
gap: min(calc(40px * var(--pagefind-ui-scale)), 3%);
padding: calc(30px * var(--pagefind-ui-scale)) 0 calc(40px * var(--pagefind-ui-scale));
border-top: solid var(--pagefind-ui-border-width) var(--pagefind-ui-border);
}
.pagefind-modular-list-result:last-of-type {
border-bottom: solid var(--pagefind-ui-border-width) var(--pagefind-ui-border);
}
.pagefind-modular-list-thumb {
width: min(30%,
calc((30% - (100px * var(--pagefind-ui-scale))) * 100000));
max-width: calc(120px * var(--pagefind-ui-scale));
margin-top: calc(10px * var(--pagefind-ui-scale));
aspect-ratio: var(--pagefind-ui-image-box-ratio);
position: relative;
}
.pagefind-modular-list-image {
display: block;
position: absolute;
left: 50%;
transform: translateX(-50%);
font-size: 0;
width: auto;
height: auto;
max-width: 100%;
max-height: 100%;
border-radius: var(--pagefind-ui-image-border-radius);
}
.pagefind-modular-list-inner {
flex: 1;
display: flex;
flex-direction: column;
align-items: flex-start;
margin-top: calc(10px * var(--pagefind-ui-scale));
}
.pagefind-modular-list-title {
display: inline-block;
font-weight: 700;
font-size: calc(21px * var(--pagefind-ui-scale));
margin-top: 0;
margin-bottom: 0;
}
.pagefind-modular-list-link {
color: var(--pagefind-ui-text);
text-decoration: none;
}
.pagefind-modular-list-link:hover {
text-decoration: underline;
}
.pagefind-modular-list-excerpt {
display: inline-block;
font-weight: 400;
font-size: calc(16px * var(--pagefind-ui-scale));
margin-top: calc(4px * var(--pagefind-ui-scale));
margin-bottom: 0;
min-width: calc(250px * var(--pagefind-ui-scale));
}
/* FilterPills */
.pagefind-modular-filter-pills-wrapper {
overflow-x: scroll;
padding: 15px 0;
}
.pagefind-modular-filter-pills {
display: flex;
gap: 6px;
}
.pagefind-modular-filter-pill {
flex-grow: 1;
flex-shrink: 1;
display: flex;
justify-content: center;
align-items: center;
background-color: var(--pagefind-ui-background);
color: var(--pagefind-ui-fade);
border: var(--pagefind-ui-border-width) solid var(--pagefind-ui-border);
border-radius: calc(25px * var(--pagefind-ui-scale));
font-size: calc(18px * var(--pagefind-ui-scale));
height: calc(50px * var(--pagefind-ui-scale));
cursor: pointer;
white-space: nowrap;
}
.pagefind-modular-filter-pill:hover {
border-color: var(--pagefind-ui-primary);
}
.pagefind-modular-filter-pill[aria-pressed="true"] {
border-color: var(--pagefind-ui-primary);
color: var(--pagefind-ui-primary);
}

File diff suppressed because one or more lines are too long

View File

@@ -0,0 +1,36 @@
/* latin-ext */
@font-face {
font-family: 'Poppins';
font-style: normal;
font-weight: 400;
font-display: swap;
src: url(https://fonts.gstatic.com/s/poppins/v21/pxiEyp8kv8JHgFVrJJnecmNE.woff2) format('woff2');
unicode-range: U+0100-02AF, U+0304, U+0308, U+0329, U+1E00-1E9F, U+1EF2-1EFF, U+2020, U+20A0-20AB, U+20AD-20C0, U+2113, U+2C60-2C7F, U+A720-A7FF;
}
/* latin */
@font-face {
font-family: 'Poppins';
font-style: normal;
font-weight: 400;
font-display: swap;
src: url(https://fonts.gstatic.com/s/poppins/v21/pxiEyp8kv8JHgFVrJJfecg.woff2) format('woff2');
unicode-range: U+0000-00FF, U+0131, U+0152-0153, U+02BB-02BC, U+02C6, U+02DA, U+02DC, U+0304, U+0308, U+0329, U+2000-206F, U+2074, U+20AC, U+2122, U+2191, U+2193, U+2212, U+2215, U+FEFF, U+FFFD;
}
/* latin-ext */
@font-face {
font-family: 'Poppins';
font-style: normal;
font-weight: 700;
font-display: swap;
src: url(https://fonts.gstatic.com/s/poppins/v21/pxiByp8kv8JHgFVrLCz7Z1JlFc-K.woff2) format('woff2');
unicode-range: U+0100-02AF, U+0304, U+0308, U+0329, U+1E00-1E9F, U+1EF2-1EFF, U+2020, U+20A0-20AB, U+20AD-20C0, U+2113, U+2C60-2C7F, U+A720-A7FF;
}
/* latin */
@font-face {
font-family: 'Poppins';
font-style: normal;
font-weight: 700;
font-display: swap;
src: url(https://fonts.gstatic.com/s/poppins/v21/pxiByp8kv8JHgFVrLCz7Z1xlFQ.woff2) format('woff2');
unicode-range: U+0000-00FF, U+0131, U+0152-0153, U+02BB-02BC, U+02C6, U+02DA, U+02DC, U+0304, U+0308, U+0329, U+2000-206F, U+2074, U+20AC, U+2122, U+2191, U+2193, U+2212, U+2215, U+FEFF, U+FFFD;
}

View File

@@ -0,0 +1,22 @@
.pagefind-ui__result-thumb {
display: none;
}
:root {
/* from the icon */
--capa-blue: #2593d7;
--capa-blue-darker: #1d74aa;
--bs-primary: var(--capa-blue);
--bs-primary-rgb: var(--capa-blue);
}
a:not(.btn) {
color: var(--capa-blue);
text-decoration: none;
}
a:not(.btn):hover {
text-decoration: underline;
text-decoration-color: var(--capa-blue) !important;
}

Binary file not shown.

After

Width:  |  Height:  |  Size: 15 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 7.4 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 9.8 KiB

File diff suppressed because one or more lines are too long

File diff suppressed because one or more lines are too long

File diff suppressed because one or more lines are too long

View File

@@ -0,0 +1,3 @@
pyyaml==6.0.2
pygments==2.18.0
-e ../.. # capa

View File

@@ -0,0 +1,343 @@
"""
Copyright (C) 2024 Mandiant, Inc. All Rights Reserved.
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at: [package root]/LICENSE.txt
Unless required by applicable law or agreed to in writing, software distributed under the License
is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and limitations under the License.
"""
import sys
import random
import logging
from typing import Dict, List
from pathlib import Path
import capa.rules
logger = logging.getLogger(__name__)
start_dir = Path(sys.argv[1])
txt_file_path = Path(sys.argv[2])
out_dir = Path(sys.argv[3])
output_html_path = out_dir / "index.html"
assert start_dir.exists(), "input directory must exist"
assert txt_file_path.exists(), "file-modification txt file must exist"
assert out_dir.exists(), "output directory must exist"
predefined_colors = [
"#9CAFAA",
"#577590",
"#a98467",
"#D6DAC8",
"#adc178",
"#f4d35e",
"#85182a",
"#d6c399",
"#dde5b6",
"#8da9c4",
"#fcd5ce",
"#706993",
"#FBF3D5",
"#1a659e",
"#c71f37",
"#EFBC9B",
"#7e7f9a",
]
def read_file_paths(txt_file_path: Path):
categorized_files: Dict[str, List[Path]] = {
"modified in the last day": [],
"modified in the last week": [],
"modified in the last month": [],
"modified in the last three months": [],
"modified in the last year": [],
"older": [],
}
lines = txt_file_path.read_text(encoding="utf-8").splitlines()
current_category = None
for line in lines:
line = line.strip()
if not line:
continue
if "===" in line:
category = line.strip("=").strip()
if category in categorized_files:
current_category = category
else:
logger.warning("Unrecognized category '%s'", category)
current_category = None
elif current_category:
parts = line.split(" ", 1)
if len(parts) == 2:
file_path, last_modified_date_str = parts
categorized_files[current_category].append(Path(file_path))
else:
logger.warning("Skipping line due to unexpected format: %s", line)
return categorized_files
def parse_rule(file_path: Path):
rule = capa.rules.Rule.from_yaml_file(file_path)
return {
"name": rule.name,
"namespace": rule.meta.get("namespace", ""),
"authors": rule.meta.get("authors", []),
"path": file_path,
"filename": file_path.name,
}
def generate_color():
return "#{:06x}".format(random.randint(0, 0xFFFFFF))
def get_first_word(namespace):
return namespace.split("/")[0] if "/" in namespace else namespace
def generate_html(categories_data, color_map):
html_content = """<!DOCTYPE html>
<html lang="en">
<head>
<meta charset="UTF-8">
<meta name="viewport" content="width=device-width, initial-scale=1.0">
<title>capa rules</title>
<link rel="stylesheet" href="./pagefind/pagefind-ui.css">
<link rel="stylesheet" href="./pagefind/pagefind-modular-ui.css">
<link href="./pagefind/pagefind-ui.css" rel="stylesheet">
<link href="./css/poppins.css" rel="stylesheet">
<link href="./css/bootstrap-5.3.3.min.css" rel="stylesheet">
<link rel="icon" href="./img/favicon.png" type="image/x-icon"> <!-- Favicon -->
<script src="./pagefind/pagefind-ui.js"></script>
<script defer src="https://cloud.umami.is/script.js" data-website-id="0bb8ff9e-fbcc-4ee2-9f9f-b337a2e8cc7f"></script>
<link rel="stylesheet" type="text/css" href="./css/style.css">
<style>
body {
background-color: #ffffff;
font-family: 'Poppins', sans-serif;
margin: 0;
padding: 0;
}
.container-fluid {
padding: 0 40px;
}
.row {
display: grid;
grid-template-columns: repeat(4, 1fr);
gap: 10px;
}
.card-wrapper {
display: flex;
align-items: stretch;
}
.card {
background-color: #FFFFFF;
border-radius: 10px;
box-shadow: 0px 0px 0px 0px #E5EDFF;
padding: 10px;
transition: box-shadow 0.3s ease-in-out;
display: flex;
flex-direction: column;
width: 100%;
}
.card:hover {
box-shadow: 10px 10px 0px 0px #E5EDFF;
}
.thin-rectangle {
width: 15px;
height: 40px;
position: absolute;
top: 20px;
left: 0;
}
.namespace {
color: grey;
font-size: 13px;
}
.rule-name a {
color: black;
font-weight: bold;
font-size: 14.5px;
}
.rule-name a:hover {
text-decoration: none;
}
.authors {
color: black;
font-size: 13px;
}
.tags {
margin-top: 10px;
}
.tag {
background-color: hsl(210, 98%, 80%);
border: none;
border-radius: 20px;
color: black;
font: 600 1.05rem/1 "Poppins", sans-serif;
padding: 0.5em 1.5em;
margin-right: 5px;
display: inline-block;
font-size: 12px;
text-transform: lowercase;
transition: transform 0.3s;
}
.tag:hover {
transform: scale(1.1);
}
.card-body {
flex: 1;
}
a {
color: inherit;
text-decoration: none;
}
a:hover {
text-decoration: underline;
}
</style>
</head>
<body>
<header
class="d-flex flex-wrap justify-content-center py-1 mb-4 border-bottom fixed-top"
style="background-color: rgba(255,255,255,0.95);
box-shadow: 0 0.5rem 1rem rgba(0,0,0,0.05),inset 0 -1px 0 rgba(0,0,0,0.15);"
>
<a href="/capa/" class="d-flex align-items-center mb-3 mb-md-0 me-md-auto">
<img src="./img/logo.png" height=48 />
</a>
<ul class="nav nav-pills">
<li class="nav-item d-flex align-items-center"><a href="/capa/#rules" class="nav-link text-dark">Rules</a></li>
<li class="nav-item d-flex align-items-center"><a href="/capa/#examples" class="nav-link text-dark">Examples</a></li>
<li class="nav-item d-flex align-items-center"><a href="/capa/#download" class="nav-link text-dark">Download</a></li>
</ul>
</header>
<div class="container-fluid" style="margin-top: 5rem !important;">
<div id="search" class="my-4"></div>
"""
for category, files in categories_data.items():
if not files:
continue
html_content += f'<h4>{category}</h4><div class="row mb-4">'
cards_data = []
for file_path in files:
try:
card_data = parse_rule(file_path)
cards_data.append(card_data)
except Exception as e:
logger.error("error parsing %s: %s", file_path, e)
for card in cards_data:
first_word = get_first_word(card["namespace"])
rectangle_color = color_map[first_word]
file_name = card["filename"].rpartition(".yml")[0]
card_html = f"""
<div class="card-wrapper">
<div class="card">
<div class="thin-rectangle" style="background-color: {rectangle_color};"></div>
<div class="card-body">
<div class="namespace">{card['namespace']}</div>
<div class="rule-name"><a href="./{file_name}.html">{card['name']}</a></div>
<div class="authors">{', '.join(card['authors'])}</div>
</div>
</div>
</div>"""
html_content += card_html
num_cards = len(cards_data)
num_empty_cells = (4 - (num_cards % 4)) % 4
if num_empty_cells > 0:
for _ in range(num_empty_cells):
html_content += """
<div class="card-wrapper">
<div class="card" style="visibility: hidden;"></div>
</div>"""
html_content += "</div>"
html_content += """
</div>
<script>
window.addEventListener('DOMContentLoaded', (event) => {
const search = new PagefindUI({
element: "#search",
showSubResults: true,
showEmptyFilters: false,
excerptLength: 15,
});
const params = new URLSearchParams(window.location.search);
const q = params.get("q");
if (q) {
console.log("initial query:", q)
search.triggerSearch(q)
}
});
</script>
</body>
</html>"""
output_html_path.write_text(html_content, encoding="utf-8")
categories_data = read_file_paths(txt_file_path)
color_map = {}
used_colors = set(predefined_colors)
color_index = 0
all_files = [file for category in categories_data.values() for file in category]
for file_path in all_files:
try:
card_data = parse_rule(file_path)
first_word = get_first_word(card_data["namespace"])
if first_word not in color_map:
if color_index < len(predefined_colors):
color_map[first_word] = predefined_colors[color_index]
color_index += 1
else:
new_color = generate_color()
while new_color in used_colors:
new_color = generate_color()
color_map[first_word] = new_color
used_colors.add(new_color)
except Exception as e:
logger.error("error parsing %s: %s", file_path, e)
generate_html(categories_data, color_map)
logger.info("HTML file has been generated: %s", output_html_path)

View File

@@ -0,0 +1,136 @@
"""
Copyright (C) 2024 Mandiant, Inc. All Rights Reserved.
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at: [package root]/LICENSE.txt
Unless required by applicable law or agreed to in writing, software distributed under the License
is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and limitations under the License.
"""
import os
import sys
import urllib.parse
from glob import glob
from pathlib import Path
import pygments
from pygments.lexers import YamlLexer
from pygments.formatters import HtmlFormatter
import capa.rules
input_directory = Path(sys.argv[1])
txt_file_path = Path(sys.argv[2])
output_directory = Path(sys.argv[3])
assert input_directory.exists(), "input directory must exist"
assert txt_file_path.exists(), "file-modification txt file must exist"
assert output_directory.exists(), "output directory must exist"
def convert_yaml_to_html(timestamps, yaml_file: Path, output_dir: Path):
rule_content = yaml_file.read_text(encoding="utf-8")
rule = capa.rules.Rule.from_yaml(rule_content, use_ruamel=True)
filename = os.path.basename(yaml_file).rpartition(".yml")[0]
namespace = rule.meta.get("namespace", "")
timestamp = timestamps[yaml_file.as_posix()]
rendered_rule = pygments.highlight(
rule_content,
YamlLexer(),
HtmlFormatter(
style="xcode",
noclasses=True,
wrapcode=True,
nobackground=True,
),
)
gh_link = f"https://github.com/mandiant/capa-rules/tree/master/{namespace}/{filename}.yml"
vt_query = 'behavior_signature:"' + rule.name + '"'
vt_fragment = urllib.parse.quote(urllib.parse.quote(vt_query))
vt_link = f"https://www.virustotal.com/gui/search/{vt_fragment}/files"
ns_query = f'"namespace: {namespace} "'
ns_link = f"./?{urllib.parse.urlencode({'q': ns_query})}"
html_content = f"""
<!DOCTYPE html>
<html lang="en">
<head>
<meta charset="UTF-8">
<meta name="viewport" content="width=device-width, initial-scale=1.0">
<title>{rule.name}</title>
<link rel="icon" href="./img/favicon.ico" type="image/x-icon">
<link rel="stylesheet" href="./css/bootstrap-5.3.3.min.css">
<script src="./js/jquery-3.5.1.slim.min.js"></script>
<script src="./js/bootstrap-5.3.3.bundle.min.js"></script>
<script defer src="https://cloud.umami.is/script.js" data-website-id="0bb8ff9e-fbcc-4ee2-9f9f-b337a2e8cc7f"></script>
<link rel="stylesheet" type="text/css" href="./css/style.css">
<style>
.rule-content .highlight pre {{
overflow: visible;
}}
</style>
</head>
<body>
<header
class="d-flex flex-wrap justify-content-center py-1 mb-4 border-bottom fixed-top"
style="background-color: rgba(255,255,255,0.95);
box-shadow: 0 0.5rem 1rem rgba(0,0,0,0.05),inset 0 -1px 0 rgba(0,0,0,0.15);"
>
<a href="/capa/" class="d-flex align-items-center mb-3 mb-md-0 me-md-auto">
<img src="./img/logo.png" height=48 />
</a>
<ul class="nav nav-pills">
<li class="nav-item d-flex align-items-center"><a href="/capa/#rules" class="nav-link text-dark">Rules</a></li>
<li class="nav-item d-flex align-items-center"><a href="/capa/#examples" class="nav-link text-dark">Examples</a></li>
<li class="nav-item d-flex align-items-center"><a href="/capa/#download" class="nav-link text-dark">Download</a></li>
</ul>
</header>
<div class="container d-flex justify-content-center" style="margin-top: 4rem !important;">
<div style="max-width: 650px;">
<p class="lead mb-0 text-secondary">
<a href="{ns_link}" class="text-secondary">
{namespace}
</a>
</p>
<h1 class="display-6">{rule.name}</h1>
<ul style="display: block; position: relative; float: right; height: 0px;" class="mt-4">
<li><a href="{gh_link}">edit on GitHub</a></li>
<li><a href="{vt_link}">search on VirusTotal</a></li>
</ul>
<div class="mt-4 rule-content" data-pagefind-body>
{rendered_rule}
</div>
<p class="text-secondary">last edited: {timestamp}</p>
</div>
</div>
</body>
</html>
"""
output_dir.mkdir(parents=True, exist_ok=True)
output_file_path = output_dir / (filename + ".html")
output_file_path.write_text(html_content, encoding="utf-8")
yaml_files = glob(os.path.join(input_directory, "**/*.yml"), recursive=True)
timestamps = {}
for line in txt_file_path.read_text(encoding="utf-8").splitlines():
if not line:
continue
if line.startswith("==="):
continue
path, _, timestamp = line.partition(" ")
timestamps[path] = timestamp
for yaml_file in yaml_files:
convert_yaml_to_html(timestamps, Path(yaml_file), output_directory)

View File

@@ -0,0 +1,92 @@
"""
Copyright (C) 2024 Mandiant, Inc. All Rights Reserved.
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at: [package root]/LICENSE.txt
Unless required by applicable law or agreed to in writing, software distributed under the License
is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and limitations under the License.
"""
import os
import sys
import logging
import subprocess
from pathlib import Path
from datetime import datetime, timedelta
logger = logging.getLogger(__name__)
start_dir = Path(sys.argv[1])
output_file = Path(sys.argv[2])
assert start_dir.exists(), "start directory must exist"
def get_yml_files_and_dates(start_dir: Path):
yml_files = []
for root, _, files in os.walk(start_dir):
for file in files:
if file.endswith(".yml") or file.endswith(".yaml"):
file_path = Path(root) / file
proc = subprocess.run(
[
"git",
"log",
"-1", # only show most recent commit
'--pretty="%ct"', # unix timestmp, https://git-scm.com/docs/pretty-formats#Documentation/pretty-formats.txt-emctem
file, # just the filename, will run from the containing directory
],
cwd=root, # the directory with the file we're inspecting
check=True,
capture_output=True,
)
last_modified_date = int(proc.stdout.decode("utf-8").partition("\n")[0].strip('"'))
yml_files.append((file_path, last_modified_date))
return yml_files
yml_files_and_dates = get_yml_files_and_dates(start_dir)
yml_files_and_dates.sort(key=lambda x: x[1], reverse=True)
current_date = datetime.now()
categories = [
("modified in the last day", current_date - timedelta(days=1)),
("modified in the last week", current_date - timedelta(days=7)),
("modified in the last month", current_date - timedelta(days=30)),
("modified in the last three months", current_date - timedelta(days=90)),
("modified in the last year", current_date - timedelta(days=365)),
]
def write_category(f, category_name, files):
f.write(f"=== {category_name} ===\n")
for file_path, last_modified_date in files:
last_modified_date_str = datetime.fromtimestamp(last_modified_date).strftime("%Y-%m-%d %H:%M:%S")
f.write(f"{file_path} {last_modified_date_str}\n")
f.write("\n")
with output_file.open("wt", encoding="utf-8") as f:
for title, delta in categories:
current_files = []
for file_path, last_modified_date in yml_files_and_dates:
last_modified_date_dt = datetime.fromtimestamp(last_modified_date)
if last_modified_date_dt > delta:
current_files.append((file_path, last_modified_date))
write_category(f, title, current_files)
for item in current_files:
yml_files_and_dates.remove(item)
write_category(f, "older", yml_files_and_dates)
logger.info("File names and modification dates have been written to %s", output_file)