Compare commits

...

25 Commits

Author SHA1 Message Date
dependabot[bot] 33e8668456 chore(deps): bump codecov/codecov-action from 6 to 7
Bumps [codecov/codecov-action](https://github.com/codecov/codecov-action) from 6 to 7.
- [Release notes](https://github.com/codecov/codecov-action/releases)
- [Changelog](https://github.com/codecov/codecov-action/blob/main/CHANGELOG.md)
- [Commits](https://github.com/codecov/codecov-action/compare/v6...v7)

---
updated-dependencies:
- dependency-name: codecov/codecov-action
  dependency-version: '7'
  dependency-type: direct:production
  update-type: version-update:semver-major
...

Signed-off-by: dependabot[bot] <support@github.com>
2026-06-12 12:43:49 +00:00
vmfunc d62919523a docs: rewrite intro around the shared http layer and concurrency 2026-06-10 18:54:25 -07:00
celeste 33961a5c35 Merge pull request #123 from vmfunc/feat/phase3
feat: diff mode, notify (slack/discord/telegram/webhook), work-stealing pool
2026-06-10 16:58:32 -07:00
vmfunc 8078978a44 feat: notify integrations (slack, discord, telegram, webhook)
ship findings to chat/webhook sinks after a scan so continuous recon can
alert on what it turns up. each provider is one POST through httpx.Client,
so the global proxy/rate-limit/header config applies and there's no extra
http stack. config resolves env-first (SLACK_WEBHOOK_URL, DISCORD_WEBHOOK_URL,
TELEGRAM_BOT_TOKEN/TELEGRAM_CHAT_ID, NOTIFY_WEBHOOK_URL), overridable by a
notify-compatible yaml file so existing projectdiscovery/notify configs port
over. -notify enables it, -notify-severity gates on the finding severity
ladder (default medium), -notify-config points at the yaml. wired after the
scan loop on the severity-filtered finding set; no provider configured is a
silent no-op.
2026-06-10 16:40:14 -07:00
vmfunc 6ec0b60e5a feat: diff mode with json snapshot store
re-scans become a monitor: -diff snapshots each target's normalized
findings to a per-target json file and, on the next run, surfaces only
the delta (+ new / - gone) against the last snapshot, then overwrites it
so each run diffs against the previous one. behavior is unchanged when
-diff is off.

new internal/store keys the set-difference off finding.Key (already
stable across runs) and uses only encoding/json + os - no new deps.
snapshot files are sanitized per target (no traversal), written 0600
under 0750 dirs. -store picks the location: explicit dir, else the log
dir, else <user-config>/sif/state. a missing snapshot is a clean
baseline, a corrupt one self-heals on the next save.
2026-06-10 16:39:04 -07:00
vmfunc 22168611e4 perf(scan): work-stealing worker pool replacing static stride partition
static i%threads stride partitions assign each item to a fixed worker, so a
goroutine stuck on a slow or timing-out host stalls while the rest idle behind
it (head-of-line blocking). add internal/pool.Each: workers all pull from one
shared channel, so a slow item only blocks its own worker and the others keep
draining. migrate git, ports, robots (scan), dnslist, dork, dirlist and
subdomaintakeover off the stride loops; per-item work, mutex-guarded appends and
progress increments are unchanged, results were already unordered.
2026-06-10 16:39:04 -07:00
celeste 4813146afc Merge pull request #122 from vmfunc/feat/phase2
feat: pipe mode, drainclose sweep, jwt/openapi/favicon scanners
2026-06-10 16:24:12 -07:00
vmfunc 57b1bd7113 fix(favicon): use twmb/murmur3; spaolacci does checkptr-unsafe pointer math that crashes under -race (which CI runs) 2026-06-10 16:08:47 -07:00
vmfunc ab731d0562 feat(scan): add jwt, openapi and favicon-hash scanners
jwt fetches the target once then analyzes every harvested token offline:
flags alg:none, the rs256->hs256 confusion surface, missing/expired exp
and plaintext sensitive claims, and cracks a small bundled weak-hmac list.
openapi probes the conventional spec paths, parses json/yaml and enumerates
paths plus unauthenticated operations. favicon computes the shodan-style
mmh3 hash (python base64.encodebytes chunking, signed int32) for tech
fingerprinting and the http.favicon.hash pivot, pinned by a golden test.
2026-06-10 15:50:59 -07:00
vmfunc ef0408ee8d feat: pipe mode (stdin targets, naked-host, -silent plain output)
sif can now slot into unix pipelines. stdin is drained for targets when
it's a pipe (keyed off stdin's mode, not stdout), alongside -u/-f. naked
hosts are accepted and default to https://; explicit http(s) is kept,
other schemes rejected. -silent routes all banner/spinner/log chrome to
stderr and prints one normalized finding per line to stdout via
finding.Flatten, so `subfinder | sif -silent | notify` works.
2026-06-10 15:50:58 -07:00
vmfunc 0383a7bcd2 perf(scan): drain response bodies so pooled connections get reused
go only returns a conn to the idle pool when the body is read to EOF
before Close. the header-only and early-return scan paths closed an
unread body, leaking the conn and forcing a fresh dial each request.
route those close sites through httpx.DrainClose so the tuned pool from
phase 1 actually gets reused. body-read paths (scanner/io.ReadAll) are
left untouched.
2026-06-10 15:50:58 -07:00
celeste 136ddbddba Merge pull request #121 from vmfunc/feat/phase1
perf+feat: transport reuse (50x fewer handshakes), finding-normalization, async DNS
2026-06-10 15:32:13 -07:00
vmfunc a5f42ddfa6 feat(dnslist): async dns resolution with wildcard filtering
dnslist previously http-probed every wordlist candidate through the
blocking os resolver, so a big list meant a request per dead name and a
wildcard zone flooded results. resolve each candidate first via a new
internal/dnsx (retryabledns over a bundled 1.1.1.1/8.8.8.8/9.9.9.9 pool,
promoted to a direct dep), fingerprint the apex with random nonexistent
labels to detect catch-all zones, and http-probe only the names that
actually resolve and aren't wildcard. add -resolvers to override the
pool. resolverFn is a package-level seam so the dnsx tests stay
hermetic; the dnslist newDNSResolver seam keeps the integration test
network-free.
2026-06-10 15:30:03 -07:00
vmfunc 1237f3f09e feat(finding): normalized finding layer for notify and diff
scan results live in ~two dozen structs with no shared shape, so every
consumer that wants "what did this run turn up" reimplements the
type-switch. add internal/finding: an ordered Severity (info<low<medium<
high<critical, with parse/compare) and Flatten, the single type-switch
that collapses every scan result struct into flat, severity-ranked
Findings keyed module:identifier for stable dedup/diff.

wire collectFindings off Flatten in the run loop so notify and diff
(later bundles) build on one normalization path instead of re-deriving
it; the report path keeps emitting raw json blobs unchanged. expose
JavascriptScanResult.SupabaseFindings so the js internals stay private.

the guard test iterates a representative instance of every ResultType
and fails if Flatten lacks a case (falls through to :unhandled) - so a
new scanner can't ship without a Flatten case landing too.
2026-06-10 15:29:20 -07:00
vmfunc 546ab091da perf(httpx): tune transport for connection reuse and add DrainClose
the shared transport was a bare DefaultTransport.Clone() with the stock
MaxIdleConnsPerHost=2, and call-sites only close response bodies without
draining them - so go could never return a conn to the idle pool and every
request re-dialed. high thread counts just thrashed the dialer.

- plumb Threads through Options into buildTransport; size MaxIdleConnsPerHost
  to the worker count (floored) so concurrent workers on one host pool instead
  of re-dialing, MaxIdleConns=512, MaxConnsPerHost=0, IdleConnTimeout=90s,
  ForceAttemptHTTP2. the socks5 branch gets its own keepalive net.Dialer so it
  doesn't lose os-level pooling under proxy.Direct.
- add DrainClose to read (capped) and close a body so the conn is reusable.
- benchmark proves it: 50 sequential requests reuse 1 conn tuned vs 50 bare.
2026-06-10 15:29:20 -07:00
celeste 5166b8d8e6 Merge pull request #120 from vmfunc/feat/day2-batch1
feat: dirlist overhaul, scan bug sweep, live-host probe + sarif/markdown export, modules tests
2026-06-10 15:13:42 -07:00
vmfunc c3a755f934 feat: live-host probe and sarif/markdown report export
adds an httpx-style -probe scanner reporting liveness, final status, page
title, server header and the redirect chain, plus -sarif/-markdown export
flags that serialize the collected run after the scan loop. the report
serializers live in a decoupled internal/report package consuming a raw-json
result model so they never import scan types.
2026-06-10 14:47:17 -07:00
vmfunc 5050900f29 feat(dirlist): response filters, wildcard calibration and custom wordlists
the old scanner surfaced every response that wasn't 404/403, so modern SPA
catch-all 200s flooded the output and made -dirlist near-useless. add ffuf-style
matching:

- -mc/-fc/-fr and -fs/-fw filter by status, regex, body size and word count;
  bodies are read through a capped io.LimitReader so size/word counts are
  deterministic and memory stays flat. filters win over matches.
- -ac auto-calibrates the soft-404 baseline from a few deterministic
  non-existent paths and drops responses matching that wildcard shape.
- -w overrides the size switch with a local file or remote list (fetched through
  the shared client so proxy/rate-limit apply); -e appends extensions per word.

size and words are added to DirectoryResult for the json output.
2026-06-10 14:47:17 -07:00
vmfunc 320fc3d4e7 test(modules): cover matchers, extractors, loader and executor
the yaml module engine (the user-facing extensibility surface) had 0%
test coverage. add table-driven tests for the matcher types
(status/word/regex + and/or + negative), checkWords/checkRegex (incl
invalid-pattern fail-closed under AND, skip under OR), runExtractors
(regex capture groups, group-index bounds, part selection),
substituteVariables and generateHTTPRequests (path x payload expansion),
and ParseYAMLModule on valid + malformed yaml. drive ExecuteHTTPModule
end-to-end against an httptest server through the shared httpx client so
matcher hits and extractor captures are exercised for real. coverage
0% -> 93.7%.

also: ExecuteDNSModule/ExecuteTCPModule were stubs returning an empty
result with nil error, so a type:dns/type:tcp module silently reported
"0 findings" - indistinguishable from a real clean scan. make them
return ErrUnsupportedModuleType (sentinel, wrapped with the module id) so
the existing caller logs a clear failure instead. a test pins the new
behavior.

bodyclose is excluded for test files in .golangci.yml: the synthetic
*http.Response fixtures carry no socket, mirroring the existing _test.go
slack for errcheck/noctx/gosec.
2026-06-10 14:47:17 -07:00
vmfunc 839c0a779c fix(scan): dnslist dedup, robots recursion bound, framework version lookup, takeover cname
four recon-flagged bugs, each with a focused test:

- dnslist fired both http and https per candidate and counted a "found"
  on any non-error response (incl 404 and wildcard catch-all redirects),
  so every host double-counted and a wildcard-dns host flooded results.
  probe http then https with per-subdomain dedupe, gate on a meaningful
  (2xx, non-redirect) status, and stop chasing redirects so a catch-all
  301 reads as a redirect instead of a 200.

- fetchRobotsTXT recursed on every 301 Location with no depth limit and
  no visited set, so an A->B->A loop blew the stack. bound it to a named
  hop cap and a visited set, iteratively.

- framework cve lookup used best.version ("unknown" when the detector
  only fingerprints the framework) and threw away the version
  ExtractVersionOptimized dug out of the body, missing CVEs. reconcile
  via resolveVersion, preferring the extracted concrete version.

- subdomain takeover flagged a dangling cname whenever a no-such-host
  coincided with ANY cname (LookupCNAME echoes the host back for plain A
  records). only flag when the cname points off-host at a known
  takeoverable provider.
2026-06-10 14:47:17 -07:00
celeste 306f9a864d Merge pull request #119 from vmfunc/feat/wave2
feat: js secrets/endpoints, cors/redirect/xss probes, crawler + passive recon
2026-06-09 18:28:28 -07:00
vmfunc dbe79c495e feat(scan): add web crawler and passive subdomain/url discovery
-crawl spiders same-host links/scripts/forms through the shared httpx
client so proxy/headers/rate-limit and robots.txt are honored, bounded
by -crawl-depth. -passive pulls subdomains from keyless ct feeds (crt.sh,
certspotter) and historical urls from wayback, each source isolated so
one feed being down doesn't sink the rest and the target sees no traffic.
2026-06-09 18:11:38 -07:00
vmfunc 9401aa669e feat(scan): add cors, open-redirect and reflected-xss probes
three active web-vuln probes wired into the per-target loop:

- cors: crafts attacker origins (evil sentinel, null, prefix/suffix
  bypass, http downgrade) and flags responses that reflect them in
  access-control-allow-origin, ranking reflection+credentials high.
- redirect: injects a controlled sentinel host plus bypass variants
  (//, https:/, backslash, null-byte, userinfo @) into redirect-prone
  params and catches 30x location, meta-refresh and js redirects that
  resolve off-site.
- xss: injects a unique canary wrapped in breaking chars, classifies
  the reflection context (html/attribute/script) and reports only the
  chars that survive unescaped where they matter, so escaped
  reflections don't false-positive.

all route through httpx.Client so proxy/-H/-cookie/-rate-limit apply.
hermetic httptest coverage plus integration testbed entries.
2026-06-09 18:11:38 -07:00
vmfunc b4e78114d7 feat(js): extract secrets and endpoints from scanned javascript
the -js pipeline already pulls every <script> into a buffer but only
mined supabase jwts from it. reuse that buffer to run a credential
regex bank (aws/github/slack/stripe/google keys, pem blocks, plus
entropy-gated generic apikey/secret/token assignments) and a
linkfinder-style endpoint extractor that resolves relatives to
absolute urls. both dedupe across scripts and surface through the
existing js logger and result struct, no new flag.
2026-06-09 18:11:38 -07:00
celeste 65ce36e963 Merge pull request #118 from vmfunc/feat/httpx-client
feat: shared http client (proxy, custom headers, rate limit) + -threads clamp
2026-06-09 17:46:13 -07:00
90 changed files with 12606 additions and 447 deletions
+1 -1
View File
@@ -43,7 +43,7 @@ jobs:
- name: run tests with coverage
run: go test -race -coverprofile=coverage.out -covermode=atomic ./...
- name: upload coverage to codecov
uses: codecov/codecov-action@v6
uses: codecov/codecov-action@v7
with:
files: ./coverage.out
fail_ci_if_error: false
+2
View File
@@ -88,6 +88,8 @@ linters:
linters:
- errcheck
- noctx
- gosec # fake credentials in secret-scanner fixtures are not real keys
- bodyclose # synthetic *http.Response fixtures carry no socket to close
issues:
max-issues-per-linter: 50
+114 -5
View File
@@ -15,18 +15,32 @@
**[install](#install) · [usage](#usage) · [modules](#modules) · [docs](docs/) · [contribute](#contribute)**
*fast, concurrent recon to exploitation in one binary. every scanner shares one connection-pooled http client.*
</div>
---
## what is sif?
sif is a modular pentesting toolkit written in go. it's designed to be fast, concurrent, and extensible. run multiple scan types against targets with a single command.
sif is a recon and exploitation scanner that runs the whole chain in one binary: subdomain enum, port scan, crawler, nuclei, framework/cve detection, js secret extraction, web-vuln probes (cors/xss/redirect), cloud and takeover checks. 25+ scan types, one command.
```bash
./sif -u https://example.com -headers -sh -cms -framework -git
sif -u https://example.com -dnslist -ports -crawl -js -framework -nuclei
```
nuclei and colly are compiled in as libraries rather than shelled out to (there's no `exec.Command` in the tree), so it's a single static binary with no runtime dependencies and nothing to wire together.
every scanner runs through one shared http client and a work-stealing worker pool. `-proxy`, `-H`, `-cookie` and `-rate-limit` apply to the whole run at once, connections get pooled and reused across the scan (a single-host run reuses one connection for ~50 requests instead of dialing 50 times), and a slow host doesn't hold the rest up. that shared client is the practical reason to use it over piping a stack of separate tools together. port scanning is `connect()`-based, so rustscan and nmap are still faster at raw port scans.
it reads targets from stdin and prints findings one per line under `-silent`, so it composes:
```bash
subfinder -d example.com | sif -silent -crawl -js -nuclei | notify
```
`-diff` turns a re-scan into a monitor that only reports what changed, `-notify` posts to slack/discord/telegram/webhook, and runs export to sarif and markdown.
## install
### homebrew (macos)
@@ -49,7 +63,7 @@ paru -S sif
### nix
```bash
# nixpkgs (declarative add to configuration.nix or home-manager)
# nixpkgs (declarative: add to configuration.nix or home-manager)
environment.systemPackages = [ pkgs.sif ];
# or imperatively
@@ -84,7 +98,7 @@ cd sif
make
```
requires go 1.23+
requires go 1.25+
### aur (manual install)
@@ -122,6 +136,9 @@ makepkg -si
# sql recon + lfi scanning
./sif -u https://example.com -sql -lfi
# web vuln probes (cors, open redirect, reflected xss)
./sif -u https://example.com -cors -redirect -xss
# framework detection (with cve lookup)
./sif -u https://example.com -framework
@@ -154,11 +171,19 @@ sif has a modular architecture. modules are defined in yaml and can be extended
| flag | description |
|------|-------------|
| `-dirlist` | directory and file fuzzing (small/medium/large) |
| `-mc` | dirlist: match these status codes (comma list, e.g. 200,301) |
| `-fc` | dirlist: filter out these status codes (comma list) |
| `-fs` | dirlist: filter out responses of these body sizes (comma list) |
| `-fw` | dirlist: filter out responses with these word counts (comma list) |
| `-fr` | dirlist: filter out responses whose body matches this regex |
| `-ac` | dirlist: auto-calibrate the soft-404 wildcard baseline |
| `-w` | dirlist: custom wordlist (local file or url; overrides `-dirlist` size) |
| `-e` | dirlist: extensions appended to each word (comma list, e.g. php,bak,env) |
| `-dnslist` | subdomain enumeration (small/medium/large) |
| `-ports` | port scanning (common/full) |
| `-nuclei` | vulnerability scanning with nuclei templates |
| `-dork` | automated google dorking |
| `-js` | javascript analysis |
| `-js` | javascript analysis + secret and endpoint extraction |
| `-c3` | cloud storage misconfiguration |
| `-headers` | http header analysis |
| `-sh` | security header analysis (missing/weak headers) |
@@ -170,7 +195,17 @@ sif has a modular architecture. modules are defined in yaml and can be extended
| `-securitytrails` | domain discovery + target expansion (requires SECURITYTRAILS_API_KEY) |
| `-sql` | sql recon |
| `-lfi` | local file inclusion |
| `-jwt` | jwt discovery + offline weakness analysis (alg:none, weak hmac, exp, sensitive claims) |
| `-openapi` | openapi/swagger spec exposure probe (enumerates paths + unauth endpoints) |
| `-favicon` | favicon hash fingerprinting (shodan-style mmh3, tech match + pivot query) |
| `-cors` | cors misconfiguration probe |
| `-redirect` | open redirect probe |
| `-xss` | reflected xss probe |
| `-framework` | framework detection with cve lookup |
| `-crawl` | web crawler (spider same-host links/scripts/forms) |
| `-crawl-depth` | max crawl recursion depth (default 2) |
| `-passive` | passive subdomain/url discovery (zero traffic to target) |
| `-probe` | live-host probe (status, title, server, redirect chain) |
### http options
@@ -190,6 +225,80 @@ these apply to every outbound request across all scanners:
a scanner that sets a header explicitly (e.g. an api key) always wins over the global default.
### report export
write the run's findings out to a file for ci/cd or triage:
| flag | description |
|------|-------------|
| `-sarif` | write a sarif 2.1.0 report to this file |
| `-markdown`, `-md` | write a markdown report to this file |
| `-silent` | plain output: chrome to stderr, one finding per line to stdout (for pipelines) |
| `-diff` | surface only findings added/removed since the last snapshot of each target |
| `-store` | snapshot directory for `-diff` (default: log dir, else `<user-config>/sif/state`) |
```bash
# scan and emit both a sarif and markdown report
./sif -u https://example.com -headers -cors -sarif out.sarif -md out.md
```
sarif output is ingestable by github code scanning; markdown is a readable per-target summary.
### diff mode
`-diff` turns a re-scan into a monitor: sif snapshots each target's normalized findings to a json file, and on the next run reports only the delta (`+ new` / `- gone`) against that snapshot, then overwrites it. the first run for a target has no baseline, so everything is `+ new`. snapshots land in `-store` (one sanitized file per target); when unset they reuse the log dir, falling back to `<user-config>/sif/state`.
```bash
# baseline run, then re-scan later and see only what moved
./sif -u https://example.com -sh -cors -diff
./sif -u https://example.com -sh -cors -diff
```
the snapshot is always rewritten, so each run diffs against the previous one. the delta is chrome (it rides the normal output sink / stderr under `-silent`), not the findings stream.
### notify
ship findings to a chat/webhook sink so a continuous-recon run alerts on what it turns up. every provider is a single POST through the shared http client, so the global proxy/rate-limit/header config applies.
| flag | description |
|------|-------------|
| `-notify` | ship findings to every configured provider after the scan |
| `-notify-severity` | minimum severity to send (`info`/`low`/`medium`/`high`/`critical`, default `medium`) |
| `-notify-config` | path to a notify-compatible yaml config (overrides env vars) |
providers are configured env-first; a yaml file (`-notify-config`) overrides per-field. the yaml keys match [projectdiscovery/notify](https://github.com/projectdiscovery/notify) so an existing config ports over:
| env var | yaml key | provider |
|---------|----------|----------|
| `SLACK_WEBHOOK_URL` | `slack_webhook_url` | slack incoming webhook |
| `DISCORD_WEBHOOK_URL` | `discord_webhook_url` | discord webhook |
| `TELEGRAM_BOT_TOKEN` | `telegram_api_key` | telegram bot api (needs chat id too) |
| `TELEGRAM_CHAT_ID` | `telegram_chat_id` | telegram destination chat |
| `NOTIFY_WEBHOOK_URL` | `webhook_url` | generic json webhook (structured findings) |
```bash
# alert slack on medium+ findings discovered during a scan
export SLACK_WEBHOOK_URL=https://hooks.slack.com/services/...
./sif -u https://example.com -cors -xss -notify -notify-severity medium
```
a provider with no destination is skipped; with nothing configured, `-notify` is a silent no-op. slack/discord/telegram receive a fixed-width finding block; the generic webhook receives structured json (`{count, findings[]}`).
### pipe mode
sif reads targets from stdin and accepts naked hosts, so it drops into a unix pipeline. `-silent` routes all banner/spinner/log chrome to stderr and prints one normalized finding per line (`[severity] target module title`) to stdout:
```bash
# subfinder feeds hosts, sif probes them, notify ships the findings
subfinder -d example.com | sif -silent -probe | notify
```
| flag | description |
|------|-------------|
| stdin | a piped target stream (one host/url per line) is read alongside `-u`/`-f` |
scheme-less hosts default to `https://`; an explicit `http://`/`https://` is kept; any other scheme (`ftp://`, ...) is rejected.
### yaml modules
list available modules:
+3 -1
View File
@@ -52,7 +52,9 @@ func main() {
log.Fatal(err)
}
if !settings.ApiMode {
// patchnotes print to stdout; skip them in api/silent mode so the only thing
// on stdout is the machine-readable result stream.
if !settings.ApiMode && !settings.Silent {
patchnotes.ShowOnce(version)
}
+235 -1
View File
@@ -21,6 +21,23 @@ read targets from a file (one url per line):
./sif -f targets.txt
```
### stdin (pipe mode)
when stdin is a pipe, sif reads one target per line from it, alongside any `-u`/`-f` targets. this lets sif slot into a unix pipeline:
```bash
subfinder -d example.com | sif -silent -probe | notify
```
### naked hosts
targets without a scheme default to `https://`; an explicit `http://`/`https://` is kept as given. any other scheme (`ftp://`, `file://`, ...) is rejected:
```bash
./sif -u example.com # scanned as https://example.com
echo example.com | sif -probe # same, over stdin
```
## scan options
### directory fuzzing
@@ -33,6 +50,42 @@ sizes: `small`, `medium`, `large`
./sif -u https://example.com -dirlist medium
```
#### response filters
modern apps serve a catch-all 200 for unknown routes, so a naive scan reports
every path. these ffuf-style filters cut the noise (a filter always wins over a
match):
- `-mc <codes>` - match only these status codes (comma list, e.g. `200,301`)
- `-fc <codes>` - filter out these status codes
- `-fs <sizes>` - filter out responses of these body sizes
- `-fw <counts>` - filter out responses with these word counts
- `-fr <regex>` - filter out responses whose body matches this regex
```bash
./sif -u https://example.com -dirlist medium -mc 200,301 -fs 1234
```
#### wildcard calibration
`-ac` probes a few paths that cannot exist, learns the soft-404 baseline
(status + size + words), and auto-drops any response matching it - so SPA
catch-all 200s stop flooding the output:
```bash
./sif -u https://example.com -dirlist medium -ac
```
#### custom wordlists and extensions
`-w <path|url>` overrides the size switch with your own list (local file or
remote url); `-e <exts>` appends each extension to every word, keeping the bare
word too:
```bash
./sif -u https://example.com -w /path/to/words.txt -e php,bak,env
```
### subdomain enumeration
`-dnslist <size>` - enumerate subdomains
@@ -79,7 +132,7 @@ scopes: `common` (top ports), `full` (all ports)
### javascript analysis
`-js` - analyze javascript files
`-js` - analyze javascript files + secret and endpoint extraction
```bash
./sif -u https://example.com -js
@@ -154,6 +207,56 @@ export SHODAN_API_KEY=your-api-key
./sif -u https://example.com -lfi
```
### cors probe
`-cors` - probe for cors misconfigurations (reflected/permissive origins)
```bash
./sif -u https://example.com -cors
```
### open redirect probe
`-redirect` - probe redirect-prone params for open redirects
```bash
./sif -u https://example.com/login?next=home -redirect
```
### reflected xss probe
`-xss` - inject a canary into params and report unescaped reflections
```bash
./sif -u https://example.com/search?q=test -xss
```
### jwt analysis
`-jwt` - fetch the target once, harvest jwts from response headers, cookies and body, then analyze each one entirely offline
flags alg:none, the rs256->hs256 confusion surface, missing/expired exp, plaintext sensitive claims, and cracks a small bundled weak-hmac wordlist. no token is ever sent off-box.
```bash
./sif -u https://example.com -jwt
```
### openapi/swagger exposure
`-openapi` - probe the conventional spec paths (`/swagger.json`, `/openapi.json`, `/v3/api-docs`, ...), parse the first hit (json or yaml) and enumerate every path+method, flagging operations with no security requirement
```bash
./sif -u https://example.com -openapi
```
### favicon fingerprint
`-favicon` - fetch `/favicon.ico` (or the declared `<link rel=icon>`), compute the shodan-style mmh3 hash, match it against a bundled tech map and print the `http.favicon.hash:<n>` pivot query
```bash
./sif -u https://example.com -favicon
```
### framework detection
`-framework` - detect web frameworks with version and cve lookup
@@ -162,6 +265,34 @@ export SHODAN_API_KEY=your-api-key
./sif -u https://example.com -framework
```
### web crawler
`-crawl` - spider the target, following same-host links, scripts and forms
`-crawl-depth` - max recursion depth (default 2). respects robots.txt and stays on the target host.
```bash
./sif -u https://example.com -crawl -crawl-depth 3
```
### passive discovery
`-passive` - gather subdomains from certificate transparency (crt.sh, certspotter) and historical urls from the wayback machine
keyless and zero traffic to the target itself - all lookups hit third-party feeds.
```bash
./sif -u https://example.com -passive
```
### live-host probe
`-probe` - check whether the target is alive and report its final status, page title, server header, content-length and the redirect chain it walked
```bash
./sif -u https://example.com -probe
```
### whois lookup
`-whois` - perform whois lookups
@@ -283,6 +414,106 @@ cap outbound requests per second (0 = unlimited, default 0):
./sif -u https://example.com -rate-limit 20
```
## output options
write the collected findings out to a file after the scan. both formats can be requested in the same run.
### -sarif
write a sarif 2.1.0 report (one run, tool `sif`, one result per finding). ingestable by github code scanning and other sarif consumers:
```bash
./sif -u https://example.com -headers -cors -sarif out.sarif
```
### -md, --markdown
write a readable markdown report grouped by target, then by module:
```bash
./sif -u https://example.com -headers -cors -md report.md
```
### -silent
plain output for pipelines: all banner/spinner/log chrome goes to stderr and stdout carries one normalized finding per line, formatted `[severity] target module title`. implies non-interactive (no spinners), so a downstream consumer sees nothing but findings:
```bash
subfinder -d example.com | sif -silent -probe -sh | notify
```
### -diff
turn a re-scan into a monitor. sif snapshots each target's normalized findings to a json file under the store dir; on the next run it loads that snapshot, diffs the current findings against it by finding key, and prints only the delta (`+ new` for findings that appeared, `- gone` for findings that vanished). it always rewrites the snapshot afterwards, so each run compares against the previous one.
the first run for a target has no snapshot, so every finding shows as `+ new`. when nothing changed, sif notes that and writes a fresh snapshot anyway.
```bash
# baseline, then re-scan and see only what moved
./sif -u https://example.com -sh -cors -diff
./sif -u https://example.com -sh -cors -diff
```
the delta is chrome, not the findings stream: under `-silent` it rides stderr with the rest of the chrome, leaving stdout for the full findings.
### -store
snapshot directory for `-diff`. precedence when unset: the `-log` dir if one is given, else `<user-config>/sif/state` (`$XDG_CONFIG_HOME/sif/state` on linux, `~/Library/Application Support/sif/state` on macos). one sanitized file per target, created at `0750`, written `0600`.
```bash
./sif -u https://example.com -sh -diff -store ./snapshots
```
## notify options
ship findings to a chat/webhook sink after the scan. every provider is a single POST through the shared http client, so the global proxy/rate-limit/header config applies. with nothing configured, `-notify` is a silent no-op.
### -notify
enable delivery to every configured provider:
```bash
export SLACK_WEBHOOK_URL=https://hooks.slack.com/services/...
./sif -u https://example.com -cors -xss -notify
```
### -notify-severity
minimum severity to send: `info`, `low`, `medium`, `high` or `critical` (default `medium`). findings below the floor are dropped, so info-level recon noise doesn't flood a channel. an unrecognized value falls back to `medium`:
```bash
./sif -u https://example.com -cors -notify -notify-severity high
```
### -notify-config
path to a yaml config that overrides the env vars per-field. the keys match [projectdiscovery/notify](https://github.com/projectdiscovery/notify) so an existing config ports over:
```yaml
slack_webhook_url: https://hooks.slack.com/services/...
discord_webhook_url: https://discord.com/api/webhooks/...
telegram_api_key: 123456:abcdef
telegram_chat_id: "987654"
webhook_url: https://example.internal/sif-findings
```
```bash
./sif -u https://example.com -cors -notify -notify-config notify.yaml
```
providers are resolved env-first, then overlaid by the yaml file:
| env var | yaml key | provider |
|---------|----------|----------|
| `SLACK_WEBHOOK_URL` | `slack_webhook_url` | slack incoming webhook |
| `DISCORD_WEBHOOK_URL` | `discord_webhook_url` | discord webhook |
| `TELEGRAM_BOT_TOKEN` | `telegram_api_key` | telegram bot api (needs chat id too) |
| `TELEGRAM_CHAT_ID` | `telegram_chat_id` | telegram destination chat |
| `NOTIFY_WEBHOOK_URL` | `webhook_url` | generic json webhook (structured findings) |
slack/discord/telegram receive a fixed-width finding block; the generic webhook receives structured json (`{count, findings[]}`) for downstream automation.
## api options
### -api
@@ -339,6 +570,9 @@ the first time you run a new release sif also prints that release's notes once.
-git \
-sql \
-lfi \
-cors \
-redirect \
-xss \
-am
```
+3 -2
View File
@@ -7,11 +7,14 @@ require (
github.com/charmbracelet/glamour v0.10.0
github.com/charmbracelet/lipgloss v1.1.1-0.20250404203927-76690c660834
github.com/charmbracelet/log v1.0.0
github.com/gocolly/colly/v2 v2.1.0
github.com/likexian/whois v1.15.7
github.com/projectdiscovery/goflags v0.1.74
github.com/projectdiscovery/nuclei/v3 v3.8.0
github.com/projectdiscovery/retryabledns v1.0.114
github.com/projectdiscovery/utils v0.10.1
github.com/rocketlaunchr/google-search v1.1.6
github.com/twmb/murmur3 v1.1.6
golang.org/x/net v0.53.0
golang.org/x/time v0.14.0
gopkg.in/yaml.v3 v3.0.1
@@ -160,7 +163,6 @@ require (
github.com/gobwas/pool v0.2.1 // indirect
github.com/gobwas/ws v1.4.0 // indirect
github.com/goccy/go-json v0.10.5 // indirect
github.com/gocolly/colly/v2 v2.1.0 // indirect
github.com/gogo/protobuf v1.3.2 // indirect
github.com/golang-jwt/jwt/v4 v4.5.2 // indirect
github.com/golang-jwt/jwt/v5 v5.2.2 // indirect
@@ -288,7 +290,6 @@ require (
github.com/projectdiscovery/ratelimit v0.0.85 // indirect
github.com/projectdiscovery/rawhttp v0.1.90 // indirect
github.com/projectdiscovery/rdap v0.9.1-0.20221108103045-9865884d1917 // indirect
github.com/projectdiscovery/retryabledns v1.0.114 // indirect
github.com/projectdiscovery/retryablehttp-go v1.3.8 // indirect
github.com/projectdiscovery/sarif v0.0.1 // indirect
github.com/projectdiscovery/tlsx v1.2.2 // indirect
+68
View File
@@ -21,7 +21,16 @@ import (
type Settings struct {
Dirlist string
DirMatchCodes string // -mc dirlist: status codes to keep
DirFilterCodes string // -fc dirlist: status codes to drop
DirFilterSizes string // -fs dirlist: body sizes to drop
DirFilterWords string // -fw dirlist: word counts to drop
DirFilterRegex string // -fr dirlist: regex; body match drops response
DirCalibrate bool // -ac dirlist: auto-calibrate soft-404 baseline
DirWordlist string // -w dirlist: custom wordlist (file path or url)
DirExtensions string // -e dirlist: extensions appended to each word
Dnslist string
Resolvers string // -resolvers dnslist: comma list overriding the bundled pool
Debug bool
LogDir string
NoScan bool
@@ -46,7 +55,22 @@ type Settings struct {
SecurityTrails bool
SQL bool
LFI bool
JWT bool
OpenAPI bool
Favicon bool
CORS bool
Redirect bool
XSS bool
Framework bool
Crawl bool
CrawlDepth int
Passive bool
Probe bool
SARIF string // path to write a sarif 2.1.0 report to ("" = off)
Markdown string // path to write a markdown report to ("" = off)
Silent bool // route chrome to stderr, print one finding per line to stdout
Diff bool // surface only findings added/removed vs the last snapshot
Store string // snapshot dir for diff mode ("" = default state dir)
Modules string // Comma-separated list of module IDs to run
ModuleTags string // Run modules matching these tags
AllModules bool // Run all loaded modules
@@ -55,6 +79,9 @@ type Settings struct {
Header goflags.StringSlice // custom request headers ("Key: Value")
Cookie string
RateLimit int
Notify bool // -notify: ship findings to configured providers
NotifySeverity string // -notify-severity: minimum severity to send (info..critical)
NotifyConfig string // -notify-config: path to a notify-compatible yaml file
}
// minThreads is the floor for the worker count. Threads feeds wg.Add across the
@@ -62,6 +89,14 @@ type Settings struct {
// "negative WaitGroup counter"; clamp the parsed value up to this.
const minThreads = 1
// defaultCrawlDepth bounds how far the spider recurses by default; deep enough
// to find linked pages without crawling an entire site.
const defaultCrawlDepth = 2
// defaultNotifySeverity is the floor notify sends at when -notify-severity is
// unset: medium drops pure recon/info noise so alerts stay actionable.
const defaultNotifySeverity = "medium"
const (
Nil goflags.EnumVariable = iota
@@ -90,7 +125,16 @@ func Parse() *Settings {
portScopes := goflags.AllowdTypes{"common": Common, "full": Full, "none": Nil}
flagSet.CreateGroup("scans", "Scans",
flagSet.EnumVar(&settings.Dirlist, "dirlist", Nil, "Directory fuzzing scan size (small/medium/large)", listSizes),
flagSet.StringVar(&settings.DirMatchCodes, "mc", "", "Dirlist: match these status codes (comma list, e.g. 200,301)"),
flagSet.StringVar(&settings.DirFilterCodes, "fc", "", "Dirlist: filter out these status codes (comma list)"),
flagSet.StringVar(&settings.DirFilterSizes, "fs", "", "Dirlist: filter out responses of these body sizes (comma list)"),
flagSet.StringVar(&settings.DirFilterWords, "fw", "", "Dirlist: filter out responses with these word counts (comma list)"),
flagSet.StringVar(&settings.DirFilterRegex, "fr", "", "Dirlist: filter out responses whose body matches this regex"),
flagSet.BoolVar(&settings.DirCalibrate, "ac", false, "Dirlist: auto-calibrate the soft-404 wildcard baseline"),
flagSet.StringVar(&settings.DirWordlist, "w", "", "Dirlist: custom wordlist (local file path or url; overrides -dirlist size)"),
flagSet.StringVar(&settings.DirExtensions, "e", "", "Dirlist: extensions appended to each word (comma list, e.g. php,bak,env)"),
flagSet.EnumVar(&settings.Dnslist, "dnslist", Nil, "DNS fuzzing scan size (small/medium/large)", listSizes),
flagSet.StringVar(&settings.Resolvers, "resolvers", "", "Dnslist: DNS resolvers to use (comma list, e.g. 1.1.1.1,8.8.8.8; overrides the bundled pool)"),
flagSet.EnumVar(&settings.Ports, "ports", Nil, "Port scanning scope (common/full)", portScopes),
flagSet.BoolVar(&settings.Dorking, "dork", false, "Enable Google dorking"),
flagSet.BoolVar(&settings.Git, "git", false, "Enable git repository scanning"),
@@ -107,7 +151,17 @@ func Parse() *Settings {
flagSet.BoolVar(&settings.SecurityTrails, "securitytrails", false, "Enable SecurityTrails domain discovery (requires SECURITYTRAILS_API_KEY env var)"),
flagSet.BoolVar(&settings.SQL, "sql", false, "Enable SQL reconnaissance (admin panels, error disclosure)"),
flagSet.BoolVar(&settings.LFI, "lfi", false, "Enable LFI (Local File Inclusion) reconnaissance"),
flagSet.BoolVar(&settings.JWT, "jwt", false, "Enable JWT discovery + offline weakness analysis"),
flagSet.BoolVar(&settings.OpenAPI, "openapi", false, "Enable OpenAPI/Swagger spec exposure probe"),
flagSet.BoolVar(&settings.Favicon, "favicon", false, "Enable favicon hash fingerprinting (shodan-style)"),
flagSet.BoolVar(&settings.CORS, "cors", false, "Enable CORS misconfiguration probe"),
flagSet.BoolVar(&settings.Redirect, "redirect", false, "Enable open redirect probe"),
flagSet.BoolVar(&settings.XSS, "xss", false, "Enable reflected XSS probe"),
flagSet.BoolVar(&settings.Framework, "framework", false, "Enable framework detection"),
flagSet.BoolVar(&settings.Crawl, "crawl", false, "Enable web crawling (spider same-host links/scripts/forms)"),
flagSet.IntVar(&settings.CrawlDepth, "crawl-depth", defaultCrawlDepth, "Max crawl recursion depth"),
flagSet.BoolVar(&settings.Passive, "passive", false, "Enable passive subdomain/url discovery (zero traffic to target)"),
flagSet.BoolVar(&settings.Probe, "probe", false, "Probe the target for liveness (status, title, server, redirect chain)"),
)
flagSet.CreateGroup("runtime", "Runtime",
@@ -125,6 +179,20 @@ func Parse() *Settings {
flagSet.IntVar(&settings.RateLimit, "rate-limit", 0, "Max requests per second (0 = unlimited)"),
)
flagSet.CreateGroup("output", "Output",
flagSet.StringVar(&settings.SARIF, "sarif", "", "Write a SARIF 2.1.0 report to this file"),
flagSet.StringVarP(&settings.Markdown, "markdown", "md", "", "Write a markdown report to this file"),
flagSet.BoolVar(&settings.Silent, "silent", false, "Plain output: chrome to stderr, one finding per line to stdout (for pipelines)"),
flagSet.BoolVar(&settings.Diff, "diff", false, "Diff mode: surface only findings added/removed since the last snapshot of each target"),
flagSet.StringVar(&settings.Store, "store", "", "Snapshot directory for -diff (default: log dir, else <user-config>/sif/state)"),
)
flagSet.CreateGroup("notify", "Notify",
flagSet.BoolVar(&settings.Notify, "notify", false, "Ship findings to configured providers (slack/discord/telegram/webhook)"),
flagSet.StringVar(&settings.NotifySeverity, "notify-severity", defaultNotifySeverity, "Minimum severity to notify on (info/low/medium/high/critical)"),
flagSet.StringVar(&settings.NotifyConfig, "notify-config", "", "Path to a notify-compatible yaml config (overrides env vars)"),
)
flagSet.CreateGroup("api", "API",
flagSet.BoolVar(&settings.ApiMode, "api", false, "Enable API mode. Only useful for internal lunchcat usage"),
)
+8
View File
@@ -61,6 +61,14 @@ func TestSettingsDefaults(t *testing.T) {
if settings.Ports != "" {
t.Errorf("expected Ports default to be empty, got %v", settings.Ports)
}
// diff mode is opt-in and its store dir defaults empty (resolved at runtime).
if settings.Diff != false {
t.Errorf("expected Diff default to be false, got %v", settings.Diff)
}
if settings.Store != "" {
t.Errorf("expected Store default to be empty, got %v", settings.Store)
}
}
func TestSettingsNoScanBehavior(t *testing.T) {
+270
View File
@@ -0,0 +1,270 @@
/*
·━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━·
: :
: █▀ █ █▀▀ · Blazing-fast pentesting suite :
: ▄█ █ █▀ · BSD 3-Clause License :
: :
: (c) 2022-2026 vmfunc, xyzeva, :
: lunchcat alumni & contributors :
: :
·━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━·
*/
// Package dnsx resolves subdomain candidates against a bundled resolver pool
// before anything is probed over http, so the slow/inaccurate path of HTTP-ing
// every wordlist entry through the OS resolver is gone. it also fingerprints
// wildcard zones (a zone that answers every random label) so a catch-all
// nameserver can't flood the caller with phantom subdomains.
package dnsx
import (
"crypto/rand"
"fmt"
"math/big"
"sort"
"strings"
retryabledns "github.com/projectdiscovery/retryabledns"
)
// bundled default resolver pool. anycast cloudflare/google/quad9 - fast, public,
// and unlikely to rate-limit a recon sweep. -resolvers overrides this set.
const (
resolverCloudflare = "1.1.1.1:53"
resolverGoogle = "8.8.8.8:53"
resolverQuad9 = "9.9.9.9:53"
)
// defaultResolvers is the bundled pool used when the caller passes none.
var defaultResolvers = []string{resolverCloudflare, resolverGoogle, resolverQuad9}
const (
// defaultRetries is how many times retryabledns rotates through the pool on a
// timeout before giving up on a name. low enough to stay fast on a big list.
defaultRetries = 3
// wildcardProbes is how many random nonexistent labels we resolve to
// fingerprint a wildcard zone. more samples make a rotating catch-all (one
// that hands back a different ip per query) harder to miss, but each is a
// real lookup so this stays small.
wildcardProbes = 3
// randomLabelLen is the length of each random wildcard-probe label. long
// enough that a collision with a real host is astronomically unlikely.
randomLabelLen = 16
)
// randomLabelAlphabet is the lowercase-alnum set wildcard probe labels draw
// from; a valid dns label so the query isn't rejected before it leaves.
const randomLabelAlphabet = "abcdefghijklmnopqrstuvwxyz0123456789"
// defaultDNSPort is appended to any resolver entry given without an explicit
// port, so "1.1.1.1" and "1.1.1.1:53" both work on the cli.
const defaultDNSPort = "53"
// ParseResolvers splits a comma list of resolvers into a normalized slice,
// appending the default port to bare ips/hosts. an empty or blank input returns
// nil so the caller falls back to the bundled pool.
func ParseResolvers(raw string) []string {
if strings.TrimSpace(raw) == "" {
return nil
}
parts := strings.Split(raw, ",")
out := make([]string, 0, len(parts))
for i := 0; i < len(parts); i++ {
entry := strings.TrimSpace(parts[i])
if entry == "" {
continue
}
// a bare ip/host gets the default port; an entry already carrying ":port"
// (or a bracketed ipv6 literal) is left as-is.
if !strings.Contains(entry, ":") {
entry += ":" + defaultDNSPort
}
out = append(out, entry)
}
return out
}
// resolution is the resolved address set for one host. empty Addrs means the
// name did not resolve (nxdomain / no records).
type resolution struct {
Addrs []string
}
// resolved reports whether the name returned any address.
func (r resolution) resolved() bool {
return len(r.Addrs) > 0
}
// resolverFn is the test seam: every lookup the package makes goes through this
// var, so a fake can answer without touching the network. real runs point it at
// a retryabledns-backed client via NewResolver.
var resolverFn func(host string) (resolution, error)
// Resolver resolves candidates against a pool and filters wildcard answers. it
// is built once per scan and shared across the worker goroutines; the
// underlying retryabledns client is safe for concurrent use.
type Resolver struct {
// wildcardSigs holds the address sets a wildcard zone answers random labels
// with. nil/empty means the zone is not wildcard. a candidate whose answer is
// covered by one of these is a catch-all hit, not a real host.
wildcardSigs []map[string]struct{}
}
// NewResolver wires resolverFn to a retryabledns client over the given pool
// (bundled default when resolvers is empty) and returns a Resolver. it does not
// fingerprint anything yet - call FingerprintWildcard with the apex first.
func NewResolver(resolvers []string) (*Resolver, error) {
pool := resolvers
if len(pool) == 0 {
pool = defaultResolvers
}
client, err := retryabledns.New(pool, defaultRetries)
if err != nil {
return nil, fmt.Errorf("dnsx: build resolver over %v: %w", pool, err)
}
// only install the real client when a test hasn't already injected a fake;
// the seam wins so hermetic tests never reach this client.
if resolverFn == nil {
resolverFn = func(host string) (resolution, error) {
data, err := client.Resolve(host)
if err != nil {
return resolution{}, fmt.Errorf("dnsx: resolve %q: %w", host, err)
}
return resolution{Addrs: mergeAddrs(data)}, nil
}
}
return &Resolver{}, nil
}
// FingerprintWildcard resolves wildcardProbes random labels under apex. any that
// answer mean the zone is a catch-all, so their address sets are recorded as
// signatures to filter real candidates against later. a clean zone leaves the
// signature list empty and nothing gets filtered.
func (r *Resolver) FingerprintWildcard(apex string) error {
apex = strings.TrimSuffix(apex, ".")
for i := 0; i < wildcardProbes; i++ {
label, err := randomLabel(randomLabelLen)
if err != nil {
return fmt.Errorf("dnsx: wildcard probe label: %w", err)
}
res, err := resolverFn(label + "." + apex)
if err != nil {
// a probe failure (timeout / nxdomain surfaced as error) just means this
// sample says "not wildcard"; don't abort the whole fingerprint on it.
continue
}
if res.resolved() {
r.wildcardSigs = append(r.wildcardSigs, toSet(res.Addrs))
}
}
return nil
}
// Resolve looks up host and reports whether it is a real, non-wildcard hit. a
// name that doesn't resolve, or whose answer matches a recorded wildcard
// signature, returns false so the caller skips probing it.
func (r *Resolver) Resolve(host string) (bool, error) {
res, err := resolverFn(host)
if err != nil {
return false, fmt.Errorf("dnsx: resolve %q: %w", host, err)
}
if !res.resolved() {
return false, nil
}
if r.isWildcard(res.Addrs) {
return false, nil
}
return true, nil
}
// isWildcard reports whether addrs is covered by any recorded wildcard
// signature. a candidate whose every address appears in a wildcard answer is a
// catch-all hit; a host with even one address outside the signature is a real,
// distinct record and survives.
func (r *Resolver) isWildcard(addrs []string) bool {
if len(r.wildcardSigs) == 0 {
return false
}
for i := 0; i < len(r.wildcardSigs); i++ {
if subset(addrs, r.wildcardSigs[i]) {
return true
}
}
return false
}
// mergeAddrs flattens the A and AAAA answers into one sorted, deduped slice so
// two equal answers compare equal regardless of record ordering.
func mergeAddrs(data *retryabledns.DNSData) []string {
if data == nil {
return nil
}
seen := make(map[string]struct{}, len(data.A)+len(data.AAAA))
for i := 0; i < len(data.A); i++ {
seen[data.A[i]] = struct{}{}
}
for i := 0; i < len(data.AAAA); i++ {
seen[data.AAAA[i]] = struct{}{}
}
addrs := make([]string, 0, len(seen))
for addr := range seen {
addrs = append(addrs, addr)
}
sort.Strings(addrs)
return addrs
}
// toSet turns addrs into a lookup set for subset checks.
func toSet(addrs []string) map[string]struct{} {
set := make(map[string]struct{}, len(addrs))
for i := 0; i < len(addrs); i++ {
set[addrs[i]] = struct{}{}
}
return set
}
// subset reports whether every addr is present in sig (and addrs is non-empty);
// an empty addrs can't be a wildcard match.
func subset(addrs []string, sig map[string]struct{}) bool {
if len(addrs) == 0 {
return false
}
for i := 0; i < len(addrs); i++ {
if _, ok := sig[addrs[i]]; !ok {
return false
}
}
return true
}
// randomLabel returns a cryptographically-random lowercase-alnum dns label of
// length n. crypto/rand (not math/rand) so a target can't predict the probe
// labels and special-case them to defeat wildcard detection.
func randomLabel(n int) (string, error) {
var b strings.Builder
b.Grow(n)
alphabetLen := big.NewInt(int64(len(randomLabelAlphabet)))
for i := 0; i < n; i++ {
idx, err := rand.Int(rand.Reader, alphabetLen)
if err != nil {
return "", fmt.Errorf("dnsx: random index: %w", err)
}
b.WriteByte(randomLabelAlphabet[idx.Int64()])
}
return b.String(), nil
}
+176
View File
@@ -0,0 +1,176 @@
/*
·━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━·
: :
: █▀ █ █▀▀ · Blazing-fast pentesting suite :
: ▄█ █ █▀ · BSD 3-Clause License :
: :
: (c) 2022-2026 vmfunc, xyzeva, :
: lunchcat alumni & contributors :
: :
·━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━·
*/
package dnsx
import (
"reflect"
"strings"
"testing"
)
// withFakeResolver swaps resolverFn for fn for the duration of one test, then
// restores it - the seam that keeps every case below network-free.
func withFakeResolver(t *testing.T, fn func(host string) (resolution, error)) {
t.Helper()
orig := resolverFn
resolverFn = fn
t.Cleanup(func() { resolverFn = orig })
}
// newFingerprinted builds a Resolver and runs the wildcard fingerprint against
// apex using the already-injected fake; fatal on error.
func newFingerprinted(t *testing.T, apex string) *Resolver {
t.Helper()
r := &Resolver{}
if err := r.FingerprintWildcard(apex); err != nil {
t.Fatalf("FingerprintWildcard: %v", err)
}
return r
}
const testApex = "example.com"
// a host that resolves to a real address, in a clean (non-wildcard) zone, is a
// genuine hit.
func TestResolve_FoundInCleanZone(t *testing.T) {
withFakeResolver(t, func(host string) (resolution, error) {
// nothing answers a random wildcard probe -> clean zone.
if strings.HasSuffix(host, "."+testApex) && host != "www."+testApex {
return resolution{}, nil
}
if host == "www."+testApex {
return resolution{Addrs: []string{"93.184.216.34"}}, nil
}
return resolution{}, nil
})
r := newFingerprinted(t, testApex)
if len(r.wildcardSigs) != 0 {
t.Fatalf("clean zone should record no wildcard signatures, got %d", len(r.wildcardSigs))
}
ok, err := r.Resolve("www." + testApex)
if err != nil {
t.Fatalf("Resolve: %v", err)
}
if !ok {
t.Error("a resolving host in a clean zone should be a hit")
}
}
// nxdomain (no addresses) is not a hit, so the caller skips probing it.
func TestResolve_NxdomainSkipped(t *testing.T) {
withFakeResolver(t, func(string) (resolution, error) {
// every name, probes included, returns no records.
return resolution{}, nil
})
r := newFingerprinted(t, testApex)
ok, err := r.Resolve("ghost." + testApex)
if err != nil {
t.Fatalf("Resolve: %v", err)
}
if ok {
t.Error("an nxdomain host must not count as found")
}
}
// a wildcard zone answers the random probe labels, so a candidate that resolves
// to the same catch-all address is filtered out.
func TestResolve_WildcardFiltered(t *testing.T) {
const catchAll = "10.0.0.1"
withFakeResolver(t, func(string) (resolution, error) {
// the zone answers everything - probes and candidates alike - with one ip.
return resolution{Addrs: []string{catchAll}}, nil
})
r := newFingerprinted(t, testApex)
if len(r.wildcardSigs) == 0 {
t.Fatal("wildcard zone should record at least one signature")
}
ok, err := r.Resolve("anything." + testApex)
if err != nil {
t.Fatalf("Resolve: %v", err)
}
if ok {
t.Error("a candidate matching the wildcard answer must be filtered")
}
}
// a real host in a wildcard zone that resolves to a distinct address (not the
// catch-all) still survives the filter - one address outside the signature is
// enough to be a genuine record.
func TestResolve_DistinctHostSurvivesWildcard(t *testing.T) {
const catchAll = "10.0.0.1"
const realHost = "api." + testApex
withFakeResolver(t, func(host string) (resolution, error) {
if host == realHost {
return resolution{Addrs: []string{"203.0.113.7"}}, nil
}
// everything else (probes + other candidates) hits the catch-all.
return resolution{Addrs: []string{catchAll}}, nil
})
r := newFingerprinted(t, testApex)
if len(r.wildcardSigs) == 0 {
t.Fatal("wildcard zone should record at least one signature")
}
ok, err := r.Resolve(realHost)
if err != nil {
t.Fatalf("Resolve: %v", err)
}
if !ok {
t.Error("a host resolving to a distinct address should survive the wildcard filter")
}
}
func TestParseResolvers(t *testing.T) {
tests := []struct {
name string
in string
want []string
}{
{"empty falls back to bundled", "", nil},
{"blank falls back to bundled", " ", nil},
{"bare ips get default port", "1.1.1.1,8.8.8.8", []string{"1.1.1.1:53", "8.8.8.8:53"}},
{"explicit port preserved", "9.9.9.9:5353", []string{"9.9.9.9:5353"}},
{"whitespace and empties trimmed", " 1.1.1.1 , ,8.8.8.8 ", []string{"1.1.1.1:53", "8.8.8.8:53"}},
{"mixed bare and ported", "1.1.1.1,9.9.9.9:5353", []string{"1.1.1.1:53", "9.9.9.9:5353"}},
}
for _, tt := range tests {
t.Run(tt.name, func(t *testing.T) {
if got := ParseResolvers(tt.in); !reflect.DeepEqual(got, tt.want) {
t.Errorf("ParseResolvers(%q) = %v, want %v", tt.in, got, tt.want)
}
})
}
}
func TestNewResolver_DefaultsToBundledPool(t *testing.T) {
// keep the seam already installed so New doesn't replace it with a real
// client; we only assert the constructor accepts an empty override.
withFakeResolver(t, func(string) (resolution, error) { return resolution{}, nil })
r, err := NewResolver(nil)
if err != nil {
t.Fatalf("NewResolver(nil): %v", err)
}
if r == nil {
t.Fatal("NewResolver returned nil resolver")
}
}
+730
View File
@@ -0,0 +1,730 @@
/*
·━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━·
: :
: █▀ █ █▀▀ · Blazing-fast pentesting suite :
: ▄█ █ █▀ · BSD 3-Clause License :
: :
: (c) 2022-2026 vmfunc, xyzeva, :
: lunchcat alumni & contributors :
: :
·━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━·
*/
// Package finding is the one normalization layer between the scan results and
// the consumers that don't want to know about ~two dozen result structs: notify
// (later) gates and renders on it, diff (later) keys runs off it. Flatten is the
// single type-switch; adding a scanner without teaching Flatten about it trips
// the guard test in flatten_test.go, on purpose.
package finding
import (
"fmt"
"strings"
"github.com/dropalldatabases/sif/internal/modules"
"github.com/dropalldatabases/sif/internal/scan"
"github.com/dropalldatabases/sif/internal/scan/frameworks"
"github.com/dropalldatabases/sif/internal/scan/js"
"github.com/projectdiscovery/nuclei/v3/pkg/output"
)
// Finding is the normalized shape every scanner result collapses to. one
// Finding is one underlying item (a single header, one cors hit, one nuclei
// match) rather than a whole module's blob, so consumers diff and notify at
// item granularity.
type Finding struct {
Target string // the url/host the scan ran against
Module string // the ResultType() of the source scanner
Severity Severity // ranked severity, SeverityUnknown when the source has none
Key string // stable identity for dedup/diff: module + ":" + identifier
Title string // short human label
Raw string // short evidence string, not the full body
}
// Line renders a finding as one stable, terse, machine-friendly line for the
// -silent plain sink: "[severity] target module title". no styling, no color -
// a downstream pipe (notify, grep, awk) keys off the bracketed severity and the
// fixed field order, so the shape stays frozen. pointer receiver: Finding is
// wide enough that copying it per line is wasteful.
func (f *Finding) Line() string {
return fmt.Sprintf("[%s] %s %s %s", f.Severity, f.Target, f.Module, f.Title)
}
// static per-module severities for results that carry no severity field of
// their own. these are the editorial baseline; a scanner that emits its own
// severity (cors, xss, nuclei, ...) overrides this on a per-item basis.
const (
// a live admin panel / takeover / public bucket is high on its own.
sevTakeover = SeverityHigh
sevPublicS3 = SeverityHigh
sevAdminPanel = SeverityHigh
// disclosure-grade signals: dberrors, secrets, supabase keys.
sevDBError = SeverityMedium
sevSecret = SeverityMedium
// pure recon/inventory: headers, crawl urls, passive hosts, ports.
sevRecon = SeverityInfo
)
// keySep joins the module id and the per-item identifier into a Key. kept as a
// const so the diff layer can split on it without re-deriving the separator.
const keySep = ":"
// key builds a stable per-item identity: module:identifier. identifier is
// whatever uniquely names the item within its module (a url, a header name, a
// subdomain) so the same finding across two runs produces the same Key.
func key(module, identifier string) string {
return module + keySep + identifier
}
// Flatten normalizes one module's result into zero or more Findings. result is
// the raw data carried in a ModuleResult; the type switch covers every scan
// result struct. an unrecognized type yields a single SeverityUnknown finding
// keyed "module:unhandled" so a new scanner surfaces loudly instead of
// vanishing - the guard test asserts this never happens for a known type.
func Flatten(target, module string, result any) []Finding {
switch r := result.(type) {
case *scan.ShodanResult:
return flattenShodan(target, r)
case *scan.SQLResult:
return flattenSQL(target, r)
case *scan.LFIResult:
return flattenLFI(target, r)
case *scan.JWTResult:
return flattenJWT(target, r)
case *scan.OpenAPIResult:
return flattenOpenAPI(target, r)
case *scan.FaviconResult:
return flattenFavicon(target, r)
case *scan.CMSResult:
return flattenCMS(target, r)
case *scan.SecurityTrailsResult:
return flattenSecurityTrails(target, r)
case *scan.CORSResult:
return flattenCORS(target, r)
case *scan.RedirectResult:
return flattenRedirect(target, r)
case *scan.XSSResult:
return flattenXSS(target, r)
case *scan.CrawlResult:
return flattenCrawl(target, r)
case *scan.PassiveResult:
return flattenPassive(target, r)
case *scan.ProbeResult:
return flattenProbe(target, r)
case scan.HeaderResults:
return flattenHeaders(target, r)
case []scan.HeaderResult:
// the headers module appends a literal []HeaderResult, not the named
// slice type; both reach here so cover both.
return flattenHeaders(target, r)
case scan.SecurityHeaderResults:
return flattenSecurityHeaders(target, r)
case []scan.SecurityHeaderResult:
return flattenSecurityHeaders(target, r)
case scan.DirectoryResults:
return flattenDirlist(target, r)
case []scan.DirectoryResult:
return flattenDirlist(target, r)
case scan.CloudStorageResults:
return flattenCloudStorage(target, r)
case []scan.CloudStorageResult:
return flattenCloudStorage(target, r)
case scan.DorkResults:
return flattenDork(target, r)
case []scan.DorkResult:
return flattenDork(target, r)
case scan.SubdomainTakeoverResults:
return flattenTakeover(target, r)
case []scan.SubdomainTakeoverResult:
return flattenTakeover(target, r)
case *frameworks.FrameworkResult:
return flattenFramework(target, r)
case *js.JavascriptScanResult:
return flattenJS(target, r)
case *modules.Result:
// yaml/builtin modules carry their own module id; honor it over the
// passed-in module so per-module findings stay attributed correctly.
return flattenModule(target, r)
case []output.ResultEvent:
return flattenNuclei(target, r)
case []string:
// dnslist/portscan/git all hand back a bare []string of discovered
// items; module disambiguates which inventory it is.
return flattenStrings(target, module, r)
default:
// unknown type: emit a loud placeholder rather than dropping it.
return []Finding{{
Target: target,
Module: module,
Severity: SeverityUnknown,
Key: key(module, "unhandled"),
Title: fmt.Sprintf("unhandled result type %T", result),
Raw: fmt.Sprintf("%T", result),
}}
}
}
func flattenShodan(target string, r *scan.ShodanResult) []Finding {
if r == nil {
return nil
}
// one host snapshot -> one inventory finding; vulns are the interesting bit
// so they bump severity and ride along in the evidence string.
sev := sevRecon
if len(r.Vulns) > 0 {
sev = SeverityHigh
}
raw := fmt.Sprintf("%d ports", len(r.Ports))
if len(r.Vulns) > 0 {
raw = fmt.Sprintf("%s, %d vulns", raw, len(r.Vulns))
}
return []Finding{{
Target: target,
Module: "shodan",
Severity: sev,
Key: key("shodan", r.IP),
Title: "shodan host " + r.IP,
Raw: raw,
}}
}
func flattenSQL(target string, r *scan.SQLResult) []Finding {
if r == nil {
return nil
}
out := make([]Finding, 0, len(r.AdminPanels)+len(r.DatabaseErrors)+len(r.ExposedPorts))
for i := 0; i < len(r.AdminPanels); i++ {
p := r.AdminPanels[i]
out = append(out, Finding{
Target: target,
Module: "sql",
Severity: sevAdminPanel,
Key: key("sql", "admin:"+p.URL),
Title: p.Type + " admin panel",
Raw: fmt.Sprintf("%s (%d)", p.URL, p.Status),
})
}
for i := 0; i < len(r.DatabaseErrors); i++ {
e := r.DatabaseErrors[i]
out = append(out, Finding{
Target: target,
Module: "sql",
Severity: sevDBError,
Key: key("sql", "dberr:"+e.URL+":"+e.DatabaseType),
Title: e.DatabaseType + " error disclosure",
Raw: e.ErrorPattern,
})
}
for i := 0; i < len(r.ExposedPorts); i++ {
p := r.ExposedPorts[i]
out = append(out, Finding{
Target: target,
Module: "sql",
Severity: SeverityMedium,
Key: key("sql", fmt.Sprintf("port:%d", p)),
Title: fmt.Sprintf("exposed db port %d", p),
Raw: fmt.Sprintf("%d", p),
})
}
return out
}
func flattenLFI(target string, r *scan.LFIResult) []Finding {
if r == nil {
return nil
}
out := make([]Finding, 0, len(r.Vulnerabilities))
for i := 0; i < len(r.Vulnerabilities); i++ {
v := r.Vulnerabilities[i]
out = append(out, Finding{
Target: target,
Module: "lfi",
Severity: ParseSeverity(v.Severity),
Key: key("lfi", v.URL+":"+v.Parameter),
Title: "lfi via " + v.Parameter,
Raw: v.Evidence,
})
}
return out
}
func flattenJWT(target string, r *scan.JWTResult) []Finding {
if r == nil {
return nil
}
out := make([]Finding, 0, len(r.Tokens))
for i := 0; i < len(r.Tokens); i++ {
t := r.Tokens[i]
// one finding per weakness, not per token: a token with alg:none and a
// weak key is two distinct issues a consumer wants to diff separately.
for j := 0; j < len(t.Issues); j++ {
iss := t.Issues[j]
out = append(out, Finding{
Target: target,
Module: "jwt",
Severity: ParseSeverity(iss.Severity),
Key: key("jwt", t.Source+":"+iss.Kind),
Title: "jwt " + iss.Kind,
Raw: iss.Detail,
})
}
}
return out
}
func flattenOpenAPI(target string, r *scan.OpenAPIResult) []Finding {
if r == nil {
return nil
}
return []Finding{{
Target: target,
Module: "openapi",
Severity: ParseSeverity(r.Severity),
Key: key("openapi", r.SpecURL),
Title: "openapi spec exposed",
Raw: fmt.Sprintf("%s (%d endpoints)", r.SpecURL, len(r.Endpoints)),
}}
}
func flattenFavicon(target string, r *scan.FaviconResult) []Finding {
if r == nil {
return nil
}
// a matched fingerprint is a real signal; an unmatched hash is just inventory
// (still useful as a shodan pivot, so we keep it at recon).
sev := sevRecon
title := fmt.Sprintf("favicon hash %d", r.Hash)
if r.Tech != "" {
sev = SeverityLow
title = r.Tech + " (favicon)"
}
return []Finding{{
Target: target,
Module: "favicon",
Severity: sev,
Key: key("favicon", fmt.Sprintf("%d", r.Hash)),
Title: title,
Raw: r.ShodanQ,
}}
}
func flattenCMS(target string, r *scan.CMSResult) []Finding {
if r == nil || r.Name == "" {
return nil
}
return []Finding{{
Target: target,
Module: "cms",
Severity: sevRecon,
Key: key("cms", r.Name),
Title: r.Name + " detected",
Raw: strings.TrimSpace(r.Name + " " + r.Version),
}}
}
func flattenSecurityTrails(target string, r *scan.SecurityTrailsResult) []Finding {
if r == nil {
return nil
}
out := make([]Finding, 0, len(r.Subdomains)+len(r.AssociatedDomains))
for i := 0; i < len(r.Subdomains); i++ {
d := r.Subdomains[i]
out = append(out, Finding{
Target: target,
Module: "securitytrails",
Severity: sevRecon,
Key: key("securitytrails", "sub:"+d),
Title: "subdomain " + d,
Raw: d,
})
}
for i := 0; i < len(r.AssociatedDomains); i++ {
d := r.AssociatedDomains[i]
out = append(out, Finding{
Target: target,
Module: "securitytrails",
Severity: sevRecon,
Key: key("securitytrails", "assoc:"+d),
Title: "associated domain " + d,
Raw: d,
})
}
return out
}
func flattenCORS(target string, r *scan.CORSResult) []Finding {
if r == nil {
return nil
}
out := make([]Finding, 0, len(r.Findings))
for i := 0; i < len(r.Findings); i++ {
f := r.Findings[i]
out = append(out, Finding{
Target: target,
Module: "cors",
Severity: ParseSeverity(f.Severity),
Key: key("cors", f.URL+":"+f.OriginTested),
Title: f.Note,
Raw: "allow-origin: " + f.AllowOrigin,
})
}
return out
}
func flattenRedirect(target string, r *scan.RedirectResult) []Finding {
if r == nil {
return nil
}
out := make([]Finding, 0, len(r.Findings))
for i := 0; i < len(r.Findings); i++ {
f := r.Findings[i]
out = append(out, Finding{
Target: target,
Module: "redirect",
Severity: ParseSeverity(f.Severity),
Key: key("redirect", f.URL+":"+f.Parameter+":"+f.Via),
Title: "open redirect via " + f.Parameter,
Raw: f.Location,
})
}
return out
}
func flattenXSS(target string, r *scan.XSSResult) []Finding {
if r == nil {
return nil
}
out := make([]Finding, 0, len(r.Findings))
for i := 0; i < len(r.Findings); i++ {
f := r.Findings[i]
out = append(out, Finding{
Target: target,
Module: "xss",
Severity: ParseSeverity(f.Severity),
Key: key("xss", f.URL+":"+f.Parameter+":"+f.Context),
Title: "reflected xss in " + f.Parameter,
Raw: strings.Join(f.SurvivedRaw, " "),
})
}
return out
}
func flattenCrawl(target string, r *scan.CrawlResult) []Finding {
if r == nil {
return nil
}
out := make([]Finding, 0, len(r.URLs))
for i := 0; i < len(r.URLs); i++ {
u := r.URLs[i]
out = append(out, Finding{
Target: target,
Module: "crawl",
Severity: sevRecon,
Key: key("crawl", u),
Title: "crawled url",
Raw: u,
})
}
return out
}
func flattenPassive(target string, r *scan.PassiveResult) []Finding {
if r == nil {
return nil
}
out := make([]Finding, 0, len(r.Subdomains)+len(r.URLs))
for i := 0; i < len(r.Subdomains); i++ {
s := r.Subdomains[i]
out = append(out, Finding{
Target: target,
Module: "passive",
Severity: sevRecon,
Key: key("passive", "sub:"+s),
Title: "passive subdomain " + s,
Raw: s,
})
}
for i := 0; i < len(r.URLs); i++ {
u := r.URLs[i]
out = append(out, Finding{
Target: target,
Module: "passive",
Severity: sevRecon,
Key: key("passive", "url:"+u),
Title: "passive url",
Raw: u,
})
}
return out
}
func flattenProbe(target string, r *scan.ProbeResult) []Finding {
if r == nil || !r.Alive {
// a dead probe isn't a finding, just an absent host.
return nil
}
return []Finding{{
Target: target,
Module: "probe",
Severity: sevRecon,
Key: key("probe", r.URL),
Title: fmt.Sprintf("alive %d", r.StatusCode),
Raw: strings.TrimSpace(fmt.Sprintf("%d %s", r.StatusCode, r.Title)),
}}
}
func flattenHeaders(target string, rs []scan.HeaderResult) []Finding {
out := make([]Finding, 0, len(rs))
for i := 0; i < len(rs); i++ {
h := rs[i]
out = append(out, Finding{
Target: target,
Module: "headers",
Severity: sevRecon,
Key: key("headers", h.Name),
Title: h.Name,
Raw: h.Value,
})
}
return out
}
func flattenSecurityHeaders(target string, rs []scan.SecurityHeaderResult) []Finding {
out := make([]Finding, 0, len(rs))
for i := 0; i < len(rs); i++ {
h := rs[i]
out = append(out, Finding{
Target: target,
Module: "security_headers",
Severity: ParseSeverity(h.Severity),
Key: key("security_headers", h.Header),
Title: h.Header,
Raw: h.Note,
})
}
return out
}
// dirInteresting bounds the "noteworthy" 3xx range for a listed directory; a
// redirect (>=300) or anything past it is worth more than a plain 200 hit.
const dirRedirectFloor = 300
func flattenDirlist(target string, rs []scan.DirectoryResult) []Finding {
out := make([]Finding, 0, len(rs))
for i := 0; i < len(rs); i++ {
d := rs[i]
sev := sevRecon
if d.StatusCode >= dirRedirectFloor {
sev = SeverityLow
}
out = append(out, Finding{
Target: target,
Module: "dirlist",
Severity: sev,
Key: key("dirlist", d.Url),
Title: fmt.Sprintf("%s [%d]", d.Url, d.StatusCode),
Raw: fmt.Sprintf("status=%d size=%d", d.StatusCode, d.Size),
})
}
return out
}
func flattenCloudStorage(target string, rs []scan.CloudStorageResult) []Finding {
out := make([]Finding, 0, len(rs))
for i := 0; i < len(rs); i++ {
b := rs[i]
sev := sevRecon
if b.IsPublic {
sev = sevPublicS3
}
title := "bucket " + b.BucketName
if b.IsPublic {
title = "public bucket " + b.BucketName
}
out = append(out, Finding{
Target: target,
Module: "cloudstorage",
Severity: sev,
Key: key("cloudstorage", b.BucketName),
Title: title,
Raw: fmt.Sprintf("public=%t", b.IsPublic),
})
}
return out
}
func flattenDork(target string, rs []scan.DorkResult) []Finding {
out := make([]Finding, 0, len(rs))
for i := 0; i < len(rs); i++ {
d := rs[i]
out = append(out, Finding{
Target: target,
Module: "dork",
Severity: sevRecon,
Key: key("dork", d.Url),
Title: "dork hit",
Raw: d.Url,
})
}
return out
}
func flattenTakeover(target string, rs []scan.SubdomainTakeoverResult) []Finding {
out := make([]Finding, 0, len(rs))
for i := 0; i < len(rs); i++ {
t := rs[i]
// only the vulnerable ones are findings; a safe cname is noise here.
if !t.Vulnerable {
continue
}
out = append(out, Finding{
Target: target,
Module: "subdomain_takeover",
Severity: sevTakeover,
Key: key("subdomain_takeover", t.Subdomain),
Title: "takeover: " + t.Subdomain,
Raw: t.Service,
})
}
return out
}
func flattenFramework(target string, r *frameworks.FrameworkResult) []Finding {
if r == nil || r.Name == "" {
return nil
}
// framework risk maps onto severity; an unset risk falls back to recon.
sev := ParseSeverity(r.RiskLevel)
if sev == SeverityUnknown {
sev = sevRecon
}
raw := strings.TrimSpace(r.Name + " " + r.Version)
if len(r.CVEs) > 0 {
raw = fmt.Sprintf("%s, %d cves", raw, len(r.CVEs))
}
return []Finding{{
Target: target,
Module: "framework",
Severity: sev,
Key: key("framework", r.Name),
Title: r.Name + " detected",
Raw: raw,
}}
}
func flattenJS(target string, r *js.JavascriptScanResult) []Finding {
if r == nil {
return nil
}
supabase := r.SupabaseFindings()
out := make([]Finding, 0, len(r.SecretMatches)+len(supabase)+len(r.Endpoints)+len(r.FoundEnvironmentVars))
for i := 0; i < len(r.SecretMatches); i++ {
s := r.SecretMatches[i]
out = append(out, Finding{
Target: target,
Module: "js",
Severity: sevSecret,
Key: key("js", "secret:"+s.Rule+":"+s.Source),
Title: "secret: " + s.Rule,
Raw: s.Source,
})
}
for i := 0; i < len(supabase); i++ {
s := supabase[i]
// a non-anon role on an exposed key is the real bug; anon is just recon.
sev := sevRecon
if s.Role != "" && s.Role != "anon" {
sev = SeverityHigh
}
out = append(out, Finding{
Target: target,
Module: "js",
Severity: sev,
Key: key("js", "supabase:"+s.ProjectId),
Title: "supabase project " + s.ProjectId,
Raw: fmt.Sprintf("role=%s collections=%d", s.Role, s.Collections),
})
}
for i := 0; i < len(r.Endpoints); i++ {
e := r.Endpoints[i]
out = append(out, Finding{
Target: target,
Module: "js",
Severity: sevRecon,
Key: key("js", "endpoint:"+e),
Title: "js endpoint",
Raw: e,
})
}
// env vars are a map; sort-free since the Key carries the name, and diff
// keys on the Key not on iteration order.
for name, value := range r.FoundEnvironmentVars {
out = append(out, Finding{
Target: target,
Module: "js",
Severity: sevSecret,
Key: key("js", "env:"+name),
Title: "env var " + name,
Raw: value,
})
}
return out
}
func flattenModule(target string, r *modules.Result) []Finding {
if r == nil {
return nil
}
module := r.ResultType()
out := make([]Finding, 0, len(r.Findings))
for i := 0; i < len(r.Findings); i++ {
f := r.Findings[i]
out = append(out, Finding{
Target: target,
Module: module,
Severity: ParseSeverity(f.Severity),
Key: key(module, f.URL),
Title: module + " finding",
Raw: f.Evidence,
})
}
return out
}
func flattenNuclei(target string, events []output.ResultEvent) []Finding {
out := make([]Finding, 0, len(events))
for i := 0; i < len(events); i++ {
e := events[i]
// host is the most reliable per-hit identifier; matched-at sharpens it
// when several templates fire on one host.
ident := e.TemplateID + ":" + e.Host
if e.Matched != "" {
ident = e.TemplateID + ":" + e.Matched
}
out = append(out, Finding{
Target: target,
Module: "nuclei",
Severity: ParseSeverity(e.Info.SeverityHolder.Severity.String()),
Key: key("nuclei", ident),
Title: e.Info.Name,
Raw: e.Matched,
})
}
return out
}
func flattenStrings(target, module string, items []string) []Finding {
out := make([]Finding, 0, len(items))
for i := 0; i < len(items); i++ {
v := items[i]
out = append(out, Finding{
Target: target,
Module: module,
Severity: sevRecon,
Key: key(module, v),
Title: module + " item",
Raw: v,
})
}
return out
}
+383
View File
@@ -0,0 +1,383 @@
/*
·━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━·
: :
: █▀ █ █▀▀ · Blazing-fast pentesting suite :
: ▄█ █ █▀ · BSD 3-Clause License :
: :
: (c) 2022-2026 vmfunc, xyzeva, :
: lunchcat alumni & contributors :
: :
·━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━·
*/
package finding
import (
"strings"
"testing"
"github.com/dropalldatabases/sif/internal/modules"
"github.com/dropalldatabases/sif/internal/scan"
"github.com/dropalldatabases/sif/internal/scan/frameworks"
"github.com/dropalldatabases/sif/internal/scan/js"
"github.com/projectdiscovery/nuclei/v3/pkg/model"
"github.com/projectdiscovery/nuclei/v3/pkg/model/types/severity"
"github.com/projectdiscovery/nuclei/v3/pkg/output"
)
// scanResultType mirrors the minimal interface the scan packages implement; the
// coverage table below carries a value per ResultType() so a new scanner whose
// ResultType isn't represented (or isn't handled by Flatten) trips a failure.
type scanResultType interface {
ResultType() string
}
// coverageCase is one representative, non-empty instance of a result type plus
// its expected module attribution. wantItems is how many findings Flatten must
// emit for the populated instance, proving the per-item fan-out works.
type coverageCase struct {
value any // the result as it reaches Flatten
typed scanResultType // same value when it implements ResultType(), else nil
module string // module id Flatten should stamp
wantItems int // findings the populated instance must produce
}
// coverageCases is the registry the guard checks against. there must be one
// entry per distinct ResultType() in the scan tree (plus the raw []string and
// nuclei []ResultEvent that flow through the report without a ResultType). add a
// scanner without adding it here and TestFlattenCoversEveryResultType fails.
func coverageCases() []coverageCase {
return []coverageCase{
{
value: &scan.ShodanResult{IP: "1.2.3.4", Ports: []int{80}, Vulns: []string{"CVE-1"}},
typed: &scan.ShodanResult{},
module: "shodan",
wantItems: 1,
},
{
value: &scan.SQLResult{
AdminPanels: []scan.SQLAdminPanel{{URL: "http://x/pma", Type: "phpMyAdmin", Status: 200}},
DatabaseErrors: []scan.SQLDatabaseError{{URL: "http://x", DatabaseType: "mysql", ErrorPattern: "syntax"}},
ExposedPorts: []int{3306},
},
typed: &scan.SQLResult{},
module: "sql",
wantItems: 3,
},
{
value: &scan.LFIResult{Vulnerabilities: []scan.LFIVulnerability{
{URL: "http://x", Parameter: "file", Evidence: "root:x", Severity: "high"},
}},
typed: &scan.LFIResult{},
module: "lfi",
wantItems: 1,
},
{
value: &scan.JWTResult{Tokens: []scan.JWTToken{{
Source: "header:Authorization",
Alg: "none",
Issues: []scan.JWTIssue{
{Kind: "alg:none", Severity: "critical", Detail: "no signature"},
{Kind: "missing exp", Severity: "medium", Detail: "no expiry"},
},
}}},
typed: &scan.JWTResult{},
module: "jwt",
wantItems: 2,
},
{
value: &scan.OpenAPIResult{
SpecURL: "http://x/openapi.json",
Severity: "high",
Endpoints: []scan.OpenAPIEndpoint{{Path: "/users", Method: "GET", Unauth: true}},
},
typed: &scan.OpenAPIResult{},
module: "openapi",
wantItems: 1,
},
{
value: &scan.FaviconResult{Hash: 116323821, Tech: "Apache Tomcat", ShodanQ: "http.favicon.hash:116323821"},
typed: &scan.FaviconResult{},
module: "favicon",
wantItems: 1,
},
{
value: &scan.CMSResult{Name: "WordPress", Version: "6.1"},
typed: &scan.CMSResult{},
module: "cms",
wantItems: 1,
},
{
value: &scan.SecurityTrailsResult{Domain: "x.com", Subdomains: []string{"a.x.com"}, AssociatedDomains: []string{"y.com"}},
typed: &scan.SecurityTrailsResult{},
module: "securitytrails",
wantItems: 2,
},
{
value: &scan.CORSResult{Findings: []scan.CORSFinding{{URL: "http://x", OriginTested: "null", AllowOrigin: "null", Severity: "medium", Note: "null origin"}}},
typed: &scan.CORSResult{},
module: "cors",
wantItems: 1,
},
{
value: &scan.RedirectResult{Findings: []scan.RedirectFinding{{URL: "http://x", Parameter: "next", Location: "http://evil", Via: "header", Severity: "medium"}}},
typed: &scan.RedirectResult{},
module: "redirect",
wantItems: 1,
},
{
value: &scan.XSSResult{Findings: []scan.XSSFinding{{URL: "http://x", Parameter: "q", Context: "html", SurvivedRaw: []string{"<"}, Severity: "high"}}},
typed: &scan.XSSResult{},
module: "xss",
wantItems: 1,
},
{
value: &scan.CrawlResult{URLs: []string{"http://x/a"}},
typed: &scan.CrawlResult{},
module: "crawl",
wantItems: 1,
},
{
value: &scan.PassiveResult{Subdomains: []string{"a.x.com"}, URLs: []string{"http://x/old"}},
typed: &scan.PassiveResult{},
module: "passive",
wantItems: 2,
},
{
value: &scan.ProbeResult{URL: "http://x", Alive: true, StatusCode: 200, Title: "home"},
typed: &scan.ProbeResult{},
module: "probe",
wantItems: 1,
},
{
value: scan.HeaderResults{{Name: "Server", Value: "nginx"}},
typed: scan.HeaderResults{},
module: "headers",
wantItems: 1,
},
{
value: scan.SecurityHeaderResults{{Header: "Content-Security-Policy", Present: false, Severity: "medium", Note: "missing"}},
typed: scan.SecurityHeaderResults{},
module: "security_headers",
wantItems: 1,
},
{
value: scan.DirectoryResults{{Url: "http://x/admin", StatusCode: 301, Size: 10, Words: 2}},
typed: scan.DirectoryResults{},
module: "dirlist",
wantItems: 1,
},
{
value: scan.CloudStorageResults{{BucketName: "x-assets", IsPublic: true}},
typed: scan.CloudStorageResults{},
module: "cloudstorage",
wantItems: 1,
},
{
value: scan.DorkResults{{Url: "http://x/leak", Count: 1}},
typed: scan.DorkResults{},
module: "dork",
wantItems: 1,
},
{
value: scan.SubdomainTakeoverResults{{Subdomain: "old.x.com", Vulnerable: true, Service: "GitHub Pages"}},
typed: scan.SubdomainTakeoverResults{},
module: "subdomain_takeover",
wantItems: 1,
},
{
value: &frameworks.FrameworkResult{Name: "Laravel", Version: "9.0", RiskLevel: "high", CVEs: []string{"CVE-2"}},
typed: &frameworks.FrameworkResult{},
module: "framework",
wantItems: 1,
},
{
value: &js.JavascriptScanResult{
SecretMatches: []js.SecretMatch{{Rule: "aws-key", Match: "AKIA...", Source: "http://x/app.js"}},
Endpoints: []string{"/api/v1"},
},
typed: &js.JavascriptScanResult{},
module: "js",
wantItems: 2,
},
{
value: &modules.Result{ModuleID: "custom-mod", Target: "http://x", Findings: []modules.Finding{{URL: "http://x", Severity: "low", Evidence: "hit"}}},
typed: &modules.Result{ModuleID: "custom-mod"},
module: "custom-mod",
wantItems: 1,
},
{
// nuclei results aren't ScanResult-typed; they ride through the report
// as a raw []ResultEvent, so cover that shape explicitly.
value: []output.ResultEvent{{TemplateID: "t1", Host: "x", Matched: "http://x", Info: model.Info{Name: "n", SeverityHolder: severity.Holder{Severity: severity.High}}}},
module: "nuclei",
wantItems: 1,
},
{
// dnslist/portscan/git all hand Flatten a bare []string keyed only by
// the module argument.
value: []string{"sub.x.com"},
module: "dnslist",
wantItems: 1,
},
}
}
const target = "http://target.example"
// TestFlattenCoversEveryResultType is the guard: every result type in the
// coverage table must flatten into the expected module without hitting the
// "unhandled" fallback. a new scanner that skips both the table and Flatten's
// switch trips this loudly.
func TestFlattenCoversEveryResultType(t *testing.T) {
for _, tc := range coverageCases() {
findings := Flatten(target, tc.module, tc.value)
if len(findings) != tc.wantItems {
t.Errorf("module %q: got %d findings, want %d", tc.module, len(findings), tc.wantItems)
}
for i := 0; i < len(findings); i++ {
f := findings[i]
if strings.HasSuffix(f.Key, keySep+"unhandled") {
t.Errorf("module %q: Flatten has no case, fell through to unhandled (key=%q)", tc.module, f.Key)
}
if f.Target != target {
t.Errorf("module %q: target=%q, want %q", tc.module, f.Target, target)
}
if f.Module != tc.module {
t.Errorf("module %q: finding stamped module=%q, want %q", tc.module, f.Module, tc.module)
}
if f.Key == "" {
t.Errorf("module %q: empty Key", tc.module)
}
if !strings.HasPrefix(f.Key, tc.module+keySep) {
t.Errorf("module %q: Key %q not prefixed with module", tc.module, f.Key)
}
}
}
}
// TestEveryResultTypeIsInCoverageTable cross-checks the table against the actual
// ResultType() registry: if a scanner type exists whose ResultType() isn't in
// the table, the coverage guard above would never exercise it. enumerate the
// known typed entries and assert each ResultType() string is present.
func TestEveryResultTypeIsInCoverageTable(t *testing.T) {
covered := make(map[string]struct{})
for _, tc := range coverageCases() {
if tc.typed == nil {
continue
}
covered[tc.typed.ResultType()] = struct{}{}
}
// the full set of ResultType() strings the scan tree exposes. keep this in
// lockstep with the ScanResult implementers; a missing entry means the table
// (and very likely Flatten) skipped a scanner.
want := []string{
"shodan", "sql", "lfi", "jwt", "openapi", "favicon", "cms", "securitytrails",
"cors", "redirect", "xss", "crawl", "passive", "probe",
"headers", "security_headers", "dirlist", "cloudstorage",
"dork", "subdomain_takeover", "framework", "js", "custom-mod",
}
for _, rt := range want {
if _, ok := covered[rt]; !ok {
t.Errorf("ResultType %q has no entry in coverageCases; Flatten coverage unverified", rt)
}
}
}
// TestFlattenStableKeysAndSeverities pins the keys and severities for a couple
// of representative items so a refactor that quietly reshuffles them is caught.
func TestFlattenStableKeysAndSeverities(t *testing.T) {
tests := []struct {
name string
value any
module string
wantKey string
wantSev Severity
}{
{
name: "cors honors source severity",
value: &scan.CORSResult{Findings: []scan.CORSFinding{{URL: "http://x", OriginTested: "null", AllowOrigin: "null", Severity: "high", Note: "n"}}},
module: "cors",
wantKey: "cors:http://x:null",
wantSev: SeverityHigh,
},
{
name: "public bucket is high",
value: scan.CloudStorageResults{{BucketName: "b", IsPublic: true}},
module: "cloudstorage",
wantKey: "cloudstorage:b",
wantSev: SeverityHigh,
},
{
name: "header is recon info",
value: scan.HeaderResults{{Name: "Server", Value: "nginx"}},
module: "headers",
wantKey: "headers:Server",
wantSev: SeverityInfo,
},
{
name: "vulnerable takeover is high",
value: scan.SubdomainTakeoverResults{{Subdomain: "old.x.com", Vulnerable: true, Service: "GitHub Pages"}},
module: "subdomain_takeover",
wantKey: "subdomain_takeover:old.x.com",
wantSev: SeverityHigh,
},
}
for _, tt := range tests {
t.Run(tt.name, func(t *testing.T) {
findings := Flatten(target, tt.module, tt.value)
if len(findings) != 1 {
t.Fatalf("got %d findings, want 1", len(findings))
}
f := findings[0]
if f.Key != tt.wantKey {
t.Errorf("Key = %q, want %q", f.Key, tt.wantKey)
}
if f.Severity != tt.wantSev {
t.Errorf("Severity = %v, want %v", f.Severity, tt.wantSev)
}
})
}
}
// TestFlattenUnhandledTypeIsLoud asserts the fallback fires for a type Flatten
// doesn't know - this is what makes the guard above meaningful.
func TestFlattenUnhandledTypeIsLoud(t *testing.T) {
type bogus struct{}
findings := Flatten(target, "mystery", bogus{})
if len(findings) != 1 {
t.Fatalf("got %d findings, want 1 placeholder", len(findings))
}
if !strings.HasSuffix(findings[0].Key, keySep+"unhandled") {
t.Errorf("unhandled type should key on :unhandled, got %q", findings[0].Key)
}
if findings[0].Severity != SeverityUnknown {
t.Errorf("unhandled severity = %v, want SeverityUnknown", findings[0].Severity)
}
}
// TestSubdomainTakeoverSkipsSafe confirms a non-vulnerable cname produces no
// finding; only the real takeover is a finding.
func TestSubdomainTakeoverSkipsSafe(t *testing.T) {
value := scan.SubdomainTakeoverResults{
{Subdomain: "safe.x.com", Vulnerable: false},
{Subdomain: "bad.x.com", Vulnerable: true, Service: "Heroku"},
}
findings := Flatten(target, "subdomain_takeover", value)
if len(findings) != 1 {
t.Fatalf("got %d findings, want 1 (only the vulnerable one)", len(findings))
}
if findings[0].Key != "subdomain_takeover:bad.x.com" {
t.Errorf("Key = %q, want subdomain_takeover:bad.x.com", findings[0].Key)
}
}
// TestDeadProbeIsNotAFinding confirms a host that didn't answer yields nothing.
func TestDeadProbeIsNotAFinding(t *testing.T) {
findings := Flatten(target, "probe", &scan.ProbeResult{URL: "http://x", Alive: false})
if len(findings) != 0 {
t.Errorf("dead probe produced %d findings, want 0", len(findings))
}
}
+48
View File
@@ -0,0 +1,48 @@
/*
·━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━·
: :
: █▀ █ █▀▀ · Blazing-fast pentesting suite :
: ▄█ █ █▀ · BSD 3-Clause License :
: :
: (c) 2022-2026 vmfunc, xyzeva, :
: lunchcat alumni & contributors :
: :
·━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━·
*/
package finding
import "testing"
// Line is the -silent wire format; its shape is frozen, so pin it.
func TestFindingLine(t *testing.T) {
tests := []struct {
name string
f Finding
want string
}{
{
name: "high severity",
f: Finding{Target: "https://x.com", Module: "sql", Severity: SeverityHigh, Title: "admin panel"},
want: "[high] https://x.com sql admin panel",
},
{
name: "info recon",
f: Finding{Target: "https://y.com", Module: "headers", Severity: SeverityInfo, Title: "Server"},
want: "[info] https://y.com headers Server",
},
{
name: "unknown severity",
f: Finding{Target: "z.com", Module: "mystery", Severity: SeverityUnknown, Title: "?"},
want: "[unknown] z.com mystery ?",
},
}
for _, tt := range tests {
t.Run(tt.name, func(t *testing.T) {
if got := tt.f.Line(); got != tt.want {
t.Errorf("Line() = %q, want %q", got, tt.want)
}
})
}
}
+78
View File
@@ -0,0 +1,78 @@
/*
·━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━·
: :
: █▀ █ █▀▀ · Blazing-fast pentesting suite :
: ▄█ █ █▀ · BSD 3-Clause License :
: :
: (c) 2022-2026 vmfunc, xyzeva, :
: lunchcat alumni & contributors :
: :
·━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━·
*/
package finding
import "strings"
// Severity is an ordered severity rank shared by every normalized finding.
// the order matters: notify gates on a threshold and diff sorts by it, so the
// underlying ints have to compare info < low < medium < high < critical.
type Severity int
// severity ranks, lowest to highest. SeverityUnknown sorts below everything so
// an unrecognized scanner string never silently outranks a real critical.
const (
SeverityUnknown Severity = iota
SeverityInfo
SeverityLow
SeverityMedium
SeverityHigh
SeverityCritical
)
// severityNames maps each rank to its canonical lowercase string. the wire
// format scanners emit ("info"/"low"/...) round-trips through this table.
var severityNames = map[Severity]string{
SeverityUnknown: "unknown",
SeverityInfo: "info",
SeverityLow: "low",
SeverityMedium: "medium",
SeverityHigh: "high",
SeverityCritical: "critical",
}
// String renders the canonical lowercase name for the rank.
func (s Severity) String() string {
if name, ok := severityNames[s]; ok {
return name
}
return severityNames[SeverityUnknown]
}
// ParseSeverity maps a scanner's free-form severity string onto a rank. it's
// case/space insensitive and folds the common synonyms ("informational",
// "warning", "moderate") so the dozen scanners that each picked their own
// spelling all land on the same ladder. an empty or unrecognized value is
// SeverityUnknown rather than a guess.
func ParseSeverity(raw string) Severity {
switch strings.ToLower(strings.TrimSpace(raw)) {
case "critical":
return SeverityCritical
case "high":
return SeverityHigh
case "medium", "moderate", "warning":
return SeverityMedium
case "low":
return SeverityLow
case "info", "informational", "information", "none":
return SeverityInfo
default:
return SeverityUnknown
}
}
// AtLeast reports whether s is at or above threshold; notify uses it to drop
// findings below the configured floor.
func (s Severity) AtLeast(threshold Severity) bool {
return s >= threshold
}
+84
View File
@@ -0,0 +1,84 @@
/*
·━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━·
: :
: █▀ █ █▀▀ · Blazing-fast pentesting suite :
: ▄█ █ █▀ · BSD 3-Clause License :
: :
: (c) 2022-2026 vmfunc, xyzeva, :
: lunchcat alumni & contributors :
: :
·━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━·
*/
package finding
import "testing"
func TestParseSeverity(t *testing.T) {
tests := []struct {
in string
want Severity
}{
{"critical", SeverityCritical},
{"CRITICAL", SeverityCritical},
{" high ", SeverityHigh},
{"medium", SeverityMedium},
{"moderate", SeverityMedium},
{"warning", SeverityMedium},
{"low", SeverityLow},
{"info", SeverityInfo},
{"informational", SeverityInfo},
{"none", SeverityInfo},
{"", SeverityUnknown},
{"bogus", SeverityUnknown},
}
for _, tt := range tests {
if got := ParseSeverity(tt.in); got != tt.want {
t.Errorf("ParseSeverity(%q) = %v, want %v", tt.in, got, tt.want)
}
}
}
func TestSeverityOrdering(t *testing.T) {
// the ladder must be strictly increasing for AtLeast/sort to behave.
ordered := []Severity{
SeverityUnknown, SeverityInfo, SeverityLow,
SeverityMedium, SeverityHigh, SeverityCritical,
}
for i := 1; i < len(ordered); i++ {
if ordered[i-1] >= ordered[i] {
t.Errorf("severity ladder not increasing at %d: %v !< %v", i, ordered[i-1], ordered[i])
}
}
}
func TestSeverityAtLeast(t *testing.T) {
tests := []struct {
sev Severity
threshold Severity
want bool
}{
{SeverityHigh, SeverityMedium, true},
{SeverityMedium, SeverityMedium, true},
{SeverityLow, SeverityMedium, false},
{SeverityCritical, SeverityInfo, true},
{SeverityUnknown, SeverityInfo, false},
}
for _, tt := range tests {
if got := tt.sev.AtLeast(tt.threshold); got != tt.want {
t.Errorf("%v.AtLeast(%v) = %v, want %v", tt.sev, tt.threshold, got, tt.want)
}
}
}
func TestSeverityStringRoundTrip(t *testing.T) {
// every named rank renders to a string ParseSeverity maps back to the same
// rank, so the wire format is lossless for known severities.
for _, sev := range []Severity{
SeverityInfo, SeverityLow, SeverityMedium, SeverityHigh, SeverityCritical,
} {
if got := ParseSeverity(sev.String()); got != sev {
t.Errorf("round-trip %v -> %q -> %v", sev, sev.String(), got)
}
}
}
+70 -7
View File
@@ -17,6 +17,8 @@ package httpx
import (
"fmt"
"io"
"net"
"net/http"
"net/url"
"strings"
@@ -41,6 +43,29 @@ const headerSep = ": "
// equal to the per-second rate keeps the cap honest over any one-second window.
const limiterBurstPerRate = 1
// transport pool tuning. go's default transport caps idle conns per host at 2
// and reuse only kicks in once a response body is fully drained, so without
// these a high thread count just thrashes the dialer instead of pooling.
const (
// total idle conns kept warm across every host we hit in a run.
maxIdleConns = 512
// floor for per-host idle conns so a single-target run still pools even
// when the thread count is tiny.
minIdleConnsPerHost = 8
// how long an idle conn lingers before the pool reaps it.
idleConnTimeout = 90 * time.Second
// keepalive probe interval for live conns; mirrors go's default dialer so
// the socks5 branch doesn't silently lose os-level keepalive.
dialKeepAlive = 30 * time.Second
// dial timeout for the socks5 branch; matches go's default dialer.
dialTimeout = 30 * time.Second
)
// drainCap bounds how much of an unread body DrainClose will copy before
// closing; a body larger than this isn't worth slurping just to reuse the
// conn, so we cap the read and let the conn be discarded instead.
const drainCap = 16 << 10
// Options carries the runtime knobs that apply to every outbound request.
// RateLimit is requests/sec (0 = unlimited); Headers are "Key: Value" strings.
type Options struct {
@@ -49,6 +74,9 @@ type Options struct {
Cookie string
UserAgent string
RateLimit int
// Threads is the scan worker count; it sizes the per-host idle pool so
// concurrent workers hitting one target reuse conns instead of dialing fresh.
Threads int
}
// configured holds the package-level transport built once by Configure. nil
@@ -63,7 +91,7 @@ var (
//
//nolint:gocritic // signature is the package's stable startup api; called once.
func Configure(opts Options) error {
base, err := buildTransport(opts.Proxy)
base, err := buildTransport(opts.Proxy, opts.Threads)
if err != nil {
return err
}
@@ -104,9 +132,10 @@ func Client(timeout time.Duration) *http.Client {
return &http.Client{Timeout: timeout, Transport: rt}
}
// buildTransport clones the default transport and applies the proxy. An empty
// proxy leaves the default behavior (respects HTTP_PROXY env) intact.
func buildTransport(proxyURL string) (*http.Transport, error) {
// buildTransport clones the default transport, tunes its pool for the worker
// count and applies the proxy. An empty proxy leaves the default behavior
// (respects HTTP_PROXY env) intact.
func buildTransport(proxyURL string, threads int) (*http.Transport, error) {
tr, ok := http.DefaultTransport.(*http.Transport)
if !ok {
// unreachable in practice, but never trust an assertion silently.
@@ -114,6 +143,15 @@ func buildTransport(proxyURL string) (*http.Transport, error) {
}
transport := tr.Clone()
// size the idle pool so every worker can keep its conn warm. per-host idle
// must clear the thread count or workers past the cap re-dial each request;
// MaxConnsPerHost stays 0 (unbounded) so the limiter, not the pool, paces us.
transport.MaxIdleConns = maxIdleConns
transport.MaxIdleConnsPerHost = idlePerHost(threads)
transport.MaxConnsPerHost = 0
transport.IdleConnTimeout = idleConnTimeout
transport.ForceAttemptHTTP2 = true
if proxyURL == "" {
return transport, nil
}
@@ -127,9 +165,11 @@ func buildTransport(proxyURL string) (*http.Transport, error) {
case schemeHTTP, schemeHTTPS:
transport.Proxy = http.ProxyURL(parsed)
case schemeSOCKS5:
// socks5 needs a custom dialer; the returned dialer implements
// ContextDialer so cancellation/timeouts propagate.
dialer, err := proxy.SOCKS5("tcp", parsed.Host, nil, proxy.Direct)
// socks5 needs a custom dialer. proxy.SOCKS5 takes a forward dialer, so
// hand it our own net.Dialer with keepalive set - the default
// proxy.Direct has none, which would kill os-level conn pooling.
fwd := &net.Dialer{Timeout: dialTimeout, KeepAlive: dialKeepAlive}
dialer, err := proxy.SOCKS5("tcp", parsed.Host, nil, fwd)
if err != nil {
return nil, fmt.Errorf("socks5 proxy %q: %w", proxyURL, err)
}
@@ -145,6 +185,29 @@ func buildTransport(proxyURL string) (*http.Transport, error) {
return transport, nil
}
// idlePerHost picks the per-host idle pool size: at least the worker count so
// no worker re-dials, never below the floor so a small thread count still pools.
func idlePerHost(threads int) int {
if threads < minIdleConnsPerHost {
return minIdleConnsPerHost
}
return threads
}
// DrainClose fully reads (up to drainCap) and closes resp.Body. go only returns
// a conn to the idle pool when the body is read to EOF, so a caller that only
// closes leaks the conn and forces a fresh dial next time. Call this instead of
// a bare resp.Body.Close() to keep the pool warm. Safe on a nil response.
func DrainClose(resp *http.Response) {
if resp == nil || resp.Body == nil {
return
}
// the read result is intentionally ignored: we're discarding the body and
// about to close it, so a copy error changes nothing we can act on.
_, _ = io.Copy(io.Discard, io.LimitReader(resp.Body, drainCap))
resp.Body.Close()
}
// parseHeaders splits each "Key: Value" entry on the first ": ". Entries
// without the separator are rejected so a typo fails loud instead of silently.
// The returned map is always non-nil so callers can range it unconditionally.
+274
View File
@@ -14,8 +14,12 @@ package httpx
import (
"context"
"io"
"net"
"net/http"
"net/http/httptest"
"strings"
"sync"
"testing"
"time"
)
@@ -215,3 +219,273 @@ func TestRateLimitUnlimited(t *testing.T) {
t.Error("expected no limiter when RateLimit is 0")
}
}
func TestIdlePerHost(t *testing.T) {
tests := []struct {
name string
threads int
want int
}{
{name: "below floor clamps up", threads: 1, want: minIdleConnsPerHost},
{name: "zero clamps up", threads: 0, want: minIdleConnsPerHost},
{name: "at floor", threads: minIdleConnsPerHost, want: minIdleConnsPerHost},
{name: "above floor passes through", threads: 64, want: 64},
}
for _, tt := range tests {
t.Run(tt.name, func(t *testing.T) {
if got := idlePerHost(tt.threads); got != tt.want {
t.Errorf("idlePerHost(%d) = %d, want %d", tt.threads, got, tt.want)
}
})
}
}
func TestBuildTransportTuning(t *testing.T) {
const threads = 32
tr, err := buildTransport("", threads)
if err != nil {
t.Fatalf("buildTransport: %v", err)
}
if tr.MaxIdleConns != maxIdleConns {
t.Errorf("MaxIdleConns = %d, want %d", tr.MaxIdleConns, maxIdleConns)
}
if tr.MaxIdleConnsPerHost != threads {
t.Errorf("MaxIdleConnsPerHost = %d, want %d", tr.MaxIdleConnsPerHost, threads)
}
if tr.MaxConnsPerHost != 0 {
t.Errorf("MaxConnsPerHost = %d, want 0 (unbounded)", tr.MaxConnsPerHost)
}
if tr.IdleConnTimeout != idleConnTimeout {
t.Errorf("IdleConnTimeout = %v, want %v", tr.IdleConnTimeout, idleConnTimeout)
}
if !tr.ForceAttemptHTTP2 {
t.Error("ForceAttemptHTTP2 = false, want true")
}
}
func TestDrainClose(t *testing.T) {
resetConfig(t)
// serve a body the caller never reads; DrainClose must drain it so the conn
// is eligible for reuse rather than abandoned mid-stream.
const body = "sif response body that the caller never reads"
srv := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, _ *http.Request) {
w.WriteHeader(http.StatusOK)
io.WriteString(w, body)
}))
t.Cleanup(srv.Close)
req, err := http.NewRequestWithContext(context.Background(), http.MethodGet, srv.URL, http.NoBody)
if err != nil {
t.Fatalf("new request: %v", err)
}
resp, err := Client(5 * time.Second).Do(req)
if err != nil {
t.Fatalf("do request: %v", err)
}
DrainClose(resp)
// after DrainClose the body is closed; a further read must fail.
if _, err := resp.Body.Read(make([]byte, 1)); err == nil {
t.Error("expected read after DrainClose to fail on a closed body")
}
}
func TestDrainCloseNil(t *testing.T) {
// a nil response (e.g. an errored request) must not panic.
DrainClose(nil)
DrainClose(&http.Response{})
}
// countConns wraps a test server with a ConnState hook that tallies how many
// distinct tcp conns the server saw. distinct conns == failed reuse.
func countConns(t *testing.T) (*httptest.Server, func() int) {
t.Helper()
var (
mu sync.Mutex
conns = make(map[net.Conn]struct{})
)
srv := httptest.NewUnstartedServer(http.HandlerFunc(func(w http.ResponseWriter, _ *http.Request) {
// always write a body so reuse depends on the caller draining it.
io.WriteString(w, "ok")
}))
srv.Config.ConnState = func(c net.Conn, state http.ConnState) {
if state != http.StateNew {
return
}
mu.Lock()
conns[c] = struct{}{}
mu.Unlock()
}
srv.Start()
t.Cleanup(srv.Close)
return srv, func() int {
mu.Lock()
defer mu.Unlock()
return len(conns)
}
}
func TestTransportReusesConnections(t *testing.T) {
resetConfig(t)
const (
threads = 8
requests = 30
)
if err := Configure(Options{Threads: threads}); err != nil {
t.Fatalf("Configure: %v", err)
}
srv, distinct := countConns(t)
// fire N sequential requests through the tuned client, draining each body so
// the conn returns to the pool. a working pool serves all of them on one conn.
client := Client(5 * time.Second)
for i := 0; i < requests; i++ {
req, err := http.NewRequestWithContext(context.Background(), http.MethodGet, srv.URL, http.NoBody)
if err != nil {
t.Fatalf("new request %d: %v", i, err)
}
resp, err := client.Do(req)
if err != nil {
t.Fatalf("do request %d: %v", i, err)
}
DrainClose(resp)
}
// sequential reuse should land on exactly one conn; allow a tiny margin for
// the rare race where a conn is reaped between requests.
const maxReuseConns = 2
if got := distinct(); got > maxReuseConns {
t.Errorf("tuned client opened %d conns for %d requests, want <= %d (pool not reusing)",
got, requests, maxReuseConns)
}
}
func TestBareClientDoesNotReuse(t *testing.T) {
srv, distinct := countConns(t)
// the control: a bare DefaultTransport client whose caller closes but never
// drains the body. go can't reuse a half-read conn, so each request dials
// fresh - this is exactly the pre-tuning behavior we're fixing.
client := &http.Client{
Timeout: 5 * time.Second,
Transport: http.DefaultTransport.(*http.Transport).Clone(),
}
const requests = 30
for i := 0; i < requests; i++ {
req, err := http.NewRequestWithContext(context.Background(), http.MethodGet, srv.URL, http.NoBody)
if err != nil {
t.Fatalf("new request %d: %v", i, err)
}
resp, err := client.Do(req)
if err != nil {
t.Fatalf("do request %d: %v", i, err)
}
// close without draining - the leak that kills reuse.
resp.Body.Close()
}
// most requests should have dialed a fresh conn. don't demand exactly N (the
// scheduler occasionally reuses one), just that it's clearly not pooling.
const minDistinct = requests / 2
if got := distinct(); got < minDistinct {
t.Errorf("bare client opened only %d conns for %d requests, want >= %d "+
"(expected near-zero reuse without draining)", got, requests, minDistinct)
}
}
// BenchmarkConnReuse contrasts the tuned, draining client against a bare client
// that closes without draining. the reported conns/op metric is the distinct
// tcp conns one pass of `requests` opened - tuned≈1, bare≈requests - so the
// README can quote real before/after reuse numbers. the conn map is reset per
// iteration so the metric stays a per-pass count and the bare path doesn't
// accumulate b.N*requests live sockets and exhaust the ephemeral port range.
//
// run the bare sub-bench with a bounded -benchtime (e.g. -benchtime 5x): its
// whole point is that it can't reuse, so a large b.N floods the local port
// space with TIME_WAIT sockets. the tuned sub-bench reuses and runs unbounded.
func BenchmarkConnReuse(b *testing.B) {
const requests = 50
run := func(b *testing.B, drain bool, client *http.Client) {
b.Helper()
var (
mu sync.Mutex
conns = make(map[net.Conn]struct{})
)
srv := httptest.NewUnstartedServer(http.HandlerFunc(func(w http.ResponseWriter, _ *http.Request) {
io.WriteString(w, strings.Repeat("x", 256))
}))
srv.Config.ConnState = func(c net.Conn, state http.ConnState) {
if state != http.StateNew {
return
}
mu.Lock()
conns[c] = struct{}{}
mu.Unlock()
}
srv.Start()
defer srv.Close()
var lastPass int
b.ResetTimer()
for n := 0; n < b.N; n++ {
mu.Lock()
conns = make(map[net.Conn]struct{})
mu.Unlock()
for i := 0; i < requests; i++ {
req, _ := http.NewRequestWithContext(context.Background(), http.MethodGet, srv.URL, http.NoBody)
resp, err := client.Do(req)
if err != nil {
b.Fatalf("do: %v", err)
}
if drain {
DrainClose(resp)
} else {
resp.Body.Close()
}
}
// close idle conns between passes so the bare client's per-pass
// sockets land in TIME_WAIT and free up before the next pass.
client.CloseIdleConnections()
mu.Lock()
lastPass = len(conns)
mu.Unlock()
}
b.StopTimer()
// distinct conns for a single pass of `requests`.
b.ReportMetric(float64(lastPass), "conns/op")
}
b.Run("tuned-drain", func(b *testing.B) {
resetBench()
tr, err := buildTransport("", 8)
if err != nil {
b.Fatalf("buildTransport: %v", err)
}
run(b, true, &http.Client{Timeout: 5 * time.Second, Transport: tr})
})
b.Run("bare-noDrain", func(b *testing.B) {
run(b, false, &http.Client{
Timeout: 5 * time.Second,
Transport: http.DefaultTransport.(*http.Transport).Clone(),
})
})
}
// resetBench clears the package transport without a *testing.T for benchmarks.
func resetBench() {
mu.Lock()
configured = nil
mu.Unlock()
}
+16 -16
View File
@@ -14,6 +14,7 @@ package modules
import (
"context"
"errors"
"fmt"
"io"
"net/http"
@@ -26,6 +27,11 @@ import (
// MaxBodySize limits response body to prevent memory exhaustion.
const MaxBodySize = 5 * 1024 * 1024
// ErrUnsupportedModuleType signals an executor for a module type that is not
// yet implemented. Returning it (rather than an empty result) keeps callers
// from mistaking "not implemented" for "scanned, found nothing".
var ErrUnsupportedModuleType = errors.New("unsupported module type")
// httpRequest represents a generated HTTP request.
type httpRequest struct {
Method string
@@ -379,22 +385,16 @@ func truncateEvidence(s string) string {
return s
}
// ExecuteDNSModule runs a DNS-based module (stub for now).
func ExecuteDNSModule(ctx context.Context, target string, def *YAMLModule, opts Options) (*Result, error) {
// TODO: Implement DNS module execution
return &Result{
ModuleID: def.ID,
Target: target,
Findings: []Finding{},
}, nil
// ExecuteDNSModule runs a DNS-based module (not yet implemented).
// returns ErrUnsupportedModuleType so the caller logs a clear failure rather
// than reporting an empty (but successful-looking) result.
func ExecuteDNSModule(_ context.Context, _ string, def *YAMLModule, _ Options) (*Result, error) {
return nil, fmt.Errorf("dns module %q: %w", def.ID, ErrUnsupportedModuleType)
}
// ExecuteTCPModule runs a TCP-based module (stub for now).
func ExecuteTCPModule(ctx context.Context, target string, def *YAMLModule, opts Options) (*Result, error) {
// TODO: Implement TCP module execution
return &Result{
ModuleID: def.ID,
Target: target,
Findings: []Finding{},
}, nil
// ExecuteTCPModule runs a TCP-based module (not yet implemented).
// returns ErrUnsupportedModuleType so the caller logs a clear failure rather
// than reporting an empty (but successful-looking) result.
func ExecuteTCPModule(_ context.Context, _ string, def *YAMLModule, _ Options) (*Result, error) {
return nil, fmt.Errorf("tcp module %q: %w", def.ID, ErrUnsupportedModuleType)
}
+270
View File
@@ -0,0 +1,270 @@
/*
·━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━·
: :
: █▀ █ █▀▀ · Blazing-fast pentesting suite :
: ▄█ █ █▀ · BSD 3-Clause License :
: :
: (c) 2022-2026 vmfunc, xyzeva, :
: lunchcat alumni & contributors :
: :
·━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━·
*/
package modules
import (
"context"
"errors"
"net/http"
"net/http/httptest"
"testing"
"time"
"github.com/dropalldatabases/sif/internal/httpx"
)
const testTimeout = 5 * time.Second
// TestExecuteHTTPModuleMatchAndExtract drives the full executor against a live
// httptest server: a request hits a path, a matcher fires, an extractor captures.
func TestExecuteHTTPModuleMatchAndExtract(t *testing.T) {
srv := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
if r.URL.Path == "/admin" {
w.Header().Set("X-App", "demo")
w.WriteHeader(http.StatusOK)
_, _ = w.Write([]byte(`flag{found-it} session=sess-4242`))
return
}
w.WriteHeader(http.StatusNotFound)
}))
defer srv.Close()
def := &YAMLModule{
ID: "test-http-hit",
Type: TypeHTTP,
Info: YAMLModuleInfo{Severity: "high"},
HTTP: &HTTPConfig{
Method: "GET",
Paths: []string{"{{BaseURL}}/admin", "{{BaseURL}}/missing"},
Matchers: []Matcher{
{Type: "status", Status: []int{200}},
{Type: "word", Part: "body", Words: []string{"flag{found-it}"}},
},
Extractors: []Extractor{
{Type: "regex", Name: "session", Part: "body", Regex: []string{`session=(\S+)`}, Group: 1},
},
},
}
// route through the shared httpx client so proxy/-H/-rate-limit would apply.
opts := Options{Timeout: testTimeout, Client: httpx.Client(testTimeout)}
result, err := ExecuteHTTPModule(context.Background(), srv.URL, def, opts)
if err != nil {
t.Fatalf("ExecuteHTTPModule: %v", err)
}
// only /admin satisfies status+word, /missing returns 404.
if len(result.Findings) != 1 {
t.Fatalf("got %d findings, want 1", len(result.Findings))
}
f := result.Findings[0]
if f.Severity != "high" {
t.Errorf("severity = %q, want high (carried from Info)", f.Severity)
}
if f.Extracted["session"] != "sess-4242" {
t.Errorf("extracted session = %q, want sess-4242", f.Extracted["session"])
}
if f.URL != srv.URL+"/admin" {
t.Errorf("finding url = %q, want %q", f.URL, srv.URL+"/admin")
}
}
// TestExecuteHTTPModuleNoMatch confirms a module that matches nothing reports
// zero findings without erroring.
func TestExecuteHTTPModuleNoMatch(t *testing.T) {
srv := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, _ *http.Request) {
w.WriteHeader(http.StatusOK)
_, _ = w.Write([]byte("nothing interesting"))
}))
defer srv.Close()
def := &YAMLModule{
ID: "test-http-miss",
Type: TypeHTTP,
HTTP: &HTTPConfig{
Paths: []string{"{{BaseURL}}/"},
Matchers: []Matcher{
{Type: "word", Part: "body", Words: []string{"never-present"}},
},
},
}
result, err := ExecuteHTTPModule(context.Background(), srv.URL, def, Options{Timeout: testTimeout, Client: httpx.Client(testTimeout)})
if err != nil {
t.Fatalf("ExecuteHTTPModule: %v", err)
}
if len(result.Findings) != 0 {
t.Fatalf("got %d findings, want 0", len(result.Findings))
}
}
// TestExecuteHTTPModulePayloadExpansion verifies payload templates reach the
// server and the matching response is captured.
func TestExecuteHTTPModulePayloadExpansion(t *testing.T) {
srv := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
// only the "boom" payload triggers the vulnerable branch.
if r.URL.Query().Get("q") == "boom" {
_, _ = w.Write([]byte("error: sql syntax near boom"))
return
}
_, _ = w.Write([]byte("ok"))
}))
defer srv.Close()
def := &YAMLModule{
ID: "test-http-payload",
Type: TypeHTTP,
HTTP: &HTTPConfig{
Paths: []string{"{{BaseURL}}/search?q={{payload}}"},
Payloads: []string{"safe", "boom"},
Matchers: []Matcher{
{Type: "word", Part: "body", Words: []string{"sql syntax"}},
},
},
}
result, err := ExecuteHTTPModule(context.Background(), srv.URL, def, Options{Timeout: testTimeout, Client: httpx.Client(testTimeout)})
if err != nil {
t.Fatalf("ExecuteHTTPModule: %v", err)
}
if len(result.Findings) != 1 {
t.Fatalf("got %d findings, want 1 (only boom payload)", len(result.Findings))
}
}
func TestExecuteHTTPModuleNoConfig(t *testing.T) {
def := &YAMLModule{ID: "x", Type: TypeHTTP}
if _, err := ExecuteHTTPModule(context.Background(), "http://h", def, Options{}); err == nil {
t.Fatal("expected error when HTTP config is nil")
}
}
// TestExecuteHTTPModuleContextCancel pins the cancellation path. The dispatch
// loop selects between ctx.Done() and the concurrency semaphore, so a cancelled
// context can either short-circuit with ctx.Err() or let the in-flight request
// fail on the dead context. Both are correct: the contract is "never hang, never
// invent a finding", which is what we assert here rather than forcing one race
// winner (that made this test flaky under -count).
func TestExecuteHTTPModuleContextCancel(t *testing.T) {
srv := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, _ *http.Request) {
w.WriteHeader(http.StatusOK)
}))
defer srv.Close()
ctx, cancel := context.WithCancel(context.Background())
cancel()
def := &YAMLModule{
ID: "test-http-cancel",
Type: TypeHTTP,
HTTP: &HTTPConfig{
Paths: []string{"{{BaseURL}}/a"},
Matchers: []Matcher{{Type: "status", Status: []int{200}}},
},
}
result, err := ExecuteHTTPModule(ctx, srv.URL, def, Options{Timeout: testTimeout, Client: httpx.Client(testTimeout)})
if err != nil {
if !errors.Is(err, context.Canceled) {
t.Fatalf("err = %v, want context.Canceled or nil", err)
}
return
}
// no error means the request was dispatched but failed on the dead context;
// either way a cancelled scan must not surface findings.
if len(result.Findings) != 0 {
t.Fatalf("cancelled scan produced %d findings, want 0", len(result.Findings))
}
}
// TestExecuteDNSModuleUnsupported pins the current behavior: DNS execution is
// not implemented and must signal it via ErrUnsupportedModuleType, not by
// quietly returning an empty (successful-looking) result.
func TestExecuteDNSModuleUnsupported(t *testing.T) {
def := &YAMLModule{ID: "dns-mod", Type: TypeDNS, DNS: &DNSConfig{Type: "A"}}
result, err := ExecuteDNSModule(context.Background(), "example.com", def, Options{})
if result != nil {
t.Errorf("result = %v, want nil for unsupported type", result)
}
if !errors.Is(err, ErrUnsupportedModuleType) {
t.Fatalf("err = %v, want ErrUnsupportedModuleType", err)
}
}
func TestExecuteTCPModuleUnsupported(t *testing.T) {
def := &YAMLModule{ID: "tcp-mod", Type: TypeTCP, TCP: &TCPConfig{Port: 22}}
result, err := ExecuteTCPModule(context.Background(), "example.com", def, Options{})
if result != nil {
t.Errorf("result = %v, want nil for unsupported type", result)
}
if !errors.Is(err, ErrUnsupportedModuleType) {
t.Fatalf("err = %v, want ErrUnsupportedModuleType", err)
}
}
// TestWrapperExecuteRoutesByType confirms the Module wrapper dispatches each
// type to the right executor and propagates the unsupported-type sentinel.
func TestWrapperExecuteRoutesByType(t *testing.T) {
t.Run("dns routes to unsupported", func(t *testing.T) {
def := &YAMLModule{ID: "d", Type: TypeDNS, DNS: &DNSConfig{}}
w := newYAMLModuleWrapper(def, "d.yaml")
if _, err := w.Execute(context.Background(), "t", Options{}); !errors.Is(err, ErrUnsupportedModuleType) {
t.Fatalf("err = %v, want ErrUnsupportedModuleType", err)
}
})
t.Run("tcp routes to unsupported", func(t *testing.T) {
def := &YAMLModule{ID: "t", Type: TypeTCP, TCP: &TCPConfig{}}
w := newYAMLModuleWrapper(def, "t.yaml")
if _, err := w.Execute(context.Background(), "t", Options{}); !errors.Is(err, ErrUnsupportedModuleType) {
t.Fatalf("err = %v, want ErrUnsupportedModuleType", err)
}
})
t.Run("missing http config errors", func(t *testing.T) {
def := &YAMLModule{ID: "h", Type: TypeHTTP}
w := newYAMLModuleWrapper(def, "h.yaml")
if _, err := w.Execute(context.Background(), "t", Options{}); err == nil {
t.Fatal("expected error for missing http config")
}
})
t.Run("unknown type errors", func(t *testing.T) {
def := &YAMLModule{ID: "z", Type: ModuleType("bogus")}
w := newYAMLModuleWrapper(def, "z.yaml")
if _, err := w.Execute(context.Background(), "t", Options{}); err == nil {
t.Fatal("expected error for unknown module type")
}
})
}
func TestTruncateEvidence(t *testing.T) {
short := "short evidence"
if got := truncateEvidence(short); got != short {
t.Errorf("short evidence changed: %q", got)
}
long := make([]byte, 600)
for i := range long {
long[i] = 'a'
}
got := truncateEvidence(string(long))
// 500 chars of content plus the ellipsis marker.
if len(got) != 503 {
t.Errorf("truncated len = %d, want 503", len(got))
}
if got[len(got)-3:] != "..." {
t.Errorf("truncated evidence missing ellipsis: %q", got[len(got)-3:])
}
}
+269
View File
@@ -0,0 +1,269 @@
/*
·━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━·
: :
: █▀ █ █▀▀ · Blazing-fast pentesting suite :
: ▄█ █ █▀ · BSD 3-Clause License :
: :
: (c) 2022-2026 vmfunc, xyzeva, :
: lunchcat alumni & contributors :
: :
·━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━·
*/
package modules
import (
"os"
"path/filepath"
"testing"
)
// writeModule drops a yaml file into a temp dir and returns its path.
func writeModule(t *testing.T, dir, name, content string) string {
t.Helper()
path := filepath.Join(dir, name)
if err := os.WriteFile(path, []byte(content), 0o600); err != nil {
t.Fatalf("write module: %v", err)
}
return path
}
func TestParseYAMLModuleValid(t *testing.T) {
const doc = `id: example-http
type: http
info:
name: Example
author: azzie
severity: medium
description: a test module
tags: [test, demo]
http:
method: GET
paths:
- "{{BaseURL}}/admin"
matchers:
- type: status
status: [200]
- type: word
part: body
words: ["admin"]
condition: and
extractors:
- type: regex
name: token
part: body
regex: ["token=(\\w+)"]
group: 1
`
dir := t.TempDir()
path := writeModule(t, dir, "ok.yaml", doc)
def, err := ParseYAMLModule(path)
if err != nil {
t.Fatalf("ParseYAMLModule: %v", err)
}
if def.ID != "example-http" {
t.Errorf("id = %q, want example-http", def.ID)
}
if def.Type != TypeHTTP {
t.Errorf("type = %q, want http", def.Type)
}
if def.Info.Severity != "medium" {
t.Errorf("severity = %q, want medium", def.Info.Severity)
}
if def.HTTP == nil {
t.Fatal("http config not parsed")
}
if len(def.HTTP.Matchers) != 2 {
t.Errorf("got %d matchers, want 2", len(def.HTTP.Matchers))
}
if len(def.HTTP.Extractors) != 1 || def.HTTP.Extractors[0].Group != 1 {
t.Errorf("extractor not parsed correctly: %+v", def.HTTP.Extractors)
}
if len(def.Info.Tags) != 2 {
t.Errorf("got %d tags, want 2", len(def.Info.Tags))
}
}
func TestParseYAMLModuleErrors(t *testing.T) {
dir := t.TempDir()
tests := []struct {
name string
content string
}{
{
name: "missing id",
content: "type: http\nhttp:\n paths: [\"/\"]\n",
},
{
name: "missing type",
content: "id: no-type\nhttp:\n paths: [\"/\"]\n",
},
{
name: "malformed yaml",
content: "id: bad\ntype: http\n paths: [unbalanced\n : nope\n",
},
{
// a scalar where a mapping is expected must fail to unmarshal.
name: "type mismatch",
content: "id: bad-shape\ntype: http\nhttp: \"should-be-a-map\"\n",
},
}
for _, tt := range tests {
t.Run(tt.name, func(t *testing.T) {
path := writeModule(t, dir, tt.name+".yaml", tt.content)
if _, err := ParseYAMLModule(path); err == nil {
t.Fatalf("expected error for %s", tt.name)
}
})
}
}
func TestParseYAMLModuleMissingFile(t *testing.T) {
if _, err := ParseYAMLModule(filepath.Join(t.TempDir(), "does-not-exist.yaml")); err == nil {
t.Fatal("expected error for missing file")
}
}
func TestYAMLModuleWrapperInfoAndType(t *testing.T) {
def := &YAMLModule{
ID: "wrap-test",
Type: TypeHTTP,
Info: YAMLModuleInfo{
Name: "Wrapped",
Author: "azzie",
Severity: "low",
Description: "desc",
Tags: []string{"a", "b"},
},
}
w := newYAMLModuleWrapper(def, "wrap.yaml")
if w.Type() != TypeHTTP {
t.Errorf("Type() = %q, want http", w.Type())
}
info := w.Info()
if info.ID != "wrap-test" || info.Name != "Wrapped" || info.Severity != "low" {
t.Errorf("Info() mismatch: %+v", info)
}
if len(info.Tags) != 2 {
t.Errorf("Info().Tags = %v, want 2 entries", info.Tags)
}
}
// TestLoaderLoadAll exercises the directory walk: a valid module registers, a
// malformed one is skipped without aborting the walk.
func TestLoaderLoadAll(t *testing.T) {
Clear()
t.Cleanup(Clear)
dir := t.TempDir()
writeModule(t, dir, "good.yaml", "id: good-mod\ntype: http\nhttp:\n paths: [\"{{BaseURL}}/\"]\n matchers:\n - type: status\n status: [200]\n")
writeModule(t, dir, "bad.yml", "id: bad-mod\n") // missing type -> skipped
writeModule(t, dir, "ignore.txt", "not a module")
l := &Loader{builtinDir: dir, userDir: filepath.Join(dir, "nonexistent-user")}
if err := l.LoadAll(); err != nil {
t.Fatalf("LoadAll: %v", err)
}
// only the good module loads; the malformed one is logged and skipped.
if l.Loaded() != 1 {
t.Errorf("Loaded() = %d, want 1", l.Loaded())
}
if _, ok := Get("good-mod"); !ok {
t.Error("good-mod not registered")
}
if _, ok := Get("bad-mod"); ok {
t.Error("bad-mod should not have registered")
}
}
func TestNewLoaderDirs(t *testing.T) {
l, err := NewLoader()
if err != nil {
t.Fatalf("NewLoader: %v", err)
}
if l.BuiltinDir() == "" {
t.Error("BuiltinDir is empty")
}
if l.UserDir() == "" {
t.Error("UserDir is empty")
}
}
// TestRegistry exercises the package-level registry: register, get, dedupe by
// id, filter by tag and type, count and clear.
func TestRegistry(t *testing.T) {
Clear()
t.Cleanup(Clear)
http1 := newYAMLModuleWrapper(&YAMLModule{ID: "h1", Type: TypeHTTP, Info: YAMLModuleInfo{Tags: []string{"web", "cve"}}}, "h1")
http2 := newYAMLModuleWrapper(&YAMLModule{ID: "h2", Type: TypeHTTP, Info: YAMLModuleInfo{Tags: []string{"web"}}}, "h2")
dns1 := newYAMLModuleWrapper(&YAMLModule{ID: "d1", Type: TypeDNS, Info: YAMLModuleInfo{Tags: []string{"dns"}}}, "d1")
Register(http1)
Register(http2)
Register(dns1)
if Count() != 3 {
t.Fatalf("Count() = %d, want 3", Count())
}
got, ok := Get("h1")
if !ok || got.Info().ID != "h1" {
t.Errorf("Get(h1) = %v, %v", got, ok)
}
if _, ok := Get("missing"); ok {
t.Error("Get(missing) should report not found")
}
if n := len(ByType(TypeHTTP)); n != 2 {
t.Errorf("ByType(http) = %d, want 2", n)
}
if n := len(ByType(TypeDNS)); n != 1 {
t.Errorf("ByType(dns) = %d, want 1", n)
}
if n := len(ByTag("web")); n != 2 {
t.Errorf("ByTag(web) = %d, want 2", n)
}
if n := len(ByTag("cve")); n != 1 {
t.Errorf("ByTag(cve) = %d, want 1", n)
}
if n := len(ByTag("none")); n != 0 {
t.Errorf("ByTag(none) = %d, want 0", n)
}
if n := len(All()); n != 3 {
t.Errorf("All() = %d, want 3", n)
}
// re-registering the same id overwrites rather than duplicating.
Register(newYAMLModuleWrapper(&YAMLModule{ID: "h1", Type: TypeHTTP}, "h1-v2"))
if Count() != 3 {
t.Errorf("Count() after re-register = %d, want 3", Count())
}
Clear()
if Count() != 0 {
t.Errorf("Count() after Clear = %d, want 0", Count())
}
}
// TestResultType pins the ScanResult interface bridge.
func TestResultType(t *testing.T) {
r := &Result{ModuleID: "abc"}
if r.ResultType() != "abc" {
t.Errorf("ResultType() = %q, want abc", r.ResultType())
}
}
// TestLoaderScriptStubNoop confirms the go-script loader is currently a no-op
// that registers nothing and returns no error.
func TestLoaderScriptStubNoop(t *testing.T) {
l := &Loader{}
if err := l.loadScript("anything.go"); err != nil {
t.Errorf("loadScript stub returned error: %v", err)
}
}
+465
View File
@@ -0,0 +1,465 @@
/*
·━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━·
: :
: █▀ █ █▀▀ · Blazing-fast pentesting suite :
: ▄█ █ █▀ · BSD 3-Clause License :
: :
: (c) 2022-2026 vmfunc, xyzeva, :
: lunchcat alumni & contributors :
: :
·━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━·
*/
package modules
import (
"net/http"
"strings"
"testing"
)
// fakeResponse builds a minimal *http.Response for matcher/extractor tests.
// it carries no real socket (Body is http.NoBody), so there is nothing to
// close; bodyclose is excluded for test files in .golangci.yml. header drives
// the header/all parts without a live server; matchers read the body string
// argument, not resp.Body.
func fakeResponse(t *testing.T, status int, header http.Header) *http.Response {
t.Helper()
if header == nil {
header = http.Header{}
}
return &http.Response{StatusCode: status, Header: header, Body: http.NoBody}
}
func TestCheckMatcherStatus(t *testing.T) {
tests := []struct {
name string
status int
want []int
expect bool
}{
{name: "single match", status: 200, want: []int{200}, expect: true},
{name: "one of many", status: 404, want: []int{200, 301, 404}, expect: true},
{name: "no match", status: 500, want: []int{200, 404}, expect: false},
{name: "empty status list", status: 200, want: nil, expect: false},
}
for _, tt := range tests {
t.Run(tt.name, func(t *testing.T) {
m := &Matcher{Type: "status", Status: tt.want}
resp := fakeResponse(t, tt.status, nil)
if got := checkMatcher(m, resp, ""); got != tt.expect {
t.Errorf("checkMatcher status = %v, want %v", got, tt.expect)
}
})
}
}
func TestCheckMatcherWord(t *testing.T) {
const body = "welcome admin dashboard"
tests := []struct {
name string
words []string
condition string
expect bool
}{
{name: "and all present", words: []string{"admin", "dashboard"}, condition: "and", expect: true},
{name: "and one missing", words: []string{"admin", "missing"}, condition: "and", expect: false},
{name: "default is and", words: []string{"admin", "missing"}, condition: "", expect: false},
{name: "or one present", words: []string{"missing", "admin"}, condition: "or", expect: true},
{name: "or none present", words: []string{"missing", "absent"}, condition: "or", expect: false},
}
for _, tt := range tests {
t.Run(tt.name, func(t *testing.T) {
m := &Matcher{Type: "word", Part: "body", Words: tt.words, Condition: tt.condition}
resp := fakeResponse(t, 200, nil)
if got := checkMatcher(m, resp, body); got != tt.expect {
t.Errorf("checkMatcher word = %v, want %v", got, tt.expect)
}
})
}
}
func TestCheckMatcherRegex(t *testing.T) {
const body = "version 1.2.3 build 99"
tests := []struct {
name string
patterns []string
condition string
expect bool
}{
{name: "and all match", patterns: []string{`version \d`, `build \d+`}, condition: "and", expect: true},
{name: "and one fails", patterns: []string{`version \d`, `nope\d`}, condition: "and", expect: false},
{name: "or one matches", patterns: []string{`nope`, `build \d+`}, condition: "or", expect: true},
{name: "or none match", patterns: []string{`nope`, `zilch`}, condition: "or", expect: false},
// an invalid pattern under AND must fail closed, not panic.
{name: "and invalid pattern fails closed", patterns: []string{`version \d`, `(`}, condition: "and", expect: false},
// under OR an invalid pattern is skipped, a later valid one can still hit.
{name: "or invalid pattern skipped", patterns: []string{`(`, `build \d+`}, condition: "or", expect: true},
}
for _, tt := range tests {
t.Run(tt.name, func(t *testing.T) {
m := &Matcher{Type: "regex", Part: "body", Regex: tt.patterns, Condition: tt.condition}
resp := fakeResponse(t, 200, nil)
if got := checkMatcher(m, resp, body); got != tt.expect {
t.Errorf("checkMatcher regex = %v, want %v", got, tt.expect)
}
})
}
}
func TestCheckMatcherHeaderPart(t *testing.T) {
header := http.Header{"X-Powered-By": []string{"PHP/8.1"}}
resp := fakeResponse(t, 200, header)
m := &Matcher{Type: "word", Part: "header", Words: []string{"PHP/8.1"}}
if !checkMatcher(m, resp, "body-content") {
t.Error("expected header-part word matcher to hit on header value")
}
// the same word lives only in the header, so a body-part matcher must miss.
mBody := &Matcher{Type: "word", Part: "body", Words: []string{"PHP/8.1"}}
if checkMatcher(mBody, resp, "body-content") {
t.Error("body-part matcher should not see header-only value")
}
}
func TestCheckMatcherUnknownType(t *testing.T) {
m := &Matcher{Type: "size", Part: "body"}
resp := fakeResponse(t, 200, nil)
if checkMatcher(m, resp, "anything") {
t.Error("unknown matcher type should not match")
}
}
func TestCheckMatchers(t *testing.T) {
resp := fakeResponse(t, 200, http.Header{"Server": []string{"nginx"}})
const body = "secret token here"
tests := []struct {
name string
matchers []Matcher
expect bool
}{
{
name: "empty matchers never match",
matchers: nil,
expect: false,
},
{
name: "all matchers pass (AND across matchers)",
matchers: []Matcher{
{Type: "status", Status: []int{200}},
{Type: "word", Part: "body", Words: []string{"secret"}},
},
expect: true,
},
{
name: "one matcher fails breaks AND",
matchers: []Matcher{
{Type: "status", Status: []int{200}},
{Type: "word", Part: "body", Words: []string{"absent"}},
},
expect: false,
},
{
name: "negative inverts a non-match into a pass",
matchers: []Matcher{
{Type: "word", Part: "body", Words: []string{"absent"}, Negative: true},
},
expect: true,
},
{
name: "negative inverts a match into a fail",
matchers: []Matcher{
{Type: "word", Part: "body", Words: []string{"secret"}, Negative: true},
},
expect: false,
},
}
for _, tt := range tests {
t.Run(tt.name, func(t *testing.T) {
if got := checkMatchers(tt.matchers, resp, body); got != tt.expect {
t.Errorf("checkMatchers = %v, want %v", got, tt.expect)
}
})
}
}
func TestCheckWords(t *testing.T) {
const content = "alpha beta gamma"
tests := []struct {
name string
words []string
condition string
expect bool
}{
{name: "and all present", words: []string{"alpha", "gamma"}, condition: "and", expect: true},
{name: "and missing", words: []string{"alpha", "delta"}, condition: "and", expect: false},
{name: "or present", words: []string{"delta", "beta"}, condition: "or", expect: true},
{name: "or absent", words: []string{"delta", "epsilon"}, condition: "or", expect: false},
{name: "empty under and matches vacuously", words: nil, condition: "and", expect: true},
{name: "empty under or matches nothing", words: nil, condition: "or", expect: false},
}
for _, tt := range tests {
t.Run(tt.name, func(t *testing.T) {
if got := checkWords(content, tt.words, tt.condition); got != tt.expect {
t.Errorf("checkWords = %v, want %v", got, tt.expect)
}
})
}
}
func TestCheckRegex(t *testing.T) {
const content = "id=42 name=root"
tests := []struct {
name string
patterns []string
condition string
expect bool
}{
{name: "and all match", patterns: []string{`id=\d+`, `name=\w+`}, condition: "and", expect: true},
{name: "and one fails", patterns: []string{`id=\d+`, `zzz`}, condition: "and", expect: false},
{name: "or first matches", patterns: []string{`id=\d+`, `zzz`}, condition: "or", expect: true},
{name: "or none match", patterns: []string{`xxx`, `zzz`}, condition: "or", expect: false},
{name: "and bad regex fails closed", patterns: []string{`(`}, condition: "and", expect: false},
{name: "or bad regex skipped then match", patterns: []string{`(`, `name=\w+`}, condition: "or", expect: true},
}
for _, tt := range tests {
t.Run(tt.name, func(t *testing.T) {
if got := checkRegex(content, tt.patterns, tt.condition); got != tt.expect {
t.Errorf("checkRegex = %v, want %v", got, tt.expect)
}
})
}
}
func TestGetPart(t *testing.T) {
header := http.Header{"Server": []string{"nginx"}}
resp := fakeResponse(t, 200, header)
const body = "page body"
if got := getPart("body", resp, body); got != body {
t.Errorf("getPart body = %q, want %q", got, body)
}
headerPart := getPart("header", resp, body)
if !strings.Contains(headerPart, "Server") || !strings.Contains(headerPart, "nginx") {
t.Errorf("getPart header = %q, want it to include the header", headerPart)
}
if strings.Contains(headerPart, body) {
t.Errorf("getPart header should not include body, got %q", headerPart)
}
all := getPart("all", resp, body)
if !strings.Contains(all, "nginx") || !strings.Contains(all, body) {
t.Errorf("getPart all = %q, want both header and body", all)
}
// an unrecognised part falls back to the body.
if got := getPart("weird", resp, body); got != body {
t.Errorf("getPart fallback = %q, want body %q", got, body)
}
// empty part behaves like "all".
if got := getPart("", resp, body); !strings.Contains(got, "nginx") || !strings.Contains(got, body) {
t.Errorf("getPart empty = %q, want both header and body", got)
}
}
func TestRunExtractors(t *testing.T) {
resp := fakeResponse(t, 200, http.Header{"X-Token": []string{"abc123"}})
const body = `{"session":"sess-7788","role":"admin"}`
tests := []struct {
name string
extractors []Extractor
wantKey string
wantVal string
wantNil bool
}{
{
name: "no extractors yields nil",
extractors: nil,
wantNil: true,
},
{
name: "regex capture group on body",
extractors: []Extractor{
{Type: "regex", Name: "session", Part: "body", Regex: []string{`"session":"([^"]+)"`}, Group: 1},
},
wantKey: "session",
wantVal: "sess-7788",
},
{
name: "group zero is the whole match",
extractors: []Extractor{
{Type: "regex", Name: "role", Part: "body", Regex: []string{`role":"admin`}, Group: 0},
},
wantKey: "role",
wantVal: `role":"admin`,
},
{
name: "extract from header part",
extractors: []Extractor{
{Type: "regex", Name: "token", Part: "header", Regex: []string{`X-Token: (\S+)`}, Group: 1},
},
wantKey: "token",
wantVal: "abc123",
},
{
name: "first matching pattern wins",
extractors: []Extractor{
{Type: "regex", Name: "session", Part: "body", Regex: []string{`nomatch(\d+)`, `"session":"([^"]+)"`}, Group: 1},
},
wantKey: "session",
wantVal: "sess-7788",
},
{
name: "group index out of range is skipped",
extractors: []Extractor{
{Type: "regex", Name: "session", Part: "body", Regex: []string{`"session":"([^"]+)"`}, Group: 5},
},
wantNil: true,
},
{
name: "invalid pattern is skipped, no capture",
extractors: []Extractor{
{Type: "regex", Name: "session", Part: "body", Regex: []string{`(`}, Group: 1},
},
wantNil: true,
},
{
name: "non-regex extractor type is ignored",
extractors: []Extractor{
{Type: "kval", Name: "session", Part: "body", Regex: []string{`"session":"([^"]+)"`}, Group: 1},
},
wantNil: true,
},
}
for _, tt := range tests {
t.Run(tt.name, func(t *testing.T) {
got := runExtractors(tt.extractors, resp, body)
if tt.wantNil {
if len(got) != 0 {
t.Errorf("runExtractors = %v, want empty", got)
}
return
}
if got[tt.wantKey] != tt.wantVal {
t.Errorf("runExtractors[%q] = %q, want %q", tt.wantKey, got[tt.wantKey], tt.wantVal)
}
})
}
}
func TestSubstituteVariables(t *testing.T) {
tests := []struct {
name string
template string
baseURL string
payload string
want string
}{
{
name: "baseurl both cases",
template: "{{BaseURL}}/x and {{baseurl}}/y",
baseURL: "http://h",
want: "http://h/x and http://h/y",
},
{
name: "payload both cases",
template: "q={{payload}}&r={{Payload}}",
payload: "<script>",
want: "q=<script>&r=<script>",
},
{
name: "combined base and payload",
template: "{{BaseURL}}/search?q={{payload}}",
baseURL: "http://h",
payload: "x",
want: "http://h/search?q=x",
},
{
name: "no placeholders untouched",
template: "/static/path",
baseURL: "http://h",
want: "/static/path",
},
}
for _, tt := range tests {
t.Run(tt.name, func(t *testing.T) {
if got := substituteVariables(tt.template, tt.baseURL, tt.payload); got != tt.want {
t.Errorf("substituteVariables = %q, want %q", got, tt.want)
}
})
}
}
func TestGenerateHTTPRequests(t *testing.T) {
t.Run("paths without payloads", func(t *testing.T) {
cfg := &HTTPConfig{
Paths: []string{"{{BaseURL}}/a", "{{BaseURL}}/b"},
}
// trailing slash on the target must be trimmed before substitution.
got := generateHTTPRequests("http://h/", cfg)
if len(got) != 2 {
t.Fatalf("got %d requests, want 2", len(got))
}
if got[0].Method != "GET" {
t.Errorf("default method = %q, want GET", got[0].Method)
}
if got[0].URL != "http://h/a" || got[1].URL != "http://h/b" {
t.Errorf("urls = %q,%q", got[0].URL, got[1].URL)
}
})
t.Run("payload expansion is path x payload", func(t *testing.T) {
cfg := &HTTPConfig{
Method: "POST",
Paths: []string{"{{BaseURL}}/q?x={{payload}}"},
Payloads: []string{"1", "2", "3"},
Body: "data={{payload}}",
}
got := generateHTTPRequests("http://h", cfg)
if len(got) != 3 {
t.Fatalf("got %d requests, want 3", len(got))
}
for i, want := range []string{"1", "2", "3"} {
if got[i].Payload != want {
t.Errorf("req %d payload = %q, want %q", i, got[i].Payload, want)
}
if got[i].URL != "http://h/q?x="+want {
t.Errorf("req %d url = %q", i, got[i].URL)
}
if got[i].Body != "data="+want {
t.Errorf("req %d body = %q", i, got[i].Body)
}
if got[i].Method != "POST" {
t.Errorf("req %d method = %q, want POST", i, got[i].Method)
}
}
})
t.Run("multiple paths times multiple payloads", func(t *testing.T) {
cfg := &HTTPConfig{
Paths: []string{"{{BaseURL}}/a", "{{BaseURL}}/b"},
Payloads: []string{"x", "y"},
}
got := generateHTTPRequests("http://h", cfg)
if len(got) != 4 {
t.Fatalf("got %d requests, want 4 (2 paths x 2 payloads)", len(got))
}
})
}
+119
View File
@@ -0,0 +1,119 @@
/*
·━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━·
: :
: █▀ █ █▀▀ · Blazing-fast pentesting suite :
: ▄█ █ █▀ · BSD 3-Clause License :
: :
: (c) 2022-2026 vmfunc, xyzeva, :
: lunchcat alumni & contributors :
: :
·━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━·
*/
package notify
import (
"fmt"
"os"
"gopkg.in/yaml.v3"
)
// env var names notify reads, env-first. these mirror the conventional names so
// an operator who already exports them for other tooling gets notify for free.
const (
envSlackWebhook = "SLACK_WEBHOOK_URL"
envDiscordWebhook = "DISCORD_WEBHOOK_URL"
// the name of the env var holding the bot token, not the token itself.
envTelegramToken = "TELEGRAM_BOT_TOKEN" //nolint:gosec // env var name, not a secret
envTelegramChat = "TELEGRAM_CHAT_ID"
envWebhookURL = "NOTIFY_WEBHOOK_URL"
)
// config holds resolved destinations for every provider. yaml tags use
// projectdiscovery/notify-compatible key names so an existing notify config file
// ports over verbatim; env supplies the same values and yaml overrides it.
type config struct {
SlackWebhook string `yaml:"slack_webhook_url"`
DiscordWebhook string `yaml:"discord_webhook_url"`
// telegram needs both a bot token and a chat id. notify spells the token
// "telegram_api_key", so accept that key for drop-in compatibility.
TelegramToken string `yaml:"telegram_api_key"`
TelegramChat string `yaml:"telegram_chat_id"`
WebhookURL string `yaml:"webhook_url"`
}
// loadConfig resolves notify destinations env-first, then overlays a yaml file
// when path is non-empty. yaml wins per-field so a file value overrides the
// matching env var; an unset yaml field leaves the env value intact. an empty
// path means env-only. a missing/unparseable file is an error - if the operator
// pointed -notify-config somewhere, a typo should fail loud, not silently drop.
func loadConfig(path string) (config, error) {
cfg := config{
SlackWebhook: os.Getenv(envSlackWebhook),
DiscordWebhook: os.Getenv(envDiscordWebhook),
TelegramToken: os.Getenv(envTelegramToken),
TelegramChat: os.Getenv(envTelegramChat),
WebhookURL: os.Getenv(envWebhookURL),
}
if path == "" {
return cfg, nil
}
data, err := os.ReadFile(path)
if err != nil {
return config{}, fmt.Errorf("read config %q: %w", path, err)
}
// decode into a separate value so only the keys present in the file overlay
// the env-derived defaults; a zero field in the yaml must not blank an env var.
var file config
if err := yaml.Unmarshal(data, &file); err != nil {
return config{}, fmt.Errorf("parse config %q: %w", path, err)
}
overlay(&cfg, &file)
return cfg, nil
}
// overlay copies non-empty fields from src onto dst. used to let a yaml file
// override env without an empty yaml key wiping out a populated env value.
func overlay(dst, src *config) {
if src.SlackWebhook != "" {
dst.SlackWebhook = src.SlackWebhook
}
if src.DiscordWebhook != "" {
dst.DiscordWebhook = src.DiscordWebhook
}
if src.TelegramToken != "" {
dst.TelegramToken = src.TelegramToken
}
if src.TelegramChat != "" {
dst.TelegramChat = src.TelegramChat
}
if src.WebhookURL != "" {
dst.WebhookURL = src.WebhookURL
}
}
// providers builds the live provider list from the resolved config: a provider
// is included only when its destination is fully specified. telegram needs both
// token and chat id, so a half-configured telegram is dropped rather than POSTing
// to a broken endpoint.
func (c *config) providers() []provider {
var out []provider
if c.SlackWebhook != "" {
out = append(out, &slackProvider{webhook: c.SlackWebhook})
}
if c.DiscordWebhook != "" {
out = append(out, &discordProvider{webhook: c.DiscordWebhook})
}
if c.TelegramToken != "" && c.TelegramChat != "" {
out = append(out, &telegramProvider{token: c.TelegramToken, chatID: c.TelegramChat})
}
if c.WebhookURL != "" {
out = append(out, &webhookProvider{url: c.WebhookURL})
}
return out
}
+153
View File
@@ -0,0 +1,153 @@
/*
·━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━·
: :
: █▀ █ █▀▀ · Blazing-fast pentesting suite :
: ▄█ █ █▀ · BSD 3-Clause License :
: :
: (c) 2022-2026 vmfunc, xyzeva, :
: lunchcat alumni & contributors :
: :
·━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━·
*/
package notify
import (
"os"
"path/filepath"
"testing"
)
// clearNotifyEnv unsets every var loadConfig reads so a test starts from a known
// blank slate; t.Setenv("", "") still records the var for cleanup restoration.
func clearNotifyEnv(t *testing.T) {
t.Helper()
for _, k := range []string{
envSlackWebhook, envDiscordWebhook,
envTelegramToken, envTelegramChat, envWebhookURL,
} {
t.Setenv(k, "")
}
}
func TestLoadConfigEnvOnly(t *testing.T) {
clearNotifyEnv(t)
t.Setenv(envSlackWebhook, "https://hooks.slack.test/a")
t.Setenv(envTelegramToken, "123:abc")
t.Setenv(envTelegramChat, "999")
cfg, err := loadConfig("")
if err != nil {
t.Fatalf("loadConfig: %v", err)
}
if cfg.SlackWebhook != "https://hooks.slack.test/a" {
t.Errorf("slack webhook = %q, want from env", cfg.SlackWebhook)
}
if cfg.TelegramToken != "123:abc" || cfg.TelegramChat != "999" {
t.Errorf("telegram = %q/%q, want from env", cfg.TelegramToken, cfg.TelegramChat)
}
// slack + telegram (both halves) configured, discord/webhook empty.
got := cfg.providers()
if len(got) != 2 {
t.Fatalf("providers = %d, want 2 (slack, telegram)", len(got))
}
wantNames := map[string]bool{"slack": false, "telegram": false}
for _, p := range got {
wantNames[p.name()] = true
}
for name, seen := range wantNames {
if !seen {
t.Errorf("provider %q missing", name)
}
}
}
func TestLoadConfigYAMLOverridesEnv(t *testing.T) {
clearNotifyEnv(t)
t.Setenv(envSlackWebhook, "https://env.slack.test/x")
t.Setenv(envWebhookURL, "https://env.webhook.test/x")
body := "" +
"slack_webhook_url: https://file.slack.test/y\n" +
"discord_webhook_url: https://file.discord.test/z\n"
path := writeTempConfig(t, body)
cfg, err := loadConfig(path)
if err != nil {
t.Fatalf("loadConfig: %v", err)
}
// yaml present -> overrides env.
if cfg.SlackWebhook != "https://file.slack.test/y" {
t.Errorf("slack = %q, want yaml override", cfg.SlackWebhook)
}
// yaml absent for webhook -> env value survives.
if cfg.WebhookURL != "https://env.webhook.test/x" {
t.Errorf("webhook = %q, want env value preserved", cfg.WebhookURL)
}
// yaml introduces discord.
if cfg.DiscordWebhook != "https://file.discord.test/z" {
t.Errorf("discord = %q, want from yaml", cfg.DiscordWebhook)
}
}
func TestLoadConfigNotifyCompatibleTelegramKey(t *testing.T) {
clearNotifyEnv(t)
// projectdiscovery/notify spells the bot token "telegram_api_key"; assert a
// drop-in config wires telegram from that key.
body := "" +
"telegram_api_key: 555:tok\n" +
"telegram_chat_id: \"42\"\n"
path := writeTempConfig(t, body)
cfg, err := loadConfig(path)
if err != nil {
t.Fatalf("loadConfig: %v", err)
}
if cfg.TelegramToken != "555:tok" || cfg.TelegramChat != "42" {
t.Fatalf("telegram = %q/%q, want from notify-compatible keys", cfg.TelegramToken, cfg.TelegramChat)
}
if len(cfg.providers()) != 1 {
t.Fatalf("providers = %d, want 1 (telegram)", len(cfg.providers()))
}
}
func TestLoadConfigMissingFileErrors(t *testing.T) {
clearNotifyEnv(t)
if _, err := loadConfig(filepath.Join(t.TempDir(), "nope.yaml")); err == nil {
t.Fatal("loadConfig with missing file: want error, got nil")
}
}
func TestLoadConfigBadYAMLErrors(t *testing.T) {
clearNotifyEnv(t)
path := writeTempConfig(t, "slack_webhook_url: [unterminated\n")
if _, err := loadConfig(path); err == nil {
t.Fatal("loadConfig with malformed yaml: want error, got nil")
}
}
func TestProvidersTelegramNeedsBothHalves(t *testing.T) {
// token without chat id must not produce a (broken) telegram provider.
cfg := config{TelegramToken: "tok"}
if got := cfg.providers(); len(got) != 0 {
t.Fatalf("providers = %d, want 0 for half-configured telegram", len(got))
}
}
func TestProvidersEmptyConfigIsNone(t *testing.T) {
var cfg config
if got := cfg.providers(); len(got) != 0 {
t.Fatalf("providers = %d, want 0 for empty config", len(got))
}
}
// writeTempConfig writes body to a temp yaml file and returns its path.
func writeTempConfig(t *testing.T, body string) string {
t.Helper()
path := filepath.Join(t.TempDir(), "notify.yaml")
if err := os.WriteFile(path, []byte(body), 0o600); err != nil {
t.Fatalf("write temp config: %v", err)
}
return path
}
+39
View File
@@ -0,0 +1,39 @@
/*
·━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━·
: :
: █▀ █ █▀▀ · Blazing-fast pentesting suite :
: ▄█ █ █▀ · BSD 3-Clause License :
: :
: (c) 2022-2026 vmfunc, xyzeva, :
: lunchcat alumni & contributors :
: :
·━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━·
*/
package notify
import (
"context"
"net/http"
"github.com/dropalldatabases/sif/internal/finding"
)
// discordProvider posts to a discord webhook. discord's incoming-webhook body
// keys the message on "content" (slack uses "text"); same code-block wrapping so
// the finding columns line up in the channel.
type discordProvider struct {
webhook string
}
func (d *discordProvider) name() string { return "discord" }
// discordPayload is the minimal webhook body: a single content field.
type discordPayload struct {
Content string `json:"content"`
}
func (d *discordProvider) send(ctx context.Context, client *http.Client, findings []finding.Finding) error {
payload := discordPayload{Content: codeBlock(renderFindings(findings))}
return postJSON(ctx, client, d.webhook, payload)
}
+74
View File
@@ -0,0 +1,74 @@
/*
·━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━·
: :
: █▀ █ █▀▀ · Blazing-fast pentesting suite :
: ▄█ █ █▀ · BSD 3-Clause License :
: :
: (c) 2022-2026 vmfunc, xyzeva, :
: lunchcat alumni & contributors :
: :
·━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━·
*/
package notify
import (
"bytes"
"context"
"encoding/json"
"fmt"
"net/http"
"strings"
"github.com/dropalldatabases/sif/internal/finding"
"github.com/dropalldatabases/sif/internal/httpx"
)
// contentTypeJSON is the body type every provider POSTs; all four speak json.
const contentTypeJSON = "application/json"
// messageHeader prefixes the rendered finding block. kept terse - chat sinks
// truncate, so the count and lead-in carry the signal.
const messageHeader = "sif found %d finding(s):"
// renderFindings turns a batch into a single plain-text block, one finding per
// line in the same "[severity] target module title" shape as the -silent sink so
// a reader sees identical lines across stdout and chat. a strings.Builder keeps
// the per-line concat to one allocation path.
func renderFindings(findings []finding.Finding) string {
var b strings.Builder
fmt.Fprintf(&b, messageHeader, len(findings))
b.WriteByte('\n')
for i := 0; i < len(findings); i++ {
b.WriteString(findings[i].Line())
b.WriteByte('\n')
}
return b.String()
}
// postJSON marshals payload and POSTs it to url through the shared client. it
// drains+closes the response so the conn returns to httpx's pool, and treats any
// non-2xx as a delivery failure so a 4xx from a bad webhook surfaces loudly.
func postJSON(ctx context.Context, client *http.Client, url string, payload any) error {
body, err := json.Marshal(payload)
if err != nil {
return fmt.Errorf("marshal payload: %w", err)
}
req, err := http.NewRequestWithContext(ctx, http.MethodPost, url, bytes.NewReader(body))
if err != nil {
return fmt.Errorf("build request: %w", err)
}
req.Header.Set("Content-Type", contentTypeJSON)
resp, err := client.Do(req) //nolint:bodyclose // drained and closed via httpx.DrainClose
if err != nil {
return fmt.Errorf("post: %w", err)
}
defer httpx.DrainClose(resp)
if resp.StatusCode < http.StatusOK || resp.StatusCode >= http.StatusMultipleChoices {
return fmt.Errorf("unexpected status %d", resp.StatusCode)
}
return nil
}
+85
View File
@@ -0,0 +1,85 @@
/*
·━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━·
: :
: █▀ █ █▀▀ · Blazing-fast pentesting suite :
: ▄█ █ █▀ · BSD 3-Clause License :
: :
: (c) 2022-2026 vmfunc, xyzeva, :
: lunchcat alumni & contributors :
: :
·━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━·
*/
// Package notify ships findings to chat/webhook sinks (slack, discord, telegram,
// generic webhook) so a continuous-recon run can alert on what it turns up. every
// provider is one POST through httpx.Client, so the global proxy/rate-limit/header
// config applies uniformly and there's no extra http stack to keep in sync.
package notify
import (
"context"
"fmt"
"net/http"
"time"
"github.com/dropalldatabases/sif/internal/finding"
"github.com/dropalldatabases/sif/internal/httpx"
"github.com/dropalldatabases/sif/internal/output"
)
// Options carries the runtime knobs Send needs. Timeout bounds each provider's
// POST; ConfigPath is an optional yaml file whose values override env. severity
// filtering is the caller's job - Send ships whatever batch it's handed.
type Options struct {
Timeout time.Duration
ConfigPath string
}
// Send dispatches findings to every configured provider. config resolves
// env-first, then a yaml file overlays it (notify-compatible key names). a
// provider with no destination is skipped, so zero configured providers makes
// Send a silent no-op - notify is opt-in and never errors just for being unwired.
// an empty findings slice is also a no-op: nothing to report.
func Send(ctx context.Context, findings []finding.Finding, opts Options) error {
if len(findings) == 0 {
return nil
}
cfg, err := loadConfig(opts.ConfigPath)
if err != nil {
return fmt.Errorf("notify config: %w", err)
}
providers := cfg.providers()
if len(providers) == 0 {
// nothing wired up; opt-in feature stays quiet rather than erroring.
return nil
}
log := output.Module("NOTIFY")
client := httpx.Client(opts.Timeout)
// run every provider; a failure on one sink must not suppress the others, so
// errors accumulate and the first is returned after all have been attempted.
var firstErr error
for i := 0; i < len(providers); i++ {
p := providers[i]
if err := p.send(ctx, client, findings); err != nil {
log.Error("%s delivery failed: %v", p.name(), err)
if firstErr == nil {
firstErr = fmt.Errorf("%s: %w", p.name(), err)
}
continue
}
log.Success("sent %d findings to %s", len(findings), p.name())
}
return firstErr
}
// provider is one delivery sink. name is for logging; send formats findings into
// the sink's payload and POSTs it through the shared client.
type provider interface {
name() string
send(ctx context.Context, client *http.Client, findings []finding.Finding) error
}
+224
View File
@@ -0,0 +1,224 @@
/*
·━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━·
: :
: █▀ █ █▀▀ · Blazing-fast pentesting suite :
: ▄█ █ █▀ · BSD 3-Clause License :
: :
: (c) 2022-2026 vmfunc, xyzeva, :
: lunchcat alumni & contributors :
: :
·━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━·
*/
package notify
import (
"context"
"encoding/json"
"io"
"net/http"
"net/http/httptest"
"strings"
"testing"
"time"
"github.com/dropalldatabases/sif/internal/finding"
)
// sampleFindings returns a small mixed-severity batch for payload assertions.
func sampleFindings() []finding.Finding {
return []finding.Finding{
{Target: "https://a.test", Module: "cors", Severity: finding.SeverityHigh, Key: "cors:a", Title: "reflected origin", Raw: "ACAO echo"},
{Target: "https://a.test", Module: "headers", Severity: finding.SeverityInfo, Key: "headers:x", Title: "Server header", Raw: "nginx"},
}
}
// capture records the method, content-type and raw body of the request a provider
// makes, so each test can assert the wire shape without a real network.
type capture struct {
method string
contentType string
path string
body []byte
}
// captureServer stands up an httptest server that records the single inbound
// request into c and replies 200, the happy path every provider expects.
func captureServer(t *testing.T, c *capture) *httptest.Server {
t.Helper()
srv := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
body, _ := io.ReadAll(r.Body)
c.method = r.Method
c.contentType = r.Header.Get("Content-Type")
c.path = r.URL.Path
c.body = body
w.WriteHeader(http.StatusOK)
}))
t.Cleanup(srv.Close)
return srv
}
func TestSlackPayloadShape(t *testing.T) {
var c capture
srv := captureServer(t, &c)
p := &slackProvider{webhook: srv.URL}
if err := p.send(context.Background(), srv.Client(), sampleFindings()); err != nil {
t.Fatalf("slack send: %v", err)
}
assertPostJSON(t, c)
var payload slackPayload
if err := json.Unmarshal(c.body, &payload); err != nil {
t.Fatalf("unmarshal slack body: %v", err)
}
// slack keys on "text"; both findings must appear, code-block fenced.
if !strings.Contains(payload.Text, "reflected origin") || !strings.Contains(payload.Text, "Server header") {
t.Errorf("slack text missing findings: %q", payload.Text)
}
if !strings.HasPrefix(payload.Text, "```") {
t.Errorf("slack text not code-block fenced: %q", payload.Text)
}
}
func TestDiscordPayloadShape(t *testing.T) {
var c capture
srv := captureServer(t, &c)
p := &discordProvider{webhook: srv.URL}
if err := p.send(context.Background(), srv.Client(), sampleFindings()); err != nil {
t.Fatalf("discord send: %v", err)
}
assertPostJSON(t, c)
var payload discordPayload
if err := json.Unmarshal(c.body, &payload); err != nil {
t.Fatalf("unmarshal discord body: %v", err)
}
// discord keys on "content", not "text".
if !strings.Contains(payload.Content, "reflected origin") {
t.Errorf("discord content missing finding: %q", payload.Content)
}
}
func TestTelegramPayloadShape(t *testing.T) {
var c capture
srv := captureServer(t, &c)
// repoint the bot api base at the test server for the lifetime of this test.
orig := telegramAPIBase
telegramAPIBase = srv.URL
t.Cleanup(func() { telegramAPIBase = orig })
p := &telegramProvider{token: "555:tok", chatID: "42"}
if err := p.send(context.Background(), srv.Client(), sampleFindings()); err != nil {
t.Fatalf("telegram send: %v", err)
}
assertPostJSON(t, c)
// the token rides the path and the method is sendMessage.
if c.path != "/bot555:tok/sendMessage" {
t.Errorf("telegram path = %q, want /bot555:tok/sendMessage", c.path)
}
var payload telegramPayload
if err := json.Unmarshal(c.body, &payload); err != nil {
t.Fatalf("unmarshal telegram body: %v", err)
}
if payload.ChatID != "42" {
t.Errorf("telegram chat_id = %q, want 42", payload.ChatID)
}
if !strings.Contains(payload.Text, "reflected origin") {
t.Errorf("telegram text missing finding: %q", payload.Text)
}
}
func TestWebhookPayloadShape(t *testing.T) {
var c capture
srv := captureServer(t, &c)
p := &webhookProvider{url: srv.URL}
if err := p.send(context.Background(), srv.Client(), sampleFindings()); err != nil {
t.Fatalf("webhook send: %v", err)
}
assertPostJSON(t, c)
var payload webhookPayload
if err := json.Unmarshal(c.body, &payload); err != nil {
t.Fatalf("unmarshal webhook body: %v", err)
}
// generic webhook carries structured findings, not a prerendered blob.
if payload.Count != 2 || len(payload.Findings) != 2 {
t.Fatalf("webhook count = %d / %d findings, want 2", payload.Count, len(payload.Findings))
}
first := payload.Findings[0]
if first.Severity != "high" {
t.Errorf("webhook severity = %q, want canonical string \"high\"", first.Severity)
}
if first.Key != "cors:a" || first.Module != "cors" {
t.Errorf("webhook finding fields wrong: %+v", first)
}
}
func TestProviderNon2xxIsError(t *testing.T) {
srv := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, _ *http.Request) {
w.WriteHeader(http.StatusForbidden)
}))
t.Cleanup(srv.Close)
p := &slackProvider{webhook: srv.URL}
if err := p.send(context.Background(), srv.Client(), sampleFindings()); err == nil {
t.Fatal("send to 403 endpoint: want error, got nil")
}
}
func TestSendNoProviderIsNoop(t *testing.T) {
clearNotifyEnv(t)
// no env, no config file -> zero providers -> Send must not error.
if err := Send(context.Background(), sampleFindings(), Options{Timeout: time.Second}); err != nil {
t.Fatalf("Send with no provider: want nil, got %v", err)
}
}
func TestSendEmptyFindingsIsNoop(t *testing.T) {
// even with a provider configured, an empty batch must not POST anything.
hit := false
srv := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, _ *http.Request) {
hit = true
w.WriteHeader(http.StatusOK)
}))
t.Cleanup(srv.Close)
clearNotifyEnv(t)
t.Setenv(envSlackWebhook, srv.URL)
if err := Send(context.Background(), nil, Options{Timeout: time.Second}); err != nil {
t.Fatalf("Send with empty findings: want nil, got %v", err)
}
if hit {
t.Fatal("Send with empty findings posted to provider, want no-op")
}
}
func TestSendDeliversToConfiguredProvider(t *testing.T) {
var c capture
srv := captureServer(t, &c)
clearNotifyEnv(t)
t.Setenv(envSlackWebhook, srv.URL)
if err := Send(context.Background(), sampleFindings(), Options{Timeout: time.Second}); err != nil {
t.Fatalf("Send: %v", err)
}
if c.method != http.MethodPost {
t.Fatalf("provider not hit (method=%q)", c.method)
}
}
// assertPostJSON checks the request was a json POST.
func assertPostJSON(t *testing.T, c capture) {
t.Helper()
if c.method != http.MethodPost {
t.Errorf("method = %q, want POST", c.method)
}
if c.contentType != contentTypeJSON {
t.Errorf("content-type = %q, want %q", c.contentType, contentTypeJSON)
}
}
+45
View File
@@ -0,0 +1,45 @@
/*
·━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━·
: :
: █▀ █ █▀▀ · Blazing-fast pentesting suite :
: ▄█ █ █▀ · BSD 3-Clause License :
: :
: (c) 2022-2026 vmfunc, xyzeva, :
: lunchcat alumni & contributors :
: :
·━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━·
*/
package notify
import (
"context"
"net/http"
"github.com/dropalldatabases/sif/internal/finding"
)
// slackProvider posts to a slack incoming webhook. the webhook url already pins
// the channel, so the payload is just the rendered text in slack's mrkdwn-aware
// "text" field wrapped in a code block to keep the fixed-width finding lines.
type slackProvider struct {
webhook string
}
func (s *slackProvider) name() string { return "slack" }
// slackPayload is the minimal incoming-webhook body: a single text field.
type slackPayload struct {
Text string `json:"text"`
}
func (s *slackProvider) send(ctx context.Context, client *http.Client, findings []finding.Finding) error {
payload := slackPayload{Text: codeBlock(renderFindings(findings))}
return postJSON(ctx, client, s.webhook, payload)
}
// codeBlock wraps body in a triple-backtick fence; both slack and discord render
// it fixed-width, which preserves the column-aligned finding lines.
func codeBlock(body string) string {
return "```\n" + body + "```"
}
+48
View File
@@ -0,0 +1,48 @@
/*
·━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━·
: :
: █▀ █ █▀▀ · Blazing-fast pentesting suite :
: ▄█ █ █▀ · BSD 3-Clause License :
: :
: (c) 2022-2026 vmfunc, xyzeva, :
: lunchcat alumni & contributors :
: :
·━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━·
*/
package notify
import (
"context"
"net/http"
"github.com/dropalldatabases/sif/internal/finding"
)
// telegramAPIBase is the bot api root. it's a var so tests can repoint it at an
// httptest server; the token is appended path-side per telegram's scheme.
var telegramAPIBase = "https://api.telegram.org"
// telegramProvider posts via the bot api's sendMessage. unlike slack/discord the
// destination isn't a single opaque webhook: it needs the bot token (in the url
// path) plus the chat id (in the body).
type telegramProvider struct {
token string
chatID string
}
func (t *telegramProvider) name() string { return "telegram" }
// telegramPayload is the sendMessage body. parse_mode "MarkdownV2" would force
// escaping every special char in the finding lines, so we send plain text and
// let the lines stand as-is.
type telegramPayload struct {
ChatID string `json:"chat_id"`
Text string `json:"text"`
}
func (t *telegramProvider) send(ctx context.Context, client *http.Client, findings []finding.Finding) error {
endpoint := telegramAPIBase + "/bot" + t.token + "/sendMessage"
payload := telegramPayload{ChatID: t.chatID, Text: renderFindings(findings)}
return postJSON(ctx, client, endpoint, payload)
}
+65
View File
@@ -0,0 +1,65 @@
/*
·━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━·
: :
: █▀ █ █▀▀ · Blazing-fast pentesting suite :
: ▄█ █ █▀ · BSD 3-Clause License :
: :
: (c) 2022-2026 vmfunc, xyzeva, :
: lunchcat alumni & contributors :
: :
·━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━·
*/
package notify
import (
"context"
"net/http"
"github.com/dropalldatabases/sif/internal/finding"
)
// webhookProvider posts a structured json payload to an arbitrary endpoint. unlike
// the chat sinks it carries the findings as data, not a prerendered blob, so
// downstream automation (a siem, a bot, ci) keys off the fields directly.
type webhookProvider struct {
url string
}
func (w *webhookProvider) name() string { return "webhook" }
// webhookFinding is the per-item wire shape: the normalized Finding fields with
// severity flattened to its canonical string so a json consumer never sees the
// internal integer rank.
type webhookFinding struct {
Target string `json:"target"`
Module string `json:"module"`
Severity string `json:"severity"`
Key string `json:"key"`
Title string `json:"title"`
Raw string `json:"raw,omitempty"`
}
// webhookPayload wraps the batch with a count so a consumer can size buffers /
// assert completeness without walking the slice first.
type webhookPayload struct {
Count int `json:"count"`
Findings []webhookFinding `json:"findings"`
}
func (w *webhookProvider) send(ctx context.Context, client *http.Client, findings []finding.Finding) error {
items := make([]webhookFinding, 0, len(findings))
for i := 0; i < len(findings); i++ {
f := findings[i]
items = append(items, webhookFinding{
Target: f.Target,
Module: f.Module,
Severity: f.Severity.String(),
Key: f.Key,
Title: f.Title,
Raw: f.Raw,
})
}
payload := webhookPayload{Count: len(items), Findings: items}
return postJSON(ctx, client, w.url, payload)
}
+61 -25
View File
@@ -14,6 +14,7 @@ package output
import (
"fmt"
"io"
"os"
"strings"
@@ -126,13 +127,47 @@ func SetAPIMode(enabled bool) {
apiMode = enabled
}
// sink is where all banner/spinner/log chrome is written. it defaults to stdout
// so normal runs are unchanged; -silent repoints it at stderr so stdout carries
// nothing but the machine-readable findings a downstream pipe consumes.
var sink io.Writer = os.Stdout
// silent is the plain-sink mode: chrome goes to stderr and interactive widgets
// (spinners, live progress) are suppressed so a piped consumer never sees them.
var silent bool
// SetSilent routes all chrome to stderr and marks the run non-interactive.
// findings are printed to stdout by the caller via Finding/PrintFinding; the
// output package itself never touches stdout once silent is on.
func SetSilent(enabled bool) {
silent = enabled
if enabled {
sink = os.Stderr
return
}
sink = os.Stdout
}
// Silent reports whether plain-sink mode is active. callers gate interactive
// behaviour (spinners, prompts) on this.
func Silent() bool {
return silent
}
// Writer is the current chrome sink (stdout normally, stderr under -silent).
// callers that render their own chrome (the startup banner) write here so it
// follows the same routing as everything else.
func Writer() io.Writer {
return sink
}
// Info prints an informational message with [*] prefix
func Info(format string, args ...interface{}) {
if apiMode {
return
}
msg := fmt.Sprintf(format, args...)
fmt.Printf("%s %s\n", prefixInfo.Render("[*]"), msg)
fmt.Fprintf(sink, "%s %s\n", prefixInfo.Render("[*]"), msg)
}
// Success prints a success message with [+] prefix
@@ -141,7 +176,7 @@ func Success(format string, args ...interface{}) {
return
}
msg := fmt.Sprintf(format, args...)
fmt.Printf("%s %s\n", prefixSuccess.Render("[+]"), msg)
fmt.Fprintf(sink, "%s %s\n", prefixSuccess.Render("[+]"), msg)
}
// Warn prints a warning message with [!] prefix
@@ -150,7 +185,7 @@ func Warn(format string, args ...interface{}) {
return
}
msg := fmt.Sprintf(format, args...)
fmt.Printf("%s %s\n", prefixWarning.Render("[!]"), msg)
fmt.Fprintf(sink, "%s %s\n", prefixWarning.Render("[!]"), msg)
}
// Error prints an error message with [-] prefix
@@ -159,7 +194,7 @@ func Error(format string, args ...interface{}) {
return
}
msg := fmt.Sprintf(format, args...)
fmt.Printf("%s %s\n", prefixError.Render("[-]"), msg)
fmt.Fprintf(sink, "%s %s\n", prefixError.Render("[-]"), msg)
}
// ScanStart prints a styled scan start message
@@ -167,7 +202,7 @@ func ScanStart(scanName string) {
if apiMode {
return
}
fmt.Printf("%s starting %s\n", prefixInfo.Render("[*]"), scanName)
fmt.Fprintf(sink, "%s starting %s\n", prefixInfo.Render("[*]"), scanName)
}
// ScanComplete prints a styled scan completion message
@@ -175,7 +210,7 @@ func ScanComplete(scanName string, resultCount int, resultType string) {
if apiMode {
return
}
fmt.Printf("%s %s complete (%d %s)\n", prefixInfo.Render("[*]"), scanName, resultCount, resultType)
fmt.Fprintf(sink, "%s %s complete (%d %s)\n", prefixInfo.Render("[*]"), scanName, resultCount, resultType)
}
// Module creates a prefixed logger for a specific module/tool
@@ -202,7 +237,7 @@ func (m *ModuleLogger) Info(format string, args ...interface{}) {
return
}
msg := fmt.Sprintf(format, args...)
fmt.Printf("%s %s\n", m.prefix(), msg)
fmt.Fprintf(sink, "%s %s\n", m.prefix(), msg)
}
// Success prints a success message with module prefix
@@ -211,7 +246,7 @@ func (m *ModuleLogger) Success(format string, args ...interface{}) {
return
}
msg := fmt.Sprintf(format, args...)
fmt.Printf("%s %s %s\n", m.prefix(), prefixSuccess.Render("✓"), msg)
fmt.Fprintf(sink, "%s %s %s\n", m.prefix(), prefixSuccess.Render("✓"), msg)
}
// Warn prints a warning message with module prefix
@@ -220,7 +255,7 @@ func (m *ModuleLogger) Warn(format string, args ...interface{}) {
return
}
msg := fmt.Sprintf(format, args...)
fmt.Printf("%s %s %s\n", m.prefix(), prefixWarning.Render("!"), msg)
fmt.Fprintf(sink, "%s %s %s\n", m.prefix(), prefixWarning.Render("!"), msg)
}
// Error prints an error message with module prefix
@@ -229,7 +264,7 @@ func (m *ModuleLogger) Error(format string, args ...interface{}) {
return
}
msg := fmt.Sprintf(format, args...)
fmt.Printf("%s %s %s\n", m.prefix(), prefixError.Render("✗"), msg)
fmt.Fprintf(sink, "%s %s %s\n", m.prefix(), prefixError.Render("✗"), msg)
}
// Start prints a scan start message with module prefix (adds newline before for separation)
@@ -237,7 +272,7 @@ func (m *ModuleLogger) Start() {
if apiMode {
return
}
fmt.Printf("\n%s starting scan\n", m.prefix())
fmt.Fprintf(sink, "\n%s starting scan\n", m.prefix())
}
// Complete prints a scan complete message with module prefix
@@ -245,15 +280,16 @@ func (m *ModuleLogger) Complete(resultCount int, resultType string) {
if apiMode {
return
}
fmt.Printf("%s complete (%d %s)\n", m.prefix(), resultCount, resultType)
fmt.Fprintf(sink, "%s complete (%d %s)\n", m.prefix(), resultCount, resultType)
}
// ClearLine clears the current line (for progress bar updates)
// ClearLine clears the current line (for progress bar updates). silent mode is
// non-interactive, so there's no live line to clear and stdout stays untouched.
func ClearLine() {
if !IsTTY {
if !IsTTY || silent {
return
}
fmt.Print("\033[2K\r")
fmt.Fprint(sink, "\033[2K\r")
}
// Summary styles
@@ -274,22 +310,22 @@ func PrintSummary(scans []string, logFiles []string) {
return
}
fmt.Println()
fmt.Println(summaryLine.Render("────────────────────────────────────────────────────────────"))
fmt.Println()
fmt.Printf(" %s\n", summaryHeader.Render("SCAN COMPLETE"))
fmt.Println()
fmt.Fprintln(sink)
fmt.Fprintln(sink, summaryLine.Render("────────────────────────────────────────────────────────────"))
fmt.Fprintln(sink)
fmt.Fprintf(sink, " %s\n", summaryHeader.Render("SCAN COMPLETE"))
fmt.Fprintln(sink)
// Print scans
scanList := strings.Join(scans, ", ")
fmt.Printf(" %s %s\n", Muted.Render("Scans:"), scanList)
fmt.Fprintf(sink, " %s %s\n", Muted.Render("Scans:"), scanList)
// Print log files if any
if len(logFiles) > 0 {
fmt.Printf(" %s %s\n", Muted.Render("Output:"), strings.Join(logFiles, ", "))
fmt.Fprintf(sink, " %s %s\n", Muted.Render("Output:"), strings.Join(logFiles, ", "))
}
fmt.Println()
fmt.Println(summaryLine.Render("────────────────────────────────────────────────────────────"))
fmt.Println()
fmt.Fprintln(sink)
fmt.Fprintln(sink, summaryLine.Render("────────────────────────────────────────────────────────────"))
fmt.Fprintln(sink)
}
+3 -3
View File
@@ -98,7 +98,7 @@ func (p *Progress) Done() {
}
func (p *Progress) render() {
if apiMode {
if apiMode || silent {
return
}
@@ -135,7 +135,7 @@ func (p *Progress) render() {
p.mu.Unlock()
if advanced {
fmt.Printf(" [%d%%] %d/%d\n", percent, current, total)
fmt.Fprintf(sink, " [%d%%] %d/%d\n", percent, current, total)
}
return
}
@@ -190,5 +190,5 @@ func (p *Progress) render() {
)
ClearLine()
fmt.Print(line)
fmt.Fprint(sink, line)
}
+113
View File
@@ -0,0 +1,113 @@
/*
·━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━·
: :
: █▀ █ █▀▀ · Blazing-fast pentesting suite :
: ▄█ █ █▀ · BSD 3-Clause License :
: :
: (c) 2022-2026 vmfunc, xyzeva, :
: lunchcat alumni & contributors :
: :
·━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━·
*/
package output
import (
"os"
"strings"
"testing"
)
// in silent mode chrome must land on stderr and leave stdout untouched, so a
// piped consumer downstream never sees a banner or log line.
func TestSetSilentRoutesChromeToStderr(t *testing.T) {
defer SetSilent(false)
outStr, errStr := captureStdoutStderr(t, func() {
// SetSilent reads os.Stderr at call time, so swap then set.
SetSilent(true)
Info("scanning %s", "example.com")
Success("done")
})
if outStr != "" {
t.Errorf("silent mode wrote chrome to stdout: %q", outStr)
}
if !strings.Contains(errStr, "scanning example.com") {
t.Errorf("silent chrome missing from stderr: %q", errStr)
}
}
// the default (non-silent) sink is stdout; flipping silent off must restore it.
func TestSetSilentOffRoutesChromeToStdout(t *testing.T) {
outStr, errStr := captureStdoutStderr(t, func() {
SetSilent(false)
Info("hello")
})
if !strings.Contains(outStr, "hello") {
t.Errorf("non-silent chrome missing from stdout: %q", outStr)
}
if strings.Contains(errStr, "hello") {
t.Errorf("non-silent chrome leaked to stderr: %q", errStr)
}
}
// Silent() reflects the toggle so callers can gate interactive widgets.
func TestSilentToggle(t *testing.T) {
defer SetSilent(false)
SetSilent(true)
if !Silent() {
t.Error("Silent() = false after SetSilent(true)")
}
SetSilent(false)
if Silent() {
t.Error("Silent() = true after SetSilent(false)")
}
}
// captureStdoutStderr swaps both real streams for pipes, runs fn, and returns
// what landed on each. SetSilent reads os.Stdout/os.Stderr at call time, so the
// swap has to happen before fn flips the sink - fn does that itself.
func captureStdoutStderr(t *testing.T, fn func()) (string, string) {
t.Helper()
outR, outW, err := os.Pipe()
if err != nil {
t.Fatalf("pipe stdout: %v", err)
}
errR, errW, err := os.Pipe()
if err != nil {
t.Fatalf("pipe stderr: %v", err)
}
savedOut, savedErr := os.Stdout, os.Stderr
os.Stdout, os.Stderr = outW, errW
outCh := drain(outR)
errCh := drain(errR)
fn()
os.Stdout, os.Stderr = savedOut, savedErr
outW.Close()
errW.Close()
return <-outCh, <-errCh
}
func drain(r *os.File) <-chan string {
ch := make(chan string, 1)
go func() {
buf := make([]byte, 0, 4096)
tmp := make([]byte, 1024)
for {
n, rerr := r.Read(tmp)
buf = append(buf, tmp[:n]...)
if rerr != nil {
break
}
}
ch <- string(buf)
}()
return ch
}
+5 -6
View File
@@ -14,7 +14,6 @@ package output
import (
"fmt"
"os"
"sync"
"time"
)
@@ -42,7 +41,7 @@ func NewSpinner(message string) *Spinner {
// Start begins the spinner animation
func (s *Spinner) Start() {
if apiMode {
if apiMode || silent {
return
}
@@ -57,7 +56,7 @@ func (s *Spinner) Start() {
// In non-TTY mode, just print the message once
if !IsTTY {
fmt.Printf(" %s...\n", s.message)
fmt.Fprintf(sink, " %s...\n", s.message)
return
}
@@ -66,7 +65,7 @@ func (s *Spinner) Start() {
// Stop halts the spinner and clears the line
func (s *Spinner) Stop() {
if apiMode {
if apiMode || silent {
return
}
@@ -112,8 +111,8 @@ func (s *Spinner) animate() {
spinnerChar := prefixInfo.Render(spinnerFrames[frame])
line := fmt.Sprintf("\r %s %s", spinnerChar, msg)
fmt.Fprint(os.Stdout, "\033[2K") // Clear line
fmt.Fprint(os.Stdout, line)
fmt.Fprint(sink, "\033[2K") // Clear line
fmt.Fprint(sink, line)
frame = (frame + 1) % len(spinnerFrames)
}
+57
View File
@@ -0,0 +1,57 @@
/*
·━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━·
: :
: █▀ █ █▀▀ · Blazing-fast pentesting suite :
: ▄█ █ █▀ · BSD 3-Clause License :
: :
: (c) 2022-2026 vmfunc, xyzeva, :
: lunchcat alumni & contributors :
: :
·━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━·
*/
// Package pool spreads independent per-item work across a fixed set of workers
// that all pull from one shared channel. that's the point over a static
// modulo-stride partition: a slow or timing-out item only stalls the one worker
// holding it, the rest keep draining the queue instead of idling behind it.
package pool
import "sync"
// Each runs fn for every item in items, concurrently, across at most workers
// goroutines. order isn't preserved - fn must be safe to call from multiple
// goroutines and guard any shared state itself. blocks until every item is done.
func Each[T any](items []T, workers int, fn func(T)) {
if len(items) == 0 {
return
}
// floor at one worker; a non-positive count would otherwise spawn nothing
// and silently drop the work.
if workers < 1 {
workers = 1
}
// never spin more workers than there is work for.
if workers > len(items) {
workers = len(items)
}
queue := make(chan T, len(items))
for i := 0; i < len(items); i++ {
queue <- items[i]
}
close(queue)
var wg sync.WaitGroup
wg.Add(workers)
for i := 0; i < workers; i++ {
go func() {
defer wg.Done()
// pull until the queue is drained; a worker that finishes its
// current item just grabs the next, which is the work-stealing.
for item := range queue {
fn(item)
}
}()
}
wg.Wait()
}
+145
View File
@@ -0,0 +1,145 @@
/*
·━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━·
: :
: █▀ █ █▀▀ · Blazing-fast pentesting suite :
: ▄█ █ █▀ · BSD 3-Clause License :
: :
: (c) 2022-2026 vmfunc, xyzeva, :
: lunchcat alumni & contributors :
: :
·━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━·
*/
package pool
import (
"sync"
"sync/atomic"
"testing"
)
// every item runs exactly once across a spread of sizes and worker counts,
// including the floors (zero/negative workers) and workers > len.
func TestEachProcessesAllExactlyOnce(t *testing.T) {
tests := []struct {
name string
items int
workers int
}{
{"empty", 0, 4},
{"single item", 1, 8},
{"workers floored from zero", 5, 0},
{"workers floored from negative", 5, -3},
{"more workers than items", 3, 16},
{"even split", 100, 4},
{"uneven split", 101, 7},
{"one worker", 50, 1},
}
for _, tt := range tests {
t.Run(tt.name, func(t *testing.T) {
items := make([]int, tt.items)
for i := 0; i < tt.items; i++ {
items[i] = i
}
var mu sync.Mutex
seen := make(map[int]int, tt.items)
Each(items, tt.workers, func(v int) {
mu.Lock()
seen[v]++
mu.Unlock()
})
if len(seen) != tt.items {
t.Fatalf("processed %d distinct items, want %d", len(seen), tt.items)
}
for v, n := range seen {
if n != 1 {
t.Errorf("item %d processed %d times, want 1", v, n)
}
}
})
}
}
// no more than `workers` (capped at len(items)) callbacks ever run at once.
func TestEachRespectsWorkerCap(t *testing.T) {
const (
items = 200
workers = 6
)
work := make([]int, items)
var inFlight, peak int64
var release = make(chan struct{})
var started sync.WaitGroup
started.Add(items)
go func() {
Each(work, workers, func(int) {
cur := atomic.AddInt64(&inFlight, 1)
for {
p := atomic.LoadInt64(&peak)
if cur <= p || atomic.CompareAndSwapInt64(&peak, p, cur) {
break
}
}
started.Done()
<-release
atomic.AddInt64(&inFlight, -1)
})
}()
// the cap means at most `workers` callbacks block on release at once, so
// release exactly that many at a time until everything drains.
done := make(chan struct{})
go func() {
for i := 0; i < items; i++ {
release <- struct{}{}
}
close(done)
}()
<-done
if got := atomic.LoadInt64(&peak); got > workers {
t.Fatalf("peak concurrency %d exceeded worker cap %d", got, workers)
}
}
// the cap is min(workers, len(items)): fewer items than workers must not spin
// idle goroutines past the item count.
func TestEachCapsAtItemCount(t *testing.T) {
const (
items = 3
workers = 32
)
work := make([]int, items)
var inFlight, peak int64
var ready sync.WaitGroup
ready.Add(items)
release := make(chan struct{})
go func() {
for i := 0; i < items; i++ {
release <- struct{}{}
}
}()
Each(work, workers, func(int) {
cur := atomic.AddInt64(&inFlight, 1)
for {
p := atomic.LoadInt64(&peak)
if cur <= p || atomic.CompareAndSwapInt64(&peak, p, cur) {
break
}
}
<-release
atomic.AddInt64(&inFlight, -1)
})
if got := atomic.LoadInt64(&peak); got > items {
t.Fatalf("peak concurrency %d exceeded item count %d", got, items)
}
}
+74
View File
@@ -0,0 +1,74 @@
/*
·━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━·
: :
: █▀ █ █▀▀ · Blazing-fast pentesting suite :
: ▄█ █ █▀ · BSD 3-Clause License :
: :
: (c) 2022-2026 vmfunc, xyzeva, :
: lunchcat alumni & contributors :
: :
·━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━·
*/
package report
import (
"bytes"
"encoding/json"
"sort"
"strings"
)
// Markdown renders results as a readable report grouped by target, then by
// module, with each module's finding pretty-printed as a json code block.
func Markdown(results []Result) []byte {
var b strings.Builder
b.WriteString("# sif scan report\n\n")
// group module results under their target so the report reads target-first
// regardless of the order results came in.
byTarget := make(map[string][]Result)
order := make([]string, 0)
for i := 0; i < len(results); i++ {
t := results[i].Target
if _, seen := byTarget[t]; !seen {
order = append(order, t)
}
byTarget[t] = append(byTarget[t], results[i])
}
for i := 0; i < len(order); i++ {
target := order[i]
b.WriteString("## ")
b.WriteString(target)
b.WriteString("\n\n")
mods := byTarget[target]
// sort modules so the report is deterministic across runs
sort.SliceStable(mods, func(a, c int) bool { return mods[a].Module < mods[c].Module })
for j := 0; j < len(mods); j++ {
b.WriteString("### ")
b.WriteString(mods[j].Module)
b.WriteString("\n\n")
b.WriteString("```json\n")
b.WriteString(prettyJSON(mods[j].Data))
b.WriteString("\n```\n\n")
}
}
return []byte(b.String())
}
// prettyJSON re-indents the raw finding for readability; if it doesn't parse as
// json (shouldn't happen, but never trust it) the raw bytes are returned as-is.
func prettyJSON(raw json.RawMessage) string {
if len(raw) == 0 {
return "null"
}
var indented bytes.Buffer
if err := json.Indent(&indented, raw, "", " "); err != nil {
return string(raw)
}
return indented.String()
}
+26
View File
@@ -0,0 +1,26 @@
/*
·━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━·
: :
: █▀ █ █▀▀ · Blazing-fast pentesting suite :
: ▄█ █ █▀ · BSD 3-Clause License :
: :
: (c) 2022-2026 vmfunc, xyzeva, :
: lunchcat alumni & contributors :
: :
·━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━·
*/
// Package report serializes collected scan results to sarif and markdown. it's
// deliberately decoupled from the scan package: callers map their own results
// into report.Result, so report never imports a scanner type.
package report
import "encoding/json"
// Result is one module's output for one target. Data is whatever the scanner
// returned, carried as raw json so report stays free of scan types.
type Result struct {
Target string
Module string
Data json.RawMessage
}
+172
View File
@@ -0,0 +1,172 @@
/*
·━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━·
: :
: █▀ █ █▀▀ · Blazing-fast pentesting suite :
: ▄█ █ █▀ · BSD 3-Clause License :
: :
: (c) 2022-2026 vmfunc, xyzeva, :
: lunchcat alumni & contributors :
: :
·━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━·
*/
package report
import (
"encoding/json"
"strings"
"testing"
)
// fakeResults are a couple of representative findings across two targets used by
// every test below.
func fakeResults() []Result {
return []Result{
{Target: "https://a.example.com", Module: "cors", Data: json.RawMessage(`{"severity":"high"}`)},
{Target: "https://a.example.com", Module: "probe", Data: json.RawMessage(`{"status_code":200}`)},
{Target: "https://b.example.com", Module: "redirect", Data: json.RawMessage(`{"parameter":"next"}`)},
}
}
func TestSARIF_ValidAndContainsFindings(t *testing.T) {
out, err := SARIF(fakeResults())
if err != nil {
t.Fatalf("SARIF: %v", err)
}
// the output must parse back into the sarif shape
var doc sarifLog
if err := json.Unmarshal(out, &doc); err != nil {
t.Fatalf("sarif output is not valid json: %v", err)
}
if doc.Version != "2.1.0" {
t.Errorf("expected sarif version 2.1.0, got %q", doc.Version)
}
if len(doc.Runs) != 1 {
t.Fatalf("expected exactly one run, got %d", len(doc.Runs))
}
run := doc.Runs[0]
if run.Tool.Driver.Name != "sif" {
t.Errorf("expected tool name sif, got %q", run.Tool.Driver.Name)
}
if len(run.Results) != 3 {
t.Fatalf("expected 3 results, got %d", len(run.Results))
}
// each finding's module id surfaces as the ruleId and its target as the uri
tests := []struct {
ruleID string
target string
}{
{"cors", "https://a.example.com"},
{"probe", "https://a.example.com"},
{"redirect", "https://b.example.com"},
}
for _, tt := range tests {
if !sarifHasResult(run.Results, tt.ruleID, tt.target) {
t.Errorf("expected sarif result rule=%q target=%q, got %+v", tt.ruleID, tt.target, run.Results)
}
}
// rules list each module id once, deduped across targets
if len(run.Tool.Driver.Rules) != 3 {
t.Errorf("expected 3 deduped rules, got %d: %+v", len(run.Tool.Driver.Rules), run.Tool.Driver.Rules)
}
}
func TestSARIF_DedupesRulesAcrossTargets(t *testing.T) {
// the same module on two targets must yield one rule but two results.
results := []Result{
{Target: "https://a.example.com", Module: "cors", Data: json.RawMessage(`{}`)},
{Target: "https://b.example.com", Module: "cors", Data: json.RawMessage(`{}`)},
}
out, err := SARIF(results)
if err != nil {
t.Fatalf("SARIF: %v", err)
}
var doc sarifLog
if err := json.Unmarshal(out, &doc); err != nil {
t.Fatalf("invalid json: %v", err)
}
run := doc.Runs[0]
if len(run.Tool.Driver.Rules) != 1 {
t.Errorf("expected 1 deduped rule, got %d", len(run.Tool.Driver.Rules))
}
if len(run.Results) != 2 {
t.Errorf("expected 2 results, got %d", len(run.Results))
}
}
func TestSARIF_Empty(t *testing.T) {
out, err := SARIF(nil)
if err != nil {
t.Fatalf("SARIF: %v", err)
}
var doc sarifLog
if err := json.Unmarshal(out, &doc); err != nil {
t.Fatalf("empty sarif is not valid json: %v", err)
}
if len(doc.Runs) != 1 {
t.Fatalf("expected one run even when empty, got %d", len(doc.Runs))
}
if len(doc.Runs[0].Results) != 0 {
t.Errorf("expected no results, got %d", len(doc.Runs[0].Results))
}
}
func TestMarkdown_ContainsTargetsAndModules(t *testing.T) {
out := string(Markdown(fakeResults()))
wants := []string{
"# sif scan report",
"## https://a.example.com",
"## https://b.example.com",
"### cors",
"### probe",
"### redirect",
`"severity": "high"`, // re-indented finding body
`"parameter": "next"`,
}
for _, want := range wants {
if !strings.Contains(out, want) {
t.Errorf("markdown report missing %q\n---\n%s", want, out)
}
}
}
func TestMarkdown_GroupsByTarget(t *testing.T) {
// a.example.com's two modules must both appear before b.example.com's header.
out := string(Markdown(fakeResults()))
aHeader := strings.Index(out, "## https://a.example.com")
bHeader := strings.Index(out, "## https://b.example.com")
if aHeader < 0 || bHeader < 0 {
t.Fatalf("missing target headers in:\n%s", out)
}
if aHeader > bHeader {
t.Errorf("expected target a before target b, got a=%d b=%d", aHeader, bHeader)
}
// both of a's modules sit between a's header and b's header
corsIdx := strings.Index(out, "### cors")
probeIdx := strings.Index(out, "### probe")
if corsIdx < aHeader || corsIdx > bHeader || probeIdx < aHeader || probeIdx > bHeader {
t.Errorf("expected a's modules grouped under a, cors=%d probe=%d (a=%d b=%d)", corsIdx, probeIdx, aHeader, bHeader)
}
}
// sarifHasResult reports whether any result carries the given rule id and target
// uri, the pairing that proves a finding survived serialization.
func sarifHasResult(results []sarifResult, ruleID, target string) bool {
for i := 0; i < len(results); i++ {
r := results[i]
if r.RuleID != ruleID {
continue
}
for j := 0; j < len(r.Locations); j++ {
if r.Locations[j].PhysicalLocation.ArtifactLocation.URI == target {
return true
}
}
}
return false
}
+133
View File
@@ -0,0 +1,133 @@
/*
·━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━·
: :
: █▀ █ █▀▀ · Blazing-fast pentesting suite :
: ▄█ █ █▀ · BSD 3-Clause License :
: :
: (c) 2022-2026 vmfunc, xyzeva, :
: lunchcat alumni & contributors :
: :
·━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━·
*/
package report
import (
"encoding/json"
"fmt"
)
// sarif format/version constants pinned to the 2.1.0 schema so the output is
// ingestable by github code scanning and other sarif consumers.
const (
sarifVersion = "2.1.0"
sarifSchema = "https://json.schemastore.org/sarif-2.1.0.json"
toolName = "sif"
)
// sarifLog is the minimal valid 2.1.0 shape: one run from one tool.
type sarifLog struct {
Schema string `json:"$schema"`
Version string `json:"version"`
Runs []sarifRun `json:"runs"`
}
type sarifRun struct {
Tool sarifTool `json:"tool"`
Results []sarifResult `json:"results"`
}
type sarifTool struct {
Driver sarifDriver `json:"driver"`
}
type sarifDriver struct {
Name string `json:"name"`
Rules []sarifRule `json:"rules"`
}
type sarifRule struct {
ID string `json:"id"`
}
type sarifResult struct {
RuleID string `json:"ruleId"`
Level string `json:"level"`
Message sarifMessage `json:"message"`
Locations []sarifLocation `json:"locations"`
}
type sarifMessage struct {
Text string `json:"text"`
}
type sarifLocation struct {
PhysicalLocation sarifPhysicalLocation `json:"physicalLocation"`
}
type sarifPhysicalLocation struct {
ArtifactLocation sarifArtifactLocation `json:"artifactLocation"`
}
type sarifArtifactLocation struct {
URI string `json:"uri"`
}
// sarifLevel is the default severity for findings; sif results don't carry a
// uniform severity field, so "warning" is the neutral middle ground.
const sarifLevel = "warning"
// SARIF serializes results to a minimal valid sarif 2.1.0 log. Each module
// result becomes one sarif result tagged with its module id (the rule) and the
// target uri, with the raw module data inlined into the message for context.
func SARIF(results []Result) ([]byte, error) {
sarifResults := make([]sarifResult, 0, len(results))
ruleSet := make(map[string]struct{}, len(results))
for i := 0; i < len(results); i++ {
res := results[i]
ruleSet[res.Module] = struct{}{}
sarifResults = append(sarifResults, sarifResult{
RuleID: res.Module,
Level: sarifLevel,
Message: sarifMessage{Text: messageFor(res)},
Locations: []sarifLocation{{
PhysicalLocation: sarifPhysicalLocation{
ArtifactLocation: sarifArtifactLocation{URI: res.Target},
},
}},
})
}
// rules must list each id exactly once; build it from the set so duplicate
// modules across targets don't duplicate the rule.
rules := make([]sarifRule, 0, len(ruleSet))
for id := range ruleSet {
rules = append(rules, sarifRule{ID: id})
}
doc := sarifLog{
Schema: sarifSchema,
Version: sarifVersion,
Runs: []sarifRun{{
Tool: sarifTool{Driver: sarifDriver{Name: toolName, Rules: rules}},
Results: sarifResults,
}},
}
out, err := json.MarshalIndent(doc, "", " ")
if err != nil {
return nil, fmt.Errorf("marshal sarif: %w", err)
}
return out, nil
}
// messageFor builds a human-readable result message: the module id plus the raw
// finding json so a sarif viewer shows what was actually found.
func messageFor(res Result) string {
if len(res.Data) == 0 {
return fmt.Sprintf("%s finding on %s", res.Module, res.Target)
}
return fmt.Sprintf("%s finding on %s: %s", res.Module, res.Target, string(res.Data))
}
+3 -2
View File
@@ -104,11 +104,12 @@ func checkS3Bucket(ctx context.Context, bucket string, client *http.Client) (boo
if err != nil {
return false, err
}
resp, err := client.Do(req)
resp, err := client.Do(req) //nolint:bodyclose // drained and closed via httpx.DrainClose
if err != nil {
return false, err
}
defer resp.Body.Close()
// status only; drain on close so the conn returns to the pool.
defer httpx.DrainClose(resp)
// If we can access the bucket listing, it's public
return resp.StatusCode == http.StatusOK, nil
+3 -2
View File
@@ -128,10 +128,11 @@ func detectWordPress(url string, client *http.Client, bodyString string) bool {
if err != nil {
continue
}
resp, err := client.Do(req)
resp, err := client.Do(req) //nolint:bodyclose // drained and closed via httpx.DrainClose
if err == nil {
found := resp.StatusCode == http.StatusOK || resp.StatusCode == http.StatusFound
resp.Body.Close()
// status only; drain so the conn returns to the pool.
httpx.DrainClose(resp)
if found {
return true
}
+236
View File
@@ -0,0 +1,236 @@
/*
·━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━·
: :
: █▀ █ █▀▀ · Blazing-fast pentesting suite :
: ▄█ █ █▀ · BSD 3-Clause License :
: :
: (c) 2022-2026 vmfunc, xyzeva, :
: lunchcat alumni & contributors :
: :
·━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━·
*/
package scan
import (
"context"
"fmt"
"net/http"
"net/url"
"strings"
"sync"
"time"
charmlog "github.com/charmbracelet/log"
"github.com/dropalldatabases/sif/internal/httpx"
"github.com/dropalldatabases/sif/internal/logger"
"github.com/dropalldatabases/sif/internal/output"
)
// CORSResult collects every cors misconfiguration found on the target.
type CORSResult struct {
Findings []CORSFinding `json:"findings,omitempty"`
}
// CORSFinding is a single reflecting/permissive cors response.
type CORSFinding struct {
URL string `json:"url"`
OriginTested string `json:"origin_tested"`
AllowOrigin string `json:"allow_origin"`
AllowCredentials bool `json:"allow_credentials"`
Severity string `json:"severity"`
Note string `json:"note"`
}
// corsMaxRedirects caps the redirect chain so we read the cors headers off the
// host we actually asked about, not whatever it bounces us to.
const corsMaxRedirects = 3
// the sentinel attacker origin; if it comes back in Access-Control-Allow-Origin
// the target reflects arbitrary origins and any site can read the response.
const corsEvilOrigin = "https://sif-cors-probe.evil.com"
// corsOrigin is a header to inject + why it matters. {host} expands to the
// target host so the prefix/suffix bypasses key off the real name.
var corsOrigins = []struct {
origin string // crafted Origin header, {host} -> target host
note string // why this case is interesting
reflects bool // true when a literal echo of this origin is exploitable
}{
// arbitrary attacker origin - the classic "reflects anything" bug
{corsEvilOrigin, "arbitrary origin reflected", true},
// the literal null origin (sandboxed iframes, redirects, file://) is forgeable
{"null", "null origin allowed", true},
// suffix bypass: attacker registers {host}.evil.com, naive endswith checks pass
{"https://{host}.evil.com", "suffix bypass (attacker subdomain)", true},
// prefix bypass: attacker registers evil-{host}, naive startswith checks pass
{"https://evil-{host}", "prefix bypass", true},
// embedded bypass: {host} appears inside an attacker domain
{"https://evil.com.{host}", "embedded-host bypass", true},
// scheme downgrade: http origin trusted lets a mitm read cross-origin data
{"http://{host}", "http scheme downgrade trusted", true},
}
// CORS probes the target for cross-origin resource sharing misconfigurations.
func CORS(targetURL string, timeout time.Duration, threads int, logdir string) (*CORSResult, error) {
log := output.Module("CORS")
log.Start()
spin := output.NewSpinner("Scanning for CORS misconfigurations")
spin.Start()
sanitizedURL := stripScheme(targetURL)
if logdir != "" {
if err := logger.WriteHeader(sanitizedURL, logdir, "CORS misconfiguration probe"); err != nil {
spin.Stop()
log.Error("error creating log file: %v", err)
return nil, fmt.Errorf("create cors log: %w", err)
}
}
parsedURL, err := url.Parse(targetURL)
if err != nil {
spin.Stop()
return nil, fmt.Errorf("parse url: %w", err)
}
host := parsedURL.Host
client := httpx.Client(timeout)
client.CheckRedirect = func(_ *http.Request, via []*http.Request) error {
if len(via) >= corsMaxRedirects {
return http.ErrUseLastResponse
}
return nil
}
result := &CORSResult{Findings: make([]CORSFinding, 0, len(corsOrigins))}
var mu sync.Mutex
var wg sync.WaitGroup
// one origin per worker item; the set is small so a buffered channel is plenty
originChan := make(chan int, len(corsOrigins))
for i := 0; i < len(corsOrigins); i++ {
originChan <- i
}
close(originChan)
wg.Add(threads)
for t := 0; t < threads; t++ {
go func() {
defer wg.Done()
for idx := range originChan {
spec := corsOrigins[idx]
// {host} is the seam that turns a template into a real attacker origin
origin := strings.ReplaceAll(spec.origin, "{host}", host)
finding, ok := probeCORS(client, targetURL, origin, spec.note)
if !ok {
continue
}
mu.Lock()
result.Findings = append(result.Findings, finding)
mu.Unlock()
spin.Stop()
log.Warn("cors %s: origin %s reflected (creds=%t)",
renderCORSSeverity(finding.Severity),
output.Highlight.Render(origin),
finding.AllowCredentials)
spin.Start()
if logdir != "" {
logger.Write(sanitizedURL, logdir,
fmt.Sprintf("CORS: %s - origin [%s] reflected as [%s] creds=%t\n",
finding.Note, origin, finding.AllowOrigin, finding.AllowCredentials))
}
}
}()
}
wg.Wait()
spin.Stop()
if len(result.Findings) == 0 {
log.Info("no cors misconfigurations detected")
log.Complete(0, "found")
return nil, nil //nolint:nilnil // no finding is not an error, mirrors the other scanners
}
log.Complete(len(result.Findings), "found")
return result, nil
}
// probeCORS sends one request with the crafted Origin and decides whether the
// response trusts it. It returns the finding and true only when the server
// reflects the origin (or "null"/"*" with credentials), which is the exploitable
// shape - a server that ignores Origin or returns its own host is fine.
func probeCORS(client *http.Client, targetURL, origin, note string) (CORSFinding, bool) {
req, err := http.NewRequestWithContext(context.TODO(), http.MethodGet, targetURL, http.NoBody)
if err != nil {
charmlog.Debugf("cors: build request for %s: %v", targetURL, err)
return CORSFinding{}, false
}
req.Header.Set("Origin", origin)
resp, err := client.Do(req) //nolint:bodyclose // drained and closed via httpx.DrainClose
if err != nil {
charmlog.Debugf("cors: request %s with origin %s: %v", targetURL, origin, err)
return CORSFinding{}, false
}
// headers are all we need; drain the body so the conn returns to the pool.
httpx.DrainClose(resp)
allowOrigin := resp.Header.Get("Access-Control-Allow-Origin")
if allowOrigin == "" {
return CORSFinding{}, false
}
allowCreds := strings.EqualFold(resp.Header.Get("Access-Control-Allow-Credentials"), "true")
// a wildcard with credentials is forbidden by browsers, so it isn't directly
// exploitable; a plain wildcard exposes only public data. neither is a finding.
if allowOrigin == "*" {
return CORSFinding{}, false
}
// the bug is reflection: the server echoed our attacker origin back. if it
// returned something else (its own host) it isn't trusting us.
reflected := allowOrigin == origin
if !reflected {
return CORSFinding{}, false
}
return CORSFinding{
URL: targetURL,
OriginTested: origin,
AllowOrigin: allowOrigin,
AllowCredentials: allowCreds,
Severity: corsSeverity(allowCreds),
Note: note,
}, true
}
// corsSeverity ranks the finding: reflection + credentials lets an attacker read
// authenticated responses, which is the high-impact case.
func corsSeverity(allowCreds bool) string {
if allowCreds {
return "high"
}
return "medium"
}
func renderCORSSeverity(severity string) string {
if severity == "high" {
return output.SeverityHigh.Render(severity)
}
return output.SeverityMedium.Render(severity)
}
// ResultType identifies cors findings for the result registry.
func (r *CORSResult) ResultType() string { return "cors" }
var _ ScanResult = (*CORSResult)(nil)
+140
View File
@@ -0,0 +1,140 @@
/*
·━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━·
: :
: █▀ █ █▀▀ · Blazing-fast pentesting suite :
: ▄█ █ █▀ · BSD 3-Clause License :
: :
: (c) 2022-2026 vmfunc, xyzeva, :
: lunchcat alumni & contributors :
: :
·━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━·
*/
package scan
import (
"net/http"
"net/http/httptest"
"testing"
"time"
)
// reflectingCORS echoes the Origin into Access-Control-Allow-Origin and sets
// credentials, the exploitable misconfiguration.
func reflectingCORS() *httptest.Server {
return httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
if origin := r.Header.Get("Origin"); origin != "" {
w.Header().Set("Access-Control-Allow-Origin", origin)
w.Header().Set("Access-Control-Allow-Credentials", "true")
}
w.WriteHeader(http.StatusOK)
}))
}
func TestCORS_ReflectsArbitraryOrigin(t *testing.T) {
srv := reflectingCORS()
defer srv.Close()
result, err := CORS(srv.URL, 5*time.Second, 3, "")
if err != nil {
t.Fatalf("CORS: %v", err)
}
if result == nil || len(result.Findings) == 0 {
t.Fatalf("expected cors findings on reflecting server, got %+v", result)
}
// the reflecting server echoes every crafted origin with credentials,
// so each finding should be high severity.
var sawEvil bool
for _, f := range result.Findings {
if f.OriginTested == corsEvilOrigin {
sawEvil = true
if !f.AllowCredentials {
t.Errorf("expected credentials flagged for evil origin, got %+v", f)
}
if f.Severity != "high" {
t.Errorf("expected high severity for reflection+creds, got %s", f.Severity)
}
}
}
if !sawEvil {
t.Errorf("expected the sentinel evil origin to be reflected, got %+v", result.Findings)
}
}
func TestCORS_SeverityWithoutCredentials(t *testing.T) {
// reflects the origin but never grants credentials - medium, not high.
srv := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
if origin := r.Header.Get("Origin"); origin != "" {
w.Header().Set("Access-Control-Allow-Origin", origin)
}
w.WriteHeader(http.StatusOK)
}))
defer srv.Close()
result, err := CORS(srv.URL, 5*time.Second, 3, "")
if err != nil {
t.Fatalf("CORS: %v", err)
}
if result == nil || len(result.Findings) == 0 {
t.Fatalf("expected reflection findings, got %+v", result)
}
for _, f := range result.Findings {
if f.AllowCredentials {
t.Errorf("did not expect credentials, got %+v", f)
}
if f.Severity != "medium" {
t.Errorf("expected medium severity without creds, got %s", f.Severity)
}
}
}
func TestCORS_NoFalsePositiveOnSafeServer(t *testing.T) {
tests := []struct {
name string
handler http.HandlerFunc
}{
{
name: "ignores origin entirely",
handler: func(w http.ResponseWriter, _ *http.Request) {
w.WriteHeader(http.StatusOK)
},
},
{
name: "returns its own fixed origin",
handler: func(w http.ResponseWriter, _ *http.Request) {
w.Header().Set("Access-Control-Allow-Origin", "https://trusted.example.com")
w.WriteHeader(http.StatusOK)
},
},
{
name: "plain wildcard, no credentials",
handler: func(w http.ResponseWriter, _ *http.Request) {
w.Header().Set("Access-Control-Allow-Origin", "*")
w.WriteHeader(http.StatusOK)
},
},
}
for _, tt := range tests {
t.Run(tt.name, func(t *testing.T) {
srv := httptest.NewServer(tt.handler)
defer srv.Close()
result, err := CORS(srv.URL, 5*time.Second, 3, "")
if err != nil {
t.Fatalf("CORS: %v", err)
}
if result != nil && len(result.Findings) > 0 {
t.Errorf("expected no findings on safe server, got %+v", result.Findings)
}
})
}
}
func TestCORSResult_ResultType(t *testing.T) {
r := &CORSResult{}
if r.ResultType() != "cors" {
t.Errorf("expected result type 'cors', got %q", r.ResultType())
}
}
+137
View File
@@ -0,0 +1,137 @@
/*
·━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━·
: :
: █▀ █ █▀▀ · Blazing-fast pentesting suite :
: ▄█ █ █▀ · BSD 3-Clause License :
: :
: (c) 2022-2026 vmfunc, xyzeva, :
: lunchcat alumni & contributors :
: :
·━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━·
*/
package scan
import (
"fmt"
"net/url"
"sort"
"sync"
"time"
"github.com/gocolly/colly/v2"
"github.com/dropalldatabases/sif/internal/httpx"
"github.com/dropalldatabases/sif/internal/logger"
"github.com/dropalldatabases/sif/internal/output"
)
// CrawlResult holds the deduped set of urls discovered by the spider.
type CrawlResult struct {
URLs []string `json:"urls"`
}
func (r *CrawlResult) ResultType() string { return "crawl" }
// compile-time check so a result-type drift fails the build, not a run.
var _ ScanResult = (*CrawlResult)(nil)
// Crawl spiders the target up to depth, following same-host links/scripts/forms.
// all traffic flows through the shared httpx client so proxy/headers/rate-limit
// apply, and robots.txt is respected (colly honors it by default).
func Crawl(targetURL string, depth int, timeout time.Duration, logdir string) (*CrawlResult, error) {
log := output.Module("CRAWL")
log.Start()
sanitizedURL := stripScheme(targetURL)
if logdir != "" {
if err := logger.WriteHeader(sanitizedURL, logdir, "web crawl"); err != nil {
log.Error("error creating log file: %v", err)
return nil, fmt.Errorf("create crawl log: %w", err)
}
}
// the host bounds the crawl; without it colly would wander the whole web.
parsed, err := url.Parse(targetURL)
if err != nil {
return nil, fmt.Errorf("parse target url %q: %w", targetURL, err)
}
host := parsed.Hostname()
if host == "" {
return nil, fmt.Errorf("target url %q has no host", targetURL)
}
collector := colly.NewCollector(
colly.MaxDepth(depth),
colly.AllowedDomains(host),
)
// reuse the shared client so proxy/cookie/-H/rate-limit are honored and the
// configured timeout applies to every fetch, robots.txt included.
collector.SetClient(httpx.Client(timeout))
// dedupe across the concurrent callbacks colly may fire.
var mu sync.Mutex
seen := make(map[string]struct{})
record := func(raw string) {
if raw == "" {
return
}
// keep the result set scoped to the target host; off-host assets
// (cdns, third-party links) are noise for an in-scope crawl.
if u, err := url.Parse(raw); err != nil || u.Hostname() != host {
return
}
mu.Lock()
if _, ok := seen[raw]; !ok {
seen[raw] = struct{}{}
log.Success("found: %s", output.Highlight.Render(raw))
if logdir != "" {
_ = logger.Write(sanitizedURL, logdir, raw+"\n")
}
}
mu.Unlock()
}
// links drive recursion; scripts/forms are recorded but not followed.
collector.OnHTML("a[href]", func(e *colly.HTMLElement) {
link := e.Request.AbsoluteURL(e.Attr("href"))
record(link)
// Visit enforces AllowedDomains/MaxDepth itself, so off-host or
// too-deep links are dropped without us re-checking.
_ = e.Request.Visit(link)
})
collector.OnHTML("script[src]", func(e *colly.HTMLElement) {
record(e.Request.AbsoluteURL(e.Attr("src")))
})
collector.OnHTML("form[action]", func(e *colly.HTMLElement) {
record(e.Request.AbsoluteURL(e.Attr("action")))
})
collector.OnError(func(_ *colly.Response, e error) {
// a single bad page shouldn't abort the crawl; note it and move on.
log.Warn("crawl error: %v", e)
})
if err := collector.Visit(targetURL); err != nil {
log.Error("crawl failed: %v", err)
return nil, fmt.Errorf("visit %q: %w", targetURL, err)
}
collector.Wait()
result := &CrawlResult{URLs: sortedKeys(seen)}
log.Complete(len(result.URLs), "urls")
return result, nil
}
// sortedKeys returns the map keys in a stable order so output is deterministic.
func sortedKeys(set map[string]struct{}) []string {
keys := make([]string, 0, len(set))
for k := range set {
keys = append(keys, k)
}
sort.Strings(keys)
return keys
}
+158
View File
@@ -0,0 +1,158 @@
/*
·━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━·
: :
: █▀ █ █▀▀ · Blazing-fast pentesting suite :
: ▄█ █ █▀ · BSD 3-Clause License :
: :
: (c) 2022-2026 vmfunc, xyzeva, :
: lunchcat alumni & contributors :
: :
·━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━·
*/
package scan
import (
"net/http"
"net/http/httptest"
"testing"
"time"
)
// crawlSite serves a small link graph:
//
// / -> links /a and an off-host page; references script.js, form action /submit
// /a -> links /b
// /b -> links /c (only reachable at depth 3)
// /c -> leaf
func crawlSite(t *testing.T) *httptest.Server {
t.Helper()
mux := http.NewServeMux()
// no robots restrictions; colly fetches this before crawling.
mux.HandleFunc("/robots.txt", func(w http.ResponseWriter, _ *http.Request) {
w.WriteHeader(http.StatusNotFound)
})
mux.HandleFunc("/", func(w http.ResponseWriter, r *http.Request) {
if r.URL.Path != "/" {
http.NotFound(w, r)
return
}
_, _ = w.Write([]byte(`<html><body>
<a href="/a">a</a>
<a href="https://off-host.example/x">off</a>
<script src="/script.js"></script>
<form action="/submit"></form>
</body></html>`))
})
mux.HandleFunc("/a", func(w http.ResponseWriter, _ *http.Request) {
_, _ = w.Write([]byte(`<a href="/b">b</a>`))
})
mux.HandleFunc("/b", func(w http.ResponseWriter, _ *http.Request) {
_, _ = w.Write([]byte(`<a href="/c">c</a>`))
})
mux.HandleFunc("/c", func(w http.ResponseWriter, _ *http.Request) {
_, _ = w.Write([]byte(`leaf`))
})
srv := httptest.NewServer(mux)
t.Cleanup(srv.Close)
return srv
}
func urlsContain(urls []string, want string) bool {
for i := 0; i < len(urls); i++ {
if urls[i] == want {
return true
}
}
return false
}
func TestCrawl_FindsLinkedPagesAndAssets(t *testing.T) {
srv := crawlSite(t)
result, err := Crawl(srv.URL, 3, 5*time.Second, "")
if err != nil {
t.Fatalf("Crawl: %v", err)
}
// links, scripts and forms must all be recorded, resolved to absolute urls.
wants := []string{
srv.URL + "/a",
srv.URL + "/b",
srv.URL + "/c",
srv.URL + "/script.js",
srv.URL + "/submit",
}
for _, w := range wants {
if !urlsContain(result.URLs, w) {
t.Errorf("expected crawl to find %q, got %v", w, result.URLs)
}
}
// AllowedDomains must keep the off-host link out of the result set.
if urlsContain(result.URLs, "https://off-host.example/x") {
t.Errorf("off-host link should be excluded, got %v", result.URLs)
}
}
func TestCrawl_RespectsDepth(t *testing.T) {
srv := crawlSite(t)
// depth 1: only links found on the root page (/a, /script.js, /submit) are
// recorded; /b lives one hop deeper and must not appear.
result, err := Crawl(srv.URL, 1, 5*time.Second, "")
if err != nil {
t.Fatalf("Crawl: %v", err)
}
if !urlsContain(result.URLs, srv.URL+"/a") {
t.Errorf("depth 1 should find /a, got %v", result.URLs)
}
if urlsContain(result.URLs, srv.URL+"/b") {
t.Errorf("depth 1 must not reach /b, got %v", result.URLs)
}
if urlsContain(result.URLs, srv.URL+"/c") {
t.Errorf("depth 1 must not reach /c, got %v", result.URLs)
}
}
func TestCrawl_Dedupes(t *testing.T) {
// a page that links the same target twice must yield a single entry.
mux := http.NewServeMux()
mux.HandleFunc("/robots.txt", func(w http.ResponseWriter, _ *http.Request) {
w.WriteHeader(http.StatusNotFound)
})
mux.HandleFunc("/", func(w http.ResponseWriter, r *http.Request) {
if r.URL.Path == "/dup" {
_, _ = w.Write([]byte(`leaf`))
return
}
_, _ = w.Write([]byte(`<a href="/dup">1</a><a href="/dup">2</a>`))
})
srv := httptest.NewServer(mux)
defer srv.Close()
result, err := Crawl(srv.URL, 2, 5*time.Second, "")
if err != nil {
t.Fatalf("Crawl: %v", err)
}
count := 0
for _, u := range result.URLs {
if u == srv.URL+"/dup" {
count++
}
}
if count != 1 {
t.Errorf("expected /dup once after dedupe, got %d in %v", count, result.URLs)
}
}
func TestCrawl_ResultType(t *testing.T) {
r := &CrawlResult{}
if r.ResultType() != "crawl" {
t.Errorf("ResultType = %q, want crawl", r.ResultType())
}
}
+389 -65
View File
@@ -16,8 +16,12 @@ import (
"bufio"
"context"
"fmt"
"io"
"net/http"
"os"
"regexp"
"strconv"
"strings"
"sync"
"time"
@@ -25,6 +29,7 @@ import (
"github.com/dropalldatabases/sif/internal/httpx"
"github.com/dropalldatabases/sif/internal/logger"
"github.com/dropalldatabases/sif/internal/output"
"github.com/dropalldatabases/sif/internal/pool"
)
// directoryURL is a var so integration tests can repoint it at a fixture.
@@ -36,13 +41,342 @@ const (
bigFile = "directory-list-2.3-big.txt"
)
// dirlistBodyCap bounds how many bytes we read per response before computing
// size/word counts. modern apps stream large html; capping keeps memory flat
// and makes size/word matching deterministic against arbitrarily large bodies.
const dirlistBodyCap = 512 * 1024
// soft-404 calibration probes. we ask for a handful of deterministic paths that
// cannot exist, then treat any response shape they share as the wildcard
// baseline. deterministic (no rng) so the workflow stays reproducible.
const (
calibrationProbes = 3
calibrationPrefix = "/sif-cal-"
)
// statusNotFound / statusForbidden are the historical default "not interesting"
// codes; they seed the filter set when no explicit -mc/-fc is given.
const (
statusNotFound = 404
statusForbidden = 403
)
type DirectoryResult struct {
Url string `json:"url"`
StatusCode int `json:"status_code"`
Size int `json:"size"`
Words int `json:"words"`
}
// Dirlist performs directory fuzzing on the target URL.
func Dirlist(size string, url string, timeout time.Duration, threads int, logdir string) ([]DirectoryResult, error) {
// DirlistOptions carries the ffuf-style matcher knobs. the zero value reproduces
// the legacy behavior (report everything that isn't 404/403), so callers that
// don't set anything keep the old defaults.
type DirlistOptions struct {
MatchCodes string // -mc comma list of status codes to keep
FilterCodes string // -fc comma list of status codes to drop
FilterSizes string // -fs comma list of body sizes to drop
FilterWords string // -fw comma list of word counts to drop
FilterRegex string // -fr regex; a body match drops the response
Calibrate bool // -ac auto-calibrate the soft-404 wildcard baseline
Wordlist string // -w local path or url; overrides the size switch
Extensions string // -e comma list appended to each word (php,bak,env)
}
// responseMeta is the shape we match on: just enough of the response to decide
// keep/drop without holding the whole body.
type responseMeta struct {
status int
size int
words int
}
// matcher decides whether a response is "interesting" using the same precedence
// as ffuf/feroxbuster: an explicit filter (-fc/-fs/-fw/-fr or a calibrated
// baseline) drops the response, otherwise the match-code set decides.
type matcher struct {
matchCodes map[int]struct{}
filterCodes map[int]struct{}
filterSizes map[int]struct{}
filterWords map[int]struct{}
filterRe *regexp.Regexp
baselines []responseMeta // calibrated soft-404 shapes to suppress
}
// newMatcher builds the matcher from raw flag strings. when -mc is empty the
// match set is left nil, which Matches reads as "keep anything not explicitly
// filtered" - i.e. the legacy behavior minus the hardcoded 404/403, which move
// into the filter set instead.
func newMatcher(opts *DirlistOptions) (*matcher, error) {
m := &matcher{
filterSizes: make(map[int]struct{}),
filterWords: make(map[int]struct{}),
}
codes, err := parseIntSet(opts.MatchCodes)
if err != nil {
return nil, fmt.Errorf("parse -mc: %w", err)
}
m.matchCodes = codes
m.filterCodes, err = parseIntSet(opts.FilterCodes)
if err != nil {
return nil, fmt.Errorf("parse -fc: %w", err)
}
// no explicit match set means we fall back to the historical "drop 404/403"
// behavior; encode it as filters so the rest of the logic is uniform.
if len(m.matchCodes) == 0 && len(m.filterCodes) == 0 {
m.filterCodes[statusNotFound] = struct{}{}
m.filterCodes[statusForbidden] = struct{}{}
}
m.filterSizes, err = parseIntSet(opts.FilterSizes)
if err != nil {
return nil, fmt.Errorf("parse -fs: %w", err)
}
m.filterWords, err = parseIntSet(opts.FilterWords)
if err != nil {
return nil, fmt.Errorf("parse -fw: %w", err)
}
if opts.FilterRegex != "" {
re, err := regexp.Compile(opts.FilterRegex)
if err != nil {
return nil, fmt.Errorf("parse -fr: %w", err)
}
m.filterRe = re
}
return m, nil
}
// Matches reports whether the response should surface as a finding. filters win
// over matches: a calibrated baseline, an -fc/-fs/-fw hit, or an -fr body match
// always drops the response; otherwise the -mc set (when set) gates it.
func (m *matcher) Matches(meta responseMeta, body []byte) bool {
// a calibrated soft-404 shape is the same response the catch-all hands every
// bogus path, so drop anything that matches a baseline exactly.
for i := 0; i < len(m.baselines); i++ {
b := m.baselines[i]
if b.status == meta.status && b.size == meta.size && b.words == meta.words {
return false
}
}
if _, drop := m.filterCodes[meta.status]; drop {
return false
}
if _, drop := m.filterSizes[meta.size]; drop {
return false
}
if _, drop := m.filterWords[meta.words]; drop {
return false
}
if m.filterRe != nil && m.filterRe.Match(body) {
return false
}
// an explicit -mc set is allow-list semantics; without it we keep whatever
// survived the filters above.
if len(m.matchCodes) > 0 {
_, keep := m.matchCodes[meta.status]
return keep
}
return true
}
// parseIntSet turns a comma list like "200,301,500" into a set. empty input is a
// nil set, not an error, so unset flags are a no-op.
func parseIntSet(raw string) (map[int]struct{}, error) {
set := make(map[int]struct{})
if raw == "" {
return set, nil
}
for _, part := range strings.Split(raw, ",") {
part = strings.TrimSpace(part)
if part == "" {
continue
}
n, err := strconv.Atoi(part)
if err != nil {
return nil, fmt.Errorf("invalid integer %q: %w", part, err)
}
set[n] = struct{}{}
}
return set, nil
}
// readMeta drains the response (capped) and returns its match shape plus the
// body bytes the regex filter needs. it never returns the raw resp; callers
// close the body before this returns.
func readMeta(resp *http.Response) (responseMeta, []byte) {
body, err := io.ReadAll(io.LimitReader(resp.Body, dirlistBodyCap))
if err != nil {
// a truncated/aborted body still has a usable status; treat what we read
// as the body rather than dropping the whole response.
charmlog.Debugf("dirlist: read body: %v", err)
}
return responseMeta{
status: resp.StatusCode,
size: len(body),
words: countWords(body),
}, body
}
// countWords counts whitespace-separated tokens; the cheap proxy ffuf uses to
// tell a soft-404 stub apart from a real page of the same byte size.
func countWords(body []byte) int {
return len(strings.Fields(string(body)))
}
// expandWords appends each extension to every base word, keeping the bare word
// too. an empty extensions list returns the words unchanged.
func expandWords(words []string, extensions string) []string {
exts := splitExtensions(extensions)
if len(exts) == 0 {
return words
}
// each word yields itself plus one entry per extension.
expanded := make([]string, 0, len(words)*(len(exts)+1))
for i := 0; i < len(words); i++ {
expanded = append(expanded, words[i])
for j := 0; j < len(exts); j++ {
expanded = append(expanded, words[i]+"."+exts[j])
}
}
return expanded
}
// splitExtensions normalizes "php, .bak ,env" into ["php","bak","env"]; a
// leading dot is tolerated so both "php" and ".php" work.
func splitExtensions(raw string) []string {
if raw == "" {
return nil
}
parts := strings.Split(raw, ",")
exts := make([]string, 0, len(parts))
for i := 0; i < len(parts); i++ {
ext := strings.TrimSpace(parts[i])
ext = strings.TrimPrefix(ext, ".")
if ext != "" {
exts = append(exts, ext)
}
}
return exts
}
// loadWordlist reads the fuzzing words. a custom -w overrides the size switch:
// an http(s) value is fetched through the shared client, anything else is a
// local file. with no -w it downloads the size-selected sif-runtime list.
func loadWordlist(opts *DirlistOptions, size string, client *http.Client) ([]string, error) {
if opts.Wordlist != "" {
if strings.HasPrefix(opts.Wordlist, "http://") || strings.HasPrefix(opts.Wordlist, "https://") {
return fetchWordlist(opts.Wordlist, client)
}
return readWordlistFile(opts.Wordlist)
}
var file string
switch size {
case "small":
file = smallFile
case "medium":
file = mediumFile
case "large":
file = bigFile
default:
return nil, fmt.Errorf("unknown dirlist size %q", size)
}
return fetchWordlist(directoryURL+file, client)
}
// fetchWordlist downloads a remote wordlist through the shared client so proxy
// and rate-limit settings apply to the fetch too.
func fetchWordlist(listURL string, client *http.Client) ([]string, error) {
req, err := http.NewRequestWithContext(context.TODO(), http.MethodGet, listURL, http.NoBody)
if err != nil {
return nil, fmt.Errorf("build wordlist request: %w", err)
}
resp, err := client.Do(req)
if err != nil {
return nil, fmt.Errorf("download wordlist %q: %w", listURL, err)
}
defer resp.Body.Close()
return scanLines(resp.Body), nil
}
// readWordlistFile loads a local wordlist file.
func readWordlistFile(path string) ([]string, error) {
f, err := os.Open(path)
if err != nil {
return nil, fmt.Errorf("open wordlist %q: %w", path, err)
}
defer f.Close()
return scanLines(f), nil
}
// scanLines reads non-empty lines into a slice.
func scanLines(r io.Reader) []string {
var lines []string
scanner := bufio.NewScanner(r)
scanner.Split(bufio.ScanLines)
for scanner.Scan() {
line := scanner.Text()
if line != "" {
lines = append(lines, line)
}
}
return lines
}
// calibrate probes a few paths that cannot exist and records the response shapes
// the catch-all hands them. those baselines feed the matcher so a soft-404 200
// (the SPA wildcard) is suppressed before the real run. deterministic by design:
// the probe paths come from the loop index, never a random source.
func calibrate(m *matcher, baseURL string, client *http.Client) {
for i := 0; i < calibrationProbes; i++ {
probe := baseURL + calibrationPrefix + strconv.Itoa(i)
req, err := http.NewRequestWithContext(context.TODO(), http.MethodGet, probe, http.NoBody)
if err != nil {
charmlog.Debugf("dirlist: build calibration request: %v", err)
continue
}
resp, err := client.Do(req)
if err != nil {
charmlog.Debugf("dirlist: calibration probe %s: %v", probe, err)
continue
}
meta, _ := readMeta(resp)
resp.Body.Close()
// a genuine hard 404 already gets filtered by code; only soft responses
// (a 200/30x catch-all) need a size/word baseline to suppress them.
if meta.status == statusNotFound {
continue
}
if !containsBaseline(m.baselines, meta) {
m.baselines = append(m.baselines, meta)
}
}
}
// containsBaseline reports whether the shape is already recorded, so repeated
// probes returning the same soft-404 don't bloat the baseline set.
func containsBaseline(baselines []responseMeta, meta responseMeta) bool {
for i := 0; i < len(baselines); i++ {
if baselines[i] == meta {
return true
}
}
return false
}
// Dirlist performs directory fuzzing on the target URL with ffuf-style response
// filtering, soft-404 calibration and custom wordlists.
//
//nolint:gocritic // opts is the scanner's stable public config; passed by value to match the other scanners' entry points.
func Dirlist(size string, url string, timeout time.Duration, threads int, logdir string, opts DirlistOptions) (DirectoryResults, error) {
log := output.Module("DIRLIST")
log.Start()
@@ -55,89 +389,79 @@ func Dirlist(size string, url string, timeout time.Duration, threads int, logdir
}
}
var list string
switch size {
case "small":
list = directoryURL + smallFile
case "medium":
list = directoryURL + mediumFile
case "large":
list = directoryURL + bigFile
matcher, err := newMatcher(&opts)
if err != nil {
log.Error("invalid matcher flags: %v", err)
return nil, err
}
client := httpx.Client(timeout)
req, err := http.NewRequestWithContext(context.TODO(), http.MethodGet, list, http.NoBody)
directories, err := loadWordlist(&opts, size, client)
if err != nil {
log.Error("Error creating directory list request: %s", err)
log.Error("Error loading directory list: %s", err)
return nil, err
}
resp, err := client.Do(req)
if err != nil {
log.Error("Error downloading directory list: %s", err)
return nil, err
}
defer resp.Body.Close()
directories = expandWords(directories, opts.Extensions)
var directories []string
scanner := bufio.NewScanner(resp.Body)
scanner.Split(bufio.ScanLines)
for scanner.Scan() {
directories = append(directories, scanner.Text())
// -ac learns the wildcard baseline before the run so catch-all 200s drop.
if opts.Calibrate {
calibrate(matcher, url, client)
if len(matcher.baselines) > 0 {
log.Info("calibrated %d soft-404 baseline(s)", len(matcher.baselines))
}
}
progress := output.NewProgress(len(directories), "fuzzing")
var wg sync.WaitGroup
var mu sync.Mutex
wg.Add(threads)
results := make([]DirectoryResult, 0, 64)
for thread := 0; thread < threads; thread++ {
go func(thread int) {
defer wg.Done()
results := make(DirectoryResults, 0, 64)
pool.Each(directories, threads, func(directory string) {
progress.Increment(directory)
for i, directory := range directories {
if i%threads != thread {
continue
}
charmlog.Debugf("%s", directory)
dirReq, err := http.NewRequestWithContext(context.TODO(), http.MethodGet, url+"/"+directory, http.NoBody)
if err != nil {
charmlog.Debugf("Error creating request for %s: %s", directory, err)
return
}
resp, err := client.Do(dirReq)
if err != nil {
charmlog.Debugf("Error %s: %s", directory, err)
return
}
progress.Increment(directory)
meta, body := readMeta(resp)
reqURL := resp.Request.URL.String()
resp.Body.Close()
charmlog.Debugf("%s", directory)
dirReq, err := http.NewRequestWithContext(context.TODO(), http.MethodGet, url+"/"+directory, http.NoBody)
if err != nil {
charmlog.Debugf("Error creating request for %s: %s", directory, err)
continue
}
resp, err := client.Do(dirReq)
if err != nil {
charmlog.Debugf("Error %s: %s", directory, err)
continue
}
if !matcher.Matches(meta, body) {
return
}
if resp.StatusCode != 404 && resp.StatusCode != 403 {
progress.Pause()
log.Success("found: %s [%s]", output.Highlight.Render(directory), output.Status.Render(strconv.Itoa(resp.StatusCode)))
progress.Resume()
progress.Pause()
log.Success("found: %s [%s] (size=%d words=%d)",
output.Highlight.Render(directory),
output.Status.Render(strconv.Itoa(meta.status)),
meta.size, meta.words)
progress.Resume()
if logdir != "" {
_ = logger.Write(sanitizedURL, logdir, fmt.Sprintf("%s [%s]\n", strconv.Itoa(resp.StatusCode), directory))
}
if logdir != "" {
_ = logger.Write(sanitizedURL, logdir,
fmt.Sprintf("%s [%s] size=%d words=%d\n", strconv.Itoa(meta.status), directory, meta.size, meta.words))
}
result := DirectoryResult{
Url: resp.Request.URL.String(),
StatusCode: resp.StatusCode,
}
mu.Lock()
results = append(results, result)
mu.Unlock()
}
resp.Body.Close()
}
}(thread)
}
wg.Wait()
result := DirectoryResult{
Url: reqURL,
StatusCode: meta.status,
Size: meta.size,
Words: meta.words,
}
mu.Lock()
results = append(results, result)
mu.Unlock()
})
progress.Done()
log.Complete(len(results), "found")
+360
View File
@@ -0,0 +1,360 @@
/*
·━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━·
: :
: █▀ █ █▀▀ · Blazing-fast pentesting suite :
: ▄█ █ █▀ · BSD 3-Clause License :
: :
: (c) 2022-2026 vmfunc, xyzeva, :
: lunchcat alumni & contributors :
: :
·━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━·
*/
package scan
import (
"net/http"
"net/http/httptest"
"os"
"path/filepath"
"reflect"
"strings"
"testing"
"time"
)
func TestMatcher_Matches(t *testing.T) {
tests := []struct {
name string
opts DirlistOptions
meta responseMeta
body string
want bool
}{
{
// default behavior: 404/403 drop, everything else surfaces
name: "default keeps 200",
opts: DirlistOptions{},
meta: responseMeta{status: 200, size: 10, words: 2},
want: true,
},
{
name: "default drops 404",
opts: DirlistOptions{},
meta: responseMeta{status: 404, size: 9, words: 1},
want: false,
},
{
name: "default drops 403",
opts: DirlistOptions{},
meta: responseMeta{status: 403, size: 9, words: 1},
want: false,
},
{
// -mc is allow-list: only listed codes survive
name: "mc allowlist keeps listed",
opts: DirlistOptions{MatchCodes: "200,301"},
meta: responseMeta{status: 301, size: 0, words: 0},
want: true,
},
{
name: "mc allowlist drops unlisted 200 already excluded",
opts: DirlistOptions{MatchCodes: "301"},
meta: responseMeta{status: 200, size: 5, words: 1},
want: false,
},
{
name: "fc drops listed code",
opts: DirlistOptions{FilterCodes: "500"},
meta: responseMeta{status: 500, size: 5, words: 1},
want: false,
},
{
// with an explicit -fc and no -mc, the implicit 404/403 filter is not
// added, so a 200 still surfaces
name: "fc leaves others",
opts: DirlistOptions{FilterCodes: "500"},
meta: responseMeta{status: 200, size: 5, words: 1},
want: true,
},
{
name: "fs drops listed size",
opts: DirlistOptions{FilterSizes: "1024"},
meta: responseMeta{status: 200, size: 1024, words: 50},
want: false,
},
{
name: "fw drops listed word count",
opts: DirlistOptions{FilterWords: "7"},
meta: responseMeta{status: 200, size: 40, words: 7},
want: false,
},
{
name: "fr drops body match",
opts: DirlistOptions{FilterRegex: "not found"},
meta: responseMeta{status: 200, size: 9, words: 2},
body: "page not found",
want: false,
},
{
name: "fr keeps non-match",
opts: DirlistOptions{FilterRegex: "not found"},
meta: responseMeta{status: 200, size: 5, words: 1},
body: "welcome",
want: true,
},
{
// filter precedence: -mc would keep it, but a size filter drops it
name: "filter wins over match",
opts: DirlistOptions{MatchCodes: "200", FilterSizes: "12"},
meta: responseMeta{status: 200, size: 12, words: 3},
want: false,
},
}
for _, tt := range tests {
t.Run(tt.name, func(t *testing.T) {
m, err := newMatcher(&tt.opts)
if err != nil {
t.Fatalf("newMatcher: %v", err)
}
if got := m.Matches(tt.meta, []byte(tt.body)); got != tt.want {
t.Errorf("Matches(%+v, %q) = %v, want %v", tt.meta, tt.body, got, tt.want)
}
})
}
}
func TestMatcher_BaselineSuppresses(t *testing.T) {
m, err := newMatcher(&DirlistOptions{})
if err != nil {
t.Fatalf("newMatcher: %v", err)
}
// a calibrated soft-404 shape drops an identical response
m.baselines = []responseMeta{{status: 200, size: 42, words: 5}}
soft := responseMeta{status: 200, size: 42, words: 5}
if m.Matches(soft, nil) {
t.Error("baseline-matching response should be suppressed")
}
// a real page with a different size must still surface
livePage := responseMeta{status: 200, size: 99, words: 12}
if !m.Matches(livePage, nil) {
t.Error("distinct response should not be suppressed by baseline")
}
}
func TestNewMatcher_InvalidFlags(t *testing.T) {
tests := []struct {
name string
opts DirlistOptions
}{
{"bad mc", DirlistOptions{MatchCodes: "abc"}},
{"bad fc", DirlistOptions{FilterCodes: "20x"}},
{"bad fs", DirlistOptions{FilterSizes: "big"}},
{"bad fw", DirlistOptions{FilterWords: "-"}},
{"bad regex", DirlistOptions{FilterRegex: "("}},
}
for _, tt := range tests {
t.Run(tt.name, func(t *testing.T) {
if _, err := newMatcher(&tt.opts); err == nil {
t.Errorf("newMatcher(%+v) expected error, got nil", tt.opts)
}
})
}
}
func TestExpandWords(t *testing.T) {
tests := []struct {
name string
words []string
exts string
want []string
}{
{
name: "no extensions unchanged",
words: []string{"admin", "login"},
exts: "",
want: []string{"admin", "login"},
},
{
name: "appends each extension and keeps bare",
words: []string{"config"},
exts: "php,bak,env",
want: []string{"config", "config.php", "config.bak", "config.env"},
},
{
name: "tolerates leading dot and spaces",
words: []string{"db"},
exts: " .sql , bak ",
want: []string{"db", "db.sql", "db.bak"},
},
}
for _, tt := range tests {
t.Run(tt.name, func(t *testing.T) {
got := expandWords(tt.words, tt.exts)
if !reflect.DeepEqual(got, tt.want) {
t.Errorf("expandWords(%v, %q) = %v, want %v", tt.words, tt.exts, got, tt.want)
}
})
}
}
// softWildcardApp serves a couple of real paths and a catch-all soft-404: every
// unknown path returns a fixed 200 body, the SPA pattern that floods dirlist.
func softWildcardApp() *httptest.Server {
const softBody = "<html><body>app shell - route handled client side</body></html>"
mux := http.NewServeMux()
mux.HandleFunc("/admin", func(w http.ResponseWriter, r *http.Request) {
w.Write([]byte("<html><body>admin control panel dashboard here</body></html>"))
})
mux.HandleFunc("/login", func(w http.ResponseWriter, r *http.Request) {
w.Write([]byte("<html><body>please sign in with your account credentials now</body></html>"))
})
// catch-all: anything else gets the identical soft-404 shell
mux.HandleFunc("/", func(w http.ResponseWriter, r *http.Request) {
if r.URL.Path == "/admin" || r.URL.Path == "/login" {
return
}
w.Write([]byte(softBody))
})
return httptest.NewServer(mux)
}
func TestDirlist_CalibrationSuppressesWildcard(t *testing.T) {
srv := softWildcardApp()
defer srv.Close()
// the wordlist mixes the two real paths with several bogus ones the catch-all
// answers with the soft-404 shell.
dir := t.TempDir()
wordlist := filepath.Join(dir, "words.txt")
if err := os.WriteFile(wordlist, []byte("admin\nlogin\nnope\nbogus\nmissing\n"), 0o600); err != nil {
t.Fatalf("write wordlist: %v", err)
}
// without calibration every bogus path is a soft-404 200 and floods output
noAC, err := Dirlist("small", srv.URL, 5*time.Second, 3, "", DirlistOptions{Wordlist: wordlist})
if err != nil {
t.Fatalf("Dirlist (no -ac): %v", err)
}
if len(noAC) < 5 {
t.Fatalf("expected the wildcard to flood all 5 paths without -ac, got %d", len(noAC))
}
// with -ac the soft-404 baseline is learned and the bogus paths drop
withAC, err := Dirlist("small", srv.URL, 5*time.Second, 3, "", DirlistOptions{
Wordlist: wordlist,
Calibrate: true,
})
if err != nil {
t.Fatalf("Dirlist (-ac): %v", err)
}
got := pathSet(withAC)
if !has(got, "/admin") || !has(got, "/login") {
t.Errorf("real paths admin/login must still surface with -ac, got %v", sortedKeys(got))
}
for _, bogus := range []string{"/nope", "/bogus", "/missing"} {
if has(got, bogus) {
t.Errorf("soft-404 path %s should be suppressed by -ac, got %v", bogus, sortedKeys(got))
}
}
}
func TestDirlist_ExtensionExpansion(t *testing.T) {
// the server only answers config.php; the bare word and other extensions hit
// the catch-all soft-404, so -e must be what surfaces config.php.
const realBody = "<?php // database connection settings live here ?>"
mux := http.NewServeMux()
mux.HandleFunc("/config.php", func(w http.ResponseWriter, r *http.Request) {
w.Write([]byte(realBody))
})
mux.HandleFunc("/", func(w http.ResponseWriter, r *http.Request) {
http.NotFound(w, r) // hard 404 for everything but config.php
})
srv := httptest.NewServer(mux)
defer srv.Close()
dir := t.TempDir()
wordlist := filepath.Join(dir, "words.txt")
if err := os.WriteFile(wordlist, []byte("config\n"), 0o600); err != nil {
t.Fatalf("write wordlist: %v", err)
}
results, err := Dirlist("small", srv.URL, 5*time.Second, 2, "", DirlistOptions{
Wordlist: wordlist,
Extensions: "php,bak",
})
if err != nil {
t.Fatalf("Dirlist: %v", err)
}
got := pathSet(results)
if !has(got, "/config.php") {
t.Errorf("expected -e to surface config.php, got %v", sortedKeys(got))
}
if has(got, "/config") || has(got, "/config.bak") {
t.Errorf("only config.php exists; bare word and .bak are 404s, got %v", sortedKeys(got))
}
}
func TestDirlist_LocalWordlistOverridesSize(t *testing.T) {
// a local -w must be used verbatim and never touch directoryURL; point the
// remote at a sink that fails the test if it's ever hit.
orig := directoryURL
directoryURL = "http://127.0.0.1:0/should-not-be-fetched/"
defer func() { directoryURL = orig }()
mux := http.NewServeMux()
mux.HandleFunc("/secret", func(w http.ResponseWriter, r *http.Request) {
w.Write([]byte("<html>top secret area found</html>"))
})
mux.HandleFunc("/", func(w http.ResponseWriter, r *http.Request) {
http.NotFound(w, r)
})
srv := httptest.NewServer(mux)
defer srv.Close()
dir := t.TempDir()
wordlist := filepath.Join(dir, "custom.txt")
if err := os.WriteFile(wordlist, []byte("secret\nabsent\n"), 0o600); err != nil {
t.Fatalf("write wordlist: %v", err)
}
results, err := Dirlist("large", srv.URL, 5*time.Second, 2, "", DirlistOptions{Wordlist: wordlist})
if err != nil {
t.Fatalf("Dirlist: %v", err)
}
got := pathSet(results)
if !has(got, "/secret") {
t.Errorf("expected the custom wordlist to find /secret, got %v", sortedKeys(got))
}
if has(got, "/absent") {
t.Errorf("/absent is a 404 and should not surface, got %v", sortedKeys(got))
}
}
// pathSet collects each result's url path for membership checks. it reuses the
// package-level sortedKeys (crawl.go) for deterministic failure output.
func pathSet(results DirectoryResults) map[string]struct{} {
set := make(map[string]struct{}, len(results))
for i := 0; i < len(results); i++ {
if idx := strings.Index(results[i].Url, "://"); idx >= 0 {
rest := results[i].Url[idx+len("://"):]
if slash := strings.Index(rest, "/"); slash >= 0 {
set[rest[slash:]] = struct{}{}
continue
}
}
set[results[i].Url] = struct{}{}
}
return set
}
// has is a tiny readability helper for set membership in assertions.
func has(set map[string]struct{}, key string) bool {
_, ok := set[key]
return ok
}
+130 -61
View File
@@ -21,9 +21,11 @@ import (
"time"
charmlog "github.com/charmbracelet/log"
"github.com/dropalldatabases/sif/internal/dnsx"
"github.com/dropalldatabases/sif/internal/httpx"
"github.com/dropalldatabases/sif/internal/logger"
"github.com/dropalldatabases/sif/internal/output"
"github.com/dropalldatabases/sif/internal/pool"
)
// dnsURL is a var so integration tests can repoint it at a fixture.
@@ -33,14 +35,55 @@ var dnsURL = "https://raw.githubusercontent.com/dropalldatabases/sif-runtime/mai
// local server instead of resolving real DNS. nil keeps http.DefaultTransport.
var dnsTransport http.RoundTripper
// hostResolver is the small slice of dnsx the dnslist worker needs: resolve a
// candidate and report whether it's a real, non-wildcard hit.
type hostResolver interface {
Resolve(host string) (bool, error)
}
// newDNSResolver builds the resolver for one run; it's a var so integration
// tests inject a fake that answers without touching real dns. the apex is
// fingerprinted for wildcards before any candidate is checked.
var newDNSResolver = func(apex string, resolvers []string) (hostResolver, error) {
r, err := dnsx.NewResolver(resolvers)
if err != nil {
return nil, fmt.Errorf("dns resolver: %w", err)
}
if err := r.FingerprintWildcard(apex); err != nil {
return nil, fmt.Errorf("wildcard fingerprint: %w", err)
}
return r, nil
}
const (
dnsSmallFile = "subdomains-100.txt"
dnsMediumFile = "subdomains-1000.txt"
dnsBigFile = "subdomains-10000.txt"
)
// Dnslist performs DNS subdomain enumeration on the target domain.
func Dnslist(size string, url string, timeout time.Duration, threads int, logdir string) ([]string, error) {
// dnsScheme labels which url won a subdomain so we don't probe the second
// scheme once the first already counted it.
type dnsScheme string
const (
dnsSchemeHTTP dnsScheme = "http"
dnsSchemeHTTPS dnsScheme = "https"
)
// meaningfulStatus reports whether a probe response is a real "this host
// exists" signal rather than a 404 or a wildcard catch-all redirect. a
// wildcard-DNS host answers every candidate with the same redirect/404, so
// gating on a successful, non-redirect status keeps it from flooding results.
func meaningfulStatus(code int) bool {
return code >= http.StatusOK && code < http.StatusMultipleChoices
}
// Dnslist performs DNS subdomain enumeration on the target domain. each
// candidate is resolved first; only names that actually resolve (and aren't a
// wildcard catch-all) are http-probed, so a big wordlist no longer means a
// http request per dead name.
func Dnslist(size string, url string, timeout time.Duration, threads int, logdir string, resolvers []string) ([]string, error) {
log := output.Module("DNS")
log.Start()
@@ -75,6 +118,15 @@ func Dnslist(size string, url string, timeout time.Duration, threads int, logdir
sanitizedURL := stripScheme(url)
// resolve against dns first, fingerprinting the apex for wildcards so a
// catch-all zone can't flood the probe step. build it once and share across
// the workers - the underlying client is concurrency-safe.
resolver, err := newDNSResolver(sanitizedURL, resolvers)
if err != nil {
log.Error("Error building DNS resolver: %s", err)
return nil, err
}
if logdir != "" {
if err := logger.WriteHeader(sanitizedURL, logdir, size+" subdomain fuzzing"); err != nil {
log.Error("Error creating log file: %v", err)
@@ -88,81 +140,98 @@ func Dnslist(size string, url string, timeout time.Duration, threads int, logdir
if dnsTransport != nil {
client.Transport = dnsTransport
}
// don't chase redirects: a wildcard catch-all that 301s every candidate to
// the same landing page must read as a redirect status, not a 200, so it
// gets gated out instead of counting as a found host.
client.CheckRedirect = func(req *http.Request, via []*http.Request) error {
return http.ErrUseLastResponse
}
progress := output.NewProgress(len(dns), "enumerating")
var wg sync.WaitGroup
var mu sync.Mutex
wg.Add(threads)
urls := make([]string, 0, 64)
for thread := 0; thread < threads; thread++ {
go func(thread int) {
defer wg.Done()
pool.Each(dns, threads, func(domain string) {
progress.Increment(domain)
for i, domain := range dns {
if i%threads != thread {
continue
}
charmlog.Debugf("Looking up: %s", domain)
progress.Increment(domain)
host := domain + "." + sanitizedURL
charmlog.Debugf("Looking up: %s", domain)
// dns gate: skip the http probe entirely for names that don't
// resolve or that a wildcard zone answers. this is the whole point -
// no request per dead candidate.
ok, err := resolver.Resolve(host)
if err != nil {
charmlog.Debugf("resolve %s: %s", host, err)
return
}
if !ok {
return
}
// Check HTTP
httpReq, err := http.NewRequestWithContext(context.TODO(), http.MethodGet, "http://"+domain+"."+sanitizedURL, http.NoBody)
if err != nil {
charmlog.Debugf("Error %s: %s", domain, err)
continue
}
resp, err := client.Do(httpReq)
if err != nil {
charmlog.Debugf("Error %s: %s", domain, err)
} else {
mu.Lock()
urls = append(urls, resp.Request.URL.String())
mu.Unlock()
resp.Body.Close()
// probe http first, then https - but a subdomain is recorded at
// most once. firing both schemes and appending on each is what
// double-counted every host on the old path.
foundURL, scheme := probeSubdomain(client, host)
if foundURL == "" {
return
}
progress.Pause()
log.Success("found: %s.%s [http]", output.Highlight.Render(domain), sanitizedURL)
progress.Resume()
mu.Lock()
urls = append(urls, foundURL)
mu.Unlock()
if logdir != "" {
logger.Write(sanitizedURL, logdir, fmt.Sprintf("[http] %s.%s\n", domain, sanitizedURL))
}
}
progress.Pause()
log.Success("found: %s [%s]", output.Highlight.Render(host), scheme)
progress.Resume()
// Check HTTPS
httpsReq, err := http.NewRequestWithContext(context.TODO(), http.MethodGet, "https://"+domain+"."+sanitizedURL, http.NoBody)
if err != nil {
charmlog.Debugf("Error %s: %s", domain, err)
continue
}
resp, err = client.Do(httpsReq)
if err != nil {
charmlog.Debugf("Error %s: %s", domain, err)
} else {
mu.Lock()
urls = append(urls, resp.Request.URL.String())
mu.Unlock()
resp.Body.Close()
progress.Pause()
log.Success("found: %s.%s [https]", output.Highlight.Render(domain), sanitizedURL)
progress.Resume()
if logdir != "" {
_ = logger.Write(sanitizedURL, logdir, fmt.Sprintf("[https] %s.%s\n", domain, sanitizedURL))
}
}
}
}(thread)
}
wg.Wait()
if logdir != "" {
_ = logger.Write(sanitizedURL, logdir, fmt.Sprintf("[%s] %s\n", scheme, host))
}
})
progress.Done()
log.Complete(len(urls), "found")
return urls, nil
}
// probeSubdomain tries http then https for one host and returns the resolved
// url + winning scheme on the first meaningful hit, or "" if neither scheme
// gave a real signal. trying https only when http didn't already count is the
// per-subdomain dedupe.
func probeSubdomain(client *http.Client, host string) (string, dnsScheme) {
schemes := []struct {
prefix string
label dnsScheme
}{
{"http://", dnsSchemeHTTP},
{"https://", dnsSchemeHTTPS},
}
for i := 0; i < len(schemes); i++ {
req, err := http.NewRequestWithContext(context.TODO(), http.MethodGet, schemes[i].prefix+host, http.NoBody)
if err != nil {
charmlog.Debugf("Error %s: %s", host, err)
continue
}
resp, err := client.Do(req) //nolint:bodyclose // drained and closed via httpx.DrainClose
if err != nil {
charmlog.Debugf("Error %s: %s", host, err)
continue
}
code := resp.StatusCode
resolved := resp.Request.URL.String()
// status/url only; drain so the conn returns to the pool.
httpx.DrainClose(resp)
if meaningfulStatus(code) {
return resolved, schemes[i].label
}
charmlog.Debugf("skip %s [%s]: status %d", host, schemes[i].label, code)
}
return "", ""
}
+98
View File
@@ -0,0 +1,98 @@
/*
·━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━·
: :
: █▀ █ █▀▀ · Blazing-fast pentesting suite :
: ▄█ █ █▀ · BSD 3-Clause License :
: :
: (c) 2022-2026 vmfunc, xyzeva, :
: lunchcat alumni & contributors :
: :
·━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━·
*/
package scan
import (
"net/http"
"net/http/httptest"
"strings"
"sync/atomic"
"testing"
"time"
)
func TestMeaningfulStatus(t *testing.T) {
tests := []struct {
name string
code int
want bool
}{
{"ok counts", http.StatusOK, true},
{"204 counts", http.StatusNoContent, true},
{"301 catch-all redirect dropped", http.StatusMovedPermanently, false},
{"302 catch-all redirect dropped", http.StatusFound, false},
{"404 dropped", http.StatusNotFound, false},
{"500 dropped", http.StatusInternalServerError, false},
}
for _, tt := range tests {
t.Run(tt.name, func(t *testing.T) {
if got := meaningfulStatus(tt.code); got != tt.want {
t.Errorf("meaningfulStatus(%d) = %v, want %v", tt.code, got, tt.want)
}
})
}
}
// a host that answers 200 over http should count exactly once, not once per
// scheme - the old path appended on both http and https.
func TestProbeSubdomain_DedupesAcrossSchemes(t *testing.T) {
var hits int32
srv := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
atomic.AddInt32(&hits, 1)
w.WriteHeader(http.StatusOK)
}))
defer srv.Close()
host := strings.TrimPrefix(srv.URL, "http://")
client := &http.Client{Timeout: 5 * time.Second}
url, scheme := probeSubdomain(client, host)
if url == "" {
t.Fatal("expected http probe to count the host")
}
if scheme != dnsSchemeHTTP {
t.Errorf("expected http scheme to win, got %q", scheme)
}
// http already counted, so https must not be tried - one request total.
if got := atomic.LoadInt32(&hits); got != 1 {
t.Errorf("expected exactly 1 probe request, got %d", got)
}
}
// a wildcard catch-all that 404s (or 301s) every candidate must not be reported
// as found - that's the flood the gating closes.
func TestProbeSubdomain_WildcardCatchAllNotFound(t *testing.T) {
for _, code := range []int{http.StatusNotFound, http.StatusMovedPermanently} {
srv := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
if code == http.StatusMovedPermanently {
w.Header().Set("Location", "https://catch-all.example/")
}
w.WriteHeader(code)
}))
host := strings.TrimPrefix(srv.URL, "http://")
client := &http.Client{
Timeout: 5 * time.Second,
CheckRedirect: func(req *http.Request, via []*http.Request) error {
return http.ErrUseLastResponse
},
}
url, _ := probeSubdomain(client, host)
if url != "" {
t.Errorf("status %d should not count as found, got %q", code, url)
}
srv.Close()
}
}
+24 -37
View File
@@ -28,6 +28,7 @@ import (
"github.com/dropalldatabases/sif/internal/httpx"
"github.com/dropalldatabases/sif/internal/logger"
"github.com/dropalldatabases/sif/internal/output"
"github.com/dropalldatabases/sif/internal/pool"
googlesearch "github.com/rocketlaunchr/google-search"
)
@@ -92,47 +93,33 @@ func Dork(url string, timeout time.Duration, threads int, logdir string) ([]Dork
}
// util.InitProgressBar()
var wg sync.WaitGroup
var mu sync.Mutex
wg.Add(threads)
dorkResults := []DorkResult{}
for thread := 0; thread < threads; thread++ {
go func(thread int) {
defer wg.Done()
for i, dork := range dorks {
if i%threads != thread {
continue
}
results, err := googlesearch.Search(context.TODO(), fmt.Sprintf("%s %s", dork, sanitizedURL))
if err != nil {
log.Debugf("error searching for dork %s: %v", dork, err)
continue
}
if len(results) > 0 {
spin.Stop()
output.Success("%s dork results found for dork %s", output.Status.Render(strconv.Itoa(len(results))), output.Highlight.Render(dork))
spin.Start()
if logdir != "" {
_ = logger.Write(sanitizedURL, logdir, strconv.Itoa(len(results))+" dork results found for dork ["+dork+"]\n")
}
result := DorkResult{
Url: dork,
Count: len(results),
}
mu.Lock()
dorkResults = append(dorkResults, result)
mu.Unlock()
}
pool.Each(dorks, threads, func(dork string) {
results, err := googlesearch.Search(context.TODO(), fmt.Sprintf("%s %s", dork, sanitizedURL))
if err != nil {
log.Debugf("error searching for dork %s: %v", dork, err)
return
}
if len(results) > 0 {
spin.Stop()
output.Success("%s dork results found for dork %s", output.Status.Render(strconv.Itoa(len(results))), output.Highlight.Render(dork))
spin.Start()
if logdir != "" {
_ = logger.Write(sanitizedURL, logdir, strconv.Itoa(len(results))+" dork results found for dork ["+dork+"]\n")
}
}(thread)
}
wg.Wait()
result := DorkResult{
Url: dork,
Count: len(results),
}
mu.Lock()
dorkResults = append(dorkResults, result)
mu.Unlock()
}
})
spin.Stop()
output.ScanComplete("URL dorking", len(dorkResults), "found")
+254
View File
@@ -0,0 +1,254 @@
/*
·━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━·
: :
: █▀ █ █▀▀ · Blazing-fast pentesting suite :
: ▄█ █ █▀ · BSD 3-Clause License :
: :
: (c) 2022-2026 vmfunc, xyzeva, :
: lunchcat alumni & contributors :
: :
·━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━·
*/
package scan
import (
"context"
"encoding/base64"
"fmt"
"io"
"net/http"
"regexp"
"strings"
"time"
"github.com/dropalldatabases/sif/internal/httpx"
"github.com/dropalldatabases/sif/internal/logger"
"github.com/dropalldatabases/sif/internal/output"
"github.com/twmb/murmur3"
)
// FaviconResult is the computed shodan-style favicon hash plus the pivot query
// and any matched tech.
type FaviconResult struct {
FaviconURL string `json:"favicon_url"` // where the icon was fetched
Hash int32 `json:"hash"` // shodan mmh3 hash (signed int32)
Tech string `json:"tech"` // matched technology, empty when unknown
ShodanQ string `json:"shodan_query"`
}
// faviconBodyReadCap bounds the icon read. real favicons are tens of kilobytes;
// a megabyte ceiling covers oversized ones without letting a hostile endpoint
// stream forever.
const faviconBodyReadCap = 1 << 20
// b64LineLen is python's base64.encodebytes line width. mmh3/shodan hash the
// chunked base64 (newline every 76 chars, trailing newline), so we must wrap at
// exactly this width to land on the same hash.
const b64LineLen = 76
// faviconLinkRegex pulls the href off a <link rel="...icon..."> tag so we can
// fall back to a declared icon when /favicon.ico is absent.
var faviconLinkRegex = regexp.MustCompile(`(?i)<link[^>]+rel=["'][^"']*icon[^"']*["'][^>]*>`)
// faviconHrefRegex extracts the href attribute value from a matched link tag.
var faviconHrefRegex = regexp.MustCompile(`(?i)href=["']([^"']+)["']`)
// faviconHashes maps a known shodan favicon hash to the tech that ships it.
// these are stable default icons for panels/frameworks/c2; a hit is a strong
// fingerprint. kept small on purpose - high-signal defaults, not an exhaustive db.
var faviconHashes = map[int32]string{
116323821: "Apache Tomcat",
81586312: "Spring Boot (default whitelabel)",
-235701012: "Jenkins",
-1255347784: "GitLab",
1278322581: "Grafana",
743365239: "Kibana",
-1462443472: "phpMyAdmin",
999357577: "Cobalt Strike (default beacon)",
-1521704893: "Metasploit",
-1893514588: "Gitea",
}
// Favicon fetches the target's favicon, computes the shodan mmh3 hash and matches
// it against the bundled fingerprint map.
func Favicon(targetURL string, timeout time.Duration, logdir string) (*FaviconResult, error) {
log := output.Module("FAVICON")
log.Start()
sanitizedURL := stripScheme(targetURL)
if logdir != "" {
if err := logger.WriteHeader(sanitizedURL, logdir, "Favicon hash fingerprint"); err != nil {
log.Error("error creating log file: %v", err)
return nil, fmt.Errorf("create favicon log: %w", err)
}
}
client := httpx.Client(timeout)
base := strings.TrimRight(targetURL, "/")
iconURL, data, err := fetchFavicon(client, base)
if err != nil {
log.Info("no favicon found: %v", err)
log.Complete(0, "found")
return nil, nil //nolint:nilnil // a missing favicon is not an error
}
hash := FaviconHash(data)
result := &FaviconResult{
FaviconURL: iconURL,
Hash: hash,
Tech: faviconHashes[hash],
ShodanQ: fmt.Sprintf("http.favicon.hash:%d", hash),
}
if result.Tech != "" {
log.Warn("favicon hash %d matches %s", hash, output.Highlight.Render(result.Tech))
} else {
log.Info("favicon hash %d (no fingerprint match)", hash)
}
log.Info("shodan pivot: %s", output.Highlight.Render(result.ShodanQ))
if logdir != "" {
_ = logger.Write(sanitizedURL, logdir,
fmt.Sprintf("Favicon %s hash=%d tech=%q query=%s\n", iconURL, hash, result.Tech, result.ShodanQ))
}
log.Complete(1, "hashed")
return result, nil
}
// fetchFavicon tries /favicon.ico first, then the <link rel=icon> declared in the
// homepage html. it returns the url it pulled the bytes from so the report shows
// exactly which icon was hashed.
func fetchFavicon(client *http.Client, base string) (string, []byte, error) {
iconURL := base + "/favicon.ico"
if data, err := getFaviconBytes(client, iconURL); err == nil {
return iconURL, data, nil
}
// no /favicon.ico; parse the homepage for a declared icon link.
href, err := declaredFaviconHref(client, base)
if err != nil {
return "", nil, err
}
iconURL = resolveFaviconURL(base, href)
data, err := getFaviconBytes(client, iconURL)
if err != nil {
return "", nil, err
}
return iconURL, data, nil
}
// getFaviconBytes GETs an icon url and returns the body, erroring on a non-200 or
// an empty body so a soft-404 html page isn't hashed as if it were an icon.
func getFaviconBytes(client *http.Client, iconURL string) ([]byte, error) {
req, err := http.NewRequestWithContext(context.TODO(), http.MethodGet, iconURL, http.NoBody)
if err != nil {
return nil, fmt.Errorf("build favicon request: %w", err)
}
resp, err := client.Do(req)
if err != nil {
return nil, fmt.Errorf("fetch favicon: %w", err)
}
defer resp.Body.Close()
if resp.StatusCode != http.StatusOK {
return nil, fmt.Errorf("favicon status %d", resp.StatusCode)
}
data, err := io.ReadAll(io.LimitReader(resp.Body, faviconBodyReadCap))
if err != nil {
return nil, fmt.Errorf("read favicon: %w", err)
}
if len(data) == 0 {
return nil, fmt.Errorf("empty favicon body")
}
return data, nil
}
// declaredFaviconHref fetches the homepage and extracts the href of the first
// <link rel="...icon..."> tag.
func declaredFaviconHref(client *http.Client, base string) (string, error) {
req, err := http.NewRequestWithContext(context.TODO(), http.MethodGet, base, http.NoBody)
if err != nil {
return "", fmt.Errorf("build homepage request: %w", err)
}
resp, err := client.Do(req)
if err != nil {
return "", fmt.Errorf("fetch homepage: %w", err)
}
defer resp.Body.Close()
body, err := io.ReadAll(io.LimitReader(resp.Body, faviconBodyReadCap))
if err != nil {
return "", fmt.Errorf("read homepage: %w", err)
}
link := faviconLinkRegex.Find(body)
if link == nil {
return "", fmt.Errorf("no favicon link in homepage")
}
href := faviconHrefRegex.FindSubmatch(link)
if href == nil {
return "", fmt.Errorf("favicon link has no href")
}
return string(href[1]), nil
}
// resolveFaviconURL turns a possibly-relative href into an absolute url against
// the target base. an absolute href is returned as-is.
func resolveFaviconURL(base, href string) string {
if strings.HasPrefix(href, "http://") || strings.HasPrefix(href, "https://") {
return href
}
if strings.HasPrefix(href, "//") {
// scheme-relative; inherit the base scheme.
scheme := "https:"
if strings.HasPrefix(base, "http://") {
scheme = "http:"
}
return scheme + href
}
if strings.HasPrefix(href, "/") {
return base + href
}
return base + "/" + href
}
// FaviconHash computes shodan's favicon hash: murmur3 32-bit over the python
// base64.encodebytes encoding of the raw icon (newline every 76 chars plus a
// trailing newline), reinterpreted as a signed int32. the chunking and the sign
// are both load-bearing - shodan stores the value python's mmh3.hash() returns,
// which is signed, over the wrapped base64, not the raw bytes. the golden test
// pins this exactly.
func FaviconHash(data []byte) int32 {
encoded := encodeFaviconBase64(data)
return int32(murmur3.Sum32(encoded)) //nolint:gosec // shodan stores the signed reinterpretation on purpose
}
// encodeFaviconBase64 mirrors python's base64.encodebytes: standard base64 with
// a newline inserted every 76 output characters and a trailing newline. this is
// the exact byte stream shodan feeds to mmh3, so it must match byte-for-byte.
func encodeFaviconBase64(data []byte) []byte {
raw := base64.StdEncoding.EncodeToString(data)
var b strings.Builder
// final size: the base64 body plus one '\n' per (full or partial) 76-char
// line. preallocate so the builder never regrows mid-loop.
b.Grow(len(raw) + len(raw)/b64LineLen + 1)
for i := 0; i < len(raw); i += b64LineLen {
end := i + b64LineLen
if end > len(raw) {
end = len(raw)
}
b.WriteString(raw[i:end])
b.WriteByte('\n')
}
return []byte(b.String())
}
// ResultType identifies favicon findings for the result registry.
func (r *FaviconResult) ResultType() string { return "favicon" }
var _ ScanResult = (*FaviconResult)(nil)
+160
View File
@@ -0,0 +1,160 @@
/*
·━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━·
: :
: █▀ █ █▀▀ · Blazing-fast pentesting suite :
: ▄█ █ █▀ · BSD 3-Clause License :
: :
: (c) 2022-2026 vmfunc, xyzeva, :
: lunchcat alumni & contributors :
: :
·━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━·
*/
package scan
import (
"net/http"
"net/http/httptest"
"strings"
"testing"
"time"
)
// goldenFaviconBytes is a fixed payload long enough to span multiple base64
// lines, so the python-style 76-char chunking is actually exercised by the hash.
var goldenFaviconBytes = []byte(strings.Repeat("sif-favicon-golden-test-bytes-", 8))
// goldenFaviconHash is the shodan mmh3 hash of goldenFaviconBytes. it is pinned:
// the value comes from feeding the python base64.encodebytes byte stream (newline
// every 76 chars + trailing newline) through murmur3-32 and reinterpreting the
// result as a signed int32 - exactly what shodan stores. if the chunking or the
// signedness regress, this number changes and the test fails.
const goldenFaviconHash int32 = -1554620260
// goldenHelloHash pins a short single-line case so a regression in the trailing
// newline (which the small case still has) is caught independently.
const goldenHelloHash int32 = 1155597304
func TestFaviconHash_Golden(t *testing.T) {
tests := []struct {
name string
in []byte
want int32
}{
{name: "multi-line fixture", in: goldenFaviconBytes, want: goldenFaviconHash},
{name: "single-line hello", in: []byte("hello"), want: goldenHelloHash},
}
for _, tt := range tests {
t.Run(tt.name, func(t *testing.T) {
got := FaviconHash(tt.in)
if got != tt.want {
t.Errorf("FaviconHash = %d, want %d", got, tt.want)
}
})
}
}
// TestFaviconBase64Chunking pins the encode step against python's
// base64.encodebytes: a 50-byte input encodes to >76 base64 chars, so it must
// wrap into two newline-terminated lines.
func TestFaviconBase64Chunking(t *testing.T) {
in := []byte(strings.Repeat("A", 60)) // 60 bytes -> 80 base64 chars -> two lines
got := string(encodeFaviconBase64(in))
lines := strings.Split(strings.TrimRight(got, "\n"), "\n")
if len(lines) != 2 {
t.Fatalf("expected 2 wrapped lines, got %d: %q", len(lines), got)
}
if len(lines[0]) != b64LineLen {
t.Errorf("first line = %d chars, want %d", len(lines[0]), b64LineLen)
}
if !strings.HasSuffix(got, "\n") {
t.Errorf("encoding must end in a trailing newline, got %q", got)
}
}
// fixtureFaviconServer serves the golden bytes at /favicon.ico.
func fixtureFaviconServer() *httptest.Server {
return httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
if r.URL.Path == "/favicon.ico" {
w.Header().Set("Content-Type", "image/x-icon")
_, _ = w.Write(goldenFaviconBytes)
return
}
w.WriteHeader(http.StatusNotFound)
}))
}
func TestFavicon_FetchAndHash(t *testing.T) {
srv := fixtureFaviconServer()
defer srv.Close()
result, err := Favicon(srv.URL, 5*time.Second, "")
if err != nil {
t.Fatalf("Favicon: %v", err)
}
if result == nil {
t.Fatal("expected a favicon result, got nil")
}
if result.Hash != goldenFaviconHash {
t.Errorf("Hash = %d, want %d", result.Hash, goldenFaviconHash)
}
wantQ := "http.favicon.hash:-1554620260"
if result.ShodanQ != wantQ {
t.Errorf("ShodanQ = %q, want %q", result.ShodanQ, wantQ)
}
}
// TestFavicon_LinkFallback covers the <link rel=icon> path when /favicon.ico is
// absent: the homepage points at /static/icon.png and that's what gets hashed.
func TestFavicon_LinkFallback(t *testing.T) {
srv := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
switch r.URL.Path {
case "/favicon.ico":
w.WriteHeader(http.StatusNotFound)
case "/static/icon.png":
_, _ = w.Write(goldenFaviconBytes)
default:
_, _ = w.Write([]byte(`<html><head><link rel="icon" href="/static/icon.png"></head></html>`))
}
}))
defer srv.Close()
result, err := Favicon(srv.URL, 5*time.Second, "")
if err != nil {
t.Fatalf("Favicon: %v", err)
}
if result == nil {
t.Fatal("expected a favicon result via link fallback, got nil")
}
if !strings.HasSuffix(result.FaviconURL, "/static/icon.png") {
t.Errorf("FaviconURL = %q, want it to end in /static/icon.png", result.FaviconURL)
}
if result.Hash != goldenFaviconHash {
t.Errorf("Hash = %d, want %d", result.Hash, goldenFaviconHash)
}
}
// TestFavicon_NoIcon confirms a target with no favicon at all yields no result
// and no error.
func TestFavicon_NoIcon(t *testing.T) {
srv := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, _ *http.Request) {
w.WriteHeader(http.StatusNotFound)
}))
defer srv.Close()
result, err := Favicon(srv.URL, 5*time.Second, "")
if err != nil {
t.Fatalf("Favicon: %v", err)
}
if result != nil {
t.Errorf("expected nil result for missing favicon, got %+v", result)
}
}
func TestFaviconResult_ResultType(t *testing.T) {
r := &FaviconResult{}
if r.ResultType() != "favicon" {
t.Errorf("expected result type 'favicon', got %q", r.ResultType())
}
}
+52 -1
View File
@@ -14,6 +14,57 @@ package frameworks
import "testing"
// the detector usually reports "unknown"; the version dug out of the body must
// win so the cve lookup runs against a concrete version instead of "unknown".
func TestResolveVersion(t *testing.T) {
tests := []struct {
name string
detector string
extracted string
want string
}{
{"detector concrete wins", "9.0.0", "8.4.1", "9.0.0"},
{"unknown detector falls back to extracted", "unknown", "8.4.1", "8.4.1"},
{"empty detector falls back to extracted", "", "8.4.1", "8.4.1"},
{"both unknown stays unknown", "unknown", "unknown", "unknown"},
{"both empty/unknown stays unknown", "", "", "unknown"},
}
for _, tt := range tests {
t.Run(tt.name, func(t *testing.T) {
if got := resolveVersion(tt.detector, tt.extracted); got != tt.want {
t.Errorf("resolveVersion(%q, %q) = %q, want %q", tt.detector, tt.extracted, got, tt.want)
}
})
}
}
// the regression itself: with the detector reporting "unknown" but a real
// version extractable from the body, the cve lookup must use the extracted
// version and surface the matching CVE - the old path looked up "unknown" and
// missed it.
func TestResolveVersionFeedsCVELookup(t *testing.T) {
const body = "Laravel 8.4.1"
// extractor pulls the concrete version out of the body...
extracted := ExtractVersionOptimized(body, "Laravel").Version
if extracted != "8.4.1" {
t.Fatalf("expected extracted version 8.4.1, got %q", extracted)
}
// ...and looking "unknown" up finds nothing, proving the old behavior missed it.
if cves, _ := getVulnerabilities("Laravel", "unknown"); len(cves) != 0 {
t.Fatalf("expected no CVEs for unknown version, got %v", cves)
}
// the reconciled version feeds the lookup and the CVE shows up.
version := resolveVersion("unknown", extracted)
cves, _ := getVulnerabilities("Laravel", version)
if len(cves) == 0 {
t.Errorf("expected Laravel %s to surface a CVE, got none", version)
}
}
func TestVersionAffected(t *testing.T) {
tests := []struct {
version string
@@ -23,7 +74,7 @@ func TestVersionAffected(t *testing.T) {
{"4.2", "4.2", true},
{"4.2.1", "4.2", true},
{"4.2.13", "4.2", true},
{"4.20", "4.2", false}, // the boundary bug: 4.20 is not a 4.2.x release
{"4.20", "4.2", false}, // the boundary bug: 4.20 is not a 4.2.x release
{"4.20.0", "4.2", false},
{"5.0", "4.2", false},
}
+28 -5
View File
@@ -118,17 +118,22 @@ func DetectFramework(url string, timeout time.Duration, logdir string) (*Framewo
return nil, nil //nolint:nilnil // no framework detected is not an error
}
// Get version match details
// Get version match details. the detector's own best.version is often
// "unknown" (it only fingerprints the framework, not always the version),
// while ExtractVersionOptimized digs the real version out of the body. prefer
// that for both the reported version and the cve lookup, otherwise CVEs that
// only match a concrete version are silently missed.
versionMatch := ExtractVersionOptimized(bodyStr, best.name)
cves, suggestions := getVulnerabilities(best.name, best.version)
version := resolveVersion(best.version, versionMatch.Version)
cves, suggestions := getVulnerabilities(best.name, version)
result := NewFrameworkResult(best.name, best.version, best.confidence, versionMatch.Confidence)
result := NewFrameworkResult(best.name, version, best.confidence, versionMatch.Confidence)
result.WithVulnerabilities(cves, suggestions)
// Log results
if logdir != "" {
logEntry := fmt.Sprintf("Detected framework: %s (version: %s, confidence: %.2f, version_confidence: %.2f)\n",
best.name, best.version, best.confidence, versionMatch.Confidence)
best.name, version, best.confidence, versionMatch.Confidence)
if len(cves) > 0 {
logEntry += fmt.Sprintf(" Risk Level: %s\n", result.RiskLevel)
logEntry += fmt.Sprintf(" CVEs: %v\n", cves)
@@ -138,7 +143,7 @@ func DetectFramework(url string, timeout time.Duration, logdir string) (*Framewo
}
log.Success("Detected %s framework (version: %s, confidence: %.2f)",
output.Highlight.Render(best.name), best.version, best.confidence)
output.Highlight.Render(best.name), version, best.confidence)
if versionMatch.Confidence > 0 {
charmlog.Debugf("Version detected from: %s (confidence: %.2f)",
@@ -160,6 +165,24 @@ func DetectFramework(url string, timeout time.Duration, logdir string) (*Framewo
return result, nil
}
// unknownVersion is the sentinel both detectors and the version extractor emit
// when no concrete version could be read from the response.
const unknownVersion = "unknown"
// resolveVersion picks the version to report and look CVEs up against. the
// detector's own value wins when it's concrete; otherwise we fall back to the
// version dug out of the body by ExtractVersionOptimized. either being
// "unknown"/empty means "no info", not a real version.
func resolveVersion(detectorVersion, extractedVersion string) string {
if detectorVersion != "" && detectorVersion != unknownVersion {
return detectorVersion
}
if extractedVersion != "" && extractedVersion != unknownVersion {
return extractedVersion
}
return unknownVersion
}
// getVulnerabilities returns CVEs and recommendations for a framework version.
func getVulnerabilities(framework, version string) ([]string, []string) {
entries, exists := knownCVEs[framework]
+27 -38
View File
@@ -25,6 +25,7 @@ import (
"github.com/dropalldatabases/sif/internal/httpx"
"github.com/dropalldatabases/sif/internal/logger"
"github.com/dropalldatabases/sif/internal/output"
"github.com/dropalldatabases/sif/internal/pool"
)
// gitURL is a var so integration tests can repoint it at a fixture.
@@ -71,49 +72,37 @@ func Git(url string, timeout time.Duration, threads int, logdir string) ([]strin
gitUrls = append(gitUrls, scanner.Text())
}
var wg sync.WaitGroup
var mu sync.Mutex
wg.Add(threads)
foundUrls := []string{}
for thread := 0; thread < threads; thread++ {
go func(thread int) {
defer wg.Done()
pool.Each(gitUrls, threads, func(repourl string) {
charmlog.Debugf("%s", repourl)
gitReq, err := http.NewRequestWithContext(context.TODO(), http.MethodGet, url+"/"+repourl, http.NoBody)
if err != nil {
charmlog.Debugf("Error creating request for %s: %s", repourl, err)
return
}
resp, err := client.Do(gitReq) //nolint:bodyclose // drained and closed via httpx.DrainClose
if err != nil {
charmlog.Debugf("Error %s: %s", repourl, err)
return
}
for i, repourl := range gitUrls {
if i%threads != thread {
continue
}
charmlog.Debugf("%s", repourl)
gitReq, err := http.NewRequestWithContext(context.TODO(), http.MethodGet, url+"/"+repourl, http.NoBody)
if err != nil {
charmlog.Debugf("Error creating request for %s: %s", repourl, err)
continue
}
resp, err := client.Do(gitReq)
if err != nil {
charmlog.Debugf("Error %s: %s", repourl, err)
continue
}
if resp.StatusCode == 200 && !strings.HasPrefix(resp.Header.Get("Content-Type"), "text/html") {
spin.Stop()
log.Success("Git found at %s [%s]", output.Highlight.Render(repourl), output.Status.Render(strconv.Itoa(resp.StatusCode)))
spin.Start()
if logdir != "" {
logger.Write(sanitizedURL, logdir, strconv.Itoa(resp.StatusCode)+" git found at ["+repourl+"]\n")
}
mu.Lock()
foundUrls = append(foundUrls, resp.Request.URL.String())
mu.Unlock()
}
resp.Body.Close()
if resp.StatusCode == 200 && !strings.HasPrefix(resp.Header.Get("Content-Type"), "text/html") {
spin.Stop()
log.Success("Git found at %s [%s]", output.Highlight.Render(repourl), output.Status.Render(strconv.Itoa(resp.StatusCode)))
spin.Start()
if logdir != "" {
logger.Write(sanitizedURL, logdir, strconv.Itoa(resp.StatusCode)+" git found at ["+repourl+"]\n")
}
}(thread)
}
wg.Wait()
mu.Lock()
foundUrls = append(foundUrls, resp.Request.URL.String())
mu.Unlock()
}
// status/headers only; drain so the conn returns to the pool.
httpx.DrainClose(resp)
})
spin.Stop()
log.Complete(len(foundUrls), "found")
+3 -2
View File
@@ -46,11 +46,12 @@ func Headers(url string, timeout time.Duration, logdir string) ([]HeaderResult,
if err != nil {
return nil, err
}
resp, err := client.Do(req)
resp, err := client.Do(req) //nolint:bodyclose // drained and closed via httpx.DrainClose
if err != nil {
return nil, err
}
defer resp.Body.Close()
// header-only scan: drain on close so the conn is returned to the pool.
defer httpx.DrainClose(resp)
var results []HeaderResult
+97 -2
View File
@@ -65,6 +65,32 @@ func newVulnApp() *httptest.Server {
w.Write([]byte("<title>phpMyAdmin</title>"))
})
// reflecting-origin endpoint for the cors probe
mux.HandleFunc("/cors", func(w http.ResponseWriter, r *http.Request) {
if origin := r.Header.Get("Origin"); origin != "" {
w.Header().Set("Access-Control-Allow-Origin", origin)
w.Header().Set("Access-Control-Allow-Credentials", "true")
}
w.WriteHeader(http.StatusOK)
})
// open-redirect endpoint: echoes the next param into Location
mux.HandleFunc("/redirect", func(w http.ResponseWriter, r *http.Request) {
if next := r.URL.Query().Get("next"); next != "" {
w.Header().Set("Location", next)
w.WriteHeader(http.StatusFound)
return
}
w.WriteHeader(http.StatusOK)
})
// reflecting endpoint for the xss probe: echoes q raw into html text
mux.HandleFunc("/xss", func(w http.ResponseWriter, r *http.Request) {
w.Header().Set("Content-Type", "text/html")
//nolint:gosec // deliberate reflected-xss fixture for the probe under test
w.Write([]byte("<html><body><div>" + r.URL.Query().Get("q") + "</div></body></html>"))
})
// homepage doubles as the cms fingerprint and the lfi sink
mux.HandleFunc("/", func(w http.ResponseWriter, r *http.Request) {
if r.URL.Path != "/" {
@@ -108,7 +134,7 @@ func TestIntegrationDirlist(t *testing.T) {
directoryURL = srv.URL + "/"
defer func() { directoryURL = orig }()
results, err := Dirlist("small", srv.URL, 5*time.Second, 3, "")
results, err := Dirlist("small", srv.URL, 5*time.Second, 3, "", DirlistOptions{})
if err != nil {
t.Fatalf("Dirlist: %v", err)
}
@@ -180,6 +206,61 @@ func TestIntegrationLFI(t *testing.T) {
}
}
func TestIntegrationCORS(t *testing.T) {
srv := newVulnApp()
defer srv.Close()
result, err := CORS(srv.URL+"/cors", 5*time.Second, 3, "")
if err != nil {
t.Fatalf("CORS: %v", err)
}
if result == nil || len(result.Findings) == 0 {
t.Fatalf("expected a cors finding from the reflecting endpoint, got %+v", result)
}
}
func TestIntegrationRedirect(t *testing.T) {
srv := newVulnApp()
defer srv.Close()
result, err := Redirect(srv.URL+"/redirect", 5*time.Second, 4, "")
if err != nil {
t.Fatalf("Redirect: %v", err)
}
if result == nil || len(result.Findings) == 0 {
t.Fatalf("expected an open-redirect finding from the next sink, got %+v", result)
}
}
func TestIntegrationXSS(t *testing.T) {
srv := newVulnApp()
defer srv.Close()
result, err := XSS(srv.URL+"/xss", 5*time.Second, 4, "")
if err != nil {
t.Fatalf("XSS: %v", err)
}
if result == nil || len(result.Findings) == 0 {
t.Fatalf("expected a reflected-xss finding from the q sink, got %+v", result)
}
}
func TestIntegrationProbe(t *testing.T) {
srv := newVulnApp()
defer srv.Close()
result, err := Probe(srv.URL, 5*time.Second, "")
if err != nil {
t.Fatalf("Probe: %v", err)
}
if result == nil || !result.Alive {
t.Fatalf("expected the vuln app to be alive, got %+v", result)
}
if result.StatusCode != http.StatusOK {
t.Errorf("expected 200 from the homepage, got %d", result.StatusCode)
}
}
func TestIntegrationPorts(t *testing.T) {
// a real listener stands in for an open port; a tiny server hands its number
// to Ports via the commonPorts wordlist.
@@ -343,7 +424,15 @@ func TestIntegrationDnslist(t *testing.T) {
}
defer func() { dnsTransport = origTr }()
found, err := Dnslist("small", "http://example.com", 5*time.Second, 2, "")
// inject a fake resolver so the run never touches real dns: every candidate
// resolves, nothing is wildcard, so all wordlist names reach the probe step.
origResolver := newDNSResolver
newDNSResolver = func(_ string, _ []string) (hostResolver, error) {
return resolveAllStub{}, nil
}
defer func() { newDNSResolver = origResolver }()
found, err := Dnslist("small", "http://example.com", 5*time.Second, 2, "", nil)
if err != nil {
t.Fatalf("Dnslist: %v", err)
}
@@ -354,6 +443,12 @@ func TestIntegrationDnslist(t *testing.T) {
}
}
// resolveAllStub answers every host as a real, non-wildcard hit so the dns gate
// is a pass-through and the probe step gets the full wordlist.
type resolveAllStub struct{}
func (resolveAllStub) Resolve(string) (bool, error) { return true, nil }
func contains(s []string, v string) bool {
for i := 0; i < len(s); i++ {
if s[i] == v {
+128
View File
@@ -0,0 +1,128 @@
/*
·━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━·
: :
: █▀ █ █▀▀ · Blazing-fast pentesting suite :
: ▄█ █ █▀ · BSD 3-Clause License :
: :
: (c) 2022-2026 vmfunc, xyzeva, :
: lunchcat alumni & contributors :
: :
·━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━·
*/
package js
import (
"net/url"
"regexp"
"slices"
"strings"
urlutil "github.com/projectdiscovery/utils/url"
)
// endpointRegex is a linkfinder-style matcher for quoted paths and urls inside
// js: full http(s) urls, root-relative (/api/...) and dotted-relative paths,
// plus bare api-ish words with an extension. the inner alternation lives in a
// single capture group so FindAllStringSubmatch hands back just the value.
var endpointRegex = regexp.MustCompile(`["'\x60]` +
`(` +
`(?:https?:)?//[^\s"'\x60]{2,}` + // protocol-relative or absolute url
`|` +
`/[A-Za-z0-9_\-./]+(?:\?[^\s"'\x60]*)?` + // root-relative path
`|` +
`\.{1,2}/[A-Za-z0-9_\-./]+(?:\?[^\s"'\x60]*)?` + // dotted-relative path
`)` +
`["'\x60]`)
// shortest thing we'll treat as an endpoint; below this it's almost always
// noise like "/" or a single slash-prefixed letter.
const minEndpointLen = 3
// mime types slip through the path regex (text/html, application/json, ...) but
// are never endpoints, so they're filtered out by their top-level type.
var mimePrefixes = []string{
"text/", "image/", "audio/", "video/", "font/",
"application/", "multipart/", "model/", "message/",
}
// ExtractEndpoints pulls candidate paths and urls out of a script body, dedupes
// them, drops obvious noise, and resolves relatives against baseURL so callers
// get absolute targets where possible. a baseURL that won't parse just leaves
// relatives as-is rather than failing the whole scan.
func ExtractEndpoints(content, baseURL string) []string {
groups := endpointRegex.FindAllStringSubmatch(content, -1)
if len(groups) == 0 {
return nil
}
base, baseErr := urlutil.Parse(baseURL)
endpoints := make([]string, 0, len(groups))
seen := make(map[string]struct{}, len(groups))
for i := 0; i < len(groups); i++ {
candidate := strings.TrimSpace(groups[i][1])
if !isEndpoint(candidate) {
continue
}
resolved := candidate
// only relatives need resolving, and only if the base parsed cleanly.
if baseErr == nil && base.URL != nil && isRelative(candidate) {
resolved = resolveRelative(base.URL, candidate)
}
if _, ok := seen[resolved]; ok {
continue
}
seen[resolved] = struct{}{}
endpoints = append(endpoints, resolved)
}
slices.Sort(endpoints)
return endpoints
}
// isEndpoint filters out the junk that the broad regex inevitably catches:
// too-short fragments, mime types, and single dotted words with no path.
func isEndpoint(s string) bool {
if len(s) < minEndpointLen {
return false
}
lower := strings.ToLower(s)
for i := 0; i < len(mimePrefixes); i++ {
// a mime type is "type/subtype" with no further path; an api route like
// /application/users has a leading slash, so anchor on the bare prefix.
if strings.HasPrefix(lower, mimePrefixes[i]) && !strings.HasPrefix(lower, "/") {
return false
}
}
// reject "word" or "a.b" with no slash at all: not a path, just a token.
if !strings.Contains(s, "/") {
return false
}
return true
}
// isRelative reports whether candidate lacks a scheme/host and so needs the
// base url to become absolute. protocol-relative (//host) and absolute urls
// are left untouched.
func isRelative(candidate string) bool {
if strings.HasPrefix(candidate, "//") {
return false
}
return !strings.HasPrefix(candidate, "http://") && !strings.HasPrefix(candidate, "https://")
}
// resolveRelative turns a relative path into an absolute url against base using
// the stdlib reference resolver; if the ref won't parse we keep the original.
func resolveRelative(base *url.URL, ref string) string {
parsed, err := url.Parse(ref)
if err != nil {
return ref
}
return base.ResolveReference(parsed).String()
}
+106
View File
@@ -0,0 +1,106 @@
/*
·━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━·
: :
: █▀ █ █▀▀ · Blazing-fast pentesting suite :
: ▄█ █ █▀ · BSD 3-Clause License :
: :
: (c) 2022-2026 vmfunc, xyzeva, :
: lunchcat alumni & contributors :
: :
·━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━·
*/
package js
import (
"slices"
"testing"
)
func TestExtractEndpoints(t *testing.T) {
const base = "https://example.com/static/app.js"
tests := []struct {
name string
content string
wantSome []string // each must appear in the result
wantAbsent []string // none of these may appear
}{
{
name: "root-relative api path resolves to absolute",
content: `fetch("/api/users")`,
wantSome: []string{"https://example.com/api/users"},
},
{
name: "absolute url passes through untouched",
content: `const u = "https://api.example.org/v1/login";`,
wantSome: []string{"https://api.example.org/v1/login"},
},
{
name: "dotted-relative path resolves against base dir",
content: `import("./chunks/main.js")`,
wantSome: []string{"https://example.com/static/chunks/main.js"},
},
{
name: "query string is preserved",
content: `axios.get("/api/search?q=test")`,
wantSome: []string{"https://example.com/api/search?q=test"},
},
{
name: "mime types are filtered out",
content: `headers["Content-Type"] = "application/json"; var t = "text/html";`,
wantAbsent: []string{"application/json", "text/html"},
},
{
name: "single words without a slash are ignored",
content: `var x = "hello"; var y = "world";`,
wantAbsent: []string{"hello", "world"},
},
{
name: "multiple endpoints deduped",
content: `fetch("/api/users"); fetch("/api/users"); fetch("/api/posts");`,
wantSome: []string{
"https://example.com/api/users",
"https://example.com/api/posts",
},
},
}
for _, tt := range tests {
t.Run(tt.name, func(t *testing.T) {
got := ExtractEndpoints(tt.content, base)
for _, want := range tt.wantSome {
if !slices.Contains(got, want) {
t.Errorf("expected %q in %v", want, got)
}
}
for _, absent := range tt.wantAbsent {
if slices.Contains(got, absent) {
t.Errorf("did not expect %q in %v", absent, got)
}
}
})
}
}
func TestExtractEndpointsDedupes(t *testing.T) {
got := ExtractEndpoints(`fetch("/api/x"); fetch("/api/x");`, "https://example.com/app.js")
count := 0
for i := 0; i < len(got); i++ {
if got[i] == "https://example.com/api/x" {
count++
}
}
if count != 1 {
t.Fatalf("expected /api/x once, got %d times in %v", count, got)
}
}
func TestExtractEndpointsBadBaseKeepsRelatives(t *testing.T) {
// a base url that won't parse must not drop findings; relatives stay as-is.
got := ExtractEndpoints(`fetch("/api/users")`, "::not a url::")
if !slices.Contains(got, "/api/users") {
t.Errorf("expected relative /api/users preserved, got %v", got)
}
}
+58 -1
View File
@@ -32,11 +32,38 @@ import (
type JavascriptScanResult struct {
SupabaseResults []supabaseScanResult `json:"supabase_results"`
FoundEnvironmentVars map[string]string `json:"environment_variables"`
SecretMatches []SecretMatch `json:"secret_matches"`
Endpoints []string `json:"endpoints"`
}
// ResultType implements the ScanResult interface.
func (r *JavascriptScanResult) ResultType() string { return "js" }
// SupabaseFinding is the exported view of one discovered supabase project. the
// raw supabaseScanResult stays package-private (it carries scan internals), so
// downstream normalizers consume this projection instead.
type SupabaseFinding struct {
ProjectId string
Role string
Collections int
}
// SupabaseFindings projects the package-private supabase results into a stable
// exported shape for the finding normalizer; role is what makes one interesting
// (a non-anon key is the real bug).
func (r *JavascriptScanResult) SupabaseFindings() []SupabaseFinding {
out := make([]SupabaseFinding, 0, len(r.SupabaseResults))
for i := 0; i < len(r.SupabaseResults); i++ {
s := r.SupabaseResults[i]
out = append(out, SupabaseFinding{
ProjectId: s.ProjectId,
Role: s.Role,
Collections: len(s.Collections),
})
}
return out
}
func JavascriptScan(url string, timeout time.Duration, threads int, logdir string) (*JavascriptScanResult, error) {
log := output.Module("JS")
log.Start()
@@ -116,6 +143,11 @@ func JavascriptScan(url string, timeout time.Duration, threads int, logdir strin
log.Info("Got %d scripts, now running scans on them", len(scripts))
supabaseResults := make([]supabaseScanResult, 0, len(scripts))
secretMatches := make([]SecretMatch, 0)
endpoints := make([]string, 0)
// dedupe secrets and endpoints across every script, not just within one.
seenSecrets := make(map[string]struct{})
seenEndpoints := make(map[string]struct{})
for _, script := range scripts {
charmlog.Debugf("Scanning %s", script)
req, err := http.NewRequestWithContext(context.TODO(), http.MethodGet, script, http.NoBody)
@@ -147,16 +179,41 @@ func JavascriptScan(url string, timeout time.Duration, threads int, logdir strin
if scriptSupabaseResults != nil {
supabaseResults = append(supabaseResults, scriptSupabaseResults...)
}
// reuse the same script buffer for credential and endpoint extraction.
for _, match := range ScanSecrets(content, script) {
key := match.Rule + "\x00" + match.Match
if _, ok := seenSecrets[key]; ok {
continue
}
seenSecrets[key] = struct{}{}
secretMatches = append(secretMatches, match)
log.Warn("found %s in %s", match.Rule, script)
}
for _, endpoint := range ExtractEndpoints(content, script) {
if _, ok := seenEndpoints[endpoint]; ok {
continue
}
seenEndpoints[endpoint] = struct{}{}
endpoints = append(endpoints, endpoint)
}
}
spin.Stop()
if len(endpoints) > 0 {
log.Info("extracted %d endpoints", len(endpoints))
}
result := JavascriptScanResult{
SupabaseResults: supabaseResults,
FoundEnvironmentVars: map[string]string{},
SecretMatches: secretMatches,
Endpoints: endpoints,
}
log.Complete(len(supabaseResults), "found")
log.Complete(len(supabaseResults)+len(secretMatches)+len(endpoints), "found")
return &result, nil
}
+171
View File
@@ -0,0 +1,171 @@
/*
·━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━·
: :
: █▀ █ █▀▀ · Blazing-fast pentesting suite :
: ▄█ █ █▀ · BSD 3-Clause License :
: :
: (c) 2022-2026 vmfunc, xyzeva, :
: lunchcat alumni & contributors :
: :
·━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━·
*/
package js
import (
"math"
"regexp"
"strings"
)
// SecretMatch is one credential the scanner pulled out of a script.
type SecretMatch struct {
Rule string `json:"rule"`
Match string `json:"match"`
Source string `json:"source"`
}
// entropy thresholds gate the noisy generic rules: provider-prefixed keys are
// trustworthy on their own, but a bare apikey="..." or a loose token blob is
// only worth reporting once its shannon entropy clears the bar for "this looks
// random, not an english word". secrets sit higher than the pem/aws-secret bar
// because the generic capture groups also catch ordinary identifiers.
const (
genericMinEntropy = 3.5
awsSecretMinEntropy = 3.0
// rules with no entropy requirement (prefix is already unique enough).
noEntropyGate = 0.0
)
// secretRules is the credential regex bank. the matching group (or the whole
// match when there's no group) is what gets reported; minEntropy gates the
// generic high-entropy rules so we don't flag every short literal.
var secretRules = []struct {
name string
re *regexp.Regexp
minEntropy float64
}{
{
// aws access key ids are fixed-shape and unmistakable.
name: "aws access key id",
re: regexp.MustCompile(`\b((?:AKIA|ABIA|ACCA|ASIA)[0-9A-Z]{16})\b`),
minEntropy: noEntropyGate,
},
{
// aws secret keys are 40-char base64-ish blobs; gate on entropy since the
// shape alone matches plenty of innocent strings.
name: "aws secret access key",
re: regexp.MustCompile(`\b((?:aws_secret_access_key|aws_secret|secret_key)["']?\s*[:=]\s*["']?)([A-Za-z0-9/+]{40})\b`),
minEntropy: awsSecretMinEntropy,
},
{
// github personal/oauth/server/refresh/app tokens share the ghX_ prefix.
name: "github token",
re: regexp.MustCompile(`\b((?:ghp|gho|ghu|ghs|ghr)_[0-9A-Za-z]{36,255})\b`),
minEntropy: noEntropyGate,
},
{
// slack bot/user/app/legacy tokens.
name: "slack token",
re: regexp.MustCompile(`\b(xox[baprs]-[0-9A-Za-z-]{10,})\b`),
minEntropy: noEntropyGate,
},
{
// stripe live secret and publishable keys (test keys are not findings).
name: "stripe live key",
re: regexp.MustCompile(`\b([sp]k_live_[0-9A-Za-z]{16,})\b`),
minEntropy: noEntropyGate,
},
{
// google api keys are a fixed AIza-prefixed 39-char shape.
name: "google api key",
re: regexp.MustCompile(`\b(AIza[0-9A-Za-z_-]{35})\b`),
minEntropy: noEntropyGate,
},
{
// pem private key blocks; the header alone is the smoking gun.
name: "private key",
re: regexp.MustCompile(`-{5}BEGIN (?:RSA |EC |DSA |OPENSSH |PGP )?PRIVATE KEY-{5}`),
minEntropy: noEntropyGate,
},
{
// generic apikey/secret/token = "<value>" assignments; the value is in
// group 2 and only reported if it looks random (entropy gate).
name: "generic secret assignment",
re: regexp.MustCompile(`(?i)\b(api[_-]?key|secret|token|password|passwd|auth)["']?\s*[:=]\s*["']([0-9A-Za-z\-._~+/]{16,})["']`),
minEntropy: genericMinEntropy,
},
}
// the value capture group lives at index 2 for the rules that prefix the
// keyword; index 0 (whole match) is used otherwise.
const (
valueGroupIndex = 2
wholeMatchIndex = 0
)
// ScanSecrets runs the regex bank over a script body and returns every gated
// match, deduped within this one source. srcURL is recorded on each find.
func ScanSecrets(content, srcURL string) []SecretMatch {
matches := make([]SecretMatch, 0)
seen := make(map[string]struct{})
for i := 0; i < len(secretRules); i++ {
rule := secretRules[i]
groups := rule.re.FindAllStringSubmatch(content, -1)
for j := 0; j < len(groups); j++ {
value := secretValue(groups[j])
if value == "" {
continue
}
// entropy gate weeds out english-y identifiers for the generic rules;
// prefixed rules pass with a zero threshold.
if rule.minEntropy > noEntropyGate && shannonEntropy(value) < rule.minEntropy {
continue
}
// dedupe per source so a key referenced twice is one finding.
key := rule.name + "\x00" + value
if _, ok := seen[key]; ok {
continue
}
seen[key] = struct{}{}
matches = append(matches, SecretMatch{Rule: rule.name, Match: value, Source: srcURL})
}
}
return matches
}
// secretValue returns the reported portion of a regex match: the dedicated
// value group when the rule captures one, otherwise the whole match.
func secretValue(groups []string) string {
if len(groups) > valueGroupIndex && groups[valueGroupIndex] != "" {
return groups[valueGroupIndex]
}
return strings.TrimSpace(groups[wholeMatchIndex])
}
// shannonEntropy is the per-character shannon entropy (bits) of s, used to tell
// random-looking secrets apart from plain words. empty input is zero entropy.
func shannonEntropy(s string) float64 {
if s == "" {
return 0
}
counts := make(map[rune]int)
for _, r := range s {
counts[r]++
}
length := float64(len([]rune(s)))
var entropy float64
for _, count := range counts {
p := float64(count) / length
entropy -= p * math.Log2(p)
}
return entropy
}
+160
View File
@@ -0,0 +1,160 @@
/*
·━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━·
: :
: █▀ █ █▀▀ · Blazing-fast pentesting suite :
: ▄█ █ █▀ · BSD 3-Clause License :
: :
: (c) 2022-2026 vmfunc, xyzeva, :
: lunchcat alumni & contributors :
: :
·━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━·
*/
package js
import (
"fmt"
"testing"
)
// the fake tokens below are assembled from two fragments on purpose: a contiguous
// provider token literal in a committed file trips github push-protection (and
// every other secret scanner) even though it's a test fixture. splitting it
// keeps the literal out of source while ScanSecrets still sees the joined value.
const (
fakeAWSKey = "AKIA" + "IOSFODNN7EXAMPLE"
fakeAWSSecret = "wJalrXUtnFEMI/K7MDENG/" + "bPxRfiCYEXAMPLEKEY"
fakeGitHub = "ghp_" + "aB3dEfGh1jKlMn0pQrStUvWxYz012345abcd"
fakeSlack = "xoxb-" + "123456789012-abcdefABCDEF1234567890ab"
fakeStripe = "sk_live_" + "4eC39HqLyjWDarjtT1zdp7dc"
fakeGoogle = "AIza" + "SyA1B2C3D4E5F6G7H8I9J0K1L2M3N4O5P6Q"
fakeGeneric = "x9Kq2Lm7Pz4Rt6Wv8Bn3Cd5Fg1Hj0As"
fakePEM = "-----BEGIN RSA PRIVATE " + "KEY-----\nMIIEpAIB..."
)
func TestScanSecrets(t *testing.T) {
tests := []struct {
name string
content string
wantRule string // rule expected on the first match, "" means no match
wantNone bool
}{
{
name: "aws access key id",
content: fmt.Sprintf(`const k = %q;`, fakeAWSKey),
wantRule: "aws access key id",
},
{
name: "github personal token",
content: fmt.Sprintf(`token: %q`, fakeGitHub),
wantRule: "github token",
},
{
name: "slack bot token",
content: fmt.Sprintf(`slack=%q`, fakeSlack),
wantRule: "slack token",
},
{
name: "stripe live secret key",
content: fmt.Sprintf(`var sk = %q;`, fakeStripe),
wantRule: "stripe live key",
},
{
name: "google api key",
content: fmt.Sprintf(`apiKey: %q`, fakeGoogle),
wantRule: "google api key",
},
{
name: "pem private key header",
content: fakePEM,
wantRule: "private key",
},
{
name: "generic high-entropy api key assignment",
content: fmt.Sprintf(`apikey = %q`, fakeGeneric),
wantRule: "generic secret assignment",
},
{
name: "aws secret with entropy",
content: fmt.Sprintf(`aws_secret_access_key=%q`, fakeAWSSecret),
wantRule: "aws secret access key",
},
{
// low-entropy assignment is a placeholder, not a real secret.
name: "low entropy generic assignment not flagged",
content: `password = "aaaaaaaaaaaaaaaaaaaaaaaa"`,
wantNone: true,
},
{
// a repetitive placeholder is low-entropy and must not trip the gate.
name: "low entropy repeated pattern not flagged",
content: `token = "abababababababababababab"`,
wantNone: true,
},
{
name: "no secrets in plain code",
content: `function add(a, b) { return a + b; }`,
wantNone: true,
},
}
for _, tt := range tests {
t.Run(tt.name, func(t *testing.T) {
got := ScanSecrets(tt.content, "https://example.com/app.js")
if tt.wantNone {
if len(got) != 0 {
t.Fatalf("expected no matches, got %+v", got)
}
return
}
if len(got) == 0 {
t.Fatalf("expected a %q match, got none", tt.wantRule)
}
if got[0].Rule != tt.wantRule {
t.Errorf("rule = %q, want %q", got[0].Rule, tt.wantRule)
}
if got[0].Match == "" {
t.Error("match value is empty")
}
if got[0].Source != "https://example.com/app.js" {
t.Errorf("source = %q, want the passed url", got[0].Source)
}
})
}
}
func TestScanSecretsDedupesWithinSource(t *testing.T) {
// the same key referenced twice in one file is one finding.
content := fmt.Sprintf(`a = %q; b = %q;`, fakeAWSKey, fakeAWSKey)
got := ScanSecrets(content, "https://example.com/app.js")
if len(got) != 1 {
t.Fatalf("expected 1 deduped match, got %d: %+v", len(got), got)
}
}
func TestShannonEntropy(t *testing.T) {
tests := []struct {
name string
input string
// random-ish strings clear the generic gate, repetitive ones don't.
wantHigh bool
}{
{name: "empty is zero", input: "", wantHigh: false},
{name: "repeated char is low", input: "aaaaaaaaaaaaaaaa", wantHigh: false},
{name: "random blob is high", input: fakeGeneric, wantHigh: true},
}
for _, tt := range tests {
t.Run(tt.name, func(t *testing.T) {
got := shannonEntropy(tt.input)
if tt.wantHigh && got < genericMinEntropy {
t.Errorf("entropy %f below generic gate %f", got, genericMinEntropy)
}
if !tt.wantHigh && got >= genericMinEntropy {
t.Errorf("entropy %f unexpectedly cleared generic gate %f", got, genericMinEntropy)
}
})
}
}
+5 -2
View File
@@ -129,7 +129,9 @@ func doSupabaseRequest(projectId, path, apikey string, auth *string, timeout tim
if err != nil {
return nil, nil, err
}
defer resp.Body.Close()
// the non-200 branch returns before reading the body, so drain on close to
// keep the conn reusable instead of leaking it.
defer httpx.DrainClose(resp)
if resp.StatusCode != 200 {
return nil, nil, errors.New("request to " + resp.Request.URL.String() + " failed with status code " + strconv.Itoa(resp.StatusCode))
@@ -215,7 +217,8 @@ func ScanSupabase(jsContent string, jsUrl string, timeout time.Duration) ([]supa
auth = authResp.AccessToken
supabaselog.Infof("Created account with JWT %s", auth)
} else {
resp.Body.Close()
// non-200 signup: body never read, so drain to reuse the conn.
httpx.DrainClose(resp)
}
var collections = []supabaseCollection{}
+396
View File
@@ -0,0 +1,396 @@
/*
·━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━·
: :
: █▀ █ █▀▀ · Blazing-fast pentesting suite :
: ▄█ █ █▀ · BSD 3-Clause License :
: :
: (c) 2022-2026 vmfunc, xyzeva, :
: lunchcat alumni & contributors :
: :
·━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━·
*/
package scan
import (
"context"
"crypto/hmac"
"crypto/sha256"
"encoding/base64"
"encoding/json"
"fmt"
"io"
"net/http"
"regexp"
"strings"
"time"
"github.com/dropalldatabases/sif/internal/httpx"
"github.com/dropalldatabases/sif/internal/logger"
"github.com/dropalldatabases/sif/internal/output"
)
// JWTResult collects every token discovered on the target plus the offline
// analysis of each one.
type JWTResult struct {
Tokens []JWTToken `json:"tokens,omitempty"`
}
// JWTToken is one decoded jwt and the weaknesses found in it. Token is trimmed
// to a short prefix so we never log a full credential.
type JWTToken struct {
Source string `json:"source"` // where we found it (header name / cookie / body)
Preview string `json:"preview"` // first chars of the raw token, never the whole thing
Alg string `json:"alg"` // header alg claim
Issues []JWTIssue `json:"issues"` // the weaknesses, ranked
Claims map[string]any `json:"claims"` // decoded payload (for reporting)
WeakKey string `json:"weak_key"` // cracked hmac secret, empty when none
}
// JWTIssue is a single weakness with a severity so the report layer can rank it.
type JWTIssue struct {
Kind string `json:"kind"`
Severity string `json:"severity"`
Detail string `json:"detail"`
}
// jwtBodyReadCap bounds how much of the response body we slurp looking for
// tokens; a jwt riding in the body is near the top, so a megabyte is plenty
// without letting a huge response exhaust memory.
const jwtBodyReadCap = 1 << 20
// jwtPreviewLen is how many leading characters of a token we keep for evidence.
// enough to identify the token in a report, short enough to never be the whole
// credential.
const jwtPreviewLen = 16
// the three structural jwt severities.
const (
jwtSevCritical = "critical"
jwtSevHigh = "high"
jwtSevMedium = "medium"
jwtSevLow = "low"
)
// jwtRegex matches a compact-serialization jwt: three base64url segments split
// by dots. the header always starts "eyJ" (base64url of `{"`), which anchors the
// match and keeps it from firing on arbitrary dotted tokens.
var jwtRegex = regexp.MustCompile(`eyJ[A-Za-z0-9_-]+\.[A-Za-z0-9_-]+\.[A-Za-z0-9_-]*`)
// jwtWeakSecrets is a tiny offline wordlist of hmac secrets seen in tutorials,
// boilerplate and leaked configs. cracking one means anyone can forge tokens, so
// a hit is critical. kept short on purpose - this is a smoke test, not john.
var jwtWeakSecrets = []string{
"secret", "secretkey", "secret_key", "your-256-bit-secret",
"changeme", "password", "jwt", "jwtsecret", "key", "test",
"admin", "supersecret", "s3cr3t", "qwerty", "123456",
}
// sensitiveClaimKeys are payload fields that should never travel in a readable
// jwt body (the payload is only base64, not encrypted). a match is a disclosure.
var sensitiveClaimKeys = []string{
"password", "passwd", "secret", "api_key", "apikey", "ssn",
"credit_card", "card_number", "private_key", "access_key",
}
// JWT fetches the target once, harvests every jwt from the response headers,
// cookies and body, then analyzes each one entirely offline.
func JWT(targetURL string, timeout time.Duration, logdir string) (*JWTResult, error) {
log := output.Module("JWT")
log.Start()
sanitizedURL := stripScheme(targetURL)
if logdir != "" {
if err := logger.WriteHeader(sanitizedURL, logdir, "JWT discovery + offline analysis"); err != nil {
log.Error("error creating log file: %v", err)
return nil, fmt.Errorf("create jwt log: %w", err)
}
}
client := httpx.Client(timeout)
req, err := http.NewRequestWithContext(context.TODO(), http.MethodGet, targetURL, http.NoBody)
if err != nil {
return nil, fmt.Errorf("build jwt request: %w", err)
}
resp, err := client.Do(req)
if err != nil {
return nil, fmt.Errorf("fetch jwt target: %w", err)
}
defer resp.Body.Close()
// one read, capped; everything past this point is offline.
body, err := io.ReadAll(io.LimitReader(resp.Body, jwtBodyReadCap))
if err != nil {
return nil, fmt.Errorf("read jwt body: %w", err)
}
raws := harvestJWTs(resp, string(body))
if len(raws) == 0 {
log.Info("no jwts found on target")
log.Complete(0, "found")
return nil, nil //nolint:nilnil // absence of a token is not an error
}
result := &JWTResult{Tokens: make([]JWTToken, 0, len(raws))}
for _, hit := range raws {
token, ok := analyzeJWT(hit.source, hit.raw)
if !ok {
continue
}
result.Tokens = append(result.Tokens, token)
for i := 0; i < len(token.Issues); i++ {
iss := token.Issues[i]
log.Warn("jwt %s: %s (%s)", renderJWTSeverity(iss.Severity), iss.Kind, hit.source)
if logdir != "" {
_ = logger.Write(sanitizedURL, logdir,
fmt.Sprintf("JWT %s: %s - %s [%s]\n", iss.Severity, iss.Kind, iss.Detail, hit.source))
}
}
}
if len(result.Tokens) == 0 {
log.Complete(0, "found")
return nil, nil //nolint:nilnil // tokens were malformed, nothing to report
}
log.Complete(len(result.Tokens), "analyzed")
return result, nil
}
// jwtHit ties a raw token to where it came from so the report can attribute it.
type jwtHit struct {
source string
raw string
}
// harvestJWTs pulls every jwt out of the response: Authorization-style headers,
// Set-Cookie values and the body. dedup keys on the raw token so the same value
// echoed in two places is reported once.
func harvestJWTs(resp *http.Response, body string) []jwtHit {
seen := make(map[string]struct{})
var hits []jwtHit
add := func(source, raw string) {
if _, ok := seen[raw]; ok {
return
}
seen[raw] = struct{}{}
hits = append(hits, jwtHit{source: source, raw: raw})
}
for name, values := range resp.Header {
for i := 0; i < len(values); i++ {
for _, m := range jwtRegex.FindAllString(values[i], -1) {
add("header:"+name, m)
}
}
}
for _, c := range resp.Cookies() {
for _, m := range jwtRegex.FindAllString(c.Value, -1) {
add("cookie:"+c.Name, m)
}
}
for _, m := range jwtRegex.FindAllString(body, -1) {
add("body", m)
}
return hits
}
// analyzeJWT decodes the header and payload (offline base64url, never verifying a
// signature against the network) and runs every weakness check. ok is false when
// the token doesn't decode into a real header+payload, so junk that matched the
// regex is dropped rather than reported.
func analyzeJWT(source, raw string) (JWTToken, bool) {
parts := strings.Split(raw, ".")
if len(parts) != 3 {
return JWTToken{}, false
}
header, err := decodeJWTSegment(parts[0])
if err != nil {
return JWTToken{}, false
}
payload, err := decodeJWTSegment(parts[1])
if err != nil {
return JWTToken{}, false
}
alg, _ := header["alg"].(string)
token := JWTToken{
Source: source,
Preview: previewToken(raw),
Alg: alg,
Claims: payload,
}
token.Issues = append(token.Issues, jwtAlgIssues(alg)...)
token.Issues = append(token.Issues, jwtClaimIssues(payload)...)
// only bother cracking when the alg is actually hmac; an asymmetric token
// has no shared secret to guess.
if isHMACAlg(alg) {
if secret, ok := crackHMAC(raw); ok {
token.WeakKey = secret
token.Issues = append(token.Issues, JWTIssue{
Kind: "weak hmac secret",
Severity: jwtSevCritical,
Detail: "signature verifies against bundled weak secret " + secret,
})
}
}
return token, true
}
// jwtAlgIssues flags the algorithm-level weaknesses: alg:none (no signature at
// all) and the RS256->HS256 confusion surface (an asymmetric-looking token whose
// header says HS*, meaning a server that loads the public key as an hmac secret
// can be forged).
func jwtAlgIssues(alg string) []JWTIssue {
var issues []JWTIssue
lower := strings.ToLower(alg)
if lower == "none" || alg == "" {
issues = append(issues, JWTIssue{
Kind: "alg:none",
Severity: jwtSevCritical,
Detail: "token declares no signature algorithm; forgeable",
})
return issues
}
if isHMACAlg(alg) {
issues = append(issues, JWTIssue{
Kind: "rs256->hs256 confusion surface",
Severity: jwtSevMedium,
Detail: "token is HMAC-signed; if the server also accepts asymmetric algs " +
"with the same verifier, a public key can be used as the HMAC secret",
})
}
return issues
}
// jwtClaimIssues inspects the decoded payload for missing/expired expiry and any
// plaintext sensitive claims (the payload is base64, not encrypted).
func jwtClaimIssues(payload map[string]any) []JWTIssue {
var issues []JWTIssue
exp, hasExp := numericClaim(payload, "exp")
switch {
case !hasExp:
issues = append(issues, JWTIssue{
Kind: "missing exp",
Severity: jwtSevMedium,
Detail: "no expiry claim; token never ages out",
})
case time.Now().After(time.Unix(int64(exp), 0)):
issues = append(issues, JWTIssue{
Kind: "expired token",
Severity: jwtSevLow,
Detail: "exp is in the past; a server still honoring it is a bug",
})
}
for i := 0; i < len(sensitiveClaimKeys); i++ {
key := sensitiveClaimKeys[i]
if _, ok := payload[key]; ok {
issues = append(issues, JWTIssue{
Kind: "sensitive plaintext claim",
Severity: jwtSevHigh,
Detail: "payload carries readable claim " + key + "; jwt bodies are not encrypted",
})
}
}
return issues
}
// crackHMAC tries every bundled weak secret against the token's HS256 signature
// offline. a verifying secret means the token is forgeable by anyone who knows
// it. only HS256 is attempted; the wordlist exists to catch lazy defaults, not
// to be a real cracker.
func crackHMAC(raw string) (string, bool) {
parts := strings.Split(raw, ".")
if len(parts) != 3 {
return "", false
}
signingInput := parts[0] + "." + parts[1]
want, err := base64.RawURLEncoding.DecodeString(parts[2])
if err != nil {
return "", false
}
for i := 0; i < len(jwtWeakSecrets); i++ {
secret := jwtWeakSecrets[i]
mac := hmac.New(sha256.New, []byte(secret))
mac.Write([]byte(signingInput))
if hmac.Equal(mac.Sum(nil), want) {
return secret, true
}
}
return "", false
}
// decodeJWTSegment base64url-decodes one jwt segment into a claims map. jwt uses
// unpadded base64url, but some emitters pad anyway, so try raw first then padded.
func decodeJWTSegment(seg string) (map[string]any, error) {
data, err := base64.RawURLEncoding.DecodeString(seg)
if err != nil {
data, err = base64.URLEncoding.DecodeString(seg)
if err != nil {
return nil, fmt.Errorf("base64url decode segment: %w", err)
}
}
var claims map[string]any
if err := json.Unmarshal(data, &claims); err != nil {
return nil, fmt.Errorf("unmarshal jwt segment: %w", err)
}
return claims, nil
}
// numericClaim pulls a numeric claim out of the payload. json numbers decode to
// float64, so that's the only shape we accept.
func numericClaim(payload map[string]any, key string) (float64, bool) {
v, ok := payload[key]
if !ok {
return 0, false
}
f, ok := v.(float64)
return f, ok
}
// isHMACAlg reports whether alg is one of the HMAC family (HS256/HS384/HS512).
func isHMACAlg(alg string) bool {
return strings.HasPrefix(strings.ToUpper(alg), "HS")
}
// previewToken trims a raw token to a short prefix so evidence never carries the
// whole credential.
func previewToken(raw string) string {
if len(raw) <= jwtPreviewLen {
return raw
}
return raw[:jwtPreviewLen] + "..."
}
func renderJWTSeverity(severity string) string {
switch severity {
case jwtSevCritical:
return output.SeverityCritical.Render(severity)
case jwtSevHigh:
return output.SeverityHigh.Render(severity)
case jwtSevMedium:
return output.SeverityMedium.Render(severity)
default:
return output.SeverityLow.Render(severity)
}
}
// ResultType identifies jwt findings for the result registry.
func (r *JWTResult) ResultType() string { return "jwt" }
var _ ScanResult = (*JWTResult)(nil)
+172
View File
@@ -0,0 +1,172 @@
/*
·━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━·
: :
: █▀ █ █▀▀ · Blazing-fast pentesting suite :
: ▄█ █ █▀ · BSD 3-Clause License :
: :
: (c) 2022-2026 vmfunc, xyzeva, :
: lunchcat alumni & contributors :
: :
·━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━·
*/
package scan
import (
"net/http"
"net/http/httptest"
"testing"
"time"
)
// fixed jwt fixtures, generated offline. each exercises a distinct weakness.
const (
// header {alg:none}, payload {sub:admin}, empty signature - forgeable.
jwtNone = "eyJhbGciOiAibm9uZSIsICJ0eXAiOiAiSldUIn0." +
"eyJzdWIiOiAiYWRtaW4iLCAicm9sZSI6ICJ1c2VyIn0."
// HS256, no exp claim, signed with the bundled weak secret "secret".
jwtWeakHS256 = "eyJhbGciOiAiSFMyNTYiLCAidHlwIjogIkpXVCJ9." +
"eyJzdWIiOiAiMTIzNDU2Nzg5MCIsICJuYW1lIjogInRlc3RlciJ9." +
"JOjVfLa8gp3cvFkNVgOnmdrI1MCHZRA_ChBmCPF-Z8w"
// HS256, exp in 2001 (long past), signed with a secret not in the wordlist.
jwtExpired = "eyJhbGciOiAiSFMyNTYiLCAidHlwIjogIkpXVCJ9." +
"eyJzdWIiOiAieCIsICJleHAiOiAxMDAwMDAwMDAwfQ." +
"gr28Ffm4wJkonHGSKmMD5Rj7e1pTt2o_EwG6lMWQeSc"
// HS256 carrying a plaintext password claim (jwt bodies are not encrypted).
jwtSensitive = "eyJhbGciOiAiSFMyNTYiLCAidHlwIjogIkpXVCJ9." +
"eyJzdWIiOiAieCIsICJwYXNzd29yZCI6ICJodW50ZXIyIiwgImV4cCI6IDk5OTk5OTk5OTl9." +
"rjEf0CUa7_qppuINi6zL9vupJIX0rzSBhul7kKM9uSA"
)
// hasIssue reports whether the analyzed token carries an issue of the given kind.
func hasIssue(token *JWTToken, kind string) bool {
for i := 0; i < len(token.Issues); i++ {
if token.Issues[i].Kind == kind {
return true
}
}
return false
}
func TestJWT_AlgNoneAndMissingExpFlagged(t *testing.T) {
// serve the alg:none token in the Authorization header echo.
srv := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, _ *http.Request) {
w.Header().Set("Authorization", "Bearer "+jwtNone)
w.WriteHeader(http.StatusOK)
}))
defer srv.Close()
result, err := JWT(srv.URL, 5*time.Second, "")
if err != nil {
t.Fatalf("JWT: %v", err)
}
if result == nil || len(result.Tokens) != 1 {
t.Fatalf("expected exactly one analyzed token, got %+v", result)
}
token := &result.Tokens[0]
if !hasIssue(token, "alg:none") {
t.Errorf("expected alg:none to be flagged, got issues %+v", token.Issues)
}
if !hasIssue(token, "missing exp") {
t.Errorf("expected missing exp to be flagged, got issues %+v", token.Issues)
}
// the preview must never carry the whole token.
if len(token.Preview) >= len(jwtNone) {
t.Errorf("preview should be trimmed, got full token %q", token.Preview)
}
}
func TestJWT_WeakSecretCracked(t *testing.T) {
// token rides in a Set-Cookie this time.
srv := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, _ *http.Request) {
http.SetCookie(w, &http.Cookie{Name: "session", Value: jwtWeakHS256})
w.WriteHeader(http.StatusOK)
}))
defer srv.Close()
result, err := JWT(srv.URL, 5*time.Second, "")
if err != nil {
t.Fatalf("JWT: %v", err)
}
if result == nil || len(result.Tokens) != 1 {
t.Fatalf("expected one token, got %+v", result)
}
token := &result.Tokens[0]
if token.WeakKey != "secret" {
t.Errorf("expected weak secret 'secret' to be cracked, got %q", token.WeakKey)
}
if !hasIssue(token, "weak hmac secret") {
t.Errorf("expected weak hmac secret issue, got %+v", token.Issues)
}
if !hasIssue(token, "rs256->hs256 confusion surface") {
t.Errorf("expected hmac confusion surface to be flagged, got %+v", token.Issues)
}
}
func TestJWT_ExpiredFlagged(t *testing.T) {
// token in the response body.
srv := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, _ *http.Request) {
_, _ = w.Write([]byte(`{"token":"` + jwtExpired + `"}`))
}))
defer srv.Close()
result, err := JWT(srv.URL, 5*time.Second, "")
if err != nil {
t.Fatalf("JWT: %v", err)
}
if result == nil || len(result.Tokens) != 1 {
t.Fatalf("expected one token, got %+v", result)
}
if !hasIssue(&result.Tokens[0], "expired token") {
t.Errorf("expected expired token to be flagged, got %+v", result.Tokens[0].Issues)
}
// a strong, unguessed secret must not be cracked.
if result.Tokens[0].WeakKey != "" {
t.Errorf("did not expect a cracked key on the strong-secret token, got %q", result.Tokens[0].WeakKey)
}
}
func TestJWT_SensitiveClaimFlagged(t *testing.T) {
srv := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, _ *http.Request) {
_, _ = w.Write([]byte(jwtSensitive))
}))
defer srv.Close()
result, err := JWT(srv.URL, 5*time.Second, "")
if err != nil {
t.Fatalf("JWT: %v", err)
}
if result == nil || len(result.Tokens) != 1 {
t.Fatalf("expected one token, got %+v", result)
}
if !hasIssue(&result.Tokens[0], "sensitive plaintext claim") {
t.Errorf("expected sensitive claim to be flagged, got %+v", result.Tokens[0].Issues)
}
}
func TestJWT_NoTokens(t *testing.T) {
srv := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, _ *http.Request) {
_, _ = w.Write([]byte("nothing to see here"))
}))
defer srv.Close()
result, err := JWT(srv.URL, 5*time.Second, "")
if err != nil {
t.Fatalf("JWT: %v", err)
}
if result != nil {
t.Errorf("expected nil result when no tokens present, got %+v", result)
}
}
func TestJWTResult_ResultType(t *testing.T) {
r := &JWTResult{}
if r.ResultType() != "jwt" {
t.Errorf("expected result type 'jwt', got %q", r.ResultType())
}
}
+322
View File
@@ -0,0 +1,322 @@
/*
·━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━·
: :
: █▀ █ █▀▀ · Blazing-fast pentesting suite :
: ▄█ █ █▀ · BSD 3-Clause License :
: :
: (c) 2022-2026 vmfunc, xyzeva, :
: lunchcat alumni & contributors :
: :
·━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━·
*/
package scan
import (
"context"
"encoding/json"
"fmt"
"io"
"net/http"
"sort"
"strings"
"sync"
"time"
charmlog "github.com/charmbracelet/log"
"github.com/dropalldatabases/sif/internal/httpx"
"github.com/dropalldatabases/sif/internal/logger"
"github.com/dropalldatabases/sif/internal/output"
"gopkg.in/yaml.v3"
)
// OpenAPIResult is the parsed spec exposure plus the endpoints enumerated from
// it.
type OpenAPIResult struct {
SpecURL string `json:"spec_url"` // the path the spec was served at
Title string `json:"title"` // info.title from the spec
Version string `json:"version"` // openapi/swagger version string
Endpoints []OpenAPIEndpoint `json:"endpoints"` // every path+method pair
Severity string `json:"severity"` // exposure severity
}
// OpenAPIEndpoint is one path+method, flagged when nothing in the spec gates it.
type OpenAPIEndpoint struct {
Path string `json:"path"`
Method string `json:"method"`
Unauth bool `json:"unauth"` // no security requirement on this operation
}
// openapiSpecPaths are the conventional locations a spec is served from. ordered
// most-common first so the typical hit is found early.
var openapiSpecPaths = []string{
"/swagger.json",
"/openapi.json",
"/v3/api-docs",
"/api-docs",
"/swagger/v1/swagger.json",
"/swagger-ui/",
}
// openapiBodyReadCap bounds spec body reads. specs are text and rarely huge, but
// an attacker-controlled endpoint could stream forever, so cap it.
const openapiBodyReadCap = 8 << 20
// the http methods an openapi path item can declare. anything outside this set
// is metadata (parameters, summary), not an operation.
var openapiHTTPMethods = []string{
http.MethodGet, http.MethodPut, http.MethodPost, http.MethodDelete,
http.MethodOptions, http.MethodHead, http.MethodPatch, http.MethodTrace,
}
// exposure severities. an enumerable spec is medium on its own; unauthenticated
// operations bump it to high.
const (
openapiSevMedium = "medium"
openapiSevHigh = "high"
)
// openapiSpec is the minimal slice of an openapi/swagger document we care about:
// the version banner, info block, top-level security and the path map. unknown
// fields are ignored by both json and yaml decoders.
type openapiSpec struct {
OpenAPI string `json:"openapi" yaml:"openapi"`
Swagger string `json:"swagger" yaml:"swagger"`
Info openapiInfo `json:"info" yaml:"info"`
Security []map[string][]string `json:"security" yaml:"security"`
Paths map[string]map[string]rawOps `json:"paths" yaml:"paths"`
}
type openapiInfo struct {
Title string `json:"title" yaml:"title"`
Version string `json:"version" yaml:"version"`
}
// rawOps captures just the per-operation security block so we can tell whether
// an operation requires auth. the rest of the operation object is irrelevant.
type rawOps struct {
Security []map[string][]string `json:"security" yaml:"security"`
}
// OpenAPI probes the candidate spec paths concurrently and, on the first hit,
// parses the spec and enumerates its endpoints.
func OpenAPI(targetURL string, timeout time.Duration, threads int, logdir string) (*OpenAPIResult, error) {
log := output.Module("OPENAPI")
log.Start()
spin := output.NewSpinner("Probing for exposed openapi/swagger specs")
spin.Start()
sanitizedURL := stripScheme(targetURL)
if logdir != "" {
if err := logger.WriteHeader(sanitizedURL, logdir, "OpenAPI/Swagger spec exposure"); err != nil {
spin.Stop()
log.Error("error creating log file: %v", err)
return nil, fmt.Errorf("create openapi log: %w", err)
}
}
client := httpx.Client(timeout)
base := strings.TrimRight(targetURL, "/")
result := probeOpenAPIPaths(client, base, threads)
spin.Stop()
if result == nil {
log.Info("no openapi/swagger spec exposed")
log.Complete(0, "found")
return nil, nil //nolint:nilnil // no exposed spec is not an error
}
unauth := 0
for i := 0; i < len(result.Endpoints); i++ {
if result.Endpoints[i].Unauth {
unauth++
}
}
log.Warn("openapi %s: spec at %s exposes %d endpoints (%d unauthenticated)",
renderOpenAPISeverity(result.Severity),
output.Highlight.Render(result.SpecURL),
len(result.Endpoints), unauth)
if logdir != "" {
_ = logger.Write(sanitizedURL, logdir,
fmt.Sprintf("OpenAPI spec exposed at %s: %d endpoints, %d unauthenticated\n",
result.SpecURL, len(result.Endpoints), unauth))
}
log.Complete(len(result.Endpoints), "endpoints")
return result, nil
}
// probeOpenAPIPaths fans the candidate paths across a worker pool and returns the
// first parseable spec. the first hit wins, so once one worker fills the result
// the rest of the channel drains without re-parsing.
func probeOpenAPIPaths(client *http.Client, base string, threads int) *OpenAPIResult {
var (
mu sync.Mutex
wg sync.WaitGroup
result *OpenAPIResult
)
pathChan := make(chan string, len(openapiSpecPaths))
for i := 0; i < len(openapiSpecPaths); i++ {
pathChan <- openapiSpecPaths[i]
}
close(pathChan)
wg.Add(threads)
for t := 0; t < threads; t++ {
go func() {
defer wg.Done()
for path := range pathChan {
// a spec already landed; stop spending requests.
mu.Lock()
done := result != nil
mu.Unlock()
if done {
return
}
hit := fetchOpenAPISpec(client, base+path)
if hit == nil {
continue
}
hit.SpecURL = base + path
mu.Lock()
if result == nil {
result = hit
}
mu.Unlock()
}
}()
}
wg.Wait()
return result
}
// fetchOpenAPISpec GETs one candidate path and parses the body as a spec. it
// returns nil on any failure (non-200, unparseable, zero paths) so a swagger-ui
// html page or a 404 doesn't masquerade as a finding.
func fetchOpenAPISpec(client *http.Client, specURL string) *OpenAPIResult {
req, err := http.NewRequestWithContext(context.TODO(), http.MethodGet, specURL, http.NoBody)
if err != nil {
charmlog.Debugf("openapi: build request for %s: %v", specURL, err)
return nil
}
resp, err := client.Do(req)
if err != nil {
charmlog.Debugf("openapi: request %s: %v", specURL, err)
return nil
}
defer resp.Body.Close()
if resp.StatusCode != http.StatusOK {
return nil
}
body, err := io.ReadAll(io.LimitReader(resp.Body, openapiBodyReadCap))
if err != nil {
charmlog.Debugf("openapi: read %s: %v", specURL, err)
return nil
}
spec, ok := parseOpenAPISpec(body)
if !ok {
return nil
}
return specToResult(spec)
}
// parseOpenAPISpec decodes the body as json first, then yaml. it only accepts a
// document that actually declares an openapi/swagger version and at least one
// path, so an unrelated json/yaml file served at the candidate path is rejected.
func parseOpenAPISpec(body []byte) (*openapiSpec, bool) {
var spec openapiSpec
if err := json.Unmarshal(body, &spec); err != nil {
if err := yaml.Unmarshal(body, &spec); err != nil {
return nil, false
}
}
versioned := spec.OpenAPI != "" || spec.Swagger != ""
if !versioned || len(spec.Paths) == 0 {
return nil, false
}
return &spec, true
}
// specToResult flattens the parsed spec into enumerated endpoints and ranks the
// exposure. an operation with no security requirement (and no top-level default)
// is flagged unauthenticated, which bumps the overall severity to high.
func specToResult(spec *openapiSpec) *OpenAPIResult {
hasGlobalSecurity := len(spec.Security) > 0
endpoints := make([]OpenAPIEndpoint, 0, len(spec.Paths))
anyUnauth := false
// stable order: sort paths so the report is deterministic across runs.
paths := make([]string, 0, len(spec.Paths))
for p := range spec.Paths {
paths = append(paths, p)
}
sort.Strings(paths)
for i := 0; i < len(paths); i++ {
path := paths[i]
ops := spec.Paths[path]
for j := 0; j < len(openapiHTTPMethods); j++ {
method := openapiHTTPMethods[j]
op, ok := ops[strings.ToLower(method)]
if !ok {
continue
}
// an operation is unauth when neither it nor the global default
// declares a security requirement.
unauth := len(op.Security) == 0 && !hasGlobalSecurity
if unauth {
anyUnauth = true
}
endpoints = append(endpoints, OpenAPIEndpoint{
Path: path,
Method: method,
Unauth: unauth,
})
}
}
severity := openapiSevMedium
if anyUnauth {
severity = openapiSevHigh
}
version := spec.OpenAPI
if version == "" {
version = spec.Swagger
}
return &OpenAPIResult{
Title: spec.Info.Title,
Version: version,
Endpoints: endpoints,
Severity: severity,
}
}
func renderOpenAPISeverity(severity string) string {
if severity == openapiSevHigh {
return output.SeverityHigh.Render(severity)
}
return output.SeverityMedium.Render(severity)
}
// ResultType identifies openapi findings for the result registry.
func (r *OpenAPIResult) ResultType() string { return "openapi" }
var _ ScanResult = (*OpenAPIResult)(nil)
+210
View File
@@ -0,0 +1,210 @@
/*
·━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━·
: :
: █▀ █ █▀▀ · Blazing-fast pentesting suite :
: ▄█ █ █▀ · BSD 3-Clause License :
: :
: (c) 2022-2026 vmfunc, xyzeva, :
: lunchcat alumni & contributors :
: :
·━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━·
*/
package scan
import (
"net/http"
"net/http/httptest"
"testing"
"time"
)
// a minimal openapi 3 doc with two paths/three operations, no security at all -
// every operation is unauthenticated.
const openapiJSONUnauth = `{
"openapi": "3.0.1",
"info": {"title": "Test API", "version": "1.0"},
"paths": {
"/users": {
"get": {"summary": "list"},
"post": {"summary": "create"}
},
"/admin": {
"delete": {"summary": "nuke"}
}
}
}`
// same doc but with a global security requirement, so nothing is flagged unauth.
const openapiJSONSecured = `{
"openapi": "3.0.1",
"info": {"title": "Secured API", "version": "1.0"},
"security": [{"bearerAuth": []}],
"paths": {
"/users": {"get": {"summary": "list"}}
}
}`
// a yaml swagger 2.0 doc, to exercise the yaml parse fallback.
const openapiYAML = `swagger: "2.0"
info:
title: YAML API
version: "2.0"
paths:
/ping:
get:
summary: health
`
// hasEndpoint reports whether the result enumerated the given path+method.
func hasEndpoint(r *OpenAPIResult, path, method string) (OpenAPIEndpoint, bool) {
for i := 0; i < len(r.Endpoints); i++ {
if r.Endpoints[i].Path == path && r.Endpoints[i].Method == method {
return r.Endpoints[i], true
}
}
return OpenAPIEndpoint{}, false
}
func TestOpenAPI_EnumeratesEndpoints(t *testing.T) {
srv := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
if r.URL.Path == "/openapi.json" {
w.Header().Set("Content-Type", "application/json")
_, _ = w.Write([]byte(openapiJSONUnauth))
return
}
w.WriteHeader(http.StatusNotFound)
}))
defer srv.Close()
result, err := OpenAPI(srv.URL, 5*time.Second, 4, "")
if err != nil {
t.Fatalf("OpenAPI: %v", err)
}
if result == nil {
t.Fatal("expected an openapi result, got nil")
}
if len(result.Endpoints) != 3 {
t.Fatalf("expected 3 enumerated endpoints, got %d: %+v", len(result.Endpoints), result.Endpoints)
}
for _, want := range []struct{ path, method string }{
{"/users", http.MethodGet},
{"/users", http.MethodPost},
{"/admin", http.MethodDelete},
} {
ep, ok := hasEndpoint(result, want.path, want.method)
if !ok {
t.Errorf("missing endpoint %s %s", want.method, want.path)
continue
}
if !ep.Unauth {
t.Errorf("expected %s %s to be flagged unauthenticated", want.method, want.path)
}
}
// no security anywhere -> high exposure.
if result.Severity != openapiSevHigh {
t.Errorf("expected high severity for fully-unauth spec, got %q", result.Severity)
}
if result.Title != "Test API" {
t.Errorf("expected title 'Test API', got %q", result.Title)
}
}
func TestOpenAPI_SecuredSpecIsMedium(t *testing.T) {
srv := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
if r.URL.Path == "/swagger.json" {
_, _ = w.Write([]byte(openapiJSONSecured))
return
}
w.WriteHeader(http.StatusNotFound)
}))
defer srv.Close()
result, err := OpenAPI(srv.URL, 5*time.Second, 4, "")
if err != nil {
t.Fatalf("OpenAPI: %v", err)
}
if result == nil {
t.Fatal("expected a result, got nil")
}
ep, ok := hasEndpoint(result, "/users", http.MethodGet)
if !ok {
t.Fatal("expected /users GET to be enumerated")
}
if ep.Unauth {
t.Errorf("global security should mark the operation authenticated, got unauth")
}
if result.Severity != openapiSevMedium {
t.Errorf("expected medium severity for a secured spec, got %q", result.Severity)
}
}
func TestOpenAPI_YAMLSpec(t *testing.T) {
srv := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
if r.URL.Path == "/v3/api-docs" {
w.Header().Set("Content-Type", "application/yaml")
_, _ = w.Write([]byte(openapiYAML))
return
}
w.WriteHeader(http.StatusNotFound)
}))
defer srv.Close()
result, err := OpenAPI(srv.URL, 5*time.Second, 4, "")
if err != nil {
t.Fatalf("OpenAPI: %v", err)
}
if result == nil {
t.Fatal("expected a yaml-parsed result, got nil")
}
if _, ok := hasEndpoint(result, "/ping", http.MethodGet); !ok {
t.Errorf("expected /ping GET from yaml spec, got %+v", result.Endpoints)
}
}
// TestOpenAPI_NoSpecExposed confirms a server with no spec at any candidate path
// produces no result.
func TestOpenAPI_NoSpecExposed(t *testing.T) {
srv := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, _ *http.Request) {
w.WriteHeader(http.StatusNotFound)
}))
defer srv.Close()
result, err := OpenAPI(srv.URL, 5*time.Second, 4, "")
if err != nil {
t.Fatalf("OpenAPI: %v", err)
}
if result != nil {
t.Errorf("expected nil result when no spec exposed, got %+v", result)
}
}
// TestOpenAPI_RejectsUnrelatedJSON makes sure a plain json document served at a
// candidate path (no openapi/swagger version) is not treated as a spec.
func TestOpenAPI_RejectsUnrelatedJSON(t *testing.T) {
srv := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
if r.URL.Path == "/openapi.json" {
_, _ = w.Write([]byte(`{"hello":"world"}`))
return
}
w.WriteHeader(http.StatusNotFound)
}))
defer srv.Close()
result, err := OpenAPI(srv.URL, 5*time.Second, 4, "")
if err != nil {
t.Fatalf("OpenAPI: %v", err)
}
if result != nil {
t.Errorf("unrelated json should not be parsed as a spec, got %+v", result)
}
}
func TestOpenAPIResult_ResultType(t *testing.T) {
r := &OpenAPIResult{}
if r.ResultType() != "openapi" {
t.Errorf("expected result type 'openapi', got %q", r.ResultType())
}
}
+268
View File
@@ -0,0 +1,268 @@
/*
·━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━·
: :
: █▀ █ █▀▀ · Blazing-fast pentesting suite :
: ▄█ █ █▀ · BSD 3-Clause License :
: :
: (c) 2022-2026 vmfunc, xyzeva, :
: lunchcat alumni & contributors :
: :
·━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━·
*/
package scan
import (
"bufio"
"context"
"encoding/json"
"fmt"
"io"
"net/http"
"net/url"
"strings"
"time"
"github.com/dropalldatabases/sif/internal/httpx"
"github.com/dropalldatabases/sif/internal/logger"
"github.com/dropalldatabases/sif/internal/output"
)
// source base urls are vars so tests can repoint them at local fixtures. they
// carry a trailing %s for the domain (or query) each source expects.
var (
crtshBaseURL = "https://crt.sh/?q=%%25.%s&output=json"
certspotterBaseURL = "https://api.certspotter.com/v1/issuances?domain=%s&include_subdomains=true&expand=dns_names"
waybackBaseURL = "http://web.archive.org/cdx/search/cdx?url=*.%s/*&output=text&fl=original&collapse=urlkey"
)
// cap the response we read from any one source so a hostile/huge feed can't
// exhaust memory.
const passiveMaxBytes = 25 * 1024 * 1024
// PassiveResult holds passively-gathered subdomains and historical urls. all
// data comes from third-party feeds; the target itself sees zero traffic.
type PassiveResult struct {
Subdomains []string `json:"subdomains"`
URLs []string `json:"urls"`
}
func (r *PassiveResult) ResultType() string { return "passive" }
// compile-time check so a result-type drift fails the build, not a run.
var _ ScanResult = (*PassiveResult)(nil)
// crtshEntry is one certificate record from crt.sh; name_value may itself hold
// several newline-separated names.
type crtshEntry struct {
NameValue string `json:"name_value"`
}
// certspotterEntry is one issuance from certspotter, expanded to dns names.
type certspotterEntry struct {
DNSNames []string `json:"dns_names"`
}
// Passive performs keyless passive recon: subdomains from certificate
// transparency feeds plus historical urls from the wayback machine. each source
// fails independently so one feed being down doesn't sink the rest.
func Passive(targetURL string, timeout time.Duration, logdir string) (*PassiveResult, error) {
log := output.Module("PASSIVE")
log.Start()
parsed, err := url.Parse(targetURL)
if err != nil {
return nil, fmt.Errorf("parse target url %q: %w", targetURL, err)
}
domain := parsed.Hostname()
if domain == "" {
return nil, fmt.Errorf("target url %q has no host", targetURL)
}
sanitizedURL := stripScheme(targetURL)
if logdir != "" {
if err := logger.WriteHeader(sanitizedURL, logdir, "passive recon"); err != nil {
log.Error("error creating log file: %v", err)
return nil, fmt.Errorf("create passive log: %w", err)
}
}
client := httpx.Client(timeout)
ctx := context.TODO()
subSet := make(map[string]struct{})
urlSet := make(map[string]struct{})
// crt.sh certificate transparency
if subs, err := fetchCrtsh(ctx, client, domain); err != nil {
log.Warn("crt.sh failed: %v", err)
} else {
addAll(subSet, subs)
}
// certspotter certificate transparency
if subs, err := fetchCertspotter(ctx, client, domain); err != nil {
log.Warn("certspotter failed: %v", err)
} else {
addAll(subSet, subs)
}
// wayback machine historical urls
if urls, err := fetchWayback(ctx, client, domain); err != nil {
log.Warn("wayback failed: %v", err)
} else {
addAll(urlSet, urls)
}
result := &PassiveResult{
Subdomains: sortedKeys(subSet),
URLs: sortedKeys(urlSet),
}
logPassiveResults(log, sanitizedURL, logdir, result)
log.Complete(len(result.Subdomains)+len(result.URLs), "discovered")
return result, nil
}
// fetchCrtsh pulls subdomains from crt.sh's certificate transparency json.
func fetchCrtsh(ctx context.Context, client *http.Client, domain string) ([]string, error) {
body, err := passiveGET(ctx, client, fmt.Sprintf(crtshBaseURL, domain))
if err != nil {
return nil, err
}
var entries []crtshEntry
if err := json.Unmarshal(body, &entries); err != nil {
return nil, fmt.Errorf("parse crt.sh json: %w", err)
}
var names []string
for i := 0; i < len(entries); i++ {
// name_value can pack several names separated by newlines.
for _, name := range strings.Split(entries[i].NameValue, "\n") {
if host := normalizeHost(name); host != "" {
names = append(names, host)
}
}
}
return names, nil
}
// fetchCertspotter pulls subdomains from certspotter's keyless issuances feed.
func fetchCertspotter(ctx context.Context, client *http.Client, domain string) ([]string, error) {
body, err := passiveGET(ctx, client, fmt.Sprintf(certspotterBaseURL, domain))
if err != nil {
return nil, err
}
var entries []certspotterEntry
if err := json.Unmarshal(body, &entries); err != nil {
return nil, fmt.Errorf("parse certspotter json: %w", err)
}
var names []string
for i := 0; i < len(entries); i++ {
for _, name := range entries[i].DNSNames {
if host := normalizeHost(name); host != "" {
names = append(names, host)
}
}
}
return names, nil
}
// fetchWayback pulls historical urls from the wayback machine cdx index, which
// returns one original url per line.
func fetchWayback(ctx context.Context, client *http.Client, domain string) ([]string, error) {
body, err := passiveGET(ctx, client, fmt.Sprintf(waybackBaseURL, domain))
if err != nil {
return nil, err
}
var urls []string
scanner := bufio.NewScanner(strings.NewReader(string(body)))
// historical urls can be long; give the scanner a generous line buffer.
scanner.Buffer(make([]byte, 0, 64*1024), 1024*1024)
for scanner.Scan() {
line := strings.TrimSpace(scanner.Text())
if line != "" {
urls = append(urls, line)
}
}
if err := scanner.Err(); err != nil {
return nil, fmt.Errorf("read wayback lines: %w", err)
}
return urls, nil
}
// passiveGET performs a bounded GET against a passive source. non-200 responses
// are treated as a source failure so the caller can skip it.
func passiveGET(ctx context.Context, client *http.Client, reqURL string) ([]byte, error) {
req, err := http.NewRequestWithContext(ctx, http.MethodGet, reqURL, http.NoBody)
if err != nil {
return nil, fmt.Errorf("create request: %w", err)
}
req.Header.Set("Accept", "application/json")
resp, err := client.Do(req) //nolint:bodyclose // drained and closed via httpx.DrainClose
if err != nil {
return nil, fmt.Errorf("request failed: %w", err)
}
// the non-200 branch returns before reading the body, so drain on close to
// keep the conn reusable instead of leaking it.
defer httpx.DrainClose(resp)
if resp.StatusCode != http.StatusOK {
return nil, fmt.Errorf("unexpected status %d", resp.StatusCode)
}
body, err := io.ReadAll(io.LimitReader(resp.Body, passiveMaxBytes))
if err != nil {
return nil, fmt.Errorf("read response: %w", err)
}
return body, nil
}
// normalizeHost lowercases a name and strips a leading wildcard label so
// "*.example.com" and "EXAMPLE.com" collapse to one canonical host.
func normalizeHost(name string) string {
host := strings.ToLower(strings.TrimSpace(name))
host = strings.TrimPrefix(host, "*.")
return host
}
// addAll inserts every value into the dedupe set.
func addAll(set map[string]struct{}, values []string) {
for _, v := range values {
set[v] = struct{}{}
}
}
func logPassiveResults(log *output.ModuleLogger, sanitizedURL, logdir string, result *PassiveResult) {
for _, sub := range result.Subdomains {
log.Success("subdomain: %s", output.Highlight.Render(sub))
}
for _, u := range result.URLs {
log.Info("url: %s", u)
}
if logdir == "" {
return
}
var sb strings.Builder
if len(result.Subdomains) > 0 {
sb.WriteString(fmt.Sprintf("Subdomains (%d):\n", len(result.Subdomains)))
for _, sub := range result.Subdomains {
sb.WriteString(" " + sub + "\n")
}
}
if len(result.URLs) > 0 {
sb.WriteString(fmt.Sprintf("\nHistorical URLs (%d):\n", len(result.URLs)))
for _, u := range result.URLs {
sb.WriteString(" " + u + "\n")
}
}
_ = logger.Write(sanitizedURL, logdir, sb.String())
}
+163
View File
@@ -0,0 +1,163 @@
/*
·━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━·
: :
: █▀ █ █▀▀ · Blazing-fast pentesting suite :
: ▄█ █ █▀ · BSD 3-Clause License :
: :
: (c) 2022-2026 vmfunc, xyzeva, :
: lunchcat alumni & contributors :
: :
·━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━·
*/
package scan
import (
"net/http"
"net/http/httptest"
"testing"
"time"
)
// sample feed payloads. crt.sh packs several names per name_value (newline
// separated) and emits wildcards; certspotter returns expanded dns_names.
const (
crtshFixture = `[
{"name_value": "www.example.com\n*.example.com"},
{"name_value": "api.example.com"},
{"name_value": "WWW.example.com"}
]`
certspotterFixture = `[
{"dns_names": ["mail.example.com", "api.example.com"]},
{"dns_names": ["*.example.com"]}
]`
waybackFixture = "http://example.com/\n" +
"http://example.com/login\n" +
"http://example.com/login\n" +
"\n" +
"http://example.com/admin\n"
)
// fixtureServer serves each passive source on its own path and repoints the
// package base-url vars at it. the vars are restored on cleanup.
func fixtureServer(t *testing.T, crtsh, certspotter, wayback string) *httptest.Server {
t.Helper()
mux := http.NewServeMux()
mux.HandleFunc("/crtsh", func(w http.ResponseWriter, _ *http.Request) {
_, _ = w.Write([]byte(crtsh))
})
mux.HandleFunc("/certspotter", func(w http.ResponseWriter, _ *http.Request) {
_, _ = w.Write([]byte(certspotter))
})
mux.HandleFunc("/wayback", func(w http.ResponseWriter, _ *http.Request) {
_, _ = w.Write([]byte(wayback))
})
srv := httptest.NewServer(mux)
t.Cleanup(srv.Close)
origCrtsh, origCertspotter, origWayback := crtshBaseURL, certspotterBaseURL, waybackBaseURL
// %s still consumes the domain so the production formatting path is exercised.
crtshBaseURL = srv.URL + "/crtsh?q=%s"
certspotterBaseURL = srv.URL + "/certspotter?domain=%s"
waybackBaseURL = srv.URL + "/wayback?url=%s"
t.Cleanup(func() {
crtshBaseURL, certspotterBaseURL, waybackBaseURL = origCrtsh, origCertspotter, origWayback
})
return srv
}
func TestPassive_ParsesAndDedupes(t *testing.T) {
fixtureServer(t, crtshFixture, certspotterFixture, waybackFixture)
result, err := Passive("https://example.com", 5*time.Second, "")
if err != nil {
t.Fatalf("Passive: %v", err)
}
// wildcards stripped, case-folded, and merged across both ct feeds.
wantSubs := map[string]bool{
"www.example.com": false,
"api.example.com": false,
"mail.example.com": false,
"example.com": false, // from "*.example.com"
}
for _, s := range result.Subdomains {
if _, ok := wantSubs[s]; !ok {
t.Errorf("unexpected subdomain %q", s)
continue
}
wantSubs[s] = true
}
for s, seen := range wantSubs {
if !seen {
t.Errorf("missing subdomain %q in %v", s, result.Subdomains)
}
}
if len(result.Subdomains) != len(wantSubs) {
t.Errorf("expected %d deduped subdomains, got %d: %v", len(wantSubs), len(result.Subdomains), result.Subdomains)
}
// wayback: blank line dropped, duplicate /login collapsed.
wantURLs := map[string]bool{
"http://example.com/": false,
"http://example.com/login": false,
"http://example.com/admin": false,
}
for _, u := range result.URLs {
if _, ok := wantURLs[u]; !ok {
t.Errorf("unexpected url %q", u)
continue
}
wantURLs[u] = true
}
if len(result.URLs) != len(wantURLs) {
t.Errorf("expected %d deduped urls, got %d: %v", len(wantURLs), len(result.URLs), result.URLs)
}
}
func TestPassive_SourceFailureIsIsolated(t *testing.T) {
// crt.sh serves garbage that fails to parse; the other feeds must still
// produce results.
fixtureServer(t, "not json", certspotterFixture, waybackFixture)
result, err := Passive("https://example.com", 5*time.Second, "")
if err != nil {
t.Fatalf("Passive should not fail when one source is down: %v", err)
}
if len(result.Subdomains) == 0 {
t.Error("expected certspotter subdomains despite crt.sh failure")
}
if len(result.URLs) == 0 {
t.Error("expected wayback urls despite crt.sh failure")
}
if urlsContain(result.Subdomains, "www.example.com") {
t.Error("crt.sh-only subdomain leaked despite parse failure")
}
}
func TestPassive_ResultType(t *testing.T) {
r := &PassiveResult{}
if r.ResultType() != "passive" {
t.Errorf("ResultType = %q, want passive", r.ResultType())
}
}
func TestNormalizeHost(t *testing.T) {
tests := []struct {
in string
want string
}{
{"www.example.com", "www.example.com"},
{"*.example.com", "example.com"},
{" WWW.Example.COM ", "www.example.com"},
{"", ""},
}
for _, tt := range tests {
if got := normalizeHost(tt.in); got != tt.want {
t.Errorf("normalizeHost(%q) = %q, want %q", tt.in, got, tt.want)
}
}
}
+18 -30
View File
@@ -26,6 +26,7 @@ import (
"github.com/dropalldatabases/sif/internal/httpx"
"github.com/dropalldatabases/sif/internal/logger"
"github.com/dropalldatabases/sif/internal/output"
"github.com/dropalldatabases/sif/internal/pool"
)
// commonPorts is a var so integration tests can repoint it at a fixture.
@@ -75,39 +76,26 @@ func Ports(ctx context.Context, scope string, url string, timeout time.Duration,
var openPorts []string
var mu sync.Mutex
var wg sync.WaitGroup
wg.Add(threads)
for thread := 0; thread < threads; thread++ {
go func(thread int) {
defer wg.Done()
pool.Each(ports, threads, func(port int) {
progress.Increment(strconv.Itoa(port))
for i, port := range ports {
if i%threads != thread {
continue
}
charmlog.Debugf("Looking up: %d", port)
addr := fmt.Sprintf("%s:%d", sanitizedURL, port)
tcp, err := (&net.Dialer{Timeout: timeout}).DialContext(ctx, "tcp", addr)
if err != nil {
charmlog.Debugf("Error %d: %v", port, err)
} else {
progress.Pause()
log.Success("open: %s:%s [tcp]", sanitizedURL, output.Highlight.Render(strconv.Itoa(port)))
progress.Resume()
progress.Increment(strconv.Itoa(port))
charmlog.Debugf("Looking up: %d", port)
addr := fmt.Sprintf("%s:%d", sanitizedURL, port)
tcp, err := (&net.Dialer{Timeout: timeout}).DialContext(ctx, "tcp", addr)
if err != nil {
charmlog.Debugf("Error %d: %v", port, err)
} else {
progress.Pause()
log.Success("open: %s:%s [tcp]", sanitizedURL, output.Highlight.Render(strconv.Itoa(port)))
progress.Resume()
mu.Lock()
openPorts = append(openPorts, strconv.Itoa(port))
mu.Unlock()
_ = tcp.Close()
}
}
}(thread)
}
wg.Wait()
mu.Lock()
openPorts = append(openPorts, strconv.Itoa(port))
mu.Unlock()
_ = tcp.Close()
}
})
progress.Done()
log.Complete(len(openPorts), "open")
+148
View File
@@ -0,0 +1,148 @@
/*
·━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━·
: :
: █▀ █ █▀▀ · Blazing-fast pentesting suite :
: ▄█ █ █▀ · BSD 3-Clause License :
: :
: (c) 2022-2026 vmfunc, xyzeva, :
: lunchcat alumni & contributors :
: :
·━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━·
*/
package scan
import (
"context"
"fmt"
"io"
"net/http"
"regexp"
"strings"
"time"
"github.com/dropalldatabases/sif/internal/httpx"
"github.com/dropalldatabases/sif/internal/logger"
"github.com/dropalldatabases/sif/internal/output"
)
// ProbeResult is the httpx-style liveness snapshot for one target: did it answer,
// where did it land, and the few fingerprint fields worth keeping.
type ProbeResult struct {
URL string `json:"url"`
Alive bool `json:"alive"`
StatusCode int `json:"status_code"`
Title string `json:"title,omitempty"`
Server string `json:"server,omitempty"`
ContentLength int64 `json:"content_length"`
RedirectChain []string `json:"redirect_chain,omitempty"`
}
// probeMaxRedirects caps the chain we'll follow so a redirect loop can't run
// forever; matches httpx's default depth.
const probeMaxRedirects = 10
// probeMaxBody bounds the body we read to extract a <title> (64KB) so a hostile
// or huge response can't exhaust memory.
const probeMaxBody = 64 * 1024
// titleRe pulls the text out of the first <title>; DOTALL so a title spanning
// lines is still caught.
var titleRe = regexp.MustCompile(`(?is)<title[^>]*>(.*?)</title>`)
// Probe checks whether the target is alive and reports its final status, page
// title, Server header, content-length and the redirect chain it walked.
func Probe(targetURL string, timeout time.Duration, logdir string) (*ProbeResult, error) {
log := output.Module("PROBE")
log.Start()
sanitizedURL := stripScheme(targetURL)
if logdir != "" {
if err := logger.WriteHeader(sanitizedURL, logdir, "Live-host probe"); err != nil {
log.Error("error creating log file: %v", err)
return nil, fmt.Errorf("create probe log: %w", err)
}
}
// follow redirects but record every hop; the chain is half the value of a
// probe. capping at probeMaxRedirects stops a loop from spinning forever.
chain := make([]string, 0, 4)
client := httpx.Client(timeout)
client.CheckRedirect = func(req *http.Request, via []*http.Request) error {
if len(via) >= probeMaxRedirects {
return fmt.Errorf("stopped after %d redirects", probeMaxRedirects)
}
chain = append(chain, req.URL.String())
return nil
}
req, err := http.NewRequestWithContext(context.TODO(), http.MethodGet, targetURL, http.NoBody)
if err != nil {
return nil, fmt.Errorf("build probe request: %w", err)
}
resp, err := client.Do(req)
if err != nil {
// a transport error means the host didn't answer; that's a dead probe,
// not a tool failure, so report it rather than bailing.
log.Warn("%s is dead: %v", output.Highlight.Render(sanitizedURL), err)
if logdir != "" {
logger.Write(sanitizedURL, logdir, fmt.Sprintf("dead: %v\n", err))
}
result := &ProbeResult{URL: targetURL, Alive: false, RedirectChain: chain}
log.Complete(0, "alive")
return result, nil
}
defer resp.Body.Close()
body, err := io.ReadAll(io.LimitReader(resp.Body, probeMaxBody))
if err != nil {
return nil, fmt.Errorf("read probe body: %w", err)
}
result := &ProbeResult{
URL: targetURL,
Alive: true,
StatusCode: resp.StatusCode,
Title: extractTitle(body),
Server: resp.Header.Get("Server"),
ContentLength: resp.ContentLength,
RedirectChain: chain,
}
log.Info("%s [%s] %s",
output.Status.Render(fmt.Sprintf("%d", result.StatusCode)),
output.Highlight.Render(result.Title),
output.Muted.Render(result.Server))
if len(chain) > 0 {
log.Info("redirect chain: %s", strings.Join(chain, " -> "))
}
if logdir != "" {
logger.Write(sanitizedURL, logdir,
fmt.Sprintf("alive status=%d title=%q server=%q length=%d\n",
result.StatusCode, result.Title, result.Server, result.ContentLength))
if len(chain) > 0 {
logger.Write(sanitizedURL, logdir, "redirect chain: "+strings.Join(chain, " -> ")+"\n")
}
}
log.Complete(1, "alive")
return result, nil
}
// extractTitle returns the trimmed text of the first <title> in body, or "" when
// there isn't one.
func extractTitle(body []byte) string {
m := titleRe.FindSubmatch(body)
if len(m) < 2 {
return ""
}
return strings.TrimSpace(string(m[1]))
}
// ResultType identifies probe results for the result registry.
func (r *ProbeResult) ResultType() string { return "probe" }
var _ ScanResult = (*ProbeResult)(nil)
+133
View File
@@ -0,0 +1,133 @@
/*
·━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━·
: :
: █▀ █ █▀▀ · Blazing-fast pentesting suite :
: ▄█ █ █▀ · BSD 3-Clause License :
: :
: (c) 2022-2026 vmfunc, xyzeva, :
: lunchcat alumni & contributors :
: :
·━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━·
*/
package scan
import (
"net/http"
"net/http/httptest"
"testing"
"time"
)
func TestProbe_TitleServerStatus(t *testing.T) {
srv := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, _ *http.Request) {
w.Header().Set("Server", "nginx/1.25.3")
w.WriteHeader(http.StatusOK)
w.Write([]byte("<html><head><title> Welcome Home </title></head><body>hi</body></html>"))
}))
defer srv.Close()
result, err := Probe(srv.URL, 5*time.Second, "")
if err != nil {
t.Fatalf("Probe: %v", err)
}
if !result.Alive {
t.Fatalf("expected alive, got %+v", result)
}
if result.StatusCode != http.StatusOK {
t.Errorf("expected status 200, got %d", result.StatusCode)
}
// title text is trimmed of surrounding whitespace
if result.Title != "Welcome Home" {
t.Errorf("expected trimmed title, got %q", result.Title)
}
if result.Server != "nginx/1.25.3" {
t.Errorf("expected server header, got %q", result.Server)
}
}
func TestProbe_RedirectChain(t *testing.T) {
// /a -> /b -> /c(final); the chain should record both intermediate hops the
// client followed before landing on the final 200.
mux := http.NewServeMux()
mux.HandleFunc("/a", func(w http.ResponseWriter, r *http.Request) {
http.Redirect(w, r, "/b", http.StatusFound)
})
mux.HandleFunc("/b", func(w http.ResponseWriter, r *http.Request) {
http.Redirect(w, r, "/c", http.StatusMovedPermanently)
})
mux.HandleFunc("/c", func(w http.ResponseWriter, _ *http.Request) {
w.WriteHeader(http.StatusOK)
w.Write([]byte("<title>final</title>"))
})
srv := httptest.NewServer(mux)
defer srv.Close()
result, err := Probe(srv.URL+"/a", 5*time.Second, "")
if err != nil {
t.Fatalf("Probe: %v", err)
}
if !result.Alive || result.StatusCode != http.StatusOK {
t.Fatalf("expected alive 200 after redirects, got %+v", result)
}
if result.Title != "final" {
t.Errorf("expected title of final hop, got %q", result.Title)
}
// two hops were followed (/b and /c are the urls requested after the first)
if len(result.RedirectChain) != 2 {
t.Fatalf("expected 2 redirect hops, got %d: %v", len(result.RedirectChain), result.RedirectChain)
}
if !hasSuffix(result.RedirectChain[0], "/b") || !hasSuffix(result.RedirectChain[1], "/c") {
t.Errorf("expected chain to walk /b then /c, got %v", result.RedirectChain)
}
}
func TestProbe_DeadHost(t *testing.T) {
// a server we immediately close so the dial fails; a dead host is a reported
// result, not an error.
srv := httptest.NewServer(http.HandlerFunc(func(http.ResponseWriter, *http.Request) {}))
deadURL := srv.URL
srv.Close()
result, err := Probe(deadURL, 2*time.Second, "")
if err != nil {
t.Fatalf("Probe should not error on a dead host: %v", err)
}
if result.Alive {
t.Errorf("expected dead host, got %+v", result)
}
}
func TestProbe_ExtractTitle(t *testing.T) {
tests := []struct {
name string
body string
want string
}{
{"simple", "<title>hello</title>", "hello"},
{"trimmed", "<title> spaced </title>", "spaced"},
{"attrs", `<title lang="en">attr</title>`, "attr"},
{"multiline", "<title>line one\nline two</title>", "line one\nline two"},
{"none", "<html><body>no title</body></html>", ""},
}
for _, tt := range tests {
t.Run(tt.name, func(t *testing.T) {
got := extractTitle([]byte(tt.body))
if got != tt.want {
t.Errorf("extractTitle(%q) = %q, want %q", tt.body, got, tt.want)
}
})
}
}
func TestProbeResult_ResultType(t *testing.T) {
r := &ProbeResult{}
if r.ResultType() != "probe" {
t.Errorf("expected result type 'probe', got %q", r.ResultType())
}
}
// hasSuffix is a tiny local helper so the redirect-chain assertions read clearly.
func hasSuffix(s, suffix string) bool {
return len(s) >= len(suffix) && s[len(s)-len(suffix):] == suffix
}
+307
View File
@@ -0,0 +1,307 @@
/*
·━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━·
: :
: █▀ █ █▀▀ · Blazing-fast pentesting suite :
: ▄█ █ █▀ · BSD 3-Clause License :
: :
: (c) 2022-2026 vmfunc, xyzeva, :
: lunchcat alumni & contributors :
: :
·━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━·
*/
package scan
import (
"context"
"fmt"
"io"
"net/http"
"net/url"
"regexp"
"strings"
"sync"
"time"
charmlog "github.com/charmbracelet/log"
"github.com/dropalldatabases/sif/internal/httpx"
"github.com/dropalldatabases/sif/internal/logger"
"github.com/dropalldatabases/sif/internal/output"
)
// RedirectResult collects every open-redirect found on the target.
type RedirectResult struct {
Findings []RedirectFinding `json:"findings,omitempty"`
TestedParams int `json:"tested_params"`
}
// RedirectFinding is a single param/payload that sends the user off-site.
type RedirectFinding struct {
URL string `json:"url"`
Parameter string `json:"parameter"`
Payload string `json:"payload"`
Location string `json:"location"`
Via string `json:"via"` // header, meta-refresh, or javascript
Severity string `json:"severity"`
}
// redirectMaxBody caps the body we scan for meta/js redirects (100KB).
const redirectMaxBody = 1024 * 100
// the controlled sentinel host we steer redirects toward; a Location that lands
// on it proves the param is attacker-controlled.
const redirectSentinel = "sif-redirect-probe.evil.com"
// params that commonly drive a server-side redirect.
var redirectParams = []string{
"url", "next", "redirect", "redirect_uri", "redirect_url",
"return", "return_url", "returnurl", "returnto", "return_to",
"dest", "destination", "continue", "goto", "go", "target",
"to", "out", "view", "image_url", "checkout_url", "rurl", "u",
}
// payload variants: a plain sentinel plus filter bypasses that browsers still
// resolve as an absolute off-site target. {host} expands to the sentinel.
var redirectPayloads = []string{
"https://{host}", // plain absolute
"//{host}", // scheme-relative
"https:/{host}", // missing slash, browsers normalise it
"https:{host}", // no slashes
"/\\{host}", // backslash trick
"/%2f%2f{host}", // encoded scheme-relative
"https://{host}%00.x.com", // null-byte truncation
"https://x.com@{host}", // userinfo confusion - real host is after @
}
// meta refresh redirect: <meta http-equiv="refresh" content="0;url=...">
var metaRefreshRe = regexp.MustCompile(`(?i)<meta[^>]+http-equiv=["']?refresh["']?[^>]+content=["'][^"']*url=([^"'>\s]+)`)
// client-side redirects baked into a script body
var jsRedirectRe = regexp.MustCompile(`(?i)(?:location\.(?:href|replace|assign)\s*(?:=|\()|window\.location\s*=)\s*["']([^"']+)["']`)
// Redirect probes the target's redirect-prone params for open-redirect.
func Redirect(targetURL string, timeout time.Duration, threads int, logdir string) (*RedirectResult, error) {
log := output.Module("REDIRECT")
log.Start()
spin := output.NewSpinner("Scanning for open redirects")
spin.Start()
sanitizedURL := stripScheme(targetURL)
if logdir != "" {
if err := logger.WriteHeader(sanitizedURL, logdir, "open redirect probe"); err != nil {
spin.Stop()
log.Error("error creating log file: %v", err)
return nil, fmt.Errorf("create redirect log: %w", err)
}
}
parsedURL, err := url.Parse(targetURL)
if err != nil {
spin.Stop()
return nil, fmt.Errorf("parse url: %w", err)
}
existingParams := parsedURL.Query()
// merge target's own params with the common redirect names so we cover both
paramsToTest := make(map[string]bool, len(existingParams)+len(redirectParams))
for param := range existingParams {
paramsToTest[param] = true
}
for _, param := range redirectParams {
paramsToTest[param] = true
}
// don't auto-follow: a 30x Location is exactly what we want to inspect.
client := httpx.Client(timeout)
client.CheckRedirect = func(_ *http.Request, _ []*http.Request) error {
return http.ErrUseLastResponse
}
result := &RedirectResult{
Findings: make([]RedirectFinding, 0, 8),
TestedParams: len(paramsToTest),
}
type workItem struct {
param string
payload string
}
workItems := make([]workItem, 0, len(paramsToTest)*len(redirectPayloads))
for param := range paramsToTest {
for _, raw := range redirectPayloads {
workItems = append(workItems, workItem{param: param, payload: strings.ReplaceAll(raw, "{host}", redirectSentinel)})
}
}
log.Info("testing %d params with %d payloads", len(paramsToTest), len(redirectPayloads))
workChan := make(chan workItem, len(workItems))
for _, item := range workItems {
workChan <- item
}
close(workChan)
seen := make(map[string]bool)
var mu sync.Mutex
var wg sync.WaitGroup
wg.Add(threads)
for t := 0; t < threads; t++ {
go func() {
defer wg.Done()
for item := range workChan {
testURL := buildRedirectURL(parsedURL, existingParams, item.param, item.payload)
location, via, ok := probeRedirect(client, testURL)
if !ok {
continue
}
key := item.param + "|" + item.payload
mu.Lock()
if seen[key] {
mu.Unlock()
continue
}
seen[key] = true
finding := RedirectFinding{
URL: testURL,
Parameter: item.param,
Payload: item.payload,
Location: location,
Via: via,
Severity: "medium",
}
result.Findings = append(result.Findings, finding)
mu.Unlock()
spin.Stop()
log.Warn("open redirect via %s in param %s -> %s",
output.SeverityMedium.Render(via),
output.Highlight.Render(item.param),
output.Status.Render(location))
spin.Start()
if logdir != "" {
logger.Write(sanitizedURL, logdir,
fmt.Sprintf("open redirect: param [%s] via %s -> [%s] (payload %s)\n",
item.param, via, location, item.payload))
}
}
}()
}
wg.Wait()
spin.Stop()
if len(result.Findings) == 0 {
log.Info("no open redirects detected")
log.Complete(0, "found")
return nil, nil //nolint:nilnil // no finding is not an error, mirrors the other scanners
}
log.Complete(len(result.Findings), "found")
return result, nil
}
// buildRedirectURL rebuilds the target with the payload injected into one param,
// preserving the rest of the original query.
func buildRedirectURL(parsedURL *url.URL, existing url.Values, param, payload string) string {
testParams := url.Values{}
for k, v := range existing {
if k != param {
testParams[k] = v
}
}
testParams.Set(param, payload)
return fmt.Sprintf("%s://%s%s?%s", parsedURL.Scheme, parsedURL.Host, parsedURL.Path, testParams.Encode())
}
// probeRedirect requests testURL and reports the first off-site redirect it
// finds, whether that's a 30x Location header, a meta-refresh, or a js
// location assignment. via names the channel; ok is false when nothing points
// at the sentinel.
func probeRedirect(client *http.Client, testURL string) (location, via string, ok bool) {
req, err := http.NewRequestWithContext(context.TODO(), http.MethodGet, testURL, http.NoBody)
if err != nil {
charmlog.Debugf("redirect: build request for %s: %v", testURL, err)
return "", "", false
}
resp, err := client.Do(req) //nolint:bodyclose // drained and closed via httpx.DrainClose
if err != nil {
charmlog.Debugf("redirect: request %s: %v", testURL, err)
return "", "", false
}
// the header-redirect branch returns before reading the body, so drain on
// close to keep that conn reusable instead of leaking it.
defer httpx.DrainClose(resp)
// header redirect: a 30x whose Location resolves to the sentinel host
if resp.StatusCode >= http.StatusMultipleChoices && resp.StatusCode < http.StatusBadRequest {
if loc := resp.Header.Get("Location"); pointsAtSentinel(loc) {
return loc, "header", true
}
}
// body redirects: meta refresh or a client-side location assignment
body, err := io.ReadAll(io.LimitReader(resp.Body, redirectMaxBody))
if err != nil {
return "", "", false
}
bodyStr := string(body)
if m := metaRefreshRe.FindStringSubmatch(bodyStr); len(m) > 1 && pointsAtSentinel(m[1]) {
return m[1], "meta-refresh", true
}
if m := jsRedirectRe.FindStringSubmatch(bodyStr); len(m) > 1 && pointsAtSentinel(m[1]) {
return m[1], "javascript", true
}
return "", "", false
}
// pointsAtSentinel reports whether a redirect target lands on our controlled
// host. We resolve the value the way a browser would so scheme-relative ("//x")
// and backslash tricks are caught, then compare hostnames - a sentinel that only
// shows up in a path or query (still same-origin) is not a redirect off-site.
func pointsAtSentinel(location string) bool {
if location == "" {
return false
}
// browsers treat backslashes in the authority as forward slashes
normalized := strings.ReplaceAll(location, "\\", "/")
parsed, err := url.Parse(normalized)
if err != nil {
// unparseable but still naming the sentinel as the leading authority is a hit
return strings.HasPrefix(strings.TrimLeft(normalized, "/:"), redirectSentinel)
}
// the resolved host is what the navigation actually targets
if strings.EqualFold(parsed.Hostname(), redirectSentinel) {
return true
}
// scheme-relative "//host" parses with an empty scheme but a populated host
if parsed.Host != "" && strings.EqualFold(stripPort(parsed.Host), redirectSentinel) {
return true
}
return false
}
// stripPort drops a trailing :port so host comparisons ignore it.
func stripPort(host string) string {
if h, _, ok := strings.Cut(host, ":"); ok {
return h
}
return host
}
// ResultType identifies open-redirect findings for the result registry.
func (r *RedirectResult) ResultType() string { return "redirect" }
var _ ScanResult = (*RedirectResult)(nil)
+163
View File
@@ -0,0 +1,163 @@
/*
·━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━·
: :
: █▀ █ █▀▀ · Blazing-fast pentesting suite :
: ▄█ █ █▀ · BSD 3-Clause License :
: :
: (c) 2022-2026 vmfunc, xyzeva, :
: lunchcat alumni & contributors :
: :
·━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━·
*/
package scan
import (
"net/http"
"net/http/httptest"
"testing"
"time"
)
func TestRedirect_HeaderLocation(t *testing.T) {
// echoes the "next" param straight into Location, the textbook open redirect.
srv := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
if next := r.URL.Query().Get("next"); next != "" {
w.Header().Set("Location", next)
w.WriteHeader(http.StatusFound)
return
}
w.WriteHeader(http.StatusOK)
}))
defer srv.Close()
result, err := Redirect(srv.URL, 5*time.Second, 4, "")
if err != nil {
t.Fatalf("Redirect: %v", err)
}
if result == nil || len(result.Findings) == 0 {
t.Fatalf("expected open redirect findings, got %+v", result)
}
var sawHeader bool
for _, f := range result.Findings {
if f.Parameter == "next" && f.Via == "header" {
sawHeader = true
}
}
if !sawHeader {
t.Errorf("expected a header redirect via 'next', got %+v", result.Findings)
}
}
func TestRedirect_MetaRefresh(t *testing.T) {
// body-based redirect: a meta refresh pointing at the injected url.
srv := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
dest := r.URL.Query().Get("url")
w.Header().Set("Content-Type", "text/html")
w.WriteHeader(http.StatusOK)
if dest != "" {
//nolint:gosec // deliberate open-redirect fixture for the probe under test
w.Write([]byte(`<html><head><meta http-equiv="refresh" content="0;url=` + dest + `"></head></html>`))
return
}
w.Write([]byte("<html>home</html>"))
}))
defer srv.Close()
result, err := Redirect(srv.URL, 5*time.Second, 4, "")
if err != nil {
t.Fatalf("Redirect: %v", err)
}
if result == nil {
t.Fatalf("expected meta-refresh findings, got nil")
}
var sawMeta bool
for _, f := range result.Findings {
if f.Via == "meta-refresh" {
sawMeta = true
}
}
if !sawMeta {
t.Errorf("expected a meta-refresh redirect finding, got %+v", result.Findings)
}
}
func TestRedirect_NoFalsePositive(t *testing.T) {
tests := []struct {
name string
handler http.HandlerFunc
}{
{
name: "never redirects",
handler: func(w http.ResponseWriter, _ *http.Request) {
w.WriteHeader(http.StatusOK)
w.Write([]byte("<html>home</html>"))
},
},
{
name: "only redirects to a fixed safe path",
handler: func(w http.ResponseWriter, _ *http.Request) {
// ignores the param, always sends users to its own login page.
w.Header().Set("Location", "/login")
w.WriteHeader(http.StatusFound)
},
},
{
name: "reflects param into body but not as a redirect",
handler: func(w http.ResponseWriter, r *http.Request) {
w.WriteHeader(http.StatusOK)
// the value lands in plain text, no meta/js redirect mechanism.
//nolint:gosec // intentional reflection fixture; asserts no false positive
w.Write([]byte("<p>you searched for " + r.URL.Query().Get("next") + "</p>"))
},
},
}
for _, tt := range tests {
t.Run(tt.name, func(t *testing.T) {
srv := httptest.NewServer(tt.handler)
defer srv.Close()
result, err := Redirect(srv.URL, 5*time.Second, 4, "")
if err != nil {
t.Fatalf("Redirect: %v", err)
}
if result != nil && len(result.Findings) > 0 {
t.Errorf("expected no findings, got %+v", result.Findings)
}
})
}
}
func TestPointsAtSentinel(t *testing.T) {
tests := []struct {
name string
location string
want bool
}{
{"absolute https", "https://" + redirectSentinel + "/path", true},
{"scheme-relative", "//" + redirectSentinel, true},
{"backslash trick", "/\\" + redirectSentinel, true},
{"with port", "https://" + redirectSentinel + ":443/", true},
{"empty", "", false},
{"same-site path", "/dashboard", false},
{"sentinel only in path", "https://safe.example.com/" + redirectSentinel, false},
{"sentinel only in query", "https://safe.example.com/?to=" + redirectSentinel, false},
}
for _, tt := range tests {
t.Run(tt.name, func(t *testing.T) {
if got := pointsAtSentinel(tt.location); got != tt.want {
t.Errorf("pointsAtSentinel(%q) = %v, want %v", tt.location, got, tt.want)
}
})
}
}
func TestRedirectResult_ResultType(t *testing.T) {
r := &RedirectResult{}
if r.ResultType() != "redirect" {
t.Errorf("expected result type 'redirect', got %q", r.ResultType())
}
}
+65 -55
View File
@@ -23,13 +23,13 @@ import (
"net/http"
"strconv"
"strings"
"sync"
"time"
"github.com/charmbracelet/log"
"github.com/dropalldatabases/sif/internal/httpx"
"github.com/dropalldatabases/sif/internal/logger"
"github.com/dropalldatabases/sif/internal/output"
"github.com/dropalldatabases/sif/internal/pool"
)
// stripScheme drops the scheme:// prefix from url, or returns it unchanged when
@@ -41,29 +41,50 @@ func stripScheme(url string) string {
return url
}
func fetchRobotsTXT(url string, client *http.Client) *http.Response {
req, err := http.NewRequestWithContext(context.TODO(), http.MethodGet, url, http.NoBody)
if err != nil {
log.Debugf("Error creating request for robots.txt: %s", err)
return nil
}
resp, err := client.Do(req)
if err != nil {
log.Debugf("Error fetching robots.txt: %s", err)
return nil
}
// maxRobotsRedirects caps how many 301 hops fetchRobotsTXT will chase. without
// a bound an A->B->A redirect loop recursed forever and blew the stack.
const maxRobotsRedirects = 10
// fetchRobotsTXT follows 301s to robots.txt iteratively, bounded by both a hop
// cap and a visited set so a redirect cycle terminates instead of recursing
// without end.
func fetchRobotsTXT(url string, client *http.Client) *http.Response {
visited := make(map[string]bool, maxRobotsRedirects)
for hop := 0; hop < maxRobotsRedirects; hop++ {
if visited[url] {
log.Debugf("redirect loop hit at %s, stopping", url)
return nil
}
visited[url] = true
req, err := http.NewRequestWithContext(context.TODO(), http.MethodGet, url, http.NoBody)
if err != nil {
log.Debugf("Error creating request for robots.txt: %s", err)
return nil
}
resp, err := client.Do(req)
if err != nil {
log.Debugf("Error fetching robots.txt: %s", err)
return nil
}
if resp.StatusCode != http.StatusMovedPermanently {
return resp
}
if resp.StatusCode == http.StatusMovedPermanently {
redirectURL := resp.Header.Get("Location")
// only the Location header is used here; drain so the conn is reusable.
httpx.DrainClose(resp)
if redirectURL == "" {
log.Debugf("Redirect location is empty for %s", url)
return nil
}
resp.Body.Close()
return fetchRobotsTXT(redirectURL, client)
url = redirectURL
}
return resp
log.Debugf("robots.txt redirect depth exceeded (%d hops)", maxRobotsRedirects)
return nil
}
// Scan performs a basic URL scan, including checks for robots.txt and other common endpoints.
@@ -91,11 +112,13 @@ func Scan(url string, timeout time.Duration, threads int, logdir string) {
return http.ErrUseLastResponse
}
resp := fetchRobotsTXT(url+"/robots.txt", client)
resp := fetchRobotsTXT(url+"/robots.txt", client) //nolint:bodyclose // drained and closed via httpx.DrainClose
if resp == nil {
return
}
defer resp.Body.Close()
// drain on close: the non-success branch never reads the body, so a bare
// close would leak the conn instead of returning it to the pool.
defer httpx.DrainClose(resp)
if resp.StatusCode != 404 && resp.StatusCode != 301 && resp.StatusCode != 302 && resp.StatusCode != 307 {
output.Success("File %s found", output.Status.Render("robots.txt"))
@@ -107,45 +130,32 @@ func Scan(url string, timeout time.Duration, threads int, logdir string) {
robotsData = append(robotsData, scanner.Text())
}
var wg sync.WaitGroup
wg.Add(threads)
for thread := 0; thread < threads; thread++ {
go func(thread int) {
defer wg.Done()
pool.Each(robotsData, threads, func(robot string) {
if robot == "" || strings.HasPrefix(robot, "#") || strings.HasPrefix(robot, "User-agent: ") || strings.HasPrefix(robot, "Sitemap: ") {
return
}
for i, robot := range robotsData {
if i%threads != thread {
continue
}
_, sanitizedRobot, _ := strings.Cut(robot, ": ")
log.Debugf("%s", robot)
robotReq, err := http.NewRequestWithContext(context.TODO(), http.MethodGet, url+"/"+sanitizedRobot, http.NoBody)
if err != nil {
log.Debugf("Error creating request for %s: %s", sanitizedRobot, err)
return
}
resp, err := client.Do(robotReq) //nolint:bodyclose // drained and closed via httpx.DrainClose
if err != nil {
log.Debugf("Error %s: %s", sanitizedRobot, err)
return
}
if robot == "" || strings.HasPrefix(robot, "#") || strings.HasPrefix(robot, "User-agent: ") || strings.HasPrefix(robot, "Sitemap: ") {
continue
}
_, sanitizedRobot, _ := strings.Cut(robot, ": ")
log.Debugf("%s", robot)
robotReq, err := http.NewRequestWithContext(context.TODO(), http.MethodGet, url+"/"+sanitizedRobot, http.NoBody)
if err != nil {
log.Debugf("Error creating request for %s: %s", sanitizedRobot, err)
continue
}
resp, err := client.Do(robotReq)
if err != nil {
log.Debugf("Error %s: %s", sanitizedRobot, err)
continue
}
if resp.StatusCode != 404 {
output.Success("%s from robots: %s", output.Status.Render(strconv.Itoa(resp.StatusCode)), output.Highlight.Render(sanitizedRobot))
if logdir != "" {
logger.Write(sanitizedURL, logdir, strconv.Itoa(resp.StatusCode)+" from robots: ["+sanitizedRobot+"]\n")
}
}
resp.Body.Close()
if resp.StatusCode != 404 {
output.Success("%s from robots: %s", output.Status.Render(strconv.Itoa(resp.StatusCode)), output.Highlight.Render(sanitizedRobot))
if logdir != "" {
logger.Write(sanitizedURL, logdir, strconv.Itoa(resp.StatusCode)+" from robots: ["+sanitizedRobot+"]\n")
}
}(thread)
}
wg.Wait()
}
// status only; drain so the conn returns to the pool.
httpx.DrainClose(resp)
})
}
}
+99
View File
@@ -3,7 +3,9 @@ package scan
import (
"net/http"
"net/http/httptest"
"strconv"
"strings"
"sync/atomic"
"testing"
"time"
)
@@ -155,6 +157,103 @@ func TestFetchRobotsTXT_Redirect(t *testing.T) {
}
}
// an A->B->A redirect loop must terminate (return nil) instead of recursing
// forever and blowing the stack.
func TestFetchRobotsTXT_RedirectLoop(t *testing.T) {
var serverA, serverB *httptest.Server
serverA = httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
w.Header().Set("Location", serverB.URL+"/robots.txt")
w.WriteHeader(http.StatusMovedPermanently)
}))
defer serverA.Close()
serverB = httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
w.Header().Set("Location", serverA.URL+"/robots.txt")
w.WriteHeader(http.StatusMovedPermanently)
}))
defer serverB.Close()
client := &http.Client{
Timeout: 5 * time.Second,
CheckRedirect: func(req *http.Request, via []*http.Request) error {
return http.ErrUseLastResponse
},
}
// the hop cap + visited set guarantee termination; a regression that drops
// either would spin forever and the test harness timeout would catch it.
resp := fetchRobotsTXT(serverA.URL+"/robots.txt", client)
if resp != nil {
resp.Body.Close()
t.Errorf("expected nil on redirect loop, got status %d", resp.StatusCode)
}
}
// a redirect chain longer than the hop cap stops at the bound rather than
// following indefinitely.
func TestFetchRobotsTXT_DepthCap(t *testing.T) {
var hops int32
srv := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
// each hop points at a fresh path so the visited set never trips; only
// the depth cap can stop this.
n := atomic.AddInt32(&hops, 1)
w.Header().Set("Location", "/r"+strconv.Itoa(int(n)))
w.WriteHeader(http.StatusMovedPermanently)
}))
defer srv.Close()
client := &http.Client{
Timeout: 5 * time.Second,
CheckRedirect: func(req *http.Request, via []*http.Request) error {
return http.ErrUseLastResponse
},
}
resp := fetchRobotsTXT(srv.URL+"/robots.txt", client)
if resp != nil {
resp.Body.Close()
t.Errorf("expected nil once depth cap exceeded, got status %d", resp.StatusCode)
}
if got := atomic.LoadInt32(&hops); got > maxRobotsRedirects {
t.Errorf("followed %d hops, expected at most %d", got, maxRobotsRedirects)
}
}
// the old code flagged a dangling cname on ANY cname, including LookupCNAME
// echoing the host back for a plain A record. only an off-host cname into a
// known takeoverable provider should count.
func TestDanglingProvider(t *testing.T) {
tests := []struct {
name string
subdomain string
cname string
wantService string
wantOK bool
}{
{"github pages dangling", "blog.example.com", "example.github.io.", "GitHub Pages", true},
{"heroku dangling", "app.example.com", "example.herokuapp.com.", "Heroku", true},
{"s3 dangling", "files.example.com", "bucket.s3.amazonaws.com.", "Amazon S3", true},
{"self-reference is not dangling", "www.example.com", "www.example.com.", "", false},
{"on-domain cname is not dangling", "www.example.com", "lb.example.com.", "", false},
{"unknown provider is not dangling", "x.example.com", "host.notaprovider.net.", "", false},
{"empty cname is not dangling", "x.example.com", "", "", false},
{"case-insensitive match", "x.example.com", "X.GitHub.IO.", "GitHub Pages", true},
}
for _, tt := range tests {
t.Run(tt.name, func(t *testing.T) {
service, ok := danglingProvider(tt.subdomain, tt.cname)
if ok != tt.wantOK {
t.Errorf("danglingProvider(%q, %q) ok = %v, want %v", tt.subdomain, tt.cname, ok, tt.wantOK)
}
if service != tt.wantService {
t.Errorf("danglingProvider(%q, %q) service = %q, want %q", tt.subdomain, tt.cname, service, tt.wantService)
}
})
}
}
func TestSubdomainTakeoverResult(t *testing.T) {
result := SubdomainTakeoverResult{
Subdomain: "test.example.com",
+3 -2
View File
@@ -71,11 +71,12 @@ func SecurityHeaders(url string, timeout time.Duration, logdir string) (Security
if err != nil {
return nil, err
}
resp, err := client.Do(req)
resp, err := client.Do(req) //nolint:bodyclose // drained and closed via httpx.DrainClose
if err != nil {
return nil, err
}
defer resp.Body.Close()
// header-only scan: drain on close so the conn is returned to the pool.
defer httpx.DrainClose(resp)
results := gradeSecurityHeaders(resp.Header, strings.HasPrefix(url, "https://"))
+4 -2
View File
@@ -187,11 +187,13 @@ func doSTRequest(client *http.Client, reqURL, apiKey string) ([]byte, error) {
req.Header.Set("APIKEY", apiKey)
req.Header.Set("Accept", "application/json")
resp, err := client.Do(req)
resp, err := client.Do(req) //nolint:bodyclose // drained and closed via httpx.DrainClose
if err != nil {
return nil, fmt.Errorf("SecurityTrails request failed: %w", err)
}
defer resp.Body.Close()
// the auth/rate-limit branches return before reading the body, so drain on
// close to keep the conn reusable instead of leaking it.
defer httpx.DrainClose(resp)
if resp.StatusCode == http.StatusForbidden || resp.StatusCode == http.StatusUnauthorized {
return nil, fmt.Errorf("invalid SecurityTrails API key (status %d)", resp.StatusCode)
+4 -2
View File
@@ -188,11 +188,13 @@ func queryShodanHost(ip string, apiKey string, timeout time.Duration) (*ShodanRe
if err != nil {
return nil, fmt.Errorf("failed to create Shodan request: %w", err)
}
resp, err := client.Do(req)
resp, err := client.Do(req) //nolint:bodyclose // drained and closed via httpx.DrainClose
if err != nil {
return nil, fmt.Errorf("failed to query Shodan: %w", err)
}
defer resp.Body.Close()
// the unauthorized/not-found branches return before reading the body, so
// drain on close to keep the conn reusable instead of leaking it.
defer httpx.DrainClose(resp)
if resp.StatusCode == http.StatusUnauthorized {
return nil, fmt.Errorf("invalid Shodan API key")
+2 -1
View File
@@ -208,7 +208,8 @@ func SQL(targetURL string, timeout time.Duration, threads int, logdir string) (*
}
}
} else {
resp.Body.Close()
// uninteresting status; body never read, so drain to reuse the conn.
httpx.DrainClose(resp)
}
}
}()
+80 -39
View File
@@ -20,12 +20,12 @@ import (
"net/http"
"os"
"strings"
"sync"
"time"
"github.com/charmbracelet/log"
"github.com/dropalldatabases/sif/internal/httpx"
"github.com/dropalldatabases/sif/internal/logger"
"github.com/dropalldatabases/sif/internal/pool"
"github.com/dropalldatabases/sif/internal/styles"
)
@@ -37,6 +37,36 @@ type SubdomainTakeoverResult struct {
Service string `json:"service,omitempty"`
}
// takeoverProviders maps a takeoverable third-party's cname apex to its service
// name. a "no such host" on a subdomain only counts as a dangling-cname takeover
// when the cname points at one of these and the target is unclaimed - a cname
// to anything else (or to the host itself) is a normal record, not a finding.
var takeoverProviders = map[string]string{
"github.io": "GitHub Pages",
"herokuapp.com": "Heroku",
"herokudns.com": "Heroku",
"myshopify.com": "Shopify",
"wordpress.com": "WordPress",
"s3.amazonaws.com": "Amazon S3",
"ghost.io": "Ghost",
"pantheonsite.io": "Pantheon",
"zendesk.com": "Zendesk",
"surge.sh": "Surge",
"bitbucket.io": "Bitbucket",
"fastly.net": "Fastly",
"helpscoutdocs.com": "Helpscout",
"cargocollective.com": "Cargo",
"uservoice.com": "Uservoice",
"webflow.io": "Webflow",
"readthedocs.io": "ReadTheDocs",
"azurewebsites.net": "Azure",
"cloudapp.net": "Azure",
"trafficmanager.net": "Azure",
"blob.core.windows.net": "Azure",
"netlify.app": "Netlify",
"netlify.com": "Netlify",
}
// SubdomainTakeover checks dnsResults for dangling subdomains pointing at
// unclaimed third-party services.
func SubdomainTakeover(url string, dnsResults []string, timeout time.Duration, threads int, logdir string) ([]SubdomainTakeoverResult, error) {
@@ -57,44 +87,29 @@ func SubdomainTakeover(url string, dnsResults []string, timeout time.Duration, t
client := httpx.Client(timeout)
var wg sync.WaitGroup
wg.Add(threads)
// buffered to the full candidate count so a send never blocks: Each only
// returns once every worker is done, and the channel is drained afterwards.
resultsChan := make(chan SubdomainTakeoverResult, len(dnsResults))
for thread := 0; thread < threads; thread++ {
go func(thread int) {
defer wg.Done()
pool.Each(dnsResults, threads, func(subdomain string) {
vulnerable, service := checkSubdomainTakeover(subdomain, client)
result := SubdomainTakeoverResult{
Subdomain: subdomain,
Vulnerable: vulnerable,
Service: service,
}
resultsChan <- result
for i, subdomain := range dnsResults {
if i%threads != thread {
continue
}
vulnerable, service := checkSubdomainTakeover(subdomain, client)
result := SubdomainTakeoverResult{
Subdomain: subdomain,
Vulnerable: vulnerable,
Service: service,
}
resultsChan <- result
if vulnerable {
subdomainlog.Warnf("Potential subdomain takeover: %s (%s)", styles.Highlight.Render(subdomain), service)
if logdir != "" {
logger.Write(sanitizedURL, logdir, fmt.Sprintf("Potential subdomain takeover: %s (%s)\n", subdomain, service))
}
} else {
subdomainlog.Infof("Subdomain not vulnerable: %s", subdomain)
}
if vulnerable {
subdomainlog.Warnf("Potential subdomain takeover: %s (%s)", styles.Highlight.Render(subdomain), service)
if logdir != "" {
logger.Write(sanitizedURL, logdir, fmt.Sprintf("Potential subdomain takeover: %s (%s)\n", subdomain, service))
}
}(thread)
}
go func() {
wg.Wait()
close(resultsChan)
}()
} else {
subdomainlog.Infof("Subdomain not vulnerable: %s", subdomain)
}
})
close(resultsChan)
var results []SubdomainTakeoverResult
for result := range resultsChan {
@@ -104,6 +119,27 @@ func SubdomainTakeover(url string, dnsResults []string, timeout time.Duration, t
return results, nil
}
// danglingProvider reports whether cname points off-host at a known
// takeoverable provider. a self-referential cname (LookupCNAME echoing an A
// record back as the host) is rejected, since that's a live host, not a
// dangling pointer.
func danglingProvider(subdomain, cname string) (string, bool) {
// LookupCNAME returns a fqdn with a trailing dot; strip it so suffix and
// self-reference checks compare like-for-like.
target := strings.ToLower(strings.TrimSuffix(cname, "."))
host := strings.ToLower(strings.TrimSuffix(subdomain, "."))
if target == "" || target == host {
return "", false
}
for apex, service := range takeoverProviders {
if target == apex || strings.HasSuffix(target, "."+apex) {
return service, true
}
}
return "", false
}
func checkSubdomainTakeover(subdomain string, client *http.Client) (bool, string) {
req, err := http.NewRequestWithContext(context.TODO(), http.MethodGet, "http://"+subdomain, http.NoBody)
if err != nil {
@@ -111,11 +147,16 @@ func checkSubdomainTakeover(subdomain string, client *http.Client) (bool, string
}
resp, err := client.Do(req)
if err != nil {
// a dead host only matters if its cname still points at an unclaimed
// third-party service. LookupCNAME echoes the host back for plain A
// records, so "any cname" is not a signal - the cname must resolve to a
// known takeoverable provider and not be the host itself.
if strings.Contains(err.Error(), "no such host") {
// Check if CNAME exists
cname, err := net.DefaultResolver.LookupCNAME(context.TODO(), subdomain)
if err == nil && cname != "" {
return true, "Dangling CNAME"
cname, lookupErr := net.DefaultResolver.LookupCNAME(context.TODO(), subdomain)
if lookupErr == nil {
if service, ok := danglingProvider(subdomain, cname); ok {
return true, service + " (Dangling CNAME)"
}
}
}
return false, ""
+342
View File
@@ -0,0 +1,342 @@
/*
··
: :
: · Blazing-fast pentesting suite :
: · BSD 3-Clause License :
: :
: (c) 2022-2026 vmfunc, xyzeva, :
: lunchcat alumni & contributors :
: :
··
*/
package scan
import (
"context"
"fmt"
"io"
"net/http"
"net/url"
"strings"
"sync"
"time"
charmlog "github.com/charmbracelet/log"
"github.com/dropalldatabases/sif/internal/httpx"
"github.com/dropalldatabases/sif/internal/logger"
"github.com/dropalldatabases/sif/internal/output"
)
// XSSResult collects every likely reflected-xss point on the target.
type XSSResult struct {
Findings []XSSFinding `json:"findings,omitempty"`
TestedParams int `json:"tested_params"`
}
// XSSFinding is a reflection where one or more breaking chars survived
// unescaped in a context that makes injection plausible.
type XSSFinding struct {
URL string `json:"url"`
Parameter string `json:"parameter"`
Context string `json:"context"` // html, attribute, or script
SurvivedRaw []string `json:"survived_raw"` // breaking chars echoed unescaped
Severity string `json:"severity"`
}
// xssMaxBody caps the body we scan for the canary (100KB).
const xssMaxBody = 1024 * 100
// canaryToken is a unique, alnum-only marker we can grep for unambiguously; it
// survives every output encoder so a missing reflection means no echo at all.
const canaryToken = "sifxss9173canary" //nolint:gosec // not a credential, just a reflection marker
// the chars that let an attacker break out of a context; we inject the canary
// wrapped in each and check which come back raw.
var xssBreakChars = []string{"<", ">", "\"", "'", "`"}
// params we test when the target carries none of its own.
var xssParams = []string{
"q", "s", "search", "query", "id", "name", "page",
"keyword", "lang", "redirect", "url", "return", "ref",
"message", "msg", "error", "title", "text", "comment",
}
// XSS probes the target's params for reflected cross-site scripting.
func XSS(targetURL string, timeout time.Duration, threads int, logdir string) (*XSSResult, error) {
log := output.Module("XSS")
log.Start()
spin := output.NewSpinner("Scanning for reflected XSS")
spin.Start()
sanitizedURL := stripScheme(targetURL)
if logdir != "" {
if err := logger.WriteHeader(sanitizedURL, logdir, "reflected XSS probe"); err != nil {
spin.Stop()
log.Error("error creating log file: %v", err)
return nil, fmt.Errorf("create xss log: %w", err)
}
}
parsedURL, err := url.Parse(targetURL)
if err != nil {
spin.Stop()
return nil, fmt.Errorf("parse url: %w", err)
}
existingParams := parsedURL.Query()
paramsToTest := make(map[string]bool, len(existingParams)+len(xssParams))
for param := range existingParams {
paramsToTest[param] = true
}
for _, param := range xssParams {
paramsToTest[param] = true
}
client := httpx.Client(timeout)
client.CheckRedirect = func(_ *http.Request, via []*http.Request) error {
if len(via) >= corsMaxRedirects {
return http.ErrUseLastResponse
}
return nil
}
result := &XSSResult{
Findings: make([]XSSFinding, 0, 8),
TestedParams: len(paramsToTest),
}
params := make([]string, 0, len(paramsToTest))
for param := range paramsToTest {
params = append(params, param)
}
log.Info("testing %d params with reflection canary", len(paramsToTest))
paramChan := make(chan string, len(params))
for _, param := range params {
paramChan <- param
}
close(paramChan)
seen := make(map[string]bool)
var mu sync.Mutex
var wg sync.WaitGroup
wg.Add(threads)
for t := 0; t < threads; t++ {
go func() {
defer wg.Done()
for param := range paramChan {
finding, ok := probeXSS(client, parsedURL, existingParams, param)
if !ok {
continue
}
mu.Lock()
if seen[param] {
mu.Unlock()
continue
}
seen[param] = true
result.Findings = append(result.Findings, finding)
mu.Unlock()
spin.Stop()
log.Warn("reflected xss in param %s (%s context, raw: %s)",
output.Highlight.Render(param),
output.SeverityHigh.Render(finding.Context),
strings.Join(finding.SurvivedRaw, ""))
spin.Start()
if logdir != "" {
logger.Write(sanitizedURL, logdir,
fmt.Sprintf("reflected XSS: param [%s] in %s context, unescaped chars [%s]\n",
param, finding.Context, strings.Join(finding.SurvivedRaw, "")))
}
}
}()
}
wg.Wait()
spin.Stop()
if len(result.Findings) == 0 {
log.Info("no reflected xss detected")
log.Complete(0, "found")
return nil, nil //nolint:nilnil // no finding is not an error, mirrors the other scanners
}
log.Complete(len(result.Findings), "found")
return result, nil
}
// probeXSS injects a canary wrapped in the breaking chars into one param, then
// inspects the reflection: it classifies where the canary landed and which
// breaking chars came back unescaped there. ok is false unless at least one
// dangerous char survived in an exploitable context.
func probeXSS(client *http.Client, parsedURL *url.URL, existing url.Values, param string) (XSSFinding, bool) {
// wrap the canary so a single request tells us both that it reflected and
// which surrounding chars survived: <canary> "canary' `canary`
payload := fmt.Sprintf("<%s>\"%s'`%s`", canaryToken, canaryToken, canaryToken)
testParams := url.Values{}
for k, v := range existing {
if k != param {
testParams[k] = v
}
}
testParams.Set(param, payload)
testURL := fmt.Sprintf("%s://%s%s?%s", parsedURL.Scheme, parsedURL.Host, parsedURL.Path, testParams.Encode())
req, err := http.NewRequestWithContext(context.TODO(), http.MethodGet, testURL, http.NoBody)
if err != nil {
charmlog.Debugf("xss: build request for %s: %v", testURL, err)
return XSSFinding{}, false
}
resp, err := client.Do(req)
if err != nil {
charmlog.Debugf("xss: request %s: %v", testURL, err)
return XSSFinding{}, false
}
body, err := io.ReadAll(io.LimitReader(resp.Body, xssMaxBody))
resp.Body.Close()
if err != nil {
return XSSFinding{}, false
}
bodyStr := string(body)
// no echo of the canary at all means the param isn't reflected; bail early.
if !strings.Contains(bodyStr, canaryToken) {
return XSSFinding{}, false
}
reflectCtx := classifyXSSContext(bodyStr)
survived := survivingBreakChars(bodyStr)
// a reflection that escaped every dangerous char can't break out, so it's not
// reported - only raw chars that matter in the detected context count.
survived = relevantForContext(reflectCtx, survived)
if len(survived) == 0 {
return XSSFinding{}, false
}
return XSSFinding{
URL: testURL,
Parameter: param,
Context: reflectCtx,
SurvivedRaw: survived,
Severity: "high",
}, true
}
// classifyXSSContext guesses where the canary was reflected. We look at the
// markup immediately around the token: a live <canary> tag means html text, a
// reflection inside a <script> block means js, otherwise it sits in an attribute
// value. The html-tag check wins because it's the most directly exploitable.
func classifyXSSContext(body string) string {
// a surviving "<canary>" means the < and > both passed through into markup
if strings.Contains(body, "<"+canaryToken+">") {
return "html"
}
// reflected between <script> ... </script> is a script context
for {
open := strings.Index(body, "<script")
if open < 0 {
break
}
closeIdx := strings.Index(body[open:], "</script>")
if closeIdx < 0 {
break
}
segment := body[open : open+closeIdx]
if strings.Contains(segment, canaryToken) {
return "script"
}
body = body[open+closeIdx+len("</script>"):]
}
// default: echoed inside an html attribute value
return "attribute"
}
// survivingBreakChars reports which dangerous chars came back next to the canary
// unescaped. We only trust occurrences adjacent to the token so unrelated chars
// elsewhere on the page don't create false positives.
func survivingBreakChars(body string) []string {
survived := make([]string, 0, len(xssBreakChars))
markers := []string{
"<" + canaryToken, // leading < survived
canaryToken + ">", // trailing > survived
"\"" + canaryToken, // leading " survived
canaryToken + "'", // trailing ' survived
"`" + canaryToken, // backtick wrap survived (token + ` and ` + token)
canaryToken + "`",
}
present := make(map[string]bool, len(xssBreakChars))
for i := 0; i < len(markers); i++ {
if !strings.Contains(body, markers[i]) {
continue
}
switch {
case strings.HasPrefix(markers[i], "<"):
present["<"] = true
case strings.HasSuffix(markers[i], ">"):
present[">"] = true
case strings.HasPrefix(markers[i], "\""):
present["\""] = true
case strings.HasSuffix(markers[i], "'"):
present["'"] = true
default:
present["`"] = true
}
}
// keep the canonical order for stable output
for i := 0; i < len(xssBreakChars); i++ {
if present[xssBreakChars[i]] {
survived = append(survived, xssBreakChars[i])
}
}
return survived
}
// relevantForContext filters surviving chars to the ones that actually enable a
// breakout in the detected context: angle brackets matter in html, quotes and
// backticks matter inside attributes/scripts.
func relevantForContext(reflectCtx string, survived []string) []string {
wanted := make(map[string]bool, len(survived))
switch reflectCtx {
case "html":
wanted["<"] = true
wanted[">"] = true
case "attribute":
// breaking out of an attribute value needs the quote that delimits it; a
// bare backtick isn't a delimiter in html, so it doesn't count here.
wanted["\""] = true
wanted["'"] = true
case "script":
// a quote, backtick, or angle bracket all let you close/escape the script
wanted["\""] = true
wanted["'"] = true
wanted["`"] = true
wanted["<"] = true
wanted[">"] = true
}
filtered := make([]string, 0, len(survived))
for i := 0; i < len(survived); i++ {
if wanted[survived[i]] {
filtered = append(filtered, survived[i])
}
}
return filtered
}
// ResultType identifies reflected-xss findings for the result registry.
func (r *XSSResult) ResultType() string { return "xss" }
var _ ScanResult = (*XSSResult)(nil)
+153
View File
@@ -0,0 +1,153 @@
/*
··
: :
: · Blazing-fast pentesting suite :
: · BSD 3-Clause License :
: :
: (c) 2022-2026 vmfunc, xyzeva, :
: lunchcat alumni & contributors :
: :
··
*/
package scan
import (
"html"
"net/http"
"net/http/httptest"
"testing"
"time"
)
// reflectsRaw echoes the named param straight into html text, so the breaking
// chars survive unescaped - a reflected xss sink.
func reflectsRaw(param string) *httptest.Server {
return httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
v := r.URL.Query().Get(param)
w.Header().Set("Content-Type", "text/html")
w.WriteHeader(http.StatusOK)
//nolint:gosec // deliberate reflected-xss fixture for the probe under test
w.Write([]byte("<html><body><div>" + v + "</div></body></html>"))
}))
}
func TestXSS_DetectsRawHTMLReflection(t *testing.T) {
srv := reflectsRaw("q")
defer srv.Close()
result, err := XSS(srv.URL, 5*time.Second, 4, "")
if err != nil {
t.Fatalf("XSS: %v", err)
}
if result == nil || len(result.Findings) == 0 {
t.Fatalf("expected reflected xss findings, got %+v", result)
}
var found *XSSFinding
for i := range result.Findings {
if result.Findings[i].Parameter == "q" {
found = &result.Findings[i]
}
}
if found == nil {
t.Fatalf("expected a finding on param 'q', got %+v", result.Findings)
}
if found.Context != "html" {
t.Errorf("expected html context, got %s", found.Context)
}
if len(found.SurvivedRaw) == 0 {
t.Errorf("expected surviving breaking chars, got none")
}
}
func TestXSS_NoFalsePositiveWhenEscaped(t *testing.T) {
// the server html-escapes the reflection, so no breaking char survives raw.
srv := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
v := r.URL.Query().Get("q")
w.Header().Set("Content-Type", "text/html")
w.WriteHeader(http.StatusOK)
w.Write([]byte("<html><body><div>" + html.EscapeString(v) + "</div></body></html>"))
}))
defer srv.Close()
result, err := XSS(srv.URL, 5*time.Second, 4, "")
if err != nil {
t.Fatalf("XSS: %v", err)
}
if result != nil && len(result.Findings) > 0 {
t.Errorf("expected no findings when reflection is escaped, got %+v", result.Findings)
}
}
func TestXSS_NoFalsePositiveWhenNotReflected(t *testing.T) {
// never echoes the input back, so nothing is injectable.
srv := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, _ *http.Request) {
w.Header().Set("Content-Type", "text/html")
w.WriteHeader(http.StatusOK)
w.Write([]byte("<html><body>static page</body></html>"))
}))
defer srv.Close()
result, err := XSS(srv.URL, 5*time.Second, 4, "")
if err != nil {
t.Fatalf("XSS: %v", err)
}
if result != nil && len(result.Findings) > 0 {
t.Errorf("expected no findings on static page, got %+v", result.Findings)
}
}
func TestClassifyXSSContext(t *testing.T) {
tests := []struct {
name string
body string
want string
}{
{
name: "live html tag",
body: "<div><" + canaryToken + "></div>",
want: "html",
},
{
name: "inside script block",
body: "<script>var x = '" + canaryToken + "';</script>",
want: "script",
},
{
name: "attribute value",
body: `<input value="` + canaryToken + `">`,
want: "attribute",
},
}
for _, tt := range tests {
t.Run(tt.name, func(t *testing.T) {
if got := classifyXSSContext(tt.body); got != tt.want {
t.Errorf("classifyXSSContext() = %q, want %q", got, tt.want)
}
})
}
}
func TestSurvivingBreakChars(t *testing.T) {
// the canary is wrapped exactly as the probe injects it; all five chars survive.
body := "<" + canaryToken + ">\"" + canaryToken + "'`" + canaryToken + "`"
got := survivingBreakChars(body)
want := map[string]bool{"<": true, ">": true, "\"": true, "'": true, "`": true}
if len(got) != len(want) {
t.Fatalf("expected %d surviving chars, got %v", len(want), got)
}
for _, c := range got {
if !want[c] {
t.Errorf("unexpected surviving char %q", c)
}
}
}
func TestXSSResult_ResultType(t *testing.T) {
r := &XSSResult{}
if r.ResultType() != "xss" {
t.Errorf("expected result type 'xss', got %q", r.ResultType())
}
}
+204
View File
@@ -0,0 +1,204 @@
/*
··
: :
: · Blazing-fast pentesting suite :
: · BSD 3-Clause License :
: :
: (c) 2022-2026 vmfunc, xyzeva, :
: lunchcat alumni & contributors :
: :
··
*/
// Package store persists a run's normalized findings as a json snapshot, one
// file per target, so a later run can diff against it and surface only what
// changed. it leans on encoding/json + os only - no new deps - and keys the
// delta off finding.Key, the identity the finding layer already guarantees is
// stable across runs.
package store
import (
"encoding/json"
"fmt"
"os"
"path/filepath"
"strings"
"github.com/dropalldatabases/sif/internal/finding"
)
// snapshotFileMode is applied to written snapshot files: owner read/write only.
// a snapshot enumerates a target's findings (urls, secrets, takeovers) and is
// not meant for other users on the box, so it stays 0600.
const snapshotFileMode = 0o600
// stateDirMode is applied to directories the store creates: owner rwx, group rx,
// no world access. matches the 0o750 the bundle asks for so the state tree isn't
// world-readable.
const stateDirMode = 0o750
// snapshotExt is the extension every snapshot file carries; makes the state dir
// self-describing and lets Load reconstruct the path from a bare target.
const snapshotExt = ".json"
// defaultDirName is the sif-owned subdirectory under the user's config dir when
// no explicit store dir is given. DefaultDir joins it under os.UserConfigDir().
const defaultDirName = "sif"
// stateSubDir separates snapshots from anything else sif might drop in its
// config dir later, so the state tree is a single sweepable directory.
const stateSubDir = "state"
// DefaultDir returns the fallback snapshot location: <user-config>/sif/state.
// callers pass it when -store is unset and there's no logdir to reuse. the dir
// is not created here - Save does that lazily so a diff-less run touches nothing.
func DefaultDir() (string, error) {
configDir, err := os.UserConfigDir()
if err != nil {
return "", fmt.Errorf("resolving user config dir: %w", err)
}
return filepath.Join(configDir, defaultDirName, stateSubDir), nil
}
// sanitize turns an arbitrary target (https://example.com:8443/path?q=1) into a
// single safe filename component. a target is attacker-influenced (it can come
// from a stdin pipe or a -f file), so every separator and path metacharacter is
// folded to '_' - no '/', '\\', '.', ':' survives to escape the state dir or
// collide with a parent reference. empty/degenerate input falls back to a fixed
// token rather than producing a dotfile or empty name.
func sanitize(target string) string {
var b strings.Builder
b.Grow(len(target))
// collapse runs of separators: a scheme like "https://" is three metachars
// in a row, and one '_' reads cleaner than three without losing uniqueness.
prevSep := false
for i := 0; i < len(target); i++ {
c := target[i]
switch {
case c >= 'a' && c <= 'z', c >= 'A' && c <= 'Z', c >= '0' && c <= '9', c == '-':
b.WriteByte(c)
prevSep = false
default:
// every other byte (path sep, dot, colon, slash, space, unicode, and a
// literal '_') is a separator; fold it so traversal and dotfiles are
// impossible and a run never balloons the filename.
if !prevSep {
b.WriteByte('_')
prevSep = true
}
}
}
name := strings.Trim(b.String(), "_")
if name == "" {
return "target"
}
return name
}
// pathFor builds the absolute snapshot path for a target under dir. kept private
// so the sanitized-filename invariant lives in one place; Save and Load both go
// through it so a target always maps to the same file.
func pathFor(dir, target string) string {
return filepath.Join(dir, sanitize(target)+snapshotExt)
}
// Save writes the run's findings for target as a json snapshot under dir,
// overwriting any prior snapshot. the dir (and parents) is created lazily with
// stateDirMode. an empty findings slice is still written - it records "this
// target had nothing", which a later diff reads as a clean baseline rather than
// a missing one.
func Save(dir, target string, findings []finding.Finding) error {
if dir == "" {
return fmt.Errorf("store: empty snapshot dir")
}
if err := os.MkdirAll(dir, stateDirMode); err != nil {
return fmt.Errorf("creating state dir %q: %w", dir, err)
}
// marshal a non-nil slice so an empty run serializes to [] not null; keeps
// the on-disk shape stable and Load's decode unambiguous.
if findings == nil {
findings = []finding.Finding{}
}
data, err := json.MarshalIndent(findings, "", " ")
if err != nil {
return fmt.Errorf("marshaling snapshot for %q: %w", target, err)
}
path := pathFor(dir, target)
if err := os.WriteFile(path, data, snapshotFileMode); err != nil {
return fmt.Errorf("writing snapshot %q: %w", path, err)
}
return nil
}
// Load reads the previously saved snapshot for target under dir. a missing
// snapshot is not an error - it's the first run for that target, so an empty
// slice comes back and the caller treats every current finding as new. a present
// but unreadable/corrupt file is a real error: silently swallowing it would make
// a broken store look like a fresh one and flag everything as added forever.
func Load(dir, target string) ([]finding.Finding, error) {
path := pathFor(dir, target)
data, err := os.ReadFile(path)
if err != nil {
if os.IsNotExist(err) {
return []finding.Finding{}, nil
}
return nil, fmt.Errorf("reading snapshot %q: %w", path, err)
}
var findings []finding.Finding
if err := json.Unmarshal(data, &findings); err != nil {
return nil, fmt.Errorf("decoding snapshot %q: %w", path, err)
}
if findings == nil {
findings = []finding.Finding{}
}
return findings, nil
}
// Diff computes the set-difference between two snapshots keyed on Finding.Key:
// added is everything in next whose Key isn't in old, removed is everything in
// old whose Key isn't in next. order follows the input slices (added in next's
// order, removed in old's) so output is deterministic for a given pair. a Key
// seen twice in one slice is deduped on first sight, so duplicate findings don't
// double-report.
func Diff(old, next []finding.Finding) (added, removed []finding.Finding) {
oldKeys := make(map[string]struct{}, len(old))
for i := 0; i < len(old); i++ {
oldKeys[old[i].Key] = struct{}{}
}
nextKeys := make(map[string]struct{}, len(next))
for i := 0; i < len(next); i++ {
nextKeys[next[i].Key] = struct{}{}
}
seen := make(map[string]struct{}, len(next))
for i := 0; i < len(next); i++ {
k := next[i].Key
if _, ok := oldKeys[k]; ok {
continue
}
if _, dup := seen[k]; dup {
continue
}
seen[k] = struct{}{}
added = append(added, next[i])
}
// reuse seen for the removed pass; the two key spaces don't overlap by
// construction (removed keys are absent from next) so a single map is safe.
clear(seen)
for i := 0; i < len(old); i++ {
k := old[i].Key
if _, ok := nextKeys[k]; ok {
continue
}
if _, dup := seen[k]; dup {
continue
}
seen[k] = struct{}{}
removed = append(removed, old[i])
}
return added, removed
}
+234
View File
@@ -0,0 +1,234 @@
/*
··
: :
: · Blazing-fast pentesting suite :
: · BSD 3-Clause License :
: :
: (c) 2022-2026 vmfunc, xyzeva, :
: lunchcat alumni & contributors :
: :
··
*/
package store
import (
"os"
"path/filepath"
"reflect"
"sort"
"testing"
"github.com/dropalldatabases/sif/internal/finding"
)
// sampleFindings is a small, stable set of findings reused across the round-trip
// and diff cases; covers two modules and two severities so marshaling exercises
// every Finding field.
func sampleFindings() []finding.Finding {
return []finding.Finding{
{
Target: "https://example.com",
Module: "headers",
Severity: finding.SeverityInfo,
Key: "headers:Server",
Title: "Server",
Raw: "nginx",
},
{
Target: "https://example.com",
Module: "cors",
Severity: finding.SeverityMedium,
Key: "cors:https://example.com:null",
Title: "null origin reflected",
Raw: "allow-origin: null",
},
}
}
func TestSaveLoadRoundTrip(t *testing.T) {
dir := t.TempDir()
const target = "https://example.com"
want := sampleFindings()
if err := Save(dir, target, want); err != nil {
t.Fatalf("Save: %v", err)
}
got, err := Load(dir, target)
if err != nil {
t.Fatalf("Load: %v", err)
}
if !reflect.DeepEqual(got, want) {
t.Fatalf("round-trip mismatch:\n got=%#v\nwant=%#v", got, want)
}
}
func TestSaveCreatesNestedDir(t *testing.T) {
// the state dir need not exist; Save mkdir's it (and parents) lazily.
dir := filepath.Join(t.TempDir(), "nested", "state")
if err := Save(dir, "https://x.test", sampleFindings()); err != nil {
t.Fatalf("Save into missing dir: %v", err)
}
info, err := os.Stat(dir)
if err != nil {
t.Fatalf("stat created dir: %v", err)
}
if !info.IsDir() {
t.Fatalf("expected %q to be a directory", dir)
}
}
func TestSaveEmptyDirRejected(t *testing.T) {
if err := Save("", "https://x.test", sampleFindings()); err == nil {
t.Fatal("Save with empty dir: want error, got nil")
}
}
func TestSaveEmptyFindingsRoundTrips(t *testing.T) {
// an empty run is a valid baseline: Save writes [], Load reads back an empty
// (non-nil) slice, never an error.
dir := t.TempDir()
const target = "https://empty.test"
if err := Save(dir, target, nil); err != nil {
t.Fatalf("Save nil findings: %v", err)
}
got, err := Load(dir, target)
if err != nil {
t.Fatalf("Load: %v", err)
}
if got == nil {
t.Fatal("Load returned nil, want non-nil empty slice")
}
if len(got) != 0 {
t.Fatalf("Load returned %d findings, want 0", len(got))
}
}
func TestLoadMissingSnapshotIsEmpty(t *testing.T) {
// no prior run for this target: a missing file is not an error, it's an empty
// baseline so the first run treats everything as added.
dir := t.TempDir()
got, err := Load(dir, "https://never-scanned.test")
if err != nil {
t.Fatalf("Load missing snapshot: %v", err)
}
if got == nil {
t.Fatal("Load returned nil, want non-nil empty slice")
}
if len(got) != 0 {
t.Fatalf("Load missing snapshot returned %d findings, want 0", len(got))
}
}
func TestLoadCorruptSnapshotErrors(t *testing.T) {
// a present-but-garbage snapshot must surface loudly: treating it as empty
// would silently re-flag every finding as new on every run.
dir := t.TempDir()
const target = "https://corrupt.test"
path := filepath.Join(dir, sanitize(target)+snapshotExt)
if err := os.WriteFile(path, []byte("{not json"), snapshotFileMode); err != nil {
t.Fatalf("seeding corrupt snapshot: %v", err)
}
if _, err := Load(dir, target); err == nil {
t.Fatal("Load corrupt snapshot: want error, got nil")
}
}
func TestDiffAddedAndRemoved(t *testing.T) {
base := sampleFindings()
// next drops the cors finding (removed) and adds a takeover (added); the
// headers finding is unchanged and must appear in neither delta.
next := []finding.Finding{
base[0], // headers - unchanged
{
Target: "https://example.com",
Module: "subdomain_takeover",
Severity: finding.SeverityHigh,
Key: "subdomain_takeover:old.example.com",
Title: "takeover: old.example.com",
Raw: "GitHub Pages",
},
}
added, removed := Diff(base, next)
if len(added) != 1 || added[0].Key != "subdomain_takeover:old.example.com" {
t.Fatalf("added = %#v, want the takeover only", added)
}
if len(removed) != 1 || removed[0].Key != "cors:https://example.com:null" {
t.Fatalf("removed = %#v, want the cors finding only", removed)
}
}
func TestDiffNoChange(t *testing.T) {
// identical snapshots produce no delta in either direction.
base := sampleFindings()
added, removed := Diff(base, base)
if len(added) != 0 || len(removed) != 0 {
t.Fatalf("identical snapshots: added=%d removed=%d, want 0/0", len(added), len(removed))
}
}
func TestDiffFirstRunAllAdded(t *testing.T) {
// no prior snapshot (empty old) means every current finding is new.
next := sampleFindings()
added, removed := Diff(nil, next)
if len(removed) != 0 {
t.Fatalf("first run removed=%d, want 0", len(removed))
}
gotKeys := keysOf(added)
wantKeys := keysOf(next)
if !reflect.DeepEqual(gotKeys, wantKeys) {
t.Fatalf("first run added keys=%v, want %v", gotKeys, wantKeys)
}
}
func TestDiffDedupesRepeatedKey(t *testing.T) {
// a Key appearing twice in the new snapshot is reported once, not twice.
f := sampleFindings()[0]
next := []finding.Finding{f, f}
added, _ := Diff(nil, next)
if len(added) != 1 {
t.Fatalf("duplicate key reported %d times, want 1", len(added))
}
}
// keysOf returns the sorted Key set of a finding slice for order-independent
// comparison.
func keysOf(fs []finding.Finding) []string {
out := make([]string, 0, len(fs))
for i := 0; i < len(fs); i++ {
out = append(out, fs[i].Key)
}
sort.Strings(out)
return out
}
func TestSanitizeNoTraversal(t *testing.T) {
// sanitize is the only barrier between an attacker-influenced target and the
// state dir; assert no separator or traversal token survives.
tests := []struct {
in string
want string
}{
{"https://example.com", "https_example_com"},
{"../../etc/passwd", "etc_passwd"},
{"a/b/c", "a_b_c"},
{"....//....//x", "x"},
{"", "target"},
{"///", "target"},
{"host:8443/path?q=1", "host_8443_path_q_1"},
}
for _, tt := range tests {
got := sanitize(tt.in)
if got != tt.want {
t.Errorf("sanitize(%q) = %q, want %q", tt.in, got, tt.want)
}
if filepath.Base(got) != got {
t.Errorf("sanitize(%q) = %q escapes its component", tt.in, got)
}
}
}
+121 -4
View File
@@ -1,5 +1,5 @@
.\" man page for sif - the blazing-fast pentesting suite
.TH sif 1 "2026-06-08" "sif" "sif manual"
.TH sif 1 "2026-06-10" "sif" "sif manual"
.SH NAME
sif \- blazing-fast pentesting suite
.SH SYNOPSIS
@@ -15,17 +15,25 @@ sif \- blazing-fast pentesting suite
.RI [ scans ]
.RI [ options ]
.br
.I "targets"
|
.B sif
.RI [ scans ]
.RI [ options ]
.br
.B sif
.RB { patchnote | version }
.SH DESCRIPTION
.B sif
is a modular recon and exploitation suite. it runs multiple scan types
concurrently against one or more targets, and can be extended with yaml
modules. targets must include a
modules. a scheme\-less target defaults to
.B https://
\&; an explicit
.B http://
or
.B https://
scheme.
is kept; any other scheme is rejected.
.SH TARGETS
.TP
.BR \-u ", " \-\-urls " \fIlist\fR"
@@ -33,11 +41,42 @@ comma\-separated list of urls to scan.
.TP
.BR \-f ", " \-\-file " \fIpath\fR"
file with one url per line.
.TP
.B stdin
when stdin is a pipe, one target per line is read from it, alongside any
.B \-u
/
.B \-f
targets. lets sif slot into a unix pipeline (e.g. \fBsubfinder | sif \-silent | notify\fR).
.SH SCANS
.TP
.BR \-dirlist " \fIsize\fR"
directory and file fuzzing (small/medium/large).
.TP
.BR \-mc " \fIcodes\fR"
dirlist: match only these status codes (comma list, e.g. 200,301).
.TP
.BR \-fc " \fIcodes\fR"
dirlist: filter out these status codes (comma list).
.TP
.BR \-fs " \fIsizes\fR"
dirlist: filter out responses of these body sizes (comma list).
.TP
.BR \-fw " \fIcounts\fR"
dirlist: filter out responses with these word counts (comma list).
.TP
.BR \-fr " \fIregex\fR"
dirlist: filter out responses whose body matches this regex.
.TP
.B \-ac
dirlist: auto\-calibrate the soft\-404 wildcard baseline so catch\-all 200s are dropped.
.TP
.BR \-w " \fIpath|url\fR"
dirlist: custom wordlist (local file or url); overrides the \fB\-dirlist\fR size.
.TP
.BR \-e " \fIexts\fR"
dirlist: extensions appended to each word (comma list, e.g. php,bak,env).
.TP
.BR \-dnslist " \fIsize\fR"
subdomain enumeration (small/medium/large).
.TP
@@ -51,7 +90,7 @@ vulnerability scanning with nuclei templates.
automated google dorking.
.TP
.B \-js
javascript analysis.
javascript analysis + secret and endpoint extraction.
.TP
.B \-c3
cloud storage misconfiguration scan.
@@ -86,9 +125,39 @@ sql reconnaissance (admin panels, error disclosure).
.B \-lfi
local file inclusion reconnaissance.
.TP
.B \-jwt
jwt discovery plus offline weakness analysis (alg:none, weak hmac secret, missing/expired exp, sensitive plaintext claims).
.TP
.B \-openapi
openapi/swagger spec exposure probe; enumerates paths, methods and unauthenticated operations.
.TP
.B \-favicon
favicon hash fingerprinting (shodan\-style mmh3); matches bundled tech and prints the http.favicon.hash pivot query.
.TP
.B \-cors
cors misconfiguration probe (reflected/permissive origins).
.TP
.B \-redirect
open redirect probe.
.TP
.B \-xss
reflected xss probe.
.TP
.B \-framework
framework detection with cve lookup.
.TP
.B \-crawl
web crawler; spiders same\-host links, scripts and forms, respecting robots.txt.
.TP
.BR \-crawl\-depth " \fIn\fR"
max crawl recursion depth (default 2).
.TP
.B \-passive
passive subdomain and historical url discovery from third\-party feeds (zero traffic to the target).
.TP
.B \-probe
live\-host probe; reports liveness, final status, page title, server header and the redirect chain.
.TP
.B \-noscan
skip the base url scan (robots.txt, etc).
.SH OPTIONS
@@ -120,6 +189,38 @@ cookie header to send with every request.
.BR \-rate\-limit " \fIn\fR"
cap outbound requests per second (0 = unlimited, default 0).
.TP
.BR \-sarif " \fIfile\fR"
write a sarif 2.1.0 report of the run to \fIfile\fR.
.TP
.BR \-md ", " \-\-markdown " \fIfile\fR"
write a markdown report of the run to \fIfile\fR.
.TP
.B \-silent
plain output for pipelines: route all chrome to stderr and print one
normalized finding per line to stdout as \fB[severity] target module title\fR.
implies non\-interactive (no spinners).
.TP
.B \-diff
diff mode: snapshot each target's findings to a json file and, on a re\-scan,
print only the delta against the last snapshot (\fB+ new\fR for findings that
appeared, \fB- gone\fR for ones that vanished), then overwrite the snapshot.
the first run for a target reports everything as new.
.TP
.BR \-store " \fIdir\fR"
snapshot directory for \fB\-diff\fR. defaults to the \fB\-log\fR dir if set,
otherwise \fI<user\-config>/sif/state\fR. one sanitized file per target.
.B \-notify
ship findings to every configured provider (slack, discord, telegram, generic
webhook) after the scan. providers are configured env\-first and overridable by a
yaml file; with nothing configured this is a silent no\-op.
.TP
.BR \-notify\-severity " \fIlevel\fR"
minimum severity to send: \fBinfo\fR, \fBlow\fR, \fBmedium\fR, \fBhigh\fR or
\fBcritical\fR (default \fBmedium\fR). findings below the floor are dropped.
.TP
.BR \-notify\-config " \fIfile\fR"
path to a notify\-compatible yaml config whose values override the env vars.
.TP
.B \-api
emit json results and suppress the interactive output.
.SH MODULES
@@ -151,6 +252,22 @@ api key used by \fB\-shodan\fR.
.B SECURITYTRAILS_API_KEY
api key used by \fB\-securitytrails\fR.
.TP
.B SLACK_WEBHOOK_URL
slack incoming webhook used by \fB\-notify\fR (yaml key \fBslack_webhook_url\fR).
.TP
.B DISCORD_WEBHOOK_URL
discord webhook used by \fB\-notify\fR (yaml key \fBdiscord_webhook_url\fR).
.TP
.B TELEGRAM_BOT_TOKEN
telegram bot token used by \fB\-notify\fR (yaml key \fBtelegram_api_key\fR);
requires \fBTELEGRAM_CHAT_ID\fR too.
.TP
.B TELEGRAM_CHAT_ID
telegram destination chat used by \fB\-notify\fR (yaml key \fBtelegram_chat_id\fR).
.TP
.B NOTIFY_WEBHOOK_URL
generic json webhook used by \fB\-notify\fR (yaml key \fBwebhook_url\fR).
.TP
.B SIF_NO_PATCHNOTES
set to any value to suppress the once\-per\-version patch note shown at startup.
.SH FILES
+157
View File
@@ -0,0 +1,157 @@
/*
··
: :
: · Blazing-fast pentesting suite :
: · BSD 3-Clause License :
: :
: (c) 2022-2026 vmfunc, xyzeva, :
: lunchcat alumni & contributors :
: :
··
*/
package sif
import (
"context"
"encoding/json"
"io"
"net/http"
"net/http/httptest"
"testing"
"time"
"github.com/dropalldatabases/sif/internal/config"
"github.com/dropalldatabases/sif/internal/finding"
)
// notifyWebhookBody is the generic-webhook wire shape, mirrored here so the
// wiring test can assert which findings crossed the severity floor without
// reaching into the notify package internals.
type notifyWebhookBody struct {
Count int `json:"count"`
Findings []struct {
Severity string `json:"severity"`
Key string `json:"key"`
} `json:"findings"`
}
// mixedSeverityFindings spans the whole ladder so a floor test has something to
// drop on either side of every threshold.
func mixedSeverityFindings() []finding.Finding {
return []finding.Finding{
{Target: "https://t.test", Module: "headers", Severity: finding.SeverityInfo, Key: "headers:s", Title: "server"},
{Target: "https://t.test", Module: "redirect", Severity: finding.SeverityLow, Key: "redirect:r", Title: "open redirect"},
{Target: "https://t.test", Module: "sql", Severity: finding.SeverityMedium, Key: "sql:e", Title: "db error"},
{Target: "https://t.test", Module: "cors", Severity: finding.SeverityHigh, Key: "cors:c", Title: "reflected origin"},
{Target: "https://t.test", Module: "lfi", Severity: finding.SeverityCritical, Key: "lfi:l", Title: "path traversal"},
}
}
func TestNotifyFindingsSeverityFilter(t *testing.T) {
tests := []struct {
name string
floor string
wantKeys []string
}{
{name: "medium drops info+low", floor: "medium", wantKeys: []string{"sql:e", "cors:c", "lfi:l"}},
{name: "high drops everything below", floor: "high", wantKeys: []string{"cors:c", "lfi:l"}},
{name: "info keeps all", floor: "info", wantKeys: []string{"headers:s", "redirect:r", "sql:e", "cors:c", "lfi:l"}},
{name: "critical keeps only critical", floor: "critical", wantKeys: []string{"lfi:l"}},
// an unrecognized floor must default to medium, not let info through.
{name: "garbage floor defaults medium", floor: "bogus", wantKeys: []string{"sql:e", "cors:c", "lfi:l"}},
}
for _, tt := range tests {
t.Run(tt.name, func(t *testing.T) {
var got notifyWebhookBody
srv := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
body, _ := io.ReadAll(r.Body)
if err := json.Unmarshal(body, &got); err != nil {
t.Errorf("unmarshal webhook body: %v", err)
}
w.WriteHeader(http.StatusOK)
}))
t.Cleanup(srv.Close)
// route notify at the test server via the generic webhook env var.
t.Setenv("SLACK_WEBHOOK_URL", "")
t.Setenv("DISCORD_WEBHOOK_URL", "")
t.Setenv("TELEGRAM_BOT_TOKEN", "")
t.Setenv("TELEGRAM_CHAT_ID", "")
t.Setenv("NOTIFY_WEBHOOK_URL", srv.URL)
app := &App{settings: &config.Settings{
Notify: true,
NotifySeverity: tt.floor,
Timeout: time.Second,
}}
if err := app.notifyFindings(context.Background(), mixedSeverityFindings()); err != nil {
t.Fatalf("notifyFindings: %v", err)
}
gotKeys := make([]string, 0, len(got.Findings))
for _, f := range got.Findings {
gotKeys = append(gotKeys, f.Key)
}
if !equalStringSets(gotKeys, tt.wantKeys) {
t.Errorf("floor %q delivered keys %v, want %v", tt.floor, gotKeys, tt.wantKeys)
}
if got.Count != len(tt.wantKeys) {
t.Errorf("floor %q count = %d, want %d", tt.floor, got.Count, len(tt.wantKeys))
}
})
}
}
func TestNotifyFindingsBelowFloorIsNoop(t *testing.T) {
// every finding below the floor -> nothing crosses -> no POST at all.
hit := false
srv := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, _ *http.Request) {
hit = true
w.WriteHeader(http.StatusOK)
}))
t.Cleanup(srv.Close)
t.Setenv("SLACK_WEBHOOK_URL", "")
t.Setenv("DISCORD_WEBHOOK_URL", "")
t.Setenv("TELEGRAM_BOT_TOKEN", "")
t.Setenv("TELEGRAM_CHAT_ID", "")
t.Setenv("NOTIFY_WEBHOOK_URL", srv.URL)
app := &App{settings: &config.Settings{
Notify: true,
NotifySeverity: "critical",
Timeout: time.Second,
}}
infoOnly := []finding.Finding{
{Target: "https://t.test", Module: "headers", Severity: finding.SeverityInfo, Key: "headers:s", Title: "server"},
}
if err := app.notifyFindings(context.Background(), infoOnly); err != nil {
t.Fatalf("notifyFindings: %v", err)
}
if hit {
t.Fatal("notifyFindings posted with everything below floor, want no-op")
}
}
// equalStringSets reports whether a and b contain the same elements regardless
// of order; the wire order mirrors input order, but order isn't the contract.
func equalStringSets(a, b []string) bool {
if len(a) != len(b) {
return false
}
seen := make(map[string]int, len(a))
for _, s := range a {
seen[s]++
}
for _, s := range b {
seen[s]--
}
for _, n := range seen {
if n != 0 {
return false
}
}
return true
}
+430 -21
View File
@@ -20,19 +20,25 @@ import (
"context"
"encoding/json"
"fmt"
"io"
"os"
"strings"
"github.com/charmbracelet/log"
"github.com/dropalldatabases/sif/internal/config"
"github.com/dropalldatabases/sif/internal/dnsx"
"github.com/dropalldatabases/sif/internal/finding"
"github.com/dropalldatabases/sif/internal/httpx"
"github.com/dropalldatabases/sif/internal/logger"
"github.com/dropalldatabases/sif/internal/modules"
"github.com/dropalldatabases/sif/internal/notify"
"github.com/dropalldatabases/sif/internal/output"
"github.com/dropalldatabases/sif/internal/report"
"github.com/dropalldatabases/sif/internal/scan"
"github.com/dropalldatabases/sif/internal/scan/builtin"
"github.com/dropalldatabases/sif/internal/scan/frameworks"
jsscan "github.com/dropalldatabases/sif/internal/scan/js"
"github.com/dropalldatabases/sif/internal/store"
)
// App represents the main application structure for sif.
@@ -46,6 +52,10 @@ type App struct {
// Version is set by main to the resolved build version and shown on the banner.
var Version = "dev"
// reportFileMode is the permission applied to written report files: owner
// read/write, group/other read. reports aren't secret but may name targets.
const reportFileMode = 0o644
type UrlResult struct {
Url string `json:"url"`
Results []ModuleResult
@@ -78,13 +88,19 @@ func NewModuleResult[T ScanResult](data T) ModuleResult {
func New(settings *config.Settings) (*App, error) {
app := &App{settings: settings}
// -silent reroutes all chrome to stderr (and suppresses spinners) before the
// banner prints, so stdout carries nothing but findings even on the banner.
if settings.Silent {
output.SetSilent(true)
}
if !settings.ApiMode {
fmt.Println(output.Box.Render(" █▀ █ █▀▀\n ▄█ █ █▀ "))
fmt.Fprintln(output.Writer(), output.Box.Render(" █▀ █ █▀▀\n ▄█ █ █▀ "))
tagline := "blazing-fast pentesting suite"
if Version != "dev" {
tagline += " · v" + Version
}
fmt.Println(output.Subheading.Render("\n" + tagline + "\n\nbsd 3-clause · (c) 2022-2026 vmfunc, xyzeva & contributors\n"))
fmt.Fprintln(output.Writer(), output.Subheading.Render("\n"+tagline+"\n\nbsd 3-clause · (c) 2022-2026 vmfunc, xyzeva & contributors\n"))
} else {
output.SetAPIMode(true)
}
@@ -94,10 +110,11 @@ func New(settings *config.Settings) (*App, error) {
return app, nil
}
switch {
case len(settings.URLs) > 0:
app.targets = settings.URLs
case settings.File != "":
// -u and -f are explicit; stdin is additive so `subfinder | sif -u extra`
// still works. order: flags first, then piped lines appended.
app.targets = append(app.targets, settings.URLs...)
if settings.File != "" {
if _, err := os.Stat(settings.File); err != nil {
return nil, err
}
@@ -113,29 +130,105 @@ func New(settings *config.Settings) (*App, error) {
for scanner.Scan() {
app.targets = append(app.targets, scanner.Text())
}
default:
return nil, fmt.Errorf("target(s) must be supplied with -u or -f\n\nSee 'sif -h' for more information")
}
// Validate all URLs early
for _, url := range app.targets {
if err := validateURL(url); err != nil {
// when stdin is a pipe (not a terminal), drain it for targets so sif slots
// into a unix pipeline: `subfinder -d x | sif -silent | notify`. keyed off
// stdin's mode, never stdout - a redirected stdout (>file) is not a pipe in.
piped, err := stdinPipedFn()
if err != nil {
return nil, err
}
if piped {
stdinTargets, err := readTargets(stdinReader)
if err != nil {
return nil, fmt.Errorf("reading targets from stdin: %w", err)
}
app.targets = append(app.targets, stdinTargets...)
}
if len(app.targets) == 0 {
return nil, fmt.Errorf("target(s) must be supplied with -u, -f, or stdin\n\nSee 'sif -h' for more information")
}
// normalize every target in place: a naked host gains a default scheme, an
// explicit scheme is kept, genuinely invalid input is rejected early.
for i := 0; i < len(app.targets); i++ {
normalized, err := normalizeTarget(app.targets[i])
if err != nil {
return nil, err
}
app.targets[i] = normalized
}
return app, nil
}
// validateURL checks that a URL has a valid HTTP/HTTPS protocol.
func validateURL(url string) error {
if url == "" {
return fmt.Errorf("empty URL provided")
// defaultScheme is prepended to scheme-less targets. https is the safer default
// for recon: it's what modern hosts serve and avoids a cleartext first hop.
const defaultScheme = "https://"
// stdin ingestion is wired through two seams so it's hermetically testable: the
// pipe check and the reader can be swapped in tests without touching real fds.
var (
stdinPipedFn = stdinPiped
stdinReader io.Reader = os.Stdin
)
// stdinPiped reports whether stdin is a pipe/redirect rather than a terminal.
// a char device (the tty) means interactive with no piped input; anything else
// (pipe, file redirect) is treated as a target stream.
func stdinPiped() (bool, error) {
info, err := os.Stdin.Stat()
if err != nil {
return false, fmt.Errorf("stat stdin: %w", err)
}
if !strings.HasPrefix(url, "http://") && !strings.HasPrefix(url, "https://") {
return fmt.Errorf("URL %s must include http:// or https:// protocol", url)
return info.Mode()&os.ModeCharDevice == 0, nil
}
// readTargets scans one target per line from r, dropping blank lines and
// trimming surrounding whitespace. shared by the stdin path; the file path keeps
// its own scanner since it preserves lines verbatim for back-compat.
func readTargets(r io.Reader) ([]string, error) {
var out []string
scanner := bufio.NewScanner(r)
scanner.Split(bufio.ScanLines)
for scanner.Scan() {
line := strings.TrimSpace(scanner.Text())
if line == "" {
continue
}
out = append(out, line)
}
return nil
if err := scanner.Err(); err != nil {
return nil, fmt.Errorf("scanning targets: %w", err)
}
return out, nil
}
// normalizeTarget canonicalizes a single target. a scheme-less host gets the
// default scheme; an http:// or https:// target is kept as-is. an empty string
// or a non-http(s) scheme (ftp://, file://, ...) is rejected so junk can't slip
// into the scan loop.
func normalizeTarget(target string) (string, error) {
target = strings.TrimSpace(target)
if target == "" {
return "", fmt.Errorf("empty target provided")
}
if strings.HasPrefix(target, "http://") || strings.HasPrefix(target, "https://") {
return target, nil
}
// reject anything that carries some other scheme; "://" present but not
// http(s) is a deliberate non-web target, not a naked host.
if strings.Contains(target, "://") {
return "", fmt.Errorf("target %s must use http:// or https:// scheme", target)
}
// a bare "host:port" or path-only token would also be ambiguous; require at
// least a host-looking first segment (no spaces) before defaulting a scheme.
if strings.ContainsAny(target, " \t") {
return "", fmt.Errorf("invalid target %q", target)
}
return defaultScheme + target, nil
}
// Run runs the pentesting suite, with the targets specified, according to the
@@ -189,6 +282,7 @@ func (app *App) Run() error {
Headers: app.settings.Header,
Cookie: app.settings.Cookie,
RateLimit: app.settings.RateLimit,
Threads: app.settings.Threads,
}); err != nil {
log.Warnf("http client config failed, continuing with defaults: %v", err)
}
@@ -204,6 +298,29 @@ func (app *App) Run() error {
scansRun := make([]string, 0, 16)
// accumulate every module result across targets so the report writers can
// serialize the full run after the loop. only collected when an export flag
// is set, so the common path pays nothing.
wantReport := app.settings.SARIF != "" || app.settings.Markdown != ""
reportResults := make([]report.Result, 0, 16)
// normalized findings for the whole run; the single Flatten-driven view that
// notify and diff consume. collected alongside the report so both describe the
// same scanners from one pass.
allFindings := make([]finding.Finding, 0, 16)
// resolve the snapshot dir once when diff mode is on; a bad default isn't
// fatal - diff just no-ops for the run rather than killing the scan.
storeDir := ""
if app.settings.Diff {
dir, err := app.resolveStoreDir()
if err != nil {
log.Warnf("diff disabled: %v", err)
} else {
storeDir = dir
}
}
for _, url := range app.targets {
output.Info("Starting scan on %s", output.Highlight.Render(url))
@@ -231,11 +348,20 @@ func (app *App) Run() error {
}
if app.settings.Dirlist != "none" {
result, err := scan.Dirlist(app.settings.Dirlist, url, app.settings.Timeout, app.settings.Threads, app.settings.LogDir)
result, err := scan.Dirlist(app.settings.Dirlist, url, app.settings.Timeout, app.settings.Threads, app.settings.LogDir, scan.DirlistOptions{
MatchCodes: app.settings.DirMatchCodes,
FilterCodes: app.settings.DirFilterCodes,
FilterSizes: app.settings.DirFilterSizes,
FilterWords: app.settings.DirFilterWords,
FilterRegex: app.settings.DirFilterRegex,
Calibrate: app.settings.DirCalibrate,
Wordlist: app.settings.DirWordlist,
Extensions: app.settings.DirExtensions,
})
if err != nil {
log.Errorf("Error while running directory scan: %s", err)
} else {
moduleResults = append(moduleResults, ModuleResult{"dirlist", result})
moduleResults = append(moduleResults, NewModuleResult(result))
scansRun = append(scansRun, "Directory Listing")
}
}
@@ -243,7 +369,7 @@ func (app *App) Run() error {
var dnsResults []string
if app.settings.Dnslist != "none" {
result, err := scan.Dnslist(app.settings.Dnslist, url, app.settings.Timeout, app.settings.Threads, app.settings.LogDir)
result, err := scan.Dnslist(app.settings.Dnslist, url, app.settings.Timeout, app.settings.Threads, app.settings.LogDir, dnsx.ParseResolvers(app.settings.Resolvers))
if err != nil {
log.Errorf("Error while running dns scan: %s", err)
} else {
@@ -391,6 +517,96 @@ func (app *App) Run() error {
}
}
if app.settings.JWT {
result, err := scan.JWT(url, app.settings.Timeout, app.settings.LogDir)
if err != nil {
log.Errorf("Error while running JWT analysis: %s", err)
} else if result != nil {
moduleResults = append(moduleResults, NewModuleResult(result))
scansRun = append(scansRun, "JWT")
}
}
if app.settings.OpenAPI {
result, err := scan.OpenAPI(url, app.settings.Timeout, app.settings.Threads, app.settings.LogDir)
if err != nil {
log.Errorf("Error while running OpenAPI probe: %s", err)
} else if result != nil {
moduleResults = append(moduleResults, NewModuleResult(result))
scansRun = append(scansRun, "OpenAPI")
}
}
if app.settings.Favicon {
result, err := scan.Favicon(url, app.settings.Timeout, app.settings.LogDir)
if err != nil {
log.Errorf("Error while running favicon fingerprint: %s", err)
} else if result != nil {
moduleResults = append(moduleResults, NewModuleResult(result))
scansRun = append(scansRun, "Favicon")
}
}
if app.settings.CORS {
result, err := scan.CORS(url, app.settings.Timeout, app.settings.Threads, app.settings.LogDir)
if err != nil {
log.Errorf("Error while running CORS probe: %s", err)
} else if result != nil {
moduleResults = append(moduleResults, NewModuleResult(result))
scansRun = append(scansRun, "CORS")
}
}
if app.settings.Redirect {
result, err := scan.Redirect(url, app.settings.Timeout, app.settings.Threads, app.settings.LogDir)
if err != nil {
log.Errorf("Error while running open redirect probe: %s", err)
} else if result != nil {
moduleResults = append(moduleResults, NewModuleResult(result))
scansRun = append(scansRun, "Open Redirect")
}
}
if app.settings.XSS {
result, err := scan.XSS(url, app.settings.Timeout, app.settings.Threads, app.settings.LogDir)
if err != nil {
log.Errorf("Error while running reflected XSS probe: %s", err)
} else if result != nil {
moduleResults = append(moduleResults, NewModuleResult(result))
scansRun = append(scansRun, "Reflected XSS")
}
}
if app.settings.Crawl {
result, err := scan.Crawl(url, app.settings.CrawlDepth, app.settings.Timeout, app.settings.LogDir)
if err != nil {
log.Errorf("Error while running web crawl: %s", err)
} else if result != nil {
moduleResults = append(moduleResults, NewModuleResult(result))
scansRun = append(scansRun, "Crawl")
}
}
if app.settings.Passive {
result, err := scan.Passive(url, app.settings.Timeout, app.settings.LogDir)
if err != nil {
log.Errorf("Error while running passive discovery: %s", err)
} else if result != nil {
moduleResults = append(moduleResults, NewModuleResult(result))
scansRun = append(scansRun, "Passive")
}
}
if app.settings.Probe {
result, err := scan.Probe(url, app.settings.Timeout, app.settings.LogDir)
if err != nil {
log.Errorf("Error while running probe: %s", err)
} else if result != nil {
moduleResults = append(moduleResults, NewModuleResult(result))
scansRun = append(scansRun, "Probe")
}
}
// Load and run modules
if app.settings.AllModules || app.settings.Modules != "" || app.settings.ModuleTags != "" {
loader, err := modules.NewLoader()
@@ -461,6 +677,48 @@ func (app *App) Run() error {
}
fmt.Println(string(marshalled))
}
targetFindings := collectFindings(url, moduleResults)
allFindings = append(allFindings, targetFindings...)
// diff mode is per-target: load this target's last snapshot, surface only
// the delta, then overwrite the snapshot so the next run diffs against now.
// storeDir is "" when diff is off or the dir couldn't resolve, in which
// case this is a no-op and behavior is unchanged.
if storeDir != "" {
app.diffTarget(storeDir, url, targetFindings)
}
// the report carries raw blobs and is only built when an export flag is
// set, so the common path skips the marshalling entirely.
if wantReport {
reportResults = append(reportResults, collectReportResults(url, moduleResults)...)
}
}
// the normalized findings are the handoff point for notify/diff; surface the
// count now so the path is live and observable without changing output.
log.Debugf("normalized %d findings across %d targets", len(allFindings), len(app.targets))
// notify: ship the severity-filtered findings to any configured provider.
// kept as an isolated block so it merges cleanly with the diff-store bundle.
if app.settings.Notify {
if err := app.notifyFindings(context.Background(), allFindings); err != nil {
log.Errorf("notify: %v", err)
}
}
// -silent: stdout is the findings stream, one terse line each. all chrome
// already went to stderr via the rerouted sink, so this is the only thing a
// downstream pipe sees.
if app.settings.Silent {
printFindings(allFindings)
}
if wantReport {
if err := app.writeReports(reportResults); err != nil {
return err
}
}
if !app.settings.ApiMode {
@@ -470,6 +728,157 @@ func (app *App) Run() error {
return nil
}
// printFindings writes one normalized finding per line to stdout for the
// -silent plain sink. a single Builder over the run avoids interleaving with
// any stray stderr chrome and keeps the write to one syscall.
func printFindings(findings []finding.Finding) {
var b strings.Builder
for i := 0; i < len(findings); i++ {
b.WriteString(findings[i].Line())
b.WriteByte('\n')
}
fmt.Print(b.String())
}
// collectFindings normalizes one target's module results through finding.Flatten
// - the single normalization path that notify and diff build on. every scan
// result struct collapses to flat, severity-ranked findings here so a scanner is
// described once, not once per consumer.
func collectFindings(target string, moduleResults []ModuleResult) []finding.Finding {
out := make([]finding.Finding, 0, len(moduleResults))
for _, mr := range moduleResults {
out = append(out, finding.Flatten(target, mr.Id, mr.Data)...)
}
return out
}
// resolveStoreDir picks the snapshot directory for diff mode. precedence: an
// explicit -store wins; else the run's log dir is reused (snapshots live next to
// logs); else the per-user default under <user-config>/sif/state. returns an
// error only when no usable location exists, so the caller can disable diff
// without failing the scan.
func (app *App) resolveStoreDir() (string, error) {
if app.settings.Store != "" {
return app.settings.Store, nil
}
if app.settings.LogDir != "" {
return app.settings.LogDir, nil
}
dir, err := store.DefaultDir()
if err != nil {
return "", fmt.Errorf("resolving snapshot dir: %w", err)
}
return dir, nil
}
// diffTarget loads target's previous snapshot, prints the added/removed delta
// against the current findings, then overwrites the snapshot so the next run
// diffs against this one. a load failure surfaces but doesn't abort the run -
// the new snapshot is still written so a corrupt baseline self-heals. always
// saves, even when the delta is empty, to advance the baseline.
func (app *App) diffTarget(dir, target string, current []finding.Finding) {
previous, err := store.Load(dir, target)
if err != nil {
log.Warnf("diff: reading snapshot for %s, treating as fresh: %v", target, err)
previous = nil
}
added, removed := store.Diff(previous, current)
printDiff(target, added, removed)
if err := store.Save(dir, target, current); err != nil {
log.Warnf("diff: saving snapshot for %s: %v", target, err)
}
}
// printDiff renders a target's diff: each added finding marked "+ new", each
// removed one "- gone", with a one-line note when nothing changed. routed
// through the shared output sink so -silent keeps it on stderr alongside the
// other chrome. a single Builder keeps the block from interleaving.
func printDiff(target string, added, removed []finding.Finding) {
if len(added) == 0 && len(removed) == 0 {
output.Info("diff %s: no changes since last snapshot", target)
return
}
var b strings.Builder
fmt.Fprintf(&b, "diff %s: %d new, %d gone\n", target, len(added), len(removed))
for i := 0; i < len(added); i++ {
fmt.Fprintf(&b, " + new %s\n", added[i].Line())
}
for i := 0; i < len(removed); i++ {
fmt.Fprintf(&b, " - gone %s\n", removed[i].Line())
}
fmt.Fprint(output.Writer(), b.String())
}
// collectReportResults flattens one target's module results into the report
// model, carrying each finding as raw json so the report package stays free of
// scan types. a result that won't marshal is skipped rather than failing the run.
func collectReportResults(target string, moduleResults []ModuleResult) []report.Result {
out := make([]report.Result, 0, len(moduleResults))
for _, mr := range moduleResults {
data, err := json.Marshal(mr.Data)
if err != nil {
log.Warnf("report: skipping %s result for %s: %v", mr.Id, target, err)
continue
}
out = append(out, report.Result{Target: target, Module: mr.Id, Data: data})
}
return out
}
// writeReports serializes the collected results to the requested export files.
// each writer runs independently so a bad path for one format doesn't suppress
// the other.
func (app *App) writeReports(results []report.Result) error {
if path := app.settings.SARIF; path != "" {
data, err := report.SARIF(results)
if err != nil {
return fmt.Errorf("build sarif report: %w", err)
}
if err := os.WriteFile(path, data, reportFileMode); err != nil {
return fmt.Errorf("write sarif report %q: %w", path, err)
}
output.Success("sarif report written to %s", path)
}
if path := app.settings.Markdown; path != "" {
data := report.Markdown(results)
if err := os.WriteFile(path, data, reportFileMode); err != nil {
return fmt.Errorf("write markdown report %q: %w", path, err)
}
output.Success("markdown report written to %s", path)
}
return nil
}
// notifyFindings filters the run's findings to the -notify-severity floor and
// ships the survivors to every configured provider. an unrecognized severity
// string parses to SeverityUnknown, which would let everything through; guard
// against that by defaulting to medium so a typo can't flood a channel with
// info noise. an empty filtered set makes notify.Send a no-op.
func (app *App) notifyFindings(ctx context.Context, findings []finding.Finding) error {
floor := finding.ParseSeverity(app.settings.NotifySeverity)
if floor == finding.SeverityUnknown {
log.Warnf("notify: unknown severity %q, defaulting to medium", app.settings.NotifySeverity)
floor = finding.SeverityMedium
}
filtered := make([]finding.Finding, 0, len(findings))
for i := 0; i < len(findings); i++ {
if findings[i].Severity.AtLeast(floor) {
filtered = append(filtered, findings[i])
}
}
return notify.Send(ctx, filtered, notify.Options{
Timeout: app.settings.Timeout,
ConfigPath: app.settings.NotifyConfig,
})
}
// expandTargets queries SecurityTrails for each original target and returns
// newly discovered domains (subdomains + associated) for target expansion
func (app *App) expandTargets() []string {
+273 -6
View File
@@ -13,11 +13,25 @@
package sif
import (
"io"
"os"
"strings"
"testing"
"github.com/dropalldatabases/sif/internal/config"
"github.com/dropalldatabases/sif/internal/finding"
"github.com/dropalldatabases/sif/internal/store"
)
// TestMain neutralizes the stdin seam for the whole package so tests that build
// an App via New() never block on the test runner's real stdin (a pipe under
// `go test`). tests that exercise ingestion set the seams explicitly.
func TestMain(m *testing.M) {
stdinPipedFn = func() (bool, error) { return false, nil }
stdinReader = strings.NewReader("")
os.Exit(m.Run())
}
// mockResult is a test implementation of ScanResult
type mockResult struct {
name string
@@ -117,20 +131,16 @@ func TestNew_URLValidation(t *testing.T) {
wantErr: false,
},
{
// naked host is now accepted and normalized, not rejected
name: "missing protocol",
url: "example.com",
wantErr: true,
wantErr: false,
},
{
name: "invalid protocol",
url: "ftp://example.com",
wantErr: true,
},
{
name: "empty url",
url: "",
wantErr: true,
},
}
for _, tt := range tests {
@@ -148,6 +158,194 @@ func TestNew_URLValidation(t *testing.T) {
}
}
func TestNormalizeTarget(t *testing.T) {
tests := []struct {
name string
in string
want string
wantErr bool
}{
{name: "naked host defaults https", in: "example.com", want: "https://example.com"},
{name: "naked host with port", in: "example.com:8443", want: "https://example.com:8443"},
{name: "naked host with path", in: "example.com/admin", want: "https://example.com/admin"},
{name: "https kept", in: "https://example.com", want: "https://example.com"},
{name: "http kept", in: "http://example.com", want: "http://example.com"},
{name: "surrounding whitespace trimmed", in: " example.com\t", want: "https://example.com"},
{name: "empty rejected", in: "", wantErr: true},
{name: "blank rejected", in: " ", wantErr: true},
{name: "ftp scheme rejected", in: "ftp://example.com", wantErr: true},
{name: "file scheme rejected", in: "file:///etc/passwd", wantErr: true},
{name: "embedded space rejected", in: "foo bar", wantErr: true},
}
for _, tt := range tests {
t.Run(tt.name, func(t *testing.T) {
got, err := normalizeTarget(tt.in)
if (err != nil) != tt.wantErr {
t.Fatalf("normalizeTarget(%q) err = %v, wantErr %v", tt.in, err, tt.wantErr)
}
if err != nil {
return
}
if got != tt.want {
t.Errorf("normalizeTarget(%q) = %q, want %q", tt.in, got, tt.want)
}
})
}
}
func TestNew_StdinIngestion(t *testing.T) {
// feed a pipe of targets and assert they're parsed and normalized alongside
// the explicit -u target. the seams stand in for a real stdin pipe.
origPiped, origReader := stdinPipedFn, stdinReader
t.Cleanup(func() { stdinPipedFn, stdinReader = origPiped, origReader })
stdinPipedFn = func() (bool, error) { return true, nil }
stdinReader = strings.NewReader("sub1.example.com\nhttps://sub2.example.com\n\n sub3.example.com \n")
settings := &config.Settings{
URLs: []string{"https://flag.example.com"},
ApiMode: true,
}
app, err := New(settings)
if err != nil {
t.Fatalf("New() unexpected error: %v", err)
}
want := []string{
"https://flag.example.com",
"https://sub1.example.com",
"https://sub2.example.com",
"https://sub3.example.com",
}
if len(app.targets) != len(want) {
t.Fatalf("targets = %v (%d), want %d", app.targets, len(app.targets), len(want))
}
for i := range want {
if app.targets[i] != want[i] {
t.Errorf("target[%d] = %q, want %q", i, app.targets[i], want[i])
}
}
}
func TestNew_StdinOnly(t *testing.T) {
// no -u/-f: a piped stream alone must satisfy the target requirement.
origPiped, origReader := stdinPipedFn, stdinReader
t.Cleanup(func() { stdinPipedFn, stdinReader = origPiped, origReader })
stdinPipedFn = func() (bool, error) { return true, nil }
stdinReader = strings.NewReader("only.example.com\n")
app, err := New(&config.Settings{ApiMode: true})
if err != nil {
t.Fatalf("New() unexpected error: %v", err)
}
if len(app.targets) != 1 || app.targets[0] != "https://only.example.com" {
t.Errorf("targets = %v, want [https://only.example.com]", app.targets)
}
}
func TestNew_NoTargets_StdinEmpty(t *testing.T) {
// an empty pipe with no flags is still "no targets" and must error.
origPiped, origReader := stdinPipedFn, stdinReader
t.Cleanup(func() { stdinPipedFn, stdinReader = origPiped, origReader })
stdinPipedFn = func() (bool, error) { return true, nil }
stdinReader = strings.NewReader("\n \n")
if _, err := New(&config.Settings{ApiMode: true}); err == nil {
t.Error("New() should error when stdin yields no targets and no flags set")
}
}
func TestReadTargets(t *testing.T) {
got, err := readTargets(strings.NewReader("a.com\n\n b.com \nc.com\n"))
if err != nil {
t.Fatalf("readTargets() error: %v", err)
}
want := []string{"a.com", "b.com", "c.com"}
if len(got) != len(want) {
t.Fatalf("readTargets() = %v, want %v", got, want)
}
for i := range want {
if got[i] != want[i] {
t.Errorf("readTargets()[%d] = %q, want %q", i, got[i], want[i])
}
}
}
// errReader fails on first read; used to assert stdin scan errors propagate.
type errReader struct{}
func (errReader) Read([]byte) (int, error) { return 0, io.ErrClosedPipe }
func TestReadTargets_Error(t *testing.T) {
if _, err := readTargets(errReader{}); err == nil {
t.Error("readTargets() should propagate a reader error")
}
}
func TestPrintFindings(t *testing.T) {
findings := []finding.Finding{
{Target: "https://a.com", Module: "sql", Severity: finding.SeverityHigh, Title: "admin panel"},
{Target: "https://b.com", Module: "headers", Severity: finding.SeverityInfo, Title: "Server"},
}
out := captureStdout(t, func() { printFindings(findings) })
wantLines := []string{
"[high] https://a.com sql admin panel",
"[info] https://b.com headers Server",
}
got := strings.Split(strings.TrimRight(out, "\n"), "\n")
if len(got) != len(wantLines) {
t.Fatalf("printFindings wrote %d lines, want %d:\n%s", len(got), len(wantLines), out)
}
for i := range wantLines {
if got[i] != wantLines[i] {
t.Errorf("line %d = %q, want %q", i, got[i], wantLines[i])
}
}
}
func TestPrintFindings_Empty(t *testing.T) {
out := captureStdout(t, func() { printFindings(nil) })
if out != "" {
t.Errorf("printFindings(nil) wrote %q, want empty", out)
}
}
// captureStdout swaps os.Stdout for a pipe, runs fn, and returns what it wrote.
func captureStdout(t *testing.T, fn func()) string {
t.Helper()
r, w, err := os.Pipe()
if err != nil {
t.Fatalf("pipe: %v", err)
}
saved := os.Stdout
os.Stdout = w
done := make(chan string, 1)
go func() {
buf := make([]byte, 0, 4096)
tmp := make([]byte, 1024)
for {
n, rerr := r.Read(tmp)
buf = append(buf, tmp[:n]...)
if rerr != nil {
break
}
}
done <- string(buf)
}()
fn()
os.Stdout = saved
w.Close()
return <-done
}
func TestModuleResult_JSON(t *testing.T) {
mr := ModuleResult{
Id: "test",
@@ -176,3 +374,72 @@ func TestUrlResult_JSON(t *testing.T) {
t.Errorf("UrlResult.Results = %d, want 1", len(ur.Results))
}
}
func TestResolveStoreDir(t *testing.T) {
// explicit -store wins over everything.
explicit := &App{settings: &config.Settings{Store: "/tmp/snaps", LogDir: "/tmp/logs"}}
if dir, err := explicit.resolveStoreDir(); err != nil || dir != "/tmp/snaps" {
t.Fatalf("explicit store: got (%q, %v), want (/tmp/snaps, nil)", dir, err)
}
// no -store: reuse the log dir.
logged := &App{settings: &config.Settings{LogDir: "/tmp/logs"}}
if dir, err := logged.resolveStoreDir(); err != nil || dir != "/tmp/logs" {
t.Fatalf("log dir fallback: got (%q, %v), want (/tmp/logs, nil)", dir, err)
}
// neither set: fall through to the per-user default (non-empty, no error).
bare := &App{settings: &config.Settings{}}
dir, err := bare.resolveStoreDir()
if err != nil {
t.Fatalf("default store dir: %v", err)
}
if dir == "" {
t.Fatal("default store dir resolved empty")
}
}
func TestDiffTargetSnapshotsAndDiffs(t *testing.T) {
dir := t.TempDir()
const target = "https://diff.example.com"
app := &App{settings: &config.Settings{Diff: true, Store: dir}}
first := []finding.Finding{
{Target: target, Module: "headers", Severity: finding.SeverityInfo, Key: "headers:Server", Title: "Server", Raw: "nginx"},
}
// first run: no prior snapshot, everything is new; the snapshot must persist.
app.diffTarget(dir, target, first)
saved, err := store.Load(dir, target)
if err != nil {
t.Fatalf("load after first run: %v", err)
}
if len(saved) != 1 || saved[0].Key != "headers:Server" {
t.Fatalf("snapshot after first run = %#v, want the headers finding", saved)
}
// second run with a different set: the snapshot must advance to the new set so
// a third run would diff against it.
second := []finding.Finding{
{Target: target, Module: "cors", Severity: finding.SeverityMedium, Key: "cors:x", Title: "null origin", Raw: "null"},
}
app.diffTarget(dir, target, second)
saved, err = store.Load(dir, target)
if err != nil {
t.Fatalf("load after second run: %v", err)
}
if len(saved) != 1 || saved[0].Key != "cors:x" {
t.Fatalf("snapshot after second run = %#v, want the cors finding", saved)
}
// the delta between the two snapshots is exactly: headers gone, cors new.
added, removed := store.Diff(first, second)
if len(added) != 1 || added[0].Key != "cors:x" {
t.Fatalf("added = %#v, want cors:x", added)
}
if len(removed) != 1 || removed[0].Key != "headers:Server" {
t.Fatalf("removed = %#v, want headers:Server", removed)
}
}