* fix(scan): tune js secret rules for false positives and missed formats
drop stripe pk_ publishable keys (public by design) and require a digit in
the generic secret value so camelCase identifiers stop tripping the entropy
gate. add stripe rk_ restricted keys, github fine-grained pat (github_pat_),
encrypted pem headers and version-anchored slack xapp tokens; drop the
trailing word boundary on the aws-secret and google rules so keys ending in
/ + or - still match.
renames the "stripe live key" rule to "stripe secret key", which changes
the rule label in the json findings output.
* feat(scan): detect more provider keys in javascript
add unique-prefix credential rules to the js secret bank: gitlab pat
(glpat-), anthropic api/admin keys (sk-ant-), npm tokens (npm_), google
oauth client secrets (GOCSPX-), stripe webhook secrets (whsec_), shopify
app tokens (shp[at|ss|pa|ca]_), sendgrid keys (SG.) and slack incoming
webhook urls. all ride the no-entropy slot since the prefix alone is
proof, so they carry near-zero false-positive risk.
* fix(logger): create log dirs and flatten url log filenames
CreateFile/Write kept the '/' from a target's url path when building the
log filename, so a target like http://host:port/ tried to open
<dir>/host:port/.log whose parent directory never existed. os.OpenFile
failed and scanTarget aborted the whole target before any scanning ran.
fold every '/' and '\\' run in the sanitized url into a single '_' via a
shared logPath helper so CreateFile and Write always resolve to the same
flat file, and mkdir the parent defensively in getWriter as a second line
of defense.
* fix(scan): route module status lines through the locking sink
subdomaintakeover, cloudstorage, and the next.js framework detector
printed their status/error lines with a bare fmt.Println straight to
os.Stdout, bypassing the output package's sink. under -concurrency>1
only the sink is lock-wrapped, so these lines could interleave or tear
against lines other targets write concurrently.
swap them for output.ScanStart / output.Error, matching every other
scanner in the package.
* fix(store): make snapshot filenames injective and writes atomic
sanitize() folds every separator run (and a literal '_') to one '_', so
distinct targets like https://a.com/x and https://a.com//x, or
host:8443/path and host_8443_path, sanitized to the identical string and
shared one snapshot file - the second target's Save silently clobbered
the first's baseline. Save's os.WriteFile also wasn't atomic, so two
targets racing on a collided path (or -concurrency>1 in general) could
interleave partial writes.
pathFor now appends 16 hex chars of the target's sha256 to the sanitized
name, so distinct targets never collide, and Save writes through a temp
file in the same dir followed by os.Rename so a reader always sees a
complete snapshot or the previous one, never a partial write.
this changes the on-disk snapshot filename scheme: existing users' -diff
baselines under the old sanitize()-only names won't be found and the
next run re-baselines from scratch (every finding reports as newly
added once). noting this as a deliberate tradeoff for a local hardening
pass rather than silently orphaning them.
* fix(store): check the deferred temp-file cleanup error
golangci-lint (errcheck) flagged the bare defer os.Remove(tmpPath) in
writeFileAtomic; wrap it so the discard is explicit.
* fix(js): stop the scanning spinner on early script-parse errors
htmlquery.Parse and QueryAll failures returned before spin.Stop(), leaving
the "Scanning JavaScript files" spinner running forever on those paths while
every other error branch in the function stops it first.
* fix(js): stop reporting quoted regex-flag literals as endpoints
Quoted regex flag combos like "/gi" or "/su" in ordinary JS (e.g.
str.replace("/gi", x)) matched the root-relative path alternative in
endpointRegex and were reported as endpoints. Denylist the known flag
bigrams by exact match instead of raising minEndpointLen, so real short
endpoints like /v1, /me, /ws still extract normally.
* fix(js): keep supabase findings from other projects on a per-project error
ScanSupabase iterates every JWT found in a script and can discover several
distinct supabase projects. Four error paths inside that loop (reading the
signup response, parsing it, fetching the openapi spec, and a missing
Paths field) returned nil and the whole error, discarding any results
already collected for earlier, successfully scanned projects. Skip the
broken project with continue instead so the scan still returns what it
found.
* test(js): drop flaky wall-clock redos timing probe
go's regexp is re2 and linear by construction; the timing threshold added
only flake under -race load, not coverage. the regex-flag-literal false
positive it sat next to is guarded by TestRegexFlagEndpointsNotReported.
GetPagesRouterScripts fetched _buildManifest.js with httpx.Client(0),
which sets no client timeout, so a slow or hostile manifest host could
hang the whole scan on the response read. thread the caller's scan
timeout through instead, matching every other fetch in the package.
Ran `gofmt -w .` accross the repo to fix formatting drift.
Mechanical `gofmt -w .` only. No functional or behavioural changes.
CONTRIBUTING.md requires gofmt-clean code; these files had slipped.
rename the go module path from github.com/dropalldatabases/sif to
github.com/vmfunc/sif across go.mod, all imports, the golangci exclude
list, release install docs and docs. pure string rename, no logic change.
the page scanner and the next.js manifest parser both reassembled the
response body line by line with a default bufio.Scanner (64k token
cap), so a line past the cap (common with minified or inlined js)
silently halted the read and dropped every script and route reference
after it. read the whole body instead, capped at 5mb.
parsing the raw page bytes also fixes script tags split across a
newline, which the old line-joined reassembly merged and missed.
go only returns a conn to the idle pool when the body is read to EOF
before Close. the header-only and early-return scan paths closed an
unread body, leaking the conn and forcing a fresh dial each request.
route those close sites through httpx.DrainClose so the tuned pool from
phase 1 actually gets reused. body-read paths (scanner/io.ReadAll) are
left untouched.
scan results live in ~two dozen structs with no shared shape, so every
consumer that wants "what did this run turn up" reimplements the
type-switch. add internal/finding: an ordered Severity (info<low<medium<
high<critical, with parse/compare) and Flatten, the single type-switch
that collapses every scan result struct into flat, severity-ranked
Findings keyed module:identifier for stable dedup/diff.
wire collectFindings off Flatten in the run loop so notify and diff
(later bundles) build on one normalization path instead of re-deriving
it; the report path keeps emitting raw json blobs unchanged. expose
JavascriptScanResult.SupabaseFindings so the js internals stay private.
the guard test iterates a representative instance of every ResultType
and fails if Flatten lacks a case (falls through to :unhandled) - so a
new scanner can't ship without a Flatten case landing too.
the -js pipeline already pulls every <script> into a buffer but only
mined supabase jwts from it. reuse that buffer to run a credential
regex bank (aws/github/slack/stripe/google keys, pem blocks, plus
entropy-gated generic apikey/secret/token assignments) and a
linkfinder-style endpoint extractor that resolves relatives to
absolute urls. both dedupe across scripts and surface through the
existing js logger and result struct, no new flag.
every scanner spun up its own &http.Client, so there was no single place
to apply a proxy, custom headers, a cookie or a rate limit. add an
internal/httpx package that builds one configured transport at startup and
hand it to every scanner via httpx.Client(timeout), keeping behavior
identical when nothing is set (plain client when Configure was never
called).
- httpx.Configure wires -proxy (http/https/socks5), -H/--header, -cookie
and -rate-limit into a package-level RoundTripper that paces via a
rate.Limiter and only sets headers the caller hasn't already, so a
scanner's explicit api key still wins.
- route the scan/wordlist downloads that used http.DefaultClient through
the shared client too; ports tcp dialing is left untouched.
- clamp -threads to a floor of 1: it feeds wg.Add across the scanners, so
0 was a silent no-op and a negative value panicked the waitgroup.
document the new flags in the readme, usage docs and man page.
doSupabaseRequest and the signup call used a bare http.Client{} with no
timeout, so a slow supabase project could hang the whole js scan. thread the
scan's --timeout down through ScanSupabase into every supabase request.
strips the emoji from the subdomaintakeover/cloudstorage/supabase log
prefixes and trims comments that just restate the code (the
parameters/returns block on SubdomainTakeover, a couple of "this provides
type safety" notes) so the scanners read like whois.go/headers.go.
rolls the (c) 2022-2025 banner to 2022-2026 across all go files, the
startup banner in sif.go, and the header-check workflow's expected
format. comment-only, nothing else changes.
- noctx: use http.NewRequestWithContext instead of http.Get/client.Get
- bodyclose: close response bodies on all code paths
- httpNoBody: use http.NoBody instead of nil for GET request bodies
- ifElseChain: convert if/else chains to switch in sif.go
- sloppyReassign: use := in logger.go where possible
- nilnil: annotate intentional nil,nil returns in lfi.go, sql.go
- errcheck: handle template install error in nuclei.go
- govet copylock: pass mutex by pointer in executor.go
- log.Fatalf: replace with log.Errorf+continue in api mode
- Add output package with colored prefixes and module loggers
- Each module gets unique background color based on name hash
- Add spinner for indeterminate operations
- Add progress bar for known-count operations
- Update all scan files to use ModuleLogger pattern
- Add clean PrintSummary for scan completion
