-crawl spiders same-host links/scripts/forms through the shared httpx
client so proxy/headers/rate-limit and robots.txt are honored, bounded
by -crawl-depth. -passive pulls subdomains from keyless ct feeds (crt.sh,
certspotter) and historical urls from wayback, each source isolated so
one feed being down doesn't sink the rest and the target sees no traffic.
three active web-vuln probes wired into the per-target loop:
- cors: crafts attacker origins (evil sentinel, null, prefix/suffix
bypass, http downgrade) and flags responses that reflect them in
access-control-allow-origin, ranking reflection+credentials high.
- redirect: injects a controlled sentinel host plus bypass variants
(//, https:/, backslash, null-byte, userinfo @) into redirect-prone
params and catches 30x location, meta-refresh and js redirects that
resolve off-site.
- xss: injects a unique canary wrapped in breaking chars, classifies
the reflection context (html/attribute/script) and reports only the
chars that survive unescaped where they matter, so escaped
reflections don't false-positive.
all route through httpx.Client so proxy/-H/-cookie/-rate-limit apply.
hermetic httptest coverage plus integration testbed entries.
the -js pipeline already pulls every <script> into a buffer but only
mined supabase jwts from it. reuse that buffer to run a credential
regex bank (aws/github/slack/stripe/google keys, pem blocks, plus
entropy-gated generic apikey/secret/token assignments) and a
linkfinder-style endpoint extractor that resolves relatives to
absolute urls. both dedupe across scripts and surface through the
existing js logger and result struct, no new flag.
every scanner spun up its own &http.Client, so there was no single place
to apply a proxy, custom headers, a cookie or a rate limit. add an
internal/httpx package that builds one configured transport at startup and
hand it to every scanner via httpx.Client(timeout), keeping behavior
identical when nothing is set (plain client when Configure was never
called).
- httpx.Configure wires -proxy (http/https/socks5), -H/--header, -cookie
and -rate-limit into a package-level RoundTripper that paces via a
rate.Limiter and only sets headers the caller hasn't already, so a
scanner's explicit api key still wins.
- route the scan/wordlist downloads that used http.DefaultClient through
the shared client too; ports tcp dialing is left untouched.
- clamp -threads to a floor of 1: it feeds wg.Add across the scanners, so
0 was a silent no-op and a negative value panicked the waitgroup.
document the new flags in the readme, usage docs and man page.
- add internal/version: resolve from the release ldflag, else the go build
info (module tag / vcs revision), else "dev"
- show the version on the boot banner and for `sif version`
- Makefile now stamps `make` builds via git describe (matching the release ci),
so local/go-install builds report a real version instead of "dev"
- patchnotes.ShowOnce skips pseudo/dev versions so non-release builds dont make
a doomed github call
- document sif version / sif patchnote / SIF_NO_PATCHNOTES in the readme + usage
- the readme headline used -all, which isn't a real flag (goflags fatals
on unknown flags), so the three -all examples now use actual flags
- document the new -sh security-header scan in the readme table, usage.md
and scans.md, and fix the -headers section (it dumps headers; -sh grades
them)
- bump the documented go version 1.23 -> 1.25 to match go.mod
- add docs/ with installation, usage, modules, scans, and api docs
- add docs link to main readme
- fix release.yml to bundle modules directory with releases
- add module system tests to runtest.yml
- standardize go version to 1.23 across workflows
vmfunc