* feat(config): add -config/-profile config file and profile overlays
extend the existing goflags yaml config mechanism (the same one -template
already uses) with an explicit -config path and named -profile overlays,
instead of adding a second config system.
resolveConfigInput unifies -config/-profile/-template into a single flat
yaml path for goflags to merge before Parse: unset, it returns "" and
goflags falls back to its ambient ~/.config/sif/config.yaml unchanged.
a profile overlays its keys onto the file's top-level keys in a go map
before handing goflags a flat temp file, so precedence stays explicit cli
flag > profile > file default > built-in default via goflags' own
DefValue sentinel merge. -config and -template share one config slot and
are mutually exclusive.
verifies empirically that goflags auto-creates and merges the ambient
config file with no explicit SetConfigFilePath call, which this feature
depends on.
* docs(config): document -config/-profile and drop the dead toml example
template-example.toml was unreferenced by any go file and did not match
how goflags actually reads config (flat long-name keys, yaml). replace it
with config-example.yaml showing the real schema, including a profiles
block, and document -config/-profile in the usage/configuration guides
and the man page.
* fix(config): validate malformed config on the no-profile path
resolveConfigInput used to return an explicit -config path unparsed when
-profile was not set, so a malformed yaml file skipped validation entirely.
goflags then silently discarded the decode error, dropping every real
setting in the file with no diagnostic and exit 0, while the same file with
-profile already errored cleanly through loadConfigMap.
route both branches through one buildFlatConfig that always loads via
loadConfigMap and only overlays a profile when one is selected, so a
malformed file errors the same way regardless of -profile.
* fix(config): let explicit cli flags override config and profile values
goflags' own merge (readConfigFile) treats a flag whose current value equals
its DefValue as "unset" and applies the config value over it. that makes an
explicit cli flag silently lose to the config file or a profile whenever the
user happens to pass the flag's own default, e.g. "-timeout 10s" against the
built-in 10s default: today the file's 1s wins even though the flag was set
explicitly on the command line. the same class of bug hits -threads,
-concurrency, -notify-severity, and any profile value on the same merge path.
scan the raw args for every flag the user actually passed (long or short
alias, space or "=" form) and strip those keys out of the resolved config/
profile map before it ever reaches goflags, so cli precedence holds
unconditionally instead of only when the value differs from the default.
flagAliasGroups derives the long/short alias groups from the real flag
registration (grouping by the shared flag.Value pointer) rather than a
second hardcoded name table, so it can't drift from registerFlags.
goflags also swallows a type-mismatched config value entirely (discards
fl.Value.Set's error); that is a separate, lower-severity gap in the
vendored dependency itself and is left alone, noted in a comment.
ship findings to chat/webhook sinks after a scan so continuous recon can
alert on what it turns up. each provider is one POST through httpx.Client,
so the global proxy/rate-limit/header config applies and there's no extra
http stack. config resolves env-first (SLACK_WEBHOOK_URL, DISCORD_WEBHOOK_URL,
TELEGRAM_BOT_TOKEN/TELEGRAM_CHAT_ID, NOTIFY_WEBHOOK_URL), overridable by a
notify-compatible yaml file so existing projectdiscovery/notify configs port
over. -notify enables it, -notify-severity gates on the finding severity
ladder (default medium), -notify-config points at the yaml. wired after the
scan loop on the severity-filtered finding set; no provider configured is a
silent no-op.
re-scans become a monitor: -diff snapshots each target's normalized
findings to a per-target json file and, on the next run, surfaces only
the delta (+ new / - gone) against the last snapshot, then overwrites it
so each run diffs against the previous one. behavior is unchanged when
-diff is off.
new internal/store keys the set-difference off finding.Key (already
stable across runs) and uses only encoding/json + os - no new deps.
snapshot files are sanitized per target (no traversal), written 0600
under 0750 dirs. -store picks the location: explicit dir, else the log
dir, else <user-config>/sif/state. a missing snapshot is a clean
baseline, a corrupt one self-heals on the next save.
jwt fetches the target once then analyzes every harvested token offline:
flags alg:none, the rs256->hs256 confusion surface, missing/expired exp
and plaintext sensitive claims, and cracks a small bundled weak-hmac list.
openapi probes the conventional spec paths, parses json/yaml and enumerates
paths plus unauthenticated operations. favicon computes the shodan-style
mmh3 hash (python base64.encodebytes chunking, signed int32) for tech
fingerprinting and the http.favicon.hash pivot, pinned by a golden test.
sif can now slot into unix pipelines. stdin is drained for targets when
it's a pipe (keyed off stdin's mode, not stdout), alongside -u/-f. naked
hosts are accepted and default to https://; explicit http(s) is kept,
other schemes rejected. -silent routes all banner/spinner/log chrome to
stderr and prints one normalized finding per line to stdout via
finding.Flatten, so `subfinder | sif -silent | notify` works.
adds an httpx-style -probe scanner reporting liveness, final status, page
title, server header and the redirect chain, plus -sarif/-markdown export
flags that serialize the collected run after the scan loop. the report
serializers live in a decoupled internal/report package consuming a raw-json
result model so they never import scan types.
the old scanner surfaced every response that wasn't 404/403, so modern SPA
catch-all 200s flooded the output and made -dirlist near-useless. add ffuf-style
matching:
- -mc/-fc/-fr and -fs/-fw filter by status, regex, body size and word count;
bodies are read through a capped io.LimitReader so size/word counts are
deterministic and memory stays flat. filters win over matches.
- -ac auto-calibrates the soft-404 baseline from a few deterministic
non-existent paths and drops responses matching that wildcard shape.
- -w overrides the size switch with a local file or remote list (fetched through
the shared client so proxy/rate-limit apply); -e appends extensions per word.
size and words are added to DirectoryResult for the json output.
-crawl spiders same-host links/scripts/forms through the shared httpx
client so proxy/headers/rate-limit and robots.txt are honored, bounded
by -crawl-depth. -passive pulls subdomains from keyless ct feeds (crt.sh,
certspotter) and historical urls from wayback, each source isolated so
one feed being down doesn't sink the rest and the target sees no traffic.
three active web-vuln probes wired into the per-target loop:
- cors: crafts attacker origins (evil sentinel, null, prefix/suffix
bypass, http downgrade) and flags responses that reflect them in
access-control-allow-origin, ranking reflection+credentials high.
- redirect: injects a controlled sentinel host plus bypass variants
(//, https:/, backslash, null-byte, userinfo @) into redirect-prone
params and catches 30x location, meta-refresh and js redirects that
resolve off-site.
- xss: injects a unique canary wrapped in breaking chars, classifies
the reflection context (html/attribute/script) and reports only the
chars that survive unescaped where they matter, so escaped
reflections don't false-positive.
all route through httpx.Client so proxy/-H/-cookie/-rate-limit apply.
hermetic httptest coverage plus integration testbed entries.
the -js pipeline already pulls every <script> into a buffer but only
mined supabase jwts from it. reuse that buffer to run a credential
regex bank (aws/github/slack/stripe/google keys, pem blocks, plus
entropy-gated generic apikey/secret/token assignments) and a
linkfinder-style endpoint extractor that resolves relatives to
absolute urls. both dedupe across scripts and surface through the
existing js logger and result struct, no new flag.
every scanner spun up its own &http.Client, so there was no single place
to apply a proxy, custom headers, a cookie or a rate limit. add an
internal/httpx package that builds one configured transport at startup and
hand it to every scanner via httpx.Client(timeout), keeping behavior
identical when nothing is set (plain client when Configure was never
called).
- httpx.Configure wires -proxy (http/https/socks5), -H/--header, -cookie
and -rate-limit into a package-level RoundTripper that paces via a
rate.Limiter and only sets headers the caller hasn't already, so a
scanner's explicit api key still wins.
- route the scan/wordlist downloads that used http.DefaultClient through
the shared client too; ports tcp dialing is left untouched.
- clamp -threads to a floor of 1: it feeds wg.Add across the scanners, so
0 was a silent no-op and a negative value panicked the waitgroup.
document the new flags in the readme, usage docs and man page.
adds man/sif.1 covering the targets/scans/options/modules flags, the
patchnote and version commands, env vars (incl SIF_NO_PATCHNOTES), and a
few examples. the install/uninstall make targets now drop it in
$(PREFIX)/share/man/man1.
