framework detection collapsed to a single argmax winner, so a page built
from several technologies (react behind a next.js server, say) reported
only one and dropped the rest. add DetectFrameworks, which keeps every
detector hit above the threshold ranked most-confident first, and wire
the -framework path to emit one finding per detected framework.
the single-winner DetectFramework stays for callers that want just the
top match; both share one fetch through the new gatherDetections helper,
so the extra coverage costs no additional request.
Co-authored-by: vmfunc <vmfunc.lc@gmail.com>
Add a -tls-cert scan that actively dials the target's tls port and mines the
leaf certificate: SAN entries become candidate subdomains, and issuer/validity
feed posture flags (self-signed, expired, expiring soon, wildcard). Unlike the
passive crt.sh and certspotter feeds this probes the target directly, so it
surfaces certs never logged to a public ct log at the cost of touching the
target. Self-signed detection compares raw issuer/subject der and verifies the
cert against its own key rather than CheckSignatureFrom, which false-negatives
on leaf certs that omit CA key-usage bits.
Co-authored-by: vmfunc <vmfunc.lc@gmail.com>
module execution built a bare http client, so -proxy, -H, -cookie and
-rate-limit applied to every other scanner but silently not to module
scans. route through httpx.Client so modules share the configured
transport like the rest of the run.
Co-authored-by: vmfunc <vmfunc.lc@gmail.com>
sarif and markdown were the only structured exports and both drop the
per-item detail the normalized finding model already holds. add a -json
flag that writes the run's findings as a json array, one object per
finding with its target, module, severity, key, title and evidence.
carry detection confidence onto the finding model too so framework
detections report their score. it rides into the json report and is
omitted for scanners that carry none. the -silent line format and the
snapshot on-disk shape stay unchanged, since the report uses a dedicated
json view rather than marshalling the finding struct directly.
sarif results were hardcoded to level "warning", so a critical finding
and an informational one landed identically in code-scanning ingests.
carry the normalized severity onto report.Result and map it: critical
and high to error, medium to warning, low and info to note. the severity
is derived by flattening each module result and taking the worst rank,
and an absent severity keeps the old "warning" default.
* refactor(cli): extract per-target scan into scanTarget
pull the target loop body into scanTarget, which returns a targetScan
holding that target's findings, report rows, scan labels and log files
instead of appending onto run-wide slices. the run loop merges those in
target order and finishRun handles the post-loop notify/silent/report/
summary steps. behavior is identical for the sequential run today; the
isolation is the seam a bounded worker pool needs next.
* feat(cli): scan targets in parallel with -concurrency
add -concurrency N (default 1) to run the per-target scan pool over
several targets at once. scanTarget already returns isolated
accumulators, so workers share only the console: output.SetConcurrent
serializes sink writes and de-animates spinners/progress so parallel
lines stay whole. results merge in input order, and concurrency 1 keeps
the sequential path unchanged.
the shared transport paced outbound requests but ignored a server
asking it to slow down, so a 429 or 503 came straight back as a failed
probe. back off and retry both through the one chokepoint every scanner
shares, honoring Retry-After (delta-seconds or http-date) and falling
back to capped exponential backoff.
gated behind -max-retries (default 2, 0 disables). bodyless GETs and
GetBody-backed requests replay safely; anything else is returned as-is.
rename the go module path from github.com/dropalldatabases/sif to
github.com/vmfunc/sif across go.mod, all imports, the golangci exclude
list, release install docs and docs. pure string rename, no logic change.
calibrate against reflecting catch-alls whose body size tracks path
length so exact-shape calibration no longer misses them; -ac now drives
both dirlist and sql. updates the admin-panel query test to the new SQL
signature. adds soft-404 + calibration coverage.
four built-in go modules (nuclei-scan, framework-detection, shodan-lookup,
whois-lookup) wrap a legacy per-target scan that also has its own flag, so
running -am alongside one of those flags (e.g. -am -nuclei) ran the same
per-target scan twice. skip the wrapper module in the run loop when its flag
already ran. securitytrails-lookup is not deduped: its -securitytrails flag
expands the target list rather than scanning a target, so the module is not a
duplicate of it.
-mt builds the run list by appending modules.ByTag per tag and -m by appending
each id, neither de-duplicated, so a module matching two requested tags
(nuclei-scan is tagged both vuln and cve, so -mt vuln,cve) or listed twice ran
twice. de-duplicate toRun by module id, preserving first-seen order, before
executing.
ship findings to chat/webhook sinks after a scan so continuous recon can
alert on what it turns up. each provider is one POST through httpx.Client,
so the global proxy/rate-limit/header config applies and there's no extra
http stack. config resolves env-first (SLACK_WEBHOOK_URL, DISCORD_WEBHOOK_URL,
TELEGRAM_BOT_TOKEN/TELEGRAM_CHAT_ID, NOTIFY_WEBHOOK_URL), overridable by a
notify-compatible yaml file so existing projectdiscovery/notify configs port
over. -notify enables it, -notify-severity gates on the finding severity
ladder (default medium), -notify-config points at the yaml. wired after the
scan loop on the severity-filtered finding set; no provider configured is a
silent no-op.
re-scans become a monitor: -diff snapshots each target's normalized
findings to a per-target json file and, on the next run, surfaces only
the delta (+ new / - gone) against the last snapshot, then overwrites it
so each run diffs against the previous one. behavior is unchanged when
-diff is off.
new internal/store keys the set-difference off finding.Key (already
stable across runs) and uses only encoding/json + os - no new deps.
snapshot files are sanitized per target (no traversal), written 0600
under 0750 dirs. -store picks the location: explicit dir, else the log
dir, else <user-config>/sif/state. a missing snapshot is a clean
baseline, a corrupt one self-heals on the next save.
jwt fetches the target once then analyzes every harvested token offline:
flags alg:none, the rs256->hs256 confusion surface, missing/expired exp
and plaintext sensitive claims, and cracks a small bundled weak-hmac list.
openapi probes the conventional spec paths, parses json/yaml and enumerates
paths plus unauthenticated operations. favicon computes the shodan-style
mmh3 hash (python base64.encodebytes chunking, signed int32) for tech
fingerprinting and the http.favicon.hash pivot, pinned by a golden test.
sif can now slot into unix pipelines. stdin is drained for targets when
it's a pipe (keyed off stdin's mode, not stdout), alongside -u/-f. naked
hosts are accepted and default to https://; explicit http(s) is kept,
other schemes rejected. -silent routes all banner/spinner/log chrome to
stderr and prints one normalized finding per line to stdout via
finding.Flatten, so `subfinder | sif -silent | notify` works.
dnslist previously http-probed every wordlist candidate through the
blocking os resolver, so a big list meant a request per dead name and a
wildcard zone flooded results. resolve each candidate first via a new
internal/dnsx (retryabledns over a bundled 1.1.1.1/8.8.8.8/9.9.9.9 pool,
promoted to a direct dep), fingerprint the apex with random nonexistent
labels to detect catch-all zones, and http-probe only the names that
actually resolve and aren't wildcard. add -resolvers to override the
pool. resolverFn is a package-level seam so the dnsx tests stay
hermetic; the dnslist newDNSResolver seam keeps the integration test
network-free.
scan results live in ~two dozen structs with no shared shape, so every
consumer that wants "what did this run turn up" reimplements the
type-switch. add internal/finding: an ordered Severity (info<low<medium<
high<critical, with parse/compare) and Flatten, the single type-switch
that collapses every scan result struct into flat, severity-ranked
Findings keyed module:identifier for stable dedup/diff.
wire collectFindings off Flatten in the run loop so notify and diff
(later bundles) build on one normalization path instead of re-deriving
it; the report path keeps emitting raw json blobs unchanged. expose
JavascriptScanResult.SupabaseFindings so the js internals stay private.
the guard test iterates a representative instance of every ResultType
and fails if Flatten lacks a case (falls through to :unhandled) - so a
new scanner can't ship without a Flatten case landing too.
the shared transport was a bare DefaultTransport.Clone() with the stock
MaxIdleConnsPerHost=2, and call-sites only close response bodies without
draining them - so go could never return a conn to the idle pool and every
request re-dialed. high thread counts just thrashed the dialer.
- plumb Threads through Options into buildTransport; size MaxIdleConnsPerHost
to the worker count (floored) so concurrent workers on one host pool instead
of re-dialing, MaxIdleConns=512, MaxConnsPerHost=0, IdleConnTimeout=90s,
ForceAttemptHTTP2. the socks5 branch gets its own keepalive net.Dialer so it
doesn't lose os-level pooling under proxy.Direct.
- add DrainClose to read (capped) and close a body so the conn is reusable.
- benchmark proves it: 50 sequential requests reuse 1 conn tuned vs 50 bare.
adds an httpx-style -probe scanner reporting liveness, final status, page
title, server header and the redirect chain, plus -sarif/-markdown export
flags that serialize the collected run after the scan loop. the report
serializers live in a decoupled internal/report package consuming a raw-json
result model so they never import scan types.
the old scanner surfaced every response that wasn't 404/403, so modern SPA
catch-all 200s flooded the output and made -dirlist near-useless. add ffuf-style
matching:
- -mc/-fc/-fr and -fs/-fw filter by status, regex, body size and word count;
bodies are read through a capped io.LimitReader so size/word counts are
deterministic and memory stays flat. filters win over matches.
- -ac auto-calibrates the soft-404 baseline from a few deterministic
non-existent paths and drops responses matching that wildcard shape.
- -w overrides the size switch with a local file or remote list (fetched through
the shared client so proxy/rate-limit apply); -e appends extensions per word.
size and words are added to DirectoryResult for the json output.
-crawl spiders same-host links/scripts/forms through the shared httpx
client so proxy/headers/rate-limit and robots.txt are honored, bounded
by -crawl-depth. -passive pulls subdomains from keyless ct feeds (crt.sh,
certspotter) and historical urls from wayback, each source isolated so
one feed being down doesn't sink the rest and the target sees no traffic.
three active web-vuln probes wired into the per-target loop:
- cors: crafts attacker origins (evil sentinel, null, prefix/suffix
bypass, http downgrade) and flags responses that reflect them in
access-control-allow-origin, ranking reflection+credentials high.
- redirect: injects a controlled sentinel host plus bypass variants
(//, https:/, backslash, null-byte, userinfo @) into redirect-prone
params and catches 30x location, meta-refresh and js redirects that
resolve off-site.
- xss: injects a unique canary wrapped in breaking chars, classifies
the reflection context (html/attribute/script) and reports only the
chars that survive unescaped where they matter, so escaped
reflections don't false-positive.
all route through httpx.Client so proxy/-H/-cookie/-rate-limit apply.
hermetic httptest coverage plus integration testbed entries.
every scanner spun up its own &http.Client, so there was no single place
to apply a proxy, custom headers, a cookie or a rate limit. add an
internal/httpx package that builds one configured transport at startup and
hand it to every scanner via httpx.Client(timeout), keeping behavior
identical when nothing is set (plain client when Configure was never
called).
- httpx.Configure wires -proxy (http/https/socks5), -H/--header, -cookie
and -rate-limit into a package-level RoundTripper that paces via a
rate.Limiter and only sets headers the caller hasn't already, so a
scanner's explicit api key still wins.
- route the scan/wordlist downloads that used http.DefaultClient through
the shared client too; ports tcp dialing is left untouched.
- clamp -threads to a floor of 1: it feeds wg.Add across the scanners, so
0 was a silent no-op and a negative value panicked the waitgroup.
document the new flags in the readme, usage docs and man page.
- add internal/version: resolve from the release ldflag, else the go build
info (module tag / vcs revision), else "dev"
- show the version on the boot banner and for `sif version`
- Makefile now stamps `make` builds via git describe (matching the release ci),
so local/go-install builds report a real version instead of "dev"
- patchnotes.ShowOnce skips pseudo/dev versions so non-release builds dont make
a doomed github call
- document sif version / sif patchnote / SIF_NO_PATCHNOTES in the readme + usage
adds a -sh/--security-headers scan that flags missing or weak response
headers (hsts, csp, x-frame-options, x-content-type-options,
referrer-policy, permissions-policy, coop) and headers that leak server
internals (server, x-powered-by, ...). hsts is only graded over https
where it actually applies. wired into App.Run and the module results.
rolls the (c) 2022-2025 banner to 2022-2026 across all go files, the
startup banner in sif.go, and the header-check workflow's expected
format. comment-only, nothing else changes.
- noctx: use http.NewRequestWithContext instead of http.Get/client.Get
- bodyclose: close response bodies on all code paths
- httpNoBody: use http.NoBody instead of nil for GET request bodies
- ifElseChain: convert if/else chains to switch in sif.go
- sloppyReassign: use := in logger.go where possible
- nilnil: annotate intentional nil,nil returns in lfi.go, sql.go
- errcheck: handle template install error in nuclei.go
- govet copylock: pass mutex by pointer in executor.go
- log.Fatalf: replace with log.Errorf+continue in api mode
- Add output package with colored prefixes and module loggers
- Each module gets unique background color based on name hash
- Add spinner for indeterminate operations
- Add progress bar for known-count operations
- Update all scan files to use ModuleLogger pattern
- Add clean PrintSummary for scan completion
Add module system integration allowing users to run YAML-defined security
modules via CLI flags. Implements --list-modules to display available modules,
and supports running modules by ID, tags, or all at once.
- create detector interface and registry for extensibility
- extract detectors to separate files: backend.go, frontend.go, cms.go, meta.go
- reduce detect.go from 785 lines to 178 lines (pure orchestrator)
- export VersionMatch and ExtractVersionOptimized for detector use
- create result.go with NewFrameworkResult and WithVulnerabilities helpers
- add url validation to New() for early error detection
- add sif_test.go with main package tests
- update detect_test.go to use external test package pattern
introduce ScanResult interface and generic NewModuleResult constructor
for compile-time type safety when creating module results.
- add pkg/scan/result.go with ScanResult interface and named slice types
- add typed shodanMetadata struct to replace map[string]interface{}
- refactor supabase.go with typed response structs and json.RawMessage
- add ResultType() methods to all scan result types
- update sif.go to use NewModuleResult generic constructor
this provides type safety without breaking JSON serialization.
replace errors.new(fmt.sprintf()) with fmt.errorf, use strings.builder
instead of string concatenation in loop, fix defer in loop issue,
preallocate slices where size is estimable
replace per-write file open/close with cached file handles and buffered
writers for significantly reduced i/o overhead. adds flush and close
methods for proper cleanup at program exit.
adds a new --lfi flag for local file inclusion vulnerability scanning:
- tests common lfi parameters with directory traversal payloads
- detects /etc/passwd, /etc/shadow, windows system files
- identifies php wrappers and encoded content
- supports various bypass techniques (null bytes, encoding)
closes#4
- remove duplicate SubdomainTakeover call that ran twice when both
dns scan and --st flag were enabled
- add comprehensive tests for config settings defaults and behavior
- fix formatting in dork.go
closes#1
adds a new --shodan flag that queries the shodan api for information
about the target host. requires SHODAN_API_KEY environment variable.
features:
- resolves hostnames to ip addresses
- queries shodan host api for reconnaissance data
- displays organization, isp, location, ports, services, and vulns
- logs results to file when logdir is specified
closes#2
- update badges to point to vmfunc/sif
- replace ascii art with banner image
- fix header check action to check first 5 lines
- remove obsolete LICENSE.md
