Commit Graph
150 Commits
Author SHA1 Message Date
TigahandGitHub 570592c317 feat(modules): add airflow, flink and kafka connect exposure modules (#214)
modules/recon/airflow-api-exposure.yaml flags an exposed Apache Airflow webserver
through its unauthenticated health endpoint, keyed on the metadatabase and
scheduler health blocks, then extracts the scheduler heartbeat.

modules/recon/flink-api-exposure.yaml flags an exposed Apache Flink dashboard,
keyed on the flink version paired with the slot total that a generic overview does
not carry, then extracts the flink version.

modules/recon/kafka-connect-api-exposure.yaml flags an exposed Kafka Connect rest
api, keyed on the kafka cluster id paired with the version, then extracts the
version.

internal/modules/data_pipeline_api_exposure_test.go drives the three modules end to
end through ExecuteHTTPModule and asserts the leak alongside the near misses a
strict review wants pinned: each service with one keying field missing, a generic
health response, a plain 200 and a 404.

verify: go test ./internal/modules, each matcher and extractor proven to bite
(break -> red, restore -> green).
2026-06-22 17:07:25 -07:00
TigahandGitHub 761e570d59 feat(modules): add sql dump, sqlite and redis rdb exposure modules (#204)
modules/recon/sql-dump-exposure.yaml flags an exposed SQL dump on its
mysqldump and pg_dump idioms paired against a guard that drops SQL shown
inside an html page, then extracts the dumped table name.

modules/recon/sqlite-database-exposure.yaml flags an exposed SQLite file on
the 16 byte format magic anchored to the start of the body, then extracts a
schema table name. anchoring the magic keeps a page that merely embeds the
header from being reported.

modules/recon/redis-dump-exposure.yaml flags an exposed Redis RDB snapshot on
the RDB magic anchored to the start of the body, then extracts the format
version.

internal/modules/database_file_exposure_test.go drives the three modules end
to end through ExecuteHTTPModule and asserts the leak alongside the near
misses a strict review wants pinned: a SQL tutorial page, a bare select, prose
that names the sqlite or redis format, a header embedded mid body, a plain 200
body and a 404, none of which may match.

verify: go test ./internal/modules, each matcher, magic anchor, guard and
extractor proven to bite (break -> red, restore -> green).
2026-06-22 17:07:15 -07:00
TigahandGitHub 28a01f0f83 feat(modules): add riak, couchbase and druid api exposure modules (#216)
modules/recon/riak-api-exposure.yaml flags an exposed Riak http api reachable
without authentication, keyed on the riak kv version paired with the core version,
then extracts the kv version.

modules/recon/couchbase-api-exposure.yaml flags an exposed Couchbase cluster
management api, keyed on the implementation version paired with the components
version that the bootstrap pool reports, then extracts the implementation version.

modules/recon/druid-api-exposure.yaml flags an exposed Apache Druid process that
runs without authentication, keyed on the druid package namespace paired with the
process memory block, then extracts the druid version.

internal/modules/distributed_db_exposure_test.go drives the three modules end to
end through ExecuteHTTPModule and asserts the leak alongside the near misses a
strict review wants pinned: each service with one keying field missing, a generic
version response, a plain 200 and a 404.

verify: go test ./internal/modules, each matcher and extractor proven to bite
(break -> red, restore -> green).
2026-06-22 17:06:51 -07:00
TigahandGitHub 7788550722 feat(modules): add qdrant, weaviate and chroma exposure modules (#222)
modules/recon/qdrant-api-exposure.yaml flags a Qdrant vector database that
serves its collections without an api key, keyed on the result envelope wrapping
a collections array paired with the ok status, then extracts the first
collection name. Qdrant gates /collections behind the api key, so an answer here
means the catalog is readable.

modules/recon/weaviate-api-exposure.yaml flags a Weaviate vector database that
leaks its host address and version over the meta api, keyed on the url-valued
hostname paired with the version field, then extracts the hostname. Weaviate
drops the modules field when none are enabled, so the match leans on the
hostname, which it always renders as a scheme and host.

modules/recon/chroma-api-exposure.yaml flags a reachable Chroma vector database
by its heartbeat api on both the v1 and v2 paths, keyed on the nanosecond
heartbeat field. The heartbeat stays anonymous by design, so this is rated as a
reachability signal rather than an auth bypass.

internal/modules/vector_db_exposure_test.go drives the three modules end to end
through ExecuteHTTPModule and asserts the leak alongside the near misses a strict
review wants pinned: each service with one keying field missing, a bare hostname,
a generic version response, a plain 200 and a 404.

verify: go test ./internal/modules, each matcher and extractor proven to bite
(break -> red, restore -> green).
2026-06-22 16:54:03 -07:00
TigahandGitHub 40482a8409 feat(modules): add argo cd exposure module (#219)
modules/recon/argocd-api-exposure.yaml flags an exposed Argo CD api server through
its unauthenticated /api/version endpoint, keyed on the KustomizeVersion and
HelmVersion fields a generic version response does not carry, then extracts the
server version.

internal/modules/argocd_exposure_test.go drives the module through ExecuteHTTPModule
and asserts the leak alongside the near misses: each keying field missing on its own,
a generic version json, a plain 200 and a 404.

verify: go test ./internal/modules, each matcher and extractor proven to bite
(break -> red, restore -> green).
2026-06-22 16:49:52 -07:00
TigahandGitHub 3ed9ea4b6f feat(modules): add kong, jolokia and nats api exposure modules (#215)
modules/recon/kong-api-exposure.yaml flags an exposed Kong admin api that grants
full control of the gateway, keyed on the available plugins map paired with the
admin listen address that the node reports, then extracts the kong version.

modules/recon/jolokia-api-exposure.yaml flags an exposed Jolokia agent that
bridges http to jmx, keyed on the agent and protocol fields of its version
response, then extracts the agent version.

modules/recon/nats-api-exposure.yaml flags an exposed NATS monitoring endpoint
that leaks the server topology, keyed on the server id paired with the max
payload, then extracts the server version.

internal/modules/management_api_exposure_test.go drives the three modules end to
end through ExecuteHTTPModule and asserts the leak alongside the near misses a
strict review wants pinned: each service with one keying field missing, a generic
version response, a plain 200 and a 404.

verify: go test ./internal/modules, each matcher and extractor proven to bite
(break -> red, restore -> green).
2026-06-22 16:49:45 -07:00
TigahandGitHub 612bb61d00 feat(modules): add solr, spark and hadoop yarn api exposure modules (#213)
modules/recon/solr-api-exposure.yaml flags an exposed Apache Solr admin api,
keyed on the solr spec version paired with the solr home that a generic json
endpoint does not carry, then extracts the solr version.

modules/recon/spark-api-exposure.yaml flags an exposed Apache Spark master whose
cluster state is reachable without authentication, keyed on a spark:// master url
paired with the alive worker count, then extracts the master url.

modules/recon/hadoop-yarn-api-exposure.yaml flags an exposed Hadoop YARN resource
manager, keyed on the cluster info wrapper paired with the resource manager
version, then extracts the hadoop version.

internal/modules/bigdata_api_exposure_test.go drives the three modules end to end
through ExecuteHTTPModule and asserts the leak alongside the near misses a strict
review wants pinned: each service with one keying field missing, a non spark url
behind the worker count, a generic json endpoint, a plain 200 and a 404.

verify: go test ./internal/modules, each matcher and extractor proven to bite
(break -> red, restore -> green).
2026-06-22 16:49:38 -07:00
TigahandGitHub 95523bc344 chore: remove stale lunchcat domain references (#134)
the lunchcat org and lunchcat.dev are inactive, so point the repo
link at github.com/vmfunc/sif and drop the dead lunchcat.dev support
link.

- Makefile: repo url -> vmfunc/sif; drop the lunchcat.dev line
- README + .all-contributorsrc: contributor links and projectOwner ->
  vmfunc
- config: drop "lunchcat" from the -api flag help text

copyright and attribution to lunchcat are kept for continuity
(license headers, LICENSE, man page) per the issue.

fixes #50
2026-06-22 16:48:49 -07:00
TigahandGitHub d0e986736d feat(modules): add influxdb, arangodb and neo4j exposure modules (#221)
modules/recon/influxdb-api-exposure.yaml flags an exposed InfluxDB instance through
its unauthenticated /health endpoint, keyed on the influxdb name paired with the
ready-for-queries health message, then extracts the version.

modules/recon/arangodb-api-exposure.yaml flags an ArangoDB instance reachable
anonymously through its /_api/version endpoint, keyed on the arango server name
paired with the version field, then extracts the version. the 200 gate is what
proves anonymous reach: an auth-enabled instance answers with a 401.

modules/recon/neo4j-api-exposure.yaml flags an exposed Neo4j instance through its
unauthenticated root discovery endpoint, keyed on the neo4j version paired with the
neo4j edition, then extracts the version.

internal/modules/http_database_exposure_test.go drives the three modules through
ExecuteHTTPModule and asserts the leak alongside the near misses a strict review
wants pinned: each service with one keying field missing, a non-arango response, an
arango that requires auth, a generic health json, a plain 200 and a 404.

verify: go test ./internal/modules, each matcher and extractor proven to bite
(break -> red, restore -> green).
2026-06-22 16:48:41 -07:00
TigahandGitHub 72f59532cf feat(modules): add metabase, zeppelin and jupyter exposure modules (#220)
modules/recon/metabase-api-exposure.yaml flags a Metabase instance that exposes
a live setup token without authentication, keyed on a non-null uuid token paired
with the anonymous tracking setting, then extracts the version tag. A live token
is the pre-auth chain behind CVE-2023-38646; a patched instance reports it as
null and is left alone.

modules/recon/zeppelin-api-exposure.yaml flags an Apache Zeppelin server that
discloses its version and build commit over the anonymous version api, keyed on
the version banner paired with the git commit id, then extracts the version. The
endpoint stays anonymous even on a shiro-secured instance, so this is rated as a
version leak rather than an auth bypass.

modules/recon/jupyter-api-exposure.yaml flags a Jupyter server whose status api
answers without a token, keyed on the activity, connections and kernels fields
it reports, then extracts the running kernel count.

internal/modules/analytics_ui_exposure_test.go drives the three modules end to
end through ExecuteHTTPModule and asserts the leak alongside the near misses a
strict review wants pinned: each service with one keying field missing, a
patched metabase that nulls its token, a generic version response, a plain 200
and a 404.

verify: go test ./internal/modules, each matcher and extractor proven to bite
(break -> red, restore -> green).
2026-06-22 16:48:34 -07:00
TigahandGitHub 24b573a368 fix(scan): scope passive ct-feed subdomains to the target (#181)
shared/multi-SAN certificates list unrelated names, so collecting every
crt.sh/certspotter entry reported off-scope hosts as subdomains of the
target. keep only the target or a subdomain of it.
2026-06-22 16:48:23 -07:00
TigahandGitHub 4d680074b8 fix(scan): don't count a reflected payload as lfi evidence (#179)
the data:// wrapper payload contains the base64 "PD9waHA" that the
base64-php evidence pattern matches, so a server reflecting the request
path was flagged as a critical inclusion. skip the self-reflected match.
2026-06-22 16:48:17 -07:00
TigahandGitHub f0aa1895e9 fix(scan): accept quoted hsts max-age values (#176)
rfc 6797 permits a quoted-string directive value, so a quoted
max-age="31536000" failed strconv.Atoi and got flagged as too short.
2026-06-22 16:48:11 -07:00
TigahandGitHub 6d903a4752 fix(scan): drop "query" from generic db admin-panel match (#174)
isAdminPanel's default branch (the generic /db/, /sql/, /mongodb/ paths) matched
the keyword "query", which is a substring of jQuery and querySelector. any site
whose catch-all returns 200/403/401 at one of those paths and runs any javascript
was reported as a database admin panel at high severity.

drop "query". the remaining keywords (database, sql, mysql, postgresql, mongodb)
only match db-topical pages, so real generic interfaces are still detected via a
sibling keyword.
2026-06-22 16:48:05 -07:00
TigahandGitHub 9c241cf185 fix(scan): key joomla detection on the generator tag (#173)
detectJoomla matched any body containing the lowercase word "joomla", so pages
that merely mention it (joomla hosting marketing like hostinger.com) were
misreported as Joomla. the canonical Joomla! generator tag is capitalised, so
the lowercase match also missed sites whose only tell is that tag.

match the Joomla! generator meta tag and the joomla asset paths
(/media/vendor/joomla, /media/system/js/core.js) instead. keying on the meta
attribute rather than the plain tagline also rejects prose that quotes it.
2026-06-22 16:47:58 -07:00
TigahandGitHub 26ccbea888 fix(scan): detect open redirects through scheme-malformed payloads (#172)
pointsAtSentinel parsed the reflected Location with url.Parse, which only
finds the authority when the scheme is followed by "//". the redirect probe
already sends "https:/host" and "https:host" payloads, but a server that
reflected them landed the host in Opaque or the path, so the off-site
redirect went unreported. normalise the scheme slash count to "//" before
parsing so those forms resolve to the host a browser would navigate to.
scoped to http(s), so an opaque "mailto:host" is left untouched.
2026-06-22 16:47:52 -07:00
TigahandGitHub bca4831df1 fix(frameworks): stop ghost detector matching the generic ghost-button (#163)
the ghost detector keyed on the bare "ghost-" body substring, which fires on
any page using the common "ghost-button" or skeleton-loader CSS class (e.g. a
WordPress theme), scoring 0.65 and reporting it as Ghost CMS. key on the
canonical `<meta name="generator" content="Ghost` tag instead, matching the
existing astro detector. the /ghost/api/ path and Ghost header markers are
unchanged.

a ghost site that emits neither the generator meta nor a /ghost/api/
reference is no longer detected; the Ghost response header alone scores
below the detection floor.
2026-06-22 16:47:43 -07:00
TigahandGitHub 291846dde5 fix(frameworks): make Phoenix and AdonisJS detection specific (#153)
the phoenix and adonis detectors matched bare substrings ("phx-", "phoenix",
"adonis") that fire on unrelated pages: a "phx-" css class on a phoenix,
arizona site, or any markup containing the word "adonis". replace them with
markers the frameworks actually emit. phoenix keys on the liveview container
attributes data-phx-main, data-phx-session and data-phx-static; adonis on its
default adonis-session cookie.

this narrows detection: plain (non-liveview) phoenix and session-less adonis
apis are no longer matched. the markers we now key on (liveview's container
attributes, adonis's default session cookie) are ones ordinary prose cannot
forge. each detector gains a true-positive test and a false-positive tripwire.
2026-06-22 16:47:34 -07:00
TigahandGitHub 21c1d1c8a5 feat(modules): implement size matcher and kv extractor (#140)
the engine declared size matchers and kv extractors but the executor
dropped them (size fell through to the default case, kv was never read).
wire both: size matches the response body length in bytes, kv records
every response header as a key-value pair namespaced by the extractor
name.

this unblocks the headers.go conversion in #52, which needs a full header
dump the known-set regex extractors cannot reproduce; the headers.yaml
module and the headers.go removal are a separate follow-up. the extractor
is named kv to match docs/modules.md (the struct comment said kval). the
declared json extractor stays deferred since it needs a json-path
dependency and a path-syntax decision.

refs #52
2026-06-22 16:47:25 -07:00
vmfunc 8078978a44 feat: notify integrations (slack, discord, telegram, webhook)
ship findings to chat/webhook sinks after a scan so continuous recon can
alert on what it turns up. each provider is one POST through httpx.Client,
so the global proxy/rate-limit/header config applies and there's no extra
http stack. config resolves env-first (SLACK_WEBHOOK_URL, DISCORD_WEBHOOK_URL,
TELEGRAM_BOT_TOKEN/TELEGRAM_CHAT_ID, NOTIFY_WEBHOOK_URL), overridable by a
notify-compatible yaml file so existing projectdiscovery/notify configs port
over. -notify enables it, -notify-severity gates on the finding severity
ladder (default medium), -notify-config points at the yaml. wired after the
scan loop on the severity-filtered finding set; no provider configured is a
silent no-op.
2026-06-10 16:40:14 -07:00
vmfunc 6ec0b60e5a feat: diff mode with json snapshot store
re-scans become a monitor: -diff snapshots each target's normalized
findings to a per-target json file and, on the next run, surfaces only
the delta (+ new / - gone) against the last snapshot, then overwrites it
so each run diffs against the previous one. behavior is unchanged when
-diff is off.

new internal/store keys the set-difference off finding.Key (already
stable across runs) and uses only encoding/json + os - no new deps.
snapshot files are sanitized per target (no traversal), written 0600
under 0750 dirs. -store picks the location: explicit dir, else the log
dir, else <user-config>/sif/state. a missing snapshot is a clean
baseline, a corrupt one self-heals on the next save.
2026-06-10 16:39:04 -07:00
vmfunc 22168611e4 perf(scan): work-stealing worker pool replacing static stride partition
static i%threads stride partitions assign each item to a fixed worker, so a
goroutine stuck on a slow or timing-out host stalls while the rest idle behind
it (head-of-line blocking). add internal/pool.Each: workers all pull from one
shared channel, so a slow item only blocks its own worker and the others keep
draining. migrate git, ports, robots (scan), dnslist, dork, dirlist and
subdomaintakeover off the stride loops; per-item work, mutex-guarded appends and
progress increments are unchanged, results were already unordered.
2026-06-10 16:39:04 -07:00
vmfunc 57b1bd7113 fix(favicon): use twmb/murmur3; spaolacci does checkptr-unsafe pointer math that crashes under -race (which CI runs) 2026-06-10 16:08:47 -07:00
vmfunc ab731d0562 feat(scan): add jwt, openapi and favicon-hash scanners
jwt fetches the target once then analyzes every harvested token offline:
flags alg:none, the rs256->hs256 confusion surface, missing/expired exp
and plaintext sensitive claims, and cracks a small bundled weak-hmac list.
openapi probes the conventional spec paths, parses json/yaml and enumerates
paths plus unauthenticated operations. favicon computes the shodan-style
mmh3 hash (python base64.encodebytes chunking, signed int32) for tech
fingerprinting and the http.favicon.hash pivot, pinned by a golden test.
2026-06-10 15:50:59 -07:00
vmfunc ef0408ee8d feat: pipe mode (stdin targets, naked-host, -silent plain output)
sif can now slot into unix pipelines. stdin is drained for targets when
it's a pipe (keyed off stdin's mode, not stdout), alongside -u/-f. naked
hosts are accepted and default to https://; explicit http(s) is kept,
other schemes rejected. -silent routes all banner/spinner/log chrome to
stderr and prints one normalized finding per line to stdout via
finding.Flatten, so `subfinder | sif -silent | notify` works.
2026-06-10 15:50:58 -07:00
vmfunc 0383a7bcd2 perf(scan): drain response bodies so pooled connections get reused
go only returns a conn to the idle pool when the body is read to EOF
before Close. the header-only and early-return scan paths closed an
unread body, leaking the conn and forcing a fresh dial each request.
route those close sites through httpx.DrainClose so the tuned pool from
phase 1 actually gets reused. body-read paths (scanner/io.ReadAll) are
left untouched.
2026-06-10 15:50:58 -07:00
vmfunc a5f42ddfa6 feat(dnslist): async dns resolution with wildcard filtering
dnslist previously http-probed every wordlist candidate through the
blocking os resolver, so a big list meant a request per dead name and a
wildcard zone flooded results. resolve each candidate first via a new
internal/dnsx (retryabledns over a bundled 1.1.1.1/8.8.8.8/9.9.9.9 pool,
promoted to a direct dep), fingerprint the apex with random nonexistent
labels to detect catch-all zones, and http-probe only the names that
actually resolve and aren't wildcard. add -resolvers to override the
pool. resolverFn is a package-level seam so the dnsx tests stay
hermetic; the dnslist newDNSResolver seam keeps the integration test
network-free.
2026-06-10 15:30:03 -07:00
vmfunc 1237f3f09e feat(finding): normalized finding layer for notify and diff
scan results live in ~two dozen structs with no shared shape, so every
consumer that wants "what did this run turn up" reimplements the
type-switch. add internal/finding: an ordered Severity (info<low<medium<
high<critical, with parse/compare) and Flatten, the single type-switch
that collapses every scan result struct into flat, severity-ranked
Findings keyed module:identifier for stable dedup/diff.

wire collectFindings off Flatten in the run loop so notify and diff
(later bundles) build on one normalization path instead of re-deriving
it; the report path keeps emitting raw json blobs unchanged. expose
JavascriptScanResult.SupabaseFindings so the js internals stay private.

the guard test iterates a representative instance of every ResultType
and fails if Flatten lacks a case (falls through to :unhandled) - so a
new scanner can't ship without a Flatten case landing too.
2026-06-10 15:29:20 -07:00
vmfunc 546ab091da perf(httpx): tune transport for connection reuse and add DrainClose
the shared transport was a bare DefaultTransport.Clone() with the stock
MaxIdleConnsPerHost=2, and call-sites only close response bodies without
draining them - so go could never return a conn to the idle pool and every
request re-dialed. high thread counts just thrashed the dialer.

- plumb Threads through Options into buildTransport; size MaxIdleConnsPerHost
  to the worker count (floored) so concurrent workers on one host pool instead
  of re-dialing, MaxIdleConns=512, MaxConnsPerHost=0, IdleConnTimeout=90s,
  ForceAttemptHTTP2. the socks5 branch gets its own keepalive net.Dialer so it
  doesn't lose os-level pooling under proxy.Direct.
- add DrainClose to read (capped) and close a body so the conn is reusable.
- benchmark proves it: 50 sequential requests reuse 1 conn tuned vs 50 bare.
2026-06-10 15:29:20 -07:00
vmfunc c3a755f934 feat: live-host probe and sarif/markdown report export
adds an httpx-style -probe scanner reporting liveness, final status, page
title, server header and the redirect chain, plus -sarif/-markdown export
flags that serialize the collected run after the scan loop. the report
serializers live in a decoupled internal/report package consuming a raw-json
result model so they never import scan types.
2026-06-10 14:47:17 -07:00
vmfunc 5050900f29 feat(dirlist): response filters, wildcard calibration and custom wordlists
the old scanner surfaced every response that wasn't 404/403, so modern SPA
catch-all 200s flooded the output and made -dirlist near-useless. add ffuf-style
matching:

- -mc/-fc/-fr and -fs/-fw filter by status, regex, body size and word count;
  bodies are read through a capped io.LimitReader so size/word counts are
  deterministic and memory stays flat. filters win over matches.
- -ac auto-calibrates the soft-404 baseline from a few deterministic
  non-existent paths and drops responses matching that wildcard shape.
- -w overrides the size switch with a local file or remote list (fetched through
  the shared client so proxy/rate-limit apply); -e appends extensions per word.

size and words are added to DirectoryResult for the json output.
2026-06-10 14:47:17 -07:00
vmfunc 320fc3d4e7 test(modules): cover matchers, extractors, loader and executor
the yaml module engine (the user-facing extensibility surface) had 0%
test coverage. add table-driven tests for the matcher types
(status/word/regex + and/or + negative), checkWords/checkRegex (incl
invalid-pattern fail-closed under AND, skip under OR), runExtractors
(regex capture groups, group-index bounds, part selection),
substituteVariables and generateHTTPRequests (path x payload expansion),
and ParseYAMLModule on valid + malformed yaml. drive ExecuteHTTPModule
end-to-end against an httptest server through the shared httpx client so
matcher hits and extractor captures are exercised for real. coverage
0% -> 93.7%.

also: ExecuteDNSModule/ExecuteTCPModule were stubs returning an empty
result with nil error, so a type:dns/type:tcp module silently reported
"0 findings" - indistinguishable from a real clean scan. make them
return ErrUnsupportedModuleType (sentinel, wrapped with the module id) so
the existing caller logs a clear failure instead. a test pins the new
behavior.

bodyclose is excluded for test files in .golangci.yml: the synthetic
*http.Response fixtures carry no socket, mirroring the existing _test.go
slack for errcheck/noctx/gosec.
2026-06-10 14:47:17 -07:00
vmfunc 839c0a779c fix(scan): dnslist dedup, robots recursion bound, framework version lookup, takeover cname
four recon-flagged bugs, each with a focused test:

- dnslist fired both http and https per candidate and counted a "found"
  on any non-error response (incl 404 and wildcard catch-all redirects),
  so every host double-counted and a wildcard-dns host flooded results.
  probe http then https with per-subdomain dedupe, gate on a meaningful
  (2xx, non-redirect) status, and stop chasing redirects so a catch-all
  301 reads as a redirect instead of a 200.

- fetchRobotsTXT recursed on every 301 Location with no depth limit and
  no visited set, so an A->B->A loop blew the stack. bound it to a named
  hop cap and a visited set, iteratively.

- framework cve lookup used best.version ("unknown" when the detector
  only fingerprints the framework) and threw away the version
  ExtractVersionOptimized dug out of the body, missing CVEs. reconcile
  via resolveVersion, preferring the extracted concrete version.

- subdomain takeover flagged a dangling cname whenever a no-such-host
  coincided with ANY cname (LookupCNAME echoes the host back for plain A
  records). only flag when the cname points off-host at a known
  takeoverable provider.
2026-06-10 14:47:17 -07:00
vmfuncandvmfunc dbe79c495e feat(scan): add web crawler and passive subdomain/url discovery
-crawl spiders same-host links/scripts/forms through the shared httpx
client so proxy/headers/rate-limit and robots.txt are honored, bounded
by -crawl-depth. -passive pulls subdomains from keyless ct feeds (crt.sh,
certspotter) and historical urls from wayback, each source isolated so
one feed being down doesn't sink the rest and the target sees no traffic.
2026-06-09 18:11:38 -07:00
vmfuncandvmfunc 9401aa669e feat(scan): add cors, open-redirect and reflected-xss probes
three active web-vuln probes wired into the per-target loop:

- cors: crafts attacker origins (evil sentinel, null, prefix/suffix
  bypass, http downgrade) and flags responses that reflect them in
  access-control-allow-origin, ranking reflection+credentials high.
- redirect: injects a controlled sentinel host plus bypass variants
  (//, https:/, backslash, null-byte, userinfo @) into redirect-prone
  params and catches 30x location, meta-refresh and js redirects that
  resolve off-site.
- xss: injects a unique canary wrapped in breaking chars, classifies
  the reflection context (html/attribute/script) and reports only the
  chars that survive unescaped where they matter, so escaped
  reflections don't false-positive.

all route through httpx.Client so proxy/-H/-cookie/-rate-limit apply.
hermetic httptest coverage plus integration testbed entries.
2026-06-09 18:11:38 -07:00
vmfuncandvmfunc b4e78114d7 feat(js): extract secrets and endpoints from scanned javascript
the -js pipeline already pulls every <script> into a buffer but only
mined supabase jwts from it. reuse that buffer to run a credential
regex bank (aws/github/slack/stripe/google keys, pem blocks, plus
entropy-gated generic apikey/secret/token assignments) and a
linkfinder-style endpoint extractor that resolves relatives to
absolute urls. both dedupe across scripts and surface through the
existing js logger and result struct, no new flag.
2026-06-09 18:11:38 -07:00
vmfunc d0bdcf1690 feat: shared http client with proxy, custom headers and rate limiting
every scanner spun up its own &http.Client, so there was no single place
to apply a proxy, custom headers, a cookie or a rate limit. add an
internal/httpx package that builds one configured transport at startup and
hand it to every scanner via httpx.Client(timeout), keeping behavior
identical when nothing is set (plain client when Configure was never
called).

- httpx.Configure wires -proxy (http/https/socks5), -H/--header, -cookie
  and -rate-limit into a package-level RoundTripper that paces via a
  rate.Limiter and only sets headers the caller hasn't already, so a
  scanner's explicit api key still wins.
- route the scan/wordlist downloads that used http.DefaultClient through
  the shared client too; ports tcp dialing is left untouched.
- clamp -threads to a floor of 1: it feeds wg.Add across the scanners, so
  0 was a silent no-op and a negative value panicked the waitgroup.

document the new flags in the readme, usage docs and man page.
2026-06-09 17:28:14 -07:00
celesteandGitHub cb194406a7 Merge pull request #116 from vmfunc/test/scanner-seams
test(scan): seam shodan/securitytrails/cloudstorage/dnslist for hermetic tests
2026-06-09 16:26:40 -07:00
vmfunc 912f6e8e0e test(scan): seam shodan/securitytrails/cloudstorage/dnslist for hermetic integration tests
the remaining hardcoded base urls had no test seam, so their drivers could
only be exercised against the live apis. promote them to package vars (matching
the dirlist/git/ports pattern from #112) and route dnslist's per-host probes
through an injectable transport, then add integration tests that pin each at a
local httptest fixture. defaults equal the old const values so behavior is
unchanged.
2026-06-09 16:07:20 -07:00
vmfunc 094f1e7806 fix(output): dedupe non-tty progress milestones
concurrent workers (-threads 40) all hit the same milestone bucket on
increment, spamming ~10 duplicate [25%] lines. track the last printed
bucket under p.mu and only print when it advances.
2026-06-09 16:03:52 -07:00
celesteandGitHub 83ac92a4b8 Merge pull request #113 from vmfunc/fix/frameworks-confidence
fix(frameworks): require a real signature match + fix cve version matching
2026-06-09 15:02:40 -07:00
celesteandGitHub 7f0e4cd128 Merge pull request #112 from vmfunc/test/e2e-integration
test: hermetic e2e integration suite
2026-06-09 14:50:54 -07:00
vmfunc 29d94e5352 fix(frameworks): require a real signature match, fix cve version matching
- recenter the detection confidence (sigmoid centered at 0.3) so a single weak
  signature match no longer clears the 0.5 threshold. before, sigmoid(0) was 0.5
  so *any* match counted as a detection - that's the magento-on-a-plain-page
  false positive from the live run. real detections match ~50%+ of signature
  weight, so the existing detector tests are unaffected
- getVulnerabilities matched affected versions with a raw string prefix, so "4.2"
  also matched "4.20"; match only on dotted boundaries now
- break confidence ties on name so the picked framework is deterministic
- add regression tests for the confidence floor and the version boundary
2026-06-09 14:46:10 -07:00
vmfunc ce3075ad91 test: hermetic e2e integration suite
- make the four wordlist base urls (dirlist/dnslist/git/ports) package vars
  instead of consts so tests can repoint them at a local fixture; the default
  values are byte-for-byte unchanged
- add internal/scan/integration_test.go behind a //go:build integration tag: it
  stands up a local "vulnerable app" httptest server with planted artifacts and
  runs git/dirlist/cms/headers/sql/lfi/ports against it, asserting real findings
- go.yml runs them via `go test -tags=integration`; the default test run is
  untouched (the tag keeps them out)
- document the integration run in docs/development.md
2026-06-09 14:32:26 -07:00
vmfunc 661480a56d feat: stamp and surface the build version
- add internal/version: resolve from the release ldflag, else the go build
  info (module tag / vcs revision), else "dev"
- show the version on the boot banner and for `sif version`
- Makefile now stamps `make` builds via git describe (matching the release ci),
  so local/go-install builds report a real version instead of "dev"
- patchnotes.ShowOnce skips pseudo/dev versions so non-release builds dont make
  a doomed github call
- document sif version / sif patchnote / SIF_NO_PATCHNOTES in the readme + usage
2026-06-09 14:18:28 -07:00
celesteandGitHub 76e8893ee2 Merge pull request #110 from vmfunc/fix/supabase-timeout
fix(js): give supabase requests a real timeout
2026-06-09 14:11:53 -07:00
vmfunc eb33321102 fix(js): give supabase requests a real timeout
doSupabaseRequest and the signup call used a bare http.Client{} with no
timeout, so a slow supabase project could hang the whole js scan. thread the
scan's --timeout down through ScanSupabase into every supabase request.
2026-06-09 13:56:01 -07:00
vmfunc 133224c348 refactor: dedupe url scheme stripping
`strings.Split(url, "://")[1]` was copy-pasted in 18 spots and panics on a
schemeless target (index out of range). add a small stripScheme helper in the
scan package - and a guarded equivalent in logger, which cant import scan - so
a bare host degrades gracefully instead of crashing the scan.
2026-06-09 13:51:04 -07:00
vmfunc 5e10c1857b feat: show release notes via patch notes
- `sif patchnote` (also `-pn`) fetches the latest github release and renders
  its notes with glamour
- on the first run of a new version those notes are shown once, then recorded
  so they dont show again - best-effort, so dev builds, the SIF_NO_PATCHNOTES
  opt-out, and any network failure stay quiet
- wire up `var version` so the release `-X main.version` ldflag actually lands,
  and add `sif version`
2026-06-08 19:13:03 -07:00
vmfunc 9326465a46 chore: drop decorative emoji and redundant comments
strips the emoji from the subdomaintakeover/cloudstorage/supabase log
prefixes and trims comments that just restate the code (the
parameters/returns block on SubdomainTakeover, a couple of "this provides
type safety" notes) so the scanners read like whois.go/headers.go.
2026-06-08 18:53:06 -07:00