numericClaim only accepted exp as a json number, but some emitters
serialize it as a quoted string. a token with a past string exp was
reported as missing exp instead of expired, an expiry false negative
and a missing-claim false positive on the same token. also parse a
string value via strconv.ParseFloat, falling back to absent when it
does not parse.
hsts grading was gated on the scheme of the originally requested url,
but the client follows redirects, so an http target that redirects to
https skipped the hsts check entirely and dropped a high-severity
finding. decide the scheme from the final response request url instead,
falling back to the requested url only when no response request is set.
paths was typed as a map of operation structs, so a spec with a $ref
string path item (valid in openapi 3.1) failed to decode and the whole
spec was dropped with no finding. decode path items into a generic tree
and extract per-operation security manually, treating a non-array
security value as absent so it inherits the global default rather than
being reported as an anonymous endpoint.
pointsAtSentinel parsed the reflected location without the whitespace
and control-byte removal a browser applies, so payloads like a
space-prefixed scheme-relative host or a tab embedded in the scheme
parsed to an empty host and a real off-site redirect was missed. strip
leading and trailing c0-or-space and all tab, cr and lf before parsing,
matching the navigation-time normalization, without removing mid-string
spaces so a same-site target is never reclassified as off-site.
the http scheme downgrade origin was queued against every target, so on
a plain-http target the http://host probe is the target's own origin and
any correct same-origin reflection was reported as a high-severity
downgrade. mark that origin https-only and skip it when the target
scheme is not https, where a downgrade is meaningless, while still
firing it on https targets that genuinely trust the http origin.
checkDatabaseErrors matched db error regexes against each response with
no baseline, so a site serving a shared error template, custom 500 or
waf block page containing a db error string reported an identical
high-severity sql injection finding on every probed path. capture the
main-url response as a baseline and skip, per pattern, any error string
already present in it while still checking other patterns, so only
differential probe-caused errors are reported.
the lfi scan matched evidence regexes against each response with no
baseline, so a page whose static content matched a pattern (a blog
containing an /etc/passwd-shaped line, a code sample with <?php) was
reported as an lfi finding for every parameter and payload combination.
take one baseline get of the unmodified target and exclude, per
evidence class, any pattern already present in it, so only
payload-caused disclosures are counted.
the live-response path flagged any host whose body matched a provider
fingerprint without checking dns, so a page merely containing a
provider 404 string was reported as a takeover. resolve the cname and
only confirm when it maps to the same provider; a contradicting cname
kills the flag. providers with a body signature but no apex to
correlate, and unresolvable lookups, degrade to a medium potential
finding via a new confidence field rather than confirming or dropping.
Crawl only capped recursion depth, not breadth per level. A link-heavy page
(pagination, faceted search) drives fetch count toward branching^depth with no
ceiling: a depth-3 crawl of a page linking 40 fresh urls issued 1641 real
fetches.
Cap total fetches at a page budget via an atomic counter in an OnRequest hook
that aborts once exceeded, and add a Truncated flag on CrawlResult so callers
can tell a run was cut short.
a detector accuracy audit surfaced two classes of bug in the framework
detectors.
bare-brand header false positives: header-only signatures matched a
brand name as a substring across every header name and value, so a
detector fired on any response that merely referenced the brand (a
vendor cdn named in a link or csp value, a cookie sharing the prefix).
add an optional Header field to Signature that scopes a header-only
match to one named header's value, and apply it (or a structural
anchor) per detector:
- express: "Express" scoped to x-powered-by, was firing on an
express_checkout cookie.
- flask: "Werkzeug" scoped to the server header.
- symfony: dropped the bare "symfony" word (symfony sets no such
header, it fired on symfony.com links); the x-debug-token header is
the marker.
- shopify: key on the x-shopify response headers instead of the bare
"Shopify" word, which fired on a cdn.shopify.com link.
- remix: dropped the bare "remix"/"_remix" substrings that fired on a
track_remix.mp3 asset; window.__remixContext is the definitive
marker.
- spring boot: anchor the whitelabel title in its h1 tag context so a
tutorial discussing the error does not fire.
the gin and fastapi detectors are removed: gin keyed on the
"gin-gonic" import-path string (appears in tutorials, never in a real
gin response) and fastapi on bare words matching the projects' doc
domains. neither framework advertises itself in a response header or a
non-prose body marker, so there is no clean passive signal to anchor
on.
version mis-extraction: drop the low-confidence ".*?" version
fallbacks (rails, django, laravel, spring), whose unbounded gap
grabbed the first version-shaped number after the framework word and
reported an unrelated asset's cache-buster when no real version was
present. let isValidVersionString accept a single integer so a bare
major such as drupal's "Drupal 10" is no longer rejected as "unknown".
each false positive and version bug is covered by a regression test.
Ran `gofmt -w .` accross the repo to fix formatting drift.
Mechanical `gofmt -w .` only. No functional or behavioural changes.
CONTRIBUTING.md requires gofmt-clean code; these files had slipped.
rename the go module path from github.com/dropalldatabases/sif to
github.com/vmfunc/sif across go.mod, all imports, the golangci exclude
list, release install docs and docs. pure string rename, no logic change.
#180 added the calibrate bool to SQL but the integration-tagged test
(only built under -tags=integration, outside normal CI) still called the
4-arg form. pass false (no calibration) to restore behavior.
replace bare gin/api/cake/svelte/ember/backbone/meteor substrings with
specific markers (Backbone.Model, ember-application, __meteor_runtime_config__,
CAKEPHP cookie, svelte/internal) so prose and CORS headers stop matching.
adds paired false-positive/positive coverage per framework.
calibrate against reflecting catch-alls whose body size tracks path
length so exact-shape calibration no longer misses them; -ac now drives
both dirlist and sql. updates the admin-panel query test to the new SQL
signature. adds soft-404 + calibration coverage.
CMS only flagged Drupal on X-Drupal-Cache: HIT or the Drupal 7 Drupal.settings
marker, so Drupal 8-11 went undetected: a MISS cache header was ignored, and
cdn-fronted sites serve none of those markers in the body at all. verified live
that london.gov.uk and georgia.gov (both Drupal) were missed.
key on the Drupal-specific headers instead: any X-Drupal-Cache or
X-Drupal-Dynamic-Cache, plus X-Generator naming Drupal. these survive cdn
caching. drupalSettings (8+) and Drupal.settings (7) cover uncached bodies.
load yaml-defined detectors from ~/.config/sif/signatures (AppData\Local
on windows), mirroring the user-modules convention, so a framework sif
does not ship can be detected without a rebuild. they load lazily once
per run from DetectFramework and register alongside the built-ins.
each file is one detector, scored by the same weighted signature match as
the built-ins. confidence is linear rather than their sigmoid (importing
it would cycle), so a detector clears the 0.5 threshold once its matched
weights pass half. a name matching a built-in overrides it and inherits
that built-in's version patterns and cves, the same as a user module. a
single unparseable file warns and is skipped rather than failing the scan.
implements the custom signature support help-wanted item in contributing.
detectWordPress treated any 2xx on /wp-login.php, /wp-admin/ or /wp-config.php
as wordpress, but the http client follows redirects and many sites soft-404
(200 for every path) or catch-all redirect to their homepage. a probed path
then returns 200 without being wordpress at all, so plain static sites and
marketing pages were flagged as wordpress.
read the probed response and require one of the existing wordpress markers in
its body before counting it. a real /wp-login.php still references wp-includes
assets, so genuine detection holds.
classifyXSSContext defaulted any non-html, non-script reflection to attribute context. a reflection echoed into element text with the angle brackets escaped but quotes left raw (common for encoders limited to < > &) was then reported as a high-severity attribute injection, even though a quote in text content has no delimiter to break out of.
classify a reflection as attribute only when the canary actually lands inside an open tag; everything else is inert element text and yields no finding. the in-tag test is a byte-scan, not a parser, so rare malformed markup can still mis-bucket.
the ghost and pantheon entries keyed on each platform's older unclaimed-domain
copy, which the live pages no longer serve, so a real dangling subdomain went
undetected. verified against live unclaimed subdomains: <random>.ghost.io now
returns "Failed to resolve DNS path for this host" and <random>.pantheonsite.io
returns a "404 - Unknown site" page. key on those instead.
ghost also serves a bare "Site unavailable." on some hosts; the body match holds
one substring per service, so it keys on the distinctive arm and a page showing
only "Site unavailable." is not detected.
The cors probe followed up to three redirects, then read the cors headers off the final response. Its own comment said the cap existed so the headers came from the host we asked about, so the code contradicted its intent: a target that only redirects to a permissive third party had that party's misconfiguration reported against it.
Stop following redirects, matching the open-redirect scanner. The redirect cap was a const shared with the xss probe, so xss now owns its own xssMaxRedirects with its follow behavior unchanged.
the extractPotentialBuckets TODO asked to handle non-adjacent label
combos and strip the tld. strip the trailing tld label so we stop
guessing it ("com", "com-s3", "example-com"), and pair every label
with every other rather than only adjacent ones, so a deep host like
shop.cdn.example yields shop-example too.
this widens the candidate set (and the s3 probes) per host. multi-part
suffixes like .co.uk still leave a junk label; publicsuffix would
refine that as a follow-up.
scanLines used a default bufio.Scanner (64k token cap) and never
checked scanner.Err(), so a wordlist line past the cap silently halted
the scan and dropped that line plus everything after it. return the
error so the wordlist loaders fail loud instead of running against a
truncated list.
move FaviconHash and its base64 chunking into a leaf package so the scan check and the module engine share one implementation instead of each carrying a copy. no behavior change; the golden test moves with it.
detectWordPress flagged any page whose body merely contained the word
"wordpress", so sites that only mention it (wp-hosting marketing, comparison
pages) were misreported: acquia.com, a Drupal site, was detected as WordPress.
wp-content, wp-includes and wp-json already identify a real install and appear
on every wordpress page, so the bare word adds only false positives. drop it.
specToResult computed an operation's auth status with len(op.Security) == 0,
which a value slice cannot tell apart from an absent block: an operation that
overrides the global requirement with an empty array (security: []) or an
empty requirement object (security: [{}]) decoded the same as one that simply
inherits the global default, so those deliberately-public endpoints were
reported as authenticated. decode the operation security into a pointer to
keep absent distinct from explicit-empty, and treat both empty forms as
anonymous-reachable.
the page scanner and the next.js manifest parser both reassembled the
response body line by line with a default bufio.Scanner (64k token
cap), so a line past the cap (common with minified or inlined js)
silently halted the read and dropped every script and route reference
after it. read the whole body instead, capped at 5mb.
parsing the raw page bytes also fixes script tags split across a
newline, which the old line-joined reassembly merged and missed.
the asp.net detector carried {Pattern: "X-Powered-By: ASP.NET", HeaderOnly:
true}, but containsHeader matches a signature against each header name and each
value separately, never against a joined "name: value" string. so that pattern
could never match anything. it only added its 0.4 to the detector's total
weight, diluting every real signal: a genuine response (an X-AspNetMvc-Version
header, an X-Powered-By: ASP.NET header, an .aspx link) scored 1.1/3.8 = 0.47,
just under the 0.5 threshold, and went undetected.
the live "ASP.NET" header signature already matches the X-Powered-By value, so
removing the dead one loses no coverage and lifts that same response to
1.1/3.4 = 0.56. adds a regression test for it.
shared/multi-SAN certificates list unrelated names, so collecting every
crt.sh/certspotter entry reported off-scope hosts as subdomains of the
target. keep only the target or a subdomain of it.
the data:// wrapper payload contains the base64 "PD9waHA" that the
base64-php evidence pattern matches, so a server reflecting the request
path was flagged as a critical inclusion. skip the self-reflected match.
isAdminPanel's default branch (the generic /db/, /sql/, /mongodb/ paths) matched
the keyword "query", which is a substring of jQuery and querySelector. any site
whose catch-all returns 200/403/401 at one of those paths and runs any javascript
was reported as a database admin panel at high severity.
drop "query". the remaining keywords (database, sql, mysql, postgresql, mongodb)
only match db-topical pages, so real generic interfaces are still detected via a
sibling keyword.
detectJoomla matched any body containing the lowercase word "joomla", so pages
that merely mention it (joomla hosting marketing like hostinger.com) were
misreported as Joomla. the canonical Joomla! generator tag is capitalised, so
the lowercase match also missed sites whose only tell is that tag.
match the Joomla! generator meta tag and the joomla asset paths
(/media/vendor/joomla, /media/system/js/core.js) instead. keying on the meta
attribute rather than the plain tagline also rejects prose that quotes it.
pointsAtSentinel parsed the reflected Location with url.Parse, which only
finds the authority when the scheme is followed by "//". the redirect probe
already sends "https:/host" and "https:host" payloads, but a server that
reflected them landed the host in Opaque or the path, so the off-site
redirect went unreported. normalise the scheme slash count to "//" before
parsing so those forms resolve to the host a browser would navigate to.
scoped to http(s), so an opaque "mailto:host" is left untouched.
the ghost detector keyed on the bare "ghost-" body substring, which fires on
any page using the common "ghost-button" or skeleton-loader CSS class (e.g. a
WordPress theme), scoring 0.65 and reporting it as Ghost CMS. key on the
canonical `<meta name="generator" content="Ghost` tag instead, matching the
existing astro detector. the /ghost/api/ path and Ghost header markers are
unchanged.
a ghost site that emits neither the generator meta nor a /ghost/api/
reference is no longer detected; the Ghost response header alone scores
below the detection floor.
the phoenix and adonis detectors matched bare substrings ("phx-", "phoenix",
"adonis") that fire on unrelated pages: a "phx-" css class on a phoenix,
arizona site, or any markup containing the word "adonis". replace them with
markers the frameworks actually emit. phoenix keys on the liveview container
attributes data-phx-main, data-phx-session and data-phx-static; adonis on its
default adonis-session cookie.
this narrows detection: plain (non-liveview) phoenix and session-less adonis
apis are no longer matched. the markers we now key on (liveview's container
attributes, adonis's default session cookie) are ones ordinary prose cannot
forge. each detector gains a true-positive test and a false-positive tripwire.
static i%threads stride partitions assign each item to a fixed worker, so a
goroutine stuck on a slow or timing-out host stalls while the rest idle behind
it (head-of-line blocking). add internal/pool.Each: workers all pull from one
shared channel, so a slow item only blocks its own worker and the others keep
draining. migrate git, ports, robots (scan), dnslist, dork, dirlist and
subdomaintakeover off the stride loops; per-item work, mutex-guarded appends and
progress increments are unchanged, results were already unordered.
jwt fetches the target once then analyzes every harvested token offline:
flags alg:none, the rs256->hs256 confusion surface, missing/expired exp
and plaintext sensitive claims, and cracks a small bundled weak-hmac list.
openapi probes the conventional spec paths, parses json/yaml and enumerates
paths plus unauthenticated operations. favicon computes the shodan-style
mmh3 hash (python base64.encodebytes chunking, signed int32) for tech
fingerprinting and the http.favicon.hash pivot, pinned by a golden test.
go only returns a conn to the idle pool when the body is read to EOF
before Close. the header-only and early-return scan paths closed an
unread body, leaking the conn and forcing a fresh dial each request.
route those close sites through httpx.DrainClose so the tuned pool from
phase 1 actually gets reused. body-read paths (scanner/io.ReadAll) are
left untouched.
