Compare commits

..

42 Commits

Author SHA1 Message Date
dependabot[bot] 86a22a553d chore(deps): bump github.com/projectdiscovery/nuclei/v3
Bumps [github.com/projectdiscovery/nuclei/v3](https://github.com/projectdiscovery/nuclei) from 3.9.0 to 3.10.0.
- [Release notes](https://github.com/projectdiscovery/nuclei/releases)
- [Commits](https://github.com/projectdiscovery/nuclei/compare/v3.9.0...v3.10.0)

---
updated-dependencies:
- dependency-name: github.com/projectdiscovery/nuclei/v3
  dependency-version: 3.10.0
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
2026-07-03 12:43:12 +00:00
Tigah 7ea1cd28d5 feat(modules): add traefik, nomad and portainer exposure modules (#249)
add recon modules for container and proxy control planes that answer without authentication: traefik serves its full routing config at /api/overview when the api is enabled, nomad dumps the agent config at /v1/agent/self when acls are disabled (403 otherwise), and portainer discloses its version and instance id at the public /api/status.
2026-07-02 17:00:50 -07:00
Tigah 9e8fb3a8d5 feat(modules): detect exposed ml platforms (#240)
add recon modules for unauthenticated ml platforms that leak cluster or deployment info without a key: h2o, mindsdb, and zenml.
2026-07-02 17:00:47 -07:00
Tigah abcc22de38 feat(modules): detect exposed rag and knowledge apps (#238)
add recon modules for unauthenticated rag/search apps that leak config or health without a key: perplexica, verba, onyx, and fastgpt.
2026-07-02 17:00:44 -07:00
Tigah 122fe5f94b feat(modules): detect exposed gpu inference servers (#236)
add recon modules for unauthenticated gpu serving stacks that leak model info without a key: triton, sglang, and torchserve.
2026-07-02 17:00:41 -07:00
Tigah e1c8e3948d feat(modules): detect exposed local llm runners (#235)
add recon modules for unauthenticated local inference servers that leak model inventory without a key: ollama, koboldcpp, text-generation-webui (oobabooga), and tabby.
2026-07-02 17:00:38 -07:00
Tigah 24bcab0a8f feat(modules): detect exposed openai-compatible llm servers (#233)
add recon modules for unauthenticated inference servers that expose an openai-style /v1/models inventory without a key: vllm, llama.cpp, lm studio, and infinity.
2026-07-02 17:00:35 -07:00
Tigah 3a289a3ac4 feat(modules): add clickhouse and dgraph exposure modules (#258)
add recon modules for self-hosted databases whose http interface is reachable
without credentials: clickhouse runs arbitrary sql because the default user has
an empty password, confirmed here by reading the server version through the
http interface, and the open-source dgraph alpha has no authentication so its
/health endpoint discloses the cluster while /query and /admin read and drop
all data; a clickhouse that requires a password returns 403 and an alpha behind
an authenticating proxy returns 401 and neither is flagged.
2026-07-02 13:22:24 -07:00
Tigah 6e022d56d9 feat(modules): detect exposed ai orchestration platforms (#242)
add recon modules for self-hosted agent builders and compute
orchestration. ray exposes an unauthenticated compute dashboard whose
job api allows code execution, skypilot exposes an open cloud and
kubernetes control plane when basic auth is disabled, dify flags a
console that allows open registration, and langflow fingerprints a
reachable instance over its public version api.
2026-07-02 13:22:21 -07:00
Tigah 1bb54393f1 feat(httpx): retry 429 and 503 with Retry-After backoff (#227)
the shared transport paced outbound requests but ignored a server
asking it to slow down, so a 429 or 503 came straight back as a failed
probe. back off and retry both through the one chokepoint every scanner
shares, honoring Retry-After (delta-seconds or http-date) and falling
back to capped exponential backoff.

gated behind -max-retries (default 2, 0 disables). bodyless GETs and
GetBody-backed requests replay safely; anything else is returned as-is.
2026-07-02 13:22:17 -07:00
dependabot[bot] f97c67695d chore(deps): bump actions/checkout from 4 to 7 (#267)
Bumps [actions/checkout](https://github.com/actions/checkout) from 4 to 7.
- [Release notes](https://github.com/actions/checkout/releases)
- [Changelog](https://github.com/actions/checkout/blob/main/CHANGELOG.md)
- [Commits](https://github.com/actions/checkout/compare/v4...v7)

---
updated-dependencies:
- dependency-name: actions/checkout
  dependency-version: '7'
  dependency-type: direct:production
  update-type: version-update:semver-major
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2026-07-02 13:16:52 -07:00
Tigah 7eb6603e77 feat(modules): add eureka and spring boot admin exposure modules (#263)
add two recon modules detecting anon-by-design service registries that leak the internal service map:

- eureka-registry-exposure (medium): probes /eureka/apps and matches the registry envelope markers apps__hashcode and versions__delta (the EurekaJacksonCodec formats them with a default '_' -> '__' replacement, so they appear verbatim in both the xml and json forms); eureka has no auth, so an open registry discloses every instance's internal hostname, ip and ports and accepts rogue registrations, while a spring-security-secured eureka returns 401; extracts the first instance ip from either form

- spring-boot-admin-exposure (medium): probes /instances and matches the registration/healthUrl/statusInfo shape; the codecentric server ships no security, so an open one discloses every registered app's internal management and health urls (a pivot to their actuators), while a secured one returns 401 or redirects to login; extracts the first internal health url

both modules carry hard-test coverage: eureka is proven on both its xml and json forms, with an N-1 trap (an apps__hashcode-less envelope stays quiet) and a prose trap (a page that merely mentions the word applications stays quiet); sba has an N-1 statusInfo trap; both have 401 secured-instance cases and cross false-positive guards both ways.
2026-07-02 12:56:34 -07:00
Tigah 12cab935ec feat(modules): add zookeeper and hadoop namenode exposure modules (#262)
add two recon modules detecting anon-by-design big-data control surfaces that leak operational topology:

- zookeeper-admin-exposure (medium): probes /commands/monitor on a zookeeper adminserver and matches the command/monitor envelope plus server_state; the adminserver has no auth on reads, so an open one discloses the version, ensemble role, connection counts and data size, while a firewalled or disabled adminserver is not reachable; extracts the version

- hadoop-namenode-exposure (medium): probes /jmx for the NameNodeInfo bean and matches the bean name plus LiveNodes/DeadNodes; the namenode jmx is unauthenticated unless kerberos spnego is set, so an open one discloses the hdfs version and every datanode internal hostname, while a kerberos-secured namenode returns 401; extracts the software version

trino/presto were dropped: /v1/info is ResourceSecurity(PUBLIC) so it answers on secured clusters too and would be a fingerprint, not an exposure.

both modules carry hard-test coverage with N-1 anchor traps (a non-monitor zookeeper command and a non-NameNodeInfo hadoop bean stay quiet), cross false-positive guards both ways and a secured-instance status case.
2026-07-02 12:56:31 -07:00
Tigah 79794638e4 feat(modules): add kafka ui and kafdrop exposure modules (#261)
add two recon modules detecting exposed kafka management consoles that ship without authentication by default:

- kafka-ui-exposure (high): probes /api/clusters on a kafbat/provectus kafka ui and matches the kafka-ui-specific defaultCluster/brokerCount keys plus the online/offline/initializing status enum; an open instance leaks the cluster topology and lets an unauthenticated session browse and produce records, while a login- or oauth-protected instance returns 401 and is not flagged

- kafdrop-exposure (high): probes / with an Accept: application/json header and matches the ClusterInfoVO summary/preferredReplicaPercent/brokers/topics shape; an open instance discloses broker hostnames, topics and messages, while an instance with the optional basic auth returns 401

both modules carry hard-test coverage with cross false-positive guards both ways, non-enum-status and missing-key negative cases, a wire-level check that the kafdrop module sends the Accept header, and 401 secured-instance cases
2026-07-02 12:56:29 -07:00
Tigah ed0cceb75a feat(modules): add n8n settings exposure module (#260)
add a recon module for an n8n instance that serves /rest/settings without
authentication; the endpoint exists so the frontend can load but it discloses
the webhook url structure, instance id, release channel, configured
authentication method and whether owner setup is still pending, which aids
reconnaissance and can indicate a claimable instance; it is unauthenticated by
design so this is reported as information disclosure.
2026-07-02 12:56:26 -07:00
Tigah 551c89ad0d feat(modules): add grafana anonymous access exposure module (#259)
add a recon module for a grafana with anonymous access enabled: by default
grafana requires a login and /api/search returns 401, but when anonymous
access is turned on the endpoint lists every dashboard without credentials,
exposing the internal metrics, hostnames and queries they contain, and the same
anonymous session can reach backend data sources through the data source proxy;
a grafana that requires login returns 401 and is not flagged.
2026-07-02 12:56:23 -07:00
Tigah 2c9bb4bad7 feat(modules): add jenkins and nifi exposure modules (#257)
add recon modules for self-hosted automation servers whose api is reachable
without credentials when misconfigured: jenkins /api/json answers when
anonymous read access is enabled and lists every job, view and node, and
apache nifi /nifi-api/flow/about answers when the instance runs without
security; each open instance reaches a script console or processor that runs
arbitrary code, while a jenkins without anonymous read returns 403 and a
secured nifi returns 401 and neither is flagged.
2026-07-02 12:56:21 -07:00
Tigah 19eb3b4292 feat(modules): add prometheus alertmanager exposure module (#256)
add a recon module for an unauthenticated prometheus alertmanager: the
/api/v2/status endpoint answers without credentials and returns config.original,
the full running configuration, which discloses receiver integrations and any
embedded credentials such as slack, pagerduty and webhook urls and smtp
passwords, along with the build version and cluster peers; an instance behind an
authenticating proxy returns 401 and is not flagged.
2026-07-02 12:56:18 -07:00
Tigah 2a49a72548 feat(modules): add selenium grid and selenoid exposure modules (#255)
add recon modules for self-hosted browser-automation grids that do not
authenticate their status endpoint: selenium grid /status discloses every
registered node with its os and browser slots, and selenoid /status discloses
the running sessions, capacity and browser images; each open grid also lets
anyone create a session that drives a real browser to arbitrary internal urls,
while a grid behind an authenticating proxy returns 401 and is not flagged.
2026-07-02 12:56:15 -07:00
Tigah 855a47c3cf feat(modules): add sidekiq, flower and rq-dashboard exposure modules (#254)
add recon modules for self-hosted background-job dashboards that ship no
authentication of their own and rely on the hosting application to protect
them: sidekiq web /sidekiq/stats discloses the redis server internals and the
job queue, celery flower /api/workers (reachable only when its api is
deliberately opened) discloses every worker's broker config and registered
tasks, and rq-dashboard /0/data/queues.json discloses the redis-backed queue
names and job counts; each open instance also allows killing, retrying or
deleting jobs, while a deployment protected by the application returns a
redirect or 401 and is not flagged.
2026-07-02 12:56:12 -07:00
Tigah b4dec11a52 feat(modules): add dagster and mage exposure modules (#253)
add recon modules for self-hosted data-orchestration webservers that ship no built-in authentication: dagster /server_info discloses the webserver and core versions, and mage /api/status discloses the scheduler status and server repository path; both reach an editor or graphql api on the same instance that can execute arbitrary code.
2026-07-02 12:56:09 -07:00
Tigah c69818ecd0 feat(modules): add caddy and envoy admin exposure modules (#252)
add recon modules for unauthenticated proxy and server admin interfaces that should be loopback-only: caddy /config/ returns the full running configuration and accepts a config-replacing post, and envoy /server_info exposes the build and command-line options of an admin interface that also offers config_dump and shutdown.
2026-07-02 12:56:07 -07:00
Tigah 64e3f540a9 feat(modules): add node-red flow api exposure module (#251)
add a recon module for an exposed node-red admin api; with adminAuth unset the /flows endpoint returns the flow configuration without a token and the open admin api allows flow deployment that runs arbitrary code through function and exec nodes, while a secured instance returns 401.
2026-07-02 12:56:04 -07:00
Tigah b9bca40a06 feat(modules): add concourse, woodpecker and gocd exposure modules (#250)
add recon modules that fingerprint ci/cd pipeline servers via their public version endpoints: concourse /api/v1/info, woodpecker /version (keyed on its source repository) and gocd /go/api/version (keyed on its gocd commit url); each answers without authentication and discloses the server version.
2026-07-02 12:56:01 -07:00
Tigah 16ffcc240b feat(modules): add loki, jaeger and zipkin exposure modules (#248)
add recon modules for unauthenticated observability backends that leak service topology and log data: loki serves its log query api when auth_enabled is false (401 otherwise), and jaeger and zipkin ship no authentication so a reachable instance exposes the service map and trace data.
2026-07-02 12:55:58 -07:00
Tigah 1b0677033c feat(modules): detect exposed one-api llm gateway (#246)
add a recon module for the one-api / new-api self-hosted llm gateway,
whose public status endpoint discloses the system name, version, and
quota configuration.
2026-07-02 12:55:55 -07:00
Tigah 84ae3a7168 feat(modules): detect exposed ai speech servers (#245)
add recon modules for self-hosted speech inference servers that are exposed without authentication: speaches (faster-whisper successor, speech-to-text and text-to-speech) keyed on its non-openai model task field, and xtts-api-server keyed on its get-folders endpoint that discloses local filesystem paths.
2026-07-02 12:55:52 -07:00
Tigah 754739bac7 feat(modules): detect exposed vector search engines (#244)
add recon modules for self-hosted vector and semantic search engines
reachable without auth: marqo, vespa, and meilisearch each allow
unauthenticated read and write of the indexed data, and their root or
version endpoints disclose the build.
2026-07-02 12:55:50 -07:00
Tigah fcccff5532 feat(modules): detect exposed ml experiment trackers (#243)
add recon modules for self-hosted training and experiment-tracking
platforms reachable without auth: mlflow, tensorboard, aim, and
determined disclose experiments, the artifact store, training run paths,
and cluster topology over unauthenticated apis.
2026-07-02 12:55:47 -07:00
Tigah a549102bb0 feat(modules): detect exposed ai image generation servers (#241)
add recon modules for self-hosted image generation servers reachable
without auth: comfyui, automatic1111, fooocus-api, and iopaint each
expose unauthenticated generation or editing and disclose the installed
models.
2026-07-02 12:55:44 -07:00
Tigah 7e3648e06d feat(modules): detect exposed data labeling tools (#239)
add recon modules for unauthenticated annotation servers that leak projects or config without a key: cvat and label studio.
2026-07-02 12:55:41 -07:00
Tigah 9e2965b777 feat(modules): detect exposed llm chat frontends (#237)
add recon modules for unauthenticated chat uis that leak config or allow open signup without a key: open webui, librechat, anythingllm, and nextchat.
2026-07-02 12:55:39 -07:00
Tigah 37a1a9e0ec feat(modules): detect exposed huggingface inference servers (#234)
add recon modules for unauthenticated text-generation-inference (tgi) and text-embeddings-inference (tei) servers that leak model info without a key.
2026-07-02 12:55:36 -07:00
Tigah 6575c2e5f7 fix(frameworks): stop false positives and version mis-extraction (#247)
a detector accuracy audit surfaced two classes of bug in the framework
detectors.

bare-brand header false positives: header-only signatures matched a
brand name as a substring across every header name and value, so a
detector fired on any response that merely referenced the brand (a
vendor cdn named in a link or csp value, a cookie sharing the prefix).
add an optional Header field to Signature that scopes a header-only
match to one named header's value, and apply it (or a structural
anchor) per detector:

- express: "Express" scoped to x-powered-by, was firing on an
  express_checkout cookie.
- flask: "Werkzeug" scoped to the server header.
- symfony: dropped the bare "symfony" word (symfony sets no such
  header, it fired on symfony.com links); the x-debug-token header is
  the marker.
- shopify: key on the x-shopify response headers instead of the bare
  "Shopify" word, which fired on a cdn.shopify.com link.
- remix: dropped the bare "remix"/"_remix" substrings that fired on a
  track_remix.mp3 asset; window.__remixContext is the definitive
  marker.
- spring boot: anchor the whitelabel title in its h1 tag context so a
  tutorial discussing the error does not fire.

the gin and fastapi detectors are removed: gin keyed on the
"gin-gonic" import-path string (appears in tutorials, never in a real
gin response) and fastapi on bare words matching the projects' doc
domains. neither framework advertises itself in a response header or a
non-prose body marker, so there is no clean passive signal to anchor
on.

version mis-extraction: drop the low-confidence ".*?" version
fallbacks (rails, django, laravel, spring), whose unbounded gap
grabbed the first version-shaped number after the framework word and
reported an unrelated asset's cache-buster when no real version was
present. let isValidVersionString accept a single integer so a bare
major such as drupal's "Drupal 10" is no longer rejected as "unknown".

each false positive and version bug is covered by a regression test.
2026-07-02 12:55:34 -07:00
dependabot[bot] 0caca05467 chore(deps): bump reviewdog/action-yamllint from 1.21.0 to 1.22.0 (#266)
Bumps [reviewdog/action-yamllint](https://github.com/reviewdog/action-yamllint) from 1.21.0 to 1.22.0.
- [Release notes](https://github.com/reviewdog/action-yamllint/releases)
- [Commits](https://github.com/reviewdog/action-yamllint/compare/v1.21.0...v1.22.0)

---
updated-dependencies:
- dependency-name: reviewdog/action-yamllint
  dependency-version: 1.22.0
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2026-07-02 12:55:31 -07:00
dependabot[bot] 61c4d33f5c chore(deps): bump reviewdog/action-markdownlint from 0.26.2 to 0.27.0 (#265)
Bumps [reviewdog/action-markdownlint](https://github.com/reviewdog/action-markdownlint) from 0.26.2 to 0.27.0.
- [Release notes](https://github.com/reviewdog/action-markdownlint/releases)
- [Commits](https://github.com/reviewdog/action-markdownlint/compare/v0.26.2...v0.27.0)

---
updated-dependencies:
- dependency-name: reviewdog/action-markdownlint
  dependency-version: 0.27.0
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2026-07-02 12:55:29 -07:00
dependabot[bot] 3971d37190 chore(deps): bump github.com/tidwall/gjson from 1.18.0 to 1.19.0 (#264)
Bumps [github.com/tidwall/gjson](https://github.com/tidwall/gjson) from 1.18.0 to 1.19.0.
- [Commits](https://github.com/tidwall/gjson/compare/v1.18.0...v1.19.0)

---
updated-dependencies:
- dependency-name: github.com/tidwall/gjson
  dependency-version: 1.19.0
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2026-07-02 12:55:26 -07:00
celeste 9d95c5b74c ci(claude-review): skip fork and dependabot PRs where auth is unavailable (#268)
pull_request runs from forks and dependabot don't receive OIDC tokens or
repo secrets, so claude-code-action can't authenticate and the check fails
on every external contributor PR. Guard the job to same-repo, non-dependabot
PRs so it skips (rather than fails) those cases.
2026-07-02 12:53:38 -07:00
jan ee4ade207d docs(readme): fix invalid discord shield invite link (#231)
* docs(readme): fix invalid discord shield invite link

- Replaced the invalid Discord invite link badge to the correct one
from the top of the README

* fix: discord invite in contributing.md
2026-07-02 12:35:03 -07:00
jan 96092dafab style: apply gofmt to source tree (#232)
Ran `gofmt -w .` accross the repo to fix formatting drift.

Mechanical `gofmt -w .` only. No functional or behavioural changes.

CONTRIBUTING.md requires gofmt-clean code; these files had slipped.
2026-06-25 18:19:17 -07:00
Tigah 39b333320e chore: migrate module path to github.com/vmfunc/sif (#194)
rename the go module path from github.com/dropalldatabases/sif to
github.com/vmfunc/sif across go.mod, all imports, the golangci exclude
list, release install docs and docs. pure string rename, no logic change.
2026-06-22 22:25:39 -07:00
celeste 7c0eb0bd4d test(scan): fix integration_test SQL arity after calibrate param (#230)
#180 added the calibrate bool to SQL but the integration-tagged test
(only built under -tags=integration, outside normal CI) still called the
4-arg form. pass false (no calibration) to restore behavior.
2026-06-22 22:10:46 -07:00
223 changed files with 8139 additions and 415 deletions
+10 -6
View File
@@ -12,11 +12,15 @@ on:
jobs:
claude-review:
# Optional: Filter by PR author
# if: |
# github.event.pull_request.user.login == 'external-contributor' ||
# github.event.pull_request.user.login == 'new-developer' ||
# github.event.pull_request.author_association == 'FIRST_TIME_CONTRIBUTOR'
# OIDC tokens and repo secrets are withheld from pull_request runs that
# originate from forks or dependabot, so the action cannot authenticate
# there and the check fails for every external PR. Skip those cases (the
# job simply does not run) instead of failing. Same-repo branch PRs still
# get reviewed. To review fork PRs too, switch the trigger to
# pull_request_target (has a security trade-off) rather than loosening this.
if: >-
github.event.pull_request.head.repo.full_name == github.repository &&
github.actor != 'dependabot[bot]'
runs-on: ubuntu-latest
permissions:
@@ -27,7 +31,7 @@ jobs:
steps:
- name: Checkout repository
uses: actions/checkout@v4
uses: actions/checkout@v7
with:
fetch-depth: 1
+1 -1
View File
@@ -26,7 +26,7 @@ jobs:
actions: read # Required for Claude to read CI results on PRs
steps:
- name: Checkout repository
uses: actions/checkout@v4
uses: actions/checkout@v7
with:
fetch-depth: 1
+1 -1
View File
@@ -16,7 +16,7 @@ jobs:
steps:
- uses: actions/checkout@v7
- name: markdownlint
uses: reviewdog/action-markdownlint@v0.26.2
uses: reviewdog/action-markdownlint@v0.27.0
with:
github_token: ${{ secrets.GITHUB_TOKEN }}
reporter: github-pr-review
+1 -1
View File
@@ -171,7 +171,7 @@ jobs:
**go install**
```bash
go install github.com/dropalldatabases/sif/cmd/sif@v${{ env.VERSION }}
go install github.com/vmfunc/sif/cmd/sif@v${{ env.VERSION }}
```
**binary download** - grab the right archive from below.
+1 -1
View File
@@ -17,7 +17,7 @@ jobs:
steps:
- uses: actions/checkout@v7
- name: yamllint
uses: reviewdog/action-yamllint@v1.21.0
uses: reviewdog/action-yamllint@v1.22.0
with:
github_token: ${{ secrets.GITHUB_TOKEN }}
reporter: github-pr-review
+1 -1
View File
@@ -36,7 +36,7 @@ linters:
check-blank: false
exclude-functions:
# log writes are best-effort
- github.com/dropalldatabases/sif/internal/logger.Write
- github.com/vmfunc/sif/internal/logger.Write
# Close on io.Closer is idiomatic best-effort
- (io.Closer).Close
- (*os.File).Close
+1 -1
View File
@@ -24,7 +24,7 @@ If you like the project, but don't have time to contribute, that's okay too! Her
## Reporting issues
If you believe you've found a bug, or you have a new feature to request, please hop on the [Discord server](https://discord.com/invite/sifcli) first to discuss it.
If you believe you've found a bug, or you have a new feature to request, please hop on the [Discord server](https://discord.gg/Yksy9J2BvE) first to discuss it.
This way, if it's an easy fix, we could help you solve it more quickly, and if it's a feature request we could workshop it together into something more mature.
When opening an issue, please use the search tool and make sure that the issue has not been discussed before. In the case of a bug report, run sif with the `-d/-debug` flag for full debug logs.
+1 -1
View File
@@ -375,7 +375,7 @@ go test ./...
join our discord for support, feature discussions, and pentesting tips:
[![discord](https://img.shields.io/badge/join%20our%20discord-5865F2?style=for-the-badge&logo=discord&logoColor=white)](https://discord.gg/sifcli)
[![discord](https://img.shields.io/badge/join%20our%20discord-5865F2?style=for-the-badge&logo=discord&logoColor=white)](https://discord.gg/Yksy9J2BvE)
## contributors
+5 -5
View File
@@ -17,13 +17,13 @@ import (
"os"
"github.com/charmbracelet/log"
"github.com/dropalldatabases/sif"
"github.com/dropalldatabases/sif/internal/config"
"github.com/dropalldatabases/sif/internal/patchnotes"
ver "github.com/dropalldatabases/sif/internal/version"
"github.com/vmfunc/sif"
"github.com/vmfunc/sif/internal/config"
"github.com/vmfunc/sif/internal/patchnotes"
ver "github.com/vmfunc/sif/internal/version"
// Register framework detectors
_ "github.com/dropalldatabases/sif/internal/scan/frameworks/detectors"
_ "github.com/vmfunc/sif/internal/scan/frameworks/detectors"
)
// version is stamped at release time via -ldflags "-X main.version=...";
+1 -1
View File
@@ -31,7 +31,7 @@ welcome to the sif documentation. sif is a modular pentesting toolkit designed t
```bash
# install
git clone https://github.com/dropalldatabases/sif.git && cd sif && make
git clone https://github.com/vmfunc/sif.git && cd sif && make
# basic scan
./sif -u https://example.com
+1 -1
View File
@@ -11,7 +11,7 @@ setting up a development environment for sif.
## clone and build
```bash
git clone https://github.com/dropalldatabases/sif.git
git clone https://github.com/vmfunc/sif.git
cd sif
make
```
+1 -1
View File
@@ -39,7 +39,7 @@ download `sif-windows-amd64.exe` from releases and add to your PATH.
requires go 1.25+
```bash
git clone https://github.com/dropalldatabases/sif.git
git clone https://github.com/vmfunc/sif.git
cd sif
make
```
+15 -21
View File
@@ -1,4 +1,4 @@
module github.com/dropalldatabases/sif
module github.com/vmfunc/sif
go 1.25.7
@@ -10,11 +10,11 @@ require (
github.com/gocolly/colly/v2 v2.3.0
github.com/likexian/whois v1.15.7
github.com/projectdiscovery/goflags v0.1.74
github.com/projectdiscovery/nuclei/v3 v3.9.0
github.com/projectdiscovery/nuclei/v3 v3.10.0
github.com/projectdiscovery/retryabledns v1.0.115
github.com/projectdiscovery/utils v0.11.1
github.com/rocketlaunchr/google-search v1.1.6
github.com/tidwall/gjson v1.18.0
github.com/tidwall/gjson v1.19.0
github.com/twmb/murmur3 v1.1.8
golang.org/x/net v0.56.0
golang.org/x/time v0.15.0
@@ -35,14 +35,12 @@ require (
github.com/Azure/go-ntlmssp v0.1.1 // indirect
github.com/AzureAD/microsoft-authentication-library-for-go v1.4.2 // indirect
github.com/FalconOpsLLC/goexec v0.3.0 // indirect
github.com/Masterminds/semver/v3 v3.4.0 // indirect
github.com/Masterminds/semver/v3 v3.5.0 // indirect
github.com/Microsoft/go-winio v0.6.2 // indirect
github.com/Mzack9999/gcache v0.0.0-20230410081825-519e28eab057 // indirect
github.com/Mzack9999/go-http-digest-auth-client v0.6.1-0.20220414142836-eb8883508809 // indirect
github.com/Mzack9999/go-rsync v0.0.0-20250821180103-81ffa574ef4d // indirect
github.com/Mzack9999/goimpacket v0.0.0-20260422121140-7085336a0415 // indirect
github.com/Mzack9999/goja v0.0.0-20250507184235-e46100e9c697 // indirect
github.com/Mzack9999/goja_nodejs v0.0.0-20250507184139-66bcbf65c883 // indirect
github.com/ProtonMail/go-crypto v1.1.6 // indirect
github.com/PuerkitoBio/goquery v1.11.0 // indirect
github.com/RedTeamPentesting/adauth v0.5.4-0.20260511073005-3d18e8a5a687 // indirect
@@ -85,7 +83,6 @@ require (
github.com/bahlo/generic-list-go v0.2.0 // indirect
github.com/bits-and-blooms/bitset v1.24.4 // indirect
github.com/bits-and-blooms/bloom/v3 v3.5.0 // indirect
github.com/bluele/gcache v0.0.2 // indirect
github.com/bodgit/plumbing v1.3.0 // indirect
github.com/bodgit/sevenzip v1.6.1 // indirect
github.com/bodgit/windows v1.0.1 // indirect
@@ -120,6 +117,7 @@ require (
github.com/ditashi/jsbeautifier-go v0.0.0-20141206144643-2520a8026a9c // indirect
github.com/djherbis/times v1.6.0 // indirect
github.com/dlclark/regexp2 v1.11.5 // indirect
github.com/dlclark/regexp2/v2 v2.2.1 // indirect
github.com/docker/go-connections v0.7.0 // indirect
github.com/docker/go-units v0.5.0 // indirect
github.com/dsnet/compress v0.0.2-0.20230904184137-39efe44ab707 // indirect
@@ -173,7 +171,6 @@ require (
github.com/golang/protobuf v1.5.4 // indirect
github.com/golang/snappy v1.0.0 // indirect
github.com/google/certificate-transparency-go v1.3.2 // indirect
github.com/google/go-github v17.0.0+incompatible // indirect
github.com/google/go-github/v30 v30.1.0 // indirect
github.com/google/go-querystring v1.1.0 // indirect
github.com/google/pprof v0.0.0-20240727154555-813a5fbdbec8 // indirect
@@ -194,7 +191,6 @@ require (
github.com/imdario/mergo v0.3.16 // indirect
github.com/indece-official/go-ebcdic v1.2.0 // indirect
github.com/invopop/jsonschema v0.13.0 // indirect
github.com/invopop/yaml v0.3.1 // indirect
github.com/itchyny/gojq v0.12.17 // indirect
github.com/itchyny/timefmt-go v0.1.6 // indirect
github.com/jbenet/go-context v0.0.0-20150711004518-d14ea06fba99 // indirect
@@ -218,8 +214,6 @@ require (
github.com/klauspost/cpuid/v2 v2.3.0 // indirect
github.com/klauspost/pgzip v1.2.6 // indirect
github.com/kylelemons/godebug v1.1.0 // indirect
github.com/labstack/echo/v4 v4.13.4 // indirect
github.com/labstack/gommon v0.4.2 // indirect
github.com/leodido/go-urn v1.4.0 // indirect
github.com/leslie-qiwa/flat v0.0.0-20230424180412-f9d1cf014baa // indirect
github.com/lib/pq v1.11.2 // indirect
@@ -276,17 +270,19 @@ require (
github.com/praetorian-inc/fingerprintx v1.1.15 // indirect
github.com/projectdiscovery/asnmap v1.1.1 // indirect
github.com/projectdiscovery/blackrock v0.0.1 // indirect
github.com/projectdiscovery/cdncheck v1.2.39 // indirect
github.com/projectdiscovery/cdncheck v1.2.42 // indirect
github.com/projectdiscovery/clistats v0.1.4 // indirect
github.com/projectdiscovery/dsl v0.8.19 // indirect
github.com/projectdiscovery/fastdialer v0.5.10 // indirect
github.com/projectdiscovery/dsl v0.8.20 // indirect
github.com/projectdiscovery/fastdialer v0.5.11 // indirect
github.com/projectdiscovery/fasttemplate v0.0.2 // indirect
github.com/projectdiscovery/freeport v0.0.7 // indirect
github.com/projectdiscovery/gcache v0.0.0-20241015120333-12546c6e3f4c // indirect
github.com/projectdiscovery/go-smb2 v0.0.0-20240129202741-052cc450c6cb // indirect
github.com/projectdiscovery/gologger v1.1.70 // indirect
github.com/projectdiscovery/goja v0.0.0-20260618133720-acb73e419534 // indirect
github.com/projectdiscovery/goja_nodejs v0.0.0-20260618132410-8519f75f703d // indirect
github.com/projectdiscovery/gologger v1.1.71 // indirect
github.com/projectdiscovery/gostruct v0.0.2 // indirect
github.com/projectdiscovery/govaluate v0.0.0-20260504230327-80320480bb6e // indirect
github.com/projectdiscovery/govaluate v0.0.0-20260615100919-5ee2581bbf7e // indirect
github.com/projectdiscovery/gozero v0.1.1-0.20260530071156-fa1dad563d76 // indirect
github.com/projectdiscovery/hmap v0.0.101 // indirect
github.com/projectdiscovery/httpx v1.9.0 // indirect
@@ -295,16 +291,16 @@ require (
github.com/projectdiscovery/machineid v0.0.0-20250715113114-c77eb3567582 // indirect
github.com/projectdiscovery/mapcidr v1.1.97 // indirect
github.com/projectdiscovery/n3iwf v0.0.0-20230523120440-b8cd232ff1f5 // indirect
github.com/projectdiscovery/networkpolicy v0.1.40 // indirect
github.com/projectdiscovery/networkpolicy v0.1.41 // indirect
github.com/projectdiscovery/ratelimit v0.0.88 // indirect
github.com/projectdiscovery/rawhttp v0.1.90 // indirect
github.com/projectdiscovery/rdap v0.9.1-0.20221108103045-9865884d1917 // indirect
github.com/projectdiscovery/retryablehttp-go v1.3.14 // indirect
github.com/projectdiscovery/retryablehttp-go v1.3.16 // indirect
github.com/projectdiscovery/sarif v0.1.0 // indirect
github.com/projectdiscovery/tlsx v1.2.2 // indirect
github.com/projectdiscovery/uncover v1.2.1 // indirect
github.com/projectdiscovery/useragent v0.0.108 // indirect
github.com/projectdiscovery/wappalyzergo v0.2.84 // indirect
github.com/projectdiscovery/wappalyzergo v0.2.87 // indirect
github.com/projectdiscovery/yamldoc-go v1.0.6 // indirect
github.com/redis/go-redis/v9 v9.11.0 // indirect
github.com/refraction-networking/utls v1.8.2 // indirect
@@ -345,7 +341,6 @@ require (
github.com/ugorji/go/codec v1.2.11 // indirect
github.com/ulikunitz/xz v0.5.15 // indirect
github.com/valyala/bytebufferpool v1.0.0 // indirect
github.com/valyala/fasttemplate v1.2.2 // indirect
github.com/vmihailenco/bufpool v0.1.11 // indirect
github.com/vmihailenco/msgpack/v5 v5.3.4 // indirect
github.com/vmihailenco/tagparser v0.1.2 // indirect
@@ -401,7 +396,6 @@ require (
gopkg.in/alecthomas/kingpin.v2 v2.2.6 // indirect
gopkg.in/corvus-ch/zbase32.v1 v1.0.0 // indirect
gopkg.in/warnings.v0 v0.1.2 // indirect
gopkg.in/yaml.v2 v2.4.0 // indirect
mellium.im/sasl v0.3.2 // indirect
moul.io/http2curl v1.0.0 // indirect
software.sslmate.com/src/go-pkcs12 v0.7.0 // indirect
+36 -43
View File
@@ -70,13 +70,13 @@ github.com/AzureAD/microsoft-authentication-extensions-for-go/cache v0.1.1/go.mo
github.com/AzureAD/microsoft-authentication-library-for-go v1.4.2 h1:oygO0locgZJe7PpYPXT5A29ZkwJaPqcva7BVeemZOZs=
github.com/AzureAD/microsoft-authentication-library-for-go v1.4.2/go.mod h1:wP83P5OoQ5p6ip3ScPr0BAq0BvuPAvacpEuSzyouqAI=
github.com/BurntSushi/toml v0.3.1/go.mod h1:xHWCNGjB5oqiDr8zfno3MHue2Ht5sIBksp03qcyfWMU=
github.com/BurntSushi/toml v1.3.2 h1:o7IhLm0Msx3BaB+n3Ag7L8EVlByGnpq14C4YWiu/gL8=
github.com/BurntSushi/toml v1.3.2/go.mod h1:CxXYINrC8qIiEnFrOxCa7Jy5BFHlXnUU2pbicEuybxQ=
github.com/BurntSushi/toml v1.4.0 h1:kuoIxZQy2WRRk1pttg9asf+WVv6tWQuBNVmK8+nqPr0=
github.com/BurntSushi/toml v1.4.0/go.mod h1:ukJfTF/6rtPPRCnwkur4qwRxa8vTRFBF0uk2lLoLwho=
github.com/BurntSushi/xgb v0.0.0-20160522181843-27f122750802/go.mod h1:IVnqGOEym/WlBOVXweHU+Q+/VP0lqqI8lqeDx9IjBqo=
github.com/FalconOpsLLC/goexec v0.3.0 h1:ryLMkrGT6asnkqdc5rFMNOSTYdMH/iCfyEuwu0D6ZhA=
github.com/FalconOpsLLC/goexec v0.3.0/go.mod h1:kiyxVbmFCGbbwXRyZmOSKlOy7PiK+JH2gq07Ztag/k8=
github.com/Masterminds/semver/v3 v3.4.0 h1:Zog+i5UMtVoCU8oKka5P7i9q9HgrJeGzI9SA1Xbatp0=
github.com/Masterminds/semver/v3 v3.4.0/go.mod h1:4V+yj/TJE1HU9XfppCwVMZq3I84lprf4nC11bSS5beM=
github.com/Masterminds/semver/v3 v3.5.0 h1:kQceYJfbupGfZOKZQg0kou0DgAKhzDg2NZPAwZ/2OOE=
github.com/Masterminds/semver/v3 v3.5.0/go.mod h1:4V+yj/TJE1HU9XfppCwVMZq3I84lprf4nC11bSS5beM=
github.com/Microsoft/go-winio v0.5.2/go.mod h1:WpS1mjBmmwHBEWmogvA2mj8546UReBk4v8QkMxJ6pZY=
github.com/Microsoft/go-winio v0.6.2 h1:F2VQgta7ecxGYO8k3ZZz3RS8fVIXVxONVUPlNERoyfY=
github.com/Microsoft/go-winio v0.6.2/go.mod h1:yd8OoFMLzJbo9gZq8j5qaps8bJ9aShtEA8Ipt1oGCvU=
@@ -88,10 +88,6 @@ github.com/Mzack9999/go-rsync v0.0.0-20250821180103-81ffa574ef4d h1:DofPB5AcjTnO
github.com/Mzack9999/go-rsync v0.0.0-20250821180103-81ffa574ef4d/go.mod h1:uzdh/m6XQJI7qRvufeBPDa+lj5SVCJO8B9eLxTbtI5U=
github.com/Mzack9999/goimpacket v0.0.0-20260422121140-7085336a0415 h1:lpSZPcbowbxvKFaFvE1reLTCStezWXcRVk0zzBtUatg=
github.com/Mzack9999/goimpacket v0.0.0-20260422121140-7085336a0415/go.mod h1:Wvb2f1Aq6NVL4Fza/dPNxv6/canpeizpgmiTCBGMD50=
github.com/Mzack9999/goja v0.0.0-20250507184235-e46100e9c697 h1:54I+OF5vS4a/rxnUrN5J3hi0VEYKcrTlpc8JosDyP+c=
github.com/Mzack9999/goja v0.0.0-20250507184235-e46100e9c697/go.mod h1:yNqYRqxYkSROY1J+LX+A0tOSA/6soXQs5m8hZSqYBac=
github.com/Mzack9999/goja_nodejs v0.0.0-20250507184139-66bcbf65c883 h1:+Is1AS20q3naP+qJophNpxuvx1daFOx9C0kLIuI0GVk=
github.com/Mzack9999/goja_nodejs v0.0.0-20250507184139-66bcbf65c883/go.mod h1:K+FhM7iKGKtalkeXGEviafPPwyVjDv1a/ehomabLF2w=
github.com/Nvveen/Gotty v0.0.0-20120604004816-cd527374f1e5 h1:TngWCqHvy9oXAN6lEVMRuU21PR1EtLVZJmdB18Gu3Rw=
github.com/Nvveen/Gotty v0.0.0-20120604004816-cd527374f1e5/go.mod h1:lmUJ/7eu/Q8D7ML55dXQrVaamCz2vxCfdQBasLZfHKk=
github.com/OneOfOne/xxhash v1.2.2 h1:KMrpdQIwFcEqXDklaen+P1axHaj9BSKzvpUUfnHldSE=
@@ -221,8 +217,6 @@ github.com/bits-and-blooms/bitset v1.24.4 h1:95H15Og1clikBrKr/DuzMXkQzECs1M6hhoG
github.com/bits-and-blooms/bitset v1.24.4/go.mod h1:7hO7Gc7Pp1vODcmWvKMRA9BNmbv6a/7QIWpPxHddWR8=
github.com/bits-and-blooms/bloom/v3 v3.5.0 h1:AKDvi1V3xJCmSR6QhcBfHbCN4Vf8FfxeWkMNQfmAGhY=
github.com/bits-and-blooms/bloom/v3 v3.5.0/go.mod h1:Y8vrn7nk1tPIlmLtW2ZPV+W7StdVMor6bC1xgpjMZFs=
github.com/bluele/gcache v0.0.2 h1:WcbfdXICg7G/DGBh1PFfcirkWOQV+v077yF1pSy3DGw=
github.com/bluele/gcache v0.0.2/go.mod h1:m15KV+ECjptwSPxKhOhQoAFQVtUFjTVkc3H8o0t/fp0=
github.com/bodgit/plumbing v1.3.0 h1:pf9Itz1JOQgn7vEOE7v7nlEfBykYqvUYioC61TwWCFU=
github.com/bodgit/plumbing v1.3.0/go.mod h1:JOTb4XiRu5xfnmdnDJo6GmSbSbtSyufrsyZFByMtKEs=
github.com/bodgit/sevenzip v1.6.1 h1:kikg2pUMYC9ljU7W9SaqHXhym5HyKm8/M/jd31fYan4=
@@ -330,6 +324,8 @@ github.com/djherbis/times v1.6.0/go.mod h1:gOHeRAz2h+VJNZ5Gmc/o7iD9k4wW7NMVqieYC
github.com/dlclark/regexp2 v1.4.0/go.mod h1:2pZnwuY/m+8K6iRw6wQdMtk+rH5tNGR1i55kozfMjCc=
github.com/dlclark/regexp2 v1.11.5 h1:Q/sSnsKerHeCkc/jSTNq1oCm7KiVgUMZRDUoRu0JQZQ=
github.com/dlclark/regexp2 v1.11.5/go.mod h1:DHkYz0B9wPfa6wondMfaivmHpzrQ3v9q8cnmRbL6yW8=
github.com/dlclark/regexp2/v2 v2.2.1 h1:mf4KkFUj0gJuarK8P+LgiS+Lit7m9N1yAwEfPbee7R0=
github.com/dlclark/regexp2/v2 v2.2.1/go.mod h1:avUrQvPaLz2DrFNHJF0taWAFFX2C1GMSSoeiqFjcBmU=
github.com/docker/cli v29.2.0+incompatible h1:9oBd9+YM7rxjZLfyMGxjraKBKE4/nVyvVfN4qNl9XRM=
github.com/docker/cli v29.2.0+incompatible/go.mod h1:JLrzqnKDaYBop7H2jaqPtU4hHvMKP+vjCwu2uszcLI8=
github.com/docker/go-connections v0.7.0 h1:6SsRfJddP22WMrCkj19x9WKjEDTB+ahsdiGYf0mN39c=
@@ -463,6 +459,8 @@ github.com/gobwas/ws v1.4.0 h1:CTaoG1tojrh4ucGPcoJFiAQUAsEWekEWvLy7GsVNqGs=
github.com/gobwas/ws v1.4.0/go.mod h1:G3gNqMNtPppf5XUz7O4shetPpcZ1VJ7zt18dlUeakrc=
github.com/goccy/go-json v0.10.5 h1:Fq85nIqj+gXn/S5ahsiTlK3TmC85qgirsdTP/+DeaC4=
github.com/goccy/go-json v0.10.5/go.mod h1:oq7eo15ShAhp70Anwd5lgX2pLfOS3QCiwU/PULtXL6M=
github.com/goccy/go-yaml v1.19.2 h1:PmFC1S6h8ljIz6gMRBopkjP1TVT7xuwrButHID66PoM=
github.com/goccy/go-yaml v1.19.2/go.mod h1:XBurs7gK8ATbW4ZPGKgcbrY1Br56PdM69F7LkFRi1kA=
github.com/gocolly/colly v1.2.0/go.mod h1:Hof5T3ZswNVsOHYmba1u03W65HDWgpV5HifSuueE0EA=
github.com/gocolly/colly/v2 v2.1.0/go.mod h1:I2MuhsLjQ+Ex+IzK3afNS8/1qP3AedHOusRPcRdC5o0=
github.com/gocolly/colly/v2 v2.3.0 h1:HSFh0ckbgVd2CSGRE+Y/iA4goUhGROJwyQDCMXGFBWM=
@@ -534,7 +532,6 @@ github.com/google/go-cmp v0.5.9/go.mod h1:17dUlkBOakJ0+DkrSSNjCkIjxS6bF9zb3elmeN
github.com/google/go-cmp v0.6.0/go.mod h1:17dUlkBOakJ0+DkrSSNjCkIjxS6bF9zb3elmeNGIjoY=
github.com/google/go-cmp v0.7.0 h1:wk8382ETsv4JYUZwIsn6YpYiWiBsYLSJiTsyBybVuN8=
github.com/google/go-cmp v0.7.0/go.mod h1:pXiqmnSA92OHEEa9HXL2W4E7lf9JzCmGVUdgjX3N/iU=
github.com/google/go-github v17.0.0+incompatible h1:N0LgJ1j65A7kfXrZnUDaYCs/Sf4rEjNlfyDHW9dolSY=
github.com/google/go-github v17.0.0+incompatible/go.mod h1:zLgOLi98H3fifZn+44m+umXrS52loVEgC2AApnigrVQ=
github.com/google/go-github/v30 v30.1.0 h1:VLDx+UolQICEOKu2m4uAoMti1SxuEBAl7RSEG16L+Oo=
github.com/google/go-github/v30 v30.1.0/go.mod h1:n8jBpHl45a/rlBUtRJMOG4GhNADUQFEufcolZ95JfU8=
@@ -614,8 +611,6 @@ github.com/indece-official/go-ebcdic v1.2.0 h1:nKCubkNoXrGvBp3MSYuplOQnhANCDEY51
github.com/indece-official/go-ebcdic v1.2.0/go.mod h1:RBddVJt0Ks0eDLRG5dhPwBDRiTNA7n+yv0dVFpSs46Q=
github.com/invopop/jsonschema v0.13.0 h1:KvpoAJWEjR3uD9Kbm2HWJmqsEaHt8lBUpd0qHcIi21E=
github.com/invopop/jsonschema v0.13.0/go.mod h1:ffZ5Km5SWWRAIN6wbDXItl95euhFz2uON45H2qjYt+0=
github.com/invopop/yaml v0.3.1 h1:f0+ZpmhfBSS4MhG+4HYseMdJhoeeopbSKbq5Rpeelso=
github.com/invopop/yaml v0.3.1/go.mod h1:PMOp3nn4/12yEZUFfmOuNHJsZToEEOwoWsT+D81KkeA=
github.com/itchyny/gojq v0.12.17 h1:8av8eGduDb5+rvEdaOO+zQUjA04MS0m3Ps8HiD+fceg=
github.com/itchyny/gojq v0.12.17/go.mod h1:WBrEMkgAfAGO1LUcGOckBl5O726KPp+OlkKug0I/FEY=
github.com/itchyny/timefmt-go v0.1.6 h1:ia3s54iciXDdzWzwaVKXZPbiXzxxnv1SPGFfM/myJ5Q=
@@ -693,10 +688,6 @@ github.com/kr/text v0.2.0 h1:5Nx0Ya0ZqY2ygV366QzturHI13Jq95ApcVaJBhpS+AY=
github.com/kr/text v0.2.0/go.mod h1:eLer722TekiGuMkidMxC/pM04lWEeraHUUmBw8l2grE=
github.com/kylelemons/godebug v1.1.0 h1:RPNrshWIDI6G2gRW9EHilWtl7Z6Sb1BR0xunSBf0SNc=
github.com/kylelemons/godebug v1.1.0/go.mod h1:9/0rRGxNHcop5bhtWyNeEfOS8JIWk580+fNqagV/RAw=
github.com/labstack/echo/v4 v4.13.4 h1:oTZZW+T3s9gAu5L8vmzihV7/lkXGZuITzTQkTEhcXEA=
github.com/labstack/echo/v4 v4.13.4/go.mod h1:g63b33BZ5vZzcIUF8AtRH40DrTlXnx4UMC8rBdndmjQ=
github.com/labstack/gommon v0.4.2 h1:F8qTUNXgG1+6WQmqoUWnz8WiEU60mXVVw0P4ht1WRA0=
github.com/labstack/gommon v0.4.2/go.mod h1:QlUFxVM+SNXhDL/Z7YhocGIBYOiwB0mXm1+1bAPHPyU=
github.com/ledongthuc/pdf v0.0.0-20220302134840-0c2507a12d80/go.mod h1:imJHygn/1yfhB7XSJJKlFZKl/J+dCPAknuiaGOshXAs=
github.com/leodido/go-urn v1.4.0 h1:WT9HwE9SGECu3lg4d/dIA+jxlljEa1/ffXKmRjqdmIQ=
github.com/leodido/go-urn v1.4.0/go.mod h1:bvxc+MVxLKB4z00jd1z+Dvzr47oO32F/QSNjSBOlFxI=
@@ -825,8 +816,8 @@ github.com/opencontainers/go-digest v1.0.0 h1:apOUWs51W5PlhuyGyz9FCeeBIOUDA/6nW8
github.com/opencontainers/go-digest v1.0.0/go.mod h1:0JzlMkj0TRzQZfJkVvzbP0HBR3IKzErnv2BNG4W4MAM=
github.com/opencontainers/image-spec v1.1.1 h1:y0fUlFfIZhPF1W537XOLg0/fcx6zcHCJwooC2xJA040=
github.com/opencontainers/image-spec v1.1.1/go.mod h1:qpqAh3Dmcf36wStyyWU+kCeDgrGnAve2nCC8+7h8Q0M=
github.com/opencontainers/runc v1.2.8 h1:RnEICeDReapbZ5lZEgHvj7E9Q3Eex9toYmaGBsbvU5Q=
github.com/opencontainers/runc v1.2.8/go.mod h1:cC0YkmZcuvr+rtBZ6T7NBoVbMGNAdLa/21vIElJDOzI=
github.com/opencontainers/runc v1.3.6 h1:SLGIymCtsk80iNPWgbc8dtjI30r+5mTVV+4dN8/17Sk=
github.com/opencontainers/runc v1.3.6/go.mod h1:o1wyv76EDlTkcf0KTFgN8bMWLPvgF/HfX709lDv+rr4=
github.com/orisano/pixelmatch v0.0.0-20220722002657-fb0b55479cde/go.mod h1:nZgzbfBr3hhjoZnS66nKrHmduYNpc34ny7RK4z5/HM0=
github.com/ory/dockertest/v3 v3.12.0 h1:3oV9d0sDzlSQfHtIaB5k6ghUCVMVLpAY8hwrqoCyRCw=
github.com/ory/dockertest/v3 v3.12.0/go.mod h1:aKNDTva3cp8dwOWwb9cWuX84aH5akkxXRvO7KCwWVjE=
@@ -855,14 +846,14 @@ github.com/projectdiscovery/asnmap v1.1.1 h1:ImJiKIaACOT7HPx4Pabb5dksolzaFYsD1kI
github.com/projectdiscovery/asnmap v1.1.1/go.mod h1:QT7jt9nQanj+Ucjr9BqGr1Q2veCCKSAVyUzLXfEcQ60=
github.com/projectdiscovery/blackrock v0.0.1 h1:lHQqhaaEFjgf5WkuItbpeCZv2DUIE45k0VbGJyft6LQ=
github.com/projectdiscovery/blackrock v0.0.1/go.mod h1:ANUtjDfaVrqB453bzToU+YB4cUbvBRpLvEwoWIwlTss=
github.com/projectdiscovery/cdncheck v1.2.39 h1:gNE2dyaK+ZqEdEWyVUFlq8PvromEhSxamhsmFZR2Voc=
github.com/projectdiscovery/cdncheck v1.2.39/go.mod h1:9oE9KKxCSHNvUf0UaMeqqUwWpC38FkNaTll0ScIBT3w=
github.com/projectdiscovery/cdncheck v1.2.42 h1:Y1Q9MPq7uuv25+aGlgjA5nToOcsk+9gNEKjicyhIwQI=
github.com/projectdiscovery/cdncheck v1.2.42/go.mod h1:9oE9KKxCSHNvUf0UaMeqqUwWpC38FkNaTll0ScIBT3w=
github.com/projectdiscovery/clistats v0.1.4 h1:kDnXoNxIdOvQElOF7k2Mt6XosGa5GbMKPtRXdPHMVzU=
github.com/projectdiscovery/clistats v0.1.4/go.mod h1:hjJYNcUubk9T3cuFvA+JkLhZGjzYW50fkC48dqUAtbU=
github.com/projectdiscovery/dsl v0.8.19 h1:qA5OFJMfghSCjKqS4AdsEtnur/SoriXDw3geE7+mReU=
github.com/projectdiscovery/dsl v0.8.19/go.mod h1:Twk93q7fxQ43v/8nR+0TJV8/eFTdBAC5tIXe3qzua9Y=
github.com/projectdiscovery/fastdialer v0.5.10 h1:dB9MSu4cSo22qne4pHiK9iYSxfOgpwlKB6zfOHvz3RI=
github.com/projectdiscovery/fastdialer v0.5.10/go.mod h1:W1ZkULr9mMR6i0oRFTztANnpVyEEzPUovK8sUM4eAw8=
github.com/projectdiscovery/dsl v0.8.20 h1:CxWcKuoHFpOSS1kzqnbJuK5No/6qoRG8IzNDMnZ6c/M=
github.com/projectdiscovery/dsl v0.8.20/go.mod h1:e1oHi7mxAxF+UhBhD5gOk90Ga6LQqvFea2voMO1E5D0=
github.com/projectdiscovery/fastdialer v0.5.11 h1:eI7jfwz0i73Ot1cowIBezQLxbg0i6INdAsFGJjfwPa0=
github.com/projectdiscovery/fastdialer v0.5.11/go.mod h1:W1ZkULr9mMR6i0oRFTztANnpVyEEzPUovK8sUM4eAw8=
github.com/projectdiscovery/fasttemplate v0.0.2 h1:h2cISk5xDhlJEinlBQS6RRx0vOlOirB2y3Yu4PJzpiA=
github.com/projectdiscovery/fasttemplate v0.0.2/go.mod h1:XYWWVMxnItd+r0GbjA1GCsUopMw1/XusuQxdyAIHMCw=
github.com/projectdiscovery/freeport v0.0.7 h1:Q6uXo/j8SaV/GlAHkEYQi8WQoPXyJWxyspx+aFmz9Qk=
@@ -873,12 +864,16 @@ github.com/projectdiscovery/go-smb2 v0.0.0-20240129202741-052cc450c6cb h1:rutG90
github.com/projectdiscovery/go-smb2 v0.0.0-20240129202741-052cc450c6cb/go.mod h1:FLjF1DmZ+POoGEiIQdWuYVwS++C/GwpX8YaCsTSm1RY=
github.com/projectdiscovery/goflags v0.1.74 h1:n85uTRj5qMosm0PFBfsvOL24I7TdWRcWq/1GynhXS7c=
github.com/projectdiscovery/goflags v0.1.74/go.mod h1:UMc9/7dFz2oln+10tv6cy+7WZKTHf9UGhaNkF95emh4=
github.com/projectdiscovery/gologger v1.1.70 h1:A1ZAsUJfRUXO6qqwTwyTWXLVlBrVu/Gpi1zzL1hg5LY=
github.com/projectdiscovery/gologger v1.1.70/go.mod h1:kpLKNafZWRN9P7WpJYtIOY/XvY/v41GDdU9NzICdKmo=
github.com/projectdiscovery/goja v0.0.0-20260618133720-acb73e419534 h1:hYd1zQA/dxO2ASyQ6Re73TcJkW1LjLQvt4+86Hxefz8=
github.com/projectdiscovery/goja v0.0.0-20260618133720-acb73e419534/go.mod h1:SO0AP+uKfYeeoR6jyVH/PKRYJE/f5gJrPMAM00iGEMc=
github.com/projectdiscovery/goja_nodejs v0.0.0-20260618132410-8519f75f703d h1:fqqH9LHpN2WDz9QuxFrhKNxXSRtzk+Sa6jAhbB7tXcQ=
github.com/projectdiscovery/goja_nodejs v0.0.0-20260618132410-8519f75f703d/go.mod h1:Ezmbgdaw4EunGGBU4MQViLoGMJc37LA3ip55YV3KeRI=
github.com/projectdiscovery/gologger v1.1.71 h1:IYU4mw9viKdSzMTIGVpYuw1Gtg7QIHIStqAQgeNXcBQ=
github.com/projectdiscovery/gologger v1.1.71/go.mod h1:mJwODZcFDg70ihINpOvZevmBtgvpP8H9/l8Y+OPhZPY=
github.com/projectdiscovery/gostruct v0.0.2 h1:s8gP8ApugGM4go1pA+sVlPDXaWqNP5BBDDSv7VEdG1M=
github.com/projectdiscovery/gostruct v0.0.2/go.mod h1:H86peL4HKwMXcQQtEa6lmC8FuD9XFt6gkNR0B/Mu5PE=
github.com/projectdiscovery/govaluate v0.0.0-20260504230327-80320480bb6e h1:o+ulEIaC2+9V2Ezr6mI5xEhKWsf0V/+FUQIS723Aj6U=
github.com/projectdiscovery/govaluate v0.0.0-20260504230327-80320480bb6e/go.mod h1:xH7bPwHxUlz1yx9UlVeTF+UVCUaKhTnZgaxHb5z362E=
github.com/projectdiscovery/govaluate v0.0.0-20260615100919-5ee2581bbf7e h1:vxzgQlz2Cy/YvizYDQx9OhucBcmBotfDhbQ4yCY2vfA=
github.com/projectdiscovery/govaluate v0.0.0-20260615100919-5ee2581bbf7e/go.mod h1:xH7bPwHxUlz1yx9UlVeTF+UVCUaKhTnZgaxHb5z362E=
github.com/projectdiscovery/gozero v0.1.1-0.20260530071156-fa1dad563d76 h1:AN70bbi6BBs7KpIM9w0LxygUN7uzT/oH+owDIQ+Fz/k=
github.com/projectdiscovery/gozero v0.1.1-0.20260530071156-fa1dad563d76/go.mod h1:cWHYnRXoYWHtTpOYyAp5laGYX8GH8ITUhgQaP8G/8FA=
github.com/projectdiscovery/hmap v0.0.101 h1:zXM6YtLmsn8Q0CUUw8QavhqWmiQYwaw+/U679Rr00pc=
@@ -895,10 +890,10 @@ github.com/projectdiscovery/mapcidr v1.1.97 h1:7FkxNNVXp+m1rIu5Nv/2SrF9k4+LwP8Qu
github.com/projectdiscovery/mapcidr v1.1.97/go.mod h1:9dgTJh1SP02gYZdpzMjm6vtYFkEHQHoTyaVNvaeJ7lA=
github.com/projectdiscovery/n3iwf v0.0.0-20230523120440-b8cd232ff1f5 h1:L/e8z8yw1pfT6bg35NiN7yd1XKtJap5Nk6lMwQ0RNi8=
github.com/projectdiscovery/n3iwf v0.0.0-20230523120440-b8cd232ff1f5/go.mod h1:pGW2ncnTxTxHtP9wzcIJAB+3/NMp6IiuQWd2NK7K+oc=
github.com/projectdiscovery/networkpolicy v0.1.40 h1:kYin4u1/dgb0nuz5fE1bz4Q0Zh66mOKIdkSHJ00bjGY=
github.com/projectdiscovery/networkpolicy v0.1.40/go.mod h1:9ULLaMbdv9UnT0C5rmuK4nIwYs0o776xMnkPUb8TtaE=
github.com/projectdiscovery/nuclei/v3 v3.9.0 h1:kNHrWZH7mM8Ntf5qacYgHNCEGzmPtywcU0feKm2YnhU=
github.com/projectdiscovery/nuclei/v3 v3.9.0/go.mod h1:6gkhTSiX+7ay5NTHM62+WUUCg7toWwHaWady+3tblbY=
github.com/projectdiscovery/networkpolicy v0.1.41 h1:6daf4A8Vj1+iQ7nH2FR4+sOYd7q1WH1qfN61EyQU74c=
github.com/projectdiscovery/networkpolicy v0.1.41/go.mod h1:9ULLaMbdv9UnT0C5rmuK4nIwYs0o776xMnkPUb8TtaE=
github.com/projectdiscovery/nuclei/v3 v3.10.0 h1:hKWXhfqKvpxGa1vFaS0TlkSPY7EZy4VG7ml4JAMJavo=
github.com/projectdiscovery/nuclei/v3 v3.10.0/go.mod h1:pWZNFVtdIHHSWCV6ouGHUPzvdARD+OGN5K9WrVw0plQ=
github.com/projectdiscovery/ratelimit v0.0.88 h1:AcurW9aLRzlEyPe9kSjnOpr3XzLMWTpiWAlW/w73ALU=
github.com/projectdiscovery/ratelimit v0.0.88/go.mod h1:CU1s+68UUG2mctSl2wi32/DHLJA6TMg+4rxgP59LfVk=
github.com/projectdiscovery/rawhttp v0.1.90 h1:LOSZ6PUH08tnKmWsIwvwv1Z/4zkiYKYOSZ6n+8RFKtw=
@@ -907,8 +902,8 @@ github.com/projectdiscovery/rdap v0.9.1-0.20221108103045-9865884d1917 h1:m03X4gB
github.com/projectdiscovery/rdap v0.9.1-0.20221108103045-9865884d1917/go.mod h1:JxXtZC9e195awe7EynrcnBJmFoad/BNDzW9mzFkK8Sg=
github.com/projectdiscovery/retryabledns v1.0.115 h1:RKV63FNIznFHUoawg/1hs53pVH3wqPtFhwstCuxVSoA=
github.com/projectdiscovery/retryabledns v1.0.115/go.mod h1:+fEMWoPigw+M0lGNKY7AZ+g8FIgj+4sONjsinMmeL3k=
github.com/projectdiscovery/retryablehttp-go v1.3.14 h1:vCBLwK8iIuua3i97jEac5/+TWkYTLhTkGblHu9ETPVc=
github.com/projectdiscovery/retryablehttp-go v1.3.14/go.mod h1:reVhQ+DzMAPYEQHdawCQ6h0tX3CpFyMH4XjcAyq9+U8=
github.com/projectdiscovery/retryablehttp-go v1.3.16 h1:M/xICwSaiSHhd5OIarU3+5JoU7VmbiSAAwYUCK7CTjw=
github.com/projectdiscovery/retryablehttp-go v1.3.16/go.mod h1:s0azLAqAbcVCjHI9t0ezPhamevYGM1eoOvFkn4QmpZ8=
github.com/projectdiscovery/sarif v0.1.0 h1:O541T+a448nSJsmIMnXXSOeDQEzpnCAYvRfe0eG5h74=
github.com/projectdiscovery/sarif v0.1.0/go.mod h1:LBC+reM3bkI3qIIhE0rZaINaYX6VG+En6u2hHa5mA7E=
github.com/projectdiscovery/stringsutil v0.0.2 h1:uzmw3IVLJSMW1kEg8eCStG/cGbYYZAja8BH3LqqJXMA=
@@ -921,8 +916,8 @@ github.com/projectdiscovery/useragent v0.0.108 h1:fb+uLuFJvC+MHZjCtxQJxtvp1X6A8n
github.com/projectdiscovery/useragent v0.0.108/go.mod h1:XdNRrlvtDmYfVL1Oybat4uMe+W6cLwsK9S18ond17CI=
github.com/projectdiscovery/utils v0.11.1 h1:PWj1KjIASxt8icxommH72C0TQqNOvGkcSODRkiq0SQw=
github.com/projectdiscovery/utils v0.11.1/go.mod h1:yktGrHGk2CTjNiccXovnvGrLHX9sV2bqz9nSnbA3V8M=
github.com/projectdiscovery/wappalyzergo v0.2.84 h1:19c+ea8KZCnZIuZPztafFKK2uczDXxcZ/z6/l6DEEEs=
github.com/projectdiscovery/wappalyzergo v0.2.84/go.mod h1:gMH0o5lBp65sKMwHx/tuUdOtW2RjodC6Ti+9QDsYMkY=
github.com/projectdiscovery/wappalyzergo v0.2.87 h1:KuUpeRSQ80L6tx9YaAPhfYWTF47bpEURYbAymr76zto=
github.com/projectdiscovery/wappalyzergo v0.2.87/go.mod h1:gMH0o5lBp65sKMwHx/tuUdOtW2RjodC6Ti+9QDsYMkY=
github.com/projectdiscovery/yamldoc-go v1.0.6 h1:GCEdIRlQjDux28xTXKszM7n3jlMf152d5nqVpVoetas=
github.com/projectdiscovery/yamldoc-go v1.0.6/go.mod h1:R5lWrNzP+7Oyn77NDVPnBsxx2/FyQZBBkIAaSaCQFxw=
github.com/prometheus/client_golang v0.9.1/go.mod h1:7SWBe2y4D6OKWSNQJUaRYU/AaXPKyh/dDVn+NZz0KFw=
@@ -1018,8 +1013,9 @@ github.com/stretchr/objx v0.1.1/go.mod h1:HFkY916IF+rwdDfMAkV7OtwuqBVzrE8GR6GFx+
github.com/stretchr/objx v0.2.0/go.mod h1:qt09Ya8vawLte6SNmTgCsAVtYtaKzEcn8ATUoHMkEqE=
github.com/stretchr/objx v0.4.0/go.mod h1:YvHI0jy2hoMjB+UWwv71VJQ9isScKT/TqJzVSSt89Yw=
github.com/stretchr/objx v0.5.0/go.mod h1:Yh+to48EsGEfYuaHDzXPcE3xhTkx73EhmCGUpEOglKo=
github.com/stretchr/objx v0.5.2 h1:xuMeJ0Sdp5ZMRXx/aWO6RZxdr3beISkG5/G/aIRr3pY=
github.com/stretchr/objx v0.5.2/go.mod h1:FRsXN1f5AsAjCGJKqEizvkpNtU+EGNCLh3NxZ/8L+MA=
github.com/stretchr/objx v0.5.3 h1:jmXUvGomnU1o3W/V5h2VEradbpJDwGrzugQQvL0POH4=
github.com/stretchr/objx v0.5.3/go.mod h1:rDQraq+vQZU7Fde9LOZLr8Tax6zZvy4kuNKF+QYS+U0=
github.com/stretchr/testify v1.2.2/go.mod h1:a8OnRcib4nhh0OaRAV+Yts87kKdq0PP7pXfy6kDkUVs=
github.com/stretchr/testify v1.3.0/go.mod h1:M5WIy9Dh21IEIfnGCwXGc5bZfKNJtfHm1UVUgZn+9EI=
github.com/stretchr/testify v1.4.0/go.mod h1:j7eGeouHqKxXV5pUuKE4zz7dFj8WfuZ+81PSLYec5m4=
@@ -1048,8 +1044,8 @@ github.com/tidwall/btree v1.8.1/go.mod h1:jBbTdUWhSZClZWoDg54VnvV7/54modSOzDN7VX
github.com/tidwall/buntdb v1.3.2 h1:qd+IpdEGs0pZci37G4jF51+fSKlkuUTMXuHhXL1AkKg=
github.com/tidwall/buntdb v1.3.2/go.mod h1:lZZrZUWzlyDJKlLQ6DKAy53LnG7m5kHyrEHvvcDmBpU=
github.com/tidwall/gjson v1.12.1/go.mod h1:/wbyibRr2FHMks5tjHJ5F8dMZh3AcwJEMf5vlfC0lxk=
github.com/tidwall/gjson v1.18.0 h1:FIDeeyB800efLX89e5a8Y0BNH+LOngJyGrIWxG2FKQY=
github.com/tidwall/gjson v1.18.0/go.mod h1:/wbyibRr2FHMks5tjHJ5F8dMZh3AcwJEMf5vlfC0lxk=
github.com/tidwall/gjson v1.19.0 h1:xwxm7n691Uf3u5OFjzngavjGTh55KX5q/9w9xHW88JU=
github.com/tidwall/gjson v1.19.0/go.mod h1:V37/opeE/JbLUOfH0QTXiNez2l0RUjYUhpT4szFQAfc=
github.com/tidwall/grect v0.1.4 h1:dA3oIgNgWdSspFzn1kS4S/RDpZFLrIxAZOdJKjYapOg=
github.com/tidwall/grect v0.1.4/go.mod h1:9FBsaYRaR0Tcy4UwefBX/UDcDcDy9V5jUcxHzv2jd5Q=
github.com/tidwall/lotsa v1.0.2 h1:dNVBH5MErdaQ/xd9s769R31/n2dXavsQ0Yf4TMEHHw8=
@@ -1086,8 +1082,6 @@ github.com/ulikunitz/xz v0.5.15 h1:9DNdB5s+SgV3bQ2ApL10xRc35ck0DuIX/isZvIk+ubY=
github.com/ulikunitz/xz v0.5.15/go.mod h1:nbz6k7qbPmH4IRqmfOplQw/tblSgqTqBwxkY0oWt/14=
github.com/valyala/bytebufferpool v1.0.0 h1:GqA5TC/0021Y/b9FG4Oi9Mr3q7XYx6KllzawFIhcdPw=
github.com/valyala/bytebufferpool v1.0.0/go.mod h1:6bBcMArwyJ5K/AmCkWv1jt77kVWyCJ6HpOuEn7z0Csc=
github.com/valyala/fasttemplate v1.2.2 h1:lxLXG0uE3Qnshl9QyaK6XJxMXlQZELvChBOCmQD0Loo=
github.com/valyala/fasttemplate v1.2.2/go.mod h1:KHLXt3tVN2HBp8eijSv/kGJopbvo7S+qRAEEKiv+SiQ=
github.com/vmihailenco/bufpool v0.1.11 h1:gOq2WmBrq0i2yW5QJ16ykccQ4wH9UyEsgLm6czKAd94=
github.com/vmihailenco/bufpool v0.1.11/go.mod h1:AFf/MOy3l2CFTKbxwt0mp2MwnqjNEs5H/UxrkA5jxTQ=
github.com/vmihailenco/msgpack/v5 v5.3.4 h1:qMKAwOV+meBw2Y8k9cVwAy7qErtYCwBzZ2ellBfvnqc=
@@ -1644,7 +1638,6 @@ gopkg.in/yaml.v2 v2.2.2/go.mod h1:hI93XBmqTisBFMUTm0b8Fm+jr3Dg1NNxqwp+5A1VGuI=
gopkg.in/yaml.v2 v2.2.4/go.mod h1:hI93XBmqTisBFMUTm0b8Fm+jr3Dg1NNxqwp+5A1VGuI=
gopkg.in/yaml.v2 v2.2.5/go.mod h1:hI93XBmqTisBFMUTm0b8Fm+jr3Dg1NNxqwp+5A1VGuI=
gopkg.in/yaml.v2 v2.3.0/go.mod h1:hI93XBmqTisBFMUTm0b8Fm+jr3Dg1NNxqwp+5A1VGuI=
gopkg.in/yaml.v2 v2.4.0 h1:D8xgwECY7CYvx+Y2n4sBz93Jn9JRvxdiyyo8CTfuKaY=
gopkg.in/yaml.v2 v2.4.0/go.mod h1:RDklbk79AGWmwhnvt/jBztapEOGDOx6ZbXqjP6csGnQ=
gopkg.in/yaml.v3 v3.0.0-20200313102051-9f266ea9e77c/go.mod h1:K4uyk7z7BCEPqu6E+C64Yfv1cQ7kz7rIZviUmN+EgEM=
gopkg.in/yaml.v3 v3.0.1 h1:fxVm/GzAzEWqLHuvctI91KS9hhNmmWOoWu0XTYJS7CA=
+2
View File
@@ -80,6 +80,7 @@ type Settings struct {
Header goflags.StringSlice // custom request headers ("Key: Value")
Cookie string
RateLimit int
MaxRetries int // -max-retries: retries on 429/503 (0 = off)
Notify bool // -notify: ship findings to configured providers
NotifySeverity string // -notify-severity: minimum severity to send (info..critical)
NotifyConfig string // -notify-config: path to a notify-compatible yaml file
@@ -178,6 +179,7 @@ func registerFlags(settings *Settings) *goflags.FlagSet {
flagSet.StringSliceVarP(&settings.Header, "header", "H", nil, "Custom header to send (repeatable or comma-separated, \"Key: Value\")", goflags.CommaSeparatedStringSliceOptions),
flagSet.StringVar(&settings.Cookie, "cookie", "", "Cookie header to send with every request"),
flagSet.IntVar(&settings.RateLimit, "rate-limit", 0, "Max requests per second (0 = unlimited)"),
flagSet.IntVar(&settings.MaxRetries, "max-retries", 2, "Retries on 429/503 with Retry-After backoff (0 = off)"),
)
flagSet.CreateGroup("output", "Output",
+4 -4
View File
@@ -21,11 +21,11 @@ import (
"fmt"
"strings"
"github.com/dropalldatabases/sif/internal/modules"
"github.com/dropalldatabases/sif/internal/scan"
"github.com/dropalldatabases/sif/internal/scan/frameworks"
"github.com/dropalldatabases/sif/internal/scan/js"
"github.com/projectdiscovery/nuclei/v3/pkg/output"
"github.com/vmfunc/sif/internal/modules"
"github.com/vmfunc/sif/internal/scan"
"github.com/vmfunc/sif/internal/scan/frameworks"
"github.com/vmfunc/sif/internal/scan/js"
)
// Finding is the normalized shape every scanner result collapses to. one
+4 -4
View File
@@ -16,13 +16,13 @@ import (
"strings"
"testing"
"github.com/dropalldatabases/sif/internal/modules"
"github.com/dropalldatabases/sif/internal/scan"
"github.com/dropalldatabases/sif/internal/scan/frameworks"
"github.com/dropalldatabases/sif/internal/scan/js"
"github.com/projectdiscovery/nuclei/v3/pkg/model"
"github.com/projectdiscovery/nuclei/v3/pkg/model/types/severity"
"github.com/projectdiscovery/nuclei/v3/pkg/output"
"github.com/vmfunc/sif/internal/modules"
"github.com/vmfunc/sif/internal/scan"
"github.com/vmfunc/sif/internal/scan/frameworks"
"github.com/vmfunc/sif/internal/scan/js"
)
// scanResultType mirrors the minimal interface the scan packages implement; the
+113 -17
View File
@@ -16,11 +16,13 @@
package httpx
import (
"context"
"fmt"
"io"
"net"
"net/http"
"net/url"
"strconv"
"strings"
"sync"
"time"
@@ -74,6 +76,8 @@ type Options struct {
Cookie string
UserAgent string
RateLimit int
// MaxRetries is how many 429/503 responses to retry with backoff (0 = off).
MaxRetries int
// Threads is the scan worker count; it sizes the per-host idle pool so
// concurrent workers hitting one target reuse conns instead of dialing fresh.
Threads int
@@ -107,11 +111,12 @@ func Configure(opts Options) error {
}
rt := &roundTripper{
base: base,
headers: headers,
cookie: opts.Cookie,
userAgent: opts.UserAgent,
limiter: limiter,
base: base,
headers: headers,
cookie: opts.Cookie,
userAgent: opts.UserAgent,
limiter: limiter,
maxRetries: opts.MaxRetries,
}
mu.Lock()
@@ -226,20 +231,15 @@ func parseHeaders(raw []string) (map[string]string, error) {
// roundTripper paces and decorates each request before delegating to base.
type roundTripper struct {
base *http.Transport
headers map[string]string
cookie string
userAgent string
limiter *rate.Limiter
base *http.Transport
headers map[string]string
cookie string
userAgent string
limiter *rate.Limiter
maxRetries int
}
func (rt *roundTripper) RoundTrip(req *http.Request) (*http.Response, error) {
if rt.limiter != nil {
if err := rt.limiter.Wait(req.Context()); err != nil {
return nil, fmt.Errorf("rate limiter: %w", err)
}
}
// only set what the caller hasn't already; a scanner that explicitly sets a
// header (e.g. an api key) must win over the global default.
for key, value := range rt.headers {
@@ -254,5 +254,101 @@ func (rt *roundTripper) RoundTrip(req *http.Request) (*http.Response, error) {
req.Header.Set("User-Agent", rt.userAgent)
}
return rt.base.RoundTrip(req)
for attempt := 0; ; attempt++ {
if rt.limiter != nil {
if err := rt.limiter.Wait(req.Context()); err != nil {
return nil, fmt.Errorf("rate limiter: %w", err)
}
}
resp, err := rt.base.RoundTrip(req)
if err != nil || attempt >= rt.maxRetries || !retryableStatus(resp.StatusCode) {
return resp, err
}
// back off and retry, unless the body can't be replayed.
if !rewind(req) {
return resp, nil
}
wait := retryAfter(resp, attempt)
DrainClose(resp)
if err := sleepCtx(req.Context(), wait); err != nil {
return nil, err
}
}
}
func retryableStatus(code int) bool {
return code == http.StatusTooManyRequests || code == http.StatusServiceUnavailable
}
const (
retryAfterCap = 20 * time.Second
retryBackoffBase = 500 * time.Millisecond
// clamp the shift so a large -max-retries can't overflow the duration.
retryBackoffMaxShift = 16
)
// retryAfter honors a Retry-After header (delta-seconds or HTTP-date) and
// otherwise falls back to capped exponential backoff.
func retryAfter(resp *http.Response, attempt int) time.Duration {
if v := strings.TrimSpace(resp.Header.Get("Retry-After")); v != "" {
if secs, err := strconv.Atoi(v); err == nil && secs >= 0 {
return capDuration(time.Duration(secs) * time.Second)
}
if t, err := http.ParseTime(v); err == nil {
return capDuration(time.Until(t))
}
}
shift := attempt
if shift > retryBackoffMaxShift {
shift = retryBackoffMaxShift
}
return capDuration(retryBackoffBase << shift)
}
// capDuration clamps d to [0, retryAfterCap].
func capDuration(d time.Duration) time.Duration {
switch {
case d < 0:
return 0
case d > retryAfterCap:
return retryAfterCap
default:
return d
}
}
// rewind restores req.Body for a resend. Only a GetBody-backed body (set by
// net/http for the in-memory bodies sif uses) is replayable; a nil or NoBody
// request needs nothing, anything else can't be retried.
func rewind(req *http.Request) bool {
if req.Body == nil || req.Body == http.NoBody {
return true
}
if req.GetBody == nil {
return false
}
body, err := req.GetBody()
if err != nil {
return false
}
req.Body = body
return true
}
// sleepCtx waits for d or until ctx is cancelled, whichever comes first.
func sleepCtx(ctx context.Context, d time.Duration) error {
if d <= 0 {
return nil
}
timer := time.NewTimer(d)
defer timer.Stop()
select {
case <-ctx.Done():
return ctx.Err()
case <-timer.C:
return nil
}
}
+226
View File
@@ -20,6 +20,7 @@ import (
"net/http/httptest"
"strings"
"sync"
"sync/atomic"
"testing"
"time"
)
@@ -489,3 +490,228 @@ func resetBench() {
configured = nil
mu.Unlock()
}
// retrySequenceServer serves codes[n] on the n-th hit, repeating the last once
// the slice runs out; retryable codes carry Retry-After: 0 to keep tests fast.
func retrySequenceServer(t *testing.T, hits *atomic.Int64, codes ...int) *httptest.Server {
t.Helper()
srv := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, _ *http.Request) {
n := int(hits.Add(1)) - 1
code := codes[len(codes)-1]
if n < len(codes) {
code = codes[n]
}
if code == http.StatusTooManyRequests || code == http.StatusServiceUnavailable {
w.Header().Set("Retry-After", "0")
}
w.WriteHeader(code)
}))
t.Cleanup(srv.Close)
return srv
}
// getStatus performs a GET and returns the final status code the caller sees.
func getStatus(t *testing.T, client *http.Client, url string) int {
t.Helper()
req, err := http.NewRequestWithContext(context.Background(), http.MethodGet, url, http.NoBody)
if err != nil {
t.Fatalf("new request: %v", err)
}
resp, err := client.Do(req)
if err != nil {
t.Fatalf("do request: %v", err)
}
resp.Body.Close()
return resp.StatusCode
}
func TestRetryRecoversAfter429(t *testing.T) {
resetConfig(t)
var hits atomic.Int64
srv := retrySequenceServer(t, &hits, http.StatusTooManyRequests, http.StatusOK)
if err := Configure(Options{MaxRetries: 2}); err != nil {
t.Fatalf("Configure: %v", err)
}
if code := getStatus(t, Client(5*time.Second), srv.URL); code != http.StatusOK {
t.Errorf("status = %d, want 200 after retry", code)
}
if got := hits.Load(); got != 2 {
t.Errorf("server hits = %d, want 2 (initial + one retry)", got)
}
}
func TestRetryRecoversAfter503(t *testing.T) {
resetConfig(t)
var hits atomic.Int64
srv := retrySequenceServer(t, &hits, http.StatusServiceUnavailable, http.StatusOK)
if err := Configure(Options{MaxRetries: 2}); err != nil {
t.Fatalf("Configure: %v", err)
}
if code := getStatus(t, Client(5*time.Second), srv.URL); code != http.StatusOK {
t.Errorf("status = %d, want 200 after retry", code)
}
if got := hits.Load(); got != 2 {
t.Errorf("server hits = %d, want 2", got)
}
}
func TestRetryDisabled(t *testing.T) {
resetConfig(t)
var hits atomic.Int64
srv := retrySequenceServer(t, &hits, http.StatusTooManyRequests, http.StatusOK)
if err := Configure(Options{MaxRetries: 0}); err != nil {
t.Fatalf("Configure: %v", err)
}
if code := getStatus(t, Client(5*time.Second), srv.URL); code != http.StatusTooManyRequests {
t.Errorf("status = %d, want 429 with retries off", code)
}
if got := hits.Load(); got != 1 {
t.Errorf("server hits = %d, want 1 (no retry)", got)
}
}
func TestRetryExhausted(t *testing.T) {
resetConfig(t)
var hits atomic.Int64
srv := retrySequenceServer(t, &hits, http.StatusTooManyRequests) // always 429
if err := Configure(Options{MaxRetries: 2}); err != nil {
t.Fatalf("Configure: %v", err)
}
if code := getStatus(t, Client(5*time.Second), srv.URL); code != http.StatusTooManyRequests {
t.Errorf("status = %d, want 429 after exhausting retries", code)
}
if got := hits.Load(); got != 3 {
t.Errorf("server hits = %d, want 3 (initial + 2 retries)", got)
}
}
func TestRetryIgnoresNonRetryableStatus(t *testing.T) {
resetConfig(t)
var hits atomic.Int64
srv := retrySequenceServer(t, &hits, http.StatusInternalServerError, http.StatusOK)
if err := Configure(Options{MaxRetries: 2}); err != nil {
t.Fatalf("Configure: %v", err)
}
if code := getStatus(t, Client(5*time.Second), srv.URL); code != http.StatusInternalServerError {
t.Errorf("status = %d, want 500 (not retried)", code)
}
if got := hits.Load(); got != 1 {
t.Errorf("server hits = %d, want 1 (500 not retried)", got)
}
}
func TestRetryReplaysRequestBody(t *testing.T) {
resetConfig(t)
var hits atomic.Int64
var bmu sync.Mutex
var bodies []string
srv := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
body, _ := io.ReadAll(r.Body)
bmu.Lock()
bodies = append(bodies, string(body))
bmu.Unlock()
if hits.Add(1) == 1 {
w.Header().Set("Retry-After", "0")
w.WriteHeader(http.StatusTooManyRequests)
return
}
w.WriteHeader(http.StatusOK)
}))
t.Cleanup(srv.Close)
if err := Configure(Options{MaxRetries: 2}); err != nil {
t.Fatalf("Configure: %v", err)
}
req, err := http.NewRequestWithContext(context.Background(), http.MethodPost, srv.URL, strings.NewReader("payload"))
if err != nil {
t.Fatalf("new request: %v", err)
}
resp, err := Client(5 * time.Second).Do(req)
if err != nil {
t.Fatalf("do request: %v", err)
}
resp.Body.Close()
if resp.StatusCode != http.StatusOK {
t.Fatalf("status = %d, want 200 after body replay", resp.StatusCode)
}
bmu.Lock()
defer bmu.Unlock()
if len(bodies) != 2 {
t.Fatalf("server saw %d requests, want 2", len(bodies))
}
for i, body := range bodies {
if body != "payload" {
t.Errorf("body[%d] = %q, want %q (rewind dropped the body)", i, body, "payload")
}
}
}
func TestRetryAfterHeader(t *testing.T) {
noHeader := &http.Response{Header: http.Header{}}
if got := retryAfter(noHeader, 0); got != retryBackoffBase {
t.Errorf("missing header: attempt 0 = %v, want %v", got, retryBackoffBase)
}
if got := retryAfter(noHeader, 1); got != 2*retryBackoffBase {
t.Errorf("missing header: attempt 1 = %v, want %v", got, 2*retryBackoffBase)
}
if got := retryAfter(noHeader, 1000); got != retryAfterCap {
t.Errorf("missing header: attempt 1000 = %v, want cap %v", got, retryAfterCap)
}
withSeconds := func(v string) *http.Response {
return &http.Response{Header: http.Header{"Retry-After": {v}}}
}
if got := retryAfter(withSeconds("3"), 0); got != 3*time.Second {
t.Errorf("Retry-After 3 = %v, want 3s", got)
}
if got := retryAfter(withSeconds("0"), 5); got != 0 {
t.Errorf("Retry-After 0 = %v, want 0", got)
}
if got := retryAfter(withSeconds("9999"), 0); got != retryAfterCap {
t.Errorf("Retry-After 9999 = %v, want cap %v", got, retryAfterCap)
}
if got := retryAfter(withSeconds("soon"), 0); got != retryBackoffBase {
t.Errorf("Retry-After junk = %v, want backoff %v", got, retryBackoffBase)
}
future := time.Now().Add(5 * time.Second).UTC().Format(http.TimeFormat)
if got := retryAfter(withSeconds(future), 0); got <= 0 || got > 5*time.Second {
t.Errorf("Retry-After http-date = %v, want (0, 5s]", got)
}
}
func TestCapDuration(t *testing.T) {
cases := []struct{ in, want time.Duration }{
{-time.Second, 0},
{0, 0},
{5 * time.Second, 5 * time.Second},
{retryAfterCap, retryAfterCap},
{retryAfterCap + time.Second, retryAfterCap},
}
for _, c := range cases {
if got := capDuration(c.in); got != c.want {
t.Errorf("capDuration(%v) = %v, want %v", c.in, got, c.want)
}
}
}
func TestSleepCtxCancelled(t *testing.T) {
ctx, cancel := context.WithCancel(context.Background())
cancel()
if err := sleepCtx(ctx, time.Hour); err == nil {
t.Error("sleepCtx on a cancelled context should return its error, not block")
}
}
@@ -0,0 +1,216 @@
package modules_test
import (
"context"
"net/http"
"net/http/httptest"
"testing"
"time"
"github.com/vmfunc/sif/internal/modules"
)
func runOrchestrationModule(t *testing.T, file string, status int, body string) *modules.Result {
t.Helper()
def, err := modules.ParseYAMLModule(file)
if err != nil {
t.Fatalf("parse %s: %v", file, err)
}
srv := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
w.WriteHeader(status)
_, _ = w.Write([]byte(body))
}))
defer srv.Close()
res, err := modules.ExecuteHTTPModule(context.Background(), srv.URL, def, modules.Options{
Timeout: 5 * time.Second,
Threads: 2,
})
if err != nil {
t.Fatalf("execute %s: %v", file, err)
}
return res
}
func orchestrationExtract(res *modules.Result, key string) string {
for _, f := range res.Findings {
if v := f.Extracted[key]; v != "" {
return v
}
}
return ""
}
func TestAIOrchestrationExposureModules(t *testing.T) {
const langflow = "../../modules/recon/langflow-exposure.yaml"
const dify = "../../modules/recon/dify-console-exposure.yaml"
const ray = "../../modules/recon/ray-dashboard-exposure.yaml"
const skypilot = "../../modules/recon/skypilot-api-exposure.yaml"
langflowVersion := `{"version":"1.0.19","main_version":"1.0.19","package":"Langflow"}`
difyFeatures := `{"enable_app_deploy":true,"sso_enforced_for_signin":false,` +
`"sso_enforced_for_signin_protocol":"","enable_marketplace":true,"enable_email_code_login":false,` +
`"enable_email_password_login":true,"enable_social_oauth_login":false,"is_allow_register":true,` +
`"is_allow_create_workspace":false,"is_email_setup":true,"license":{"status":"none","expired_at":""}}`
rayVersion := `{"version":"4","ray_version":"2.9.3","ray_commit":"a1b2c3d4e5","session_name":"session_2024"}`
skypilotHealth := `{"status":"healthy","api_version":"14","version":"0.9.3","version_on_disk":"0.9.3",` +
`"commit":"abc1234def","basic_auth_enabled":false}`
t.Run("a langflow version api is flagged", func(t *testing.T) {
res := runOrchestrationModule(t, langflow, 200, langflowVersion)
if len(res.Findings) == 0 {
t.Fatal("expected a langflow finding")
}
if v := orchestrationExtract(res, "langflow_version"); v != "1.0.19" {
t.Errorf("langflow_version=%q, want 1.0.19", v)
}
})
t.Run("a langflow base build is still flagged", func(t *testing.T) {
body := `{"version":"1.0.19","main_version":"1.0.19","package":"Langflow Base"}`
if res := runOrchestrationModule(t, langflow, 200, body); len(res.Findings) == 0 {
t.Fatal("expected a finding for a langflow base build")
}
})
t.Run("a version api from another package is not flagged as langflow", func(t *testing.T) {
body := `{"version":"1.0","main_version":"1.0","package":"SomeApp"}`
if res := runOrchestrationModule(t, langflow, 200, body); len(res.Findings) > 0 {
t.Errorf("another package should not match langflow, got %d findings", len(res.Findings))
}
})
t.Run("a langflow package without main_version is not flagged", func(t *testing.T) {
if res := runOrchestrationModule(t, langflow, 200, `{"package":"Langflow"}`); len(res.Findings) > 0 {
t.Errorf("a package-only body should not match langflow, got %d findings", len(res.Findings))
}
})
t.Run("a dify system-features is flagged and reports open registration", func(t *testing.T) {
res := runOrchestrationModule(t, dify, 200, difyFeatures)
if len(res.Findings) == 0 {
t.Fatal("expected a dify finding")
}
if v := orchestrationExtract(res, "dify_allow_register"); v != "true" {
t.Errorf("dify_allow_register=%q, want true", v)
}
})
t.Run("a body without sso_enforced_for_signin is not flagged as dify", func(t *testing.T) {
body := `{"enable_email_password_login":true,"is_allow_create_workspace":false,"is_allow_register":true}`
if res := runOrchestrationModule(t, dify, 200, body); len(res.Findings) > 0 {
t.Errorf("a body without sso_enforced_for_signin should not match dify, got %d findings", len(res.Findings))
}
})
t.Run("a body without enable_email_password_login is not flagged as dify", func(t *testing.T) {
body := `{"sso_enforced_for_signin":false,"is_allow_create_workspace":false}`
if res := runOrchestrationModule(t, dify, 200, body); len(res.Findings) > 0 {
t.Errorf("a body without enable_email_password_login should not match dify, got %d findings", len(res.Findings))
}
})
t.Run("a body without is_allow_create_workspace is not flagged as dify", func(t *testing.T) {
body := `{"sso_enforced_for_signin":false,"enable_email_password_login":true}`
if res := runOrchestrationModule(t, dify, 200, body); len(res.Findings) > 0 {
t.Errorf("a body without is_allow_create_workspace should not match dify, got %d findings", len(res.Findings))
}
})
t.Run("a dify with registration disabled is not flagged", func(t *testing.T) {
body := `{"sso_enforced_for_signin":false,"enable_email_password_login":true,"is_allow_create_workspace":true,"is_allow_register":false}`
if res := runOrchestrationModule(t, dify, 200, body); len(res.Findings) > 0 {
t.Errorf("a closed-registration dify should not be flagged, got %d findings", len(res.Findings))
}
})
t.Run("a ray dashboard version is flagged", func(t *testing.T) {
res := runOrchestrationModule(t, ray, 200, rayVersion)
if len(res.Findings) == 0 {
t.Fatal("expected a ray finding")
}
if v := orchestrationExtract(res, "ray_version"); v != "2.9.3" {
t.Errorf("ray_version=%q, want 2.9.3", v)
}
})
t.Run("a generic version api is not flagged as ray", func(t *testing.T) {
body := `{"version":"4","api_version":"v1","build":"123"}`
if res := runOrchestrationModule(t, ray, 200, body); len(res.Findings) > 0 {
t.Errorf("a generic version api should not match ray, got %d findings", len(res.Findings))
}
})
t.Run("a ray_version without a ray_commit is not flagged", func(t *testing.T) {
body := `{"version":"4","ray_version":"2.9.3"}`
if res := runOrchestrationModule(t, ray, 200, body); len(res.Findings) > 0 {
t.Errorf("ray_version alone should not match ray, got %d findings", len(res.Findings))
}
})
t.Run("a skypilot health is flagged with its version and auth state", func(t *testing.T) {
res := runOrchestrationModule(t, skypilot, 200, skypilotHealth)
if len(res.Findings) == 0 {
t.Fatal("expected a skypilot finding")
}
if v := orchestrationExtract(res, "skypilot_version"); v != "0.9.3" {
t.Errorf("skypilot_version=%q, want 0.9.3", v)
}
if v := orchestrationExtract(res, "skypilot_basic_auth"); v != "false" {
t.Errorf("skypilot_basic_auth=%q, want false", v)
}
})
t.Run("a bare status health is not flagged as skypilot", func(t *testing.T) {
if res := runOrchestrationModule(t, skypilot, 200, `{"status":"healthy"}`); len(res.Findings) > 0 {
t.Errorf("an auth-gated bare health should not match skypilot, got %d findings", len(res.Findings))
}
})
t.Run("a body without version_on_disk is not flagged as skypilot", func(t *testing.T) {
body := `{"status":"healthy","api_version":"14","commit":"abc","basic_auth_enabled":false}`
if res := runOrchestrationModule(t, skypilot, 200, body); len(res.Findings) > 0 {
t.Errorf("a body without version_on_disk should not match skypilot, got %d findings", len(res.Findings))
}
})
t.Run("a body without basic_auth_enabled is not flagged as skypilot", func(t *testing.T) {
body := `{"status":"healthy","version_on_disk":"0.9.3","commit":"abc"}`
if res := runOrchestrationModule(t, skypilot, 200, body); len(res.Findings) > 0 {
t.Errorf("a body without basic_auth_enabled should not match skypilot, got %d findings", len(res.Findings))
}
})
t.Run("a body without commit is not flagged as skypilot", func(t *testing.T) {
body := `{"status":"healthy","version_on_disk":"0.9.3","basic_auth_enabled":false}`
if res := runOrchestrationModule(t, skypilot, 200, body); len(res.Findings) > 0 {
t.Errorf("a body without commit should not match skypilot, got %d findings", len(res.Findings))
}
})
t.Run("a skypilot with basic auth enabled is not flagged", func(t *testing.T) {
body := `{"status":"healthy","version_on_disk":"0.9.3","commit":"abc","basic_auth_enabled":true}`
if res := runOrchestrationModule(t, skypilot, 200, body); len(res.Findings) > 0 {
t.Errorf("an auth-enabled skypilot should not be flagged, got %d findings", len(res.Findings))
}
})
t.Run("a plain 200 body is not a leak", func(t *testing.T) {
for _, file := range []string{langflow, dify, ray, skypilot} {
if res := runOrchestrationModule(t, file, 200, "ok"); len(res.Findings) > 0 {
t.Errorf("%s: a plain 200 body should not match, got %d findings", file, len(res.Findings))
}
}
})
t.Run("a 404 is not a leak", func(t *testing.T) {
for _, file := range []string{langflow, dify, ray, skypilot} {
if res := runOrchestrationModule(t, file, 404, "not found"); len(res.Findings) > 0 {
t.Errorf("%s: a 404 should not match, got %d findings", file, len(res.Findings))
}
}
})
}
@@ -0,0 +1,87 @@
package modules_test
import (
"context"
"net/http"
"net/http/httptest"
"testing"
"time"
"github.com/vmfunc/sif/internal/modules"
)
func runAlertmanagerModule(t *testing.T, file string, status int, body string) *modules.Result {
t.Helper()
def, err := modules.ParseYAMLModule(file)
if err != nil {
t.Fatalf("parse %s: %v", file, err)
}
srv := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
w.WriteHeader(status)
_, _ = w.Write([]byte(body))
}))
defer srv.Close()
res, err := modules.ExecuteHTTPModule(context.Background(), srv.URL, def, modules.Options{
Timeout: 5 * time.Second,
Threads: 2,
})
if err != nil {
t.Fatalf("execute %s: %v", file, err)
}
return res
}
func alertmanagerExtract(res *modules.Result, key string) string {
for _, f := range res.Findings {
if v := f.Extracted[key]; v != "" {
return v
}
}
return ""
}
func TestAlertmanagerExposureModule(t *testing.T) {
const am = "../../modules/recon/alertmanager-status-exposure.yaml"
t.Run("an alertmanager status is flagged with its version", func(t *testing.T) {
body := `{"cluster":{"name":"01HXYZ","status":"ready","peers":[{"name":"01HX","address":"10.0.0.7:9094"}]},` +
`"versionInfo":{"branch":"HEAD","buildDate":"20240228","buildUser":"root@host","goVersion":"go1.21.7",` +
`"revision":"0aa3c2a","version":"0.27.0"},"config":{"original":"global:\n smtp_smarthost: 'smtp:587'\n ` +
`smtp_auth_password: 'hunter2'\nreceivers:\n- name: team\n slack_configs:\n - api_url: 'https://hooks.slack.com/services/T/B/X'\n"},` +
`"uptime":"2024-06-01T10:00:00.000Z"}`
res := runAlertmanagerModule(t, am, 200, body)
if len(res.Findings) == 0 {
t.Fatal("expected an alertmanager finding")
}
if v := alertmanagerExtract(res, "alertmanager_version"); v != "0.27.0" {
t.Errorf("alertmanager_version=%q, want 0.27.0", v)
}
})
t.Run("a versionInfo+cluster body without config is not flagged", func(t *testing.T) {
body := `{"cluster":{"name":"01HXYZ","status":"ready"},"versionInfo":{"version":"0.27.0"},"uptime":"x"}`
if res := runAlertmanagerModule(t, am, 200, body); len(res.Findings) > 0 {
t.Errorf("a configless status should not match alertmanager, got %d findings", len(res.Findings))
}
})
t.Run("a config+versionInfo body without cluster is not flagged", func(t *testing.T) {
body := `{"versionInfo":{"version":"0.27.0"},"config":{"original":"global:\n"},"uptime":"x"}`
if res := runAlertmanagerModule(t, am, 200, body); len(res.Findings) > 0 {
t.Errorf("a clusterless status should not match alertmanager, got %d findings", len(res.Findings))
}
})
t.Run("a plain 200 body is not a leak", func(t *testing.T) {
if res := runAlertmanagerModule(t, am, 200, "ok"); len(res.Findings) > 0 {
t.Errorf("a plain 200 body should not match, got %d findings", len(res.Findings))
}
})
t.Run("a 404 is not a leak", func(t *testing.T) {
if res := runAlertmanagerModule(t, am, 404, "not found"); len(res.Findings) > 0 {
t.Errorf("a 404 should not match, got %d findings", len(res.Findings))
}
})
}
@@ -7,7 +7,7 @@ import (
"testing"
"time"
"github.com/dropalldatabases/sif/internal/modules"
"github.com/vmfunc/sif/internal/modules"
)
func runAnalyticsModule(t *testing.T, file string, status int, body string) *modules.Result {
+1 -1
View File
@@ -7,7 +7,7 @@ import (
"testing"
"time"
"github.com/dropalldatabases/sif/internal/modules"
"github.com/vmfunc/sif/internal/modules"
)
func runAppCfgModule(t *testing.T, file string, status int, body string) *modules.Result {
+1 -1
View File
@@ -7,7 +7,7 @@ import (
"testing"
"time"
"github.com/dropalldatabases/sif/internal/modules"
"github.com/vmfunc/sif/internal/modules"
)
func runArgocdModule(t *testing.T, status int, body string) *modules.Result {
+1 -1
View File
@@ -23,7 +23,7 @@ import (
"sync"
"testing"
"github.com/dropalldatabases/sif/internal/httpx"
"github.com/vmfunc/sif/internal/httpx"
)
func reqURLs(reqs []*httpRequest) []string {
@@ -0,0 +1,110 @@
package modules_test
import (
"context"
"net/http"
"net/http/httptest"
"testing"
"time"
"github.com/vmfunc/sif/internal/modules"
)
func runAutomationModule(t *testing.T, file string, status int, body string) *modules.Result {
t.Helper()
def, err := modules.ParseYAMLModule(file)
if err != nil {
t.Fatalf("parse %s: %v", file, err)
}
srv := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
w.WriteHeader(status)
_, _ = w.Write([]byte(body))
}))
defer srv.Close()
res, err := modules.ExecuteHTTPModule(context.Background(), srv.URL, def, modules.Options{
Timeout: 5 * time.Second,
Threads: 2,
})
if err != nil {
t.Fatalf("execute %s: %v", file, err)
}
return res
}
func automationExtract(res *modules.Result, key string) string {
for _, f := range res.Findings {
if v := f.Extracted[key]; v != "" {
return v
}
}
return ""
}
func TestAutomationServerExposureModules(t *testing.T) {
const jenkins = "../../modules/recon/jenkins-api-exposure.yaml"
const nifi = "../../modules/recon/nifi-api-exposure.yaml"
t.Run("a jenkins controller api is flagged with the first job", func(t *testing.T) {
body := `{"_class":"hudson.model.Hudson","assignedLabels":[{}],"mode":"NORMAL","nodeDescription":` +
`"the master Jenkins node","nodeName":"","numExecutors":2,"jobs":[{"_class":` +
`"hudson.model.FreeStyleProject","name":"deploy-prod","url":"http://ci/job/deploy-prod/","color":"blue"}],` +
`"useSecurity":true,"views":[{"_class":"hudson.model.AllView","name":"all","url":"http://ci/"}]}`
res := runAutomationModule(t, jenkins, 200, body)
if len(res.Findings) == 0 {
t.Fatal("expected a jenkins finding")
}
if v := automationExtract(res, "jenkins_job"); v != "deploy-prod" {
t.Errorf("jenkins_job=%q, want deploy-prod", v)
}
})
t.Run("a non-root jenkins object is not flagged as the controller", func(t *testing.T) {
body := `{"_class":"hudson.model.FreeStyleProject","name":"deploy-prod","jobs":[],"color":"blue"}`
if res := runAutomationModule(t, jenkins, 200, body); len(res.Findings) > 0 {
t.Errorf("a FreeStyleProject object should not match the controller, got %d findings", len(res.Findings))
}
})
t.Run("a hudson root without a jobs key is not flagged", func(t *testing.T) {
if res := runAutomationModule(t, jenkins, 200, `{"_class":"hudson.model.Hudson","mode":"NORMAL"}`); len(res.Findings) > 0 {
t.Errorf("a jobless controller blob should not match, got %d findings", len(res.Findings))
}
})
t.Run("a nifi about is flagged with its version", func(t *testing.T) {
body := `{"about":{"title":"NiFi","version":"1.28.1","uri":"https://nifi:8443/nifi-api/",` +
`"contentViewerUrl":"../nifi-content-viewer/","timezone":"UTC","buildTag":"nifi-1.28.1-RC1",` +
`"buildRevision":"abc123","buildBranch":"support/nifi-1.x","buildTimestamp":"06/01/2024 10:00:00 UTC"}}`
res := runAutomationModule(t, nifi, 200, body)
if len(res.Findings) == 0 {
t.Fatal("expected a nifi finding")
}
if v := automationExtract(res, "nifi_version"); v != "1.28.1" {
t.Errorf("nifi_version=%q, want 1.28.1", v)
}
})
t.Run("an about block for another product is not flagged as nifi", func(t *testing.T) {
if res := runAutomationModule(t, nifi, 200, `{"about":{"title":"SomeApp","version":"2.0.0"}}`); len(res.Findings) > 0 {
t.Errorf("a non-nifi about block should not match, got %d findings", len(res.Findings))
}
})
t.Run("a plain 200 body is not a leak", func(t *testing.T) {
for _, file := range []string{jenkins, nifi} {
if res := runAutomationModule(t, file, 200, "ok"); len(res.Findings) > 0 {
t.Errorf("%s: a plain 200 body should not match, got %d findings", file, len(res.Findings))
}
}
})
t.Run("a 403 or 401 is not a leak", func(t *testing.T) {
if res := runAutomationModule(t, jenkins, 403, `{"_class":"hudson.model.Hudson","jobs":[]}`); len(res.Findings) > 0 {
t.Errorf("a 403 jenkins should not match, got %d findings", len(res.Findings))
}
if res := runAutomationModule(t, nifi, 401, `{"about":{"title":"NiFi","version":"1.28.1"}}`); len(res.Findings) > 0 {
t.Errorf("a 401 nifi should not match, got %d findings", len(res.Findings))
}
})
}
@@ -7,7 +7,7 @@ import (
"testing"
"time"
"github.com/dropalldatabases/sif/internal/modules"
"github.com/vmfunc/sif/internal/modules"
)
func runBigDataModule(t *testing.T, file string, status int, body string) *modules.Result {
@@ -0,0 +1,115 @@
package modules_test
import (
"context"
"net/http"
"net/http/httptest"
"testing"
"time"
"github.com/vmfunc/sif/internal/modules"
)
func runGridModule(t *testing.T, file string, status int, body string) *modules.Result {
t.Helper()
def, err := modules.ParseYAMLModule(file)
if err != nil {
t.Fatalf("parse %s: %v", file, err)
}
srv := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
w.WriteHeader(status)
_, _ = w.Write([]byte(body))
}))
defer srv.Close()
res, err := modules.ExecuteHTTPModule(context.Background(), srv.URL, def, modules.Options{
Timeout: 5 * time.Second,
Threads: 2,
})
if err != nil {
t.Fatalf("execute %s: %v", file, err)
}
return res
}
func gridExtract(res *modules.Result, key string) string {
for _, f := range res.Findings {
if v := f.Extracted[key]; v != "" {
return v
}
}
return ""
}
const seleniumStatusBody = `{"value":{"ready":true,"message":"Selenium Grid ready.","nodes":[` +
`{"id":"028ca108-bfc4-430e-806c-6477b6b8569e","uri":"http://10.0.0.5:5555","maxSessions":1,` +
`"osInfo":{"arch":"amd64","name":"Linux","version":"5.15.0"},"heartbeatPeriod":60000,` +
`"availability":"UP","version":"4.18.1 (revision b1d3319b48)","slots":[{"lastStarted":` +
`"2024-06-01T10:00:00Z","session":null,"id":{"hostId":"028ca108","id":"fdd41c10"},` +
`"stereotype":{"browserName":"chrome","platformName":"LINUX"}}]}]}}`
const selenoidStatusBody = `{"total":80,"used":10,"queued":0,"pending":1,"browsers":{"chrome":` +
`{"124.0":{"user1":{"count":1,"sessions":[{"id":"abc","container":"sel-abc"}]}}},"firefox":{"125.0":{}}}}`
func TestBrowserGridExposureModules(t *testing.T) {
const selenium = "../../modules/recon/selenium-grid-exposure.yaml"
const selenoid = "../../modules/recon/selenoid-exposure.yaml"
t.Run("a selenium grid status is flagged with a node version", func(t *testing.T) {
res := runGridModule(t, selenium, 200, seleniumStatusBody)
if len(res.Findings) == 0 {
t.Fatal("expected a selenium finding")
}
if v := gridExtract(res, "selenium_version"); v != "4.18.1 (revision b1d3319b48)" {
t.Errorf("selenium_version=%q, want the node build string", v)
}
})
t.Run("a value+nodes envelope without the selenium grid message is not flagged", func(t *testing.T) {
body := `{"value":{"ready":true,"nodes":[{"id":"n1","name":"router"}]}}`
if res := runGridModule(t, selenium, 200, body); len(res.Findings) > 0 {
t.Errorf("a generic value/nodes blob should not match selenium, got %d findings", len(res.Findings))
}
})
t.Run("a selenoid status is flagged with the first browser", func(t *testing.T) {
res := runGridModule(t, selenoid, 200, selenoidStatusBody)
if len(res.Findings) == 0 {
t.Fatal("expected a selenoid finding")
}
if v := gridExtract(res, "selenoid_browser"); v != "chrome" {
t.Errorf("selenoid_browser=%q, want chrome", v)
}
})
t.Run("a capacity blob without browsers is not flagged as selenoid", func(t *testing.T) {
if res := runGridModule(t, selenoid, 200, `{"total":80,"used":10,"queued":0,"pending":1}`); len(res.Findings) > 0 {
t.Errorf("a browserless capacity blob should not match selenoid, got %d findings", len(res.Findings))
}
})
t.Run("the two grid modules do not cross-match each other", func(t *testing.T) {
if res := runGridModule(t, selenoid, 200, seleniumStatusBody); len(res.Findings) > 0 {
t.Errorf("selenium status should not match selenoid, got %d findings", len(res.Findings))
}
if res := runGridModule(t, selenium, 200, selenoidStatusBody); len(res.Findings) > 0 {
t.Errorf("selenoid status should not match selenium, got %d findings", len(res.Findings))
}
})
t.Run("a plain 200 body is not a leak", func(t *testing.T) {
for _, file := range []string{selenium, selenoid} {
if res := runGridModule(t, file, 200, "ok"); len(res.Findings) > 0 {
t.Errorf("%s: a plain 200 body should not match, got %d findings", file, len(res.Findings))
}
}
})
t.Run("a 404 is not a leak", func(t *testing.T) {
for _, file := range []string{selenium, selenoid} {
if res := runGridModule(t, file, 404, "not found"); len(res.Findings) > 0 {
t.Errorf("%s: a 404 should not match, got %d findings", file, len(res.Findings))
}
}
})
}
@@ -19,7 +19,7 @@ import (
"testing"
"time"
"github.com/dropalldatabases/sif/internal/modules"
"github.com/vmfunc/sif/internal/modules"
)
func runBuildCredModule(t *testing.T, file string, status int, body string) *modules.Result {
@@ -0,0 +1,120 @@
package modules_test
import (
"context"
"net/http"
"net/http/httptest"
"testing"
"time"
"github.com/vmfunc/sif/internal/modules"
)
func runCICDModule(t *testing.T, file string, status int, body string) *modules.Result {
t.Helper()
def, err := modules.ParseYAMLModule(file)
if err != nil {
t.Fatalf("parse %s: %v", file, err)
}
srv := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
w.WriteHeader(status)
_, _ = w.Write([]byte(body))
}))
defer srv.Close()
res, err := modules.ExecuteHTTPModule(context.Background(), srv.URL, def, modules.Options{
Timeout: 5 * time.Second,
Threads: 2,
})
if err != nil {
t.Fatalf("execute %s: %v", file, err)
}
return res
}
func cicdExtract(res *modules.Result, key string) string {
for _, f := range res.Findings {
if v := f.Extracted[key]; v != "" {
return v
}
}
return ""
}
func TestCICDServerExposureModules(t *testing.T) {
const concourse = "../../modules/recon/concourse-info-exposure.yaml"
const woodpecker = "../../modules/recon/woodpecker-version-exposure.yaml"
const gocd = "../../modules/recon/gocd-version-exposure.yaml"
t.Run("a concourse info is flagged with its version", func(t *testing.T) {
body := `{"version":"7.11.2","worker_version":"2.4","external_url":"https://ci.example.com","cluster_name":"prod"}`
res := runCICDModule(t, concourse, 200, body)
if len(res.Findings) == 0 {
t.Fatal("expected a concourse finding")
}
if v := cicdExtract(res, "concourse_version"); v != "7.11.2" {
t.Errorf("concourse_version=%q, want 7.11.2", v)
}
})
t.Run("an info without worker_version is not flagged as concourse", func(t *testing.T) {
body := `{"version":"1.0","external_url":"https://x"}`
if res := runCICDModule(t, concourse, 200, body); len(res.Findings) > 0 {
t.Errorf("a body without worker_version should not match concourse, got %d findings", len(res.Findings))
}
})
t.Run("a woodpecker version is flagged with its version", func(t *testing.T) {
body := `{"source":"https://github.com/woodpecker-ci/woodpecker","version":"2.1.0"}`
res := runCICDModule(t, woodpecker, 200, body)
if len(res.Findings) == 0 {
t.Fatal("expected a woodpecker finding")
}
if v := cicdExtract(res, "woodpecker_version"); v != "2.1.0" {
t.Errorf("woodpecker_version=%q, want 2.1.0", v)
}
})
t.Run("a drone version with the same shape is not flagged as woodpecker", func(t *testing.T) {
body := `{"source":"https://github.com/harness/drone","version":"2.20.0"}`
if res := runCICDModule(t, woodpecker, 200, body); len(res.Findings) > 0 {
t.Errorf("a drone source should not match woodpecker, got %d findings", len(res.Findings))
}
})
t.Run("a gocd version is flagged with its version", func(t *testing.T) {
body := `{"_links":{"self":{"href":"https://ci/go/api/version"}},"version":"21.4.0",` +
`"build_number":"13183","git_sha":"abc123","full_version":"21.4.0 (13183-abc123)",` +
`"commit_url":"https://github.com/gocd/gocd/commit/abc123"}`
res := runCICDModule(t, gocd, 200, body)
if len(res.Findings) == 0 {
t.Fatal("expected a gocd finding")
}
if v := cicdExtract(res, "gocd_version"); v != "21.4.0" {
t.Errorf("gocd_version=%q, want 21.4.0", v)
}
})
t.Run("a version with a non-gocd commit url is not flagged as gocd", func(t *testing.T) {
body := `{"version":"1.0","full_version":"1.0 (1-x)","commit_url":"https://github.com/other/repo/commit/x"}`
if res := runCICDModule(t, gocd, 200, body); len(res.Findings) > 0 {
t.Errorf("a non-gocd commit url should not match gocd, got %d findings", len(res.Findings))
}
})
t.Run("a plain 200 body is not a leak", func(t *testing.T) {
for _, file := range []string{concourse, woodpecker, gocd} {
if res := runCICDModule(t, file, 200, "ok"); len(res.Findings) > 0 {
t.Errorf("%s: a plain 200 body should not match, got %d findings", file, len(res.Findings))
}
}
})
t.Run("a 404 is not a leak", func(t *testing.T) {
for _, file := range []string{concourse, woodpecker, gocd} {
if res := runCICDModule(t, file, 404, "not found"); len(res.Findings) > 0 {
t.Errorf("%s: a 404 should not match, got %d findings", file, len(res.Findings))
}
}
})
}
+1 -1
View File
@@ -7,7 +7,7 @@ import (
"testing"
"time"
"github.com/dropalldatabases/sif/internal/modules"
"github.com/vmfunc/sif/internal/modules"
)
func runCMSCfgModule(t *testing.T, file string, status int, body string) *modules.Result {
+1 -1
View File
@@ -7,7 +7,7 @@ import (
"testing"
"time"
"github.com/dropalldatabases/sif/internal/modules"
"github.com/vmfunc/sif/internal/modules"
)
func runCredModule(t *testing.T, file string, status int, body string) *modules.Result {
@@ -0,0 +1,99 @@
package modules_test
import (
"context"
"net/http"
"net/http/httptest"
"testing"
"time"
"github.com/vmfunc/sif/internal/modules"
)
func runDataOrchModule(t *testing.T, file string, status int, body string) *modules.Result {
t.Helper()
def, err := modules.ParseYAMLModule(file)
if err != nil {
t.Fatalf("parse %s: %v", file, err)
}
srv := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
w.WriteHeader(status)
_, _ = w.Write([]byte(body))
}))
defer srv.Close()
res, err := modules.ExecuteHTTPModule(context.Background(), srv.URL, def, modules.Options{
Timeout: 5 * time.Second,
Threads: 2,
})
if err != nil {
t.Fatalf("execute %s: %v", file, err)
}
return res
}
func dataOrchExtract(res *modules.Result, key string) string {
for _, f := range res.Findings {
if v := f.Extracted[key]; v != "" {
return v
}
}
return ""
}
func TestDataOrchestrationExposureModules(t *testing.T) {
const dagster = "../../modules/recon/dagster-webserver-exposure.yaml"
const mage = "../../modules/recon/mage-status-exposure.yaml"
t.Run("a dagster server_info is flagged with its version", func(t *testing.T) {
body := `{"dagster_webserver_version":"1.7.0","dagster_version":"1.7.0","dagster_graphql_version":"1.7.0"}`
res := runDataOrchModule(t, dagster, 200, body)
if len(res.Findings) == 0 {
t.Fatal("expected a dagster finding")
}
if v := dataOrchExtract(res, "dagster_version"); v != "1.7.0" {
t.Errorf("dagster_version=%q, want 1.7.0", v)
}
})
t.Run("a bare core-version body is not flagged as dagster", func(t *testing.T) {
if res := runDataOrchModule(t, dagster, 200, `{"dagster_version":"1.7.0"}`); len(res.Findings) > 0 {
t.Errorf("a body without dagster_webserver_version should not match, got %d findings", len(res.Findings))
}
})
t.Run("a mage status is flagged with its repo path", func(t *testing.T) {
body := `{"statuses":[{"is_instance_manager":false,"repo_path":"/home/src/default_repo",` +
`"repo_path_relative":"default_repo","scheduler_status":"running","project_type":"standalone",` +
`"project_uuid":"abc-123"}]}`
res := runDataOrchModule(t, mage, 200, body)
if len(res.Findings) == 0 {
t.Fatal("expected a mage finding")
}
if v := dataOrchExtract(res, "mage_repo_path"); v != "/home/src/default_repo" {
t.Errorf("mage_repo_path=%q, want /home/src/default_repo", v)
}
})
t.Run("a statuses collection without scheduler fields is not flagged as mage", func(t *testing.T) {
if res := runDataOrchModule(t, mage, 200, `{"statuses":[{"id":1,"name":"ok"}]}`); len(res.Findings) > 0 {
t.Errorf("a generic statuses array should not match mage, got %d findings", len(res.Findings))
}
})
t.Run("a plain 200 body is not a leak", func(t *testing.T) {
for _, file := range []string{dagster, mage} {
if res := runDataOrchModule(t, file, 200, "ok"); len(res.Findings) > 0 {
t.Errorf("%s: a plain 200 body should not match, got %d findings", file, len(res.Findings))
}
}
})
t.Run("a 404 is not a leak", func(t *testing.T) {
for _, file := range []string{dagster, mage} {
if res := runDataOrchModule(t, file, 404, "not found"); len(res.Findings) > 0 {
t.Errorf("%s: a 404 should not match, got %d findings", file, len(res.Findings))
}
}
})
}
@@ -7,7 +7,7 @@ import (
"testing"
"time"
"github.com/dropalldatabases/sif/internal/modules"
"github.com/vmfunc/sif/internal/modules"
)
func runPipelineModule(t *testing.T, file string, status int, body string) *modules.Result {
@@ -8,7 +8,7 @@ import (
"testing"
"time"
"github.com/dropalldatabases/sif/internal/modules"
"github.com/vmfunc/sif/internal/modules"
)
func runDBFileModule(t *testing.T, file string, status int, body string) *modules.Result {
@@ -0,0 +1,111 @@
package modules_test
import (
"context"
"net/http"
"net/http/httptest"
"testing"
"time"
"github.com/vmfunc/sif/internal/modules"
)
func runDBHTTPModule(t *testing.T, file string, status int, body string) *modules.Result {
t.Helper()
def, err := modules.ParseYAMLModule(file)
if err != nil {
t.Fatalf("parse %s: %v", file, err)
}
srv := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
w.WriteHeader(status)
_, _ = w.Write([]byte(body))
}))
defer srv.Close()
res, err := modules.ExecuteHTTPModule(context.Background(), srv.URL, def, modules.Options{
Timeout: 5 * time.Second,
Threads: 2,
})
if err != nil {
t.Fatalf("execute %s: %v", file, err)
}
return res
}
func dbHTTPExtract(res *modules.Result, key string) string {
for _, f := range res.Findings {
if v := f.Extracted[key]; v != "" {
return v
}
}
return ""
}
func TestDatabaseHTTPExposureModules(t *testing.T) {
const clickhouse = "../../modules/recon/clickhouse-http-exposure.yaml"
const dgraph = "../../modules/recon/dgraph-api-exposure.yaml"
t.Run("a clickhouse FORMAT JSON result is flagged with the version", func(t *testing.T) {
body := `{"meta":[{"name":"version()","type":"String"}],"data":[{"version()":"24.3.1.2672"}],` +
`"rows":1,"statistics":{"elapsed":0.000123,"rows_read":1,"bytes_read":1}}`
res := runDBHTTPModule(t, clickhouse, 200, body)
if len(res.Findings) == 0 {
t.Fatal("expected a clickhouse finding")
}
if v := dbHTTPExtract(res, "clickhouse_version"); v != "24.3.1.2672" {
t.Errorf("clickhouse_version=%q, want 24.3.1.2672", v)
}
})
t.Run("a json result without the statistics envelope is not flagged as clickhouse", func(t *testing.T) {
body := `{"meta":[{"name":"x"}],"data":[{"x":1}],"rows":1}`
if res := runDBHTTPModule(t, clickhouse, 200, body); len(res.Findings) > 0 {
t.Errorf("a statless json result should not match clickhouse, got %d findings", len(res.Findings))
}
})
t.Run("a dgraph alpha health is flagged with its version", func(t *testing.T) {
body := `[{"instance":"alpha","address":"localhost:7080","status":"healthy","group":"0",` +
`"version":"v23.1.0","uptime":3600,"lastEcho":1700000000,"ongoing":["opRollup"],"max_assigned":30002}]`
res := runDBHTTPModule(t, dgraph, 200, body)
if len(res.Findings) == 0 {
t.Fatal("expected a dgraph finding")
}
if v := dbHTTPExtract(res, "dgraph_version"); v != "v23.1.0" {
t.Errorf("dgraph_version=%q, want v23.1.0", v)
}
})
t.Run("a dgraph alpha health without max_assigned is not flagged", func(t *testing.T) {
body := `[{"instance":"alpha","status":"healthy","lastEcho":1700000000}]`
if res := runDBHTTPModule(t, dgraph, 200, body); len(res.Findings) > 0 {
t.Errorf("a partial alpha health should not match dgraph, got %d findings", len(res.Findings))
}
})
t.Run("a non-alpha instance health is not flagged as dgraph", func(t *testing.T) {
body := `[{"instance":"zero","max_assigned":30002,"lastEcho":1700000000}]`
if res := runDBHTTPModule(t, dgraph, 200, body); len(res.Findings) > 0 {
t.Errorf("a zero-node health should not match dgraph alpha, got %d findings", len(res.Findings))
}
})
t.Run("a plain 200 body is not a leak", func(t *testing.T) {
for _, file := range []string{clickhouse, dgraph} {
if res := runDBHTTPModule(t, file, 200, "ok"); len(res.Findings) > 0 {
t.Errorf("%s: a plain 200 body should not match, got %d findings", file, len(res.Findings))
}
}
})
t.Run("a 403 or 404 is not a leak", func(t *testing.T) {
if res := runDBHTTPModule(t, clickhouse, 403, "Authentication failed"); len(res.Findings) > 0 {
t.Errorf("a 403 clickhouse should not match, got %d findings", len(res.Findings))
}
for _, file := range []string{clickhouse, dgraph} {
if res := runDBHTTPModule(t, file, 404, "not found"); len(res.Findings) > 0 {
t.Errorf("%s: a 404 should not match, got %d findings", file, len(res.Findings))
}
}
})
}
+1 -1
View File
@@ -7,7 +7,7 @@ import (
"testing"
"time"
"github.com/dropalldatabases/sif/internal/modules"
"github.com/vmfunc/sif/internal/modules"
)
func runDBModule(t *testing.T, file string, status int, headers map[string]string, body string) *modules.Result {
+1 -1
View File
@@ -7,7 +7,7 @@ import (
"testing"
"time"
"github.com/dropalldatabases/sif/internal/modules"
"github.com/vmfunc/sif/internal/modules"
)
func runDebugModule(t *testing.T, file string, status int, body string) *modules.Result {
@@ -7,7 +7,7 @@ import (
"testing"
"time"
"github.com/dropalldatabases/sif/internal/modules"
"github.com/vmfunc/sif/internal/modules"
)
func runDeployModule(t *testing.T, file string, status int, body string) *modules.Result {
@@ -7,7 +7,7 @@ import (
"testing"
"time"
"github.com/dropalldatabases/sif/internal/modules"
"github.com/vmfunc/sif/internal/modules"
)
func runDistDBModule(t *testing.T, file string, status int, body string) *modules.Result {
@@ -7,7 +7,7 @@ import (
"testing"
"time"
"github.com/dropalldatabases/sif/internal/modules"
"github.com/vmfunc/sif/internal/modules"
)
func runDotfileModule(t *testing.T, file string, status int, body string) *modules.Result {
+1 -1
View File
@@ -19,7 +19,7 @@ import (
"testing"
"time"
"github.com/dropalldatabases/sif/internal/modules"
"github.com/vmfunc/sif/internal/modules"
)
// runEnvModule runs the env exposure module end to end against a server that
+1 -1
View File
@@ -22,7 +22,7 @@ import (
"testing"
"time"
"github.com/dropalldatabases/sif/internal/httpx"
"github.com/vmfunc/sif/internal/httpx"
)
const testTimeout = 5 * time.Second
+1 -1
View File
@@ -16,7 +16,7 @@ import (
"fmt"
"math"
"github.com/dropalldatabases/sif/internal/fingerprint"
"github.com/vmfunc/sif/internal/fingerprint"
)
// checkFaviconHash reports whether the body's shodan mmh3 hash matches any
+2 -2
View File
@@ -21,8 +21,8 @@ import (
"strings"
"testing"
"github.com/dropalldatabases/sif/internal/fingerprint"
"github.com/dropalldatabases/sif/internal/httpx"
"github.com/vmfunc/sif/internal/fingerprint"
"github.com/vmfunc/sif/internal/httpx"
)
// faviconFixture hashes to a negative int32, so its signed and unsigned forms
@@ -0,0 +1,93 @@
package modules_test
import (
"context"
"net/http"
"net/http/httptest"
"testing"
"time"
"github.com/vmfunc/sif/internal/modules"
)
func runGrafanaModule(t *testing.T, file string, status int, body string) *modules.Result {
t.Helper()
def, err := modules.ParseYAMLModule(file)
if err != nil {
t.Fatalf("parse %s: %v", file, err)
}
srv := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
w.WriteHeader(status)
_, _ = w.Write([]byte(body))
}))
defer srv.Close()
res, err := modules.ExecuteHTTPModule(context.Background(), srv.URL, def, modules.Options{
Timeout: 5 * time.Second,
Threads: 2,
})
if err != nil {
t.Fatalf("execute %s: %v", file, err)
}
return res
}
func grafanaExtract(res *modules.Result, key string) string {
for _, f := range res.Findings {
if v := f.Extracted[key]; v != "" {
return v
}
}
return ""
}
func TestGrafanaAnonymousExposureModule(t *testing.T) {
const grafana = "../../modules/recon/grafana-anonymous-exposure.yaml"
t.Run("an anonymous search result is flagged with a dashboard title", func(t *testing.T) {
body := `[{"id":1,"uid":"abc123","title":"Production Overview","uri":"db/production-overview",` +
`"url":"/d/abc123/production-overview","slug":"","type":"dash-db","tags":["prod"],` +
`"isStarred":false,"sortMeta":0}]`
res := runGrafanaModule(t, grafana, 200, body)
if len(res.Findings) == 0 {
t.Fatal("expected a grafana finding")
}
if v := grafanaExtract(res, "grafana_dashboard"); v != "Production Overview" {
t.Errorf("grafana_dashboard=%q, want Production Overview", v)
}
})
t.Run("a folder-only result is not flagged", func(t *testing.T) {
body := `[{"id":1,"uid":"f","title":"General","uri":"db/general","url":"/dashboards/f/general",` +
`"type":"dash-folder","isStarred":false}]`
if res := runGrafanaModule(t, grafana, 200, body); len(res.Findings) > 0 {
t.Errorf("a folder-only search should not match, got %d findings", len(res.Findings))
}
})
t.Run("a dash-db result without isStarred is not flagged", func(t *testing.T) {
body := `[{"uid":"abc","title":"x","uri":"db/x","type":"dash-db"}]`
if res := runGrafanaModule(t, grafana, 200, body); len(res.Findings) > 0 {
t.Errorf("a partial search blob should not match, got %d findings", len(res.Findings))
}
})
t.Run("a login-required grafana is not flagged", func(t *testing.T) {
body := `{"message":"Unauthorized"}`
if res := runGrafanaModule(t, grafana, 401, body); len(res.Findings) > 0 {
t.Errorf("a 401 grafana should not match, got %d findings", len(res.Findings))
}
})
t.Run("a plain 200 body is not a leak", func(t *testing.T) {
if res := runGrafanaModule(t, grafana, 200, "ok"); len(res.Findings) > 0 {
t.Errorf("a plain 200 body should not match, got %d findings", len(res.Findings))
}
})
t.Run("a 404 is not a leak", func(t *testing.T) {
if res := runGrafanaModule(t, grafana, 404, "not found"); len(res.Findings) > 0 {
t.Errorf("a 404 should not match, got %d findings", len(res.Findings))
}
})
}
@@ -0,0 +1,128 @@
package modules_test
import (
"context"
"net/http"
"net/http/httptest"
"testing"
"time"
"github.com/vmfunc/sif/internal/modules"
)
const zkMonitorBody = `{"command":"monitor","error":null,"version":"3.9.2-a8d7f3e",` +
`"server_state":"leader","avg_latency":0,"max_latency":12,"min_latency":0,` +
`"packets_received":1532,"packets_sent":1531,"num_alive_connections":3,` +
`"outstanding_requests":0,"znode_count":127,"watch_count":4,"ephemerals_count":2,` +
`"approximate_data_size":45219,"open_file_descriptor_count":67,"max_file_descriptor_count":1048576}`
const nameNodeInfoBody = `{"beans":[{"name":"Hadoop:service=NameNode,name=NameNodeInfo",` +
`"modelerType":"org.apache.hadoop.hdfs.server.namenode.FSNamesystem",` +
`"SoftwareVersion":"3.3.6","Version":"3.3.6, rUNKNOWN","Total":2147483648,"Free":1073741824,` +
`"Safemode":"","ClusterId":"CID-abc123","BlockPoolId":"BP-998-10.0.0.1-170",` +
`"LiveNodes":"{\"dn1.hdfs.internal:9866\":{\"infoAddr\":\"10.0.0.2:9864\"}}",` +
`"DeadNodes":"{}","DecomNodes":"{}","TotalBlocks":1024,"TotalFiles":512}]}`
func runHadoopZKModule(t *testing.T, file string, status int, body string) *modules.Result {
t.Helper()
def, err := modules.ParseYAMLModule(file)
if err != nil {
t.Fatalf("parse %s: %v", file, err)
}
srv := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
w.WriteHeader(status)
_, _ = w.Write([]byte(body))
}))
defer srv.Close()
res, err := modules.ExecuteHTTPModule(context.Background(), srv.URL, def, modules.Options{
Timeout: 5 * time.Second,
Threads: 2,
})
if err != nil {
t.Fatalf("execute %s: %v", file, err)
}
return res
}
func hadoopZKExtract(res *modules.Result, key string) string {
for _, f := range res.Findings {
if v := f.Extracted[key]; v != "" {
return v
}
}
return ""
}
func TestHadoopZooKeeperExposureModules(t *testing.T) {
const zk = "../../modules/recon/zookeeper-admin-exposure.yaml"
const namenode = "../../modules/recon/hadoop-namenode-exposure.yaml"
t.Run("an open zookeeper monitor is flagged with the version", func(t *testing.T) {
res := runHadoopZKModule(t, zk, 200, zkMonitorBody)
if len(res.Findings) == 0 {
t.Fatal("expected a zookeeper finding")
}
if v := hadoopZKExtract(res, "zookeeper_version"); v != "3.9.2-a8d7f3e" {
t.Errorf("zookeeper_version=%q, want 3.9.2-a8d7f3e", v)
}
})
t.Run("a different adminserver command is not flagged", func(t *testing.T) {
body := `{"command":"configuration","error":null,"version":"3.9.2","clientPort":2181}`
if res := runHadoopZKModule(t, zk, 200, body); len(res.Findings) > 0 {
t.Errorf("a non-monitor command should not match, got %d findings", len(res.Findings))
}
})
t.Run("a firewalled or absent adminserver is not flagged", func(t *testing.T) {
if res := runHadoopZKModule(t, zk, 404, "not found"); len(res.Findings) > 0 {
t.Errorf("a 404 should not match zookeeper, got %d findings", len(res.Findings))
}
})
t.Run("a namenode jmx body does not match the zookeeper module", func(t *testing.T) {
if res := runHadoopZKModule(t, zk, 200, nameNodeInfoBody); len(res.Findings) > 0 {
t.Errorf("a namenode body should not match zookeeper, got %d findings", len(res.Findings))
}
})
t.Run("an open namenode jmx is flagged with the hdfs version", func(t *testing.T) {
res := runHadoopZKModule(t, namenode, 200, nameNodeInfoBody)
if len(res.Findings) == 0 {
t.Fatal("expected a namenode finding")
}
if v := hadoopZKExtract(res, "hdfs_version"); v != "3.3.6" {
t.Errorf("hdfs_version=%q, want 3.3.6", v)
}
})
t.Run("a different hadoop jmx bean is not flagged", func(t *testing.T) {
body := `{"beans":[{"name":"Hadoop:service=DataNode,name=DataNodeInfo",` +
`"SoftwareVersion":"3.3.6","LiveNodes":"","DeadNodes":""}]}`
if res := runHadoopZKModule(t, namenode, 200, body); len(res.Findings) > 0 {
t.Errorf("a non-NameNodeInfo bean should not match, got %d findings", len(res.Findings))
}
})
t.Run("a kerberos-secured namenode is not flagged", func(t *testing.T) {
if res := runHadoopZKModule(t, namenode, 401, "Authentication required"); len(res.Findings) > 0 {
t.Errorf("a 401 namenode should not match, got %d findings", len(res.Findings))
}
})
t.Run("a zookeeper monitor body does not match the namenode module", func(t *testing.T) {
if res := runHadoopZKModule(t, namenode, 200, zkMonitorBody); len(res.Findings) > 0 {
t.Errorf("a zookeeper body should not match namenode, got %d findings", len(res.Findings))
}
})
t.Run("plain 200 bodies are not leaks", func(t *testing.T) {
if res := runHadoopZKModule(t, zk, 200, "ok"); len(res.Findings) > 0 {
t.Errorf("a plain 200 should not match zookeeper, got %d findings", len(res.Findings))
}
if res := runHadoopZKModule(t, namenode, 200, "ok"); len(res.Findings) > 0 {
t.Errorf("a plain 200 should not match namenode, got %d findings", len(res.Findings))
}
})
}
@@ -7,7 +7,7 @@ import (
"testing"
"time"
"github.com/dropalldatabases/sif/internal/modules"
"github.com/vmfunc/sif/internal/modules"
)
func runHTTPDBModule(t *testing.T, file string, status int, body string) *modules.Result {
+195
View File
@@ -0,0 +1,195 @@
package modules_test
import (
"context"
"net/http"
"net/http/httptest"
"testing"
"time"
"github.com/vmfunc/sif/internal/modules"
)
func runImageGenModule(t *testing.T, file string, status int, body string) *modules.Result {
t.Helper()
def, err := modules.ParseYAMLModule(file)
if err != nil {
t.Fatalf("parse %s: %v", file, err)
}
srv := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
w.WriteHeader(status)
_, _ = w.Write([]byte(body))
}))
defer srv.Close()
res, err := modules.ExecuteHTTPModule(context.Background(), srv.URL, def, modules.Options{
Timeout: 5 * time.Second,
Threads: 2,
})
if err != nil {
t.Fatalf("execute %s: %v", file, err)
}
return res
}
func imageGenExtract(res *modules.Result, key string) string {
for _, f := range res.Findings {
if v := f.Extracted[key]; v != "" {
return v
}
}
return ""
}
func TestImageGenExposureModules(t *testing.T) {
const comfyui = "../../modules/recon/comfyui-api-exposure.yaml"
const a1111 = "../../modules/recon/automatic1111-api-exposure.yaml"
const fooocus = "../../modules/recon/fooocus-api-exposure.yaml"
const iopaint = "../../modules/recon/iopaint-api-exposure.yaml"
comfyStats := `{"system":{"os":"posix","ram_total":67430219776,"ram_free":12345678,` +
`"comfyui_version":"0.3.40","python_version":"3.11.9","pytorch_version":"2.3.1",` +
`"embedded_python":false,"argv":["main.py"]},"devices":[{"name":"cuda:0 NVIDIA RTX 4090",` +
`"type":"cuda","index":0,"vram_total":25757220864,"vram_free":24000000000,` +
`"torch_vram_total":268435456,"torch_vram_free":260000000}]}`
a1111Models := `[{"title":"sd_xl_base_1.0.safetensors [31e35c80fc]","model_name":"sd_xl_base_1.0",` +
`"hash":"31e35c80fc","sha256":"31e35c80fc4829d14f90153f4c74cd59c90b779f6afe05a74cd6120b893f7e5b",` +
`"filename":"/home/sd/models/Stable-diffusion/sd_xl_base_1.0.safetensors","config":null}]`
fooocusModels := `{"model_filenames":["juggernautXL_v8.safetensors","sd_xl_base_1.0.safetensors"],` +
`"lora_filenames":["sdxl_lcm_lora.safetensors"]}`
iopaintConfig := `{"plugins":[{"name":"RemoveBG","support_gen_image":true,"support_gen_mask":false}],` +
`"modelInfos":[{"name":"lama","path":"lama","model_type":"inpaint"}],` +
`"removeBGModel":"briaai/RMBG-1.4","removeBGModels":["briaai/RMBG-1.4","u2net"],` +
`"realesrganModel":"realesr-general-x4v3","realesrganModels":["realesr-general-x4v3"],` +
`"interactiveSegModel":"sam2_1_tiny","interactiveSegModels":["vit_b","sam2_1_tiny"],` +
`"enableFileManager":true,"enableAutoSaving":false,"enableControlnet":false,` +
`"controlnetMethod":null,"disableModelSwitch":false,"isDesktop":false,` +
`"samplers":["DPM++ 2M","Euler","Euler a"]}`
t.Run("a comfyui system_stats is flagged with its version", func(t *testing.T) {
res := runImageGenModule(t, comfyui, 200, comfyStats)
if len(res.Findings) == 0 {
t.Fatal("expected a comfyui finding")
}
if v := imageGenExtract(res, "comfyui_version"); v != "0.3.40" {
t.Errorf("comfyui_version=%q, want 0.3.40", v)
}
})
t.Run("a system_stats without comfyui keys is not flagged", func(t *testing.T) {
body := `{"system":{"os":"linux","ram_total":123},"devices":[]}`
if res := runImageGenModule(t, comfyui, 200, body); len(res.Findings) > 0 {
t.Errorf("a generic system stats should not match comfyui, got %d findings", len(res.Findings))
}
})
t.Run("a comfyui version without device memory is not flagged", func(t *testing.T) {
body := `{"system":{"comfyui_version":"0.3.40"}}`
if res := runImageGenModule(t, comfyui, 200, body); len(res.Findings) > 0 {
t.Errorf("a version-only body should not match comfyui, got %d findings", len(res.Findings))
}
})
t.Run("an automatic1111 sd-models list is flagged with its checkpoint", func(t *testing.T) {
res := runImageGenModule(t, a1111, 200, a1111Models)
if len(res.Findings) == 0 {
t.Fatal("expected an automatic1111 finding")
}
if v := imageGenExtract(res, "sd_model_name"); v != "sd_xl_base_1.0" {
t.Errorf("sd_model_name=%q, want sd_xl_base_1.0", v)
}
})
t.Run("a list with a model_name but no filename is not flagged as a1111", func(t *testing.T) {
body := `[{"title":"some entry","model_name":"thing"}]`
if res := runImageGenModule(t, a1111, 200, body); len(res.Findings) > 0 {
t.Errorf("a partial entry should not match automatic1111, got %d findings", len(res.Findings))
}
})
t.Run("a generic titled list is not flagged as a1111", func(t *testing.T) {
body := `[{"title":"My Blog Post","filename":"post.md","author":"someone"}]`
if res := runImageGenModule(t, a1111, 200, body); len(res.Findings) > 0 {
t.Errorf("a generic list should not match automatic1111, got %d findings", len(res.Findings))
}
})
t.Run("a model_name and filename without a title is not flagged as a1111", func(t *testing.T) {
body := `[{"model_name":"thing","filename":"/tmp/thing.bin"}]`
if res := runImageGenModule(t, a1111, 200, body); len(res.Findings) > 0 {
t.Errorf("a titleless entry should not match automatic1111, got %d findings", len(res.Findings))
}
})
t.Run("a fooocus-api all-models is flagged with its checkpoint", func(t *testing.T) {
res := runImageGenModule(t, fooocus, 200, fooocusModels)
if len(res.Findings) == 0 {
t.Fatal("expected a fooocus finding")
}
if v := imageGenExtract(res, "fooocus_model"); v != "juggernautXL_v8.safetensors" {
t.Errorf("fooocus_model=%q, want juggernautXL_v8.safetensors", v)
}
})
t.Run("a body without model_filenames is not flagged as fooocus", func(t *testing.T) {
if res := runImageGenModule(t, fooocus, 200, `{"lora_filenames":["x.safetensors"]}`); len(res.Findings) > 0 {
t.Errorf("a body without model_filenames should not match fooocus, got %d findings", len(res.Findings))
}
})
t.Run("a body without lora_filenames is not flagged as fooocus", func(t *testing.T) {
if res := runImageGenModule(t, fooocus, 200, `{"model_filenames":["x.safetensors"]}`); len(res.Findings) > 0 {
t.Errorf("a body without lora_filenames should not match fooocus, got %d findings", len(res.Findings))
}
})
t.Run("an iopaint server-config is flagged with its file-manager state", func(t *testing.T) {
res := runImageGenModule(t, iopaint, 200, iopaintConfig)
if len(res.Findings) == 0 {
t.Fatal("expected an iopaint finding")
}
if v := imageGenExtract(res, "iopaint_file_manager"); v != "true" {
t.Errorf("iopaint_file_manager=%q, want true", v)
}
})
t.Run("a config without interactiveSegModel is not flagged as iopaint", func(t *testing.T) {
body := `{"modelInfos":[],"enableFileManager":true,"disableModelSwitch":false,"samplers":[]}`
if res := runImageGenModule(t, iopaint, 200, body); len(res.Findings) > 0 {
t.Errorf("a body without interactiveSegModel should not match iopaint, got %d findings", len(res.Findings))
}
})
t.Run("a config without enableFileManager is not flagged as iopaint", func(t *testing.T) {
body := `{"interactiveSegModel":"vit_b","disableModelSwitch":false,"samplers":[]}`
if res := runImageGenModule(t, iopaint, 200, body); len(res.Findings) > 0 {
t.Errorf("a body without enableFileManager should not match iopaint, got %d findings", len(res.Findings))
}
})
t.Run("a config without disableModelSwitch is not flagged as iopaint", func(t *testing.T) {
body := `{"interactiveSegModel":"vit_b","enableFileManager":true,"samplers":[]}`
if res := runImageGenModule(t, iopaint, 200, body); len(res.Findings) > 0 {
t.Errorf("a body without disableModelSwitch should not match iopaint, got %d findings", len(res.Findings))
}
})
t.Run("a plain 200 body is not a leak", func(t *testing.T) {
for _, file := range []string{comfyui, a1111, fooocus, iopaint} {
if res := runImageGenModule(t, file, 200, "ok"); len(res.Findings) > 0 {
t.Errorf("%s: a plain 200 body should not match, got %d findings", file, len(res.Findings))
}
}
})
t.Run("a 404 is not a leak", func(t *testing.T) {
for _, file := range []string{comfyui, a1111, fooocus, iopaint} {
if res := runImageGenModule(t, file, 404, "not found"); len(res.Findings) > 0 {
t.Errorf("%s: a 404 should not match, got %d findings", file, len(res.Findings))
}
}
})
}
@@ -19,7 +19,7 @@ import (
"testing"
"time"
"github.com/dropalldatabases/sif/internal/modules"
"github.com/vmfunc/sif/internal/modules"
)
func runInfraModule(t *testing.T, file string, status int, body string) *modules.Result {
@@ -0,0 +1,129 @@
package modules_test
import (
"context"
"net/http"
"net/http/httptest"
"testing"
"time"
"github.com/vmfunc/sif/internal/modules"
)
func runInfraControlplaneModule(t *testing.T, file string, status int, body string) *modules.Result {
t.Helper()
def, err := modules.ParseYAMLModule(file)
if err != nil {
t.Fatalf("parse %s: %v", file, err)
}
srv := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
w.WriteHeader(status)
_, _ = w.Write([]byte(body))
}))
defer srv.Close()
res, err := modules.ExecuteHTTPModule(context.Background(), srv.URL, def, modules.Options{
Timeout: 5 * time.Second,
Threads: 2,
})
if err != nil {
t.Fatalf("execute %s: %v", file, err)
}
return res
}
func controlplaneExtract(res *modules.Result, key string) string {
for _, f := range res.Findings {
if v := f.Extracted[key]; v != "" {
return v
}
}
return ""
}
func TestInfraControlplaneExposureModules(t *testing.T) {
const traefik = "../../modules/recon/traefik-api-exposure.yaml"
const nomad = "../../modules/recon/nomad-agent-exposure.yaml"
const portainer = "../../modules/recon/portainer-status-exposure.yaml"
t.Run("a traefik overview is flagged with its first provider", func(t *testing.T) {
body := `{"http":{"routers":{"total":12,"warnings":0,"errors":1},"services":{"total":8,"warnings":0,` +
`"errors":0},"middlewares":{"total":5,"warnings":0,"errors":0}},"tcp":{"routers":{"total":0},` +
`"services":{"total":0}},"udp":{"routers":{"total":0},"services":{"total":0}},` +
`"features":{"tracing":"Noop","metrics":"Prometheus","accessLog":true},"providers":["Docker","File"]}`
res := runInfraControlplaneModule(t, traefik, 200, body)
if len(res.Findings) == 0 {
t.Fatal("expected a traefik finding")
}
if v := controlplaneExtract(res, "traefik_provider"); v != "Docker" {
t.Errorf("traefik_provider=%q, want Docker", v)
}
})
t.Run("a routing summary without features is not flagged as traefik", func(t *testing.T) {
body := `{"http":{"routers":{"total":1}},"providers":["Docker"]}`
if res := runInfraControlplaneModule(t, traefik, 200, body); len(res.Findings) > 0 {
t.Errorf("a body without features should not match traefik, got %d findings", len(res.Findings))
}
})
t.Run("an open nomad agent self is flagged with its version", func(t *testing.T) {
body := `{"config":{"Region":"global","Datacenter":"dc1","BindAddr":"0.0.0.0"},` +
`"member":{"Name":"node1.global","Addr":"10.0.0.5","Port":4648,` +
`"Tags":{"role":"nomad","region":"global","dc":"dc1","build":"1.7.2","vsn":"1"},"Status":"alive"},` +
`"stats":{"nomad":{"server":"true"},"runtime":{"version":"go1.21"}}}`
res := runInfraControlplaneModule(t, nomad, 200, body)
if len(res.Findings) == 0 {
t.Fatal("expected a nomad finding")
}
if v := controlplaneExtract(res, "nomad_version"); v != "1.7.2" {
t.Errorf("nomad_version=%q, want 1.7.2", v)
}
})
t.Run("an acl-enabled nomad returns 403 and is not flagged", func(t *testing.T) {
if res := runInfraControlplaneModule(t, nomad, 403, `{"errors":["Permission denied"]}`); len(res.Findings) > 0 {
t.Errorf("a 403 from an acl-enabled nomad should not match, got %d findings", len(res.Findings))
}
})
t.Run("a config+stats body without member is not flagged as nomad", func(t *testing.T) {
body := `{"config":{"a":1},"stats":{"b":2}}`
if res := runInfraControlplaneModule(t, nomad, 200, body); len(res.Findings) > 0 {
t.Errorf("a body without member should not match nomad, got %d findings", len(res.Findings))
}
})
t.Run("a portainer status is flagged with its instance id", func(t *testing.T) {
body := `{"Version":"2.19.4","InstanceID":"299ab403-70a8-4c05-92f7-bf7a994d50df"}`
res := runInfraControlplaneModule(t, portainer, 200, body)
if len(res.Findings) == 0 {
t.Fatal("expected a portainer finding")
}
if v := controlplaneExtract(res, "portainer_instance_id"); v != "299ab403-70a8-4c05-92f7-bf7a994d50df" {
t.Errorf("portainer_instance_id=%q, want the uuid", v)
}
})
t.Run("a bare version body is not flagged as portainer", func(t *testing.T) {
if res := runInfraControlplaneModule(t, portainer, 200, `{"Version":"2.19.4"}`); len(res.Findings) > 0 {
t.Errorf("a bare version should not match portainer, got %d findings", len(res.Findings))
}
})
t.Run("a plain 200 body is not a leak", func(t *testing.T) {
for _, file := range []string{traefik, nomad, portainer} {
if res := runInfraControlplaneModule(t, file, 200, "ok"); len(res.Findings) > 0 {
t.Errorf("%s: a plain 200 body should not match, got %d findings", file, len(res.Findings))
}
}
})
t.Run("a 404 is not a leak", func(t *testing.T) {
for _, file := range []string{traefik, nomad, portainer} {
if res := runInfraControlplaneModule(t, file, 404, "not found"); len(res.Findings) > 0 {
t.Errorf("%s: a 404 should not match, got %d findings", file, len(res.Findings))
}
}
})
}
@@ -0,0 +1,126 @@
package modules_test
import (
"context"
"net/http"
"net/http/httptest"
"testing"
"time"
"github.com/vmfunc/sif/internal/modules"
)
func runJobDashModule(t *testing.T, file string, status int, body string) *modules.Result {
t.Helper()
def, err := modules.ParseYAMLModule(file)
if err != nil {
t.Fatalf("parse %s: %v", file, err)
}
srv := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
w.WriteHeader(status)
_, _ = w.Write([]byte(body))
}))
defer srv.Close()
res, err := modules.ExecuteHTTPModule(context.Background(), srv.URL, def, modules.Options{
Timeout: 5 * time.Second,
Threads: 2,
})
if err != nil {
t.Fatalf("execute %s: %v", file, err)
}
return res
}
func jobDashExtract(res *modules.Result, key string) string {
for _, f := range res.Findings {
if v := f.Extracted[key]; v != "" {
return v
}
}
return ""
}
func TestJobDashboardExposureModules(t *testing.T) {
const sidekiq = "../../modules/recon/sidekiq-web-exposure.yaml"
const flower = "../../modules/recon/celery-flower-exposure.yaml"
const rq = "../../modules/recon/rq-dashboard-exposure.yaml"
t.Run("a sidekiq stats dump is flagged with its redis version", func(t *testing.T) {
body := `{"sidekiq":{"processed":12345,"failed":67,"busy":3,"processes":2,"enqueued":10,` +
`"scheduled":5,"retries":1,"dead":0,"default_latency":0},"redis":{"redis_version":"7.2.4",` +
`"uptime_in_days":"12","connected_clients":"8","used_memory_human":"2.50M",` +
`"used_memory_peak_human":"3.10M"},"server_utc_time":"18:00:00 UTC"}`
res := runJobDashModule(t, sidekiq, 200, body)
if len(res.Findings) == 0 {
t.Fatal("expected a sidekiq finding")
}
if v := jobDashExtract(res, "redis_version"); v != "7.2.4" {
t.Errorf("redis_version=%q, want 7.2.4", v)
}
})
t.Run("a bare redis-info body without default_latency is not flagged as sidekiq", func(t *testing.T) {
if res := runJobDashModule(t, sidekiq, 200, `{"redis_version":"7.2.4","server_utc_time":"x"}`); len(res.Findings) > 0 {
t.Errorf("a redis info blob should not match sidekiq, got %d findings", len(res.Findings))
}
})
t.Run("a flower workers api is flagged with the celery version", func(t *testing.T) {
body := `{"celery@worker1":{"active_queues":[{"name":"celery","exchange":{"name":"celery",` +
`"type":"direct"},"routing_key":"celery"}],"conf":{"broker_url":"redis://localhost:6379/0",` +
`"result_backend":"redis://localhost:6379/0"},"registered":["tasks.add","tasks.send_email"],` +
`"stats":{"sw_ident":"py-celery","sw_ver":"5.3.6","sw_sys":"Linux","pool":{"max-concurrency":4},` +
`"broker":{"hostname":"localhost","transport":"redis"}},"timestamp":1719345600.0}}`
res := runJobDashModule(t, flower, 200, body)
if len(res.Findings) == 0 {
t.Fatal("expected a flower finding")
}
if v := jobDashExtract(res, "celery_version"); v != "5.3.6" {
t.Errorf("celery_version=%q, want 5.3.6", v)
}
})
t.Run("a worker blob without conf is not flagged as flower", func(t *testing.T) {
if res := runJobDashModule(t, flower, 200, `{"celery@w":{"active_queues":[],"registered":["tasks.add"]}}`); len(res.Findings) > 0 {
t.Errorf("a confless worker blob should not match flower, got %d findings", len(res.Findings))
}
})
t.Run("an rq queues dump is flagged with the first queue name", func(t *testing.T) {
body := `{"queues":[{"name":"default","count":42,"queued_url":"/0/view/jobs/default/queued/...",` +
`"failed_job_registry_count":3,"failed_url":"...","started_job_registry_count":1,"started_url":"...",` +
`"deferred_job_registry_count":0,"deferred_url":"...","finished_job_registry_count":100,` +
`"finished_url":"...","canceled_job_registry_count":0,"canceled_url":"...",` +
`"scheduled_job_registry_count":5,"scheduled_url":"..."}]}`
res := runJobDashModule(t, rq, 200, body)
if len(res.Findings) == 0 {
t.Fatal("expected an rq finding")
}
if v := jobDashExtract(res, "rq_queue_name"); v != "default" {
t.Errorf("rq_queue_name=%q, want default", v)
}
})
t.Run("a queues blob without the registry counts is not flagged as rq", func(t *testing.T) {
if res := runJobDashModule(t, rq, 200, `{"queues":[{"name":"q","failed_job_registry_count":0}]}`); len(res.Findings) > 0 {
t.Errorf("a partial queues blob should not match rq, got %d findings", len(res.Findings))
}
})
t.Run("a plain 200 body is not a leak", func(t *testing.T) {
for _, file := range []string{sidekiq, flower, rq} {
if res := runJobDashModule(t, file, 200, "ok"); len(res.Findings) > 0 {
t.Errorf("%s: a plain 200 body should not match, got %d findings", file, len(res.Findings))
}
}
})
t.Run("a 404 is not a leak", func(t *testing.T) {
for _, file := range []string{sidekiq, flower, rq} {
if res := runJobDashModule(t, file, 404, "not found"); len(res.Findings) > 0 {
t.Errorf("%s: a 404 should not match, got %d findings", file, len(res.Findings))
}
}
})
}
+1 -1
View File
@@ -18,7 +18,7 @@ import (
"net/http/httptest"
"testing"
"github.com/dropalldatabases/sif/internal/httpx"
"github.com/vmfunc/sif/internal/httpx"
)
func TestRunExtractorsJSON(t *testing.T) {
+130
View File
@@ -0,0 +1,130 @@
package modules_test
import (
"context"
"net/http"
"net/http/httptest"
"testing"
"time"
"github.com/vmfunc/sif/internal/modules"
)
const kafkaUIClustersBody = `[{"name":"production","defaultCluster":true,"status":"online",` +
`"brokerCount":3,"onlinePartitionCount":128,"topicCount":42,"bytesInPerSec":10485.5,` +
`"bytesOutPerSec":20971.0,"readOnly":false,"version":"3.7.0",` +
`"features":["TOPIC_DELETION","KAFKA_CONNECT","KSQL_DB"]}]`
const kafdropOverviewBody = `{"summary":{"topicCount":42,"partitionCount":128,` +
`"underReplicatedCount":0,"preferredReplicaPercent":1.0,"brokerLeaderPartitionCount":{"1":43},` +
`"brokerPreferredLeaderPartitionCount":{"1":43},"expectedBrokerIds":[1,2,3]},` +
`"brokers":[{"id":1,"host":"kafka-0.svc.internal","port":9092,"controller":true,"rack":"r1"}],` +
`"topics":[{"name":"orders","partitions":[]}]}`
func runKafkaModule(t *testing.T, file string, status int, body string) (*modules.Result, http.Header) {
t.Helper()
def, err := modules.ParseYAMLModule(file)
if err != nil {
t.Fatalf("parse %s: %v", file, err)
}
var gotHeaders http.Header
srv := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
gotHeaders = r.Header.Clone()
w.WriteHeader(status)
_, _ = w.Write([]byte(body))
}))
defer srv.Close()
res, err := modules.ExecuteHTTPModule(context.Background(), srv.URL, def, modules.Options{
Timeout: 5 * time.Second,
Threads: 2,
})
if err != nil {
t.Fatalf("execute %s: %v", file, err)
}
return res, gotHeaders
}
func kafkaExtract(res *modules.Result, key string) string {
for _, f := range res.Findings {
if v := f.Extracted[key]; v != "" {
return v
}
}
return ""
}
func TestKafkaManagementUIExposureModules(t *testing.T) {
const kafkaUI = "../../modules/recon/kafka-ui-exposure.yaml"
const kafdrop = "../../modules/recon/kafdrop-exposure.yaml"
t.Run("an open kafka-ui /api/clusters is flagged with the version", func(t *testing.T) {
res, _ := runKafkaModule(t, kafkaUI, 200, kafkaUIClustersBody)
if len(res.Findings) == 0 {
t.Fatal("expected a kafka-ui finding")
}
if v := kafkaExtract(res, "kafka_version"); v != "3.7.0" {
t.Errorf("kafka_version=%q, want 3.7.0", v)
}
})
t.Run("a kafka-ui cluster with a non-enum status is not flagged", func(t *testing.T) {
body := `[{"name":"x","defaultCluster":true,"status":"degraded","brokerCount":1}]`
if res, _ := runKafkaModule(t, kafkaUI, 200, body); len(res.Findings) > 0 {
t.Errorf("a non-enum status should not match, got %d findings", len(res.Findings))
}
})
t.Run("a login-protected kafka-ui is not flagged", func(t *testing.T) {
if res, _ := runKafkaModule(t, kafkaUI, 401, `{"message":"Unauthorized"}`); len(res.Findings) > 0 {
t.Errorf("a 401 kafka-ui should not match, got %d findings", len(res.Findings))
}
})
t.Run("a kafdrop overview does not match the kafka-ui module", func(t *testing.T) {
if res, _ := runKafkaModule(t, kafkaUI, 200, kafdropOverviewBody); len(res.Findings) > 0 {
t.Errorf("a kafdrop body should not match kafka-ui, got %d findings", len(res.Findings))
}
})
t.Run("an open kafdrop overview is flagged with a broker host", func(t *testing.T) {
res, hdr := runKafkaModule(t, kafdrop, 200, kafdropOverviewBody)
if len(res.Findings) == 0 {
t.Fatal("expected a kafdrop finding")
}
if v := kafkaExtract(res, "kafka_broker"); v != "kafka-0.svc.internal" {
t.Errorf("kafka_broker=%q, want kafka-0.svc.internal", v)
}
if got := hdr.Get("Accept"); got != "application/json" {
t.Errorf("Accept header=%q, want application/json", got)
}
})
t.Run("a cluster overview without preferredReplicaPercent is not flagged", func(t *testing.T) {
body := `{"summary":{"topicCount":1},"brokers":[{"id":1,"host":"h"}],"topics":[]}`
if res, _ := runKafkaModule(t, kafdrop, 200, body); len(res.Findings) > 0 {
t.Errorf("a preferredReplicaPercent-less overview should not match, got %d findings", len(res.Findings))
}
})
t.Run("a basic-auth-protected kafdrop is not flagged", func(t *testing.T) {
if res, _ := runKafkaModule(t, kafdrop, 401, "Unauthorized"); len(res.Findings) > 0 {
t.Errorf("a 401 kafdrop should not match, got %d findings", len(res.Findings))
}
})
t.Run("a kafka-ui cluster list does not match the kafdrop module", func(t *testing.T) {
if res, _ := runKafkaModule(t, kafdrop, 200, kafkaUIClustersBody); len(res.Findings) > 0 {
t.Errorf("a kafka-ui body should not match kafdrop, got %d findings", len(res.Findings))
}
})
t.Run("plain 200 bodies are not leaks", func(t *testing.T) {
if res, _ := runKafkaModule(t, kafkaUI, 200, "ok"); len(res.Findings) > 0 {
t.Errorf("a plain 200 should not match kafka-ui, got %d findings", len(res.Findings))
}
if res, _ := runKafkaModule(t, kafdrop, 404, "not found"); len(res.Findings) > 0 {
t.Errorf("a 404 should not match kafdrop, got %d findings", len(res.Findings))
}
})
}
@@ -0,0 +1,85 @@
package modules_test
import (
"context"
"net/http"
"net/http/httptest"
"testing"
"time"
"github.com/vmfunc/sif/internal/modules"
)
func runGatewayModule(t *testing.T, file string, status int, body string) *modules.Result {
t.Helper()
def, err := modules.ParseYAMLModule(file)
if err != nil {
t.Fatalf("parse %s: %v", file, err)
}
srv := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
w.WriteHeader(status)
_, _ = w.Write([]byte(body))
}))
defer srv.Close()
res, err := modules.ExecuteHTTPModule(context.Background(), srv.URL, def, modules.Options{
Timeout: 5 * time.Second,
Threads: 2,
})
if err != nil {
t.Fatalf("execute %s: %v", file, err)
}
return res
}
func gatewayExtract(res *modules.Result, key string) string {
for _, f := range res.Findings {
if v := f.Extracted[key]; v != "" {
return v
}
}
return ""
}
func TestLLMGatewayExposureModules(t *testing.T) {
const oneapi = "../../modules/recon/oneapi-status-exposure.yaml"
oneapiStatus := `{"success":true,"message":"","data":{"version":"v0.8.4","start_time":1719100000,` +
`"system_name":"New API","quota_per_unit":500000,"github_oauth":false}}`
t.Run("a oneapi status is flagged with its version", func(t *testing.T) {
res := runGatewayModule(t, oneapi, 200, oneapiStatus)
if len(res.Findings) == 0 {
t.Fatal("expected a oneapi finding")
}
if v := gatewayExtract(res, "oneapi_version"); v != "v0.8.4" {
t.Errorf("oneapi_version=%q, want v0.8.4", v)
}
})
t.Run("a body without system_name is not flagged as oneapi", func(t *testing.T) {
body := `{"data":{"start_time":1,"quota_per_unit":500000}}`
if res := runGatewayModule(t, oneapi, 200, body); len(res.Findings) > 0 {
t.Errorf("a body without system_name should not match oneapi, got %d findings", len(res.Findings))
}
})
t.Run("a body without quota_per_unit is not flagged as oneapi", func(t *testing.T) {
body := `{"data":{"system_name":"X","start_time":1}}`
if res := runGatewayModule(t, oneapi, 200, body); len(res.Findings) > 0 {
t.Errorf("a body without quota_per_unit should not match oneapi, got %d findings", len(res.Findings))
}
})
t.Run("a plain 200 body is not a leak", func(t *testing.T) {
if res := runGatewayModule(t, oneapi, 200, "ok"); len(res.Findings) > 0 {
t.Errorf("a plain 200 body should not match oneapi, got %d findings", len(res.Findings))
}
})
t.Run("a 404 is not a leak", func(t *testing.T) {
if res := runGatewayModule(t, oneapi, 404, "not found"); len(res.Findings) > 0 {
t.Errorf("a 404 should not match oneapi, got %d findings", len(res.Findings))
}
})
}
@@ -0,0 +1,161 @@
package modules_test
import (
"context"
"net/http"
"net/http/httptest"
"testing"
"time"
"github.com/vmfunc/sif/internal/modules"
)
func runGPUServingModule(t *testing.T, file string, status int, body string) *modules.Result {
t.Helper()
def, err := modules.ParseYAMLModule(file)
if err != nil {
t.Fatalf("parse %s: %v", file, err)
}
srv := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
w.WriteHeader(status)
_, _ = w.Write([]byte(body))
}))
defer srv.Close()
res, err := modules.ExecuteHTTPModule(context.Background(), srv.URL, def, modules.Options{
Timeout: 5 * time.Second,
Threads: 2,
})
if err != nil {
t.Fatalf("execute %s: %v", file, err)
}
return res
}
func gpuServingExtract(res *modules.Result, key string) string {
for _, f := range res.Findings {
if v := f.Extracted[key]; v != "" {
return v
}
}
return ""
}
func TestLLMGPUServingExposureModules(t *testing.T) {
const triton = "../../modules/recon/triton-api-exposure.yaml"
const sglang = "../../modules/recon/sglang-api-exposure.yaml"
const torchserve = "../../modules/recon/torchserve-api-exposure.yaml"
tritonMeta := `{"name":"triton","version":"2.45.0","extensions":["classification","sequence",` +
`"model_repository","schedule_policy","model_configuration","statistics","trace","logging"]}`
sglangInfo := `{"model_path":"meta-llama/Llama-3-8B","tokenizer_path":"meta-llama/Llama-3-8B",` +
`"is_generation":true,"preferred_sampling_params":null,"weight_version":"default",` +
`"has_image_understanding":false,"has_audio_understanding":false,"model_type":"llama",` +
`"architectures":["LlamaForCausalLM"]}`
torchserveModels := `{"nextPageToken":"4","models":[{"modelName":"resnet-18","modelUrl":"resnet-18.mar"},` +
`{"modelName":"noop","modelUrl":"noop-v1.0"}]}`
t.Run("a triton metadata api is flagged with its version", func(t *testing.T) {
res := runGPUServingModule(t, triton, 200, tritonMeta)
if len(res.Findings) == 0 {
t.Fatal("expected a triton finding")
}
if v := gpuServingExtract(res, "triton_version"); v != "2.45.0" {
t.Errorf("triton_version=%q, want 2.45.0", v)
}
})
t.Run("a sglang model_info is flagged with its model path", func(t *testing.T) {
res := runGPUServingModule(t, sglang, 200, sglangInfo)
if len(res.Findings) == 0 {
t.Fatal("expected a sglang finding")
}
if v := gpuServingExtract(res, "sglang_model"); v != "meta-llama/Llama-3-8B" {
t.Errorf("sglang_model=%q, want meta-llama/Llama-3-8B", v)
}
})
t.Run("a torchserve models api is flagged with its model name", func(t *testing.T) {
res := runGPUServingModule(t, torchserve, 200, torchserveModels)
if len(res.Findings) == 0 {
t.Fatal("expected a torchserve finding")
}
if v := gpuServingExtract(res, "torchserve_model"); v != "resnet-18" {
t.Errorf("torchserve_model=%q, want resnet-18", v)
}
})
t.Run("a hugging face config with model_type is not flagged as sglang", func(t *testing.T) {
body := `{"model_type":"llama","architectures":["LlamaForCausalLM"],"hidden_size":4096}`
if res := runGPUServingModule(t, sglang, 200, body); len(res.Findings) > 0 {
t.Errorf("a model config should not match sglang, got %d findings", len(res.Findings))
}
})
t.Run("a generation flag alone is not flagged as sglang", func(t *testing.T) {
body := `{"is_generation":true,"model":"x"}`
if res := runGPUServingModule(t, sglang, 200, body); len(res.Findings) > 0 {
t.Errorf("is_generation alone should not match sglang, got %d findings", len(res.Findings))
}
})
t.Run("an image understanding flag alone is not flagged as sglang", func(t *testing.T) {
body := `{"has_image_understanding":false,"model_path":"x"}`
if res := runGPUServingModule(t, sglang, 200, body); len(res.Findings) > 0 {
t.Errorf("has_image_understanding alone should not match sglang, got %d findings", len(res.Findings))
}
})
t.Run("a model url without a page token is not flagged as torchserve", func(t *testing.T) {
body := `{"models":[{"modelName":"x","modelUrl":"x.mar"}]}`
if res := runGPUServingModule(t, torchserve, 200, body); len(res.Findings) > 0 {
t.Errorf("modelUrl without nextPageToken should not match torchserve, got %d findings", len(res.Findings))
}
})
t.Run("an ollama style models list is not flagged as torchserve", func(t *testing.T) {
body := `{"models":[{"name":"llama3:latest","model":"llama3:latest"}]}`
if res := runGPUServingModule(t, torchserve, 200, body); len(res.Findings) > 0 {
t.Errorf("an ollama model list should not match torchserve, got %d findings", len(res.Findings))
}
})
t.Run("a paginated list without a model url is not flagged as torchserve", func(t *testing.T) {
body := `{"nextPageToken":"4","items":[{"id":"a"}]}`
if res := runGPUServingModule(t, torchserve, 200, body); len(res.Findings) > 0 {
t.Errorf("a generic paginated list should not match torchserve, got %d findings", len(res.Findings))
}
})
t.Run("a kserve server that is not triton is not flagged", func(t *testing.T) {
body := `{"name":"my-model-server","version":"1.0","extensions":["model_repository"]}`
if res := runGPUServingModule(t, triton, 200, body); len(res.Findings) > 0 {
t.Errorf("a non-triton kserve server should not match, got %d findings", len(res.Findings))
}
})
t.Run("a triton name without extensions is not flagged", func(t *testing.T) {
if res := runGPUServingModule(t, triton, 200, `{"name":"triton"}`); len(res.Findings) > 0 {
t.Errorf("a name-only body should not match triton, got %d findings", len(res.Findings))
}
})
t.Run("a plain 200 body is not a leak", func(t *testing.T) {
for _, file := range []string{triton, sglang, torchserve} {
if res := runGPUServingModule(t, file, 200, "ok"); len(res.Findings) > 0 {
t.Errorf("%s: a plain 200 body should not match, got %d findings", file, len(res.Findings))
}
}
})
t.Run("a 404 is not a leak", func(t *testing.T) {
for _, file := range []string{triton, sglang, torchserve} {
if res := runGPUServingModule(t, file, 404, "not found"); len(res.Findings) > 0 {
t.Errorf("%s: a 404 should not match, got %d findings", file, len(res.Findings))
}
}
})
}
@@ -0,0 +1,125 @@
package modules_test
import (
"context"
"net/http"
"net/http/httptest"
"testing"
"time"
"github.com/vmfunc/sif/internal/modules"
)
func runInferenceExposureModule(t *testing.T, file string, status int, body string) *modules.Result {
t.Helper()
def, err := modules.ParseYAMLModule(file)
if err != nil {
t.Fatalf("parse %s: %v", file, err)
}
srv := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
w.WriteHeader(status)
_, _ = w.Write([]byte(body))
}))
defer srv.Close()
res, err := modules.ExecuteHTTPModule(context.Background(), srv.URL, def, modules.Options{
Timeout: 5 * time.Second,
Threads: 2,
})
if err != nil {
t.Fatalf("execute %s: %v", file, err)
}
return res
}
func inferenceExtract(res *modules.Result, key string) string {
for _, f := range res.Findings {
if v := f.Extracted[key]; v != "" {
return v
}
}
return ""
}
func TestLLMInferenceExposureModules(t *testing.T) {
const tgi = "../../modules/recon/tgi-api-exposure.yaml"
const tei = "../../modules/recon/tei-api-exposure.yaml"
tgiInfo := `{"model_id":"meta-llama/Llama-2-7b-chat-hf","model_sha":"abc","model_pipeline_tag":"text-generation",` +
`"max_concurrent_requests":128,"max_best_of":2,"max_input_tokens":4095,"max_total_tokens":4096,` +
`"max_batch_total_tokens":16384,"router":"text-generation-router","version":"2.0.4","sha":"deadbeef"}`
teiInfo := `{"model_id":"BAAI/bge-large-en-v1.5","model_sha":"abc","model_dtype":"float16",` +
`"model_type":{"embedding":{"pooling":"cls"}},"max_concurrent_requests":512,"max_input_length":512,` +
`"max_batch_tokens":16384,"max_client_batch_size":32,"tokenization_workers":8,"version":"1.5.0"}`
t.Run("a tgi info is flagged with its model id", func(t *testing.T) {
res := runInferenceExposureModule(t, tgi, 200, tgiInfo)
if len(res.Findings) == 0 {
t.Fatal("expected a tgi finding")
}
if v := inferenceExtract(res, "tgi_model"); v != "meta-llama/Llama-2-7b-chat-hf" {
t.Errorf("tgi_model=%q, want meta-llama/Llama-2-7b-chat-hf", v)
}
})
t.Run("a tei info is flagged with its model id", func(t *testing.T) {
res := runInferenceExposureModule(t, tei, 200, teiInfo)
if len(res.Findings) == 0 {
t.Fatal("expected a tei finding")
}
if v := inferenceExtract(res, "tei_model"); v != "BAAI/bge-large-en-v1.5" {
t.Errorf("tei_model=%q, want BAAI/bge-large-en-v1.5", v)
}
})
t.Run("a tgi info is not flagged as tei", func(t *testing.T) {
if res := runInferenceExposureModule(t, tei, 200, tgiInfo); len(res.Findings) > 0 {
t.Errorf("a tgi info should not match the tei module, got %d findings", len(res.Findings))
}
})
t.Run("a tei info is not flagged as tgi", func(t *testing.T) {
if res := runInferenceExposureModule(t, tgi, 200, teiInfo); len(res.Findings) > 0 {
t.Errorf("a tei info should not match the tgi module, got %d findings", len(res.Findings))
}
})
t.Run("a hugging face model config is not flagged as tei", func(t *testing.T) {
body := `{"model_type":"bert","hidden_size":768,"num_attention_heads":12,"vocab_size":30522}`
if res := runInferenceExposureModule(t, tei, 200, body); len(res.Findings) > 0 {
t.Errorf("a model config.json should not match tei, got %d findings", len(res.Findings))
}
})
t.Run("a batch-tokens body without model_type is not flagged as tei", func(t *testing.T) {
body := `{"max_batch_tokens":16384,"max_concurrent_requests":512}`
if res := runInferenceExposureModule(t, tei, 200, body); len(res.Findings) > 0 {
t.Errorf("a body without model_type should not match tei, got %d findings", len(res.Findings))
}
})
t.Run("a different router is not flagged as tgi", func(t *testing.T) {
body := `{"router":"some-other-router","max_concurrent_requests":10}`
if res := runInferenceExposureModule(t, tgi, 200, body); len(res.Findings) > 0 {
t.Errorf("a non-tgi router should not match, got %d findings", len(res.Findings))
}
})
t.Run("a plain 200 body is not a leak", func(t *testing.T) {
for _, file := range []string{tgi, tei} {
if res := runInferenceExposureModule(t, file, 200, "ok"); len(res.Findings) > 0 {
t.Errorf("%s: a plain 200 body should not match, got %d findings", file, len(res.Findings))
}
}
})
t.Run("a 404 is not a leak", func(t *testing.T) {
for _, file := range []string{tgi, tei} {
if res := runInferenceExposureModule(t, file, 404, "not found"); len(res.Findings) > 0 {
t.Errorf("%s: a 404 should not match, got %d findings", file, len(res.Findings))
}
}
})
}
@@ -0,0 +1,212 @@
package modules_test
import (
"context"
"net/http"
"net/http/httptest"
"testing"
"time"
"github.com/vmfunc/sif/internal/modules"
)
func runLocalRunnerModule(t *testing.T, file string, status int, body string) *modules.Result {
t.Helper()
def, err := modules.ParseYAMLModule(file)
if err != nil {
t.Fatalf("parse %s: %v", file, err)
}
srv := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
w.WriteHeader(status)
_, _ = w.Write([]byte(body))
}))
defer srv.Close()
res, err := modules.ExecuteHTTPModule(context.Background(), srv.URL, def, modules.Options{
Timeout: 5 * time.Second,
Threads: 2,
})
if err != nil {
t.Fatalf("execute %s: %v", file, err)
}
return res
}
func localRunnerExtract(res *modules.Result, key string) string {
for _, f := range res.Findings {
if v := f.Extracted[key]; v != "" {
return v
}
}
return ""
}
func TestLLMLocalRunnerExposureModules(t *testing.T) {
const ollama = "../../modules/recon/ollama-api-exposure.yaml"
const koboldcpp = "../../modules/recon/koboldcpp-api-exposure.yaml"
const tabby = "../../modules/recon/tabby-api-exposure.yaml"
const oobabooga = "../../modules/recon/oobabooga-api-exposure.yaml"
ollamaTags := `{"models":[{"name":"deepseek-r1:latest","model":"deepseek-r1:latest",` +
`"modified_at":"2025-05-10T08:06:48.639712648-07:00","size":4683075271,` +
`"digest":"0a8c266910232fd3291e71e5ba1e058cc5af9d411192cf88b6d30e92b6e73163",` +
`"details":{"parent_model":"","format":"gguf","family":"qwen2","families":["qwen2"],` +
`"parameter_size":"7.6B","quantization_level":"Q4_K_M"}}]}`
koboldVersion := `{"result":"KoboldCpp","version":"1.71.1","protected":false,"llm":true,` +
`"txt2img":true,"vision":false,"audio":false,"transcribe":false,"multiplayer":false,` +
`"websearch":false,"tts":false,"embeddings":false,"music":false,"savedata":false,` +
`"admin":0,"router":false,"guidance":false,"jinja":true,"mcp":false}`
tabbyHealth := `{"model":"TabbyML/StarCoder-1B","chat_model":"Qwen2.5-Coder-7B-Instruct",` +
`"device":"cuda","cuda_devices":["NVIDIA GeForce RTX 4090"],"models":{"completion":{"vram":1234}},` +
`"arch":"x86_64","cpu_info":"AMD Ryzen 9 5950X","cpu_count":32,` +
`"version":{"build_date":"2024-06-01","build_timestamp":"2024-06-01T00:00:00Z",` +
`"git_sha":"deadbeef","git_describe":"v0.13.0"},"webserver":true}`
oobaModelInfo := `{"model_name":"TheBloke_Llama-2-13B-chat-GPTQ","lora_names":["alpaca-lora"],` +
`"loader":"ExLlamav2_HF"}`
t.Run("an ollama tags api is flagged with its model name", func(t *testing.T) {
res := runLocalRunnerModule(t, ollama, 200, ollamaTags)
if len(res.Findings) == 0 {
t.Fatal("expected an ollama finding")
}
if v := localRunnerExtract(res, "ollama_model"); v != "deepseek-r1:latest" {
t.Errorf("ollama_model=%q, want deepseek-r1:latest", v)
}
})
t.Run("an ollama tags list without model details is not flagged", func(t *testing.T) {
body := `{"models":[{"name":"llama3:latest","digest":"abc"}]}`
if res := runLocalRunnerModule(t, ollama, 200, body); len(res.Findings) > 0 {
t.Errorf("a bare model list should not match ollama, got %d findings", len(res.Findings))
}
})
t.Run("an ollama version response is not flagged", func(t *testing.T) {
if res := runLocalRunnerModule(t, ollama, 200, `{"version":"0.5.1"}`); len(res.Findings) > 0 {
t.Errorf("a version response should not match ollama, got %d findings", len(res.Findings))
}
})
t.Run("a koboldcpp version probe is flagged with its version", func(t *testing.T) {
res := runLocalRunnerModule(t, koboldcpp, 200, koboldVersion)
if len(res.Findings) == 0 {
t.Fatal("expected a koboldcpp finding")
}
if v := localRunnerExtract(res, "koboldcpp_version"); v != "1.71.1" {
t.Errorf("koboldcpp_version=%q, want 1.71.1", v)
}
if v := localRunnerExtract(res, "koboldcpp_protected"); v != "false" {
t.Errorf("koboldcpp_protected=%q, want false", v)
}
})
t.Run("a koboldcpp result without txt2img is not flagged", func(t *testing.T) {
body := `{"result":"KoboldCpp","protected":false}`
if res := runLocalRunnerModule(t, koboldcpp, 200, body); len(res.Findings) > 0 {
t.Errorf("a probe without txt2img should not match koboldcpp, got %d findings", len(res.Findings))
}
})
t.Run("a koboldcpp result without protected is not flagged", func(t *testing.T) {
body := `{"result":"KoboldCpp","txt2img":true}`
if res := runLocalRunnerModule(t, koboldcpp, 200, body); len(res.Findings) > 0 {
t.Errorf("a probe without protected should not match koboldcpp, got %d findings", len(res.Findings))
}
})
t.Run("another server reporting capabilities is not flagged as koboldcpp", func(t *testing.T) {
body := `{"result":"SomeOtherServer","protected":false,"txt2img":true}`
if res := runLocalRunnerModule(t, koboldcpp, 200, body); len(res.Findings) > 0 {
t.Errorf("a non-kobold result should not match koboldcpp, got %d findings", len(res.Findings))
}
})
t.Run("a tabby health is flagged with its model", func(t *testing.T) {
res := runLocalRunnerModule(t, tabby, 200, tabbyHealth)
if len(res.Findings) == 0 {
t.Fatal("expected a tabby finding")
}
if v := localRunnerExtract(res, "tabby_model"); v != "TabbyML/StarCoder-1B" {
t.Errorf("tabby_model=%q, want TabbyML/StarCoder-1B", v)
}
})
t.Run("a tabby health without git_describe is not flagged", func(t *testing.T) {
body := `{"cpu_info":"AMD Ryzen","cuda_devices":["RTX 4090"],"device":"cuda"}`
if res := runLocalRunnerModule(t, tabby, 200, body); len(res.Findings) > 0 {
t.Errorf("a health body without git_describe should not match tabby, got %d findings", len(res.Findings))
}
})
t.Run("a tabby health without cuda_devices is not flagged", func(t *testing.T) {
body := `{"cpu_info":"AMD Ryzen","version":{"git_describe":"v0.13.0"}}`
if res := runLocalRunnerModule(t, tabby, 200, body); len(res.Findings) > 0 {
t.Errorf("a health body without cuda_devices should not match tabby, got %d findings", len(res.Findings))
}
})
t.Run("a build info without cpu_info is not flagged as tabby", func(t *testing.T) {
body := `{"cuda_devices":["RTX 4090"],"version":{"git_describe":"v0.13.0"}}`
if res := runLocalRunnerModule(t, tabby, 200, body); len(res.Findings) > 0 {
t.Errorf("a body without cpu_info should not match tabby, got %d findings", len(res.Findings))
}
})
t.Run("an oobabooga model info is flagged with its model name", func(t *testing.T) {
res := runLocalRunnerModule(t, oobabooga, 200, oobaModelInfo)
if len(res.Findings) == 0 {
t.Fatal("expected an oobabooga finding")
}
if v := localRunnerExtract(res, "oobabooga_model"); v != "TheBloke_Llama-2-13B-chat-GPTQ" {
t.Errorf("oobabooga_model=%q, want TheBloke_Llama-2-13B-chat-GPTQ", v)
}
})
t.Run("a body without lora_names is not flagged as oobabooga", func(t *testing.T) {
body := `{"model_name":"some-model","loader":"Transformers"}`
if res := runLocalRunnerModule(t, oobabooga, 200, body); len(res.Findings) > 0 {
t.Errorf("a body without lora_names should not match oobabooga, got %d findings", len(res.Findings))
}
})
t.Run("a body without model_name is not flagged as oobabooga", func(t *testing.T) {
body := `{"lora_names":["x"],"loader":"Transformers"}`
if res := runLocalRunnerModule(t, oobabooga, 200, body); len(res.Findings) > 0 {
t.Errorf("a body without model_name should not match oobabooga, got %d findings", len(res.Findings))
}
})
t.Run("a model_names plural list is not flagged as oobabooga", func(t *testing.T) {
body := `{"model_names":["TheBloke_Llama-2-13B-chat-GPTQ","mistral-7b-instruct"]}`
if res := runLocalRunnerModule(t, oobabooga, 200, body); len(res.Findings) > 0 {
t.Errorf("a model_names plural list should not match oobabooga, got %d findings", len(res.Findings))
}
})
t.Run("an idle oobabooga with no model loaded is still flagged", func(t *testing.T) {
body := `{"model_name":"None","lora_names":[],"loader":"Transformers"}`
if res := runLocalRunnerModule(t, oobabooga, 200, body); len(res.Findings) == 0 {
t.Error("expected an idle oobabooga (model_name None) to still be flagged")
}
})
t.Run("a plain 200 body is not a leak", func(t *testing.T) {
for _, file := range []string{ollama, koboldcpp, oobabooga, tabby} {
if res := runLocalRunnerModule(t, file, 200, "ok"); len(res.Findings) > 0 {
t.Errorf("%s: a plain 200 body should not match, got %d findings", file, len(res.Findings))
}
}
})
t.Run("a 404 is not a leak", func(t *testing.T) {
for _, file := range []string{ollama, koboldcpp, oobabooga, tabby} {
if res := runLocalRunnerModule(t, file, 404, "not found"); len(res.Findings) > 0 {
t.Errorf("%s: a 404 should not match, got %d findings", file, len(res.Findings))
}
}
})
}
@@ -0,0 +1,225 @@
package modules_test
import (
"context"
"net/http"
"net/http/httptest"
"testing"
"time"
"github.com/vmfunc/sif/internal/modules"
)
func runOpenAICompatModule(t *testing.T, file string, status int, body string) *modules.Result {
t.Helper()
def, err := modules.ParseYAMLModule(file)
if err != nil {
t.Fatalf("parse %s: %v", file, err)
}
srv := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
w.WriteHeader(status)
_, _ = w.Write([]byte(body))
}))
defer srv.Close()
res, err := modules.ExecuteHTTPModule(context.Background(), srv.URL, def, modules.Options{
Timeout: 5 * time.Second,
Threads: 2,
})
if err != nil {
t.Fatalf("execute %s: %v", file, err)
}
return res
}
func openAICompatExtract(res *modules.Result, key string) string {
for _, f := range res.Findings {
if v := f.Extracted[key]; v != "" {
return v
}
}
return ""
}
func TestLLMOpenAICompatExposureModules(t *testing.T) {
const vllm = "../../modules/recon/vllm-api-exposure.yaml"
const llamacpp = "../../modules/recon/llamacpp-api-exposure.yaml"
const infinity = "../../modules/recon/infinity-embedding-api-exposure.yaml"
const lmstudio = "../../modules/recon/lmstudio-api-exposure.yaml"
vllmModels := `{"object":"list","data":[{"id":"meta-llama/Llama-3.1-8B-Instruct","object":"model",` +
`"created":1718900000,"owned_by":"vllm","root":"meta-llama/Llama-3.1-8B-Instruct",` +
`"parent":null,"max_model_len":131072}]}`
llamacppModels := `{"object":"list","data":[{"id":"/models/llama-2-7b.Q4_K_M.gguf","object":"model",` +
`"created":1718900000,"owned_by":"llamacpp"}]}`
infinityModels := `{"data":[{"id":"BAAI/bge-small-en-v1.5","stats":{"queue_fraction":0.0,` +
`"queue_absolute":0,"results_pending":0,"results_queue":0},"object":"model","owned_by":"infinity",` +
`"created":1718900000,"backend":"torch","capabilities":["embed"]}],"object":"list"}`
lmstudioModels := `{"object":"list","data":[{"id":"qwen2-vl-7b-instruct","object":"model",` +
`"type":"vlm","publisher":"mlx-community","arch":"qwen2_vl","compatibility_type":"mlx",` +
`"quantization":"4bit","state":"not-loaded","max_context_length":32768},` +
`{"id":"text-embedding-nomic-embed-text-v1.5","object":"model","type":"embeddings",` +
`"publisher":"nomic-ai","arch":"nomic-bert","compatibility_type":"gguf","quantization":"Q4_0",` +
`"state":"not-loaded","max_context_length":2048}]}`
t.Run("a vllm models api is flagged with its model id", func(t *testing.T) {
res := runOpenAICompatModule(t, vllm, 200, vllmModels)
if len(res.Findings) == 0 {
t.Fatal("expected a vllm finding")
}
if v := openAICompatExtract(res, "vllm_model"); v != "meta-llama/Llama-3.1-8B-Instruct" {
t.Errorf("vllm_model=%q, want meta-llama/Llama-3.1-8B-Instruct", v)
}
})
t.Run("a vllm-prefixed but distinct owned_by is not flagged as vllm", func(t *testing.T) {
body := `{"object":"list","data":[{"id":"meta-llama/Llama-3.1-8B-Instruct","object":"model",` +
`"created":1718900000,"owned_by":"vllm-frontend-rs","root":"meta-llama/Llama-3.1-8B-Instruct"}]}`
if res := runOpenAICompatModule(t, vllm, 200, body); len(res.Findings) > 0 {
t.Errorf("owned_by vllm-frontend-rs should not match the anchored vllm regex, got %d findings", len(res.Findings))
}
})
t.Run("a llamacpp models api is flagged with its model id", func(t *testing.T) {
res := runOpenAICompatModule(t, llamacpp, 200, llamacppModels)
if len(res.Findings) == 0 {
t.Fatal("expected a llamacpp finding")
}
if v := openAICompatExtract(res, "llamacpp_model"); v != "/models/llama-2-7b.Q4_K_M.gguf" {
t.Errorf("llamacpp_model=%q, want /models/llama-2-7b.Q4_K_M.gguf", v)
}
})
t.Run("a llamacpp models api is not flagged as vllm", func(t *testing.T) {
if res := runOpenAICompatModule(t, vllm, 200, llamacppModels); len(res.Findings) > 0 {
t.Errorf("a llamacpp list should not match the vllm module, got %d findings", len(res.Findings))
}
})
t.Run("a vllm models api is not flagged as llamacpp", func(t *testing.T) {
if res := runOpenAICompatModule(t, llamacpp, 200, vllmModels); len(res.Findings) > 0 {
t.Errorf("a vllm list should not match the llamacpp module, got %d findings", len(res.Findings))
}
})
t.Run("an openai compatible list owned by openai is not flagged as vllm", func(t *testing.T) {
body := `{"object":"list","data":[{"id":"gpt-4o","object":"model","owned_by":"openai"}]}`
if res := runOpenAICompatModule(t, vllm, 200, body); len(res.Findings) > 0 {
t.Errorf("an openai-owned list should not match vllm, got %d findings", len(res.Findings))
}
})
t.Run("an infinity models list is flagged with its model id", func(t *testing.T) {
res := runOpenAICompatModule(t, infinity, 200, infinityModels)
if len(res.Findings) == 0 {
t.Fatal("expected an infinity finding")
}
if v := openAICompatExtract(res, "infinity_model"); v != "BAAI/bge-small-en-v1.5" {
t.Errorf("infinity_model=%q, want BAAI/bge-small-en-v1.5", v)
}
})
t.Run("an infinity list without owned_by is not flagged", func(t *testing.T) {
body := `{"data":[{"id":"x","backend":"torch","capabilities":["embed"]}],"object":"list"}`
if res := runOpenAICompatModule(t, infinity, 200, body); len(res.Findings) > 0 {
t.Errorf("a list without owned_by should not match infinity, got %d findings", len(res.Findings))
}
})
t.Run("an infinity list without a backend is not flagged", func(t *testing.T) {
body := `{"data":[{"id":"x","owned_by":"infinity","capabilities":["embed"]}],"object":"list"}`
if res := runOpenAICompatModule(t, infinity, 200, body); len(res.Findings) > 0 {
t.Errorf("a list without backend should not match infinity, got %d findings", len(res.Findings))
}
})
t.Run("an infinity list without capabilities is not flagged", func(t *testing.T) {
body := `{"data":[{"id":"x","owned_by":"infinity","backend":"torch"}],"object":"list"}`
if res := runOpenAICompatModule(t, infinity, 200, body); len(res.Findings) > 0 {
t.Errorf("a list without capabilities should not match infinity, got %d findings", len(res.Findings))
}
})
t.Run("a non-infinity server with backend and capabilities is not flagged", func(t *testing.T) {
body := `{"object":"list","data":[{"id":"m","object":"model","owned_by":"acme-org",` +
`"backend":"vllm","capabilities":["chat"]}]}`
if res := runOpenAICompatModule(t, infinity, 200, body); len(res.Findings) > 0 {
t.Errorf("backend+capabilities with a non-infinity owned_by should not match, got %d findings", len(res.Findings))
}
})
t.Run("a vllm models list is not flagged as infinity", func(t *testing.T) {
if res := runOpenAICompatModule(t, infinity, 200, vllmModels); len(res.Findings) > 0 {
t.Errorf("a vllm list should not match infinity, got %d findings", len(res.Findings))
}
})
t.Run("an infinity list is not flagged as vllm", func(t *testing.T) {
if res := runOpenAICompatModule(t, vllm, 200, infinityModels); len(res.Findings) > 0 {
t.Errorf("an infinity list should not match the vllm module, got %d findings", len(res.Findings))
}
})
t.Run("an lmstudio models api is flagged with its model id", func(t *testing.T) {
res := runOpenAICompatModule(t, lmstudio, 200, lmstudioModels)
if len(res.Findings) == 0 {
t.Fatal("expected an lmstudio finding")
}
if v := openAICompatExtract(res, "lmstudio_model"); v != "qwen2-vl-7b-instruct" {
t.Errorf("lmstudio_model=%q, want qwen2-vl-7b-instruct", v)
}
})
t.Run("a body without compatibility_type is not flagged as lmstudio", func(t *testing.T) {
body := `{"object":"list","data":[{"id":"x","quantization":"4bit","max_context_length":4096}]}`
if res := runOpenAICompatModule(t, lmstudio, 200, body); len(res.Findings) > 0 {
t.Errorf("a body without compatibility_type should not match lmstudio, got %d findings", len(res.Findings))
}
})
t.Run("a body without quantization is not flagged as lmstudio", func(t *testing.T) {
body := `{"object":"list","data":[{"id":"x","compatibility_type":"gguf","max_context_length":4096}]}`
if res := runOpenAICompatModule(t, lmstudio, 200, body); len(res.Findings) > 0 {
t.Errorf("a body without quantization should not match lmstudio, got %d findings", len(res.Findings))
}
})
t.Run("a body without max_context_length is not flagged as lmstudio", func(t *testing.T) {
body := `{"object":"list","data":[{"id":"x","compatibility_type":"gguf","quantization":"4bit"}]}`
if res := runOpenAICompatModule(t, lmstudio, 200, body); len(res.Findings) > 0 {
t.Errorf("a body without max_context_length should not match lmstudio, got %d findings", len(res.Findings))
}
})
t.Run("an openai compatible v1 models list is not flagged as lmstudio", func(t *testing.T) {
if res := runOpenAICompatModule(t, lmstudio, 200, vllmModels); len(res.Findings) > 0 {
t.Errorf("a plain v1 models list should not match lmstudio, got %d findings", len(res.Findings))
}
})
t.Run("an lmstudio models list is not flagged as vllm", func(t *testing.T) {
if res := runOpenAICompatModule(t, vllm, 200, lmstudioModels); len(res.Findings) > 0 {
t.Errorf("an lmstudio list should not match the vllm module, got %d findings", len(res.Findings))
}
})
t.Run("a plain 200 body is not a leak", func(t *testing.T) {
for _, file := range []string{vllm, llamacpp, lmstudio, infinity} {
if res := runOpenAICompatModule(t, file, 200, "ok"); len(res.Findings) > 0 {
t.Errorf("%s: a plain 200 body should not match, got %d findings", file, len(res.Findings))
}
}
})
t.Run("a 404 is not a leak", func(t *testing.T) {
for _, file := range []string{vllm, llamacpp, lmstudio, infinity} {
if res := runOpenAICompatModule(t, file, 404, "not found"); len(res.Findings) > 0 {
t.Errorf("%s: a 404 should not match, got %d findings", file, len(res.Findings))
}
}
})
}
@@ -0,0 +1,201 @@
package modules_test
import (
"context"
"net/http"
"net/http/httptest"
"testing"
"time"
"github.com/vmfunc/sif/internal/modules"
)
func runRAGUIModule(t *testing.T, file string, status int, body string) *modules.Result {
t.Helper()
def, err := modules.ParseYAMLModule(file)
if err != nil {
t.Fatalf("parse %s: %v", file, err)
}
srv := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
w.WriteHeader(status)
_, _ = w.Write([]byte(body))
}))
defer srv.Close()
res, err := modules.ExecuteHTTPModule(context.Background(), srv.URL, def, modules.Options{
Timeout: 5 * time.Second,
Threads: 2,
})
if err != nil {
t.Fatalf("execute %s: %v", file, err)
}
return res
}
func ragUIExtract(res *modules.Result, key string) string {
for _, f := range res.Findings {
if v := f.Extracted[key]; v != "" {
return v
}
}
return ""
}
func TestLLMRAGUIExposureModules(t *testing.T) {
const fastgpt = "../../modules/recon/fastgpt-init-exposure.yaml"
const perplexica = "../../modules/recon/perplexica-config-exposure.yaml"
const onyx = "../../modules/recon/onyx-auth-exposure.yaml"
const verba = "../../modules/recon/verba-health-exposure.yaml"
fastgptInit := `{"code":200,"data":{"bufferId":"unAuth_x","feConfigs":{"systemTitle":"FastGPT",` +
`"docUrl":"https://doc.fastgpt.io","show_git":true},"modelProviders":[{"provider":"openai"}],` +
`"aiproxyChannels":[]}}`
perplexicaConfig := `{"values":{"general":{"theme":"dark","measureUnit":"Metric","autoMediaSearch":true,` +
`"showWeatherWidget":true},"modelProviders":[{"id":"openai","name":"OpenAI",` +
`"config":{"apiKey":"sk-secret","baseURL":""},"chatModels":[],"embeddingModels":[]}],` +
`"search":{"searxngURL":"http://searxng:8080"}},"fields":[{"key":"measureUnit"},{"key":"autoMediaSearch"},` +
`{"key":"searxngURL"}]}`
onyxAuth := `{"auth_type":"basic","requires_verification":false,"anonymous_user_enabled":false,` +
`"password_min_length":8,"has_users":true,"oauth_enabled":false}`
verbaHealth := `{"message":"Alive!","production":"Local","gtag":"",` +
`"deployments":{"WEAVIATE_URL_VERBA":"https://my-cluster.weaviate.network",` +
`"WEAVIATE_API_KEY_VERBA":"sk-weaviate-AbC123secret"},"default_deployment":"Weaviate"}`
t.Run("a fastgpt init is flagged with its system title", func(t *testing.T) {
res := runRAGUIModule(t, fastgpt, 200, fastgptInit)
if len(res.Findings) == 0 {
t.Fatal("expected a fastgpt finding")
}
if v := ragUIExtract(res, "fastgpt_title"); v != "FastGPT" {
t.Errorf("fastgpt_title=%q, want FastGPT", v)
}
})
t.Run("a body without feConfigs is not flagged as fastgpt", func(t *testing.T) {
body := `{"data":{"modelProviders":[],"aiproxyChannels":[]}}`
if res := runRAGUIModule(t, fastgpt, 200, body); len(res.Findings) > 0 {
t.Errorf("a body without feConfigs should not match fastgpt, got %d findings", len(res.Findings))
}
})
t.Run("a body without aiproxyChannels is not flagged as fastgpt", func(t *testing.T) {
body := `{"data":{"feConfigs":{},"modelProviders":[]}}`
if res := runRAGUIModule(t, fastgpt, 200, body); len(res.Findings) > 0 {
t.Errorf("a body without aiproxyChannels should not match fastgpt, got %d findings", len(res.Findings))
}
})
t.Run("a perplexica config is flagged with its searxng url", func(t *testing.T) {
res := runRAGUIModule(t, perplexica, 200, perplexicaConfig)
if len(res.Findings) == 0 {
t.Fatal("expected a perplexica finding")
}
if v := ragUIExtract(res, "perplexica_searxng"); v != "http://searxng:8080" {
t.Errorf("perplexica_searxng=%q, want http://searxng:8080", v)
}
})
t.Run("a body without searxngURL is not flagged as perplexica", func(t *testing.T) {
body := `{"values":{"general":{"measureUnit":"Metric","autoMediaSearch":true}}}`
if res := runRAGUIModule(t, perplexica, 200, body); len(res.Findings) > 0 {
t.Errorf("a body without searxngURL should not match perplexica, got %d findings", len(res.Findings))
}
})
t.Run("a body without autoMediaSearch is not flagged as perplexica", func(t *testing.T) {
body := `{"values":{"general":{"measureUnit":"Metric"},"search":{"searxngURL":"http://x"}}}`
if res := runRAGUIModule(t, perplexica, 200, body); len(res.Findings) > 0 {
t.Errorf("a body without autoMediaSearch should not match perplexica, got %d findings", len(res.Findings))
}
})
t.Run("an onyx auth type is flagged with its auth scheme", func(t *testing.T) {
res := runRAGUIModule(t, onyx, 200, onyxAuth)
if len(res.Findings) == 0 {
t.Fatal("expected an onyx finding")
}
if v := ragUIExtract(res, "onyx_auth_type"); v != "basic" {
t.Errorf("onyx_auth_type=%q, want basic", v)
}
})
t.Run("a body without anonymous_user_enabled is not flagged as onyx", func(t *testing.T) {
body := `{"auth_type":"basic","requires_verification":false,"password_min_length":8}`
if res := runRAGUIModule(t, onyx, 200, body); len(res.Findings) > 0 {
t.Errorf("a body without anonymous_user_enabled should not match onyx, got %d findings", len(res.Findings))
}
})
t.Run("a body without password_min_length is not flagged as onyx", func(t *testing.T) {
body := `{"auth_type":"basic","requires_verification":false,"anonymous_user_enabled":false}`
if res := runRAGUIModule(t, onyx, 200, body); len(res.Findings) > 0 {
t.Errorf("a body without password_min_length should not match onyx, got %d findings", len(res.Findings))
}
})
t.Run("a verba health is flagged and leaks the weaviate url", func(t *testing.T) {
res := runRAGUIModule(t, verba, 200, verbaHealth)
if len(res.Findings) == 0 {
t.Fatal("expected a verba finding")
}
if v := ragUIExtract(res, "verba_weaviate_url"); v != "https://my-cluster.weaviate.network" {
t.Errorf("verba_weaviate_url=%q, want https://my-cluster.weaviate.network", v)
}
})
t.Run("a body without WEAVIATE_API_KEY_VERBA is not flagged as verba", func(t *testing.T) {
body := `{"message":"Alive!","deployments":{"WEAVIATE_URL_VERBA":"http://x"},"default_deployment":""}`
if res := runRAGUIModule(t, verba, 200, body); len(res.Findings) > 0 {
t.Errorf("a body without WEAVIATE_API_KEY_VERBA should not match verba, got %d findings", len(res.Findings))
}
})
t.Run("a body without WEAVIATE_URL_VERBA is not flagged as verba", func(t *testing.T) {
body := `{"message":"Alive!","deployments":{"WEAVIATE_API_KEY_VERBA":"k"},"default_deployment":""}`
if res := runRAGUIModule(t, verba, 200, body); len(res.Findings) > 0 {
t.Errorf("a body without WEAVIATE_URL_VERBA should not match verba, got %d findings", len(res.Findings))
}
})
t.Run("a body without default_deployment is not flagged as verba", func(t *testing.T) {
body := `{"message":"Alive!","deployments":{"WEAVIATE_URL_VERBA":"http://x","WEAVIATE_API_KEY_VERBA":"k"}}`
if res := runRAGUIModule(t, verba, 200, body); len(res.Findings) > 0 {
t.Errorf("a body without default_deployment should not match verba, got %d findings", len(res.Findings))
}
})
t.Run("a verba health with an empty weaviate url is not flagged", func(t *testing.T) {
body := `{"message":"Alive!","production":"Local","deployments":{"WEAVIATE_URL_VERBA":"",` +
`"WEAVIATE_API_KEY_VERBA":""},"default_deployment":""}`
if res := runRAGUIModule(t, verba, 200, body); len(res.Findings) > 0 {
t.Errorf("an embedded verba that leaks no backend url should not match, got %d findings", len(res.Findings))
}
})
t.Run("a generic alive health is not flagged as verba", func(t *testing.T) {
body := `{"message":"Alive!","status":"ok","uptime":1234}`
if res := runRAGUIModule(t, verba, 200, body); len(res.Findings) > 0 {
t.Errorf("a generic health should not match verba, got %d findings", len(res.Findings))
}
})
t.Run("a plain 200 body is not a leak", func(t *testing.T) {
for _, file := range []string{perplexica, verba, onyx, fastgpt} {
if res := runRAGUIModule(t, file, 200, "ok"); len(res.Findings) > 0 {
t.Errorf("%s: a plain 200 body should not match, got %d findings", file, len(res.Findings))
}
}
})
t.Run("a 404 is not a leak", func(t *testing.T) {
for _, file := range []string{perplexica, verba, onyx, fastgpt} {
if res := runRAGUIModule(t, file, 404, "not found"); len(res.Findings) > 0 {
t.Errorf("%s: a 404 should not match, got %d findings", file, len(res.Findings))
}
}
})
}
+204
View File
@@ -0,0 +1,204 @@
package modules_test
import (
"context"
"net/http"
"net/http/httptest"
"testing"
"time"
"github.com/vmfunc/sif/internal/modules"
)
func runUIExposureModule(t *testing.T, file string, status int, body string) *modules.Result {
t.Helper()
def, err := modules.ParseYAMLModule(file)
if err != nil {
t.Fatalf("parse %s: %v", file, err)
}
srv := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
w.WriteHeader(status)
_, _ = w.Write([]byte(body))
}))
defer srv.Close()
res, err := modules.ExecuteHTTPModule(context.Background(), srv.URL, def, modules.Options{
Timeout: 5 * time.Second,
Threads: 2,
})
if err != nil {
t.Fatalf("execute %s: %v", file, err)
}
return res
}
func uiExtract(res *modules.Result, key string) string {
for _, f := range res.Findings {
if v := f.Extracted[key]; v != "" {
return v
}
}
return ""
}
func TestLLMUIExposureModules(t *testing.T) {
const openWebUI = "../../modules/recon/open-webui-exposure.yaml"
const librechat = "../../modules/recon/librechat-exposure.yaml"
const nextchat = "../../modules/recon/nextchat-config-exposure.yaml"
const anythingllm = "../../modules/recon/anythingllm-exposure.yaml"
openWebUIConfig := `{"status":true,"name":"Open WebUI","version":"0.6.15","default_locale":"",` +
`"oauth":{"providers":{}},"features":{"auth":false,"auth_trusted_header":false,` +
`"enable_ldap":false,"enable_signup":true,"enable_login_form":true,"enable_websocket":true}}`
librechatConfig := `{"appTitle":"LibreChat","serverDomain":"https://chat.example.com",` +
`"emailLoginEnabled":true,"registrationEnabled":true,"socialLogins":["google","github"]}`
nextchatConfig := `{"needCode":false,"hideUserApiKey":false,"disableGPT4":false,` +
`"hideBalanceQuery":true,"disableFastLink":false,"customModels":"","defaultModel":"gpt-4o-mini",` +
`"visionModels":""}`
anythingllmSetup := `{"results":{"RequiresAuth":false,"MultiUserMode":false,"EmbeddingEngine":"native",` +
`"VectorDB":"lancedb","LLMProvider":"openai","LLMModel":"gpt-4o","WhisperProvider":"local"}}`
t.Run("an open webui with auth disabled is flagged with its version", func(t *testing.T) {
res := runUIExposureModule(t, openWebUI, 200, openWebUIConfig)
if len(res.Findings) == 0 {
t.Fatal("expected an open webui finding")
}
if v := uiExtract(res, "open_webui_version"); v != "0.6.15" {
t.Errorf("open_webui_version=%q, want 0.6.15", v)
}
})
t.Run("a rebranded open webui with auth disabled is still flagged", func(t *testing.T) {
body := `{"status":true,"name":"Acme AI Portal","version":"0.6.15",` +
`"features":{"auth":false,"auth_trusted_header":false,"enable_signup":true}}`
res := runUIExposureModule(t, openWebUI, 200, body)
if len(res.Findings) == 0 {
t.Fatal("expected a finding for a rebranded open webui")
}
if v := uiExtract(res, "open_webui_version"); v != "0.6.15" {
t.Errorf("open_webui_version=%q, want 0.6.15", v)
}
})
t.Run("a librechat with open registration is flagged with its title", func(t *testing.T) {
res := runUIExposureModule(t, librechat, 200, librechatConfig)
if len(res.Findings) == 0 {
t.Fatal("expected a librechat finding")
}
if v := uiExtract(res, "librechat_title"); v != "LibreChat" {
t.Errorf("librechat_title=%q, want LibreChat", v)
}
})
t.Run("an open webui with auth enabled is not flagged", func(t *testing.T) {
body := `{"status":true,"name":"Open WebUI","version":"0.6.15",` +
`"features":{"auth":true,"auth_trusted_header":false,"enable_signup":true}}`
if res := runUIExposureModule(t, openWebUI, 200, body); len(res.Findings) > 0 {
t.Errorf("an auth-enabled open webui should not match, got %d findings", len(res.Findings))
}
})
t.Run("an open webui auth_trusted_header false does not satisfy the auth regex", func(t *testing.T) {
body := `{"status":true,"name":"Open WebUI","version":"0.6.15",` +
`"features":{"auth":true,"auth_trusted_header":false}}`
if res := runUIExposureModule(t, openWebUI, 200, body); len(res.Findings) > 0 {
t.Errorf("auth_trusted_header false should not match the auth regex, got %d findings", len(res.Findings))
}
})
t.Run("a librechat with registration disabled is not flagged", func(t *testing.T) {
body := `{"appTitle":"LibreChat","emailLoginEnabled":true,"registrationEnabled":false,"socialLogins":[]}`
if res := runUIExposureModule(t, librechat, 200, body); len(res.Findings) > 0 {
t.Errorf("a closed-registration librechat should not match, got %d findings", len(res.Findings))
}
})
t.Run("an unrelated app with open registration is not flagged as librechat", func(t *testing.T) {
body := `{"name":"otherapp","registrationEnabled":true}`
if res := runUIExposureModule(t, librechat, 200, body); len(res.Findings) > 0 {
t.Errorf("an unrelated app should not match librechat, got %d findings", len(res.Findings))
}
})
t.Run("a nextchat config is flagged and reports its access-code gate", func(t *testing.T) {
res := runUIExposureModule(t, nextchat, 200, nextchatConfig)
if len(res.Findings) == 0 {
t.Fatal("expected a nextchat finding")
}
if v := uiExtract(res, "nextchat_needcode"); v != "false" {
t.Errorf("nextchat_needcode=%q, want false", v)
}
})
t.Run("a body without needCode is not flagged as nextchat", func(t *testing.T) {
body := `{"hideUserApiKey":false,"hideBalanceQuery":true}`
if res := runUIExposureModule(t, nextchat, 200, body); len(res.Findings) > 0 {
t.Errorf("a body without needCode should not match nextchat, got %d findings", len(res.Findings))
}
})
t.Run("a body without hideUserApiKey is not flagged as nextchat", func(t *testing.T) {
body := `{"needCode":false,"hideBalanceQuery":true}`
if res := runUIExposureModule(t, nextchat, 200, body); len(res.Findings) > 0 {
t.Errorf("a body without hideUserApiKey should not match nextchat, got %d findings", len(res.Findings))
}
})
t.Run("a body without hideBalanceQuery is not flagged as nextchat", func(t *testing.T) {
body := `{"needCode":false,"hideUserApiKey":false}`
if res := runUIExposureModule(t, nextchat, 200, body); len(res.Findings) > 0 {
t.Errorf("a body without hideBalanceQuery should not match nextchat, got %d findings", len(res.Findings))
}
})
t.Run("a code-gated nextchat is not flagged", func(t *testing.T) {
body := `{"needCode":true,"hideUserApiKey":false,"hideBalanceQuery":true,"disableGPT4":false}`
if res := runUIExposureModule(t, nextchat, 200, body); len(res.Findings) > 0 {
t.Errorf("a needCode:true nextchat is access-code gated and should not match, got %d findings", len(res.Findings))
}
})
t.Run("an anythingllm setup is flagged with its model", func(t *testing.T) {
res := runUIExposureModule(t, anythingllm, 200, anythingllmSetup)
if len(res.Findings) == 0 {
t.Fatal("expected an anythingllm finding")
}
if v := uiExtract(res, "anythingllm_model"); v != "gpt-4o" {
t.Errorf("anythingllm_model=%q, want gpt-4o", v)
}
})
t.Run("a body without LLMProvider is not flagged as anythingllm", func(t *testing.T) {
body := `{"results":{"VectorDB":"lancedb","EmbeddingEngine":"native"}}`
if res := runUIExposureModule(t, anythingllm, 200, body); len(res.Findings) > 0 {
t.Errorf("a body without LLMProvider should not match anythingllm, got %d findings", len(res.Findings))
}
})
t.Run("a body without a VectorDB is not flagged as anythingllm", func(t *testing.T) {
body := `{"results":{"LLMProvider":"openai","EmbeddingEngine":"native"}}`
if res := runUIExposureModule(t, anythingllm, 200, body); len(res.Findings) > 0 {
t.Errorf("a body without VectorDB should not match anythingllm, got %d findings", len(res.Findings))
}
})
t.Run("a plain 200 body is not a leak", func(t *testing.T) {
for _, file := range []string{openWebUI, librechat, anythingllm, nextchat} {
if res := runUIExposureModule(t, file, 200, "ok"); len(res.Findings) > 0 {
t.Errorf("%s: a plain 200 body should not match, got %d findings", file, len(res.Findings))
}
}
})
t.Run("a 404 is not a leak", func(t *testing.T) {
for _, file := range []string{openWebUI, librechat, anythingllm, nextchat} {
if res := runUIExposureModule(t, file, 404, "not found"); len(res.Findings) > 0 {
t.Errorf("%s: a 404 should not match, got %d findings", file, len(res.Findings))
}
}
})
}
+1 -1
View File
@@ -19,7 +19,7 @@ import (
"runtime"
"github.com/charmbracelet/log"
"github.com/dropalldatabases/sif/internal/output"
"github.com/vmfunc/sif/internal/output"
)
// Loader handles module discovery and loading.
+1 -1
View File
@@ -19,7 +19,7 @@ import (
"testing"
"time"
"github.com/dropalldatabases/sif/internal/modules"
"github.com/vmfunc/sif/internal/modules"
)
func runLoginModule(t *testing.T, file string, status int, headers map[string]string, body string) *modules.Result {
@@ -7,7 +7,7 @@ import (
"testing"
"time"
"github.com/dropalldatabases/sif/internal/modules"
"github.com/vmfunc/sif/internal/modules"
)
func runMgmtModule(t *testing.T, file string, status int, body string) *modules.Result {
+1 -1
View File
@@ -21,7 +21,7 @@ import (
"path/filepath"
"testing"
"github.com/dropalldatabases/sif/internal/httpx"
"github.com/vmfunc/sif/internal/httpx"
)
func TestCheckMatchersCondition(t *testing.T) {
+1 -1
View File
@@ -19,7 +19,7 @@ import (
"testing"
"time"
"github.com/dropalldatabases/sif/internal/modules"
"github.com/vmfunc/sif/internal/modules"
)
func runMetricsModule(t *testing.T, file string, status int, body string) *modules.Result {
@@ -0,0 +1,133 @@
package modules_test
import (
"context"
"net/http"
"net/http/httptest"
"testing"
"time"
"github.com/vmfunc/sif/internal/modules"
)
func runMLPlatformModule(t *testing.T, file string, status int, body string) *modules.Result {
t.Helper()
def, err := modules.ParseYAMLModule(file)
if err != nil {
t.Fatalf("parse %s: %v", file, err)
}
srv := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
w.WriteHeader(status)
_, _ = w.Write([]byte(body))
}))
defer srv.Close()
res, err := modules.ExecuteHTTPModule(context.Background(), srv.URL, def, modules.Options{
Timeout: 5 * time.Second,
Threads: 2,
})
if err != nil {
t.Fatalf("execute %s: %v", file, err)
}
return res
}
func mlPlatformExtract(res *modules.Result, key string) string {
for _, f := range res.Findings {
if v := f.Extracted[key]; v != "" {
return v
}
}
return ""
}
func TestMLPlatformExposureModules(t *testing.T) {
const labelStudio = "../../modules/recon/label-studio-exposure.yaml"
const cvat = "../../modules/recon/cvat-server-exposure.yaml"
labelStudioVersion := `{"release":"1.13.1","label-studio-os-package":{"version":"1.13.1",` +
`"short_version":"1.13","description":"Label Studio"},"label-studio-os-backend":{"message":"release",` +
`"commit":"abc1234","date":"2024-06-01"}}`
cvatAbout := `{"name":"Computer Vision Annotation Tool","description":"CVAT is a re-designed annotation tool",` +
`"version":"2.20.0","logo_url":"http://host/static/logo.png","subtitle":""}`
t.Run("a label studio version api is flagged with its release", func(t *testing.T) {
res := runMLPlatformModule(t, labelStudio, 200, labelStudioVersion)
if len(res.Findings) == 0 {
t.Fatal("expected a label studio finding")
}
if v := mlPlatformExtract(res, "label_studio_release"); v != "1.13.1" {
t.Errorf("label_studio_release=%q, want 1.13.1", v)
}
})
t.Run("a generic release version is not flagged as label studio", func(t *testing.T) {
body := `{"release":"1.0","name":"some-app"}`
if res := runMLPlatformModule(t, labelStudio, 200, body); len(res.Findings) > 0 {
t.Errorf("a generic release should not match label studio, got %d findings", len(res.Findings))
}
})
t.Run("a label studio package without the backend key is not flagged", func(t *testing.T) {
body := `{"release":"1.13.1","label-studio-os-package":{"version":"1.13.1"}}`
if res := runMLPlatformModule(t, labelStudio, 200, body); len(res.Findings) > 0 {
t.Errorf("a package-only body should not match label studio, got %d findings", len(res.Findings))
}
})
t.Run("a label studio backend without the package key is not flagged", func(t *testing.T) {
body := `{"release":"1.13.1","label-studio-os-backend":{"commit":"abc"}}`
if res := runMLPlatformModule(t, labelStudio, 200, body); len(res.Findings) > 0 {
t.Errorf("a backend-only body should not match label studio, got %d findings", len(res.Findings))
}
})
t.Run("a cvat about is flagged with its version", func(t *testing.T) {
res := runMLPlatformModule(t, cvat, 200, cvatAbout)
if len(res.Findings) == 0 {
t.Fatal("expected a cvat finding")
}
if v := mlPlatformExtract(res, "cvat_version"); v != "2.20.0" {
t.Errorf("cvat_version=%q, want 2.20.0", v)
}
})
t.Run("another annotation tool is not flagged as cvat", func(t *testing.T) {
body := `{"name":"Some Other Tool","version":"1.0","logo_url":"http://x/l.png"}`
if res := runMLPlatformModule(t, cvat, 200, body); len(res.Findings) > 0 {
t.Errorf("a non-cvat name should not match cvat, got %d findings", len(res.Findings))
}
})
t.Run("an html page mentioning cvat is not flagged", func(t *testing.T) {
body := `<html><body><h1>Computer Vision Annotation Tool</h1> version 2.20.0 logo_url</body></html>`
if res := runMLPlatformModule(t, cvat, 200, body); len(res.Findings) > 0 {
t.Errorf("prose mentioning cvat should not match the structured response, got %d findings", len(res.Findings))
}
})
t.Run("a cvat about without a logo_url is not flagged", func(t *testing.T) {
body := `{"name":"Computer Vision Annotation Tool","version":"2.20.0"}`
if res := runMLPlatformModule(t, cvat, 200, body); len(res.Findings) > 0 {
t.Errorf("a body without logo_url should not match cvat, got %d findings", len(res.Findings))
}
})
t.Run("a plain 200 body is not a leak", func(t *testing.T) {
for _, file := range []string{cvat, labelStudio} {
if res := runMLPlatformModule(t, file, 200, "ok"); len(res.Findings) > 0 {
t.Errorf("%s: a plain 200 body should not match, got %d findings", file, len(res.Findings))
}
}
})
t.Run("a 404 is not a leak", func(t *testing.T) {
for _, file := range []string{cvat, labelStudio} {
if res := runMLPlatformModule(t, file, 404, "not found"); len(res.Findings) > 0 {
t.Errorf("%s: a 404 should not match, got %d findings", file, len(res.Findings))
}
}
})
}
@@ -0,0 +1,161 @@
package modules_test
import (
"context"
"net/http"
"net/http/httptest"
"testing"
"time"
"github.com/vmfunc/sif/internal/modules"
)
func runMLPlatformServerModule(t *testing.T, file string, status int, body string) *modules.Result {
t.Helper()
def, err := modules.ParseYAMLModule(file)
if err != nil {
t.Fatalf("parse %s: %v", file, err)
}
srv := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
w.WriteHeader(status)
_, _ = w.Write([]byte(body))
}))
defer srv.Close()
res, err := modules.ExecuteHTTPModule(context.Background(), srv.URL, def, modules.Options{
Timeout: 5 * time.Second,
Threads: 2,
})
if err != nil {
t.Fatalf("execute %s: %v", file, err)
}
return res
}
func mlPlatformServerExtract(res *modules.Result, key string) string {
for _, f := range res.Findings {
if v := f.Extracted[key]; v != "" {
return v
}
}
return ""
}
func TestMLPlatformServerExposureModules(t *testing.T) {
const h2o = "../../modules/recon/h2o-cluster-exposure.yaml"
const mindsdb = "../../modules/recon/mindsdb-api-exposure.yaml"
const zenml = "../../modules/recon/zenml-server-exposure.yaml"
h2oCloud := `{"__meta":{"schema_name":"CloudV3"},"version":"3.46.0.6","branch_name":"rel-x",` +
`"cloud_name":"H2O_started_from_python","cloud_size":1,"cloud_uptime_millis":123456,` +
`"cloud_healthy":true,"build_too_old":false}`
mindsdbStatus := `{"environment":"local","mindsdb_version":"25.13.1",` +
`"auth":{"confirmed":false,"required":false,"provider":"disabled"}}`
zenmlInfo := `{"id":"abc","version":"0.70.0","deployment_type":"docker","database_type":"sqlite",` +
`"secrets_store_type":"sql","auth_scheme":"OAUTH2_PASSWORD_BEARER","analytics_enabled":true}`
t.Run("an h2o cloud is flagged with its version", func(t *testing.T) {
res := runMLPlatformServerModule(t, h2o, 200, h2oCloud)
if len(res.Findings) == 0 {
t.Fatal("expected an h2o finding")
}
if v := mlPlatformServerExtract(res, "h2o_version"); v != "3.46.0.6" {
t.Errorf("h2o_version=%q, want 3.46.0.6", v)
}
})
t.Run("a body without cloud_name is not flagged as h2o", func(t *testing.T) {
body := `{"cloud_uptime_millis":1,"build_too_old":false,"version":"3.46"}`
if res := runMLPlatformServerModule(t, h2o, 200, body); len(res.Findings) > 0 {
t.Errorf("a body without cloud_name should not match h2o, got %d findings", len(res.Findings))
}
})
t.Run("a body without build_too_old is not flagged as h2o", func(t *testing.T) {
body := `{"cloud_name":"x","cloud_uptime_millis":1}`
if res := runMLPlatformServerModule(t, h2o, 200, body); len(res.Findings) > 0 {
t.Errorf("a body without build_too_old should not match h2o, got %d findings", len(res.Findings))
}
})
t.Run("a generic cloud status is not flagged as h2o", func(t *testing.T) {
body := `{"cloud_name":"x","build_too_old":false}`
if res := runMLPlatformServerModule(t, h2o, 200, body); len(res.Findings) > 0 {
t.Errorf("a body without cloud_uptime_millis should not match h2o, got %d findings", len(res.Findings))
}
})
t.Run("a mindsdb status is flagged with its version", func(t *testing.T) {
res := runMLPlatformServerModule(t, mindsdb, 200, mindsdbStatus)
if len(res.Findings) == 0 {
t.Fatal("expected a mindsdb finding")
}
if v := mlPlatformServerExtract(res, "mindsdb_version"); v != "25.13.1" {
t.Errorf("mindsdb_version=%q, want 25.13.1", v)
}
})
t.Run("a body without mindsdb_version is not flagged", func(t *testing.T) {
body := `{"environment":"local","auth":{"provider":"disabled"}}`
if res := runMLPlatformServerModule(t, mindsdb, 200, body); len(res.Findings) > 0 {
t.Errorf("a body without mindsdb_version should not match mindsdb, got %d findings", len(res.Findings))
}
})
t.Run("a body without an auth provider is not flagged as mindsdb", func(t *testing.T) {
body := `{"environment":"local","mindsdb_version":"25.13.1","auth":{"required":false}}`
if res := runMLPlatformServerModule(t, mindsdb, 200, body); len(res.Findings) > 0 {
t.Errorf("a body without provider should not match mindsdb, got %d findings", len(res.Findings))
}
})
t.Run("a zenml info is flagged with its version", func(t *testing.T) {
res := runMLPlatformServerModule(t, zenml, 200, zenmlInfo)
if len(res.Findings) == 0 {
t.Fatal("expected a zenml finding")
}
if v := mlPlatformServerExtract(res, "zenml_version"); v != "0.70.0" {
t.Errorf("zenml_version=%q, want 0.70.0", v)
}
})
t.Run("a body without deployment_type is not flagged as zenml", func(t *testing.T) {
body := `{"secrets_store_type":"sql","auth_scheme":"OAUTH2","version":"0.70.0"}`
if res := runMLPlatformServerModule(t, zenml, 200, body); len(res.Findings) > 0 {
t.Errorf("a body without deployment_type should not match zenml, got %d findings", len(res.Findings))
}
})
t.Run("a body without secrets_store_type is not flagged as zenml", func(t *testing.T) {
body := `{"deployment_type":"docker","auth_scheme":"OAUTH2"}`
if res := runMLPlatformServerModule(t, zenml, 200, body); len(res.Findings) > 0 {
t.Errorf("a body without secrets_store_type should not match zenml, got %d findings", len(res.Findings))
}
})
t.Run("a body without auth_scheme is not flagged as zenml", func(t *testing.T) {
body := `{"deployment_type":"docker","secrets_store_type":"sql"}`
if res := runMLPlatformServerModule(t, zenml, 200, body); len(res.Findings) > 0 {
t.Errorf("a body without auth_scheme should not match zenml, got %d findings", len(res.Findings))
}
})
t.Run("a plain 200 body is not a leak", func(t *testing.T) {
for _, file := range []string{h2o, mindsdb, zenml} {
if res := runMLPlatformServerModule(t, file, 200, "ok"); len(res.Findings) > 0 {
t.Errorf("%s: a plain 200 body should not match, got %d findings", file, len(res.Findings))
}
}
})
t.Run("a 404 is not a leak", func(t *testing.T) {
for _, file := range []string{h2o, mindsdb, zenml} {
if res := runMLPlatformServerModule(t, file, 404, "not found"); len(res.Findings) > 0 {
t.Errorf("%s: a 404 should not match, got %d findings", file, len(res.Findings))
}
}
})
}
@@ -0,0 +1,223 @@
package modules_test
import (
"context"
"net/http"
"net/http/httptest"
"testing"
"time"
"github.com/vmfunc/sif/internal/modules"
)
func runTrackingModule(t *testing.T, file string, status int, body string) *modules.Result {
t.Helper()
def, err := modules.ParseYAMLModule(file)
if err != nil {
t.Fatalf("parse %s: %v", file, err)
}
srv := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
w.WriteHeader(status)
_, _ = w.Write([]byte(body))
}))
defer srv.Close()
res, err := modules.ExecuteHTTPModule(context.Background(), srv.URL, def, modules.Options{
Timeout: 5 * time.Second,
Threads: 2,
})
if err != nil {
t.Fatalf("execute %s: %v", file, err)
}
return res
}
func trackingExtract(res *modules.Result, key string) string {
for _, f := range res.Findings {
if v := f.Extracted[key]; v != "" {
return v
}
}
return ""
}
func TestMLTrackingExposureModules(t *testing.T) {
const mlflow = "../../modules/recon/mlflow-api-exposure.yaml"
const tensorboard = "../../modules/recon/tensorboard-exposure.yaml"
const aim = "../../modules/recon/aim-exposure.yaml"
const determined = "../../modules/recon/determined-master-exposure.yaml"
mlflowExperiment := `{"experiment":{"experiment_id":"0","name":"Default",` +
`"artifact_location":"file:///mlflow/mlruns/0","lifecycle_stage":"active",` +
`"creation_time":1700000000000,"last_update_time":1700000000000,"tags":[]}}`
tensorboardEnv := `{"data_location":"/home/ml/runs/exp-2024","window_title":"",` +
`"experiment_name":"","experiment_description":"","creation_time":0,"version":"2.16.2"}`
aimProject := `{"name":"my-aim-repo","path":"/home/ml/.aim","description":"",` +
`"telemetry_enabled":0,"warn_index":false,"warn_runs":false}`
determinedMaster := `{"version":"0.27.1","master_id":"6f1f2a9c","cluster_id":"a1b2c3d4-e5f6-7890",` +
`"cluster_name":"prod-cluster","telemetry_enabled":true,"rbac_enabled":false,` +
`"strict_job_queue_control":false,"has_custom_logo":false,"branding":"determined"}`
t.Run("an mlflow experiment is flagged with its artifact store", func(t *testing.T) {
res := runTrackingModule(t, mlflow, 200, mlflowExperiment)
if len(res.Findings) == 0 {
t.Fatal("expected an mlflow finding")
}
if v := trackingExtract(res, "mlflow_artifact_location"); v != "file:///mlflow/mlruns/0" {
t.Errorf("mlflow_artifact_location=%q, want file:///mlflow/mlruns/0", v)
}
})
t.Run("an experiment_id without lifecycle_stage is not flagged as mlflow", func(t *testing.T) {
body := `{"experiment_id":"5","artifact_location":"s3://bucket/x"}`
if res := runTrackingModule(t, mlflow, 200, body); len(res.Findings) > 0 {
t.Errorf("a partial body should not match mlflow, got %d findings", len(res.Findings))
}
})
t.Run("an experiment without an artifact_location is not flagged as mlflow", func(t *testing.T) {
body := `{"experiment":{"experiment_id":"0","name":"Default","lifecycle_stage":"active"}}`
if res := runTrackingModule(t, mlflow, 200, body); len(res.Findings) > 0 {
t.Errorf("an artifactless experiment should not match mlflow, got %d findings", len(res.Findings))
}
})
t.Run("an experiment without an experiment_id is not flagged as mlflow", func(t *testing.T) {
body := `{"experiment":{"name":"Default","artifact_location":"file:///x","lifecycle_stage":"active"}}`
if res := runTrackingModule(t, mlflow, 200, body); len(res.Findings) > 0 {
t.Errorf("an idless experiment should not match mlflow, got %d findings", len(res.Findings))
}
})
t.Run("a generic lifecycle body is not flagged as mlflow", func(t *testing.T) {
body := `{"lifecycle_stage":"production","name":"some-service"}`
if res := runTrackingModule(t, mlflow, 200, body); len(res.Findings) > 0 {
t.Errorf("a generic lifecycle body should not match mlflow, got %d findings", len(res.Findings))
}
})
t.Run("a model checkpoint list is not flagged as mlflow", func(t *testing.T) {
body := `[{"title":"x","model_name":"y","filename":"z"}]`
if res := runTrackingModule(t, mlflow, 200, body); len(res.Findings) > 0 {
t.Errorf("a model list should not match mlflow, got %d findings", len(res.Findings))
}
})
t.Run("a tensorboard environment is flagged with its version and run path", func(t *testing.T) {
res := runTrackingModule(t, tensorboard, 200, tensorboardEnv)
if len(res.Findings) == 0 {
t.Fatal("expected a tensorboard finding")
}
if v := trackingExtract(res, "tensorboard_version"); v != "2.16.2" {
t.Errorf("tensorboard_version=%q, want 2.16.2", v)
}
if v := trackingExtract(res, "tensorboard_data_location"); v != "/home/ml/runs/exp-2024" {
t.Errorf("tensorboard_data_location=%q, want /home/ml/runs/exp-2024", v)
}
})
t.Run("a body without data_location is not flagged as tensorboard", func(t *testing.T) {
body := `{"window_title":"","version":"2.16.2"}`
if res := runTrackingModule(t, tensorboard, 200, body); len(res.Findings) > 0 {
t.Errorf("a body without data_location should not match tensorboard, got %d findings", len(res.Findings))
}
})
t.Run("a body without window_title is not flagged as tensorboard", func(t *testing.T) {
body := `{"data_location":"/runs","version":"2.16.2"}`
if res := runTrackingModule(t, tensorboard, 200, body); len(res.Findings) > 0 {
t.Errorf("a body without window_title should not match tensorboard, got %d findings", len(res.Findings))
}
})
t.Run("a body without a version is not flagged as tensorboard", func(t *testing.T) {
body := `{"data_location":"/runs","window_title":"my board"}`
if res := runTrackingModule(t, tensorboard, 200, body); len(res.Findings) > 0 {
t.Errorf("a body without version should not match tensorboard, got %d findings", len(res.Findings))
}
})
t.Run("an aim project is flagged with its repo path", func(t *testing.T) {
res := runTrackingModule(t, aim, 200, aimProject)
if len(res.Findings) == 0 {
t.Fatal("expected an aim finding")
}
if v := trackingExtract(res, "aim_project_path"); v != "/home/ml/.aim" {
t.Errorf("aim_project_path=%q, want /home/ml/.aim", v)
}
})
t.Run("a body without telemetry_enabled is not flagged as aim", func(t *testing.T) {
body := `{"name":"x","path":"/y","warn_index":false,"warn_runs":false}`
if res := runTrackingModule(t, aim, 200, body); len(res.Findings) > 0 {
t.Errorf("a body without telemetry_enabled should not match aim, got %d findings", len(res.Findings))
}
})
t.Run("a body without warn_index is not flagged as aim", func(t *testing.T) {
body := `{"telemetry_enabled":0,"warn_runs":false,"name":"x"}`
if res := runTrackingModule(t, aim, 200, body); len(res.Findings) > 0 {
t.Errorf("a body without warn_index should not match aim, got %d findings", len(res.Findings))
}
})
t.Run("a body without warn_runs is not flagged as aim", func(t *testing.T) {
body := `{"telemetry_enabled":0,"warn_index":false,"name":"x"}`
if res := runTrackingModule(t, aim, 200, body); len(res.Findings) > 0 {
t.Errorf("a body without warn_runs should not match aim, got %d findings", len(res.Findings))
}
})
t.Run("a determined master is flagged with its version and cluster id", func(t *testing.T) {
res := runTrackingModule(t, determined, 200, determinedMaster)
if len(res.Findings) == 0 {
t.Fatal("expected a determined finding")
}
if v := trackingExtract(res, "determined_version"); v != "0.27.1" {
t.Errorf("determined_version=%q, want 0.27.1", v)
}
if v := trackingExtract(res, "determined_cluster_id"); v != "a1b2c3d4-e5f6-7890" {
t.Errorf("determined_cluster_id=%q, want a1b2c3d4-e5f6-7890", v)
}
})
t.Run("a cluster info without a master_id is not flagged as determined", func(t *testing.T) {
body := `{"cluster_id":"x","cluster_name":"y","version":"1.0"}`
if res := runTrackingModule(t, determined, 200, body); len(res.Findings) > 0 {
t.Errorf("a masterless cluster info should not match determined, got %d findings", len(res.Findings))
}
})
t.Run("a master without a cluster_id is not flagged as determined", func(t *testing.T) {
body := `{"master_id":"x","cluster_name":"y","version":"1.0"}`
if res := runTrackingModule(t, determined, 200, body); len(res.Findings) > 0 {
t.Errorf("a body without cluster_id should not match determined, got %d findings", len(res.Findings))
}
})
t.Run("a master without a cluster_name is not flagged as determined", func(t *testing.T) {
body := `{"master_id":"x","cluster_id":"y","version":"1.0"}`
if res := runTrackingModule(t, determined, 200, body); len(res.Findings) > 0 {
t.Errorf("a body without cluster_name should not match determined, got %d findings", len(res.Findings))
}
})
t.Run("a plain 200 body is not a leak", func(t *testing.T) {
for _, file := range []string{mlflow, tensorboard, aim, determined} {
if res := runTrackingModule(t, file, 200, "ok"); len(res.Findings) > 0 {
t.Errorf("%s: a plain 200 body should not match, got %d findings", file, len(res.Findings))
}
}
})
t.Run("a 404 is not a leak", func(t *testing.T) {
for _, file := range []string{mlflow, tensorboard, aim, determined} {
if res := runTrackingModule(t, file, 404, "not found"); len(res.Findings) > 0 {
t.Errorf("%s: a 404 should not match, got %d findings", file, len(res.Findings))
}
}
})
}
+1 -1
View File
@@ -93,7 +93,7 @@ type Matcher struct {
Status []int `yaml:"status,omitempty"`
Size []int `yaml:"size,omitempty"`
Hash []int64 `yaml:"hash,omitempty"` // favicon: shodan mmh3 hashes (signed or unsigned)
Condition string `yaml:"condition"` // and, or
Condition string `yaml:"condition"` // and, or
Negative bool `yaml:"negative"`
}
@@ -0,0 +1,84 @@
package modules_test
import (
"context"
"net/http"
"net/http/httptest"
"testing"
"time"
"github.com/vmfunc/sif/internal/modules"
)
func runN8nModule(t *testing.T, file string, status int, body string) *modules.Result {
t.Helper()
def, err := modules.ParseYAMLModule(file)
if err != nil {
t.Fatalf("parse %s: %v", file, err)
}
srv := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
w.WriteHeader(status)
_, _ = w.Write([]byte(body))
}))
defer srv.Close()
res, err := modules.ExecuteHTTPModule(context.Background(), srv.URL, def, modules.Options{
Timeout: 5 * time.Second,
Threads: 2,
})
if err != nil {
t.Fatalf("execute %s: %v", file, err)
}
return res
}
func n8nExtract(res *modules.Result, key string) string {
for _, f := range res.Findings {
if v := f.Extracted[key]; v != "" {
return v
}
}
return ""
}
func TestN8nSettingsExposureModule(t *testing.T) {
const n8n = "../../modules/recon/n8n-settings-exposure.yaml"
t.Run("an n8n settings response is flagged with the version", func(t *testing.T) {
body := `{"data":{"endpointWebhook":"webhook","endpointWebhookTest":"webhook-test",` +
`"urlBaseWebhook":"https://n8n.example.com/","urlBaseEditor":"https://n8n.example.com/",` +
`"versionCli":"1.45.1","releaseChannel":"stable","instanceId":"abc123def","n8nMetadata":{},` +
`"userManagement":{"showSetupOnFirstLoad":false,"smtpSetup":true,"authenticationMethod":"email"}}}`
res := runN8nModule(t, n8n, 200, body)
if len(res.Findings) == 0 {
t.Fatal("expected an n8n finding")
}
if v := n8nExtract(res, "n8n_version"); v != "1.45.1" {
t.Errorf("n8n_version=%q, want 1.45.1", v)
}
})
t.Run("a settings blob without instanceId is not flagged", func(t *testing.T) {
if res := runN8nModule(t, n8n, 200, `{"data":{"endpointWebhook":"webhook","versionCli":"1.45.1"}}`); len(res.Findings) > 0 {
t.Errorf("an instanceless settings blob should not match, got %d findings", len(res.Findings))
}
})
t.Run("a settings blob without endpointWebhook is not flagged", func(t *testing.T) {
if res := runN8nModule(t, n8n, 200, `{"data":{"versionCli":"1.45.1","instanceId":"abc"}}`); len(res.Findings) > 0 {
t.Errorf("a webhookless settings blob should not match, got %d findings", len(res.Findings))
}
})
t.Run("a plain 200 body is not a leak", func(t *testing.T) {
if res := runN8nModule(t, n8n, 200, "ok"); len(res.Findings) > 0 {
t.Errorf("a plain 200 body should not match, got %d findings", len(res.Findings))
}
})
t.Run("a 404 is not a leak", func(t *testing.T) {
if res := runN8nModule(t, n8n, 404, "not found"); len(res.Findings) > 0 {
t.Errorf("a 404 should not match, got %d findings", len(res.Findings))
}
})
}
@@ -0,0 +1,89 @@
package modules_test
import (
"context"
"net/http"
"net/http/httptest"
"testing"
"time"
"github.com/vmfunc/sif/internal/modules"
)
func runNodeRedModule(t *testing.T, file string, status int, body string) *modules.Result {
t.Helper()
def, err := modules.ParseYAMLModule(file)
if err != nil {
t.Fatalf("parse %s: %v", file, err)
}
srv := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
w.WriteHeader(status)
_, _ = w.Write([]byte(body))
}))
defer srv.Close()
res, err := modules.ExecuteHTTPModule(context.Background(), srv.URL, def, modules.Options{
Timeout: 5 * time.Second,
Threads: 2,
})
if err != nil {
t.Fatalf("execute %s: %v", file, err)
}
return res
}
func noderedExtract(res *modules.Result, key string) string {
for _, f := range res.Findings {
if v := f.Extracted[key]; v != "" {
return v
}
}
return ""
}
func TestNodeREDFlowExposureModule(t *testing.T) {
const nodered = "../../modules/recon/nodered-flow-exposure.yaml"
t.Run("an open node-red flows export is flagged with its tab label", func(t *testing.T) {
body := `[{"id":"396c2376.c693dc","type":"tab","label":"Sheet 1"},` +
`{"id":"a1","type":"inject","z":"396c2376.c693dc","wires":[["b2"]]},` +
`{"id":"b2","type":"function","z":"396c2376.c693dc","func":"return msg;","wires":[[]]}]`
res := runNodeRedModule(t, nodered, 200, body)
if len(res.Findings) == 0 {
t.Fatal("expected a node-red finding")
}
if v := noderedExtract(res, "nodered_flow_label"); v != "Sheet 1" {
t.Errorf("nodered_flow_label=%q, want Sheet 1", v)
}
})
t.Run("an adminAuth node-red returns 401 and is not flagged", func(t *testing.T) {
if res := runNodeRedModule(t, nodered, 401, `{"message":"Unauthorized"}`); len(res.Findings) > 0 {
t.Errorf("a 401 from a secured node-red should not match, got %d findings", len(res.Findings))
}
})
t.Run("a tabs-only flow without wires is not flagged", func(t *testing.T) {
if res := runNodeRedModule(t, nodered, 200, `[{"id":"x","type":"tab","label":"Home"}]`); len(res.Findings) > 0 {
t.Errorf("a tabs-only flow should not match, got %d findings", len(res.Findings))
}
})
t.Run("a wired graph without a tab is not flagged as node-red", func(t *testing.T) {
if res := runNodeRedModule(t, nodered, 200, `[{"id":"x","type":"section","wires":[["y"]]}]`); len(res.Findings) > 0 {
t.Errorf("a wired graph without a tab should not match, got %d findings", len(res.Findings))
}
})
t.Run("a plain 200 body is not a leak", func(t *testing.T) {
if res := runNodeRedModule(t, nodered, 200, "ok"); len(res.Findings) > 0 {
t.Errorf("a plain 200 body should not match, got %d findings", len(res.Findings))
}
})
t.Run("a 404 is not a leak", func(t *testing.T) {
if res := runNodeRedModule(t, nodered, 404, "not found"); len(res.Findings) > 0 {
t.Errorf("a 404 should not match, got %d findings", len(res.Findings))
}
})
}
@@ -0,0 +1,135 @@
package modules_test
import (
"context"
"net/http"
"net/http/httptest"
"testing"
"time"
"github.com/vmfunc/sif/internal/modules"
)
func runObservabilityModule(t *testing.T, file string, status int, body string) *modules.Result {
t.Helper()
def, err := modules.ParseYAMLModule(file)
if err != nil {
t.Fatalf("parse %s: %v", file, err)
}
srv := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
w.WriteHeader(status)
_, _ = w.Write([]byte(body))
}))
defer srv.Close()
res, err := modules.ExecuteHTTPModule(context.Background(), srv.URL, def, modules.Options{
Timeout: 5 * time.Second,
Threads: 2,
})
if err != nil {
t.Fatalf("execute %s: %v", file, err)
}
return res
}
func observExtract(res *modules.Result, key string) string {
for _, f := range res.Findings {
if v := f.Extracted[key]; v != "" {
return v
}
}
return ""
}
func TestObservabilityExposureModules(t *testing.T) {
const loki = "../../modules/recon/loki-api-exposure.yaml"
const jaeger = "../../modules/recon/jaeger-query-exposure.yaml"
const zipkin = "../../modules/recon/zipkin-exposure.yaml"
t.Run("an open loki labels response is flagged with a label", func(t *testing.T) {
body := `{"status":"success","data":["app","filename","job","namespace","pod"]}`
res := runObservabilityModule(t, loki, 200, body)
if len(res.Findings) == 0 {
t.Fatal("expected a loki finding")
}
if v := observExtract(res, "loki_label"); v != "app" {
t.Errorf("loki_label=%q, want app", v)
}
})
t.Run("an open loki with no ingested labels is still flagged", func(t *testing.T) {
if res := runObservabilityModule(t, loki, 200, `{"status":"success","data":[]}`); len(res.Findings) == 0 {
t.Error("expected a loki finding for an empty-but-open instance")
}
})
t.Run("a multi-tenant loki returns 401 and is not flagged", func(t *testing.T) {
if res := runObservabilityModule(t, loki, 401, `no org id\n`); len(res.Findings) > 0 {
t.Errorf("a 401 from a secured loki should not match, got %d findings", len(res.Findings))
}
})
t.Run("a non-loki success envelope is not flagged as loki", func(t *testing.T) {
if res := runObservabilityModule(t, loki, 200, `{"ok":true,"items":[]}`); len(res.Findings) > 0 {
t.Errorf("a body without the loki shape should not match, got %d findings", len(res.Findings))
}
})
t.Run("a jaeger service list is flagged with a service name", func(t *testing.T) {
body := `{"data":["customer","driver","frontend","route"],"total":0,"limit":0,"offset":0,"errors":null}`
res := runObservabilityModule(t, jaeger, 200, body)
if len(res.Findings) == 0 {
t.Fatal("expected a jaeger finding")
}
if v := observExtract(res, "jaeger_service"); v != "customer" {
t.Errorf("jaeger_service=%q, want customer", v)
}
})
t.Run("a generic pagination envelope without errors is not flagged as jaeger", func(t *testing.T) {
body := `{"data":["a","b"],"total":2,"limit":10,"offset":0}`
if res := runObservabilityModule(t, jaeger, 200, body); len(res.Findings) > 0 {
t.Errorf("an envelope without errors should not match jaeger, got %d findings", len(res.Findings))
}
})
t.Run("a bare data array is not flagged as jaeger", func(t *testing.T) {
if res := runObservabilityModule(t, jaeger, 200, `{"data":["x"]}`); len(res.Findings) > 0 {
t.Errorf("a bare data array should not match jaeger, got %d findings", len(res.Findings))
}
})
t.Run("a zipkin config is flagged with its environment", func(t *testing.T) {
body := `{"environment":"prod","queryLimit":10,"defaultLookback":900000,"searchEnabled":true,` +
`"dependency":{"enabled":true,"lowErrorRate":0.5,"highErrorRate":0.75}}`
res := runObservabilityModule(t, zipkin, 200, body)
if len(res.Findings) == 0 {
t.Fatal("expected a zipkin finding")
}
if v := observExtract(res, "zipkin_environment"); v != "prod" {
t.Errorf("zipkin_environment=%q, want prod", v)
}
})
t.Run("a config with searchEnabled alone is not flagged as zipkin", func(t *testing.T) {
if res := runObservabilityModule(t, zipkin, 200, `{"searchEnabled":true,"foo":1}`); len(res.Findings) > 0 {
t.Errorf("a partial config should not match zipkin, got %d findings", len(res.Findings))
}
})
t.Run("a plain 200 body is not a leak", func(t *testing.T) {
for _, file := range []string{loki, jaeger, zipkin} {
if res := runObservabilityModule(t, file, 200, "ok"); len(res.Findings) > 0 {
t.Errorf("%s: a plain 200 body should not match, got %d findings", file, len(res.Findings))
}
}
})
t.Run("a 404 is not a leak", func(t *testing.T) {
for _, file := range []string{loki, jaeger, zipkin} {
if res := runObservabilityModule(t, file, 404, "not found"); len(res.Findings) > 0 {
t.Errorf("%s: a 404 should not match, got %d findings", file, len(res.Findings))
}
}
})
}
+1 -1
View File
@@ -7,7 +7,7 @@ import (
"testing"
"time"
"github.com/dropalldatabases/sif/internal/modules"
"github.com/vmfunc/sif/internal/modules"
)
// runOpsModule runs a shipped module end to end against a server that returns
@@ -7,7 +7,7 @@ import (
"testing"
"time"
"github.com/dropalldatabases/sif/internal/modules"
"github.com/vmfunc/sif/internal/modules"
)
func runOrchModule(t *testing.T, file string, status int, body string) *modules.Result {
@@ -7,7 +7,7 @@ import (
"testing"
"time"
"github.com/dropalldatabases/sif/internal/modules"
"github.com/vmfunc/sif/internal/modules"
)
func runRailsModule(t *testing.T, file string, status int, body string) *modules.Result {
+1 -1
View File
@@ -7,7 +7,7 @@ import (
"testing"
"time"
"github.com/dropalldatabases/sif/internal/modules"
"github.com/vmfunc/sif/internal/modules"
)
func runRegistryModule(t *testing.T, file string, status int, headers map[string]string, body string) *modules.Result {
@@ -19,7 +19,7 @@ import (
"testing"
"time"
"github.com/dropalldatabases/sif/internal/modules"
"github.com/vmfunc/sif/internal/modules"
)
func runRuntimeModule(t *testing.T, file string, status int, body string) *modules.Result {
@@ -7,7 +7,7 @@ import (
"testing"
"time"
"github.com/dropalldatabases/sif/internal/modules"
"github.com/vmfunc/sif/internal/modules"
)
func runSecretModule(t *testing.T, file string, status int, body string) *modules.Result {
@@ -0,0 +1,101 @@
package modules_test
import (
"context"
"net/http"
"net/http/httptest"
"testing"
"time"
"github.com/vmfunc/sif/internal/modules"
)
func runServerAdminModule(t *testing.T, file string, status int, body string) *modules.Result {
t.Helper()
def, err := modules.ParseYAMLModule(file)
if err != nil {
t.Fatalf("parse %s: %v", file, err)
}
srv := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
w.WriteHeader(status)
_, _ = w.Write([]byte(body))
}))
defer srv.Close()
res, err := modules.ExecuteHTTPModule(context.Background(), srv.URL, def, modules.Options{
Timeout: 5 * time.Second,
Threads: 2,
})
if err != nil {
t.Fatalf("execute %s: %v", file, err)
}
return res
}
func serverAdminExtract(res *modules.Result, key string) string {
for _, f := range res.Findings {
if v := f.Extracted[key]; v != "" {
return v
}
}
return ""
}
func TestServerAdminExposureModules(t *testing.T) {
const caddy = "../../modules/recon/caddy-admin-exposure.yaml"
const envoy = "../../modules/recon/envoy-admin-exposure.yaml"
t.Run("a caddy config dump is flagged with a handler", func(t *testing.T) {
body := `{"apps":{"http":{"servers":{"srv0":{"listen":[":443"],"routes":[{"match":[{"host":` +
`["example.com"]}],"handle":[{"handler":"reverse_proxy","upstreams":[{"dial":"localhost:8080"}]}]}]}}},` +
`"tls":{"automation":{"policies":[{"issuers":[{"module":"acme"}]}]}}},"admin":{"listen":"0.0.0.0:2019"}}`
res := runServerAdminModule(t, caddy, 200, body)
if len(res.Findings) == 0 {
t.Fatal("expected a caddy finding")
}
if v := serverAdminExtract(res, "caddy_handler"); v != "reverse_proxy" {
t.Errorf("caddy_handler=%q, want reverse_proxy", v)
}
})
t.Run("an apps block without servers or handler is not flagged as caddy", func(t *testing.T) {
if res := runServerAdminModule(t, caddy, 200, `{"apps":{"tls":{"automation":{}}}}`); len(res.Findings) > 0 {
t.Errorf("an apps-only tls block should not match caddy, got %d findings", len(res.Findings))
}
})
t.Run("an envoy server_info is flagged with its version", func(t *testing.T) {
body := `{"version":"1.28.0/abcd/Clean/RELEASE/BoringSSL","state":"LIVE","uptime_current_epoch":"3600s",` +
`"uptime_all_epochs":"3600s","hot_restart_version":"11.104","command_line_options":{"base_id":"0",` +
`"concurrency":4,"config_path":"/etc/envoy/envoy.yaml"},"node":{"id":"node-1","cluster":"prod"}}`
res := runServerAdminModule(t, envoy, 200, body)
if len(res.Findings) == 0 {
t.Fatal("expected an envoy finding")
}
if v := serverAdminExtract(res, "envoy_version"); v != "1.28.0/abcd/Clean/RELEASE/BoringSSL" {
t.Errorf("envoy_version=%q, want the build string", v)
}
})
t.Run("a bare version+state body is not flagged as envoy", func(t *testing.T) {
if res := runServerAdminModule(t, envoy, 200, `{"version":"1.0","state":"LIVE"}`); len(res.Findings) > 0 {
t.Errorf("a bare version+state should not match envoy, got %d findings", len(res.Findings))
}
})
t.Run("a plain 200 body is not a leak", func(t *testing.T) {
for _, file := range []string{caddy, envoy} {
if res := runServerAdminModule(t, file, 200, "ok"); len(res.Findings) > 0 {
t.Errorf("%s: a plain 200 body should not match, got %d findings", file, len(res.Findings))
}
}
})
t.Run("a 404 is not a leak", func(t *testing.T) {
for _, file := range []string{caddy, envoy} {
if res := runServerAdminModule(t, file, 404, "not found"); len(res.Findings) > 0 {
t.Errorf("%s: a 404 should not match, got %d findings", file, len(res.Findings))
}
}
})
}
@@ -0,0 +1,150 @@
package modules_test
import (
"context"
"net/http"
"net/http/httptest"
"testing"
"time"
"github.com/vmfunc/sif/internal/modules"
)
const eurekaAppsXML = `<applications><versions__delta>1</versions__delta>` +
`<apps__hashcode>UP_5_</apps__hashcode><application><name>PAYMENT-SERVICE</name>` +
`<instance><instanceId>10.0.0.5:payment-service:8443</instanceId>` +
`<hostName>payment-1.svc.internal</hostName><app>PAYMENT-SERVICE</app>` +
`<ipAddr>10.0.0.5</ipAddr><status>UP</status><port enabled="true">8443</port>` +
`<vipAddress>payment-service</vipAddress></instance></application></applications>`
const eurekaAppsJSON = `{"applications":{"versions__delta":"1","apps__hashcode":"UP_5_",` +
`"application":[{"name":"PAYMENT-SERVICE","instance":[{"instanceId":"p1",` +
`"hostName":"payment-1.svc.internal","ipAddr":"10.0.0.5","status":"UP",` +
`"vipAddress":"payment-service"}]}]}}`
const sbaInstancesJSON = `[{"id":"a1b2c3","version":12,"registration":{"name":"order-service",` +
`"managementUrl":"http://order-1.internal:8080/actuator",` +
`"healthUrl":"http://order-1.internal:8080/actuator/health",` +
`"serviceUrl":"http://order-1.internal:8080/","source":"http-api","metadata":{}},` +
`"registered":true,"statusInfo":{"status":"UP","timestamp":"2026-06-25T20:00:00Z","details":{}},` +
`"buildVersion":"1.0.0","tags":{}}]`
func runServiceRegistryModule(t *testing.T, file string, status int, body string) *modules.Result {
t.Helper()
def, err := modules.ParseYAMLModule(file)
if err != nil {
t.Fatalf("parse %s: %v", file, err)
}
srv := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
w.WriteHeader(status)
_, _ = w.Write([]byte(body))
}))
defer srv.Close()
res, err := modules.ExecuteHTTPModule(context.Background(), srv.URL, def, modules.Options{
Timeout: 5 * time.Second,
Threads: 2,
})
if err != nil {
t.Fatalf("execute %s: %v", file, err)
}
return res
}
func svcRegExtract(res *modules.Result, key string) string {
for _, f := range res.Findings {
if v := f.Extracted[key]; v != "" {
return v
}
}
return ""
}
func TestServiceRegistryExposureModules(t *testing.T) {
const eureka = "../../modules/recon/eureka-registry-exposure.yaml"
const sba = "../../modules/recon/spring-boot-admin-exposure.yaml"
t.Run("an open eureka registry (xml) is flagged with the instance ip", func(t *testing.T) {
res := runServiceRegistryModule(t, eureka, 200, eurekaAppsXML)
if len(res.Findings) == 0 {
t.Fatal("expected a eureka finding")
}
if v := svcRegExtract(res, "eureka_instance_ip"); v != "10.0.0.5" {
t.Errorf("eureka_instance_ip=%q, want 10.0.0.5", v)
}
})
t.Run("an open eureka registry (json) is also flagged with the instance ip", func(t *testing.T) {
res := runServiceRegistryModule(t, eureka, 200, eurekaAppsJSON)
if len(res.Findings) == 0 {
t.Fatal("expected a eureka finding for the json form")
}
if v := svcRegExtract(res, "eureka_instance_ip"); v != "10.0.0.5" {
t.Errorf("eureka_instance_ip=%q, want 10.0.0.5", v)
}
})
t.Run("an envelope without apps__hashcode is not flagged", func(t *testing.T) {
body := `<applications><versions__delta>1</versions__delta></applications>`
if res := runServiceRegistryModule(t, eureka, 200, body); len(res.Findings) > 0 {
t.Errorf("an apps__hashcode-less envelope should not match, got %d findings", len(res.Findings))
}
})
t.Run("a page that merely mentions applications is not flagged", func(t *testing.T) {
body := `<html><body>Our applications support multiple versions of the eureka client.</body></html>`
if res := runServiceRegistryModule(t, eureka, 200, body); len(res.Findings) > 0 {
t.Errorf("a prose page should not match eureka, got %d findings", len(res.Findings))
}
})
t.Run("a secured eureka is not flagged", func(t *testing.T) {
if res := runServiceRegistryModule(t, eureka, 401, "Unauthorized"); len(res.Findings) > 0 {
t.Errorf("a 401 eureka should not match, got %d findings", len(res.Findings))
}
})
t.Run("a spring boot admin body does not match the eureka module", func(t *testing.T) {
if res := runServiceRegistryModule(t, eureka, 200, sbaInstancesJSON); len(res.Findings) > 0 {
t.Errorf("an sba body should not match eureka, got %d findings", len(res.Findings))
}
})
t.Run("an open spring boot admin is flagged with the health url", func(t *testing.T) {
res := runServiceRegistryModule(t, sba, 200, sbaInstancesJSON)
if len(res.Findings) == 0 {
t.Fatal("expected an sba finding")
}
if v := svcRegExtract(res, "sba_health_url"); v != "http://order-1.internal:8080/actuator/health" {
t.Errorf("sba_health_url=%q, want the internal actuator health url", v)
}
})
t.Run("a registration without statusInfo is not flagged", func(t *testing.T) {
body := `[{"registration":{"name":"x","healthUrl":"http://h:8080/health"}}]`
if res := runServiceRegistryModule(t, sba, 200, body); len(res.Findings) > 0 {
t.Errorf("a statusInfo-less body should not match sba, got %d findings", len(res.Findings))
}
})
t.Run("a secured spring boot admin is not flagged", func(t *testing.T) {
if res := runServiceRegistryModule(t, sba, 401, `{"error":"Unauthorized"}`); len(res.Findings) > 0 {
t.Errorf("a 401 sba should not match, got %d findings", len(res.Findings))
}
})
t.Run("a eureka body does not match the spring boot admin module", func(t *testing.T) {
if res := runServiceRegistryModule(t, sba, 200, eurekaAppsJSON); len(res.Findings) > 0 {
t.Errorf("a eureka body should not match sba, got %d findings", len(res.Findings))
}
})
t.Run("plain 200 bodies are not leaks", func(t *testing.T) {
if res := runServiceRegistryModule(t, eureka, 200, "ok"); len(res.Findings) > 0 {
t.Errorf("a plain 200 should not match eureka, got %d findings", len(res.Findings))
}
if res := runServiceRegistryModule(t, sba, 404, "not found"); len(res.Findings) > 0 {
t.Errorf("a 404 should not match sba, got %d findings", len(res.Findings))
}
})
}
@@ -0,0 +1,143 @@
package modules_test
import (
"context"
"net/http"
"net/http/httptest"
"testing"
"time"
"github.com/vmfunc/sif/internal/modules"
)
func runSpeechModule(t *testing.T, file string, status int, body string) *modules.Result {
t.Helper()
def, err := modules.ParseYAMLModule(file)
if err != nil {
t.Fatalf("parse %s: %v", file, err)
}
srv := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
w.WriteHeader(status)
_, _ = w.Write([]byte(body))
}))
defer srv.Close()
res, err := modules.ExecuteHTTPModule(context.Background(), srv.URL, def, modules.Options{
Timeout: 5 * time.Second,
Threads: 2,
})
if err != nil {
t.Fatalf("execute %s: %v", file, err)
}
return res
}
func speechExtract(res *modules.Result, key string) string {
for _, f := range res.Findings {
if v := f.Extracted[key]; v != "" {
return v
}
}
return ""
}
func TestSpeechAudioExposureModules(t *testing.T) {
const speaches = "../../modules/recon/speaches-api-exposure.yaml"
const xtts = "../../modules/recon/xtts-api-server-exposure.yaml"
speachesSTT := `{"data":[{"id":"Systran/faster-whisper-small","created":0,"object":"model",` +
`"owned_by":"Systran","language":["en"],"task":"automatic-speech-recognition"}],"object":"list"}`
speachesTTS := `{"data":[{"id":"speaches-ai/Kokoro-82M-v1.0-ONNX","created":0,"object":"model",` +
`"owned_by":"speaches-ai","language":["en"],"task":"text-to-speech"}],"object":"list"}`
xttsFolders := `{"speaker_folder":"/app/speakers","output_folder":"/app/output",` +
`"model_folder":"/app/xtts_models"}`
t.Run("a speaches stt model list is flagged with its model id", func(t *testing.T) {
res := runSpeechModule(t, speaches, 200, speachesSTT)
if len(res.Findings) == 0 {
t.Fatal("expected a speaches finding")
}
if v := speechExtract(res, "speaches_model"); v != "Systran/faster-whisper-small" {
t.Errorf("speaches_model=%q, want Systran/faster-whisper-small", v)
}
})
t.Run("a speaches tts-only model list is flagged", func(t *testing.T) {
if res := runSpeechModule(t, speaches, 200, speachesTTS); len(res.Findings) == 0 {
t.Error("expected a speaches finding for a text-to-speech model list")
}
})
t.Run("a generic openai-compatible model list is not flagged as speaches", func(t *testing.T) {
body := `{"data":[{"id":"meta-llama/Llama-3","created":1234,"object":"model","owned_by":"vllm"}],"object":"list"}`
if res := runSpeechModule(t, speaches, 200, body); len(res.Findings) > 0 {
t.Errorf("a taskless model list should not match speaches, got %d findings", len(res.Findings))
}
})
t.Run("a model list with a non-speech task is not flagged as speaches", func(t *testing.T) {
body := `{"data":[{"id":"bert-base","object":"model","owned_by":"hf","task":"fill-mask"}],"object":"list"}`
if res := runSpeechModule(t, speaches, 200, body); len(res.Findings) > 0 {
t.Errorf("a non-speech task should not match speaches, got %d findings", len(res.Findings))
}
})
t.Run("an empty speaches model list is not flagged", func(t *testing.T) {
// A fresh speaches server with no models downloaded returns an empty data
// array with nothing speaches-specific to anchor on; this known miss is
// preferable to firing on every empty OpenAI-shaped list.
body := `{"data":[],"object":"list"}`
if res := runSpeechModule(t, speaches, 200, body); len(res.Findings) > 0 {
t.Errorf("an empty model list should not match speaches, got %d findings", len(res.Findings))
}
})
t.Run("an xtts get-folders is flagged with its model folder", func(t *testing.T) {
res := runSpeechModule(t, xtts, 200, xttsFolders)
if len(res.Findings) == 0 {
t.Fatal("expected an xtts finding")
}
if v := speechExtract(res, "xtts_model_folder"); v != "/app/xtts_models" {
t.Errorf("xtts_model_folder=%q, want /app/xtts_models", v)
}
})
t.Run("a body without speaker_folder is not flagged as xtts", func(t *testing.T) {
body := `{"output_folder":"/app/output","model_folder":"/app/xtts_models"}`
if res := runSpeechModule(t, xtts, 200, body); len(res.Findings) > 0 {
t.Errorf("a body without speaker_folder should not match xtts, got %d findings", len(res.Findings))
}
})
t.Run("a body without model_folder is not flagged as xtts", func(t *testing.T) {
body := `{"speaker_folder":"/app/speakers","output_folder":"/app/output"}`
if res := runSpeechModule(t, xtts, 200, body); len(res.Findings) > 0 {
t.Errorf("a body without model_folder should not match xtts, got %d findings", len(res.Findings))
}
})
t.Run("a body without output_folder is not flagged as xtts", func(t *testing.T) {
body := `{"speaker_folder":"/app/speakers","model_folder":"/app/xtts_models"}`
if res := runSpeechModule(t, xtts, 200, body); len(res.Findings) > 0 {
t.Errorf("a body without output_folder should not match xtts, got %d findings", len(res.Findings))
}
})
t.Run("a plain 200 body is not a leak", func(t *testing.T) {
for _, file := range []string{speaches, xtts} {
if res := runSpeechModule(t, file, 200, "ok"); len(res.Findings) > 0 {
t.Errorf("%s: a plain 200 body should not match, got %d findings", file, len(res.Findings))
}
}
})
t.Run("a 404 is not a leak", func(t *testing.T) {
for _, file := range []string{speaches, xtts} {
if res := runSpeechModule(t, file, 404, "not found"); len(res.Findings) > 0 {
t.Errorf("%s: a 404 should not match, got %d findings", file, len(res.Findings))
}
}
})
}
@@ -8,7 +8,7 @@ import (
"testing"
"time"
"github.com/dropalldatabases/sif/internal/modules"
"github.com/vmfunc/sif/internal/modules"
)
func runVCSModule(t *testing.T, file string, status int, body string) *modules.Result {
+1 -1
View File
@@ -7,7 +7,7 @@ import (
"testing"
"time"
"github.com/dropalldatabases/sif/internal/modules"
"github.com/vmfunc/sif/internal/modules"
)
func runVectorDBModule(t *testing.T, file string, status int, body string) *modules.Result {
@@ -0,0 +1,150 @@
package modules_test
import (
"context"
"net/http"
"net/http/httptest"
"testing"
"time"
"github.com/vmfunc/sif/internal/modules"
)
func runVectorSearchModule(t *testing.T, file string, status int, body string) *modules.Result {
t.Helper()
def, err := modules.ParseYAMLModule(file)
if err != nil {
t.Fatalf("parse %s: %v", file, err)
}
srv := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
w.WriteHeader(status)
_, _ = w.Write([]byte(body))
}))
defer srv.Close()
res, err := modules.ExecuteHTTPModule(context.Background(), srv.URL, def, modules.Options{
Timeout: 5 * time.Second,
Threads: 2,
})
if err != nil {
t.Fatalf("execute %s: %v", file, err)
}
return res
}
func vectorSearchExtract(res *modules.Result, key string) string {
for _, f := range res.Findings {
if v := f.Extracted[key]; v != "" {
return v
}
}
return ""
}
func TestVectorSearchExposureModules(t *testing.T) {
const marqo = "../../modules/recon/marqo-exposure.yaml"
const vespa = "../../modules/recon/vespa-status-exposure.yaml"
const meilisearch = "../../modules/recon/meilisearch-exposure.yaml"
marqoRoot := `{"message":"Welcome to Marqo","version":"2.11.0"}`
vespaStatus := `{"application":{"vespa":{"version":"8.43.64"},"meta":{"name":"default","generation":11}},` +
`"abstractComponents":[],"handlers":[],"clients":[],"servers":[],"httpRequestFilters":[],` +
`"httpResponseFilters":[],"processingChains":[]}`
meiliVersion := `{"commitSha":"b46889b5","commitDate":"2026-01-15T00:00:00Z","pkgVersion":"1.12.0"}`
t.Run("a marqo root is flagged with its version", func(t *testing.T) {
res := runVectorSearchModule(t, marqo, 200, marqoRoot)
if len(res.Findings) == 0 {
t.Fatal("expected a marqo finding")
}
if v := vectorSearchExtract(res, "marqo_version"); v != "2.11.0" {
t.Errorf("marqo_version=%q, want 2.11.0", v)
}
})
t.Run("a generic root with a version is not flagged as marqo", func(t *testing.T) {
body := `{"message":"Welcome","version":"2.11.0","service":"something-else"}`
if res := runVectorSearchModule(t, marqo, 200, body); len(res.Findings) > 0 {
t.Errorf("a generic welcome should not match marqo, got %d findings", len(res.Findings))
}
})
t.Run("a marqo welcome without a version is not flagged", func(t *testing.T) {
if res := runVectorSearchModule(t, marqo, 200, `{"message":"Welcome to Marqo"}`); len(res.Findings) > 0 {
t.Errorf("a versionless welcome should not match marqo, got %d findings", len(res.Findings))
}
})
t.Run("an html page mentioning marqo is not flagged", func(t *testing.T) {
body := `<html><body><h1>Welcome to Marqo</h1><p>version 2.11.0 docs</p></body></html>`
if res := runVectorSearchModule(t, marqo, 200, body); len(res.Findings) > 0 {
t.Errorf("prose mentioning marqo should not match the structured response, got %d findings", len(res.Findings))
}
})
t.Run("a vespa status is flagged with its version", func(t *testing.T) {
res := runVectorSearchModule(t, vespa, 200, vespaStatus)
if len(res.Findings) == 0 {
t.Fatal("expected a vespa finding")
}
if v := vectorSearchExtract(res, "vespa_version"); v != "8.43.64" {
t.Errorf("vespa_version=%q, want 8.43.64", v)
}
})
t.Run("a body without abstractComponents is not flagged as vespa", func(t *testing.T) {
body := `{"handlers":[],"processingChains":[],"httpRequestFilters":[]}`
if res := runVectorSearchModule(t, vespa, 200, body); len(res.Findings) > 0 {
t.Errorf("a body without abstractComponents should not match vespa, got %d findings", len(res.Findings))
}
})
t.Run("a body without processingChains is not flagged as vespa", func(t *testing.T) {
body := `{"abstractComponents":[],"httpRequestFilters":[]}`
if res := runVectorSearchModule(t, vespa, 200, body); len(res.Findings) > 0 {
t.Errorf("a body without processingChains should not match vespa, got %d findings", len(res.Findings))
}
})
t.Run("a meilisearch version is flagged", func(t *testing.T) {
res := runVectorSearchModule(t, meilisearch, 200, meiliVersion)
if len(res.Findings) == 0 {
t.Fatal("expected a meilisearch finding")
}
if v := vectorSearchExtract(res, "meilisearch_version"); v != "1.12.0" {
t.Errorf("meilisearch_version=%q, want 1.12.0", v)
}
})
t.Run("a body without commitSha is not flagged as meilisearch", func(t *testing.T) {
body := `{"commitDate":"2026","pkgVersion":"1.12.0"}`
if res := runVectorSearchModule(t, meilisearch, 200, body); len(res.Findings) > 0 {
t.Errorf("a body without commitSha should not match meilisearch, got %d findings", len(res.Findings))
}
})
t.Run("a body without pkgVersion is not flagged as meilisearch", func(t *testing.T) {
body := `{"commitSha":"abc","commitDate":"2026"}`
if res := runVectorSearchModule(t, meilisearch, 200, body); len(res.Findings) > 0 {
t.Errorf("a body without pkgVersion should not match meilisearch, got %d findings", len(res.Findings))
}
})
t.Run("a plain 200 body is not a leak", func(t *testing.T) {
for _, file := range []string{marqo, vespa, meilisearch} {
if res := runVectorSearchModule(t, file, 200, "ok"); len(res.Findings) > 0 {
t.Errorf("%s: a plain 200 body should not match, got %d findings", file, len(res.Findings))
}
}
})
t.Run("a 404 is not a leak", func(t *testing.T) {
for _, file := range []string{marqo, vespa, meilisearch} {
if res := runVectorSearchModule(t, file, 404, "not found"); len(res.Findings) > 0 {
t.Errorf("%s: a 404 should not match, got %d findings", file, len(res.Findings))
}
}
})
}
@@ -7,7 +7,7 @@ import (
"testing"
"time"
"github.com/dropalldatabases/sif/internal/modules"
"github.com/vmfunc/sif/internal/modules"
)
func runWebSrvModule(t *testing.T, file string, status int, body string) *modules.Result {
+1 -1
View File
@@ -16,7 +16,7 @@ import (
"context"
"net/http"
"github.com/dropalldatabases/sif/internal/finding"
"github.com/vmfunc/sif/internal/finding"
)
// discordProvider posts to a discord webhook. discord's incoming-webhook body
+2 -2
View File
@@ -20,8 +20,8 @@ import (
"net/http"
"strings"
"github.com/dropalldatabases/sif/internal/finding"
"github.com/dropalldatabases/sif/internal/httpx"
"github.com/vmfunc/sif/internal/finding"
"github.com/vmfunc/sif/internal/httpx"
)
// contentTypeJSON is the body type every provider POSTs; all four speak json.
+3 -3
View File
@@ -22,9 +22,9 @@ import (
"net/http"
"time"
"github.com/dropalldatabases/sif/internal/finding"
"github.com/dropalldatabases/sif/internal/httpx"
"github.com/dropalldatabases/sif/internal/output"
"github.com/vmfunc/sif/internal/finding"
"github.com/vmfunc/sif/internal/httpx"
"github.com/vmfunc/sif/internal/output"
)
// Options carries the runtime knobs Send needs. Timeout bounds each provider's
+1 -1
View File
@@ -22,7 +22,7 @@ import (
"testing"
"time"
"github.com/dropalldatabases/sif/internal/finding"
"github.com/vmfunc/sif/internal/finding"
)
// sampleFindings returns a small mixed-severity batch for payload assertions.
+1 -1
View File
@@ -16,7 +16,7 @@ import (
"context"
"net/http"
"github.com/dropalldatabases/sif/internal/finding"
"github.com/vmfunc/sif/internal/finding"
)
// slackProvider posts to a slack incoming webhook. the webhook url already pins
+1 -1
View File
@@ -16,7 +16,7 @@ import (
"context"
"net/http"
"github.com/dropalldatabases/sif/internal/finding"
"github.com/vmfunc/sif/internal/finding"
)
// telegramAPIBase is the bot api root. it's a var so tests can repoint it at an
+1 -1
View File
@@ -16,7 +16,7 @@ import (
"context"
"net/http"
"github.com/dropalldatabases/sif/internal/finding"
"github.com/vmfunc/sif/internal/finding"
)
// webhookProvider posts a structured json payload to an arbitrary endpoint. unlike
+1 -1
View File
@@ -13,8 +13,8 @@
package format
import (
"github.com/dropalldatabases/sif/internal/styles"
nucleiout "github.com/projectdiscovery/nuclei/v3/pkg/output"
"github.com/vmfunc/sif/internal/styles"
)
func FormatLine(event *nucleiout.ResultEvent) string {
+2 -2
View File
@@ -17,8 +17,8 @@ import (
"fmt"
"strings"
"github.com/dropalldatabases/sif/internal/modules"
"github.com/dropalldatabases/sif/internal/scan/frameworks"
"github.com/vmfunc/sif/internal/modules"
"github.com/vmfunc/sif/internal/scan/frameworks"
)
type FrameworksModule struct{}
+2 -2
View File
@@ -16,8 +16,8 @@ import (
"context"
"fmt"
"github.com/dropalldatabases/sif/internal/modules"
"github.com/dropalldatabases/sif/internal/scan"
"github.com/vmfunc/sif/internal/modules"
"github.com/vmfunc/sif/internal/scan"
)
type NucleiModule struct{}
+1 -1
View File
@@ -12,7 +12,7 @@
package builtin
import "github.com/dropalldatabases/sif/internal/modules"
import "github.com/vmfunc/sif/internal/modules"
// Register registers all Go-based built-in scans as modules.
// Allows complex Go scans to participate in the module system
@@ -17,8 +17,8 @@ import (
"fmt"
"strings"
"github.com/dropalldatabases/sif/internal/modules"
"github.com/dropalldatabases/sif/internal/scan"
"github.com/vmfunc/sif/internal/modules"
"github.com/vmfunc/sif/internal/scan"
)
type SecurityTrailsModule struct{}
+2 -2
View File
@@ -17,8 +17,8 @@ import (
"fmt"
"strings"
"github.com/dropalldatabases/sif/internal/modules"
"github.com/dropalldatabases/sif/internal/scan"
"github.com/vmfunc/sif/internal/modules"
"github.com/vmfunc/sif/internal/scan"
)
type ShodanModule struct{}
+2 -2
View File
@@ -15,8 +15,8 @@ package builtin
import (
"context"
"github.com/dropalldatabases/sif/internal/modules"
"github.com/dropalldatabases/sif/internal/scan"
"github.com/vmfunc/sif/internal/modules"
"github.com/vmfunc/sif/internal/scan"
)
type WhoisModule struct{}

Some files were not shown because too many files have changed in this diff Show More