diff --git a/README.md b/README.md index a493d38..74293fb 100644 --- a/README.md +++ b/README.md @@ -220,6 +220,8 @@ write the run's findings out to a file for ci/cd or triage: | `-sarif` | write a sarif 2.1.0 report to this file | | `-markdown`, `-md` | write a markdown report to this file | | `-silent` | plain output: chrome to stderr, one finding per line to stdout (for pipelines) | +| `-diff` | surface only findings added/removed since the last snapshot of each target | +| `-store` | snapshot directory for `-diff` (default: log dir, else `/sif/state`) | ```bash # scan and emit both a sarif and markdown report @@ -228,6 +230,46 @@ write the run's findings out to a file for ci/cd or triage: sarif output is ingestable by github code scanning; markdown is a readable per-target summary. +### diff mode + +`-diff` turns a re-scan into a monitor: sif snapshots each target's normalized findings to a json file, and on the next run reports only the delta (`+ new` / `- gone`) against that snapshot, then overwrites it. the first run for a target has no baseline, so everything is `+ new`. snapshots land in `-store` (one sanitized file per target); when unset they reuse the log dir, falling back to `/sif/state`. + +```bash +# baseline run, then re-scan later and see only what moved +./sif -u https://example.com -sh -cors -diff +./sif -u https://example.com -sh -cors -diff +``` + +the snapshot is always rewritten, so each run diffs against the previous one. the delta is chrome (it rides the normal output sink / stderr under `-silent`), not the findings stream. + +### notify + +ship findings to a chat/webhook sink so a continuous-recon run alerts on what it turns up. every provider is a single POST through the shared http client, so the global proxy/rate-limit/header config applies. + +| flag | description | +|------|-------------| +| `-notify` | ship findings to every configured provider after the scan | +| `-notify-severity` | minimum severity to send (`info`/`low`/`medium`/`high`/`critical`, default `medium`) | +| `-notify-config` | path to a notify-compatible yaml config (overrides env vars) | + +providers are configured env-first; a yaml file (`-notify-config`) overrides per-field. the yaml keys match [projectdiscovery/notify](https://github.com/projectdiscovery/notify) so an existing config ports over: + +| env var | yaml key | provider | +|---------|----------|----------| +| `SLACK_WEBHOOK_URL` | `slack_webhook_url` | slack incoming webhook | +| `DISCORD_WEBHOOK_URL` | `discord_webhook_url` | discord webhook | +| `TELEGRAM_BOT_TOKEN` | `telegram_api_key` | telegram bot api (needs chat id too) | +| `TELEGRAM_CHAT_ID` | `telegram_chat_id` | telegram destination chat | +| `NOTIFY_WEBHOOK_URL` | `webhook_url` | generic json webhook (structured findings) | + +```bash +# alert slack on medium+ findings discovered during a scan +export SLACK_WEBHOOK_URL=https://hooks.slack.com/services/... +./sif -u https://example.com -cors -xss -notify -notify-severity medium +``` + +a provider with no destination is skipped; with nothing configured, `-notify` is a silent no-op. slack/discord/telegram receive a fixed-width finding block; the generic webhook receives structured json (`{count, findings[]}`). + ### pipe mode sif reads targets from stdin and accepts naked hosts, so it drops into a unix pipeline. `-silent` routes all banner/spinner/log chrome to stderr and prints one normalized finding per line (`[severity] target module title`) to stdout: diff --git a/docs/usage.md b/docs/usage.md index 8325e75..6a7499d 100644 --- a/docs/usage.md +++ b/docs/usage.md @@ -442,6 +442,78 @@ plain output for pipelines: all banner/spinner/log chrome goes to stderr and std subfinder -d example.com | sif -silent -probe -sh | notify ``` +### -diff + +turn a re-scan into a monitor. sif snapshots each target's normalized findings to a json file under the store dir; on the next run it loads that snapshot, diffs the current findings against it by finding key, and prints only the delta (`+ new` for findings that appeared, `- gone` for findings that vanished). it always rewrites the snapshot afterwards, so each run compares against the previous one. + +the first run for a target has no snapshot, so every finding shows as `+ new`. when nothing changed, sif notes that and writes a fresh snapshot anyway. + +```bash +# baseline, then re-scan and see only what moved +./sif -u https://example.com -sh -cors -diff +./sif -u https://example.com -sh -cors -diff +``` + +the delta is chrome, not the findings stream: under `-silent` it rides stderr with the rest of the chrome, leaving stdout for the full findings. + +### -store + +snapshot directory for `-diff`. precedence when unset: the `-log` dir if one is given, else `/sif/state` (`$XDG_CONFIG_HOME/sif/state` on linux, `~/Library/Application Support/sif/state` on macos). one sanitized file per target, created at `0750`, written `0600`. + +```bash +./sif -u https://example.com -sh -diff -store ./snapshots +``` + + +## notify options + +ship findings to a chat/webhook sink after the scan. every provider is a single POST through the shared http client, so the global proxy/rate-limit/header config applies. with nothing configured, `-notify` is a silent no-op. + +### -notify + +enable delivery to every configured provider: + +```bash +export SLACK_WEBHOOK_URL=https://hooks.slack.com/services/... +./sif -u https://example.com -cors -xss -notify +``` + +### -notify-severity + +minimum severity to send: `info`, `low`, `medium`, `high` or `critical` (default `medium`). findings below the floor are dropped, so info-level recon noise doesn't flood a channel. an unrecognized value falls back to `medium`: + +```bash +./sif -u https://example.com -cors -notify -notify-severity high +``` + +### -notify-config + +path to a yaml config that overrides the env vars per-field. the keys match [projectdiscovery/notify](https://github.com/projectdiscovery/notify) so an existing config ports over: + +```yaml +slack_webhook_url: https://hooks.slack.com/services/... +discord_webhook_url: https://discord.com/api/webhooks/... +telegram_api_key: 123456:abcdef +telegram_chat_id: "987654" +webhook_url: https://example.internal/sif-findings +``` + +```bash +./sif -u https://example.com -cors -notify -notify-config notify.yaml +``` + +providers are resolved env-first, then overlaid by the yaml file: + +| env var | yaml key | provider | +|---------|----------|----------| +| `SLACK_WEBHOOK_URL` | `slack_webhook_url` | slack incoming webhook | +| `DISCORD_WEBHOOK_URL` | `discord_webhook_url` | discord webhook | +| `TELEGRAM_BOT_TOKEN` | `telegram_api_key` | telegram bot api (needs chat id too) | +| `TELEGRAM_CHAT_ID` | `telegram_chat_id` | telegram destination chat | +| `NOTIFY_WEBHOOK_URL` | `webhook_url` | generic json webhook (structured findings) | + +slack/discord/telegram receive a fixed-width finding block; the generic webhook receives structured json (`{count, findings[]}`) for downstream automation. + ## api options ### -api diff --git a/internal/config/config.go b/internal/config/config.go index e0f7583..d6ebc77 100644 --- a/internal/config/config.go +++ b/internal/config/config.go @@ -69,6 +69,8 @@ type Settings struct { SARIF string // path to write a sarif 2.1.0 report to ("" = off) Markdown string // path to write a markdown report to ("" = off) Silent bool // route chrome to stderr, print one finding per line to stdout + Diff bool // surface only findings added/removed vs the last snapshot + Store string // snapshot dir for diff mode ("" = default state dir) Modules string // Comma-separated list of module IDs to run ModuleTags string // Run modules matching these tags AllModules bool // Run all loaded modules @@ -77,6 +79,9 @@ type Settings struct { Header goflags.StringSlice // custom request headers ("Key: Value") Cookie string RateLimit int + Notify bool // -notify: ship findings to configured providers + NotifySeverity string // -notify-severity: minimum severity to send (info..critical) + NotifyConfig string // -notify-config: path to a notify-compatible yaml file } // minThreads is the floor for the worker count. Threads feeds wg.Add across the @@ -88,6 +93,10 @@ const minThreads = 1 // to find linked pages without crawling an entire site. const defaultCrawlDepth = 2 +// defaultNotifySeverity is the floor notify sends at when -notify-severity is +// unset: medium drops pure recon/info noise so alerts stay actionable. +const defaultNotifySeverity = "medium" + const ( Nil goflags.EnumVariable = iota @@ -174,6 +183,14 @@ func Parse() *Settings { flagSet.StringVar(&settings.SARIF, "sarif", "", "Write a SARIF 2.1.0 report to this file"), flagSet.StringVarP(&settings.Markdown, "markdown", "md", "", "Write a markdown report to this file"), flagSet.BoolVar(&settings.Silent, "silent", false, "Plain output: chrome to stderr, one finding per line to stdout (for pipelines)"), + flagSet.BoolVar(&settings.Diff, "diff", false, "Diff mode: surface only findings added/removed since the last snapshot of each target"), + flagSet.StringVar(&settings.Store, "store", "", "Snapshot directory for -diff (default: log dir, else /sif/state)"), + ) + + flagSet.CreateGroup("notify", "Notify", + flagSet.BoolVar(&settings.Notify, "notify", false, "Ship findings to configured providers (slack/discord/telegram/webhook)"), + flagSet.StringVar(&settings.NotifySeverity, "notify-severity", defaultNotifySeverity, "Minimum severity to notify on (info/low/medium/high/critical)"), + flagSet.StringVar(&settings.NotifyConfig, "notify-config", "", "Path to a notify-compatible yaml config (overrides env vars)"), ) flagSet.CreateGroup("api", "API", diff --git a/internal/config/config_test.go b/internal/config/config_test.go index 918cd91..96aaeba 100644 --- a/internal/config/config_test.go +++ b/internal/config/config_test.go @@ -61,6 +61,14 @@ func TestSettingsDefaults(t *testing.T) { if settings.Ports != "" { t.Errorf("expected Ports default to be empty, got %v", settings.Ports) } + + // diff mode is opt-in and its store dir defaults empty (resolved at runtime). + if settings.Diff != false { + t.Errorf("expected Diff default to be false, got %v", settings.Diff) + } + if settings.Store != "" { + t.Errorf("expected Store default to be empty, got %v", settings.Store) + } } func TestSettingsNoScanBehavior(t *testing.T) { diff --git a/internal/notify/config.go b/internal/notify/config.go new file mode 100644 index 0000000..dbf931f --- /dev/null +++ b/internal/notify/config.go @@ -0,0 +1,119 @@ +/* +·━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━· +: : +: █▀ █ █▀▀ · Blazing-fast pentesting suite : +: ▄█ █ █▀ · BSD 3-Clause License : +: : +: (c) 2022-2026 vmfunc, xyzeva, : +: lunchcat alumni & contributors : +: : +·━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━· +*/ + +package notify + +import ( + "fmt" + "os" + + "gopkg.in/yaml.v3" +) + +// env var names notify reads, env-first. these mirror the conventional names so +// an operator who already exports them for other tooling gets notify for free. +const ( + envSlackWebhook = "SLACK_WEBHOOK_URL" + envDiscordWebhook = "DISCORD_WEBHOOK_URL" + // the name of the env var holding the bot token, not the token itself. + envTelegramToken = "TELEGRAM_BOT_TOKEN" //nolint:gosec // env var name, not a secret + envTelegramChat = "TELEGRAM_CHAT_ID" + envWebhookURL = "NOTIFY_WEBHOOK_URL" +) + +// config holds resolved destinations for every provider. yaml tags use +// projectdiscovery/notify-compatible key names so an existing notify config file +// ports over verbatim; env supplies the same values and yaml overrides it. +type config struct { + SlackWebhook string `yaml:"slack_webhook_url"` + DiscordWebhook string `yaml:"discord_webhook_url"` + // telegram needs both a bot token and a chat id. notify spells the token + // "telegram_api_key", so accept that key for drop-in compatibility. + TelegramToken string `yaml:"telegram_api_key"` + TelegramChat string `yaml:"telegram_chat_id"` + WebhookURL string `yaml:"webhook_url"` +} + +// loadConfig resolves notify destinations env-first, then overlays a yaml file +// when path is non-empty. yaml wins per-field so a file value overrides the +// matching env var; an unset yaml field leaves the env value intact. an empty +// path means env-only. a missing/unparseable file is an error - if the operator +// pointed -notify-config somewhere, a typo should fail loud, not silently drop. +func loadConfig(path string) (config, error) { + cfg := config{ + SlackWebhook: os.Getenv(envSlackWebhook), + DiscordWebhook: os.Getenv(envDiscordWebhook), + TelegramToken: os.Getenv(envTelegramToken), + TelegramChat: os.Getenv(envTelegramChat), + WebhookURL: os.Getenv(envWebhookURL), + } + + if path == "" { + return cfg, nil + } + + data, err := os.ReadFile(path) + if err != nil { + return config{}, fmt.Errorf("read config %q: %w", path, err) + } + + // decode into a separate value so only the keys present in the file overlay + // the env-derived defaults; a zero field in the yaml must not blank an env var. + var file config + if err := yaml.Unmarshal(data, &file); err != nil { + return config{}, fmt.Errorf("parse config %q: %w", path, err) + } + overlay(&cfg, &file) + + return cfg, nil +} + +// overlay copies non-empty fields from src onto dst. used to let a yaml file +// override env without an empty yaml key wiping out a populated env value. +func overlay(dst, src *config) { + if src.SlackWebhook != "" { + dst.SlackWebhook = src.SlackWebhook + } + if src.DiscordWebhook != "" { + dst.DiscordWebhook = src.DiscordWebhook + } + if src.TelegramToken != "" { + dst.TelegramToken = src.TelegramToken + } + if src.TelegramChat != "" { + dst.TelegramChat = src.TelegramChat + } + if src.WebhookURL != "" { + dst.WebhookURL = src.WebhookURL + } +} + +// providers builds the live provider list from the resolved config: a provider +// is included only when its destination is fully specified. telegram needs both +// token and chat id, so a half-configured telegram is dropped rather than POSTing +// to a broken endpoint. +func (c *config) providers() []provider { + var out []provider + if c.SlackWebhook != "" { + out = append(out, &slackProvider{webhook: c.SlackWebhook}) + } + if c.DiscordWebhook != "" { + out = append(out, &discordProvider{webhook: c.DiscordWebhook}) + } + if c.TelegramToken != "" && c.TelegramChat != "" { + out = append(out, &telegramProvider{token: c.TelegramToken, chatID: c.TelegramChat}) + } + if c.WebhookURL != "" { + out = append(out, &webhookProvider{url: c.WebhookURL}) + } + return out +} diff --git a/internal/notify/config_test.go b/internal/notify/config_test.go new file mode 100644 index 0000000..0f3ae3c --- /dev/null +++ b/internal/notify/config_test.go @@ -0,0 +1,153 @@ +/* +·━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━· +: : +: █▀ █ █▀▀ · Blazing-fast pentesting suite : +: ▄█ █ █▀ · BSD 3-Clause License : +: : +: (c) 2022-2026 vmfunc, xyzeva, : +: lunchcat alumni & contributors : +: : +·━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━· +*/ + +package notify + +import ( + "os" + "path/filepath" + "testing" +) + +// clearNotifyEnv unsets every var loadConfig reads so a test starts from a known +// blank slate; t.Setenv("", "") still records the var for cleanup restoration. +func clearNotifyEnv(t *testing.T) { + t.Helper() + for _, k := range []string{ + envSlackWebhook, envDiscordWebhook, + envTelegramToken, envTelegramChat, envWebhookURL, + } { + t.Setenv(k, "") + } +} + +func TestLoadConfigEnvOnly(t *testing.T) { + clearNotifyEnv(t) + t.Setenv(envSlackWebhook, "https://hooks.slack.test/a") + t.Setenv(envTelegramToken, "123:abc") + t.Setenv(envTelegramChat, "999") + + cfg, err := loadConfig("") + if err != nil { + t.Fatalf("loadConfig: %v", err) + } + if cfg.SlackWebhook != "https://hooks.slack.test/a" { + t.Errorf("slack webhook = %q, want from env", cfg.SlackWebhook) + } + if cfg.TelegramToken != "123:abc" || cfg.TelegramChat != "999" { + t.Errorf("telegram = %q/%q, want from env", cfg.TelegramToken, cfg.TelegramChat) + } + + // slack + telegram (both halves) configured, discord/webhook empty. + got := cfg.providers() + if len(got) != 2 { + t.Fatalf("providers = %d, want 2 (slack, telegram)", len(got)) + } + wantNames := map[string]bool{"slack": false, "telegram": false} + for _, p := range got { + wantNames[p.name()] = true + } + for name, seen := range wantNames { + if !seen { + t.Errorf("provider %q missing", name) + } + } +} + +func TestLoadConfigYAMLOverridesEnv(t *testing.T) { + clearNotifyEnv(t) + t.Setenv(envSlackWebhook, "https://env.slack.test/x") + t.Setenv(envWebhookURL, "https://env.webhook.test/x") + + body := "" + + "slack_webhook_url: https://file.slack.test/y\n" + + "discord_webhook_url: https://file.discord.test/z\n" + path := writeTempConfig(t, body) + + cfg, err := loadConfig(path) + if err != nil { + t.Fatalf("loadConfig: %v", err) + } + // yaml present -> overrides env. + if cfg.SlackWebhook != "https://file.slack.test/y" { + t.Errorf("slack = %q, want yaml override", cfg.SlackWebhook) + } + // yaml absent for webhook -> env value survives. + if cfg.WebhookURL != "https://env.webhook.test/x" { + t.Errorf("webhook = %q, want env value preserved", cfg.WebhookURL) + } + // yaml introduces discord. + if cfg.DiscordWebhook != "https://file.discord.test/z" { + t.Errorf("discord = %q, want from yaml", cfg.DiscordWebhook) + } +} + +func TestLoadConfigNotifyCompatibleTelegramKey(t *testing.T) { + clearNotifyEnv(t) + // projectdiscovery/notify spells the bot token "telegram_api_key"; assert a + // drop-in config wires telegram from that key. + body := "" + + "telegram_api_key: 555:tok\n" + + "telegram_chat_id: \"42\"\n" + path := writeTempConfig(t, body) + + cfg, err := loadConfig(path) + if err != nil { + t.Fatalf("loadConfig: %v", err) + } + if cfg.TelegramToken != "555:tok" || cfg.TelegramChat != "42" { + t.Fatalf("telegram = %q/%q, want from notify-compatible keys", cfg.TelegramToken, cfg.TelegramChat) + } + if len(cfg.providers()) != 1 { + t.Fatalf("providers = %d, want 1 (telegram)", len(cfg.providers())) + } +} + +func TestLoadConfigMissingFileErrors(t *testing.T) { + clearNotifyEnv(t) + if _, err := loadConfig(filepath.Join(t.TempDir(), "nope.yaml")); err == nil { + t.Fatal("loadConfig with missing file: want error, got nil") + } +} + +func TestLoadConfigBadYAMLErrors(t *testing.T) { + clearNotifyEnv(t) + path := writeTempConfig(t, "slack_webhook_url: [unterminated\n") + if _, err := loadConfig(path); err == nil { + t.Fatal("loadConfig with malformed yaml: want error, got nil") + } +} + +func TestProvidersTelegramNeedsBothHalves(t *testing.T) { + // token without chat id must not produce a (broken) telegram provider. + cfg := config{TelegramToken: "tok"} + if got := cfg.providers(); len(got) != 0 { + t.Fatalf("providers = %d, want 0 for half-configured telegram", len(got)) + } +} + +func TestProvidersEmptyConfigIsNone(t *testing.T) { + var cfg config + if got := cfg.providers(); len(got) != 0 { + t.Fatalf("providers = %d, want 0 for empty config", len(got)) + } +} + +// writeTempConfig writes body to a temp yaml file and returns its path. +func writeTempConfig(t *testing.T, body string) string { + t.Helper() + path := filepath.Join(t.TempDir(), "notify.yaml") + if err := os.WriteFile(path, []byte(body), 0o600); err != nil { + t.Fatalf("write temp config: %v", err) + } + return path +} diff --git a/internal/notify/discord.go b/internal/notify/discord.go new file mode 100644 index 0000000..2908a49 --- /dev/null +++ b/internal/notify/discord.go @@ -0,0 +1,39 @@ +/* +·━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━· +: : +: █▀ █ █▀▀ · Blazing-fast pentesting suite : +: ▄█ █ █▀ · BSD 3-Clause License : +: : +: (c) 2022-2026 vmfunc, xyzeva, : +: lunchcat alumni & contributors : +: : +·━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━· +*/ + +package notify + +import ( + "context" + "net/http" + + "github.com/dropalldatabases/sif/internal/finding" +) + +// discordProvider posts to a discord webhook. discord's incoming-webhook body +// keys the message on "content" (slack uses "text"); same code-block wrapping so +// the finding columns line up in the channel. +type discordProvider struct { + webhook string +} + +func (d *discordProvider) name() string { return "discord" } + +// discordPayload is the minimal webhook body: a single content field. +type discordPayload struct { + Content string `json:"content"` +} + +func (d *discordProvider) send(ctx context.Context, client *http.Client, findings []finding.Finding) error { + payload := discordPayload{Content: codeBlock(renderFindings(findings))} + return postJSON(ctx, client, d.webhook, payload) +} diff --git a/internal/notify/message.go b/internal/notify/message.go new file mode 100644 index 0000000..34445b0 --- /dev/null +++ b/internal/notify/message.go @@ -0,0 +1,74 @@ +/* +·━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━· +: : +: █▀ █ █▀▀ · Blazing-fast pentesting suite : +: ▄█ █ █▀ · BSD 3-Clause License : +: : +: (c) 2022-2026 vmfunc, xyzeva, : +: lunchcat alumni & contributors : +: : +·━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━· +*/ + +package notify + +import ( + "bytes" + "context" + "encoding/json" + "fmt" + "net/http" + "strings" + + "github.com/dropalldatabases/sif/internal/finding" + "github.com/dropalldatabases/sif/internal/httpx" +) + +// contentTypeJSON is the body type every provider POSTs; all four speak json. +const contentTypeJSON = "application/json" + +// messageHeader prefixes the rendered finding block. kept terse - chat sinks +// truncate, so the count and lead-in carry the signal. +const messageHeader = "sif found %d finding(s):" + +// renderFindings turns a batch into a single plain-text block, one finding per +// line in the same "[severity] target module title" shape as the -silent sink so +// a reader sees identical lines across stdout and chat. a strings.Builder keeps +// the per-line concat to one allocation path. +func renderFindings(findings []finding.Finding) string { + var b strings.Builder + fmt.Fprintf(&b, messageHeader, len(findings)) + b.WriteByte('\n') + for i := 0; i < len(findings); i++ { + b.WriteString(findings[i].Line()) + b.WriteByte('\n') + } + return b.String() +} + +// postJSON marshals payload and POSTs it to url through the shared client. it +// drains+closes the response so the conn returns to httpx's pool, and treats any +// non-2xx as a delivery failure so a 4xx from a bad webhook surfaces loudly. +func postJSON(ctx context.Context, client *http.Client, url string, payload any) error { + body, err := json.Marshal(payload) + if err != nil { + return fmt.Errorf("marshal payload: %w", err) + } + + req, err := http.NewRequestWithContext(ctx, http.MethodPost, url, bytes.NewReader(body)) + if err != nil { + return fmt.Errorf("build request: %w", err) + } + req.Header.Set("Content-Type", contentTypeJSON) + + resp, err := client.Do(req) //nolint:bodyclose // drained and closed via httpx.DrainClose + if err != nil { + return fmt.Errorf("post: %w", err) + } + defer httpx.DrainClose(resp) + + if resp.StatusCode < http.StatusOK || resp.StatusCode >= http.StatusMultipleChoices { + return fmt.Errorf("unexpected status %d", resp.StatusCode) + } + return nil +} diff --git a/internal/notify/notify.go b/internal/notify/notify.go new file mode 100644 index 0000000..bfbae66 --- /dev/null +++ b/internal/notify/notify.go @@ -0,0 +1,85 @@ +/* +·━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━· +: : +: █▀ █ █▀▀ · Blazing-fast pentesting suite : +: ▄█ █ █▀ · BSD 3-Clause License : +: : +: (c) 2022-2026 vmfunc, xyzeva, : +: lunchcat alumni & contributors : +: : +·━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━· +*/ + +// Package notify ships findings to chat/webhook sinks (slack, discord, telegram, +// generic webhook) so a continuous-recon run can alert on what it turns up. every +// provider is one POST through httpx.Client, so the global proxy/rate-limit/header +// config applies uniformly and there's no extra http stack to keep in sync. +package notify + +import ( + "context" + "fmt" + "net/http" + "time" + + "github.com/dropalldatabases/sif/internal/finding" + "github.com/dropalldatabases/sif/internal/httpx" + "github.com/dropalldatabases/sif/internal/output" +) + +// Options carries the runtime knobs Send needs. Timeout bounds each provider's +// POST; ConfigPath is an optional yaml file whose values override env. severity +// filtering is the caller's job - Send ships whatever batch it's handed. +type Options struct { + Timeout time.Duration + ConfigPath string +} + +// Send dispatches findings to every configured provider. config resolves +// env-first, then a yaml file overlays it (notify-compatible key names). a +// provider with no destination is skipped, so zero configured providers makes +// Send a silent no-op - notify is opt-in and never errors just for being unwired. +// an empty findings slice is also a no-op: nothing to report. +func Send(ctx context.Context, findings []finding.Finding, opts Options) error { + if len(findings) == 0 { + return nil + } + + cfg, err := loadConfig(opts.ConfigPath) + if err != nil { + return fmt.Errorf("notify config: %w", err) + } + + providers := cfg.providers() + if len(providers) == 0 { + // nothing wired up; opt-in feature stays quiet rather than erroring. + return nil + } + + log := output.Module("NOTIFY") + client := httpx.Client(opts.Timeout) + + // run every provider; a failure on one sink must not suppress the others, so + // errors accumulate and the first is returned after all have been attempted. + var firstErr error + for i := 0; i < len(providers); i++ { + p := providers[i] + if err := p.send(ctx, client, findings); err != nil { + log.Error("%s delivery failed: %v", p.name(), err) + if firstErr == nil { + firstErr = fmt.Errorf("%s: %w", p.name(), err) + } + continue + } + log.Success("sent %d findings to %s", len(findings), p.name()) + } + + return firstErr +} + +// provider is one delivery sink. name is for logging; send formats findings into +// the sink's payload and POSTs it through the shared client. +type provider interface { + name() string + send(ctx context.Context, client *http.Client, findings []finding.Finding) error +} diff --git a/internal/notify/notify_test.go b/internal/notify/notify_test.go new file mode 100644 index 0000000..6d2d55b --- /dev/null +++ b/internal/notify/notify_test.go @@ -0,0 +1,224 @@ +/* +·━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━· +: : +: █▀ █ █▀▀ · Blazing-fast pentesting suite : +: ▄█ █ █▀ · BSD 3-Clause License : +: : +: (c) 2022-2026 vmfunc, xyzeva, : +: lunchcat alumni & contributors : +: : +·━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━· +*/ + +package notify + +import ( + "context" + "encoding/json" + "io" + "net/http" + "net/http/httptest" + "strings" + "testing" + "time" + + "github.com/dropalldatabases/sif/internal/finding" +) + +// sampleFindings returns a small mixed-severity batch for payload assertions. +func sampleFindings() []finding.Finding { + return []finding.Finding{ + {Target: "https://a.test", Module: "cors", Severity: finding.SeverityHigh, Key: "cors:a", Title: "reflected origin", Raw: "ACAO echo"}, + {Target: "https://a.test", Module: "headers", Severity: finding.SeverityInfo, Key: "headers:x", Title: "Server header", Raw: "nginx"}, + } +} + +// capture records the method, content-type and raw body of the request a provider +// makes, so each test can assert the wire shape without a real network. +type capture struct { + method string + contentType string + path string + body []byte +} + +// captureServer stands up an httptest server that records the single inbound +// request into c and replies 200, the happy path every provider expects. +func captureServer(t *testing.T, c *capture) *httptest.Server { + t.Helper() + srv := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) { + body, _ := io.ReadAll(r.Body) + c.method = r.Method + c.contentType = r.Header.Get("Content-Type") + c.path = r.URL.Path + c.body = body + w.WriteHeader(http.StatusOK) + })) + t.Cleanup(srv.Close) + return srv +} + +func TestSlackPayloadShape(t *testing.T) { + var c capture + srv := captureServer(t, &c) + + p := &slackProvider{webhook: srv.URL} + if err := p.send(context.Background(), srv.Client(), sampleFindings()); err != nil { + t.Fatalf("slack send: %v", err) + } + + assertPostJSON(t, c) + var payload slackPayload + if err := json.Unmarshal(c.body, &payload); err != nil { + t.Fatalf("unmarshal slack body: %v", err) + } + // slack keys on "text"; both findings must appear, code-block fenced. + if !strings.Contains(payload.Text, "reflected origin") || !strings.Contains(payload.Text, "Server header") { + t.Errorf("slack text missing findings: %q", payload.Text) + } + if !strings.HasPrefix(payload.Text, "```") { + t.Errorf("slack text not code-block fenced: %q", payload.Text) + } +} + +func TestDiscordPayloadShape(t *testing.T) { + var c capture + srv := captureServer(t, &c) + + p := &discordProvider{webhook: srv.URL} + if err := p.send(context.Background(), srv.Client(), sampleFindings()); err != nil { + t.Fatalf("discord send: %v", err) + } + + assertPostJSON(t, c) + var payload discordPayload + if err := json.Unmarshal(c.body, &payload); err != nil { + t.Fatalf("unmarshal discord body: %v", err) + } + // discord keys on "content", not "text". + if !strings.Contains(payload.Content, "reflected origin") { + t.Errorf("discord content missing finding: %q", payload.Content) + } +} + +func TestTelegramPayloadShape(t *testing.T) { + var c capture + srv := captureServer(t, &c) + + // repoint the bot api base at the test server for the lifetime of this test. + orig := telegramAPIBase + telegramAPIBase = srv.URL + t.Cleanup(func() { telegramAPIBase = orig }) + + p := &telegramProvider{token: "555:tok", chatID: "42"} + if err := p.send(context.Background(), srv.Client(), sampleFindings()); err != nil { + t.Fatalf("telegram send: %v", err) + } + + assertPostJSON(t, c) + // the token rides the path and the method is sendMessage. + if c.path != "/bot555:tok/sendMessage" { + t.Errorf("telegram path = %q, want /bot555:tok/sendMessage", c.path) + } + var payload telegramPayload + if err := json.Unmarshal(c.body, &payload); err != nil { + t.Fatalf("unmarshal telegram body: %v", err) + } + if payload.ChatID != "42" { + t.Errorf("telegram chat_id = %q, want 42", payload.ChatID) + } + if !strings.Contains(payload.Text, "reflected origin") { + t.Errorf("telegram text missing finding: %q", payload.Text) + } +} + +func TestWebhookPayloadShape(t *testing.T) { + var c capture + srv := captureServer(t, &c) + + p := &webhookProvider{url: srv.URL} + if err := p.send(context.Background(), srv.Client(), sampleFindings()); err != nil { + t.Fatalf("webhook send: %v", err) + } + + assertPostJSON(t, c) + var payload webhookPayload + if err := json.Unmarshal(c.body, &payload); err != nil { + t.Fatalf("unmarshal webhook body: %v", err) + } + // generic webhook carries structured findings, not a prerendered blob. + if payload.Count != 2 || len(payload.Findings) != 2 { + t.Fatalf("webhook count = %d / %d findings, want 2", payload.Count, len(payload.Findings)) + } + first := payload.Findings[0] + if first.Severity != "high" { + t.Errorf("webhook severity = %q, want canonical string \"high\"", first.Severity) + } + if first.Key != "cors:a" || first.Module != "cors" { + t.Errorf("webhook finding fields wrong: %+v", first) + } +} + +func TestProviderNon2xxIsError(t *testing.T) { + srv := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, _ *http.Request) { + w.WriteHeader(http.StatusForbidden) + })) + t.Cleanup(srv.Close) + + p := &slackProvider{webhook: srv.URL} + if err := p.send(context.Background(), srv.Client(), sampleFindings()); err == nil { + t.Fatal("send to 403 endpoint: want error, got nil") + } +} + +func TestSendNoProviderIsNoop(t *testing.T) { + clearNotifyEnv(t) + // no env, no config file -> zero providers -> Send must not error. + if err := Send(context.Background(), sampleFindings(), Options{Timeout: time.Second}); err != nil { + t.Fatalf("Send with no provider: want nil, got %v", err) + } +} + +func TestSendEmptyFindingsIsNoop(t *testing.T) { + // even with a provider configured, an empty batch must not POST anything. + hit := false + srv := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, _ *http.Request) { + hit = true + w.WriteHeader(http.StatusOK) + })) + t.Cleanup(srv.Close) + + clearNotifyEnv(t) + t.Setenv(envSlackWebhook, srv.URL) + if err := Send(context.Background(), nil, Options{Timeout: time.Second}); err != nil { + t.Fatalf("Send with empty findings: want nil, got %v", err) + } + if hit { + t.Fatal("Send with empty findings posted to provider, want no-op") + } +} + +func TestSendDeliversToConfiguredProvider(t *testing.T) { + var c capture + srv := captureServer(t, &c) + + clearNotifyEnv(t) + t.Setenv(envSlackWebhook, srv.URL) + if err := Send(context.Background(), sampleFindings(), Options{Timeout: time.Second}); err != nil { + t.Fatalf("Send: %v", err) + } + if c.method != http.MethodPost { + t.Fatalf("provider not hit (method=%q)", c.method) + } +} + +// assertPostJSON checks the request was a json POST. +func assertPostJSON(t *testing.T, c capture) { + t.Helper() + if c.method != http.MethodPost { + t.Errorf("method = %q, want POST", c.method) + } + if c.contentType != contentTypeJSON { + t.Errorf("content-type = %q, want %q", c.contentType, contentTypeJSON) + } +} diff --git a/internal/notify/slack.go b/internal/notify/slack.go new file mode 100644 index 0000000..0caf330 --- /dev/null +++ b/internal/notify/slack.go @@ -0,0 +1,45 @@ +/* +·━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━· +: : +: █▀ █ █▀▀ · Blazing-fast pentesting suite : +: ▄█ █ █▀ · BSD 3-Clause License : +: : +: (c) 2022-2026 vmfunc, xyzeva, : +: lunchcat alumni & contributors : +: : +·━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━· +*/ + +package notify + +import ( + "context" + "net/http" + + "github.com/dropalldatabases/sif/internal/finding" +) + +// slackProvider posts to a slack incoming webhook. the webhook url already pins +// the channel, so the payload is just the rendered text in slack's mrkdwn-aware +// "text" field wrapped in a code block to keep the fixed-width finding lines. +type slackProvider struct { + webhook string +} + +func (s *slackProvider) name() string { return "slack" } + +// slackPayload is the minimal incoming-webhook body: a single text field. +type slackPayload struct { + Text string `json:"text"` +} + +func (s *slackProvider) send(ctx context.Context, client *http.Client, findings []finding.Finding) error { + payload := slackPayload{Text: codeBlock(renderFindings(findings))} + return postJSON(ctx, client, s.webhook, payload) +} + +// codeBlock wraps body in a triple-backtick fence; both slack and discord render +// it fixed-width, which preserves the column-aligned finding lines. +func codeBlock(body string) string { + return "```\n" + body + "```" +} diff --git a/internal/notify/telegram.go b/internal/notify/telegram.go new file mode 100644 index 0000000..dbaef18 --- /dev/null +++ b/internal/notify/telegram.go @@ -0,0 +1,48 @@ +/* +·━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━· +: : +: █▀ █ █▀▀ · Blazing-fast pentesting suite : +: ▄█ █ █▀ · BSD 3-Clause License : +: : +: (c) 2022-2026 vmfunc, xyzeva, : +: lunchcat alumni & contributors : +: : +·━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━· +*/ + +package notify + +import ( + "context" + "net/http" + + "github.com/dropalldatabases/sif/internal/finding" +) + +// telegramAPIBase is the bot api root. it's a var so tests can repoint it at an +// httptest server; the token is appended path-side per telegram's scheme. +var telegramAPIBase = "https://api.telegram.org" + +// telegramProvider posts via the bot api's sendMessage. unlike slack/discord the +// destination isn't a single opaque webhook: it needs the bot token (in the url +// path) plus the chat id (in the body). +type telegramProvider struct { + token string + chatID string +} + +func (t *telegramProvider) name() string { return "telegram" } + +// telegramPayload is the sendMessage body. parse_mode "MarkdownV2" would force +// escaping every special char in the finding lines, so we send plain text and +// let the lines stand as-is. +type telegramPayload struct { + ChatID string `json:"chat_id"` + Text string `json:"text"` +} + +func (t *telegramProvider) send(ctx context.Context, client *http.Client, findings []finding.Finding) error { + endpoint := telegramAPIBase + "/bot" + t.token + "/sendMessage" + payload := telegramPayload{ChatID: t.chatID, Text: renderFindings(findings)} + return postJSON(ctx, client, endpoint, payload) +} diff --git a/internal/notify/webhook.go b/internal/notify/webhook.go new file mode 100644 index 0000000..3ef1d56 --- /dev/null +++ b/internal/notify/webhook.go @@ -0,0 +1,65 @@ +/* +·━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━· +: : +: █▀ █ █▀▀ · Blazing-fast pentesting suite : +: ▄█ █ █▀ · BSD 3-Clause License : +: : +: (c) 2022-2026 vmfunc, xyzeva, : +: lunchcat alumni & contributors : +: : +·━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━· +*/ + +package notify + +import ( + "context" + "net/http" + + "github.com/dropalldatabases/sif/internal/finding" +) + +// webhookProvider posts a structured json payload to an arbitrary endpoint. unlike +// the chat sinks it carries the findings as data, not a prerendered blob, so +// downstream automation (a siem, a bot, ci) keys off the fields directly. +type webhookProvider struct { + url string +} + +func (w *webhookProvider) name() string { return "webhook" } + +// webhookFinding is the per-item wire shape: the normalized Finding fields with +// severity flattened to its canonical string so a json consumer never sees the +// internal integer rank. +type webhookFinding struct { + Target string `json:"target"` + Module string `json:"module"` + Severity string `json:"severity"` + Key string `json:"key"` + Title string `json:"title"` + Raw string `json:"raw,omitempty"` +} + +// webhookPayload wraps the batch with a count so a consumer can size buffers / +// assert completeness without walking the slice first. +type webhookPayload struct { + Count int `json:"count"` + Findings []webhookFinding `json:"findings"` +} + +func (w *webhookProvider) send(ctx context.Context, client *http.Client, findings []finding.Finding) error { + items := make([]webhookFinding, 0, len(findings)) + for i := 0; i < len(findings); i++ { + f := findings[i] + items = append(items, webhookFinding{ + Target: f.Target, + Module: f.Module, + Severity: f.Severity.String(), + Key: f.Key, + Title: f.Title, + Raw: f.Raw, + }) + } + payload := webhookPayload{Count: len(items), Findings: items} + return postJSON(ctx, client, w.url, payload) +} diff --git a/internal/pool/pool.go b/internal/pool/pool.go new file mode 100644 index 0000000..063402b --- /dev/null +++ b/internal/pool/pool.go @@ -0,0 +1,57 @@ +/* +·━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━· +: : +: █▀ █ █▀▀ · Blazing-fast pentesting suite : +: ▄█ █ █▀ · BSD 3-Clause License : +: : +: (c) 2022-2026 vmfunc, xyzeva, : +: lunchcat alumni & contributors : +: : +·━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━· +*/ + +// Package pool spreads independent per-item work across a fixed set of workers +// that all pull from one shared channel. that's the point over a static +// modulo-stride partition: a slow or timing-out item only stalls the one worker +// holding it, the rest keep draining the queue instead of idling behind it. +package pool + +import "sync" + +// Each runs fn for every item in items, concurrently, across at most workers +// goroutines. order isn't preserved - fn must be safe to call from multiple +// goroutines and guard any shared state itself. blocks until every item is done. +func Each[T any](items []T, workers int, fn func(T)) { + if len(items) == 0 { + return + } + // floor at one worker; a non-positive count would otherwise spawn nothing + // and silently drop the work. + if workers < 1 { + workers = 1 + } + // never spin more workers than there is work for. + if workers > len(items) { + workers = len(items) + } + + queue := make(chan T, len(items)) + for i := 0; i < len(items); i++ { + queue <- items[i] + } + close(queue) + + var wg sync.WaitGroup + wg.Add(workers) + for i := 0; i < workers; i++ { + go func() { + defer wg.Done() + // pull until the queue is drained; a worker that finishes its + // current item just grabs the next, which is the work-stealing. + for item := range queue { + fn(item) + } + }() + } + wg.Wait() +} diff --git a/internal/pool/pool_test.go b/internal/pool/pool_test.go new file mode 100644 index 0000000..c8160c0 --- /dev/null +++ b/internal/pool/pool_test.go @@ -0,0 +1,145 @@ +/* +·━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━· +: : +: █▀ █ █▀▀ · Blazing-fast pentesting suite : +: ▄█ █ █▀ · BSD 3-Clause License : +: : +: (c) 2022-2026 vmfunc, xyzeva, : +: lunchcat alumni & contributors : +: : +·━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━· +*/ + +package pool + +import ( + "sync" + "sync/atomic" + "testing" +) + +// every item runs exactly once across a spread of sizes and worker counts, +// including the floors (zero/negative workers) and workers > len. +func TestEachProcessesAllExactlyOnce(t *testing.T) { + tests := []struct { + name string + items int + workers int + }{ + {"empty", 0, 4}, + {"single item", 1, 8}, + {"workers floored from zero", 5, 0}, + {"workers floored from negative", 5, -3}, + {"more workers than items", 3, 16}, + {"even split", 100, 4}, + {"uneven split", 101, 7}, + {"one worker", 50, 1}, + } + + for _, tt := range tests { + t.Run(tt.name, func(t *testing.T) { + items := make([]int, tt.items) + for i := 0; i < tt.items; i++ { + items[i] = i + } + + var mu sync.Mutex + seen := make(map[int]int, tt.items) + Each(items, tt.workers, func(v int) { + mu.Lock() + seen[v]++ + mu.Unlock() + }) + + if len(seen) != tt.items { + t.Fatalf("processed %d distinct items, want %d", len(seen), tt.items) + } + for v, n := range seen { + if n != 1 { + t.Errorf("item %d processed %d times, want 1", v, n) + } + } + }) + } +} + +// no more than `workers` (capped at len(items)) callbacks ever run at once. +func TestEachRespectsWorkerCap(t *testing.T) { + const ( + items = 200 + workers = 6 + ) + work := make([]int, items) + + var inFlight, peak int64 + var release = make(chan struct{}) + var started sync.WaitGroup + started.Add(items) + + go func() { + Each(work, workers, func(int) { + cur := atomic.AddInt64(&inFlight, 1) + for { + p := atomic.LoadInt64(&peak) + if cur <= p || atomic.CompareAndSwapInt64(&peak, p, cur) { + break + } + } + started.Done() + <-release + atomic.AddInt64(&inFlight, -1) + }) + }() + + // the cap means at most `workers` callbacks block on release at once, so + // release exactly that many at a time until everything drains. + done := make(chan struct{}) + go func() { + for i := 0; i < items; i++ { + release <- struct{}{} + } + close(done) + }() + <-done + + if got := atomic.LoadInt64(&peak); got > workers { + t.Fatalf("peak concurrency %d exceeded worker cap %d", got, workers) + } +} + +// the cap is min(workers, len(items)): fewer items than workers must not spin +// idle goroutines past the item count. +func TestEachCapsAtItemCount(t *testing.T) { + const ( + items = 3 + workers = 32 + ) + work := make([]int, items) + + var inFlight, peak int64 + var ready sync.WaitGroup + ready.Add(items) + release := make(chan struct{}) + + go func() { + for i := 0; i < items; i++ { + release <- struct{}{} + } + }() + + Each(work, workers, func(int) { + cur := atomic.AddInt64(&inFlight, 1) + for { + p := atomic.LoadInt64(&peak) + if cur <= p || atomic.CompareAndSwapInt64(&peak, p, cur) { + break + } + } + <-release + atomic.AddInt64(&inFlight, -1) + }) + + if got := atomic.LoadInt64(&peak); got > items { + t.Fatalf("peak concurrency %d exceeded item count %d", got, items) + } +} diff --git a/internal/scan/dirlist.go b/internal/scan/dirlist.go index fcbc5f4..0666f2c 100644 --- a/internal/scan/dirlist.go +++ b/internal/scan/dirlist.go @@ -29,6 +29,7 @@ import ( "github.com/dropalldatabases/sif/internal/httpx" "github.com/dropalldatabases/sif/internal/logger" "github.com/dropalldatabases/sif/internal/output" + "github.com/dropalldatabases/sif/internal/pool" ) // directoryURL is a var so integration tests can repoint it at a fixture. @@ -413,67 +414,54 @@ func Dirlist(size string, url string, timeout time.Duration, threads int, logdir progress := output.NewProgress(len(directories), "fuzzing") - var wg sync.WaitGroup var mu sync.Mutex - wg.Add(threads) results := make(DirectoryResults, 0, 64) - for thread := 0; thread < threads; thread++ { - go func(thread int) { - defer wg.Done() + pool.Each(directories, threads, func(directory string) { + progress.Increment(directory) - for i, directory := range directories { - if i%threads != thread { - continue - } + charmlog.Debugf("%s", directory) + dirReq, err := http.NewRequestWithContext(context.TODO(), http.MethodGet, url+"/"+directory, http.NoBody) + if err != nil { + charmlog.Debugf("Error creating request for %s: %s", directory, err) + return + } + resp, err := client.Do(dirReq) + if err != nil { + charmlog.Debugf("Error %s: %s", directory, err) + return + } - progress.Increment(directory) + meta, body := readMeta(resp) + reqURL := resp.Request.URL.String() + resp.Body.Close() - charmlog.Debugf("%s", directory) - dirReq, err := http.NewRequestWithContext(context.TODO(), http.MethodGet, url+"/"+directory, http.NoBody) - if err != nil { - charmlog.Debugf("Error creating request for %s: %s", directory, err) - continue - } - resp, err := client.Do(dirReq) - if err != nil { - charmlog.Debugf("Error %s: %s", directory, err) - continue - } + if !matcher.Matches(meta, body) { + return + } - meta, body := readMeta(resp) - reqURL := resp.Request.URL.String() - resp.Body.Close() + progress.Pause() + log.Success("found: %s [%s] (size=%d words=%d)", + output.Highlight.Render(directory), + output.Status.Render(strconv.Itoa(meta.status)), + meta.size, meta.words) + progress.Resume() - if !matcher.Matches(meta, body) { - continue - } + if logdir != "" { + _ = logger.Write(sanitizedURL, logdir, + fmt.Sprintf("%s [%s] size=%d words=%d\n", strconv.Itoa(meta.status), directory, meta.size, meta.words)) + } - progress.Pause() - log.Success("found: %s [%s] (size=%d words=%d)", - output.Highlight.Render(directory), - output.Status.Render(strconv.Itoa(meta.status)), - meta.size, meta.words) - progress.Resume() - - if logdir != "" { - _ = logger.Write(sanitizedURL, logdir, - fmt.Sprintf("%s [%s] size=%d words=%d\n", strconv.Itoa(meta.status), directory, meta.size, meta.words)) - } - - result := DirectoryResult{ - Url: reqURL, - StatusCode: meta.status, - Size: meta.size, - Words: meta.words, - } - mu.Lock() - results = append(results, result) - mu.Unlock() - } - }(thread) - } - wg.Wait() + result := DirectoryResult{ + Url: reqURL, + StatusCode: meta.status, + Size: meta.size, + Words: meta.words, + } + mu.Lock() + results = append(results, result) + mu.Unlock() + }) progress.Done() log.Complete(len(results), "found") diff --git a/internal/scan/dnslist.go b/internal/scan/dnslist.go index 9f2ef72..0d488dc 100644 --- a/internal/scan/dnslist.go +++ b/internal/scan/dnslist.go @@ -25,6 +25,7 @@ import ( "github.com/dropalldatabases/sif/internal/httpx" "github.com/dropalldatabases/sif/internal/logger" "github.com/dropalldatabases/sif/internal/output" + "github.com/dropalldatabases/sif/internal/pool" ) // dnsURL is a var so integration tests can repoint it at a fixture. @@ -148,61 +149,48 @@ func Dnslist(size string, url string, timeout time.Duration, threads int, logdir progress := output.NewProgress(len(dns), "enumerating") - var wg sync.WaitGroup var mu sync.Mutex - wg.Add(threads) urls := make([]string, 0, 64) - for thread := 0; thread < threads; thread++ { - go func(thread int) { - defer wg.Done() + pool.Each(dns, threads, func(domain string) { + progress.Increment(domain) - for i, domain := range dns { - if i%threads != thread { - continue - } + charmlog.Debugf("Looking up: %s", domain) - progress.Increment(domain) + host := domain + "." + sanitizedURL - charmlog.Debugf("Looking up: %s", domain) + // dns gate: skip the http probe entirely for names that don't + // resolve or that a wildcard zone answers. this is the whole point - + // no request per dead candidate. + ok, err := resolver.Resolve(host) + if err != nil { + charmlog.Debugf("resolve %s: %s", host, err) + return + } + if !ok { + return + } - host := domain + "." + sanitizedURL + // probe http first, then https - but a subdomain is recorded at + // most once. firing both schemes and appending on each is what + // double-counted every host on the old path. + foundURL, scheme := probeSubdomain(client, host) + if foundURL == "" { + return + } - // dns gate: skip the http probe entirely for names that don't - // resolve or that a wildcard zone answers. this is the whole point - - // no request per dead candidate. - ok, err := resolver.Resolve(host) - if err != nil { - charmlog.Debugf("resolve %s: %s", host, err) - continue - } - if !ok { - continue - } + mu.Lock() + urls = append(urls, foundURL) + mu.Unlock() - // probe http first, then https - but a subdomain is recorded at - // most once. firing both schemes and appending on each is what - // double-counted every host on the old path. - foundURL, scheme := probeSubdomain(client, host) - if foundURL == "" { - continue - } + progress.Pause() + log.Success("found: %s [%s]", output.Highlight.Render(host), scheme) + progress.Resume() - mu.Lock() - urls = append(urls, foundURL) - mu.Unlock() - - progress.Pause() - log.Success("found: %s [%s]", output.Highlight.Render(host), scheme) - progress.Resume() - - if logdir != "" { - _ = logger.Write(sanitizedURL, logdir, fmt.Sprintf("[%s] %s\n", scheme, host)) - } - } - }(thread) - } - wg.Wait() + if logdir != "" { + _ = logger.Write(sanitizedURL, logdir, fmt.Sprintf("[%s] %s\n", scheme, host)) + } + }) progress.Done() log.Complete(len(urls), "found") diff --git a/internal/scan/dork.go b/internal/scan/dork.go index c685285..585fdf7 100644 --- a/internal/scan/dork.go +++ b/internal/scan/dork.go @@ -28,6 +28,7 @@ import ( "github.com/dropalldatabases/sif/internal/httpx" "github.com/dropalldatabases/sif/internal/logger" "github.com/dropalldatabases/sif/internal/output" + "github.com/dropalldatabases/sif/internal/pool" googlesearch "github.com/rocketlaunchr/google-search" ) @@ -92,47 +93,33 @@ func Dork(url string, timeout time.Duration, threads int, logdir string) ([]Dork } // util.InitProgressBar() - var wg sync.WaitGroup var mu sync.Mutex - wg.Add(threads) dorkResults := []DorkResult{} - for thread := 0; thread < threads; thread++ { - go func(thread int) { - defer wg.Done() - - for i, dork := range dorks { - - if i%threads != thread { - continue - } - - results, err := googlesearch.Search(context.TODO(), fmt.Sprintf("%s %s", dork, sanitizedURL)) - if err != nil { - log.Debugf("error searching for dork %s: %v", dork, err) - continue - } - if len(results) > 0 { - spin.Stop() - output.Success("%s dork results found for dork %s", output.Status.Render(strconv.Itoa(len(results))), output.Highlight.Render(dork)) - spin.Start() - if logdir != "" { - _ = logger.Write(sanitizedURL, logdir, strconv.Itoa(len(results))+" dork results found for dork ["+dork+"]\n") - } - - result := DorkResult{ - Url: dork, - Count: len(results), - } - - mu.Lock() - dorkResults = append(dorkResults, result) - mu.Unlock() - } + pool.Each(dorks, threads, func(dork string) { + results, err := googlesearch.Search(context.TODO(), fmt.Sprintf("%s %s", dork, sanitizedURL)) + if err != nil { + log.Debugf("error searching for dork %s: %v", dork, err) + return + } + if len(results) > 0 { + spin.Stop() + output.Success("%s dork results found for dork %s", output.Status.Render(strconv.Itoa(len(results))), output.Highlight.Render(dork)) + spin.Start() + if logdir != "" { + _ = logger.Write(sanitizedURL, logdir, strconv.Itoa(len(results))+" dork results found for dork ["+dork+"]\n") } - }(thread) - } - wg.Wait() + + result := DorkResult{ + Url: dork, + Count: len(results), + } + + mu.Lock() + dorkResults = append(dorkResults, result) + mu.Unlock() + } + }) spin.Stop() output.ScanComplete("URL dorking", len(dorkResults), "found") diff --git a/internal/scan/git.go b/internal/scan/git.go index 9e8b425..2b2b6fb 100644 --- a/internal/scan/git.go +++ b/internal/scan/git.go @@ -25,6 +25,7 @@ import ( "github.com/dropalldatabases/sif/internal/httpx" "github.com/dropalldatabases/sif/internal/logger" "github.com/dropalldatabases/sif/internal/output" + "github.com/dropalldatabases/sif/internal/pool" ) // gitURL is a var so integration tests can repoint it at a fixture. @@ -71,50 +72,37 @@ func Git(url string, timeout time.Duration, threads int, logdir string) ([]strin gitUrls = append(gitUrls, scanner.Text()) } - var wg sync.WaitGroup var mu sync.Mutex - wg.Add(threads) foundUrls := []string{} - for thread := 0; thread < threads; thread++ { - go func(thread int) { - defer wg.Done() + pool.Each(gitUrls, threads, func(repourl string) { + charmlog.Debugf("%s", repourl) + gitReq, err := http.NewRequestWithContext(context.TODO(), http.MethodGet, url+"/"+repourl, http.NoBody) + if err != nil { + charmlog.Debugf("Error creating request for %s: %s", repourl, err) + return + } + resp, err := client.Do(gitReq) //nolint:bodyclose // drained and closed via httpx.DrainClose + if err != nil { + charmlog.Debugf("Error %s: %s", repourl, err) + return + } - for i, repourl := range gitUrls { - if i%threads != thread { - continue - } - - charmlog.Debugf("%s", repourl) - gitReq, err := http.NewRequestWithContext(context.TODO(), http.MethodGet, url+"/"+repourl, http.NoBody) - if err != nil { - charmlog.Debugf("Error creating request for %s: %s", repourl, err) - continue - } - resp, err := client.Do(gitReq) //nolint:bodyclose // drained and closed via httpx.DrainClose - if err != nil { - charmlog.Debugf("Error %s: %s", repourl, err) - continue - } - - if resp.StatusCode == 200 && !strings.HasPrefix(resp.Header.Get("Content-Type"), "text/html") { - spin.Stop() - log.Success("Git found at %s [%s]", output.Highlight.Render(repourl), output.Status.Render(strconv.Itoa(resp.StatusCode))) - spin.Start() - if logdir != "" { - logger.Write(sanitizedURL, logdir, strconv.Itoa(resp.StatusCode)+" git found at ["+repourl+"]\n") - } - - mu.Lock() - foundUrls = append(foundUrls, resp.Request.URL.String()) - mu.Unlock() - } - // status/headers only; drain so the conn returns to the pool. - httpx.DrainClose(resp) + if resp.StatusCode == 200 && !strings.HasPrefix(resp.Header.Get("Content-Type"), "text/html") { + spin.Stop() + log.Success("Git found at %s [%s]", output.Highlight.Render(repourl), output.Status.Render(strconv.Itoa(resp.StatusCode))) + spin.Start() + if logdir != "" { + logger.Write(sanitizedURL, logdir, strconv.Itoa(resp.StatusCode)+" git found at ["+repourl+"]\n") } - }(thread) - } - wg.Wait() + + mu.Lock() + foundUrls = append(foundUrls, resp.Request.URL.String()) + mu.Unlock() + } + // status/headers only; drain so the conn returns to the pool. + httpx.DrainClose(resp) + }) spin.Stop() log.Complete(len(foundUrls), "found") diff --git a/internal/scan/ports.go b/internal/scan/ports.go index a0566fa..202e4b4 100644 --- a/internal/scan/ports.go +++ b/internal/scan/ports.go @@ -26,6 +26,7 @@ import ( "github.com/dropalldatabases/sif/internal/httpx" "github.com/dropalldatabases/sif/internal/logger" "github.com/dropalldatabases/sif/internal/output" + "github.com/dropalldatabases/sif/internal/pool" ) // commonPorts is a var so integration tests can repoint it at a fixture. @@ -75,39 +76,26 @@ func Ports(ctx context.Context, scope string, url string, timeout time.Duration, var openPorts []string var mu sync.Mutex - var wg sync.WaitGroup - wg.Add(threads) - for thread := 0; thread < threads; thread++ { - go func(thread int) { - defer wg.Done() + pool.Each(ports, threads, func(port int) { + progress.Increment(strconv.Itoa(port)) - for i, port := range ports { - if i%threads != thread { - continue - } + charmlog.Debugf("Looking up: %d", port) + addr := fmt.Sprintf("%s:%d", sanitizedURL, port) + tcp, err := (&net.Dialer{Timeout: timeout}).DialContext(ctx, "tcp", addr) + if err != nil { + charmlog.Debugf("Error %d: %v", port, err) + } else { + progress.Pause() + log.Success("open: %s:%s [tcp]", sanitizedURL, output.Highlight.Render(strconv.Itoa(port))) + progress.Resume() - progress.Increment(strconv.Itoa(port)) - - charmlog.Debugf("Looking up: %d", port) - addr := fmt.Sprintf("%s:%d", sanitizedURL, port) - tcp, err := (&net.Dialer{Timeout: timeout}).DialContext(ctx, "tcp", addr) - if err != nil { - charmlog.Debugf("Error %d: %v", port, err) - } else { - progress.Pause() - log.Success("open: %s:%s [tcp]", sanitizedURL, output.Highlight.Render(strconv.Itoa(port))) - progress.Resume() - - mu.Lock() - openPorts = append(openPorts, strconv.Itoa(port)) - mu.Unlock() - _ = tcp.Close() - } - } - }(thread) - } - wg.Wait() + mu.Lock() + openPorts = append(openPorts, strconv.Itoa(port)) + mu.Unlock() + _ = tcp.Close() + } + }) progress.Done() log.Complete(len(openPorts), "open") diff --git a/internal/scan/scan.go b/internal/scan/scan.go index 4145f2b..7a309a6 100644 --- a/internal/scan/scan.go +++ b/internal/scan/scan.go @@ -23,13 +23,13 @@ import ( "net/http" "strconv" "strings" - "sync" "time" "github.com/charmbracelet/log" "github.com/dropalldatabases/sif/internal/httpx" "github.com/dropalldatabases/sif/internal/logger" "github.com/dropalldatabases/sif/internal/output" + "github.com/dropalldatabases/sif/internal/pool" ) // stripScheme drops the scheme:// prefix from url, or returns it unchanged when @@ -130,46 +130,32 @@ func Scan(url string, timeout time.Duration, threads int, logdir string) { robotsData = append(robotsData, scanner.Text()) } - var wg sync.WaitGroup - wg.Add(threads) - for thread := 0; thread < threads; thread++ { - go func(thread int) { - defer wg.Done() + pool.Each(robotsData, threads, func(robot string) { + if robot == "" || strings.HasPrefix(robot, "#") || strings.HasPrefix(robot, "User-agent: ") || strings.HasPrefix(robot, "Sitemap: ") { + return + } - for i, robot := range robotsData { - if i%threads != thread { - continue - } + _, sanitizedRobot, _ := strings.Cut(robot, ": ") + log.Debugf("%s", robot) + robotReq, err := http.NewRequestWithContext(context.TODO(), http.MethodGet, url+"/"+sanitizedRobot, http.NoBody) + if err != nil { + log.Debugf("Error creating request for %s: %s", sanitizedRobot, err) + return + } + resp, err := client.Do(robotReq) //nolint:bodyclose // drained and closed via httpx.DrainClose + if err != nil { + log.Debugf("Error %s: %s", sanitizedRobot, err) + return + } - if robot == "" || strings.HasPrefix(robot, "#") || strings.HasPrefix(robot, "User-agent: ") || strings.HasPrefix(robot, "Sitemap: ") { - continue - } - - _, sanitizedRobot, _ := strings.Cut(robot, ": ") - log.Debugf("%s", robot) - robotReq, err := http.NewRequestWithContext(context.TODO(), http.MethodGet, url+"/"+sanitizedRobot, http.NoBody) - if err != nil { - log.Debugf("Error creating request for %s: %s", sanitizedRobot, err) - continue - } - resp, err := client.Do(robotReq) //nolint:bodyclose // drained and closed via httpx.DrainClose - if err != nil { - log.Debugf("Error %s: %s", sanitizedRobot, err) - continue - } - - if resp.StatusCode != 404 { - output.Success("%s from robots: %s", output.Status.Render(strconv.Itoa(resp.StatusCode)), output.Highlight.Render(sanitizedRobot)) - if logdir != "" { - logger.Write(sanitizedURL, logdir, strconv.Itoa(resp.StatusCode)+" from robots: ["+sanitizedRobot+"]\n") - } - } - // status only; drain so the conn returns to the pool. - httpx.DrainClose(resp) + if resp.StatusCode != 404 { + output.Success("%s from robots: %s", output.Status.Render(strconv.Itoa(resp.StatusCode)), output.Highlight.Render(sanitizedRobot)) + if logdir != "" { + logger.Write(sanitizedURL, logdir, strconv.Itoa(resp.StatusCode)+" from robots: ["+sanitizedRobot+"]\n") } - - }(thread) - } - wg.Wait() + } + // status only; drain so the conn returns to the pool. + httpx.DrainClose(resp) + }) } } diff --git a/internal/scan/subdomaintakeover.go b/internal/scan/subdomaintakeover.go index d1389f4..fed7a30 100644 --- a/internal/scan/subdomaintakeover.go +++ b/internal/scan/subdomaintakeover.go @@ -20,12 +20,12 @@ import ( "net/http" "os" "strings" - "sync" "time" "github.com/charmbracelet/log" "github.com/dropalldatabases/sif/internal/httpx" "github.com/dropalldatabases/sif/internal/logger" + "github.com/dropalldatabases/sif/internal/pool" "github.com/dropalldatabases/sif/internal/styles" ) @@ -87,44 +87,29 @@ func SubdomainTakeover(url string, dnsResults []string, timeout time.Duration, t client := httpx.Client(timeout) - var wg sync.WaitGroup - wg.Add(threads) - + // buffered to the full candidate count so a send never blocks: Each only + // returns once every worker is done, and the channel is drained afterwards. resultsChan := make(chan SubdomainTakeoverResult, len(dnsResults)) - for thread := 0; thread < threads; thread++ { - go func(thread int) { - defer wg.Done() + pool.Each(dnsResults, threads, func(subdomain string) { + vulnerable, service := checkSubdomainTakeover(subdomain, client) + result := SubdomainTakeoverResult{ + Subdomain: subdomain, + Vulnerable: vulnerable, + Service: service, + } + resultsChan <- result - for i, subdomain := range dnsResults { - if i%threads != thread { - continue - } - - vulnerable, service := checkSubdomainTakeover(subdomain, client) - result := SubdomainTakeoverResult{ - Subdomain: subdomain, - Vulnerable: vulnerable, - Service: service, - } - resultsChan <- result - - if vulnerable { - subdomainlog.Warnf("Potential subdomain takeover: %s (%s)", styles.Highlight.Render(subdomain), service) - if logdir != "" { - logger.Write(sanitizedURL, logdir, fmt.Sprintf("Potential subdomain takeover: %s (%s)\n", subdomain, service)) - } - } else { - subdomainlog.Infof("Subdomain not vulnerable: %s", subdomain) - } + if vulnerable { + subdomainlog.Warnf("Potential subdomain takeover: %s (%s)", styles.Highlight.Render(subdomain), service) + if logdir != "" { + logger.Write(sanitizedURL, logdir, fmt.Sprintf("Potential subdomain takeover: %s (%s)\n", subdomain, service)) } - }(thread) - } - - go func() { - wg.Wait() - close(resultsChan) - }() + } else { + subdomainlog.Infof("Subdomain not vulnerable: %s", subdomain) + } + }) + close(resultsChan) var results []SubdomainTakeoverResult for result := range resultsChan { diff --git a/internal/store/store.go b/internal/store/store.go new file mode 100644 index 0000000..dcc2d6b --- /dev/null +++ b/internal/store/store.go @@ -0,0 +1,204 @@ +/* +·━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━· +: : +: █▀ █ █▀▀ · Blazing-fast pentesting suite : +: ▄█ █ █▀ · BSD 3-Clause License : +: : +: (c) 2022-2026 vmfunc, xyzeva, : +: lunchcat alumni & contributors : +: : +·━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━· +*/ + +// Package store persists a run's normalized findings as a json snapshot, one +// file per target, so a later run can diff against it and surface only what +// changed. it leans on encoding/json + os only - no new deps - and keys the +// delta off finding.Key, the identity the finding layer already guarantees is +// stable across runs. +package store + +import ( + "encoding/json" + "fmt" + "os" + "path/filepath" + "strings" + + "github.com/dropalldatabases/sif/internal/finding" +) + +// snapshotFileMode is applied to written snapshot files: owner read/write only. +// a snapshot enumerates a target's findings (urls, secrets, takeovers) and is +// not meant for other users on the box, so it stays 0600. +const snapshotFileMode = 0o600 + +// stateDirMode is applied to directories the store creates: owner rwx, group rx, +// no world access. matches the 0o750 the bundle asks for so the state tree isn't +// world-readable. +const stateDirMode = 0o750 + +// snapshotExt is the extension every snapshot file carries; makes the state dir +// self-describing and lets Load reconstruct the path from a bare target. +const snapshotExt = ".json" + +// defaultDirName is the sif-owned subdirectory under the user's config dir when +// no explicit store dir is given. DefaultDir joins it under os.UserConfigDir(). +const defaultDirName = "sif" + +// stateSubDir separates snapshots from anything else sif might drop in its +// config dir later, so the state tree is a single sweepable directory. +const stateSubDir = "state" + +// DefaultDir returns the fallback snapshot location: /sif/state. +// callers pass it when -store is unset and there's no logdir to reuse. the dir +// is not created here - Save does that lazily so a diff-less run touches nothing. +func DefaultDir() (string, error) { + configDir, err := os.UserConfigDir() + if err != nil { + return "", fmt.Errorf("resolving user config dir: %w", err) + } + return filepath.Join(configDir, defaultDirName, stateSubDir), nil +} + +// sanitize turns an arbitrary target (https://example.com:8443/path?q=1) into a +// single safe filename component. a target is attacker-influenced (it can come +// from a stdin pipe or a -f file), so every separator and path metacharacter is +// folded to '_' - no '/', '\\', '.', ':' survives to escape the state dir or +// collide with a parent reference. empty/degenerate input falls back to a fixed +// token rather than producing a dotfile or empty name. +func sanitize(target string) string { + var b strings.Builder + b.Grow(len(target)) + // collapse runs of separators: a scheme like "https://" is three metachars + // in a row, and one '_' reads cleaner than three without losing uniqueness. + prevSep := false + for i := 0; i < len(target); i++ { + c := target[i] + switch { + case c >= 'a' && c <= 'z', c >= 'A' && c <= 'Z', c >= '0' && c <= '9', c == '-': + b.WriteByte(c) + prevSep = false + default: + // every other byte (path sep, dot, colon, slash, space, unicode, and a + // literal '_') is a separator; fold it so traversal and dotfiles are + // impossible and a run never balloons the filename. + if !prevSep { + b.WriteByte('_') + prevSep = true + } + } + } + name := strings.Trim(b.String(), "_") + if name == "" { + return "target" + } + return name +} + +// pathFor builds the absolute snapshot path for a target under dir. kept private +// so the sanitized-filename invariant lives in one place; Save and Load both go +// through it so a target always maps to the same file. +func pathFor(dir, target string) string { + return filepath.Join(dir, sanitize(target)+snapshotExt) +} + +// Save writes the run's findings for target as a json snapshot under dir, +// overwriting any prior snapshot. the dir (and parents) is created lazily with +// stateDirMode. an empty findings slice is still written - it records "this +// target had nothing", which a later diff reads as a clean baseline rather than +// a missing one. +func Save(dir, target string, findings []finding.Finding) error { + if dir == "" { + return fmt.Errorf("store: empty snapshot dir") + } + if err := os.MkdirAll(dir, stateDirMode); err != nil { + return fmt.Errorf("creating state dir %q: %w", dir, err) + } + + // marshal a non-nil slice so an empty run serializes to [] not null; keeps + // the on-disk shape stable and Load's decode unambiguous. + if findings == nil { + findings = []finding.Finding{} + } + data, err := json.MarshalIndent(findings, "", " ") + if err != nil { + return fmt.Errorf("marshaling snapshot for %q: %w", target, err) + } + + path := pathFor(dir, target) + if err := os.WriteFile(path, data, snapshotFileMode); err != nil { + return fmt.Errorf("writing snapshot %q: %w", path, err) + } + return nil +} + +// Load reads the previously saved snapshot for target under dir. a missing +// snapshot is not an error - it's the first run for that target, so an empty +// slice comes back and the caller treats every current finding as new. a present +// but unreadable/corrupt file is a real error: silently swallowing it would make +// a broken store look like a fresh one and flag everything as added forever. +func Load(dir, target string) ([]finding.Finding, error) { + path := pathFor(dir, target) + data, err := os.ReadFile(path) + if err != nil { + if os.IsNotExist(err) { + return []finding.Finding{}, nil + } + return nil, fmt.Errorf("reading snapshot %q: %w", path, err) + } + + var findings []finding.Finding + if err := json.Unmarshal(data, &findings); err != nil { + return nil, fmt.Errorf("decoding snapshot %q: %w", path, err) + } + if findings == nil { + findings = []finding.Finding{} + } + return findings, nil +} + +// Diff computes the set-difference between two snapshots keyed on Finding.Key: +// added is everything in next whose Key isn't in old, removed is everything in +// old whose Key isn't in next. order follows the input slices (added in next's +// order, removed in old's) so output is deterministic for a given pair. a Key +// seen twice in one slice is deduped on first sight, so duplicate findings don't +// double-report. +func Diff(old, next []finding.Finding) (added, removed []finding.Finding) { + oldKeys := make(map[string]struct{}, len(old)) + for i := 0; i < len(old); i++ { + oldKeys[old[i].Key] = struct{}{} + } + nextKeys := make(map[string]struct{}, len(next)) + for i := 0; i < len(next); i++ { + nextKeys[next[i].Key] = struct{}{} + } + + seen := make(map[string]struct{}, len(next)) + for i := 0; i < len(next); i++ { + k := next[i].Key + if _, ok := oldKeys[k]; ok { + continue + } + if _, dup := seen[k]; dup { + continue + } + seen[k] = struct{}{} + added = append(added, next[i]) + } + + // reuse seen for the removed pass; the two key spaces don't overlap by + // construction (removed keys are absent from next) so a single map is safe. + clear(seen) + for i := 0; i < len(old); i++ { + k := old[i].Key + if _, ok := nextKeys[k]; ok { + continue + } + if _, dup := seen[k]; dup { + continue + } + seen[k] = struct{}{} + removed = append(removed, old[i]) + } + return added, removed +} diff --git a/internal/store/store_test.go b/internal/store/store_test.go new file mode 100644 index 0000000..3816948 --- /dev/null +++ b/internal/store/store_test.go @@ -0,0 +1,234 @@ +/* +·━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━· +: : +: █▀ █ █▀▀ · Blazing-fast pentesting suite : +: ▄█ █ █▀ · BSD 3-Clause License : +: : +: (c) 2022-2026 vmfunc, xyzeva, : +: lunchcat alumni & contributors : +: : +·━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━· +*/ + +package store + +import ( + "os" + "path/filepath" + "reflect" + "sort" + "testing" + + "github.com/dropalldatabases/sif/internal/finding" +) + +// sampleFindings is a small, stable set of findings reused across the round-trip +// and diff cases; covers two modules and two severities so marshaling exercises +// every Finding field. +func sampleFindings() []finding.Finding { + return []finding.Finding{ + { + Target: "https://example.com", + Module: "headers", + Severity: finding.SeverityInfo, + Key: "headers:Server", + Title: "Server", + Raw: "nginx", + }, + { + Target: "https://example.com", + Module: "cors", + Severity: finding.SeverityMedium, + Key: "cors:https://example.com:null", + Title: "null origin reflected", + Raw: "allow-origin: null", + }, + } +} + +func TestSaveLoadRoundTrip(t *testing.T) { + dir := t.TempDir() + const target = "https://example.com" + want := sampleFindings() + + if err := Save(dir, target, want); err != nil { + t.Fatalf("Save: %v", err) + } + + got, err := Load(dir, target) + if err != nil { + t.Fatalf("Load: %v", err) + } + if !reflect.DeepEqual(got, want) { + t.Fatalf("round-trip mismatch:\n got=%#v\nwant=%#v", got, want) + } +} + +func TestSaveCreatesNestedDir(t *testing.T) { + // the state dir need not exist; Save mkdir's it (and parents) lazily. + dir := filepath.Join(t.TempDir(), "nested", "state") + if err := Save(dir, "https://x.test", sampleFindings()); err != nil { + t.Fatalf("Save into missing dir: %v", err) + } + info, err := os.Stat(dir) + if err != nil { + t.Fatalf("stat created dir: %v", err) + } + if !info.IsDir() { + t.Fatalf("expected %q to be a directory", dir) + } +} + +func TestSaveEmptyDirRejected(t *testing.T) { + if err := Save("", "https://x.test", sampleFindings()); err == nil { + t.Fatal("Save with empty dir: want error, got nil") + } +} + +func TestSaveEmptyFindingsRoundTrips(t *testing.T) { + // an empty run is a valid baseline: Save writes [], Load reads back an empty + // (non-nil) slice, never an error. + dir := t.TempDir() + const target = "https://empty.test" + + if err := Save(dir, target, nil); err != nil { + t.Fatalf("Save nil findings: %v", err) + } + got, err := Load(dir, target) + if err != nil { + t.Fatalf("Load: %v", err) + } + if got == nil { + t.Fatal("Load returned nil, want non-nil empty slice") + } + if len(got) != 0 { + t.Fatalf("Load returned %d findings, want 0", len(got)) + } +} + +func TestLoadMissingSnapshotIsEmpty(t *testing.T) { + // no prior run for this target: a missing file is not an error, it's an empty + // baseline so the first run treats everything as added. + dir := t.TempDir() + got, err := Load(dir, "https://never-scanned.test") + if err != nil { + t.Fatalf("Load missing snapshot: %v", err) + } + if got == nil { + t.Fatal("Load returned nil, want non-nil empty slice") + } + if len(got) != 0 { + t.Fatalf("Load missing snapshot returned %d findings, want 0", len(got)) + } +} + +func TestLoadCorruptSnapshotErrors(t *testing.T) { + // a present-but-garbage snapshot must surface loudly: treating it as empty + // would silently re-flag every finding as new on every run. + dir := t.TempDir() + const target = "https://corrupt.test" + path := filepath.Join(dir, sanitize(target)+snapshotExt) + if err := os.WriteFile(path, []byte("{not json"), snapshotFileMode); err != nil { + t.Fatalf("seeding corrupt snapshot: %v", err) + } + if _, err := Load(dir, target); err == nil { + t.Fatal("Load corrupt snapshot: want error, got nil") + } +} + +func TestDiffAddedAndRemoved(t *testing.T) { + base := sampleFindings() + + // next drops the cors finding (removed) and adds a takeover (added); the + // headers finding is unchanged and must appear in neither delta. + next := []finding.Finding{ + base[0], // headers - unchanged + { + Target: "https://example.com", + Module: "subdomain_takeover", + Severity: finding.SeverityHigh, + Key: "subdomain_takeover:old.example.com", + Title: "takeover: old.example.com", + Raw: "GitHub Pages", + }, + } + + added, removed := Diff(base, next) + + if len(added) != 1 || added[0].Key != "subdomain_takeover:old.example.com" { + t.Fatalf("added = %#v, want the takeover only", added) + } + if len(removed) != 1 || removed[0].Key != "cors:https://example.com:null" { + t.Fatalf("removed = %#v, want the cors finding only", removed) + } +} + +func TestDiffNoChange(t *testing.T) { + // identical snapshots produce no delta in either direction. + base := sampleFindings() + added, removed := Diff(base, base) + if len(added) != 0 || len(removed) != 0 { + t.Fatalf("identical snapshots: added=%d removed=%d, want 0/0", len(added), len(removed)) + } +} + +func TestDiffFirstRunAllAdded(t *testing.T) { + // no prior snapshot (empty old) means every current finding is new. + next := sampleFindings() + added, removed := Diff(nil, next) + if len(removed) != 0 { + t.Fatalf("first run removed=%d, want 0", len(removed)) + } + gotKeys := keysOf(added) + wantKeys := keysOf(next) + if !reflect.DeepEqual(gotKeys, wantKeys) { + t.Fatalf("first run added keys=%v, want %v", gotKeys, wantKeys) + } +} + +func TestDiffDedupesRepeatedKey(t *testing.T) { + // a Key appearing twice in the new snapshot is reported once, not twice. + f := sampleFindings()[0] + next := []finding.Finding{f, f} + added, _ := Diff(nil, next) + if len(added) != 1 { + t.Fatalf("duplicate key reported %d times, want 1", len(added)) + } +} + +// keysOf returns the sorted Key set of a finding slice for order-independent +// comparison. +func keysOf(fs []finding.Finding) []string { + out := make([]string, 0, len(fs)) + for i := 0; i < len(fs); i++ { + out = append(out, fs[i].Key) + } + sort.Strings(out) + return out +} + +func TestSanitizeNoTraversal(t *testing.T) { + // sanitize is the only barrier between an attacker-influenced target and the + // state dir; assert no separator or traversal token survives. + tests := []struct { + in string + want string + }{ + {"https://example.com", "https_example_com"}, + {"../../etc/passwd", "etc_passwd"}, + {"a/b/c", "a_b_c"}, + {"....//....//x", "x"}, + {"", "target"}, + {"///", "target"}, + {"host:8443/path?q=1", "host_8443_path_q_1"}, + } + for _, tt := range tests { + got := sanitize(tt.in) + if got != tt.want { + t.Errorf("sanitize(%q) = %q, want %q", tt.in, got, tt.want) + } + if filepath.Base(got) != got { + t.Errorf("sanitize(%q) = %q escapes its component", tt.in, got) + } + } +} diff --git a/man/sif.1 b/man/sif.1 index 9799fbf..b364fa8 100644 --- a/man/sif.1 +++ b/man/sif.1 @@ -200,6 +200,27 @@ plain output for pipelines: route all chrome to stderr and print one normalized finding per line to stdout as \fB[severity] target module title\fR. implies non\-interactive (no spinners). .TP +.B \-diff +diff mode: snapshot each target's findings to a json file and, on a re\-scan, +print only the delta against the last snapshot (\fB+ new\fR for findings that +appeared, \fB- gone\fR for ones that vanished), then overwrite the snapshot. +the first run for a target reports everything as new. +.TP +.BR \-store " \fIdir\fR" +snapshot directory for \fB\-diff\fR. defaults to the \fB\-log\fR dir if set, +otherwise \fI/sif/state\fR. one sanitized file per target. +.B \-notify +ship findings to every configured provider (slack, discord, telegram, generic +webhook) after the scan. providers are configured env\-first and overridable by a +yaml file; with nothing configured this is a silent no\-op. +.TP +.BR \-notify\-severity " \fIlevel\fR" +minimum severity to send: \fBinfo\fR, \fBlow\fR, \fBmedium\fR, \fBhigh\fR or +\fBcritical\fR (default \fBmedium\fR). findings below the floor are dropped. +.TP +.BR \-notify\-config " \fIfile\fR" +path to a notify\-compatible yaml config whose values override the env vars. +.TP .B \-api emit json results and suppress the interactive output. .SH MODULES @@ -231,6 +252,22 @@ api key used by \fB\-shodan\fR. .B SECURITYTRAILS_API_KEY api key used by \fB\-securitytrails\fR. .TP +.B SLACK_WEBHOOK_URL +slack incoming webhook used by \fB\-notify\fR (yaml key \fBslack_webhook_url\fR). +.TP +.B DISCORD_WEBHOOK_URL +discord webhook used by \fB\-notify\fR (yaml key \fBdiscord_webhook_url\fR). +.TP +.B TELEGRAM_BOT_TOKEN +telegram bot token used by \fB\-notify\fR (yaml key \fBtelegram_api_key\fR); +requires \fBTELEGRAM_CHAT_ID\fR too. +.TP +.B TELEGRAM_CHAT_ID +telegram destination chat used by \fB\-notify\fR (yaml key \fBtelegram_chat_id\fR). +.TP +.B NOTIFY_WEBHOOK_URL +generic json webhook used by \fB\-notify\fR (yaml key \fBwebhook_url\fR). +.TP .B SIF_NO_PATCHNOTES set to any value to suppress the once\-per\-version patch note shown at startup. .SH FILES diff --git a/notify_test.go b/notify_test.go new file mode 100644 index 0000000..c8f111d --- /dev/null +++ b/notify_test.go @@ -0,0 +1,157 @@ +/* +·━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━· +: : +: █▀ █ █▀▀ · Blazing-fast pentesting suite : +: ▄█ █ █▀ · BSD 3-Clause License : +: : +: (c) 2022-2026 vmfunc, xyzeva, : +: lunchcat alumni & contributors : +: : +·━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━· +*/ + +package sif + +import ( + "context" + "encoding/json" + "io" + "net/http" + "net/http/httptest" + "testing" + "time" + + "github.com/dropalldatabases/sif/internal/config" + "github.com/dropalldatabases/sif/internal/finding" +) + +// notifyWebhookBody is the generic-webhook wire shape, mirrored here so the +// wiring test can assert which findings crossed the severity floor without +// reaching into the notify package internals. +type notifyWebhookBody struct { + Count int `json:"count"` + Findings []struct { + Severity string `json:"severity"` + Key string `json:"key"` + } `json:"findings"` +} + +// mixedSeverityFindings spans the whole ladder so a floor test has something to +// drop on either side of every threshold. +func mixedSeverityFindings() []finding.Finding { + return []finding.Finding{ + {Target: "https://t.test", Module: "headers", Severity: finding.SeverityInfo, Key: "headers:s", Title: "server"}, + {Target: "https://t.test", Module: "redirect", Severity: finding.SeverityLow, Key: "redirect:r", Title: "open redirect"}, + {Target: "https://t.test", Module: "sql", Severity: finding.SeverityMedium, Key: "sql:e", Title: "db error"}, + {Target: "https://t.test", Module: "cors", Severity: finding.SeverityHigh, Key: "cors:c", Title: "reflected origin"}, + {Target: "https://t.test", Module: "lfi", Severity: finding.SeverityCritical, Key: "lfi:l", Title: "path traversal"}, + } +} + +func TestNotifyFindingsSeverityFilter(t *testing.T) { + tests := []struct { + name string + floor string + wantKeys []string + }{ + {name: "medium drops info+low", floor: "medium", wantKeys: []string{"sql:e", "cors:c", "lfi:l"}}, + {name: "high drops everything below", floor: "high", wantKeys: []string{"cors:c", "lfi:l"}}, + {name: "info keeps all", floor: "info", wantKeys: []string{"headers:s", "redirect:r", "sql:e", "cors:c", "lfi:l"}}, + {name: "critical keeps only critical", floor: "critical", wantKeys: []string{"lfi:l"}}, + // an unrecognized floor must default to medium, not let info through. + {name: "garbage floor defaults medium", floor: "bogus", wantKeys: []string{"sql:e", "cors:c", "lfi:l"}}, + } + + for _, tt := range tests { + t.Run(tt.name, func(t *testing.T) { + var got notifyWebhookBody + srv := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) { + body, _ := io.ReadAll(r.Body) + if err := json.Unmarshal(body, &got); err != nil { + t.Errorf("unmarshal webhook body: %v", err) + } + w.WriteHeader(http.StatusOK) + })) + t.Cleanup(srv.Close) + + // route notify at the test server via the generic webhook env var. + t.Setenv("SLACK_WEBHOOK_URL", "") + t.Setenv("DISCORD_WEBHOOK_URL", "") + t.Setenv("TELEGRAM_BOT_TOKEN", "") + t.Setenv("TELEGRAM_CHAT_ID", "") + t.Setenv("NOTIFY_WEBHOOK_URL", srv.URL) + + app := &App{settings: &config.Settings{ + Notify: true, + NotifySeverity: tt.floor, + Timeout: time.Second, + }} + if err := app.notifyFindings(context.Background(), mixedSeverityFindings()); err != nil { + t.Fatalf("notifyFindings: %v", err) + } + + gotKeys := make([]string, 0, len(got.Findings)) + for _, f := range got.Findings { + gotKeys = append(gotKeys, f.Key) + } + if !equalStringSets(gotKeys, tt.wantKeys) { + t.Errorf("floor %q delivered keys %v, want %v", tt.floor, gotKeys, tt.wantKeys) + } + if got.Count != len(tt.wantKeys) { + t.Errorf("floor %q count = %d, want %d", tt.floor, got.Count, len(tt.wantKeys)) + } + }) + } +} + +func TestNotifyFindingsBelowFloorIsNoop(t *testing.T) { + // every finding below the floor -> nothing crosses -> no POST at all. + hit := false + srv := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, _ *http.Request) { + hit = true + w.WriteHeader(http.StatusOK) + })) + t.Cleanup(srv.Close) + + t.Setenv("SLACK_WEBHOOK_URL", "") + t.Setenv("DISCORD_WEBHOOK_URL", "") + t.Setenv("TELEGRAM_BOT_TOKEN", "") + t.Setenv("TELEGRAM_CHAT_ID", "") + t.Setenv("NOTIFY_WEBHOOK_URL", srv.URL) + + app := &App{settings: &config.Settings{ + Notify: true, + NotifySeverity: "critical", + Timeout: time.Second, + }} + infoOnly := []finding.Finding{ + {Target: "https://t.test", Module: "headers", Severity: finding.SeverityInfo, Key: "headers:s", Title: "server"}, + } + if err := app.notifyFindings(context.Background(), infoOnly); err != nil { + t.Fatalf("notifyFindings: %v", err) + } + if hit { + t.Fatal("notifyFindings posted with everything below floor, want no-op") + } +} + +// equalStringSets reports whether a and b contain the same elements regardless +// of order; the wire order mirrors input order, but order isn't the contract. +func equalStringSets(a, b []string) bool { + if len(a) != len(b) { + return false + } + seen := make(map[string]int, len(a)) + for _, s := range a { + seen[s]++ + } + for _, s := range b { + seen[s]-- + } + for _, n := range seen { + if n != 0 { + return false + } + } + return true +} diff --git a/sif.go b/sif.go index 5044f89..acc2aa2 100644 --- a/sif.go +++ b/sif.go @@ -31,12 +31,14 @@ import ( "github.com/dropalldatabases/sif/internal/httpx" "github.com/dropalldatabases/sif/internal/logger" "github.com/dropalldatabases/sif/internal/modules" + "github.com/dropalldatabases/sif/internal/notify" "github.com/dropalldatabases/sif/internal/output" "github.com/dropalldatabases/sif/internal/report" "github.com/dropalldatabases/sif/internal/scan" "github.com/dropalldatabases/sif/internal/scan/builtin" "github.com/dropalldatabases/sif/internal/scan/frameworks" jsscan "github.com/dropalldatabases/sif/internal/scan/js" + "github.com/dropalldatabases/sif/internal/store" ) // App represents the main application structure for sif. @@ -303,10 +305,22 @@ func (app *App) Run() error { reportResults := make([]report.Result, 0, 16) // normalized findings for the whole run; the single Flatten-driven view that - // notify and diff (later) consume. collected alongside the report so both - // describe the same scanners from one pass. + // notify and diff consume. collected alongside the report so both describe the + // same scanners from one pass. allFindings := make([]finding.Finding, 0, 16) + // resolve the snapshot dir once when diff mode is on; a bad default isn't + // fatal - diff just no-ops for the run rather than killing the scan. + storeDir := "" + if app.settings.Diff { + dir, err := app.resolveStoreDir() + if err != nil { + log.Warnf("diff disabled: %v", err) + } else { + storeDir = dir + } + } + for _, url := range app.targets { output.Info("Starting scan on %s", output.Highlight.Render(url)) @@ -664,7 +678,17 @@ func (app *App) Run() error { fmt.Println(string(marshalled)) } - allFindings = append(allFindings, collectFindings(url, moduleResults)...) + targetFindings := collectFindings(url, moduleResults) + allFindings = append(allFindings, targetFindings...) + + // diff mode is per-target: load this target's last snapshot, surface only + // the delta, then overwrite the snapshot so the next run diffs against now. + // storeDir is "" when diff is off or the dir couldn't resolve, in which + // case this is a no-op and behavior is unchanged. + if storeDir != "" { + app.diffTarget(storeDir, url, targetFindings) + } + // the report carries raw blobs and is only built when an export flag is // set, so the common path skips the marshalling entirely. if wantReport { @@ -676,6 +700,14 @@ func (app *App) Run() error { // count now so the path is live and observable without changing output. log.Debugf("normalized %d findings across %d targets", len(allFindings), len(app.targets)) + // notify: ship the severity-filtered findings to any configured provider. + // kept as an isolated block so it merges cleanly with the diff-store bundle. + if app.settings.Notify { + if err := app.notifyFindings(context.Background(), allFindings); err != nil { + log.Errorf("notify: %v", err) + } + } + // -silent: stdout is the findings stream, one terse line each. all chrome // already went to stderr via the rerouted sink, so this is the only thing a // downstream pipe sees. @@ -709,9 +741,9 @@ func printFindings(findings []finding.Finding) { } // collectFindings normalizes one target's module results through finding.Flatten -// - the single normalization path that notify and diff (later bundles) build on. -// every scan result struct collapses to flat, severity-ranked findings here so a -// scanner is described once, not once per consumer. +// - the single normalization path that notify and diff build on. every scan +// result struct collapses to flat, severity-ranked findings here so a scanner is +// described once, not once per consumer. func collectFindings(target string, moduleResults []ModuleResult) []finding.Finding { out := make([]finding.Finding, 0, len(moduleResults)) for _, mr := range moduleResults { @@ -720,6 +752,66 @@ func collectFindings(target string, moduleResults []ModuleResult) []finding.Find return out } +// resolveStoreDir picks the snapshot directory for diff mode. precedence: an +// explicit -store wins; else the run's log dir is reused (snapshots live next to +// logs); else the per-user default under /sif/state. returns an +// error only when no usable location exists, so the caller can disable diff +// without failing the scan. +func (app *App) resolveStoreDir() (string, error) { + if app.settings.Store != "" { + return app.settings.Store, nil + } + if app.settings.LogDir != "" { + return app.settings.LogDir, nil + } + dir, err := store.DefaultDir() + if err != nil { + return "", fmt.Errorf("resolving snapshot dir: %w", err) + } + return dir, nil +} + +// diffTarget loads target's previous snapshot, prints the added/removed delta +// against the current findings, then overwrites the snapshot so the next run +// diffs against this one. a load failure surfaces but doesn't abort the run - +// the new snapshot is still written so a corrupt baseline self-heals. always +// saves, even when the delta is empty, to advance the baseline. +func (app *App) diffTarget(dir, target string, current []finding.Finding) { + previous, err := store.Load(dir, target) + if err != nil { + log.Warnf("diff: reading snapshot for %s, treating as fresh: %v", target, err) + previous = nil + } + + added, removed := store.Diff(previous, current) + printDiff(target, added, removed) + + if err := store.Save(dir, target, current); err != nil { + log.Warnf("diff: saving snapshot for %s: %v", target, err) + } +} + +// printDiff renders a target's diff: each added finding marked "+ new", each +// removed one "- gone", with a one-line note when nothing changed. routed +// through the shared output sink so -silent keeps it on stderr alongside the +// other chrome. a single Builder keeps the block from interleaving. +func printDiff(target string, added, removed []finding.Finding) { + if len(added) == 0 && len(removed) == 0 { + output.Info("diff %s: no changes since last snapshot", target) + return + } + + var b strings.Builder + fmt.Fprintf(&b, "diff %s: %d new, %d gone\n", target, len(added), len(removed)) + for i := 0; i < len(added); i++ { + fmt.Fprintf(&b, " + new %s\n", added[i].Line()) + } + for i := 0; i < len(removed); i++ { + fmt.Fprintf(&b, " - gone %s\n", removed[i].Line()) + } + fmt.Fprint(output.Writer(), b.String()) +} + // collectReportResults flattens one target's module results into the report // model, carrying each finding as raw json so the report package stays free of // scan types. a result that won't marshal is skipped rather than failing the run. @@ -762,6 +854,31 @@ func (app *App) writeReports(results []report.Result) error { return nil } +// notifyFindings filters the run's findings to the -notify-severity floor and +// ships the survivors to every configured provider. an unrecognized severity +// string parses to SeverityUnknown, which would let everything through; guard +// against that by defaulting to medium so a typo can't flood a channel with +// info noise. an empty filtered set makes notify.Send a no-op. +func (app *App) notifyFindings(ctx context.Context, findings []finding.Finding) error { + floor := finding.ParseSeverity(app.settings.NotifySeverity) + if floor == finding.SeverityUnknown { + log.Warnf("notify: unknown severity %q, defaulting to medium", app.settings.NotifySeverity) + floor = finding.SeverityMedium + } + + filtered := make([]finding.Finding, 0, len(findings)) + for i := 0; i < len(findings); i++ { + if findings[i].Severity.AtLeast(floor) { + filtered = append(filtered, findings[i]) + } + } + + return notify.Send(ctx, filtered, notify.Options{ + Timeout: app.settings.Timeout, + ConfigPath: app.settings.NotifyConfig, + }) +} + // expandTargets queries SecurityTrails for each original target and returns // newly discovered domains (subdomains + associated) for target expansion func (app *App) expandTargets() []string { diff --git a/sif_test.go b/sif_test.go index c6455b4..36a0be2 100644 --- a/sif_test.go +++ b/sif_test.go @@ -20,6 +20,7 @@ import ( "github.com/dropalldatabases/sif/internal/config" "github.com/dropalldatabases/sif/internal/finding" + "github.com/dropalldatabases/sif/internal/store" ) // TestMain neutralizes the stdin seam for the whole package so tests that build @@ -373,3 +374,72 @@ func TestUrlResult_JSON(t *testing.T) { t.Errorf("UrlResult.Results = %d, want 1", len(ur.Results)) } } + +func TestResolveStoreDir(t *testing.T) { + // explicit -store wins over everything. + explicit := &App{settings: &config.Settings{Store: "/tmp/snaps", LogDir: "/tmp/logs"}} + if dir, err := explicit.resolveStoreDir(); err != nil || dir != "/tmp/snaps" { + t.Fatalf("explicit store: got (%q, %v), want (/tmp/snaps, nil)", dir, err) + } + + // no -store: reuse the log dir. + logged := &App{settings: &config.Settings{LogDir: "/tmp/logs"}} + if dir, err := logged.resolveStoreDir(); err != nil || dir != "/tmp/logs" { + t.Fatalf("log dir fallback: got (%q, %v), want (/tmp/logs, nil)", dir, err) + } + + // neither set: fall through to the per-user default (non-empty, no error). + bare := &App{settings: &config.Settings{}} + dir, err := bare.resolveStoreDir() + if err != nil { + t.Fatalf("default store dir: %v", err) + } + if dir == "" { + t.Fatal("default store dir resolved empty") + } +} + +func TestDiffTargetSnapshotsAndDiffs(t *testing.T) { + dir := t.TempDir() + const target = "https://diff.example.com" + app := &App{settings: &config.Settings{Diff: true, Store: dir}} + + first := []finding.Finding{ + {Target: target, Module: "headers", Severity: finding.SeverityInfo, Key: "headers:Server", Title: "Server", Raw: "nginx"}, + } + + // first run: no prior snapshot, everything is new; the snapshot must persist. + app.diffTarget(dir, target, first) + + saved, err := store.Load(dir, target) + if err != nil { + t.Fatalf("load after first run: %v", err) + } + if len(saved) != 1 || saved[0].Key != "headers:Server" { + t.Fatalf("snapshot after first run = %#v, want the headers finding", saved) + } + + // second run with a different set: the snapshot must advance to the new set so + // a third run would diff against it. + second := []finding.Finding{ + {Target: target, Module: "cors", Severity: finding.SeverityMedium, Key: "cors:x", Title: "null origin", Raw: "null"}, + } + app.diffTarget(dir, target, second) + + saved, err = store.Load(dir, target) + if err != nil { + t.Fatalf("load after second run: %v", err) + } + if len(saved) != 1 || saved[0].Key != "cors:x" { + t.Fatalf("snapshot after second run = %#v, want the cors finding", saved) + } + + // the delta between the two snapshots is exactly: headers gone, cors new. + added, removed := store.Diff(first, second) + if len(added) != 1 || added[0].Key != "cors:x" { + t.Fatalf("added = %#v, want cors:x", added) + } + if len(removed) != 1 || removed[0].Key != "headers:Server" { + t.Fatalf("removed = %#v, want headers:Server", removed) + } +}