Commit Graph

441 Commits

Author SHA1 Message Date
Justin Bollinger
de2b400f6d chore: align CI and tooling with global development standards
- Remove 6 duplicate per-version pytest workflows (matrix build covers all)
- Pin GitHub Actions to SHA hashes with version comments
- Add persist-credentials: false to checkout steps
- Replace mypy with ty for type checking (faster, stricter)
- Pin dev deps to exact versions (ty==0.0.17, ruff==0.15.1, pytest==9.0.2, pytest-cov==7.0.0)
- Remove types-* stub packages (ty doesn't need them)
- Remove stale [dependency-groups] section from pyproject.toml
- Update shell scripts to use set -euo pipefail
- Add prek.toml for git hook management (pre-push, post-commit)
- Add lint-infra.yml workflow (shellcheck + actionlint)
- Fix actionlint warning: pass github.head_ref through env var
- Track CLAUDE.md and .claude/ scripts in git
- Update README.md and Makefile references from mypy to ty

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-19 12:42:51 -05:00
Justin Bollinger
e7f4ed815b fix: exclude omen directory from mypy checking
Vendored third-party OMEN utils were also failing mypy on push.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-19 11:43:22 -05:00
Justin Bollinger
2142732bfa fix: exclude omen directory from ruff linting
Vendored third-party OMEN utils were failing ruff checks on push.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-19 11:43:04 -05:00
Justin Bollinger
f0bba73225 fix: auto-detect training device instead of defaulting to CUDA
The PassGPT training device menu now uses _detect_device() to default
to the best available device (CUDA > MPS > CPU) rather than always
defaulting to CUDA, which fails on systems without NVIDIA GPUs.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-18 18:47:41 -05:00
Justin Bollinger
00a60af9a6 Merge pull request #77 from trustedsec/bug/debug-flag-show-http-requests
fix: set HF_HUB_DISABLE_TELEMETRY before HuggingFace imports
v2.0.2
2026-02-18 15:36:24 -05:00
Justin Bollinger
893533c200 fix: set HF_HUB_DISABLE_TELEMETRY before HuggingFace imports in main.py
The env var was only set in the subprocess scripts (passgpt_generate.py,
passgpt_train.py) but not in main.py, where torch/transformers are imported
at module level. This ensures telemetry is disabled before any HF-related
imports and is inherited by spawned subprocesses.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-18 15:34:11 -05:00
Justin Bollinger
546c608c33 fix: pass --debug flag to PassGPT subprocesses and create GitHub Releases on version bump
PassGPT generate/train scripts now accept --debug to log HuggingFace HTTP
requests. Version-bump workflow creates a GitHub Release (not just a tag)
so check_for_updates can find /releases/latest. Bump logic now uses minor
for feat/ branches and patch for everything else.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-18 15:26:53 -05:00
Justin Bollinger
446fbd6d95 fix: --debug flag now outputs HTTP requests to screen
Enable urllib3 debug logging when --debug is active so all requests
made via the requests library are visible on stderr.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-18 15:17:36 -05:00
Justin Bollinger
c67a2f6800 Merge pull request #76 from trustedsec/feat/passgpt-attack
feat: add PassGPT attack, version bump workflow, and editable install
v2.0.1
2026-02-18 15:00:16 -05:00
Justin Bollinger
41b51818c1 feat: auto-bump patch version on PR merge to main
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-18 14:58:47 -05:00
Justin Bollinger
c0d2cad2c1 fix: skip ML-dependent tests in CI and mock version in version check test
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-18 14:52:16 -05:00
Justin Bollinger
603375d83f fix: use editable install so updates apply to the repo directory
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-18 14:47:15 -05:00
Justin Bollinger
b6524cbdc4 feat: add training time estimates and device selection to PassGPT menu
Show estimated training times for CUDA/MPS/CPU before starting a
training run. Add device selection prompt with cuda as the default.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-18 11:27:09 -05:00
Justin Bollinger
cda9364071 fix: add accelerate to ml optional dependencies
Trainer from transformers requires accelerate>=1.1.0 at runtime.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-18 10:51:51 -05:00
Justin Bollinger
fcfe6890f6 feat: add memory pre-checks and optimize PassGPT training for large wordlists
Training previously loaded entire wordlists into RAM and tokenized all at
once, causing OOM on large files like rockyou.txt. This adds memory
estimation, lazy dataset loading, and training optimizations.

- Add _get_available_memory_mb() for cross-platform RAM detection
- Add _estimate_training_memory_mb() to predict peak usage before loading
- Replace bulk tokenization with LazyPasswordDataset (file offset index + on-the-fly tokenization)
- Add --max-lines flag to limit training to first N lines
- Add --memory-limit flag to auto-tune --max-lines based on available RAM
- Enable gradient checkpointing and gradient accumulation (steps=4)
- Enable fp16 on CUDA devices

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-18 10:47:44 -05:00
Justin Bollinger
56aaa9b47d feat: add PassGPT model fine-tuning and training menu integration
Add ability to fine-tune PassGPT models on custom password wordlists.
Models save locally to ~/.hate_crack/passgpt/ with no data uploaded to
HuggingFace (push_to_hub=False, HF_HUB_DISABLE_TELEMETRY=1). The
PassGPT menu now shows available models (default + local fine-tuned)
and a training option. Adds datasets to [ml] deps and passgptTrainingList
config key.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-18 09:51:06 -05:00
Justin Bollinger
4a7f0724d9 feat: add startup version check, fix PassGPT MPS/output issues, hide menu without ML deps
- Add optional startup version check against GitHub releases (check_for_updates config option)
- Add packaging dependency for version comparison
- Fix PassGPT OOM on MPS by capping batch size to 64 and setting memory watermark limits
- Fix PassGPT output having spaces between every character
- Hide PassGPT menu item (17) unless torch/transformers are installed
- Fix mypy errors in passgpt_generate.py with type: ignore comments
- Update README with version check docs, optional ML deps section, and PassGPT CLI options
- Add test_version_check.py with 8 tests covering update check behavior

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-18 09:32:40 -05:00
Justin Bollinger
87535b9828 feat: add PassGPT attack (#17) - GPT-2 based ML password generator
Add PassGPT as attack mode 17, using a GPT-2 model trained on leaked
password datasets to generate candidate passwords. The generator pipes
candidates to hashcat via stdin, matching the existing OMEN pipe pattern.

- Add standalone generator module (python -m hate_crack.passgpt_generate)
- Add [ml] optional dependency group (torch, transformers)
- Add config keys: passgptModel, passgptMaxCandidates, passgptBatchSize
- Wire up menu entries in main.py, attacks.py, and hate_crack.py
- Auto-detect GPU (CUDA/MPS) with CPU fallback
- Add unit tests for pipe construction, handler, and ML deps check

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-18 08:41:22 -05:00
Justin Bollinger
ae47d453c0 docs: add OMEN attack documentation to README
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-17 15:44:31 -05:00
Justin Bollinger
169d945546 Merge pull request #75 from trustedsec/feat/omen-attack
feat: add OMEN attack as menu option 16
2026-02-17 15:38:35 -05:00
Justin Bollinger
0c27b8fb28 chore: add generated file patterns to .gitignore
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-17 15:36:33 -05:00
Justin Bollinger
a53ed293b6 fix: add types-requests dev dependency to resolve mypy import-untyped error
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-17 15:21:07 -05:00
Justin Bollinger
79e3c0f6e9 fix: detect lowercase makefile in submodule build loop
OMEN uses lowercase `makefile` which was missed by the capital-M check.
On case-sensitive filesystems (Linux) this meant OMEN wouldn't be built
during `make submodules`.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-17 14:04:11 -05:00
Justin Bollinger
61ef838673 fix: bail out early in omen_attack handler when binaries are missing
The handler was prompting for training source and max candidates even
when the OMEN binaries weren't built, leading to confusing error output.
Now checks for createNG and enumNG up front and returns with a build
instruction if either is missing.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-17 14:03:39 -05:00
Justin Bollinger
0991701024 feat: add OMEN attack as menu option 16
Add OMEN (Ordered Markov ENumerator) as a probability-ordered password
candidate generator. Trains n-gram models on leaked passwords via
createNG, then pipes candidates from enumNG into hashcat.

Also fix a pre-existing bug where ensure_binary() used quit(1) instead
of sys.exit(1) - quit() closes stdin before raising SystemExit, which
caused "ValueError: I/O operation on closed file" when any optional
binary check failed and the program continued to use input().

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-17 14:01:58 -05:00
Justin Bollinger
5c3eee9f04 Merge pull request #74 from trustedsec/feat/netntlm-computer-filter
feat: NetNTLM computer account filtering
2026-02-17 13:24:02 -05:00
Justin Bollinger
97997daf15 feat: add computer account filtering for NetNTLM hash types (5500/5600)
Reuses existing _count_computer_accounts() and _filter_computer_accounts()
to optionally strip computer accounts before NetNTLM deduplication.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-17 13:23:36 -05:00
Justin Bollinger
4ae7a2b94e test: add E2E preprocessing flow tests for computer account filtering
Add TestE2EPreprocessingFlow class that simulates the exact main()
preprocessing logic (format detection, filtering, NT/LM extraction)
with realistic secretsdump.py output. Covers: filter accept/decline,
no computers, all computers, LM hash detection, domain\computer$
format.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-17 13:10:38 -05:00
Justin Bollinger
5b754e7f2f Merge pull request #73 from trustedsec/feat/ntlm-preprocessing
Add NTLM computer account filtering and NetNTLM dedup
2026-02-17 13:00:33 -05:00
Justin Bollinger
73bb6cf596 fix: QA hardening for NTLM preprocessing
- Catch PermissionError/OSError in file operations (not just FileNotFoundError)
- Refactor _dedup_netntlm_by_username to two-pass streaming (memory safe)
- Handle CRLF line endings in filter and dedup functions
- Add KeyboardInterrupt handling with temp file cleanup during preprocessing
- Track .filtered/.dedup temp files for cleanup on interruption
- Add CRLF line ending tests for both filter and dedup

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-17 12:59:09 -05:00
Justin Bollinger
26cd21af16 test: add comprehensive pwdump filter pipeline and edge case tests
Add TestWriteFieldSortedUnique (7 tests) and TestPwdumpFilterPipeline
(8 tests) covering the full filter -> extract NT/LM pipeline. Add
edge case tests for unicode, BOM, long lines, permissions, and
delimiter handling.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-17 12:53:20 -05:00
Justin Bollinger
53d42fbe96 fix: address QA review issues for NTLM preprocessing (#27, #28)
- Add type hints to _filter_computer_accounts and _dedup_netntlm_by_username
- Fix unclosed file handle when reading hash file for format detection
- Extract _count_computer_accounts helper to eliminate duplicate file reads
- Stream _filter_computer_accounts output instead of collecting in memory
- Only write .dedup file when duplicates actually exist
- Add tests for _count_computer_accounts, malformed lines, and no-file-on-zero-dupes

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-17 12:46:48 -05:00
Justin Bollinger
e417e3d928 feat: add computer account filtering and NetNTLM dedup
- Detect and optionally filter Windows computer accounts (username
  ending with $) from pwdump-format NTLM hash files (type 1000)
- Detect and optionally deduplicate NetNTLMv1/v2 hashes (types
  5500/5600) by username, keeping first occurrence
- Add 10 tests covering both features

Fixes #27
Fixes #28

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-17 12:43:22 -05:00
Justin Bollinger
dabdcbe579 fix: remove unnecessary submodule checkout from CI workflows
Tests use HATE_CRACK_SKIP_INIT=1 and don't need submodules.
Shallow submodule clones caused flaky CI failures on PR events.

Fixes #72

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-17 12:42:42 -05:00
Justin Bollinger
75a9986e2d Merge pull request #71 from trustedsec/fix/70-hashview-download-path
Fix Hashview wordlist download saving to cwd instead of configured directory
2026-02-17 11:20:21 -05:00
Justin Bollinger
ea4cb6c0fa docs: update README Ollama defaults to match config
Update ollamaModel from qwen2.5 to mistral and ollamaNumCtx from
8192 to 2048 to reflect recent config changes.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-17 11:13:47 -05:00
Justin Bollinger
7a56c7f506 fix: resolve Hashview wordlist downloads to configured directory
download_wordlist() now resolves relative filenames against
get_hcat_wordlists_dir() instead of saving to cwd. Also ensures
the parent directory exists before writing.

Fixes #70

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-17 11:05:23 -05:00
Justin Bollinger
9d91c3b42a Add openpyxl to project dependencies
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-14 13:36:12 -05:00
Justin Bollinger
467976fe57 update to ollama_benchmark.py 2026-02-14 13:27:23 -05:00
Justin Bollinger
a53420532f feat: change default Ollama model to mistral and context to 2048
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-14 10:06:45 -05:00
Justin Bollinger
b6347d5012 feat: default to mistral model only in benchmark
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-14 10:00:21 -05:00
Justin Bollinger
b2eb19c130 feat: update LLM attack prompt to CTF-style framing
Replace the denylist-oriented prompt with a capture-the-flag scenario
prompt that focuses on recovering passwords using industry terms and
company name permutations.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-14 09:36:45 -05:00
Justin Bollinger
9652c26986 feat: add --stdout flag to toggle printing model responses
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-14 09:27:05 -05:00
Justin Bollinger
30a6e8c6a7 feat: include candidates and response in benchmark JSON output
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-14 09:25:48 -05:00
Justin Bollinger
ee914a6822 Merge pull request #69 from trustedsec/ollama
Add LLM Attack via Ollama integration
2026-02-13 22:31:09 -05:00
Justin Bollinger
4c5c954eb1 fix: add --force --reinstall to uv tool install and add make update target
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-13 22:30:38 -05:00
Justin Bollinger
80857d03c6 fix: add missing rulesDirectory mock in ollama 404 retry test
The test built its own mock context instead of using the shared
ollama_globals helper, missing the rulesDirectory and hcatPotfilePath
patches. This caused FileNotFoundError on CI where /path/to/hashcat/rules
doesn't exist.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-13 22:25:48 -05:00
Justin Bollinger
a96868df71 docs: update README for LLM attack, Ollama config, and accuracy fixes
Add LLM Attack (option 15) to menu listing and attack descriptions,
document Ollama config keys, fix broken code block, update pre-push
hook example, add make update target, and correct CI Python versions.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-13 22:20:19 -05:00
Justin Bollinger
abe8f2ae73 fix: resolve CI test failures in ollama and hashview tests
- Mock rulesDirectory in ollama test fixture so hcatOllama doesn't
  fail with FileNotFoundError on CI where /path/to/hashcat/rules
  doesn't exist
- Mock potfile path in hashview auto-merge test so found file cleanup
  isn't blocked by missing ~/.hashcat directory
- Update pre-push hook to match CI env vars (HATE_CRACK_SKIP_INIT=1)

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
v2.0
2026-02-13 22:15:48 -05:00
Justin Bollinger
b56a91fef1 fix: clean version string to show 2.0+g<hash> format
Strip .postN.devN and .dYYYYMMDD suffixes from setuptools-scm version
in __init__.py so the banner displays a clean version like 2.0+g05b5d6dc7.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-13 22:11:31 -05:00