Adds Rule File Tools submenu (menu option 81) with three operations:
- Clean: removes invalid/duplicate rules via cleanup-rules.bin
- Optimize: consolidates redundant operations via rules_optimize.bin
- Clean and optimize: both in sequence with temp file handling
Wired through the standard three-layer pattern: main.py utility
functions + dispatcher, attacks.py handlers + submenu, root
hate_crack.py menu registration.
hashcat's Lzma2Decode() expects a raw LZMA2 stream starting with 0xff chunk
headers, not an XZ container format (0xfd 37 7a...). FORMAT_RAW with
FILTER_LZMA2 produces the correct format.
Hashcat requires hcstat2 files to be LZMA2-compressed. The hcstat2gen.bin utility outputs uncompressed binary data, so we need to compress the output after generation.
Root cause: Hashcat's source code (mpsp.c) uses hc_lzma2_decompress() to read hcstat2 files. The bundled hashcat.hcstat2 file uses LZMA2 compression (XZ format). hcstat2gen.bin outputs plain hcstat format which hashcat cannot decompress, causing "Could not uncompress data" error.
Fix: After hcstat2gen.bin completes, compress the output file using Python's lzma module with XZ format (LZMA2-based).
Also restore proper subprocess invocation: pass output path as argument to hcstat2gen.bin instead of using stdout redirection.
Tests: All 8 markov E2E tests pass, 479 total tests pass
The previous commit attempted to simplify by passing hcstat2_path as an argument and removing stdout redirection. However, hcstat2gen.bin writes directly to the file specified as its first argument and doesn't use stdout. The code was missing the proper invocation.
Root cause: Removed stdout=PIPE capture but forgot that the binary doesn't write to stdout - it writes directly to the file argument. This left the .hcstat2 file empty/uncreated.
Fix: Restore proper subprocess invocation - pass output path as argument, stdin from wordlist, stderr for error reporting. Remove stdout=PIPE since the binary doesn't use it.
Tests: All 8 markov E2E tests pass, 479 total tests pass
- Remove unnecessary gzip decompression logic
- hcstat2gen.bin can read both plain text and gzipped files directly
- Simplified code: just pass source file and output path as arguments
- Removed gzip and tempfile imports (no longer needed)
- Add comprehensive end-to-end tests covering all markov attack flows
- Tests verify training with plain text and gzipped wordlists
- Tests verify all handler menu paths (use existing, generate new, cancel, training failure)
- Fixes 'cannot uncompress data' error by not trying to decompress
- hcstat2gen.bin expects: hcstat2gen.bin <outfile> < dictionary
- Previously tried to write to stdout which caused 'usage' error
- Now correctly passes output path as command argument
- Markov training now works for both plain text and gzipped wordlists
- Fixes 'Markov table generation failed' error
- Check hcstat2gen.bin exists before running
- Verify source file is readable
- Capture stdout and stderr from hcstat2gen.bin
- Report hcstat2gen.bin errors with stderr output
- Verify output file created and non-empty
- Add 300 second timeout to prevent hanging
- Write proper output to .hcstat2 file from stdout
- Helps debug training failures with specific error messages
- Add tempfile import for secure temporary file handling
- Detect gzipped files by magic bytes (0x1f 0x8b)
- Decompress gzipped files to temporary file first
- Pass uncompressed data directly to hcstat2gen.bin
- Properly clean up temporary files on success or error
- Fixes 'hcstat2: Could not uncompress data' error
- Add gzip import to main.py
- Detect gzipped files by magic bytes (0x1f 0x8b)
- Use gzip.open() for compressed files, regular open() for plain text
- Properly close file handles in all cases
- Allows markov training from both .txt and .txt.gz wordlists
- Update menu listing to show combinator submenu and new attacks (17, 18)
- Remove keys 10/11/12 from main menu (consolidated into submenu)
- Add detailed descriptions for Combinator Attacks submenu, Ad-hoc Mask Attack, and Markov Brute Force Attack
- Document Markov training source selection and table persistence
- Update version history with new feature details
- Update test cases to reflect combinator_submenu for key 6
- Remove test cases for keys 10/11/12 (moved to sub-menu)
- Add test cases for new keys 17 and 18
- Simplify adhoc_mask tests to avoid global state issues
- Add three hashcat wrapper functions: hcatAdHocMask, hcatMarkovTrain, hcatMarkovBruteForce
- Add corresponding attack handlers in attacks.py with OMEN-style training flow
- Consolidate 4 combinator attacks (keys 10/11/12) into interactive sub-menu (key 6)
- Add key 17 for ad-hoc mask attack and key 18 for markov brute force
- Update both main.py and hate_crack.py menu systems
- Add comprehensive test coverage for new handlers and wrappers
- Training source picker supports cracked passwords or any wordlist
Extract _select_rules() helper from quick_crack/loopback_attack and wire
it into omen_attack so OMEN can run with rule chains. Extend hcatOmen()
to accept and apply an hcatChains argument including debug mode injection.
Fix resolve_path() to honour HATE_CRACK_ORIG_CWD (set by the install shim)
so relative hash/wordlist paths resolve against the caller's working directory
instead of the repo root. Increase default omenMaxCandidates to 50M.
- Add _omen_model_is_valid() checking all 5 required model files
- Add _omen_model_info() reading model_info.json metadata
- hcatOmenTrain returns bool and writes training metadata
- Capture enumNG stderr and report errors instead of silent failure
- Rewrite omen_attack with train/use/cancel menu and wordlist picker
- Rewrite list_and_download_hashmob_rules with ThreadPoolExecutor
- Update quick_crack to use list_wordlist_files
- Add parallel download tests with stdin TTY mock
Config and assets were not found when running hate_crack from outside the
repo directory. The shim uses `uv run --directory` to always execute from
the repo root, and _candidate_roots() now includes _repo_root and
_package_path as fallback search locations.
api.py's get_rules_dir() independently re-parsed config.json and could
resolve a different rules path than main.py's rulesDirectory global.
Thread the already-resolved rulesDirectory through download_hashmob_rules
and list_and_download_hashmob_rules so all callers (menu option 90,
--rules flag, quick_crack, loopback_attack) use the same path from config.
A blank first line caused the format detection regex chain to fall
through to the error exit. Read lines in a loop and skip empty ones
before matching. Add tests for blank lines with LF, CRLF, whitespace,
and BOM variations.
Use utf-8-sig encoding to natively strip BOM and remove null bytes from
UTF-16 artifacts so the bare hash regex matches correctly. Replace the
unhelpful "unknown format" error with a diagnostic message showing the
actual first-line content and expected formats.
When running as root, ~/.hashcat/ exists but hashcat.potfile does not.
Hashcat refuses to create the file when given an explicit --potfile-path.
Create the parent directory and touch the file in _append_potfile_arg()
before appending the flag.
- Remove vendor-assets and clean-vendor Makefile targets
- Lima VM test and Dockerfile.test now use make install (editable)
- Remove hate_crack/hashcat-utils from submodules-pre expander generation
- Update README to reflect no-vendoring install
When hashcat is not installed via the standard method, ~/.hashcat/
does not exist. Fall back to hashcat.potfile in the current working
directory instead of referencing a path that does not exist.
The uv installer adds ~/.local/bin/uv but does not update the current
shell PATH. Use the discovered path or fall back to ~/.local/bin/uv
so the subsequent uv tool install succeeds on first run.
Config keys added after initial release (ollamaModel, ollamaNumCtx,
omenTrainingList, omenMaxCandidates, check_for_updates) raised KeyError
when config.json.example was not found for auto-injection.
- Add JSON parsing error handling with user-friendly messages for
malformed config.json (shows line/column and fix instructions)
- Handle corrupt config.json.example with package reinstall guidance
- Consolidate auto-inject messaging to single summary line instead
of one message per missing key
- Remove ~20 redundant KeyError handlers since auto-inject ensures
all keys exist before they're read
Install a custom SIGINT handler that tracks interrupt timing. A single
Ctrl+C raises KeyboardInterrupt as before (kills the current subprocess
and continues). A second Ctrl+C within 2 seconds raises DoubleInterrupt,
which bypasses all existing per-subprocess handlers and is caught at the
main menu loop, printing "[!] Returning to main menu..." and resuming
the menu.
Preprocessing and cleanup sections also catch DoubleInterrupt to ensure
temp file cleanup runs before re-raising.
In dev checkouts where submodules are built in the repo root rather than
vendored into hate_path, OMEN binaries were not found. Introduces _omen_dir
that checks hate_path/omen first and falls back to the repo root omen dir.
Also removes vendor-assets from install/update targets and drops vendored
submodule paths from pyproject.toml package data.
When the Hashview server returns HTTP 200 with an error message and no
job_id (due to its internal notify_email bug), the CLI and interactive
paths now:
- exit 1 (not 0) in the CLI path
- print "✗ Error" instead of "✓ Success"
- print a hint to check the Hashview UI before retrying, preventing
duplicate job creation
Adds test for the error response path in test_cli_flags.py.
notify_email is not supported by all Hashview servers and caused job
creation to fail. Changed default to None so it is omitted from the
request unless explicitly passed.
Also reworded the file format override prompt so the detected name does
not appear twice in the output.
hashcat is now a system dependency (installed via apt/brew or PATH) rather
than a compiled submodule. The Lima VM provision script installs it via apt,
and local installs are expected to have hashcat available in PATH.
- Remove hashcat entry from .gitmodules
- Remove hashcat/ submodule checkout
- Remove hashcat submodule skip logic from Makefile submodules target
- Simplify submodules-pre hashcat check to PATH-only
- Update vendor-assets and clean targets to remove submodule references
- Update README: hashcat is now a required prerequisite, not optional
- Document Lima VM E2E prerequisites (lima, rsync) and list all packages
provisioned automatically by the test VM
- Add rsync to _require_lima() prerequisite check; missing rsync now
skips cleanly instead of failing with an opaque command-not-found
- Add _truncate_output() helper and apply to all assertion messages to
keep failure output readable when make/install emits thousands of lines
- Increase limactl start timeout from 300s to 600s to accommodate slow
Ubuntu image downloads
- Add limactl stop before delete in cleanup for more reliable teardown
- Add flag verification to test_lima_vm_install_and_run: checks 10 CLI
flags in --help output, matching the local install test pattern
- Add 3 unit tests: test_truncate_output_trims_long_text,
test_truncate_output_short_text_unchanged,
test_require_lima_skips_without_rsync
OpenCL/device build errors are environment-specific issues, not code
bugs. Detect clCreateProgramWithBinary and kernel build failures in
stderr and pytest.skip instead of pytest.fail.
Quality checks run locally via prek pre-push hooks. CI workflows
(ruff, ty, pytest, lint-infra, version-bump) are no longer needed.
Updated docs to remove all GitHub Actions references.
Justin Bollinger