AGENTS.md
Guidance for coding agents working on the Insane Search validation and safety role.
Project Overview
Validation decides whether a response is usable evidence or only another challenge page. Safety decides whether a target or redirect destination may be fetched at all. Together they prevent two expensive mistakes: declaring blocked pages successful and letting attacker-controlled redirects reach private infrastructure.
Project Map
- skills/insane-search/engine/validators.py defines Verdict, ValidationResult, marker checks, JSON handling, selector proof, byte-size heuristics, and validate().
- skills/insane-search/engine/safety.py defines URL classification, private-address blocking, redirect resolution, and redirect detection.
- skills/insane-search/engine/fetch_chain.py uses validate() after every curl attempt and safety through transport.
- skills/insane-search/engine/transport.py validates initial URLs and redirect hops before requests.
- skills/insane-search/engine/tests/test_smoke.py and test_u1.py cover validation behavior.
- skills/insane-search/engine/tests/test_u7.py covers URL and redirect safety.
Setup Commands
bashcd skills/insane-search
python3 engine/tests/test_smoke.py
python3 engine/tests/test_u1.py
python3 engine/tests/test_u7.py
Change Rules For Agents
- Keep hard challenge markers decisive.
- Keep soft markers overridable by positive selector proof.
- Keep small non-empty JSON responses classified as usable API hits.
- Keep empty JSON and unresolved sensor cookies non-terminal when ambiguity remains.
- Compare body size in bytes, not character count.
- Keep rate limits transient; do not treat them as permanent walls.
- Keep auth and not-found verdicts terminal for the curl grid.
- Validate every redirect destination before following it.
Verification
bashcd skills/insane-search
python3 engine/tests/test_smoke.py
python3 engine/tests/test_u1.py
python3 engine/tests/test_u7.py
python3 engine/bias_check.py
Add tests for every new verdict branch. A validation change is incomplete unless it proves both the false-positive and false-negative side of the rule.
Security Notes
Safety defaults must block private, loopback, link-local, metadata, reserved, multicast, and unspecified targets. Private testing should require explicit opt-in and must never be enabled in docs, defaults, or examples.
PR Checklist
- New markers are generic, not site-specific.
- JSON handling still happens before tiny-body rejection.
- Selector proof still outranks soft text markers.
- Redirect safety is unchanged or strengthened.
- test_u1.py and test_u7.py pass.