AGENTS.md
Guidance for coding agents working on the Insane Search WAF detection role.
Project Overview
WAF detection converts a live response into a ranked list of profile candidates. It should never produce a single brittle verdict and should never inspect hostnames. The planner consumes profile rankings to choose TLS impersonation groups, referer strategies, URL transforms, known bad sizes, and fallback routes.
Project Map
- skills/insane-search/engine/waf_detector.py loads profiles, scores cookie/header/server/body signals, returns DetectionHit objects, and exposes last_load_error().
- skills/insane-search/engine/waf_profiles.yaml stores profile definitions and planner hints.
- skills/insane-search/engine/fetch_chain.py calls detect() after the probe and passes ranked hits to _build_plan().
- skills/insane-search/engine/validators.py supplies verdicts that decide whether detection is needed.
- skills/insane-search/engine/tests/test_smoke.py checks profile loading.
- skills/insane-search/engine/bias_check.py enforces the no-site-name rule.
Setup Commands
bashcd skills/insane-search
python3 engine/tests/test_smoke.py
python3 engine/bias_check.py
python3 -m engine "<URL>" --trace --json
Change Rules For Agents
- Match WAF-vendor artifacts only: cookies, headers, server substrings, and challenge body markers.
- Do not match or branch on customer domains, content brands, or site-specific selectors.
- Keep detect() returning a ranked list; callers need fallback candidates when detection is uncertain.
- Keep unknown_challenge available as the conservative fallback.
- Surface profile loader errors through trace diagnostics rather than raising from normal fetch().
- Treat missing PyYAML, missing profile files, and invalid profile shapes as degraded mode, not total failure.
- Keep confidence thresholds simple and testable.
- Update waf_profiles.yaml and detector tests together.
Verification
bashcd skills/insane-search
python3 engine/tests/test_smoke.py
python3 engine/tests/test_u1.py
python3 engine/bias_check.py
For a new profile, test that the detector ranks it above unknown_challenge when its generic signals are present and that fetch_chain still tries alternate profile candidates.
Security Notes
Do not publish, store, or hardcode site-private indicators. WAF profiles can describe product artifacts, not target identities or user-specific bypass data.
PR Checklist
- Profile loader degrades safely.
- unknown_challenge remains usable.
- Detector does not inspect hostnames.
- Profile hints are generic.
- Bias check passes.