AlexandrAI AGENTS.md
agents-md

Insane Search Session Pool Transport Agent Guide

Project-specific AGENTS.md guidance for the Insane Search transport layer, per-host curl session reuse, root warmup, browser cookie injection, redirect safety, and pool diagnostics.

AGENTS.md

Guidance for coding agents working on the Insane Search session pool and transport role.

Project Overview

The transport layer turns repeated retrieval attempts into a stateful, safety-checked sequence. It reuses curl_cffi sessions by host and impersonation, warms the origin root before deep requests, accepts browser-cleared cookies from fallback execution, and validates every redirect hop before following it.

Project Map

  • skills/insane-search/engine/transport.py defines SessionPool, POOL, warmup(), inject_cookies(), request(), _fetch_following(), stats(), reset(), and pool_enabled().
  • skills/insane-search/engine/safety.py classifies target URLs and redirect destinations before network access.
  • skills/insane-search/engine/executor.py calls the cookie bridge after browser fallback.
  • skills/insane-search/engine/fetch_chain.py uses POOL.request() for every curl probe and grid attempt.
  • skills/insane-search/engine/tests/test_u4.py covers session reuse, cookie injection, envelope parsing, warmup, and fetch_many reuse.
  • skills/insane-search/engine/tests/test_u7.py covers SSRF and redirect guards.

Setup Commands

bashcd skills/insane-search
python3 engine/tests/test_u4.py
python3 engine/tests/test_u7.py
python3 -m engine "<URL>" --trace --json

Change Rules For Agents

  • Keep pool keys scoped to host and impersonation identity.
  • Do not persist cookies in source, docs, fixtures, or traces.
  • Keep root warmup idempotent for each session entry.
  • Preserve one-shot fallback behavior when curl_cffi Session cannot be created.
  • Validate the initial URL and every redirect hop before any follow-up request.
  • Leave pool_enabled() as the switch for one-shot mode.
  • Keep browser cookie injection best-effort; failure must not break normal fetching.
  • Avoid adding domain-specific pool rules or hostname allowlists.

Verification

bashcd skills/insane-search
python3 engine/tests/test_u4.py
python3 engine/tests/test_u7.py
python3 engine/bias_check.py

When changing redirect behavior, add a test that covers both a safe redirect and a blocked internal redirect. When changing session behavior, assert reuse through pool statistics or request counts rather than timing.

Security Notes

This layer is the SSRF boundary. Never weaken the default-deny posture for private, loopback, link-local, metadata, reserved, multicast, or unspecified destinations unless the caller explicitly opts into private testing.

PR Checklist

  • Redirect validation still checks each hop.
  • Browser cookies are never printed.
  • Pool reset closes sessions.
  • One-shot fallback still works without Session support.
  • test_u4.py, test_u7.py, and bias_check.py pass.