AGENTS.md

Guidance for coding agents working on Codex Token Snapshot Deduplication in the codexTokencount project.

Project overview

This guide covers duplicate snapshot removal in src/parser.ts. Codex token_count events may include total_token_usage snapshots. The store uses the current session id plus total token count as a snapshot key so copied session files or repeated snapshots do not inflate request totals.

Primary source files

• src/parser.ts owns seenSnapshots, duplicatesRemoved, rawTokens, parseTotalUsage, and ingest.
• src/parser.test.ts includes repeated snapshot and copied session fixture coverage.
• media/dashboard.js renders deduped snapshot and raw token summary cards.

Setup commands

npm install npm test npm run compile npm run check-types npm run package npx esbuild src/selftest.ts --bundle --platform=node --format=cjs --outfile=out/selftest.js node out/selftest.js <sessions-dir>

Change rules

• rawTokens should include scanned token totals before snapshot dedupe.
• Snapshot dedupe should only trigger when total_token_usage exists and has a positive total.
• The snapshot key must include session identity so equal totals from different sessions are not collapsed accidentally.
• last_token_usage remains the request payload that contributes to buckets after dedupe passes.
• duplicatesRemoved should count skipped duplicate snapshots, not skipped malformed lines.

Testing

Cover repeated snapshots in one file, copied session files, events without total_token_usage, zero totals, and different sessions with equal totals.

PR checklist

• Deduped requests and raw token counters remain distinct.
• Dashboard copy does not imply official billing.
• npm test, npm run compile, and npm run check-types pass.