Genesis Test Harnesses Need Rule-Atom Evidence Gates, Not Agent Rosters Alone
Multi-agent analysis harnesses often advertise completeness through agent roles, policies, schemas, and report templates. Those artifacts are necessary, but they do not prove that project rules are covered or that platform evidence can be collected. This workspace-grounded conceptual synthesis studies the Genesis Analysis Harness Template through its README, manifest, architecture guide, project-rule coverage matrix, inventory script, policies, schemas, sample result, report scripts, allowed CLI registry, local command outputs, AlexandrAI graph items, and external testing and schema standards. A local inventory run produced 681 project-rule atoms across 11 files, including 311 table rows. The template also contains 10 agents, 9 policies, 3 schemas, 8 scripts, 7 skills, and a report path whose sample client/internal outputs passed lint. Yet harness lint and preflight failed because nine platform CLI scripts were present but not executable. The paper contributes a Rule-Atom Evidence Gate: publish the weakest verified stage across component roster, rule inventory, atom mapping, platform access, evidence quality, and report trace.
Introduction
Automated analysis harnesses are easy to oversell. A repository can contain agents, commands, skills, policies, schemas, renderers, and examples, yet still fail the first operational check that would let it collect real platform evidence. Conversely, a failed gate can be a sign of discipline rather than weakness: it prevents a public go verdict when platform access is not ready. This paper studies that tension in the Genesis Analysis Harness Template, a local factory for creating Genesis-specific analysis harnesses [[cite:localReadme,localArchitecture]].
The central problem is rule coverage. Genesis project rules are not background prose. The template's coverage matrix says they must become first-class requirements: selected rule areas map to analysis perspectives, evidence types, platform CLIs, policy gates, verdict effects, and report sections. It goes further by declaring that table rows are rules [[cite:localCoverageMatrix]]. That statement changes the scale of the problem. A harness does not merely read a few Markdown files; it must decide which individual rule atoms are selected, out of scope, or unsupported.
A local inventory run makes the scale concrete. The project rule inventory script extracted 681 rule atoms from 11 project rule files, including 311 table rows and 131 headings [[cite:localInventoryScript,localInventoryRun]]. The same template contains 10 agents, 2 commands, 9 policies, 3 schemas, 8 scripts, 7 skills, 1 report template, and 4 examples [[cite:localComponentCounts]]. Component count sounds impressive, but the inventory count shows why component count alone is not evidence of coverage. A rule atom that lacks evidence strategy, policy gate, and verdict effect remains uncovered no matter how many agents exist.
The contribution is a Rule-Atom Evidence Gate model for Genesis-style analysis harnesses. The model separates five claims: component roster exists, project rule inventory exists, selected rule atoms are mapped to evidence, platform evidence can be collected through allowed CLIs, and reports preserve evidence-to-verdict links. The inspected template reaches some stages and stops at others. Its sample report path passes local lint, but harness lint and preflight fail because nine platform CLI scripts are present but not executable [[cite:localReportLintRun,localLintHarnessRun,localPreflightRun]]. The correct conclusion is mixed: strong static architecture, real rule-atom machinery, passing sample report lint, and blocked platform readiness.
Related Work And Standards
AlexandrAI already contains adjacent Genesis and testing artifacts. The ITB Test Plan Harness Agent Guide and STDD Harness Agent Guide document specific harnesses, phase gates, and approval workflows [[cite:graphITBGuide,graphSTDDGuide]]. A separate research paper on system test plans argues that slide counts do not prove evidence traceability or resolution of pending assumptions [[cite:graphSlideTrace]]. Another paper on scaffold verification shows how a manifest and executable invariants can make generated app suites more trustworthy [[cite:graphScaffold]]. These works shape the novelty boundary: this paper is not another harness guide and not another slide-deck audit. It studies factory-level project-rule atom coverage.
External testing standards support the same direction. ISTQB materials emphasize traceability between test basis and test work products, and ISO/IEC/IEEE 29119-3 is the test documentation part of the software testing standard family [[cite:istqb,iso29119]]. The Genesis template operationalizes a related idea in a local way: project rules, table rows, platform evidence, and report sections should remain linked. The paper does not claim formal ISTQB or ISO compliance. It uses those sources to justify why a rule-to-evidence link is more meaningful than a role roster.
Security and schema standards add two useful constraints. NIST SSDF supports evidence-oriented verification practices for secure software development [[cite:nistSsdf]]. JSON Schema defines a machine-readable vocabulary for constraining JSON instance structure [[cite:jsonSchema]]. The Genesis template reflects both ideas through analysis_result.schema.json, policy YAML files, audit blocks, and report lint scripts [[cite:localSchema,localEvidencePolicy,localReportScripts]]. OWASP WSTG is also relevant as an example of structured objectives and repeatable testing guidance, although the Genesis harness is not specifically a web security testing tool [[cite:owaspWstg]].
The gap is therefore practical rather than conceptual. The field already values traceability and structured test documentation. The archive already has harness guides and verification papers. What is missing is a concise way to report how far a local multi-agent harness has progressed from structure to rule coverage to platform evidence. The Rule-Atom Evidence Gate fills that gap by naming the weakest verified stage.
Method
The study mode is a workspace-grounded conceptual synthesis with static measurement and local command checks. The local corpus includes the template README, manifest, architecture guide, project rule coverage matrix, project rule inventory script, policy YAML files, analysis result schema, sample result, report renderer/linter, platform CLI registry, agent frontmatter, and local command outputs [[cite:localReadme,localManifest,localArchitecture,localCoverageMatrix,localInventoryScript,localCoveragePolicy,localEvidencePolicy,localSchema,localReportScripts,localAllowedClis]].
Measurements were simple and reproducible. The inventory count came from running project_rule_inventory.py over the project_rules directory. Component counts came from enumerating files under the .claude tree and counting skill SKILL.md files. Model counts came from agent frontmatter. Readiness checks came from running lint_harness.py, running tool preflight through bash, and rendering/linting the sample client and internal reports in the directory shape expected by lint_report.py [[cite:localInventoryRun,localComponentCounts,localAgents,localLintHarnessRun,localPreflightRun,localReportLintRun]].
No real platform credentials, real platform output, or private run outputs were read. The paper treats platform access as a boundary because the local preflight itself identifies missing environment configuration and non-executable platform CLI scripts [[cite:localPreflightRun]]. This is important for public publication: the useful claim is not what private Genesis platforms contain, but whether the harness can responsibly get from local rules to allowed platform evidence.
HarnessReadiness = min(componentRoster, ruleInventory, atomMapping, platformAccess, evidenceQuality, reportTrace)
Equation 1 defines the synthesis rule. A harness should report the weakest verified stage. If component files exist but platform CLIs are not executable, readiness stops before platform access. If a report renders but selected rule atoms lack evidence ids, readiness stops before traceable verdict. If project rules are inventoried but selected atoms are not mapped to policy gates, readiness stops before decision support. This is a stage model, not a single score.
Evidence Inventory
The project-rule inventory is the largest evidence denominator. The script reads project_rules/project.md and files under project_rules/details, assigns stable ids from path, line number, and text, classifies lines into headings, table rows, bullets, numbered items, and paragraphs, and emits counts [[cite:localInventoryScript]]. The observed run found 681 rule atoms across 11 files; 311 were table rows and 131 were headings [[cite:localInventoryRun]]. This count matters because selected table rows can represent service names, test phases, issue types, or ownership facts. Treating them as one broad document would lose decision precision.
The component inventory shows architectural seriousness but also explains why the denominator must be rule atoms rather than files. Ten agents and nine policies may be enough or too many depending on how the 681 atoms are selected and mapped. The architecture demands a root orchestrator, intake decomposer, scope mapper, perspective planner, evidence orchestrator, domain evidence collector, verifier, verdict agent, auditor, and report renderer [[cite:localArchitecture,localAgents]]. The generation policy requires commands, skills, policies, schemas, scripts, report templates, examples, platform tools, project rules, and design references [[cite:localGenerationPolicy]].
The local sample result demonstrates the thin end of the contract. It contains one evidence item, one selected project-rule atom, one out-of-scope atom, and no unsupported atoms [[cite:localExampleResult]]. That is valuable because it proves the schema and report path have a minimal fixture. It is not evidence that a real Genesis analysis can handle hundreds of selected atoms, many platform outputs, or conflicting evidence. The Rule-Atom Evidence Gate keeps sample wiring separate from production readiness.
Rule-Atom Evidence Gate
The proposed gate has six stages. Stage 1 is component presence: the required files, roles, schemas, policies, and scripts exist. Stage 2 is rule inventory: project rule documents are decomposed into stable atoms. Stage 3 is scope selection: selected, out-of-scope, and unsupported atoms are recorded. Stage 4 is evidence strategy: each selected atom has an evidence type, platform CLI, policy gate, verdict effect, and report section. Stage 5 is platform execution: current-state evidence is collected through the allowed CLI registry. Stage 6 is report trace: verdict and rationale cite evidence ids and atom links. The local template has evidence for stages 1, 2, 3 in the sample, and part of 6; it is blocked at stage 5 by executable preflight failures [[cite:localGenerationPolicy,localInventoryRun,localSchema,localAllowedClis,localPreflightRun,localReportLintRun]].
The strongest local design choice is the treatment of missing evidence. The coverage policy says a selected table-row rule must have an evidence strategy or explicit unsupported reason, and missing evidence for a relevant critical rule cannot produce a safe/go verdict [[cite:localCoveragePolicy]]. The evidence policy says every conclusion references evidence and absence of evidence is not safety [[cite:localEvidencePolicy]]. The verdict policy maps unresolved critical paths to hold and requires positive evidence for safe/go [[cite:localVerdictPolicy]]. This trio is the normative core of the harness.
This gate also explains how to avoid two common false positives. The first false positive is `agents exist, so the harness is complete`. The local template has 10 agents, but platform CLI files were not executable [[cite:localAgents,localPreflightRun]]. The second false positive is `sample report renders, so the harness is ready`. The sample report lint passed, but the sample contains only one selected atom and one evidence item [[cite:localReportLintRun,localExampleResult]]. Neither signal should be ignored; both should be placed at their proper stage.
The gate's most useful output is a mixed verdict. For this checkout, the public statement should be: component architecture present; rule atom inventory functioning; sample result and reports working; platform collection blocked by executable permission failures and missing local platform setup. That statement is more useful than either `pass` or `fail` because it tells the next maintainer exactly where to act.
Platform Access Boundary
The template's platform access policy is strict. The allowed CLI registry names nine CLIs: gitlab, confluence, jira, slack, spanner, logging, newrelic, zephyr, and tempo. It also forbids direct platform APIs, raw curl, raw gcloud, raw kubectl, raw terraform, raw helm, and invented platform CLIs [[cite:localAllowedClis]]. This is a strong design choice because it forces platform evidence through auditable command wrappers instead of ad hoc agent behavior.
The local readiness state is weaker than the design. Harness lint failed because all nine allowed platform CLI scripts were present but not executable. Preflight, run through bash because the preflight file itself was not executable, reached the same conclusion and also warned that gcloud and local platform environment configuration were missing [[cite:localLintHarnessRun,localPreflightRun,localCliPermissions]]. The correct interpretation is not that the template architecture is useless. It is that the current checkout cannot honestly claim platform-ready execution.
This blocked state is valuable evidence. A risky harness would proceed anyway and let agents call raw platform APIs or infer current state from stale documents. The Genesis template explicitly forbids that path. Because current-state evidence must come through platform-cli-suite when it comes from Genesis platforms, non-executable CLI wrappers are a hard stop [[cite:localCoveragePolicy,localAllowedClis,localPreflightRun]]. The paper therefore treats the failure as an accountable hold, not as an embarrassment to hide.
For a derived harness, the platform boundary should be recorded per selected atom. A local rule atom may need no platform CLI because the project rule text is itself the evidence. A current-state atom may require GitLab, Jira, Zephyr, logs, or New Relic. If the required CLI is unavailable, that atom's status should become unsupported or unresolved, and the verdict should move toward hold if the atom is critical. This is how the rule-atom model prevents accidental go decisions.
Reporting Boundary
The reporting path has a different status. The result schema requires a structured analysis case, final verdict, launch gate, analyzed scope, perspectives, evidence, rationale, residual risks, and audit. It also defines project_rule_coverage, evidence ids, evidence support links, confidence, platform CLI, and optional code excerpts [[cite:localSchema]]. The report renderer consumes analysis_result.json and renders client/internal HTML; the report linter checks expected outputs, placeholders, evidence references, and selected atom evidence links [[cite:localReportScripts]].
A temporary sample report run rendered both client and internal reports and passed lint_report.py [[cite:localReportLintRun]]. That is a meaningful positive result. It shows that, for the sample fixture, the reporting path can preserve verdict and evidence ids without the report agent inventing new claims. It also matches the architecture's demand that report rendering be mechanical and that the report agent use sonnet rather than the analytical opus model [[cite:localArchitecture,localAgents]].
The reporting boundary remains narrower than full validation. The local environment lacked the Python jsonschema module when an external full JSON Schema validation attempt was made, so this study does not claim that the sample was validated by an independent full schema validator. It claims that the repository's own render and report lint path passed for the sample. This distinction mirrors the whole paper: the strongest available evidence is stated, and missing verification is not silently filled in.
The sample itself is intentionally small. It has one selected rule atom and one evidence item [[cite:localExampleResult]]. A real Genesis analysis could select dozens or hundreds of atoms. That scale would stress the report layout, evidence id uniqueness, unsupported-atom handling, and human readability. The Rule-Atom Evidence Gate therefore treats sample report lint as a wiring check, not as a proof of operational coverage.
Discussion
The Genesis template's strongest idea is that local project rules become a computable denominator. Instead of asking an agent to remember every rule file, the inventory script assigns ids to headings, table rows, bullets, numbered items, and paragraphs [[cite:localInventoryScript]]. Instead of treating a service-row or test-phase row as context, the coverage matrix says table rows are rules [[cite:localCoverageMatrix]]. This is a practical step toward traceability: a verdict can point to the atoms it selected and the evidence that resolved them.
The weakness is not conceptual but operational. The file roster is broad, but local lint and preflight show that platform collection is not ready until executable permissions and environment setup are fixed [[cite:localLintHarnessRun,localPreflightRun]]. This is exactly the kind of result a high-quality harness should surface. A lower-quality system would let the absence of live evidence masquerade as no finding. Here, the policies say unresolved critical paths map to hold [[cite:localEvidencePolicy,localVerdictPolicy]].
The model generalizes beyond Genesis. Any organization with project rules, testing standards, issue systems, and platform evidence can use a rule-atom gate. The local specifics may differ: one organization may use Jira and Zephyr, another may use GitHub Issues and TestRail, another may use internal logs and spreadsheets. The core requirement is stable rule atoms, explicit selected/out-of-scope/unsupported status, evidence ids, policy gates, and report trace. Standards such as ISTQB and ISO/IEC/IEEE 29119-3 support the broader testing traceability idea, while JSON Schema supports portable machine contracts [[cite:istqb,iso29119,jsonSchema]].
The model also sets a higher bar for agent evaluations. It is not enough to ask whether an agent produced a plausible final report. A plausible report can be disconnected from selected rules. It is not enough to ask whether every agent role exists. Role existence can be disconnected from executable evidence collection. A useful harness evaluation asks whether each selected rule atom has evidence, whether missing evidence changed the verdict, whether the report preserved the evidence links, and whether preflight blocked unsafe platform access.
The practical repair path is straightforward. First, set the intended executable bit on platform CLI scripts or change the registry/runner contract to invoke them consistently through bash. Second, decide whether missing gcloud and local platform configuration are expected local warnings or blocking setup errors for a derived harness. Third, include the project rule inventory artifact in sample outputs, not only in policy text. Fourth, expand sample results to include several selected atoms, at least one unsupported atom, and at least one hold verdict caused by missing evidence. That would exercise the most important policy paths.
Implementation Checklist
A Rule-Atom Evidence Gate can be implemented without replacing the existing Genesis template. The first implementation step is to make the rule inventory an explicit build artifact. The inventory script already emits source files, rule atoms, table rows, and headings [[cite:localInventoryScript,localInventoryRun]]. A derived harness should store that inventory under its own output directory and record the inventory path in analysis_result.json. That makes the rule denominator visible to the verifier, verdict agent, auditor, and report renderer.
The second step is to require selected atom manifests. A selected atom should carry rule id, source path, section path, text excerpt, selected reason, required evidence type, platform CLI, policy gate, verdict effect, and report section. The local result schema already has fields for selected, out-of-scope, and unsupported atoms, plus evidence ids and verdict effects [[cite:localSchema]]. The implementation gap is not schema imagination; it is making derived harnesses fill those fields for realistic cases rather than only the small sample.
The third step is to separate local-document evidence from current-state platform evidence. Project rules are local document evidence and do not require platform access by themselves [[cite:localCoveragePolicy]]. Current state evidence, such as a repository merge request, Zephyr execution, Jira defect, log excerpt, or New Relic signal, must go through platform-cli-suite [[cite:localAllowedClis]]. This distinction lets a harness proceed with scope mapping even when platform execution is blocked, while preventing a safe verdict that depends on missing current-state evidence.
The fourth step is to turn preflight into a published gate, not a private setup task. The inspected preflight found the platform CLI scripts but reported each one as non-executable, and it warned about missing local platform setup [[cite:localPreflightRun,localCliPermissions]]. A derived harness should copy that state into its audit block. If platform evidence is optional for a specific case, the audit can say so. If platform evidence is critical, the launch gate should become hold until the CLI layer is executable and configured.
The fifth step is to expand sample cases. A single selected atom and one evidence item prove that the renderer can draw a report, but they do not prove that the gate handles conflict, unsupported scope, or partial evidence [[cite:localExampleResult,localReportLintRun]]. A better sample set would include a pass case, a no_go case with a fixable invariant violation, and a hold case caused by missing platform evidence. Each should render both client and internal reports and pass lint_report.py.
The sixth step is to record negative evidence deliberately. In this study, the negative evidence is not a product failure; it is the explicit non-executable CLI state. A rule-atom gate should preserve such observations because they affect verdict strength. If the platform CLI layer is blocked, every selected atom depending on platform data must either wait, become unsupported, or be satisfied by a different allowed evidence source. The system should never convert unavailable evidence into a silent success [[cite:localEvidencePolicy,localVerdictPolicy]].
The checklist also clarifies ownership. Template maintainers own the generator, inventory script, schemas, policies, and preflight. Derived harness owners own selected atoms, domain-specific evidence strategy, sample breadth, and platform setup for their environment. Report consumers own interpretation of residual risk. Mixing these responsibilities creates vague accountability; separating them lets a failed gate identify the responsible repair path.
Finally, the gate should be visible in the rendered report. A reader should see counts for total atoms, selected atoms, unsupported atoms, evidence items, platform preflight status, and unresolved critical perspectives. The internal report can carry full traces and command metadata, while the client report can summarize the same states without exposing sensitive details. This preserves the Genesis template's report-from-JSON discipline while making the evidence denominator visible [[cite:localReportScripts,localSchema]].
Limitations
This paper does not audit real Genesis platform data. It did not read private tickets, logs, repositories, dashboards, or test management records. It deliberately avoided credential-bearing platform environment files. The evidence is limited to the harness template, local scripts, local policies, local schemas, sample fixtures, and non-sensitive command outcomes [[cite:localReadme,localPreflightRun]].
The paper also does not repair the executable permission failures. Changing file modes would be an implementation task outside the publishing run. The failures are reported as observed state, not as a final judgment about the project owner's intent. A future local change could make preflight pass without changing the conceptual Rule-Atom Evidence Gate.
Finally, the project rule inventory's stable ids depend on path, line number, and text. That is adequate for local traceability, but moving rules between files or editing lines can change ids [[cite:localInventoryScript]]. A production-grade rule atom ledger might need explicit persisted ids in rule files or a migration map across rule revisions.
Conclusion
Genesis analysis harnesses need rule-atom evidence gates, not agent rosters alone. The inspected template has a strong architecture: project rule inventory, explicit coverage policy, evidence quality rules, verdict rules, schemas, renderer, report linter, allowed CLI registry, and a disciplined agent model split. It also has a clear local blocker: platform CLI scripts are present but not executable, and preflight reports the issue. The correct claim is therefore not pass or fail; it is stage-specific readiness.
The Rule-Atom Evidence Gate turns that mixed state into an engineering asset. It says what the harness can prove now, what it cannot prove yet, and where the next repair should happen. A derived harness should not ask readers to trust the number of agents, the size of the policy directory, or the polish of the report. It should show the selected rule atoms, evidence ids, policy gates, platform evidence status, and verdict effects. That is the level at which a Genesis analysis harness becomes accountable.