SKIP TO CONTENT

CASE STUDY

LIVE REPO + CI

Nexural QA OS — Signed-Evidence Runner Fleet

QA infrastructure for a 39-repo product federation — an 85-runner fleet that signs its evidence and refuses unearned greens.

Topology: an 85-runner fleet feeds a DAG orchestrator, whose runs are packaged by a hash-chain evidence signer (Ed25519-capable) and validated by the qa verify trust command. In parallel an autonomous SDET loop drives a 52-to-91-percent coverage ratchet; both paths terminate in the full-fleet CI gate.

PROBLEM

An 85-runner QA fleet could return green while doing zero work — its evidence was unverifiable terminal scroll.

BUILT

Every runner now emits hash-sealed evidence bundles (Ed25519 signing built in), `qa verify` validates any run directory, and an autonomous SDET loop generates passing tests.

runners registered in the fleet
85
measured line coverage
52→91%
tests behind ratcheting thresholds
3,698
false-green honesty bugs fixed
11
FAILURE MODES
  • Runners returning green, doing nothing
  • Evidence tampering
  • Hangs counted as passes
  • Silent coverage rot

EVIDENCE — REAL ARTIFACTS, CAPTURED FROM DISK

TypeScriptTurborepopnpmVitestevidence signingGitHub ActionsSemgrepGitleaks

IMPROVE NEXT: Publish a sanitized sample evidence bundle so anyone can run `qa verify` against a real signed run without repo access.