Field Notes — Jason Teixeira https://agency.sageideas.dev/field-notes.html Working notes from the proof-first trenches: AI automation, QA, and LLM evaluation. en-us Eighteen agents audited a live curriculum. One lesson was teaching a false error. https://agency.sageideas.dev/notes/eighteen-agent-audit-gauntlet.html https://agency.sageideas.dev/notes/eighteen-agent-audit-gauntlet.html Sun, 16 Aug 2026 12:00:00 GMT I pointed an audit fleet at 34 published programming lessons: six auditors executing every code claim, six rewriters, six independent verifiers. 73 defects, 17 critical or high — every one anchored to a verbatim quote. Thirteen minutes of wall clock. Five pages, five agents, one design file — zero merge conflicts https://agency.sageideas.dev/notes/five-pages-five-agents.html https://agency.sageideas.dev/notes/five-pages-five-agents.html Sun, 16 Aug 2026 12:00:00 GMT A parallel agent fleet implemented five production marketing pages 1:1 from design files in about five minutes of wall clock. The interesting part isn't the speed — it's the three rules that made the output shippable. The day my own quality gate blocked me https://agency.sageideas.dev/notes/gate-blocked-me.html https://agency.sageideas.dev/notes/gate-blocked-me.html Sat, 15 Aug 2026 12:00:00 GMT At 11:39 my proof loop refused the PROVEN verdict — 15 high/critical CVEs had drifted into production deps. By 14:35 it was green. Both runs are published verbatim. LLM regression testing with Promptfoo in CI: the minimum viable gate https://agency.sageideas.dev/notes/promptfoo-ci-minimum-gate.html https://agency.sageideas.dev/notes/promptfoo-ci-minimum-gate.html Sat, 15 Aug 2026 12:00:00 GMT 30 golden traces, one judge, one failing exit code. The smallest setup that stops a bad prompt change from reaching production — runnable this afternoon. No fake green: what a proof ledger taught me about AI agents https://agency.sageideas.dev/notes/no-fake-green.html https://agency.sageideas.dev/notes/no-fake-green.html Tue, 04 Aug 2026 12:00:00 GMT Agents will happily report success they never earned. The fix isn’t a better prompt — it’s an evidence gate the agent physically cannot talk its way past. Your LLM feature needs a regression suite more than a better prompt https://agency.sageideas.dev/notes/llm-regression-suite.html https://agency.sageideas.dev/notes/llm-regression-suite.html Sat, 18 Jul 2026 12:00:00 GMT Prompt tweaks feel like progress because nobody is measuring. Golden traces + LLM-as-judge in CI turn "it seems better" into a diff you can gate on. Automation that never talks to your customer https://agency.sageideas.dev/notes/automation-that-never-talks.html https://agency.sageideas.dev/notes/automation-that-never-talks.html Mon, 22 Jun 2026 12:00:00 GMT The best-converting AI workflow I’ve shipped sends zero AI-written messages. Where to put the human approval point, and why it beats full autonomy.