Reference
LLM evaluation metrics explained →
The cornerstone field guide: the four metric families, when to reach for each, and how they become a gate.
01 · Pillar
LLM Evaluation & Quality
Does your AI feature actually work? How to measure whether an LLM feature is good enough to ship — metrics, judges, golden sets, and the evidence that a change did not quietly regress.
Guide
The eval gate, explained →
Animated walkthrough of the gate that blocks a regression before it ships.
Guide
Golden sets, explained →
What a golden set is and why it is the backbone of LLM regression testing.
Guide
Adversarial probes, explained →
How probes surface prompt-injection and jailbreak failures on purpose.
Reference
The evaluation method →
The end-to-end method behind every engagement — how quality is actually proven.
Reference
Evaluation & quality (reference) →
The reference page for the evaluation and quality capability.
Tool
Run a live eval in your browser →
An eval harness that runs client-side so you can watch it score, honestly.
Field note
The day my own quality gate blocked me →
A field note on being stopped by your own gate — the point of the whole thing.
Field note
No fake green →
What a proof ledger taught me about trusting AI agents.
Service
LLM evaluation & QA (service) →
The engagement: measure, gate, and prove your AI feature works.
Tutorial
Build a Golden Set for LLM Regression Testing →
Assemble and grow a golden set of cases for LLM regression testing.
Tutorial
Write Adversarial Probes for Prompt Injection Testing →
Write adversarial probes that test for prompt injection and jailbreaks.
Tutorial
Version Golden Datasets Like Code →
Keep eval datasets under version control for reproducible scores.
Tutorial
Build a Red-Team Test Suite for a Customer-Facing Chatbot →
Build a repeatable red-team suite for a customer-facing chatbot.
Tutorial
Structure a Monorepo for an AI Product Plus Eval Suite →
Structure an AI product monorepo with app, prompts, datasets, and evals.
Guide
LLM-as-a-Judge: How It Works and When It Fails →
How LLM-as-judge works, its biases, and how to calibrate one you can trust.
Guide
Why Your LLM Eval Scores Do Not Match Production →
Why eval scores diverge from production reality and how to close the gap.
Guide
Testing vs Evaluating an LLM: What Is the Difference →
The difference between testing and evaluating an LLM, and when to use each.
Guide
How Often Should You Re-Run Your LLM Evals →
A practical cadence for when to run LLM evals: per-change, nightly, pre-release.
Continue across the library
Need this on your product, not just in theory?
Book a 15-minute call and we'll scope the smallest useful next step.