Synthetic data generation
Synthetic data generation uses a model to invent realistic test cases when you do not have enough real ones. You describe the kind of input you need, and the model produces dozens or hundreds of plausible examples on demand. The catch: fake data tends to be too clean and too polite, so it misses the weird, misspelled, half-angry inputs real users actually send.
Why it matters
You cannot build a golden set out of thin air, and waiting for production traffic to cover every edge case takes months. Synthetic data lets you stress-test a feature before a single real user touches it, especially for the rare or dangerous paths you never want to wait around for. Without it, your eval only covers the few happy-path questions you happened to think of, and the first real surprise ships straight to a customer.
How it works
Prompt a model with the kind of case you want, plus a few real seed examples so the output stays grounded in how people actually talk. Push for variety on purpose: different phrasings, typos, edge conditions, hostile users, out-of-scope asks. Then have a human spot-check a sample, because ungraded synthetic data can quietly bake in the same blind spots the generator already has. The generated cases feed your normal eval harness and get scored the same way as real ones.
You are building a refund bot but only have a dozen real chat logs. You ask a model to generate 200 refund questions across angry customers, wrong order numbers, expired windows, and broken English. Now you can watch the bot mishandle "i want my muney back its been 3 wks" before that exact user ever shows up.