JTjason.teixeira() Glossary
Home / Learn / Glossary / Flake
CI / CD

Flake

also called Flaky test

A flake is a test that passes and fails on the same code, without you changing a thing. Run it ten times and you get eight greens and two reds from identical inputs. The failure is noise, and that noise quietly poisons your trust in every other test.

Why it matters

Once a test flakes, people stop trusting it, and soon they stop trusting the whole suite. The habit becomes "just hit retry until it goes green," which is how a genuine failure sails through disguised as a flake. That is fake green: your CI says pass, but it no longer means anything.

How it works

You find flakes by running the same test many times on unchanged code and watching which ones disagree with themselves. The usual culprits are timing (a fixed sleep instead of waiting for a condition), shared state between tests, real network calls, and randomness with no fixed seed. The honest fix is to quarantine the flaky test so it stops blocking merges, then repair the root cause instead of adding retries that just hide it.

In practice

A support bot's end-to-end test clicks "send" and immediately checks for a reply, but the model sometimes answers in 400ms and sometimes in 1.2 seconds. On fast runs it passes, on slow runs it fails, and the code never changed. Replace the fixed wait with "wait until the reply appears," and the flake disappears.

Want this checked on your own AI feature?
Get a free mini-eval — real findings on your live feature, no call required.
Free mini-eval →
© 2026 Jason Teixeira · Sage Ideas LLC · Glossary · Learn · privacy