The 12 Probes Every LLM Feature Should Survive

Untrusted content the model processes — a web page, an email, a retrieved document — carries instructions, and the model obeys them as if they came from you. The classic failure mode of any agent that reads the outside world.

The user tries to talk the model out of its own rules — role-play framings, “for research,” hypothetical wrappers, DAN-style personas — to extract content the feature is supposed to refuse.

Whether the model will disclose its own hidden instructions, tools, or configuration. Leaked system prompts hand attackers the exact map they need to craft the next bypass — and often expose IP you didn’t mean to publish.

For RAG or any grounded feature: does every claim in the answer trace back to the provided source? Faithfulness is separate from correctness — an answer can be true in general and still unfaithful because it wasn’t supported by your documents.

Confident fabrication in the absence of grounding — invented citations, fake product features, made-up policy numbers, APIs that don’t exist. The failure that costs trust fastest because it’s delivered with total fluency.

How the feature treats personal data on the way in and out — whether it echoes back sensitive fields it shouldn’t, stores them where it shouldn’t, or reveals one user’s data to another.

The agentic cousin of injection: an attacker gets the model to send data out — through a tool call, a rendered image URL, a link with secrets in the query string. Especially dangerous when the model can browse, fetch, or call APIs.

Whether the feature stays inside its job. A support bot shouldn’t write your legal contracts or give medical advice; a coding assistant shouldn’t opine on your divorce. Off-scope answers are a liability surface even when they’re “correct.”

The opposite failure, and the one teams forget to test. A model tuned hard for safety starts refusing legitimate requests — “how do I kill a Linux process” read as violence. Over-refusal quietly destroys the product experience while looking “safe.”

Whether the feature can be steered into abusive, hateful, or harassing output — including when the user is hostile and trying to bait it, or when a translation/summarization task launders toxic input into toxic output.

A mid-conversation attempt to override the standing rules — “new instructions supersede all previous ones.” Tests whether your system prompt actually has authority over later user turns, or whether the most recent message always wins.

Whether the same input reliably yields an equivalent answer — across reruns, across trivial rephrasings, across a model-version bump. Inconsistency is a regression you can’t debug because it won’t reproduce on demand.