Faithfulness
Faithfulness measures whether an answer is actually backed by the sources the model was given. A faithful answer only says things the retrieved documents support. An unfaithful one adds claims from nowhere, even if they happen to sound right.
Why it matters
In a retrieval system, being fluent is easy and being grounded is the whole job. An answer can read perfectly and still be quietly invented, which is just a hallucination in a nicer outfit. Faithfulness is how you catch that: it separates "the model used the sources" from "the model ignored the sources and improvised."
How it works
You score faithfulness by breaking an answer into its individual claims and checking each one against the retrieved context. Every claim that the sources support scores well; every claim that appears from nowhere drags the score down. This is often automated with an LLM judge or a natural-language-inference model that rates whether the context entails each statement. It pairs with answer relevancy, which checks the other half: that the answer is on-topic.
A user asks about a refund policy. The retrieved doc says refunds are available for 14 days. The model answers "refunds within 30 days, and store credit after that." The 14-day part is faithful; the 30 days and the store credit are invented, so the answer scores low on faithfulness even though it reads smoothly.