Hallucination

Q: Hallucination or Faithfulness — which one should I actually use?

Use HallucinationMetric when you have a curated, ground-truth context. Use Faithfulness for RAG, where the source of truth is the your retriever fetched.

Q: Is a higher or lower Hallucination score better?

Higher is better. The score is the fraction of context documents the stays consistent with, so 1 is perfect. threshold is therefore a minimum — output passes at or above it. forces the threshold to 1.

Q: It compares against context, not retrieval context — what's the difference?

context is ground-truth reference material you supply; is what your retriever fetched. This metric checks contradictions against context only, so it measures the generator, not the retriever.

Q: When should I NOT reach for this metric?

Don't use it on a live RAG system. It assumes context is trusted ground truth, so feeding it noisy retrieved chunks misleads — use Faithfulness for runtime instead.

LLM-as-a-judge

Single-turn

Reference-based

Multimodal

The hallucination metric uses LLM-as-a-judge to determine whether your LLM generates factually correct information by comparing the actual_output to the provided context.

Required Arguments

To use the HallucinationMetric, you'll have to provide the following arguments when creating an LLMTestCase:

input
actual_output
context

Read the How Is It Calculated section below to learn how test case parameters are used for metric calculation.

Usage

The HallucinationMetric() can be used for end-to-end evaluation:

from deepeval.metrics import HallucinationMetric
from deepeval.test_case import LLMTestCase
from deepeval import evaluate

# Replace this with the actual documents that you are passing as input to your LLM.
context=["A man with blond-hair, and a brown shirt drinking out of a public water fountain."]

# Replace this with the actual output from your LLM application
actual_output="A blond drinking water in public."

test_case = LLMTestCase(
    input="What was the blond doing?",
    actual_output=actual_output,
    context=context
)
metric = HallucinationMetric(threshold=0.5)

# To run metric as a standalone
# metric.measure(test_case)
# print(metric.score, metric.reason)

evaluate(test_cases=[test_case], metrics=[metric])

There are SEVEN optional parameters when creating a HallucinationMetric:

[Optional] threshold: a number representing the minimum passing threshold. Can also be set to None to run the metric in score-only mode. Defaulted to 0.5.
[Optional] model: a string specifying which of OpenAI's GPT models to use, OR any custom LLM model of type DeepEvalBaseLLM. Defaulted to gpt-5.4.
[Optional] include_reason: a boolean which when set to True, will include a reason for its evaluation score. Defaulted to True.
[Optional] strict_mode: a boolean which when set to True, enforces a binary metric score: 1 for perfection, 0 otherwise. It also overrides the current threshold and sets it to 1. Defaulted to False.
[Optional] async_mode: a boolean which when set to True, enables concurrent execution within the measure() method. Defaulted to True.
[Optional] verbose_mode: a boolean which when set to True, prints the intermediate steps used to calculate said metric to the console, as outlined in the How Is It Calculated section. Defaulted to False.
[Optional] flaky: a boolean which when set to True, marks the metric as flaky. Defaulted to False.

Within components

You can also run the HallucinationMetric within nested components for component-level evaluation.

from deepeval.dataset import EvaluationDataset, Golden
from deepeval.tracing import observe, update_current_span
...

@observe(metrics=[metric])
def inner_component():
    # Set test case at runtime
    test_case = LLMTestCase(input="...", actual_output="...")
    update_current_span(test_case=test_case)
    return

@observe
def llm_app(input: str):
    # Component can be anything from an LLM call, retrieval, agent, tool use, etc.
    inner_component()
    return

dataset = EvaluationDataset(goldens=[Golden(input="Hi!")])
for golden in dataset.evals_iterator():
    llm_app(golden.input)

As a standalone

You can also run the HallucinationMetric on a single test case as a standalone, one-off execution.

...

metric.measure(test_case)
print(metric.score, metric.reason)

How Is It Calculated?

The HallucinationMetric score is calculated according to the following equation:

\text{Hallucination} = \frac{\text{Number of Aligned Contexts}}{\text{Total Number of Contexts}}

The HallucinationMetric uses an LLM to determine, for each context in contexts, whether there are any contradictions to the actual_output.

FAQs

Hallucination or Faithfulness — which one should I actually use?

Use HallucinationMetric when you have a curated, ground-truth context. Use Faithfulness for RAG, where the source of truth is the retrieval_context your retriever fetched.

Is a higher or lower Hallucination score better?

Higher is better. The score is the fraction of context documents the actual_output stays consistent with, so 1 is perfect. threshold is therefore a minimum — output passes at or above it. strict_mode forces the threshold to 1.

It compares against context, not retrieval context — what's the difference?

context is ground-truth reference material you supply; retrieval_context is what your retriever fetched. This metric checks contradictions against context only, so it measures the generator, not the retriever.

When should I NOT reach for this metric?

Don't use it on a live RAG system. It assumes context is trusted ground truth, so feeding it noisy retrieved chunks misleads — use Faithfulness for runtime retrieval_context instead.

Required Arguments

Usage

Within components

As a standalone

How Is It Calculated?

FAQs

On this page