Hallucination

The hallucination metric uses LLM-as-a-judge to determine whether your LLM generates factually correct information by comparing the actual_output to the provided context.

Required Arguments

To use the HallucinationMetric, you'll have to provide the following arguments when creating an LLMTestCase:

• input
• actual_output
• context

Read the How Is It Calculated section below to learn how test case parameters are used for metric calculation.

Usage

The HallucinationMetric() can be used for end-to-end evaluation:

from deepeval import evaluate
from deepeval.metrics import HallucinationMetric
from deepeval.test_case import LLMTestCase

# Replace this with the actual documents that you are passing as input to your LLM.
context=["A man with blond-hair, and a brown shirt drinking out of a public water fountain."]

# Replace this with the actual output from your LLM application
actual_output="A blond drinking water in public."

test_case = LLMTestCase(
    input="What was the blond doing?",
    actual_output=actual_output,
    context=context
)
metric = HallucinationMetric(threshold=0.5)

# To run metric as a standalone
# metric.measure(test_case)
# print(metric.score, metric.reason)

evaluate(test_cases=[test_case], metrics=[metric])

There are SIX optional parameters when creating a HallucinationMetric:

• [Optional] threshold: a float representing the maximum passing threshold, defaulted to 0.5.
• [Optional] model: a string specifying which of OpenAI's GPT models to use, OR any custom LLM model of type DeepEvalBaseLLM. Defaulted to gpt-5.4.
• [Optional] include_reason: a boolean which when set to True, will include a reason for its evaluation score. Defaulted to True.
• [Optional] strict_mode: a boolean which when set to True, enforces a binary metric score: 0 for perfection, 1 otherwise. It also overrides the current threshold and sets it to 0. Defaulted to False.
• [Optional] async_mode: a boolean which when set to True, enables concurrent execution within the measure() method. Defaulted to True.
• [Optional] verbose_mode: a boolean which when set to True, prints the intermediate steps used to calculate said metric to the console, as outlined in the How Is It Calculated section. Defaulted to False.

Within components

You can also run the HallucinationMetric within nested components for component-level evaluation.

from deepeval.dataset import Golden
from deepeval.tracing import observe, update_current_span
...

@observe(metrics=[metric])
def inner_component():
    # Set test case at runtime
    test_case = LLMTestCase(input="...", actual_output="...")
    update_current_span(test_case=test_case)
    return

@observe
def llm_app(input: str):
    # Component can be anything from an LLM call, retrieval, agent, tool use, etc.
    inner_component()
    return

evaluate(observed_callback=llm_app, goldens=[Golden(input="Hi!")])

As a standalone

You can also run the HallucinationMetric on a single test case as a standalone, one-off execution.

...

metric.measure(test_case)
print(metric.score, metric.reason)

How Is It Calculated?

The HallucinationMetric score is calculated according to the following equation:

The HallucinationMetric uses an LLM to determine, for each context in contexts, whether there are any contradictions to the actual_output.