## 示例代码

ragas版本：0.4.2

`ContextRecall`指标使用:

https://docs.ragas.io/en/stable/concepts/metrics/available_metrics/context_recall/

```python
import os

from openai import AsyncOpenAI
from ragas.llms import llm_factory
from ragas.metrics.collections import ContextRecall

# 1. 设置 LLM 客户端
# 初始化异步 OpenAI 客户端
# 从环境变量中获取 API Key 和 Base URL (通常用于兼容 OpenAI 接口的模型，如 DeepSeek, Kimi 等)
async_openai_client = AsyncOpenAI(api_key=os.environ.get("OPENAI_API_KEY"),
                                  base_url=os.environ.get("OPENAI_BASE_URL"))

# 2. 创建评估用 LLM 实例
# 使用 ragas 的 llm_factory 包装客户端
# 这里的模型被指定为 "deepseek-v3.2" (也就是用 DeepSeek 模型作为裁判/评估者)
# max_tokens: 设置生成的最大 token 数，防止输出截断
# evaluator_llm = llm_factory("kimi-k2-thinking-turbo", client=async_openai_client)
evaluator_llm = llm_factory("deepseek-v3.2", client=async_openai_client, max_tokens=32768)

# 3. 创建指标对象
# 初始化 ContextRecall 指标
# ContextRecall 用于衡量检索到的上下文 (retrieved_contexts) 是否足以推导出参考答案 (reference)
contextRecall = ContextRecall(llm=evaluator_llm)

# 4. 执行评估
# 调用 score 方法计算分数
result = contextRecall.score(
    user_input="埃菲尔铁塔位于哪里？",  # 用户的原始查询
    retrieved_contexts=[
        "巴黎是法国的首都。",
        "埃菲尔铁塔位于巴黎。"
    ],                                # RAG 系统检索到的文档片段列表
    reference="埃菲尔铁塔位于巴黎。他的名字叫汤姆。"     # 标准答案 (Ground Truth)
)

# 5. 输出结果
# 打印上下文召回率得分 (范围通常是 0 到 1，1 表示完全召回)
print(f"Context Recall Score: {result.value}")
```



## 源码分析

源码位置：ragas\metrics\collections\\context_recall\metric.py

```python
async def ascore(
    self,
    user_input: str,
    retrieved_contexts: List[str],
    reference: str,
) -> MetricResult:
    """
    Calculate context recall score asynchronously.

    Components are guaranteed to be validated and non-None by the base class.

    Args:
        user_input: The original question
        retrieved_contexts: List of retrieved context strings
        reference: The reference answer to evaluate

    Returns:
        MetricResult with recall score (0.0-1.0, higher is better)
    """
    # Input validation
    if not user_input:
        raise ValueError("user_input cannot be empty")
    if not reference:
        raise ValueError("reference cannot be empty")
    if not retrieved_contexts:
        raise ValueError("retrieved_contexts cannot be empty")

    # Combine contexts into a single string
    context = "\n".join(retrieved_contexts) if retrieved_contexts else ""

    # Create input data and generate prompt
    input_data = ContextRecallInput(
        question=user_input, context=context, answer=reference
    )
    prompt_string = self.prompt.to_string(input_data)

    # Get classifications from LLM
    result = await self.llm.agenerate(prompt_string, ContextRecallOutput)

    # Calculate score
    if not result.classifications:
        return MetricResult(value=np.nan)

    # Count attributions
    attributions = [c.attributed for c in result.classifications]
    score = sum(attributions) / len(attributions) if attributions else np.nan

    return MetricResult(value=float(score))
```

核心算法:
$$
\text{Context Recall} = \frac{\text{Number of claims in the reference supported by the retrieved context}}{\text{Total number of claims in the reference}}
$$
prompt_string的值:

```json
Given a context and an answer, analyze each statement in the answer and classify if the statement can be attributed to the given context or not.
Use only binary classification: 1 if the statement can be attributed to the context, 0 if it cannot.
Provide detailed reasoning for each classification.
Please return the output in a JSON format that complies with the following schema as specified in JSON Schema:
{"$defs": {"ContextRecallClassification": {"description": "Classification of a single statement.", "properties": {"statement": {"description": "Individual statement extracted from the answer", "title": "Statement", "type": "string"}, "reason": {"description": "Reasoning for why the statement is or isn't attributable to context", "title": "Reason", "type": "string"}, "attributed": {"description": "Binary classification: 1 if the statement can be attributed to context, 0 otherwise", "title": "Attributed", "type": "integer"}}, "required": ["statement", "reason", "attributed"], "title": "ContextRecallClassification", "type": "object"}}, "description": "Structured output for context recall classifications.", "properties": {"classifications": {"description": "List of statement classifications", "items": {"$ref": "#/$defs/ContextRecallClassification"}, "title": "Classifications", "type": "array"}}, "required": ["classifications"], "title": "ContextRecallOutput", "type": "object"}Do not use single quotes in your response but double quotes,properly escaped with a backslash.

--------EXAMPLES-----------
Example 1
Input: {
    "question": "What can you tell me about Albert Einstein?",
    "context": "Albert Einstein (14 March 1879 - 18 April 1955) was a German-born theoretical physicist, widely held to be one of the greatest and most influential scientists of all time. Best known for developing the theory of relativity, he also made important contributions to quantum mechanics, and was thus a central figure in the revolutionary reshaping of the scientific understanding of nature that modern physics accomplished in the first decades of the twentieth century. His mass-energy equivalence formula E = mc2, which arises from relativity theory, has been called 'the world's most famous equation'. He received the 1921 Nobel Prize in Physics 'for his services to theoretical physics, and especially for his discovery of the law of the photoelectric effect', a pivotal step in the development of quantum theory. His work is also known for its influence on the philosophy of science. In a 1999 poll of 130 leading physicists worldwide by the British journal Physics World, Einstein was ranked the greatest physicist of all time. His intellectual achievements and originality have made Einstein synonymous with genius.",
    "answer": "Albert Einstein, born on 14 March 1879, was a German-born theoretical physicist, widely held to be one of the greatest and most influential scientists of all time. He received the 1921 Nobel Prize in Physics for his services to theoretical physics. He published 4 papers in 1905. Einstein moved to Switzerland in 1895."
}
Output: {
    "classifications": [
        {
            "statement": "Albert Einstein, born on 14 March 1879, was a German-born theoretical physicist, widely held to be one of the greatest and most influential scientists of all time.",
            "reason": "The date of birth of Einstein is mentioned clearly in the context.",
            "attributed": 1
        },
        {
            "statement": "He received the 1921 Nobel Prize in Physics for his services to theoretical physics.",
            "reason": "The exact sentence is present in the given context.",
            "attributed": 1
        },
        {
            "statement": "He published 4 papers in 1905.",
            "reason": "There is no mention about papers he wrote in the given context.",
            "attributed": 0
        },
        {
            "statement": "Einstein moved to Switzerland in 1895.",
            "reason": "There is no supporting evidence for this in the given context.",
            "attributed": 0
        }
    ]
}

Example 2
Input: {
    "question": "who won 2020 icc world cup?",
    "context": "The 2022 ICC Men's T20 World Cup, held from October 16 to November 13, 2022, in Australia, was the eighth edition of the tournament. Originally scheduled for 2020, it was postponed due to the COVID-19 pandemic. England emerged victorious, defeating Pakistan by five wickets in the final to clinch their second ICC Men's T20 World Cup title.",
    "answer": "England"
}
Output: {
    "classifications": [
        {
            "statement": "England",
            "reason": "The context clarifies that England won the 2022 edition (which was originally scheduled for 2020).",
            "attributed": 1
        }
    ]
}

Example 3
Input: {
    "question": "What is the tallest mountain in the world?",
    "context": "The Andes is the longest continental mountain range in the world, located in South America. It stretches across seven countries and features many of the highest peaks in the Western Hemisphere. The range is known for its diverse ecosystems, including the high-altitude Andean Plateau and the Amazon rainforest.",
    "answer": "Mount Everest."
}
Output: {
    "classifications": [
        {
            "statement": "Mount Everest.",
            "reason": "The provided context discusses the Andes mountain range, which does not include Mount Everest or directly relate to the world's tallest mountain.",
            "attributed": 0
        }
    ]
}
-----------------------------

Now perform the same with the following input
input: {
    "question": "埃菲尔铁塔位于哪里？",
    "context": "巴黎是法国的首都。\n埃菲尔铁塔位于巴黎。",
    "answer": "埃菲尔铁塔位于巴黎。他的名字叫汤姆。"
}
Output: 
```

result的值:

```json
ContextRecallOutput(classifications=[
    ContextRecallClassification(statement='埃菲尔铁塔位于巴黎。',
    reason='该陈述与上下文中的句子"埃菲尔铁塔位于巴黎。"完全一致。',
    attributed=1),
    ContextRecallClassification(statement='他的名字叫汤姆。',
    reason='上下文没有提及埃菲尔铁塔的名字，也没有提到任何叫汤姆的人。',
    attributed=0)
])
```

ContextRecallClassification结构如下:

```python
class ContextRecallClassification(BaseModel):
    """Classification of a single statement."""

    statement: str = Field(
        ..., description="Individual statement extracted from the answer"
    )
    reason: str = Field(
        ...,
        description="Reasoning for why the statement is or isn't attributable to context",
    )
    attributed: int = Field(
        ...,
        description="Binary classification: 1 if the statement can be attributed to context, 0 otherwise",
    )
```

attributions的值:

```python
[1, 0]
```

得分算法:

```python
score = sum(attributions) / len(attributions) if attributions else np.nan
```

即score = 1 / 2 = 0.5

## 超出大模型长度

相关代码设置：

```python
evaluator_llm = llm_factory("deepseek-v3.2", client=async_openai_client, max_tokens=32768)
```

max_tokens不够可能报错:

```
<generation number="1">
<exception>
    The output is incomplete due to a max_tokens length limit.
</exception>
<completion>
    ......
</completion>
</generation>

<generation number="2">
<exception>
    The output is incomplete due to a max_tokens length limit.
</exception>
<completion>
    ......
</completion>
</generation>

<generation number="3">
<exception>
    The output is incomplete due to a max_tokens length limit.
</exception>
<completion>
    ......
</completion>
</generation>

</failed_attempts>

<last_exception>
    The output is incomplete due to a max_tokens length limit.
</last_exception>
```



## 核心流程

简而言之，它通过大语言模型（LLM）来评估：**你检索到的文档（Context）是否包含了标准答案（Reference）中提到的所有关键信息。**

核心流程步骤

1. **输入校验:**

   - 首先检查 `user_input`（问题）、`reference`（标准答案/真值）和 `retrieved_contexts`（检索到的文档片段）是否为空。任何一项缺失都会抛出错误，保证评估的有效性。

2. **数据预处理:**

   - `context = "\n".join(retrieved_contexts)`: 将检索到的多个文档片段拼接成一个长字符串。这是为了将其作为一段完整的上下文传给 LLM 进行阅读。

3. **构建提示词与 LLM 交互:**

   - `ContextRecallInput`: 将问题、上下文和参考答案封装成结构化输入。
   - `self.prompt.to_string(input_data)`: 将这些数据填充到预设的 Prompt 模板中。
   - `await self.llm.agenerate(...)`: **这是最关键的一步。** 它调用 LLM（作为裁判），要求 LLM 输出一个 `ContextRecallOutput` 类型的结构化数据。
   - **LLM 内部在做什么？** 根据代码推断，LLM 被要求将 `reference`（标准答案）拆解为多个**陈述句（Statements）**，并逐一判断每个陈述句是否能从 `context` 中找到依据（即 `attributed`）。

4. **计算得分 (Score Calculation):**

   - 代码提取了 LLM 返回的分类结果：`attributions = [c.attributed for c in result.classifications]`。这是一个布尔值列表（例如 `[True, True, False]`）。

   - 计算公式：

     

     $$Score = \frac{\sum (\text{Attributed Statements})}{\text{Total Statements}}$$

   - 也就是：**(被上下文支持的陈述句数量) / (标准答案中的陈述句总数)**。


https://docs.ragas.io/en/stable/concepts/metrics/available_metrics/context_recall/

RAGAS的ContextRecall指标分析

人工智能

Ragas库的ContextRecall指标用于评估RAG系统检索内容是否覆盖标准答案，通过LLM分析答案陈述与上下文的匹配度，计算召回率得分。示例代码展示了如何使用DeepSeek模型评估"埃菲尔铁塔位置"问题的上下文召回情况，最终得分为0.5。

结构化数据

数据预处理

AI驱动 智领未来

文章

问答

视频

学习中心

腾讯云实验室

直播

竞赛

腾讯云代码分析专区

腾讯iOA零信任安全管理系统专区

腾讯云架构师技术同盟交流圈

腾讯云数据库专区

腾讯云智能顾问专区

腾讯云原生专区

腾讯混元专区

腾讯云TCE专区

腾讯云Lighthouse专区

腾讯云HAI专区

腾讯云Edgeone专区

腾讯云存储专区

腾讯云智能专区

腾讯轻联专区 

腾讯云开发专区

TAPD专区

腾讯轻量云游戏服专区

腾讯云最具价值专家

腾讯云架构师技术同盟

腾讯云创作之星

腾讯云开发者先锋

腾讯云代码助手

云原生构建

TAPD 敏捷项目管理

Cloud Studio

SDK中心

API中心

命令行工具

涵盖代码开发、场景应用、自动测试全流程，助你从零构建专属AI助手

一站式MCP教程库，解锁AI应用新玩法

聚焦“写作效率、视觉美观与运行性能”三方面进行全面升级，为您提供更高效、稳定的创作环境

社区富文本&Markdown编辑器全新改版上线，欢迎大家体验!

诚挚邀请您参与本次调研，分享您的真实使用感受与建议。您的反馈至关重要，感谢您的支持与参与！

社区新版编辑器体验调研

RAGAS的ContextRecall指标分析-腾讯云开发者社区-腾讯云

RAGAS的ContextRecall指标分析

RAGAS的ContextRecall指标分析

社区

活动

圈层

关于

腾讯云开发者

热门产品

热门推荐

更多推荐