## 示例代码

ragas版本：0.4.2

`ContextPrecision`指标使用:

https://docs.ragas.io/en/stable/concepts/metrics/available_metrics/context_precision/

```python
import os

from openai import AsyncOpenAI
from ragas.llms import llm_factory
from ragas.metrics.collections import ContextPrecision

# -----------------------------------------------------------------------------
# 1. 设置 LLM 客户端
# -----------------------------------------------------------------------------
# 初始化 OpenAI 异步客户端。
# 这里从环境变量中获取 API Key 和 Base URL，通常用于连接兼容 OpenAI 接口的服务商（如 DeepSeek, Moonshot 等）。
async_openai_client = AsyncOpenAI(
    api_key=os.environ.get("OPENAI_API_KEY"),
    base_url=os.environ.get("OPENAI_BASE_URL")
)

# -----------------------------------------------------------------------------
# 2. 配置评估模型 (Evaluator LLM)
# -----------------------------------------------------------------------------
# 使用 ragas 的工厂方法创建一个 LLM 实例。
#evaluator_llm = llm_factory("kimi-k2-thinking-turbo", client=async_openai_client)
# max_tokens: 设置生成的最大 token 数，防止输出截断
evaluator_llm = llm_factory("deepseek-v3.2", client=async_openai_client, max_tokens=32768)

# -----------------------------------------------------------------------------
# 3. 创建评估指标
# -----------------------------------------------------------------------------
# 初始化 ContextPrecision 指标对象。
# ContextPrecision 用于衡量检索到的上下文中有多少是与标准答案（Reference）真正相关的。
# 我们将上面配置好的 evaluator_llm 传递给它，作为裁判模型。
contextPrecision = ContextPrecision(llm=evaluator_llm)

# -----------------------------------------------------------------------------
# 4. 执行评估
# -----------------------------------------------------------------------------
# 调用 score 方法计算分数。
# - user_input: 用户提出的问题。
# - reference: 问题的标准答案 (Ground Truth)。
# - retrieved_contexts: RAG 系统检索回来的文档片段列表（Contexts）。
result = contextPrecision.score(
    user_input="埃菲尔铁塔位于哪里？",
    reference="埃菲尔铁塔位于巴黎。",
    retrieved_contexts=[
        "埃菲尔铁塔位于巴黎。",       # 相关文档 (主要依据)
        "勃兰登堡门位于柏林。"         # 干扰/不相关文档
    ]
)

# -----------------------------------------------------------------------------
# 5. 输出结果
# -----------------------------------------------------------------------------
# 打印计算出的精确度分数。
# 分数越高 (接近 1.0)，说明检索到的内容中相关内容的排序越靠前且干扰越少。
print(f"Context Precision Score: {result.value}")
```



## 源码分析

源码位置：ragas\metrics\collections\context_precision\metric.py

```python
async def ascore(
    self, user_input: str, reference: str, retrieved_contexts: List[str]
) -> MetricResult:
    """
    Calculate context precision score using reference.

    Args:
        user_input: The question being asked
        reference: The reference answer to compare against
        retrieved_contexts: The retrieved contexts to evaluate

    Returns:
        MetricResult with context precision score (0.0-1.0, higher is better)
    """
    # Input validation
    if not user_input:
        raise ValueError("user_input cannot be empty")
    if not reference:
        raise ValueError("reference cannot be empty")
    if not retrieved_contexts:
        raise ValueError("retrieved_contexts cannot be empty")

    # Evaluate each retrieved context
    verdicts = []
    for context in retrieved_contexts:
        # Create input data and generate prompt
        input_data = ContextPrecisionInput(
            question=user_input, context=context, answer=reference
        )
        prompt_string = self.prompt.to_string(input_data)
        result = await self.llm.agenerate(prompt_string, ContextPrecisionOutput)
        verdicts.append(result.verdict)

    # Calculate average precision
    score = self._calculate_average_precision(verdicts)
    return MetricResult(value=float(score))

def _calculate_average_precision(self, verdicts: List[int]) -> float:
    """Calculate average precision from binary verdicts. Matches legacy logic exactly."""
    verdict_list = verdicts
    denominator = sum(verdict_list) + 1e-10
    numerator = sum(
        [
            (sum(verdict_list[: i + 1]) / (i + 1)) * verdict_list[i]
            for i in range(len(verdict_list))
        ]
    )
    score = numerator / denominator

    if np.isnan(score):
        # Match legacy warning behavior
        import logging

        logging.warning(
            "Invalid response format. Expected a list of dictionaries with keys 'verdict'"
        )

    return score
```

针对retrieved_contexts进行循环处理：

```python
for context in retrieved_contexts:
    # Create input data and generate prompt
    input_data = ContextPrecisionInput(
        question=user_input, context=context, answer=reference
    )
    prompt_string = self.prompt.to_string(input_data)
    result = await self.llm.agenerate(prompt_string, ContextPrecisionOutput)
    verdicts.append(result.verdict)
```

retrieved_contexts的值:

```python
['埃菲尔铁塔位于巴黎。', '勃兰登堡门位于柏林。']
```

第一次循环:

prompt_string的值:

```json
Given question, answer and context verify if the context was useful in arriving at the given answer. Give verdict as "1" if useful and "0" if not with json output.
Please return the output in a JSON format that complies with the following schema as specified in JSON Schema:
{"description": "Structured output for context precision evaluation.", "properties": {"reason": {"description": "Reason for verification", "title": "Reason", "type": "string"}, "verdict": {"description": "Binary (0/1) verdict of verification", "title": "Verdict", "type": "integer"}}, "required": ["reason", "verdict"], "title": "ContextPrecisionOutput", "type": "object"}Do not use single quotes in your response but double quotes,properly escaped with a backslash.

--------EXAMPLES-----------
Example 1
Input: {
    "question": "What can you tell me about Albert Einstein?",
    "context": "Albert Einstein (14 March 1879 – 18 April 1955) was a German-born theoretical physicist, widely held to be one of the greatest and most influential scientists of all time. Best known for developing the theory of relativity, he also made important contributions to quantum mechanics, and was thus a central figure in the revolutionary reshaping of the scientific understanding of nature that modern physics accomplished in the first decades of the twentieth century. His mass–energy equivalence formula E = mc2, which arises from relativity theory, has been called 'the world's most famous equation'. He received the 1921 Nobel Prize in Physics 'for his services to theoretical physics, and especially for his discovery of the law of the photoelectric effect', a pivotal step in the development of quantum theory. His work is also known for its influence on the philosophy of science. In a 1999 poll of 130 leading physicists worldwide by the British journal Physics World, Einstein was ranked the greatest physicist of all time. His intellectual achievements and originality have made Einstein synonymous with genius.",
    "answer": "Albert Einstein, born on 14 March 1879, was a German-born theoretical physicist, widely held to be one of the greatest and most influential scientists of all time. He received the 1921 Nobel Prize in Physics for his services to theoretical physics."
}
Output: {
    "reason": "The provided context was indeed useful in arriving at the given answer. The context includes key information about Albert Einstein's life and contributions, which are reflected in the answer.",
    "verdict": 1
}

Example 2
Input: {
    "question": "who won 2020 icc world cup?",
    "context": "The 2022 ICC Men's T20 World Cup, held from October 16 to November 13, 2022, in Australia, was the eighth edition of the tournament. Originally scheduled for 2020, it was postponed due to the COVID-19 pandemic. England emerged victorious, defeating Pakistan by five wickets in the final to clinch their second ICC Men's T20 World Cup title.",
    "answer": "England"
}
Output: {
    "reason": "the context was useful in clarifying the situation regarding the 2020 ICC World Cup and indicating that England was the winner of the tournament that was intended to be held in 2020 but actually took place in 2022.",
    "verdict": 1
}

Example 3
Input: {
    "question": "What is the tallest mountain in the world?",
    "context": "The Andes is the longest continental mountain range in the world, located in South America. It stretches across seven countries and features many of the highest peaks in the Western Hemisphere. The range is known for its diverse ecosystems, including the high-altitude Andean Plateau and the Amazon rainforest.",
    "answer": "Mount Everest."
}
Output: {
    "reason": "the provided context discusses the Andes mountain range, which, while impressive, does not include Mount Everest or directly relate to the question about the world's tallest mountain.",
    "verdict": 0
}
-----------------------------

Now perform the same with the following input
input: {
    "question": "埃菲尔铁塔位于哪里？",
    "context": "埃菲尔铁塔位于巴黎。",
    "answer": "埃菲尔铁塔位于巴黎。"
}
Output: 
```

result的值:

```python
reason='The provided context directly states the location of the Eiffel Tower as Paris, which is exactly the information used to form the answer. Therefore, the context was essential and useful in arriving at the given answer.' verdict=1
```

第二次循环:

prompt_string的值:

```json
Given question, answer and context verify if the context was useful in arriving at the given answer. Give verdict as "1" if useful and "0" if not with json output.
Please return the output in a JSON format that complies with the following schema as specified in JSON Schema:
{"description": "Structured output for context precision evaluation.", "properties": {"reason": {"description": "Reason for verification", "title": "Reason", "type": "string"}, "verdict": {"description": "Binary (0/1) verdict of verification", "title": "Verdict", "type": "integer"}}, "required": ["reason", "verdict"], "title": "ContextPrecisionOutput", "type": "object"}Do not use single quotes in your response but double quotes,properly escaped with a backslash.

--------EXAMPLES-----------
Example 1
Input: {
    "question": "What can you tell me about Albert Einstein?",
    "context": "Albert Einstein (14 March 1879 – 18 April 1955) was a German-born theoretical physicist, widely held to be one of the greatest and most influential scientists of all time. Best known for developing the theory of relativity, he also made important contributions to quantum mechanics, and was thus a central figure in the revolutionary reshaping of the scientific understanding of nature that modern physics accomplished in the first decades of the twentieth century. His mass–energy equivalence formula E = mc2, which arises from relativity theory, has been called 'the world's most famous equation'. He received the 1921 Nobel Prize in Physics 'for his services to theoretical physics, and especially for his discovery of the law of the photoelectric effect', a pivotal step in the development of quantum theory. His work is also known for its influence on the philosophy of science. In a 1999 poll of 130 leading physicists worldwide by the British journal Physics World, Einstein was ranked the greatest physicist of all time. His intellectual achievements and originality have made Einstein synonymous with genius.",
    "answer": "Albert Einstein, born on 14 March 1879, was a German-born theoretical physicist, widely held to be one of the greatest and most influential scientists of all time. He received the 1921 Nobel Prize in Physics for his services to theoretical physics."
}
Output: {
    "reason": "The provided context was indeed useful in arriving at the given answer. The context includes key information about Albert Einstein's life and contributions, which are reflected in the answer.",
    "verdict": 1
}

Example 2
Input: {
    "question": "who won 2020 icc world cup?",
    "context": "The 2022 ICC Men's T20 World Cup, held from October 16 to November 13, 2022, in Australia, was the eighth edition of the tournament. Originally scheduled for 2020, it was postponed due to the COVID-19 pandemic. England emerged victorious, defeating Pakistan by five wickets in the final to clinch their second ICC Men's T20 World Cup title.",
    "answer": "England"
}
Output: {
    "reason": "the context was useful in clarifying the situation regarding the 2020 ICC World Cup and indicating that England was the winner of the tournament that was intended to be held in 2020 but actually took place in 2022.",
    "verdict": 1
}

Example 3
Input: {
    "question": "What is the tallest mountain in the world?",
    "context": "The Andes is the longest continental mountain range in the world, located in South America. It stretches across seven countries and features many of the highest peaks in the Western Hemisphere. The range is known for its diverse ecosystems, including the high-altitude Andean Plateau and the Amazon rainforest.",
    "answer": "Mount Everest."
}
Output: {
    "reason": "the provided context discusses the Andes mountain range, which, while impressive, does not include Mount Everest or directly relate to the question about the world's tallest mountain.",
    "verdict": 0
}
-----------------------------

Now perform the same with the following input
input: {
    "question": "埃菲尔铁塔位于哪里？",
    "context": "勃兰登堡门位于柏林。",
    "answer": "埃菲尔铁塔位于巴黎。"
}
Output: 
```

result的值:

```python
ContextPrecisionOutput(reason='The provided context discusses the Brandenburg Gate in Berlin, which is unrelated to the question about the location of the Eiffel Tower. The answer is correct but not derived from the context.', verdict=0)
```

此时verdicts的值:

```python
[1, 0]
```

计算平均精度:

```python
score = self._calculate_average_precision(verdicts)
```

_calculate_average_precision()函数的核心公式逻辑:
$$
Context Precision@K =  \frac{\sum_{k=1}^{K} \text{(Precision@k} \times v_k)}{\text{Total number of relevant items in the top } K \text{ results}} 
$$
$$
\text{Precision@k} = \frac{\text{true positives@k}}{\text{(true positives@k + false positives@k)}}
$$

下面解释这个公式的含义。

举个例子说明:

假设你的检索器找回了 3 个文档，其中真实相关的文档有 2 个。

情况 A（排名很好）：

1.**相关文档**

2.**相关文档**

3.不相关文档

计算过程：

- $k=1$: 相关 (1/1 = 100%)
- $k=2$: 相关 (2/2 = 100%)
- $k=3$: 不相关 (跳过)
- 平均分：$(1 + 1) / 2 = \mathbf{1.0}$ (满分)

情况 B（排名很差）：

1.不相关文档

2.**相关文档**

3.**相关文档**

计算过程：

- $k=1$: 不相关 (跳过)
- $k=2$: 相关 (前2个里有1个相关 = 1/2 = 50%)
- $k=3$: 相关 (前3个里有2个相关 = 2/3 = 66.7%)
- 平均分：$(0.5 + 0.667) / 2 = \mathbf{0.58}$ (低分)

## 超出大模型长度

相关代码设置：

```python
evaluator_llm = llm_factory("deepseek-v3.2", client=async_openai_client, max_tokens=32768)
```

max_tokens不够可能报错:

```
<generation number="1">
<exception>
    The output is incomplete due to a max_tokens length limit.
</exception>
<completion>
    ......
</completion>
</generation>

<generation number="2">
<exception>
    The output is incomplete due to a max_tokens length limit.
</exception>
<completion>
    ......
</completion>
</generation>

<generation number="3">
<exception>
    The output is incomplete due to a max_tokens length limit.
</exception>
<completion>
    ......
</completion>
</generation>

</failed_attempts>

<last_exception>
    The output is incomplete due to a max_tokens length limit.
</last_exception>
```



## 核心流程

这段代码实现了一个用于评估 **RAG (检索增强生成)** 系统中 **Context Precision (上下文精度)** 指标的核心逻辑。

简而言之，它衡量的是：**在检索到的上下文中，包含正确答案（Reference）的相关片段是否被排在了靠前的位置。**

以下是对这段代码核心流程的详细分析：

详细执行流程:

代码执行分为三个主要阶段：**输入校验**、**LLM 判别 (打分)**、**数学计算**。

**第一阶段：输入校验**

- 代码首先检查 `user_input`、`reference` 和 `retrieved_contexts` 是否为空。
- **目的**：确保评估有意义。如果没有任何上下文被检索到，或者没有问题/答案，则无法计算精度，直接抛出 `ValueError`。

**第二阶段：相关性判别**

这是代码中最耗时的部分，涉及与 LLM 的交互。

1. **遍历上下文**：代码通过 `for context in retrieved_contexts:` 遍历每一个检索到的文档片段。
2. **构建 Prompt**：
   - 它将 `question` (问题)、`context` (当前片段) 和 `answer` (参考答案) 封装进 `ContextPrecisionInput`。
   - 利用 `self.prompt.to_string` 生成发给 LLM 的提示词。
3. **LLM 异步调用**：
   - `await self.llm.agenerate(...)`：调用大模型来判断当前这个 `context` 是否包含了推导出 `reference` 所需的信息。
   - **注意**：尽管使用了 `await`，但由于是在 `for` 循环中按顺序执行，这里实际上是**串行**处理每个 context（这通常是一个性能瓶颈）。
4. **收集判决 (Verdicts)**：
   - `verdicts.append(result.verdict)`：将 LLM 的判断结果（通常是 `1` 代表相关，`0` 代表不相关）存入列表。

**第三阶段：计算平均精度**

这一步发生在 `_calculate_average_precision` 函数中。

数学原理可视化：

假设 `retrieved_contexts` 有 3 个文档，LLM 判断的相关性结果 `verdicts` 为 `[1, 0, 1]`（第1个相关，第2个不相关，第3个相关）。

$$\text{Score} = \frac{\sum (\text{Precision}@k \times \text{IsRelevant}_k)}{\text{Total Relevant Items}}$$
$$
\text{Precision@k} = \frac{\text{true positives@k}}{\text{(true positives@k + false positives@k)}}
$$


1. **Rank 1 (相关)**:
   - Precision@1 = $1/1 = 1.0$
   - 贡献 = $1.0 \times 1 = 1.0$
2. **Rank 2 (不相关)**:
   - Precision@2 = $1/2 = 0.5$
   - 贡献 = $0.5 \times 0 = 0$ (因为当前项不相关，不计入)
3. **Rank 3 (相关)**:
   - Precision@3 = $2/3 \approx 0.66$
   - 贡献 = $0.66 \times 1 = 0.66$

- **总相关数**：2
- **计算结果**：$(1.0 + 0.66) / 2 = 0.83$

对比案例：

如果顺序是 [0, 1, 1]（相关文档被排在后面）：

- Rank 1: 0
- Rank 2: P@2 = $1/2$, 贡献 $0.5$
- Rank 3: P@3 = $2/3$, 贡献 $0.66$
- 结果：$(0.5 + 0.66) / 2 = 0.58$

**结论**：该算法极其看重**排序 (Ranking)**。相关内容越靠前，分数越高。


https://docs.ragas.io/en/stable/concepts/metrics/available_metrics/context_precision/

RAGAS的ContextPrecision指标分析

这段代码实现了一个用于评估 **RAG (检索增强生成)** 系统中 **Context Precision (上下文精度)** 指标的核心逻辑。
简而言之，它衡量的是：**在检索到的上下文中，包含正确答案（Reference）的相关片段是否被排在了靠前的位置。**

人工智能

本文详解RAG系统中ContextPrecision指标的计算方法，通过Python代码演示如何评估检索结果的相关性排序质量。包含OpenAI API调用、LLM评分、平均精度计算等核心实现，帮助开发者优化检索增强生成系统的上下文质量。

AI驱动 智领未来

文章

问答

视频

学习中心

腾讯云实验室

直播

竞赛

腾讯云代码分析专区

腾讯iOA零信任安全管理系统专区

腾讯云架构师技术同盟交流圈

腾讯云数据库专区

腾讯云智能顾问专区

腾讯云原生专区

腾讯混元专区

腾讯云TCE专区

腾讯云Lighthouse专区

腾讯云HAI专区

腾讯云Edgeone专区

腾讯云存储专区

腾讯云智能专区

腾讯轻联专区 

腾讯云开发专区

TAPD专区

腾讯轻量云游戏服专区

腾讯云最具价值专家

腾讯云架构师技术同盟

腾讯云创作之星

腾讯云开发者先锋

腾讯云代码助手

云原生构建

TAPD 敏捷项目管理

Cloud Studio

SDK中心

API中心

命令行工具

涵盖代码开发、场景应用、自动测试全流程，助你从零构建专属AI助手

一站式MCP教程库，解锁AI应用新玩法

聚焦“写作效率、视觉美观与运行性能”三方面进行全面升级，为您提供更高效、稳定的创作环境

社区富文本&Markdown编辑器全新改版上线，欢迎大家体验!

诚挚邀请您参与本次调研，分享您的真实使用感受与建议。您的反馈至关重要，感谢您的支持与参与！

社区新版编辑器体验调研

RAGAS的ContextPrecision指标分析-腾讯云开发者社区-腾讯云

RAGAS的ContextPrecision指标分析

RAGAS的ContextPrecision指标分析

社区

活动

圈层

关于

腾讯云开发者

热门产品

热门推荐

更多推荐