Files
ai-agent-dev/07_输出解析器.ipynb
2026-07-08 10:09:42 +08:00

561 lines
20 KiB
Plaintext
Raw Blame History

This file contains ambiguous Unicode characters

This file contains Unicode characters that might be confused with other characters. If you think that this is intentional, you can safely ignore this warning. Use the Escape button to reveal them.

{
"cells": [
{
"cell_type": "markdown",
"metadata": {},
"source": [
"# 07 输出解析器\n",
"\n",
"## 学习目标\n",
"1. 理解输出解析器Output Parsers在 LangChain 链中的作用\n",
"2. 掌握常见解析器的用法StrOutputParser、JsonOutputParser、PydanticOutputParser、CommaSeparatedListOutputParser\n",
"3. 学会使用 `手动 try-except + LLM 修复` 处理模型输出格式异常\n",
"4. 了解如何编写自定义输出解析器\n",
"5. 理解「Prompt 中说明格式 + 解析器约束」配合的重要性"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## 1. 为什么需要输出解析器\n",
"\n",
"大模型返回的通常是**自由文本**,但很多时候我们需要:\n",
"\n",
"- 提取结构化的 JSON 数据\n",
"- 把文本转换成 Python 列表、字典或对象\n",
"- 验证输出是否符合预期的数据格式\n",
"- 过滤掉模型多余的解释,只保留关键信息\n",
"\n",
"**输出解析器**就是链中负责把模型输出转换为结构化数据的组件。\n",
"\n",
"在 LCEL 中,链的常见结构是:\n",
"\n",
"```\n",
"chain = prompt | llm | output_parser\n",
"```\n",
"\n",
"其中 `output_parser` 接收 `AIMessage` 对象,输出我们需要的 Python 数据结构。"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## 2. StrOutputParser最简单的字符串解析器\n",
"\n",
"`StrOutputParser` 会把模型返回的 `AIMessage` 对象直接转换成纯字符串。\n",
"\n",
"它适合的场景:\n",
"- 只需要文本回答\n",
"- 不想每次手动写 `.content`\n",
"- 作为链的最后一步统一输出类型"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"from langchain_openai import ChatOpenAI\n",
"from langchain_core.prompts import ChatPromptTemplate\n",
"from langchain_core.output_parsers import StrOutputParser\n",
"from dotenv import load_dotenv\n",
"\n",
"load_dotenv()\n",
"\n",
"\n",
"# 创建模型\n",
"llm = ChatOpenAI(model='qwen3.6-35b-A3b', temperature=0.7)\n",
"\n",
"# 创建 Prompt 模板\n",
"prompt = ChatPromptTemplate.from_messages([\n",
" ('system', '你是一个简洁的助手。'),\n",
" ('user', '用一句话介绍{topic}')\n",
"])\n",
"\n",
"# 构建链prompt -> llm -> 字符串解析器\n",
"chain = prompt | llm | StrOutputParser()\n",
"\n",
"# 调用链,直接得到字符串\n",
"result = chain.invoke({'topic': '机器学习'})\n",
"\n",
"print('返回类型:', type(result))\n",
"print('返回内容:', result)"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"### 代码解释\n",
"\n",
"| 代码 | 作用 |\n",
"| --- | --- |\n",
"| `prompt \\| llm \\| StrOutputParser()` | 用管道符串联三个组件 |\n",
"| `StrOutputParser()` | 自动提取 `.content` 并返回字符串 |\n",
"| `type(result)` | 验证返回结果是否为 `str` |\n",
"\n",
"不加 `StrOutputParser()` 时,`chain.invoke()` 返回的是 `AIMessage` 对象;加了之后返回的是 `str`。"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## 3. JsonOutputParser解析 JSON 输出\n",
"\n",
"`JsonOutputParser` 会自动把模型输出的 JSON 字符串解析成 Python 字典。\n",
"\n",
"使用要点:\n",
"1. 在 Prompt 中明确要求模型输出 JSON\n",
"2. JSON 中的大括号 `{` 在 Prompt 模板中需要写成 `{{` 进行转义\n",
"3. 模型输出必须能被 `json.loads()` 解析"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"from langchain_openai import ChatOpenAI\n",
"from langchain_core.prompts import ChatPromptTemplate\n",
"from langchain_core.output_parsers import JsonOutputParser\n",
"\n",
"llm = ChatOpenAI(model='qwen3.6-35b-A3b', temperature=0.1)\n",
"\n",
"# 注意:模板中的 {{ 和 }} 会被 LangChain 渲染成单个 { 和 }\n",
"system_template = '''你是一个信息提取助手。请只输出 JSON 格式,不要包含任何解释。\n",
"\n",
"输出格式如下:\n",
"{{\\n\n",
" \"names\": [...],\\n\n",
" \"locations\": [...],\\n\n",
" \"time\": \"\"\\n\n",
"}}'''\n",
"\n",
"prompt = ChatPromptTemplate.from_messages([\n",
" ('system', system_template),\n",
" ('user', '文本:{text}')\n",
"])\n",
"\n",
"chain = prompt | llm | JsonOutputParser()\n",
"\n",
"result = chain.invoke({\n",
" 'text': '2024年5月1日李明和王芳一起去了北京故宫参观。'\n",
"})\n",
"\n",
"print('返回类型:', type(result))\n",
"print('返回内容:', result)"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"### 代码解释\n",
"\n",
"- `system_template` 中的 `{{` 和 `}}` 是**LangChain 模板转义语法**,表示最终 Prompt 中显示为单个 `{` 和 `}`\n",
"- `JsonOutputParser()` 会调用 `json.loads()` 解析模型输出\n",
"- 如果模型输出不是合法 JSON会抛出异常\n",
"\n",
"返回类型是 `dict`,可以直接像 `result['names']` 这样访问字段。"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## 4. PydanticOutputParser解析为 Pydantic 对象\n",
"\n",
"如果你需要更严格的类型校验和结构化数据,可以使用 `PydanticOutputParser`。\n",
"\n",
"Pydantic 是 Python 中非常流行的数据验证库。通过定义数据模型,可以:\n",
"- 明确每个字段的类型\n",
"- 自动验证数据格式\n",
"- 把模型输出转换成可操作的 Python 对象"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"from langchain_openai import ChatOpenAI\n",
"from langchain_core.prompts import ChatPromptTemplate\n",
"from langchain_core.output_parsers import PydanticOutputParser\n",
"from pydantic import BaseModel, Field\n",
"\n",
"llm = ChatOpenAI(model='qwen3.6-35b-A3b', temperature=0.1)\n",
"\n",
"# 第一步:定义 Pydantic 数据模型\n",
"class PersonInfo(BaseModel):\n",
" name: str = Field(description='人物姓名')\n",
" age: int = Field(description='人物年龄')\n",
" city: str = Field(description='所在城市')\n",
" hobbies: list[str] = Field(description='兴趣爱好列表')\n",
"\n",
"# 第二步:创建解析器\n",
"parser = PydanticOutputParser(pydantic_object=PersonInfo)\n",
"\n",
"# 第三步:在 Prompt 中嵌入格式说明\n",
"prompt = ChatPromptTemplate.from_messages([\n",
" ('system', '''你是一个信息提取助手。请从文本中提取人物信息,按指定格式输出。\\n\n",
"{format_instructions}'''),\n",
" ('user', '文本:{text}')\n",
"])\n",
"\n",
"# format_instructions 会自动生成 JSON Schema 说明\n",
"prompt_with_format = prompt.partial(format_instructions=parser.get_format_instructions())\n",
"\n",
"# 构建链\n",
"chain = prompt_with_format | llm | parser\n",
"\n",
"result = chain.invoke({\n",
" 'text': '张三今年25岁住在杭州喜欢打篮球和编程。'\n",
"})\n",
"\n",
"print('返回类型:', type(result))\n",
"print('返回对象:', result)\n",
"print('姓名:', result.name)\n",
"print('年龄:', result.age)"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"### 代码解释\n",
"\n",
"| 步骤 | 代码 | 说明 |\n",
"| --- | --- | --- |\n",
"| 1 | `class PersonInfo(BaseModel)` | 定义数据结构,每个字段都有类型和描述 |\n",
"| 2 | `PydanticOutputParser(pydantic_object=PersonInfo)` | 创建解析器,指定要解析的模型 |\n",
"| 3 | `parser.get_format_instructions()` | 自动生成格式说明,告诉模型如何输出 |\n",
"| 4 | `prompt.partial(format_instructions=...)` | 把格式说明预先填充到 Prompt 中 |\n",
"| 5 | `chain.invoke(...)` | 返回的是 `PersonInfo` 对象 |\n",
"\n",
"Pydantic 会自动校验类型,比如 `age` 必须是整数,否则报错。"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## 5. CommaSeparatedListOutputParser解析逗号分隔列表\n",
"\n",
"当你需要模型返回一个列表时,可以使用 `CommaSeparatedListOutputParser`。\n",
"\n",
"它适合的场景:\n",
"- 提取关键词\n",
"- 生成待办事项\n",
"- 输出多个选项"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"from langchain_openai import ChatOpenAI\n",
"from langchain_core.prompts import ChatPromptTemplate\n",
"from langchain_core.output_parsers import CommaSeparatedListOutputParser\n",
"\n",
"llm = ChatOpenAI(model='qwen3.6-35b-A3b', temperature=0.7)\n",
"\n",
"# 创建解析器\n",
"parser = CommaSeparatedListOutputParser()\n",
"\n",
"# 获取格式说明\n",
"format_instructions = parser.get_format_instructions()\n",
"\n",
"prompt = ChatPromptTemplate.from_messages([\n",
" ('system', '你是一个关键词提取助手。{format_instructions}'),\n",
" ('user', '请从以下文本中提取3-5个关键词{text}')\n",
"])\n",
"\n",
"# 预填充格式说明\n",
"prompt_with_format = prompt.partial(format_instructions=format_instructions)\n",
"\n",
"chain = prompt_with_format | llm | parser\n",
"\n",
"result = chain.invoke({\n",
" 'text': '人工智能、机器学习和深度学习正在改变各个行业的运作方式。'\n",
"})\n",
"\n",
"print('返回类型:', type(result))\n",
"print('关键词列表:', result)\n",
"print('第一个关键词:', result[0])"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"### 代码解释\n",
"\n",
"- `CommaSeparatedListOutputParser()` 会自动生成格式说明:「用逗号分隔各项,不要带编号和多余解释」\n",
"- `parser.get_format_instructions()` 返回一段英文说明文本\n",
"- 返回结果是 Python 列表 `list[str]`\n",
"- 可以直接用索引访问,比如 `result[0]`"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## 6. 处理格式异常:手动 try-except + LLM 修复\n",
"\n",
"大模型有时不会严格按 JSON 格式输出,可能包含多余文字、缺失引号或注释。\n",
"\n",
"`手动 try-except + LLM 修复` 可以包装另一个解析器,当第一次解析失败时,自动调用 LLM 修复输出格式。"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"from langchain_openai import ChatOpenAI\n",
"from langchain_core.prompts import ChatPromptTemplate\n",
"from langchain_core.output_parsers import JsonOutputParser\n",
"import json\n",
"\n",
"llm = ChatOpenAI(model='qwen3.6-35b-A3b', temperature=0.1)\n",
"\n",
"# 定义一个 JSON 解析器\n",
"parser = JsonOutputParser()\n",
"\n",
"# 提示模型输出 JSON但不强制约束格式\n",
"prompt = ChatPromptTemplate.from_messages([\n",
" ('system', '请把用户信息转换成 JSON 格式输出。'),\n",
" ('user', '内容:{text}')\n",
"])\n",
"\n",
"# 先获取模型的原始文本输出\n",
"raw_chain = prompt | llm\n",
"raw_output = raw_chain.invoke({'text': '姓名李四年龄30城市上海'})\n",
"\n",
"print('===== 原始模型输出 =====')\n",
"print(raw_output.content)\n",
"\n",
"# 尝试解析 JSON\n",
"try:\n",
" result = parser.invoke(raw_output)\n",
" print('\\n===== 直接解析成功 =====')\n",
" print(result)\n",
"except Exception as e:\n",
" print('\\n===== 解析失败,尝试用 LLM 修复 =====')\n",
" print('错误信息:', e)\n",
"\n",
" # 构造修复 Prompt让模型把错误输出修正为合法 JSON\n",
" fix_prompt = ChatPromptTemplate.from_messages([\n",
" ('system', '你是一个 JSON 修复专家。请把用户提供的内容修正为合法 JSON只输出 JSON 字符串,不要任何解释。'),\n",
" ('user', '原始内容:\\n{raw_output}\\n\\n错误信息{error}')\n",
" ])\n",
"\n",
" fix_chain = fix_prompt | llm\n",
" fixed_output = fix_chain.invoke({\n",
" 'raw_output': raw_output.content,\n",
" 'error': str(e)\n",
" })\n",
"\n",
" # 手动用 json.loads 解析修复后的内容\n",
" result = json.loads(fixed_output.content)\n",
" print('\\n===== 修复后的结果 =====')\n",
" print(result)"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"### 代码解释\n",
"\n",
"- `手动 try-except + LLM 修复.from_llm(parser=parser, llm=llm)` 用同一个 LLM 修复格式错误\n",
"- 修复逻辑:第一次解析失败 → 把原始输出和错误信息传给 LLM → 请求模型修正为合法格式 → 再次解析\n",
"- 适合用于对稳定性要求较高的场景\n",
"- 注意:修复不一定 100% 成功,极端情况下仍会报错"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## 7. 自定义输出解析器\n",
"\n",
"如果内置解析器不能满足需求,你可以继承 `BaseOutputParser` 自己实现。\n",
"\n",
"下面是一个简单的自定义解析器:把模型输出按指定分隔符切分成列表。"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"from langchain_openai import ChatOpenAI\n",
"from langchain_core.prompts import ChatPromptTemplate\n",
"from langchain_core.output_parsers import BaseOutputParser\n",
"\n",
"llm = ChatOpenAI(model='qwen3.6-35b-A3b', temperature=0.7)\n",
"\n",
"# 自定义解析器:按换行符切分,并去除空行和序号\n",
"class NumberedListParser(BaseOutputParser[list[str]]):\n",
" \"\"\"把模型输出的编号列表解析成字符串列表。\"\"\"\n",
"\n",
" def parse(self, text: str) -> list[str]:\n",
" # 按行切分\n",
" lines = text.strip().split('\\n')\n",
" # 去除空行,并去掉每行开头的 1. 2. 3. 等序号\n",
" items = []\n",
" for line in lines:\n",
" line = line.strip()\n",
" if not line:\n",
" continue\n",
" # 去掉行首的数字和点,例如 \"1. \"\n",
" if '. ' in line[:4]:\n",
" line = line.split('. ', 1)[1]\n",
" items.append(line)\n",
" return items\n",
"\n",
"prompt = ChatPromptTemplate.from_messages([\n",
" ('system', '你是一个清单生成助手。请输出编号列表,每行一个项目。'),\n",
" ('user', '请列出学习{topic}的5个步骤')\n",
"])\n",
"\n",
"chain = prompt | llm | NumberedListParser()\n",
"\n",
"result = chain.invoke({'topic': 'Python 编程'})\n",
"\n",
"print('返回类型:', type(result))\n",
"print('返回内容:', result)\n",
"print('步骤数量:', len(result))"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"### 代码解释\n",
"\n",
"- 继承 `BaseOutputParser[T]`,泛型 `T` 表示返回类型\n",
"- 必须实现 `parse(self, text: str) -> T` 方法\n",
"- `text` 是模型输出的字符串(已经过 `StrOutputParser` 提取 `.content`\n",
"- 可以在 `parse` 中做任何字符串处理:切分、清洗、正则匹配、校验等\n",
"\n",
"自定义解析器的优势是完全可控,适合处理模型输出不规范的情况。"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## 8. 完整示例:结构化数据提取\n",
"\n",
"下面是一个综合示例:从一段商品评价中同时提取评分、优点、缺点和关键词。"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"from langchain_openai import ChatOpenAI\n",
"from langchain_core.prompts import ChatPromptTemplate\n",
"from langchain_core.output_parsers import PydanticOutputParser\n",
"from pydantic import BaseModel, Field\n",
"\n",
"llm = ChatOpenAI(model='qwen3.6-35b-A3b', temperature=0.1)\n",
"\n",
"class Review(BaseModel):\n",
" sentiment: str = Field(description='整体情感,正面/负面/中性')\n",
" pros: list[str] = Field(description='优点列表')\n",
" cons: list[str] = Field(description='缺点列表')\n",
" keywords: list[str] = Field(description='关键词列表')\n",
"\n",
"parser = PydanticOutputParser(pydantic_object=Review)\n",
"\n",
"prompt = ChatPromptTemplate.from_messages([\n",
" ('system', '''你是一个商品评价分析助手。请从评价中提取结构化信息。\\n\n",
"{format_instructions}\\n\n",
"只输出 JSON不要其他内容。'''),\n",
" ('user', '商品评价:{review}')\n",
"])\n",
"\n",
"prompt_with_format = prompt.partial(format_instructions=parser.get_format_instructions())\n",
"\n",
"chain = prompt_with_format | llm | parser\n",
"\n",
"review_text = '''这款手机外观很漂亮,拍照效果也不错。\n",
"但是电池续航一般,充电速度有点慢。\n",
"总体来说性价比还可以。'''\n",
"\n",
"result = chain.invoke({'review': review_text})\n",
"\n",
"print('情感:', result.sentiment)\n",
"print('优点:', result.pros)\n",
"print('缺点:', result.cons)\n",
"print('关键词:', result.keywords)"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## 9. 常见解析器对比\n",
"\n",
"| 解析器 | 返回类型 | 适用场景 |\n",
"| --- | --- | --- |\n",
"| `StrOutputParser` | `str` | 只需要文本内容 |\n",
"| `JsonOutputParser` | `dict` | 需要 JSON 字典 |\n",
"| `PydanticOutputParser` | Pydantic 对象 | 需要类型校验和结构化对象 |\n",
"| `CommaSeparatedListOutputParser` | `list[str]` | 逗号分隔的列表 |\n",
"| `手动 try-except + LLM 修复` | 依赖被包装的解析器 | 自动修复格式错误 |\n",
"| 自定义 `BaseOutputParser` | 任意类型 | 特殊格式或复杂后处理 |"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## 10. 本节课练习\n",
"\n",
"1. 使用 `StrOutputParser()` 构建一个链,让模型介绍一门编程语言,验证返回类型是否为 `str`\n",
"2. 使用 `JsonOutputParser()` 从一段地址文本中提取省、市、区、街道,输出为 JSON 字典\n",
"3. 定义一个 Pydantic 模型 `BookInfo`(书名、作者、出版年份、类型),用 `PydanticOutputParser` 从文本中提取图书信息\n",
"4. 使用 `CommaSeparatedListOutputParser()` 让模型列出学习 LangChain 的 5 个关键知识点\n",
"5. 尝试让模型故意输出不规范的 JSON然后用 `手动 try-except + LLM 修复` 包装 `JsonOutputParser` 观察是否能修复"
]
}
],
"metadata": {
"kernelspec": {
"display_name": ".venv",
"language": "python",
"name": "python3"
},
"language_info": {
"codemirror_mode": {
"name": "ipython",
"version": 3
},
"file_extension": ".py",
"mimetype": "text/x-python",
"name": "python",
"nbconvert_exporter": "python",
"pygments_lexer": "ipython3",
"version": "3.11.3"
}
},
"nbformat": 4,
"nbformat_minor": 4
}