{ "metadata": { "id": "ch22", "title": "第22章：错误处理与重试策略", "volume": "vol7", "volume_title": "Agent编程技法", "word_count": 1462, "difficulty": "intermediate", "prerequisites": [ "ch06" ], "key_concepts": [ "引言", "本章学习目标", "错误分类体系", "Agent错误分类", "错误严重等级", "LLM输出解析错误处理", "JSON解析失败的处理", "结构化输出重试", "工具调用失败处理", "工具执行器", "TypeScript工具执行器", "超时与降级策略", "分层超时设计", "断路器模式", "指数退避重试" ], "learning_objectives": [], "estimated_tokens": 877, "source_file": "vol7/ch22_错误处理与重试策略.md" }, "overview": "", "sections": [ { "id": "22.1", "title": "22.1 引言", "level": 2, "content": "在传统软件中，错误是例外（Exception）。在 Agent 系统中，错误是常态（Norm）。LLM 的输出不确定性、外部 API 的不可靠性、网络波动、Token 超限——这些\"故障\"时刻都在发生。\n\n一个健壮的 Agent 系统不是不犯错，而是犯了错能优雅地恢复。本章将系统讲解 Agent 系统中的错误处理策略，从 LLM 输出解析到工具调用失败，从超时降级到指数退避，构建一个\"不怕失败\"的容错体系。", "subsections": [ { "id": "本章学习目标", "title": "本章学习目标", "content": "- 理解 Agent 系统中错误的分类和特征\n- 掌握 LLM 输出解析错误的处理方法\n- 实现工具调用失败的容错机制\n- 设计超时与降级策略\n- 构建指数退避重试框架\n- 建立错误日志与报警体系\n\n---" } ] }, { "id": "22.2", "title": "22.2 错误分类体系", "level": 2, "content": "", "subsections": [ { "id": "22.2.1", "title": "22.2.1 Agent错误分类", "content": "" }, { "id": "22.2.2", "title": "22.2.2 错误严重等级", "content": "---" } ] }, { "id": "22.3", "title": "22.3 LLM输出解析错误处理", "level": 2, "content": "", "subsections": [ { "id": "22.3.1", "title": "22.3.1 JSON解析失败的处理", "content": "LLM 返回的 JSON 经常格式不完美——缺少引号、多余逗号、混合了自然语言等：\n\n{\n \"sentiment\": \"positive\",\n \"confidence\": 0.85,\n \"reason\": \"用户表达了满意\",\n // 这是一个注释\n}" }, { "id": "22.3.2", "title": "22.3.2 结构化输出重试", "content": "---" } ] }, { "id": "22.4", "title": "22.4 工具调用失败处理", "level": 2, "content": "", "subsections": [ { "id": "22.4.1", "title": "22.4.1 工具执行器", "content": "" }, { "id": "22.4.2", "title": "22.4.2 TypeScript工具执行器", "content": "---" } ] }, { "id": "22.5", "title": "22.5 超时与降级策略", "level": 2, "content": "", "subsections": [ { "id": "22.5.1", "title": "22.5.1 分层超时设计", "content": "" }, { "id": "22.5.2", "title": "22.5.2 断路器模式", "content": "---" } ] }, { "id": "22.6", "title": "22.6 指数退避重试", "level": 2, "content": "", "subsections": [ { "id": "22.6.1", "title": "22.6.1 指数退避实现", "content": "" }, { "id": "22.6.2", "title": "22.6.2 自适应退避", "content": "---" } ] }, { "id": "22.7", "title": "22.7 错误日志与报警", "level": 2, "content": "", "subsections": [ { "id": "22.7.1", "title": "22.7.1 结构化错误日志", "content": "" }, { "id": "22.7.2", "title": "22.7.2 错误仪表板数据结构", "content": "---" } ] }, { "id": "22.8", "title": "22.8 最佳实践", "level": 2, "content": "", "subsections": [ { "id": "22.8.1", "title": "22.8.1 错误处理策略清单", "content": "| 场景 | 策略 | 实现方式 |\n|------|------|----------|\n| LLM输出格式错误 | 重试 + 自修复 | OutputParser + StructuredOutputRetry |\n| 工具API失败 | 指数退避重试 | ExponentialBackoff |\n| 工具API持续故障 | 断路器 | CircuitBreaker |\n| 服务降级 | 多级降级 | GracefulDegradation |\n| 超时 | 分层超时 | TimeoutConfig |\n| 用户输入无效 | 友好提示 | 输入验证 + 引导 |\n| 未知错误 | 降级响应 + 日志 | try-catch + AgentErrorLogger |" }, { "id": "22.8.2", "title": "22.8.2 常见陷阱", "content": "**陷阱1：吞掉错误**\n\n\n**陷阱2：无限重试**\n\n\n**陷阱3：重试非幂等操作**\n\n\n---" } ] }, { "id": "22.9", "title": "22.9 本章小结", "level": 2, "content": "本章系统介绍了 Agent 系统的错误处理策略：\n\n1. **错误分类**是第一步——理解错误类型才能对症下药\n2. **输出容错**（JSON修复、Schema验证）应对 LLM 的不确定性\n3. **工具执行器**（超时、重试、降级）是工具调用的安全网\n4. **断路器**防止故障级联扩散\n5. **指数退避**避免雷群效应\n6. **结构化日志**是故障诊断的基础\n\n> **记住**：在 Agent 系统中，假设一切都会失败。你的任务是确保失败时的行为是可预测和可恢复的。", "subsections": [] } ], "code_blocks": [ { "id": "code-1", "language": "text", "description": "", "code": "Agent 错误\n├── 输入错误\n│ ├── 用户输入无效/不完整\n│ ├── Prompt 注入\n│ └── 输入超长（Token 超限）\n├── 模型错误\n│ ├── 输出格式不符（JSON 解析失败等）\n│ ├── 输出内容偏离（幻觉、无关内容）\n│ ├── 输出被截断（max_tokens 不足）\n│ └── 模型拒绝回答（安全过滤）\n├── 工具错误\n│ ├── API 调用失败（网络、认证、限流）\n│ ├── 工具参数错误\n│ ├── 工具执行超时\n│ └── 工具返回无效数据\n├── 系统错误\n│ ├── 内存不足\n│ ├── 并发超限\n│ ├── 服务不可用\n│ └── 配置错误\n└── 业务错误\n ├── 权限不足\n ├── 数据不存在\n ├── 业务规则冲突\n └── 状态不一致", "section_ref": "22.2.1", "runnable": false, "dependencies": [] }, { "id": "code-2", "language": "python", "description": "", "code": "from enum import Enum\n\n\nclass ErrorSeverity(Enum):\n \"\"\"错误严重等级\"\"\"\n LOW = \"low\" # 可忽略，不影响主流程\n MEDIUM = \"medium\" # 影响部分功能，可降级处理\n HIGH = \"high\" # 影响主流程，需要重试或降级\n CRITICAL = \"critical\" # 系统级故障，需要立即干预\n\n\nclass AgentError:\n \"\"\"统一的错误类\"\"\"\n \n def __init__(\n self,\n error_type: str,\n message: str,\n severity: ErrorSeverity = ErrorSeverity.MEDIUM,\n recoverable: bool = True,\n context: dict = None,\n cause: Exception = None,\n ):\n self.error_type = error_type\n self.message = message\n self.severity = severity\n self.recoverable = recoverable\n self.context = context or {}\n self.cause = cause\n self.timestamp = __import__('time').time()\n \n def __str__(self):\n return f\"[{self.severity.value}] {self.error_type}: {self.message}\"\n \n def to_dict(self) -> dict:\n return {\n \"type\": self.error_type,\n \"message\": self.message,\n \"severity\": self.severity.value,\n \"recoverable\": self.recoverable,\n \"context\": self.context,\n \"cause\": str(self.cause) if self.cause else None,\n }", "section_ref": "22.2.2", "runnable": true, "dependencies": [] }, { "id": "code-3", "language": "python", "description": "LLM 返回的 JSON 经常格式不完美——缺少引号、多余逗号、混合了自然语言等：", "code": "import json\nimport re\nfrom typing import Any, TypeVar, Type\n\n\nT = TypeVar('T')\n\n\nclass OutputParser:\n \"\"\"LLM 输出解析器——容错版\"\"\"\n \n @staticmethod\n def parse_json(text: str, strict: bool = False) -> dict:\n \"\"\"\n 解析 LLM 输出的 JSON，支持多种容错模式。\n \n 容错策略按优先级：\n 1. 直接解析\n 2. 提取JSON代码块\n 3. 修复常见格式问题\n 4. 降级为结构化文本\n \"\"\"\n # 策略1：直接解析\n try:\n return json.loads(text)\n except json.JSONDecodeError:\n pass\n \n if strict:\n raise ValueError(f\"严格的JSON解析失败: {text[:200]}\")\n \n # 策略2：提取 ```json ... ``` 代码块\n json_match = re.search(r'```(?:json)?\\s*\\n?(.*?)\\n?\\s*```', text, re.DOTALL)\n if json_match:\n try:\n return json.loads(json_match.group(1))\n except json.JSONDecodeError:\n pass\n \n # 策略3：提取 { ... } 或 [ ... ]\n brace_match = re.search(r'\\{.*\\}', text, re.DOTALL)\n bracket_match = re.search(r'\\[.*\\]', text, re.DOTALL)\n \n for match in [brace_match, bracket_match]:\n if match:\n try:\n return json.loads(match.group(0))\n except json.JSONDecodeError:\n cleaned = OutputParser._fix_common_issues(match.group(0))\n try:\n return json.loads(cleaned)\n except json.JSONDecodeError:\n pass\n \n # 策略4：降级为键值对提取\n return OutputParser._fallback_parse(text)\n \n @staticmethod\n def _fix_common_issues(text: str) -> str:\n \"\"\"修复常见的JSON格式问题\"\"\"\n # 移除注释（// ...）\n text = re.sub(r'//.*', '', text)\n # 移除尾随逗号\n text = re.sub(r',\\s*([}\\]])', r'\\1', text)\n # 单引号转双引号\n text = re.sub(r\"'\", '\"', text)\n # None -> null\n text = re.sub(r'\\bNone\\b', 'null', text)\n # True/False -> true/false\n text = re.sub(r'\\bTrue\\b', 'true', text)\n re.sub(r'\\bFalse\\b', 'false', text)\n return text\n \n @staticmethod\n def _fallback_parse(text: str) -> dict:\n \"\"\"降级解析——提取键值对\"\"\"\n result = {}\n # 匹配 key: value 或 key=value\n pairs = re.findall(r'[\"\\']?(\\w+)[\"\\']?\\s*[:=]\\s*[\"\\']?([^\"\\',\\n]+)[\"\\']?', text)\n for key, value in pairs:\n result[key.strip()] = value.strip()\n return result if result else {\"raw_text\": text}\n \n @staticmethod\n def parse_with_schema(text: str, schema: dict, max_retries: int = 2) -> dict:\n \"\"\"\n 使用Schema约束解析。如果直接解析失败，\n 使用schema信息尝试修复。\n \"\"\"\n result = OutputParser.parse_json(text)\n \n if not OutputParser._validate_schema(result, schema):\n # 尝试用schema的默认值补全缺失字段\n for key, config in schema.get(\"properties\", {}).items():\n if key not in result and \"default\" in config:\n result[key] = config[\"default\"]\n \n if not OutputParser._validate_schema(result, schema):\n raise ValueError(\n f\"输出不符合Schema。期望字段: {list(schema.get('properties', {}).keys())}，\"\n f\"实际字段: {list(result.keys())}\"\n )\n \n return result\n \n @staticmethod\n def _validate_schema(data: dict, schema: dict) -> bool:\n \"\"\"简易Schema验证\"\"\"\n required = schema.get(\"required\", [])\n return all(k in data for k in required)\n\n\n# 使用示例\nraw_output = \"\"\"\n让我分析一下这个请求：\n", "section_ref": "22.3.1", "runnable": true, "dependencies": [] }, { "id": "code-4", "language": "text", "description": "}", "code": "\"\"\"\n\nresult = OutputParser.parse_json(raw_output)\nprint(result) # {\"sentiment\": \"positive\", \"confidence\": 0.85, \"reason\": \"用户表达了满意\"}", "section_ref": "22.3.1", "runnable": false, "dependencies": [] }, { "id": "code-5", "language": "python", "description": "", "code": "from typing import Callable, Optional\n\n\nclass StructuredOutputRetry:\n \"\"\"结构化输出重试器\"\"\"\n \n def __init__(self, llm, max_retries: int = 3):\n self.llm = llm\n self.max_retries = max_retries\n \n def call(\n self,\n prompt: str,\n output_schema: dict,\n fix_hint: str = None,\n ) -> dict:\n \"\"\"调用LLM并确保返回结构化输出\"\"\"\n last_error = None\n \n for attempt in range(self.max_retries):\n # 构建带Schema的Prompt\n schema_prompt = self._build_schema_prompt(prompt, output_schema)\n \n if attempt > 0 and last_error:\n schema_prompt += f\"\\n\\n注意：上次解析失败（{last_error}）。请严格按照JSON格式输出。\"\n \n raw = self.llm.generate(user=schema_prompt, temperature=0.1)\n \n try:\n result = OutputParser.parse_json(raw)\n if OutputParser._validate_schema(result, output_schema):\n return result\n last_error = f\"缺少必需字段\"\n except ValueError as e:\n last_error = str(e)\n \n # 所有重试失败，返回最佳尝试\n return {\"error\": \"无法获取结构化输出\", \"raw\": raw, \"attempts\": self.max_retries}\n \n def _build_schema_prompt(self, prompt: str, schema: dict) -> str:\n \"\"\"构建带Schema约束的Prompt\"\"\"\n schema_str = json.dumps(schema, ensure_ascii=False, indent=2)\n return f\"\"\"{prompt}\n\n请以JSON格式回复，Schema如下：\n{schema_str}\n\n只输出JSON，不要包含其他文字。\"\"\"", "section_ref": "22.3.2", "runnable": true, "dependencies": [] }, { "id": "code-6", "language": "python", "description": "", "code": "import time\nfrom typing import Any, Callable, Optional\nfrom dataclasses import dataclass, field\n\n\n@dataclass\nclass ToolResult:\n \"\"\"工具执行结果\"\"\"\n success: bool\n data: Any = None\n error: str = \"\"\n duration_ms: float = 0\n retries_used: int = 0\n\n\n@dataclass\nclass ToolConfig:\n \"\"\"工具配置\"\"\"\n name: str\n handler: Callable\n timeout_seconds: float = 30.0\n max_retries: int = 3\n retry_delay_seconds: float = 1.0\n required_params: list[str] = field(default_factory=list)\n fallback: Optional[Callable] = None # 降级函数\n\n\nclass ToolExecutor:\n \"\"\"工具执行器——带超时、重试和降级\"\"\"\n \n def __init__(self):\n self._tools: dict[str, ToolConfig] = {}\n self._execution_log: list[dict] = []\n \n def register(self, config: ToolConfig):\n self._tools[config.name] = config\n \n async def execute(self, tool_name: str, params: dict = None) -> ToolResult:\n config = self._tools.get(tool_name)\n if not config:\n return ToolResult(success=False, error=f\"工具 {tool_name} 未注册\")\n \n # 参数验证\n missing = [p for p in config.required_params if not params.get(p)]\n if missing:\n return ToolResult(\n success=False, \n error=f\"缺少必需参数: {missing}\"\n )\n \n last_error = \"\"\n retries = 0\n \n for attempt in range(config.max_retries + 1):\n try:\n t0 = time.time()\n \n # 异步执行（带超时）\n result = await self._execute_with_timeout(\n config.handler, params, config.timeout_seconds\n )\n \n duration = (time.time() - t0) * 1000\n \n self._log(tool_name, True, duration, retries)\n \n return ToolResult(\n success=True,\n data=result,\n duration_ms=duration,\n retries_used=retries,\n )\n \n except TimeoutError:\n last_error = f\"执行超时（{config.timeout_seconds}s）\"\n retries += 1\n \n except Exception as e:\n last_error = str(e)\n retries += 1\n \n if retries <= config.max_retries:\n await self._wait(config.retry_delay_seconds * (2 ** (retries - 1)))\n \n # 所有重试失败——尝试降级\n if config.fallback:\n try:\n fallback_result = await config.fallback(params)\n return ToolResult(\n success=True, data=fallback_result,\n error=f\"主工具失败，使用降级: {last_error}\",\n retries_used=retries,\n )\n except Exception as e:\n last_error = f\"降级也失败: {e}\"\n \n self._log(tool_name, False, 0, retries)\n \n return ToolResult(\n success=False,\n error=f\"工具 {tool_name} 执行失败（{retries}次重试后）: {last_error}\",\n retries_used=retries,\n )\n \n async def _execute_with_timeout(self, fn, params, timeout):\n \"\"\"带超时的异步执行\"\"\"\n import asyncio\n try:\n return await asyncio.wait_for(fn(params), timeout=timeout)\n except asyncio.TimeoutError:\n raise TimeoutError()\n \n @staticmethod\n async def _wait(seconds: float):\n import asyncio\n await asyncio.sleep(seconds)\n \n def _log(self, tool_name: str, success: bool, duration_ms: float, retries: int):\n self._execution_log.append({\n \"tool\": tool_name,\n \"success\": success,\n \"duration_ms\": duration_ms,\n \"retries\": retries,\n \"timestamp\": time.time(),\n })", "section_ref": "22.4.1", "runnable": true, "dependencies": [] }, { "id": "code-7", "language": "typescript", "description": "", "code": "interface ToolConfig {\n name: string;\n handler: (params: any) => Promise;\n timeoutMs: number;\n maxRetries: number;\n fallback?: (params: any) => Promise;\n}\n\ninterface ToolResult {\n success: boolean;\n data?: any;\n error?: string;\n durationMs: number;\n retriesUsed: number;\n}\n\nexport class ToolExecutor {\n private tools = new Map();\n\n register(config: ToolConfig): void {\n this.tools.set(config.name, config);\n }\n\n async execute(toolName: string, params: any): Promise {\n const config = this.tools.get(toolName);\n if (!config) return { success: false, error: `Tool ${toolName} not found`, durationMs: 0, retriesUsed: 0 };\n\n let lastError = \"\";\n for (let attempt = 0; attempt <= config.maxRetries; attempt++) {\n try {\n const t0 = Date.now();\n const result = await Promise.race([\n config.handler(params),\n new Promise((_, reject) => \n setTimeout(() => reject(new Error('Timeout')), config.timeoutMs)\n ),\n ]);\n return { success: true, data: result, durationMs: Date.now() - t0, retriesUsed: attempt };\n } catch (e: any) {\n lastError = e.message;\n if (attempt < config.maxRetries) {\n await new Promise(r => setTimeout(r, 1000 * Math.pow(2, attempt)));\n }\n }\n }\n\n // Fallback\n if (config.fallback) {\n try {\n const data = await config.fallback(params);\n return { success: true, data, error: `Fallback used: ${lastError}`, durationMs: 0, retriesUsed: config.maxRetries };\n } catch {}\n }\n\n return { success: false, error: lastError, durationMs: 0, retriesUsed: config.maxRetries };\n }\n}", "section_ref": "22.4.2", "runnable": true, "dependencies": [] }, { "id": "code-8", "language": "python", "description": "", "code": "class TimeoutConfig:\n \"\"\"分层超时配置\"\"\"\n # 整体请求超时\n REQUEST_TIMEOUT = 60.0 # 秒\n \n # LLM 调用超时\n LLM_CALL_TIMEOUT = 30.0\n \n # 工具执行超时\n TOOL_EXECUTION_TIMEOUT = 15.0\n \n # 外部 API 超时\n EXTERNAL_API_TIMEOUT = 10.0\n \n # 流式响应首个 Token 超时\n FIRST_TOKEN_TIMEOUT = 5.0\n\n\nclass GracefulDegradation:\n \"\"\"优雅降级策略\"\"\"\n \n def __init__(self):\n self._strategies: dict[str, list[Callable]] = {}\n \n def register(self, service: str, strategies: list[Callable]):\n \"\"\"\n 注册降级策略，按优先级排列。\n 第一个是首选策略，最后是最终降级方案。\n \"\"\"\n self._strategies[service] = strategies\n \n async def call(self, service: str, **kwargs) -> any:\n \"\"\"尝试调用服务，失败时按策略降级\"\"\"\n strategies = self._strategies.get(service, [])\n \n if not strategies:\n raise RuntimeError(f\"服务 {service} 没有注册任何策略\")\n \n errors = []\n for i, strategy in enumerate(strategies):\n try:\n result = await strategy(**kwargs)\n if i > 0:\n print(f\"[降级] 服务 {service} 使用了第 {i+1} 降级策略\")\n return result\n except Exception as e:\n errors.append(str(e))\n continue\n \n raise RuntimeError(\n f\"服务 {service} 所有策略均失败: {errors}\"\n )\n\n\n# 使用示例\ndegradation = GracefulDegradation()\n\ndegradation.register(\"search\", [\n # 策略1：实时搜索\n lambda query: realtime_search(query),\n # 策略2：缓存搜索\n lambda query: cached_search(query),\n # 策略3：返回提示信息\n lambda query: {\"results\": [], \"message\": \"搜索服务暂时不可用，请稍后重试\"},\n])\n\ndegradation.register(\"translation\", [\n lambda text, lang: llm_translate(text, lang), # LLM翻译\n lambda text, lang: rule_based_translate(text, lang), # 规则翻译\n lambda text, lang: {\"text\": text, \"note\": \"翻译服务不可用，返回原文\"},\n])", "section_ref": "22.5.1", "runnable": true, "dependencies": [] }, { "id": "code-9", "language": "python", "description": "", "code": "import time\nfrom enum import Enum\nfrom collections import deque\n\n\nclass CircuitState(Enum):\n CLOSED = \"closed\" # 正常工作\n OPEN = \"open\" # 断开（拒绝请求）\n HALF_OPEN = \"half_open\" # 半开（试探性放行）\n\n\nclass CircuitBreaker:\n \"\"\"\n 断路器——防止故障级联扩散。\n \n 工作原理：\n 1. CLOSED（关闭）：正常转发请求，统计失败次数\n 2. OPEN（打开）：失败次数超过阈值，拒绝请求\n 3. HALF_OPEN（半开）：超时后尝试放行一个请求，\n 成功则恢复，失败则继续打开\n \"\"\"\n \n def __init__(\n self,\n failure_threshold: int = 5, # 触发断路的失败次数\n recovery_timeout: float = 30.0, # 断路恢复超时（秒）\n half_open_max_calls: int = 3, # 半开状态下最多试探请求数\n window_seconds: float = 60.0, # 统计窗口\n ):\n self.failure_threshold = failure_threshold\n self.recovery_timeout = recovery_timeout\n self.half_open_max_calls = half_open_max_calls\n self.window_seconds = window_seconds\n \n self._state = CircuitState.CLOSED\n self._failure_count = 0\n self._success_count = 0\n self._last_failure_time: float = 0\n self._half_open_calls = 0\n self._window: deque = deque() # (timestamp, success)\n \n @property\n def state(self) -> CircuitState:\n self._check_state()\n return self._state\n \n def _check_state(self):\n \"\"\"检查是否应该从OPEN转到HALF_OPEN\"\"\"\n if self._state == CircuitState.OPEN:\n if time.time() - self._last_failure_time > self.recovery_timeout:\n self._state = CircuitState.HALF_OPEN\n self._half_open_calls = 0\n \n def _prune_window(self):\n \"\"\"清理过期的统计窗口\"\"\"\n cutoff = time.time() - self.window_seconds\n while self._window and self._window[0][0] < cutoff:\n self._window.popleft()\n \n def allow_request(self) -> bool:\n \"\"\"判断是否允许请求通过\"\"\"\n state = self.state\n \n if state == CircuitState.CLOSED:\n return True\n \n if state == CircuitState.HALF_OPEN:\n if self._half_open_calls < self.half_open_max_calls:\n self._half_open_calls += 1\n return True\n return False\n \n return False # OPEN\n \n def record_success(self):\n \"\"\"记录成功\"\"\"\n self._window.append((time.time(), True))\n self._success_count += 1\n \n if self._state == CircuitState.HALF_OPEN:\n # 半开状态下成功，恢复\n self._state = CircuitState.CLOSED\n self._failure_count = 0\n \n def record_failure(self):\n \"\"\"记录失败\"\"\"\n self._window.append((time.time(), False))\n self._failure_count += 1\n self._last_failure_time = time.time()\n \n self._prune_window()\n recent_failures = sum(1 for _, s in self._window if not s)\n \n if recent_failures >= self.failure_threshold:\n self._state = CircuitState.OPEN\n \n if self._state == CircuitState.HALF_OPEN:\n # 半开状态下失败，回到OPEN\n self._state = CircuitState.OPEN\n \n def get_stats(self) -> dict:\n self._prune_window()\n return {\n \"state\": self._state.value,\n \"failure_count\": self._failure_count,\n \"success_count\": self._success_count,\n \"window_size\": len(self._window),\n \"recent_failures\": sum(1 for _, s in self._window if not s),\n }\n\n\n# 使用示例\nbreaker = CircuitBreaker(failure_threshold=3, recovery_timeout=10)\n\nasync def call_with_breaker(fn, *args, **kwargs):\n if not breaker.allow_request():\n return {\"error\": \"服务暂时不可用（断路器已打开）\", \"fallback\": True}\n \n try:\n result = await fn(*args, **kwargs)\n breaker.record_success()\n return result\n except Exception as e:\n breaker.record_failure()\n raise", "section_ref": "22.5.2", "runnable": true, "dependencies": [] }, { "id": "code-10", "language": "python", "description": "", "code": "import random\nimport asyncio\nfrom dataclasses import dataclass\n\n\n@dataclass\nclass RetryResult:\n success: bool\n data: any = None\n error: str = \"\"\n attempts: int = 0\n total_delay_ms: float = 0\n\n\nclass ExponentialBackoff:\n \"\"\"\n 指数退避重试器。\n \n 退避公式: delay = min(base * 2^attempt + jitter, max_delay)\n \n 支持以下特性：\n - 指数增长延迟\n - 随机抖动（Jitter）避免雷群效应\n - 可配置的退避参数\n - 可重试错误过滤\n \"\"\"\n \n def __init__(\n self,\n max_retries: int = 5,\n base_delay: float = 1.0,\n max_delay: float = 60.0,\n jitter: bool = True,\n retryable_errors: tuple = None,\n ):\n self.max_retries = max_retries\n self.base_delay = base_delay\n self.max_delay = max_delay\n self.jitter = jitter\n self.retryable_errors = retryable_errors or (\n TimeoutError, ConnectionError, \n )\n \n def calculate_delay(self, attempt: int) -> float:\n \"\"\"计算第N次重试的延迟\"\"\"\n delay = self.base_delay * (2 ** attempt)\n delay = min(delay, self.max_delay)\n \n if self.jitter:\n # 全抖动：在 [0, delay] 范围内随机\n delay = random.uniform(0, delay)\n \n return delay\n \n def should_retry(self, error: Exception) -> bool:\n \"\"\"判断错误是否应该重试\"\"\"\n return isinstance(error, self.retryable_errors)\n \n async def execute(self, fn, *args, **kwargs) -> RetryResult:\n \"\"\"执行带重试的操作\"\"\"\n total_delay = 0\n \n for attempt in range(self.max_retries + 1):\n try:\n result = await fn(*args, **kwargs)\n return RetryResult(\n success=True, data=result,\n attempts=attempt + 1, total_delay_ms=total_delay * 1000,\n )\n except Exception as e:\n if attempt == self.max_retries or not self.should_retry(e):\n return RetryResult(\n success=False, error=str(e),\n attempts=attempt + 1, total_delay_ms=total_delay * 1000,\n )\n \n delay = self.calculate_delay(attempt)\n total_delay += delay\n print(f\"[重试] 第 {attempt + 1} 次失败 ({e})，\"\n f\"{delay:.1f}s 后重试...\")\n await asyncio.sleep(delay)\n \n def get_schedule(self) -> list[float]:\n \"\"\"获取重试时间表（预览）\"\"\"\n return [self.calculate_delay(i) for i in range(self.max_retries)]\n\n\n# 使用示例\nbackoff = ExponentialBackoff(max_retries=5, base_delay=1.0, max_delay=60)\nprint(f\"重试时间表: {backoff.get_schedule()}\")\n# [0.3, 1.7, 3.2, 7.8, 15.4] （带抖动，每次不同）", "section_ref": "22.6.1", "runnable": true, "dependencies": [] }, { "id": "code-11", "language": "python", "description": "", "code": "class AdaptiveBackoff:\n \"\"\"\n 自适应退避——根据服务器响应动态调整退避策略。\n \n 如果服务器返回 Retry-After 头，优先使用；\n 如果检测到限流（429），增加退避时间；\n 如果连续成功，逐步减少退避时间。\n \"\"\"\n \n def __init__(\n self,\n max_retries: int = 5,\n initial_delay: float = 1.0,\n max_delay: float = 120.0,\n backoff_factor: float = 2.0,\n ):\n self.max_retries = max_retries\n self.initial_delay = initial_delay\n self.max_delay = max_delay\n self.backoff_factor = backoff_factor\n self._current_delay = initial_delay\n self._consecutive_successes = 0\n \n def get_delay(self, response_headers: dict = None) -> float:\n \"\"\"计算下次重试的延迟\"\"\"\n # 优先使用服务器的 Retry-After\n if response_headers:\n retry_after = response_headers.get(\"Retry-After\")\n if retry_after:\n try:\n return float(retry_after)\n except ValueError:\n pass\n \n return min(self._current_delay, self.max_delay)\n \n def record_success(self):\n \"\"\"记录成功——逐步恢复初始延迟\"\"\"\n self._consecutive_successes += 1\n if self._consecutive_successes >= 3:\n self._current_delay = max(\n self.initial_delay, \n self._current_delay / self.backoff_factor\n )\n self._consecutive_successes = 0\n \n def record_failure(self, status_code: int = 0):\n \"\"\"记录失败——增加退避时间\"\"\"\n if status_code == 429: # 限流\n self._current_delay *= self.backoff_factor * 1.5\n else:\n self._current_delay *= self.backoff_factor\n \n self._consecutive_successes = 0", "section_ref": "22.6.2", "runnable": true, "dependencies": [] }, { "id": "code-12", "language": "python", "description": "", "code": "import logging\nimport json\nfrom datetime import datetime\n\n\nclass AgentErrorLogger:\n \"\"\"Agent 专用错误日志器\"\"\"\n \n def __init__(self, service_name: str):\n self.service_name = service_name\n self._logger = logging.getLogger(f\"agent.{service_name}\")\n self._error_counts: dict[str, int] = {}\n self._alert_thresholds: dict[str, tuple[int, float]] = {}\n # {error_type: (count_threshold, time_window_seconds)}\n self._last_alert_time: dict[str, float] = {}\n \n def set_alert_threshold(\n self, error_type: str, count: int, window_seconds: float = 300\n ):\n \"\"\"设置报警阈值\"\"\"\n self._alert_thresholds[error_type] = (count, window_seconds)\n \n def log_error(\n self,\n error_type: str,\n message: str,\n severity: str = \"medium\",\n context: dict = None,\n ):\n \"\"\"记录错误\"\"\"\n entry = {\n \"timestamp\": datetime.now().isoformat(),\n \"service\": self.service_name,\n \"error_type\": error_type,\n \"severity\": severity,\n \"message\": message,\n \"context\": context or {},\n }\n \n # 写入日志\n self._logger.error(json.dumps(entry, ensure_ascii=False))\n \n # 更新计数\n self._error_counts[error_type] = self._error_counts.get(error_type, 0) + 1\n \n # 检查报警\n self._check_alert(error_type)\n \n def _check_alert(self, error_type: str):\n \"\"\"检查是否需要报警\"\"\"\n if error_type not in self._alert_thresholds:\n return\n \n threshold, window = self._alert_thresholds[error_type]\n count = self._error_counts.get(error_type, 0)\n now = time.time()\n \n # 检查是否在冷却期\n last_alert = self._last_alert_time.get(error_type, 0)\n if now - last_alert < window:\n return\n \n if count >= threshold:\n self._send_alert(error_type, count, threshold)\n self._last_alert_time[error_type] = now\n self._error_counts[error_type] = 0\n \n def _send_alert(self, error_type: str, count: int, threshold: int):\n \"\"\"发送报警（实际中对接通知系统）\"\"\"\n alert_msg = (\n f\"🚨 [Agent报警] 服务: {self.service_name}\\n\"\n f\"错误类型: {error_type}\\n\"\n f\"发生次数: {count}（阈值: {threshold}）\\n\"\n f\"时间: {datetime.now().isoformat()}\"\n )\n print(alert_msg) # 实际中发送到 Slack/邮件/短信等", "section_ref": "22.7.1", "runnable": true, "dependencies": [] }, { "id": "code-13", "language": "python", "description": "", "code": "@dataclass\nclass ErrorMetrics:\n \"\"\"错误指标\"\"\"\n period: str\n total_errors: int = 0\n by_type: dict[str, int] = field(default_factory=dict)\n by_severity: dict[str, int] = field(default_factory=dict)\n top_errors: list[dict] = field(default_factory=list)\n \n def to_dashboard(self) -> dict:\n return {\n \"period\": self.period,\n \"total_errors\": self.total_errors,\n \"error_rate\": self._calc_error_rate(),\n \"by_type\": dict(sorted(\n self.by_type.items(), key=lambda x: -x[1]\n )[:10]),\n \"by_severity\": self.by_severity,\n \"top_errors\": self.top_errors[:5],\n }", "section_ref": "22.7.2", "runnable": true, "dependencies": [] }, { "id": "code-14", "language": "python", "description": "陷阱1：吞掉错误", "code": "# ❌ 吞掉所有错误\ntry:\n result = call_llm()\nexcept:\n result = \"出错了\"\n\n# ✅ 分类处理\ntry:\n result = call_llm()\nexcept LLMTimeoutError:\n result = fallback_response()\n logger.warning(\"LLM超时，使用降级\")\nexcept Exception as e:\n result = error_response()\n logger.error(f\"未知错误: {e}\")", "section_ref": "22.8.2", "runnable": true, "dependencies": [] }, { "id": "code-15", "language": "python", "description": "陷阱2：无限重试", "code": "# ❌ 无限重试\nwhile True:\n try:\n return call_api()\n except:\n pass\n\n# ✅ 有限重试 + 退避\nbackoff = ExponentialBackoff(max_retries=3)\nresult = await backoff.execute(call_api)", "section_ref": "22.8.2", "runnable": true, "dependencies": [] }, { "id": "code-16", "language": "python", "description": "陷阱3：重试非幂等操作", "code": "# ❌ 重试非幂等操作（可能导致重复扣款）\nbackoff.execute(process_payment, amount=100)\n\n# ✅ 非幂等操作需要先检查状态\nasync def safe_process_payment(params):\n if await check_payment_exists(params[\"order_id\"]):\n return {\"status\": \"already_processed\"}\n return await process_payment(params)", "section_ref": "22.8.2", "runnable": true, "dependencies": [] } ], "tables": [ { "headers": [ "场景", "策略", "实现方式" ], "data": [ [ "LLM输出格式错误", "重试 + 自修复", "OutputParser + StructuredOutputRetry" ], [ "工具API失败", "指数退避重试", "ExponentialBackoff" ], [ "工具API持续故障", "断路器", "CircuitBreaker" ], [ "服务降级", "多级降级", "GracefulDegradation" ], [ "超时", "分层超时", "TimeoutConfig" ], [ "用户输入无效", "友好提示", "输入验证 + 引导" ], [ "未知错误", "降级响应 + 日志", "try-catch + AgentErrorLogger" ] ] } ], "key_takeaways": [], "common_pitfalls": [], "related_chapters": [ "ch06" ] }