{ "metadata": { "id": "ch40", "title": "第40章：CI/CD与版本管理", "volume": "vol10", "volume_title": "生产级Agent平台", "word_count": 1182, "difficulty": "advanced", "prerequisites": [ "ch36" ], "key_concepts": [ "Agent系统的CI/CD特殊性", "Prompt版本管理", "Prompt版本化存储", "Prompt变更检测", "自动化评估Pipeline", "评估框架", "回归检测", "部署策略", "蓝绿部署", "金丝雀发布", "配置版本化", "配置即代码", "GitOps工作流", "持续监控与自动回滚", "最佳实践" ], "learning_objectives": [], "estimated_tokens": 709, "source_file": "vol10/ch40_CI_CD与版本管理.md" }, "overview": "Agent 系统的 CI/CD（持续集成/持续部署）与传统软件有显著差异——除了代码变更，Prompt 调整、工具配置变更、模型切换都可能影响 Agent 行为。这意味着传统的\"代码提交 → 测试 → 部署\"流水线需要扩展为\"代码+配置+Prompt → 评估 → 金丝雀部署 → 监控回滚\"的完整链路。本章将系统讲解 Agent 系统的 CI/CD 实践。", "sections": [ { "id": "40.1", "title": "40.1 Agent系统的CI/CD特殊性", "level": 2, "content": "| 变更类型 | 影响范围 | 回归风险 | 测试方法 |\n|---------|---------|---------|---------|\n| 代码变更 | 工具逻辑、Agent框架 | 中 | 单元测试 + 集成测试 |\n| Prompt变更 | 输出质量、行为模式 | 高 | 自动评估 + 人工抽检 |\n| 工具配置 | Agent能力范围 | 高 | 工具可用性测试 |\n| 模型变更 | 推理质量、成本 | 极高 | A/B测试 + 金丝雀 |\n| 知识库变更 | RAG准确性 | 中 | 检索质量评估 |", "subsections": [] }, { "id": "40.2", "title": "40.2 Prompt版本管理", "level": 2, "content": "", "subsections": [ { "id": "40.2.1", "title": "40.2.1 Prompt版本化存储", "content": "" }, { "id": "40.2.2", "title": "40.2.2 Prompt变更检测", "content": "" } ] }, { "id": "40.3", "title": "40.3 自动化评估Pipeline", "level": 2, "content": "", "subsections": [ { "id": "40.3.1", "title": "40.3.1 评估框架", "content": "" }, { "id": "40.3.2", "title": "40.3.2 回归检测", "content": "" } ] }, { "id": "40.4", "title": "40.4 部署策略", "level": 2, "content": "", "subsections": [ { "id": "40.4.1", "title": "40.4.1 蓝绿部署", "content": "" }, { "id": "40.4.2", "title": "40.4.2 金丝雀发布", "content": "" } ] }, { "id": "40.5", "title": "40.5 配置版本化", "level": 2, "content": "", "subsections": [ { "id": "40.5.1", "title": "40.5.1 配置即代码", "content": "" } ] }, { "id": "40.6", "title": "40.6 GitOps工作流", "level": 2, "content": "", "subsections": [] }, { "id": "40.7", "title": "40.7 持续监控与自动回滚", "level": 2, "content": "", "subsections": [] }, { "id": "最佳实践", "title": "最佳实践", "level": 2, "content": "1. **一切变更版本化**：Prompt、配置、测试用例都要版本化\n2. **Prompt即代码**：使用Git管理，变更走PR流程\n3. **自动化评估**：每次Prompt变更都跑评估\n4. **渐进式发布**：金丝雀 5%→20%→50%→100%\n5. **自动回滚**：部署后持续监控，异常自动回滚\n6. **基线对比**：保留评估基线，每次发布检测回归", "subsections": [] }, { "id": "常见陷阱", "title": "常见陷阱", "level": 2, "content": "1. **只测代码不测Prompt**：Prompt变更无评估直接上线\n2. **全量发布**：直接100%流量切换，风险巨大\n3. **回滚无数据**：不知道回滚到哪个版本\n4. **监控不足**：问题发现太晚\n5. **忽略模型变更**：切换LLM不做A/B测试", "subsections": [] }, { "id": "小结", "title": "小结", "level": 2, "content": "Agent CI/CD 的核心原则：**版本化一切、自动化评估、渐进式发布、持续监控、快速回滚**。通过完善的流水线，将 Agent 从\"实验\"升级为\"生产\"。", "subsections": [] }, { "id": "延伸阅读", "title": "延伸阅读", "level": 2, "content": "1. **GitOps**: https://www.gitops.tech/\n2. **ArgoCD**: https://argoproj.github.io/cd/\n3. **PromptFoo**: https://promptfoo.dev/\n4. **书籍**: \"Accelerate\" (Nicole Forsgren)", "subsections": [] } ], "code_blocks": [ { "id": "code-1", "language": "python", "description": "", "code": "import hashlib\nimport json\nfrom pathlib import Path\n\nclass PromptVersionControl:\n \"\"\"Prompt版本控制\"\"\"\n \n def __init__(self, repo_path: str = \"prompts/\"):\n self.repo_path = Path(repo_path)\n self.repo_path.mkdir(exist_ok=True)\n self._index_path = self.repo_path / \"index.json\"\n self._load_index()\n \n def _load_index(self):\n if self._index_path.exists():\n with open(self._index_path) as f:\n self.index = json.load(f)\n else:\n self.index = {\"prompts\": {}}\n \n def save_version(self, name: str, content: str, \n author: str = \"\", message: str = \"\"):\n \"\"\"保存Prompt新版本\"\"\"\n versions = self.index[\"prompts\"].get(name, {}).get(\"versions\", [])\n new_version = len(versions) + 1\n content_hash = hashlib.sha256(content.encode()).hexdigest()[:12]\n \n version_dir = self.repo_path / name\n version_dir.mkdir(exist_ok=True)\n filepath = version_dir / f\"v{new_version}.txt\"\n filepath.write_text(content, encoding=\"utf-8\")\n \n meta = {\n \"version\": new_version,\n \"hash\": content_hash,\n \"author\": author,\n \"message\": message,\n \"created_at\": datetime.now().isoformat(),\n }\n \n if name not in self.index[\"prompts\"]:\n self.index[\"prompts\"][name] = {\"active_version\": 0, \"versions\": []}\n self.index[\"prompts\"][name][\"versions\"].append(meta)\n self.index[\"prompts\"][name][\"active_version\"] = new_version\n self._save_index()\n return new_version\n \n def load_version(self, name: str, version: int = None) -> str:\n if version is None:\n version = self.index[\"prompts\"][name][\"active_version\"]\n filepath = self.repo_path / name / f\"v{version}.txt\"\n return filepath.read_text(encoding=\"utf-8\")\n \n def diff(self, name: str, v1: int, v2: int) -> str:\n import difflib\n c1 = self.load_version(name, v1)\n c2 = self.load_version(name, v2)\n diff = difflib.unified_diff(c1.splitlines(), c2.splitlines(),\n fromfile=f\"v{v1}\", tofile=f\"v{v2}\")\n return \"\\n\".join(diff)\n \n def rollback(self, name: str, target_version: int):\n self.index[\"prompts\"][name][\"active_version\"] = target_version\n self._save_index()\n \n def _save_index(self):\n with open(self._index_path, \"w\") as f:\n json.dump(self.index, f, indent=2, ensure_ascii=False)", "section_ref": "40.2.1", "runnable": true, "dependencies": [] }, { "id": "code-2", "language": "python", "description": "", "code": "class PromptChangeDetector:\n def detect_changes(self, pvc: PromptVersionControl, \n name: str, new_content: str) -> dict:\n try:\n current = pvc.load_version(name)\n except FileNotFoundError:\n return {\"type\": \"new\", \"impact\": \"high\"}\n \n if current == new_content:\n return {\"type\": \"none\", \"impact\": \"none\"}\n \n old_lines, new_lines = current.splitlines(), new_content.splitlines()\n changed_ratio = self._diff_ratio(old_lines, new_lines)\n \n critical = sum(1 for kw in [\"禁止\", \"必须\", \"格式\", \"约束\"]\n if kw in new_content and kw not in current)\n \n impact = \"high\" if changed_ratio > 0.3 or critical > 0 else (\n \"medium\" if changed_ratio > 0.1 else \"low\"\n )\n return {\"type\": \"modified\", \"impact\": impact, \"critical_changes\": critical}\n \n def _diff_ratio(self, old, new):\n import difflib\n m = difflib.SequenceMatcher(None, old, new)\n matching = sum(b.size for b in m.get_matching_blocks())\n total = max(len(old), len(new))\n return 1 - matching / total if total else 0", "section_ref": "40.2.2", "runnable": true, "dependencies": [] }, { "id": "code-3", "language": "python", "description": "", "code": "class AgentEvaluationPipeline:\n \"\"\"Agent自动化评估\"\"\"\n \n def __init__(self, agent_factory, evaluator):\n self.agent_factory = agent_factory\n self.evaluator = evaluator\n \n async def run(self, config: dict, test_suite: list[dict]) -> dict:\n agent = self.agent_factory(config)\n results = []\n \n for case in test_suite:\n output = await agent.run(case[\"input\"])\n score = await self.evaluator.evaluate(\n expected=case.get(\"expected\", \"\"),\n actual=output,\n criteria=case.get(\"criteria\", []),\n )\n results.append({\n \"case_id\": case[\"id\"],\n \"score\": score,\n \"passed\": score >= 0.7,\n })\n \n passed = sum(1 for r in results if r[\"passed\"])\n return {\n \"total\": len(results),\n \"passed\": passed,\n \"pass_rate\": round(passed / len(results) * 100, 1),\n \"failed\": [r for r in results if not r[\"passed\"]],\n }\n\n# 质量门禁\nQUALITY_GATES = {\"min_pass_rate\": 0.85, \"min_avg_score\": 0.75, \"max_regression\": 0.05}", "section_ref": "40.3.1", "runnable": true, "dependencies": [] }, { "id": "code-4", "language": "python", "description": "", "code": "class RegressionDetector:\n async def detect(self, baseline: dict, current: dict) -> dict:\n regressions = []\n \n if current[\"pass_rate\"] < baseline[\"pass_rate\"] * (1 - QUALITY_GATES[\"max_regression\"]):\n regressions.append({\n \"type\": \"pass_rate\",\n \"baseline\": baseline[\"pass_rate\"],\n \"current\": current[\"pass_rate\"],\n })\n \n return {\n \"has_regression\": len(regressions) > 0,\n \"regressions\": regressions,\n \"decision\": \"block\" if any(\n r[\"current\"] < QUALITY_GATES[\"min_pass_rate\"] * 100 \n for r in regressions\n ) else \"warn\",\n }", "section_ref": "40.3.2", "runnable": true, "dependencies": [] }, { "id": "code-5", "language": "python", "description": "", "code": "class BlueGreenDeployer:\n def __init__(self, k8s_client):\n self.k8s = k8s_client\n \n async def deploy(self, agent_name: str, new_image: str):\n green = f\"{agent_name}-green\"\n await self.k8s.create_deployment(name=green, image=new_image, replicas=2)\n \n healthy = await self._wait_healthy(green, timeout=120)\n if not healthy:\n await self.k8s.delete_deployment(green)\n raise RuntimeError(\"健康检查失败\")\n \n await self.k8s.switch_service(service=agent_name, target=green)\n return {\"status\": \"deployed\", \"active\": green}\n \n async def rollback(self, agent_name: str):\n blue = f\"{agent_name}-blue\"\n await self.k8s.switch_service(service=agent_name, target=blue)\n return {\"status\": \"rolled_back\"}", "section_ref": "40.4.1", "runnable": true, "dependencies": [] }, { "id": "code-6", "language": "python", "description": "", "code": "class CanaryDeployer:\n def __init__(self, k8s_client, metrics_client):\n self.k8s = k8s_client\n self.metrics = metrics_client\n \n async def deploy(self, agent_name: str, new_image: str) -> dict:\n stages = [\n {\"weight\": 0.05, \"duration\": 300},\n {\"weight\": 0.20, \"duration\": 600},\n {\"weight\": 0.50, \"duration\": 600},\n {\"weight\": 1.00, \"duration\": 0},\n ]\n \n canary = f\"{agent_name}-canary\"\n stable = f\"{agent_name}-stable\"\n await self.k8s.create_deployment(name=canary, image=new_image)\n \n for i, stage in enumerate(stages):\n await self.k8s.set_traffic_weights(\n agent_name, {stable: 1 - stage[\"weight\"], canary: stage[\"weight\"]}\n )\n \n if stage[\"duration\"] > 0:\n await asyncio.sleep(stage[\"duration\"])\n \n if not await self._check_metrics(agent_name):\n await self.k8s.set_traffic_weights(agent_name, {stable: 1.0, canary: 0.0})\n await self.k8s.delete_deployment(canary)\n return {\"status\": \"rolled_back\", \"stage\": i}\n \n await self.k8s.update_deployment(stable, image=new_image)\n return {\"status\": \"fully_deployed\"}", "section_ref": "40.4.2", "runnable": true, "dependencies": [] }, { "id": "code-7", "language": "yaml", "description": "", "code": "# agent_config.yaml\nagent:\n name: \"customer-service\"\n version: \"2.3.1\"\n\nmodel:\n primary: \"gpt-4o\"\n fallback: \"gpt-4o-mini\"\n temperature: 0.1\n\ntools:\n - name: \"knowledge_search\"\n enabled: true\n config:\n index: \"customer_kb\"\n top_k: 5\n\nsafety:\n max_steps: 15\n budget_daily_usd: 100.0", "section_ref": "40.5.1", "runnable": false, "dependencies": [] }, { "id": "code-8", "language": "python", "description": "budgetdailyusd: 100.0", "code": "class ConfigManager:\n def load(self, path: str) -> dict:\n with open(path) as f:\n config = yaml.safe_load(f)\n return self._resolve_env(config)\n \n def _resolve_env(self, obj):\n if isinstance(obj, str):\n import re\n return re.sub(r'\\$\\{(\\w+)\\}', lambda m: os.environ.get(m.group(1), \"\"), obj)\n elif isinstance(obj, dict):\n return {k: self._resolve_env(v) for k, v in obj.items()}\n return obj\n \n def validate(self, config: dict) -> list[str]:\n errors = []\n if not config.get(\"model\", {}).get(\"primary\"):\n errors.append(\"未配置主模型\")\n return errors", "section_ref": "40.5.1", "runnable": true, "dependencies": [] }, { "id": "code-9", "language": "text", "description": "", "code": "agent-project/\n├── agents/ # Agent代码\n├── prompts/ # Prompt版本 (v1.txt, v2.txt, ...)\n├── configs/ # 配置 (staging.yaml, production.yaml)\n├── tests/\n│ ├── unit/\n│ ├── integration/\n│ └── eval/ # 评估测试用例\n├── pipelines/ # CI/CD流水线定义\n└── monitoring/ # 监控和告警配置", "section_ref": "40.6", "runnable": false, "dependencies": [] }, { "id": "code-10", "language": "python", "description": "", "code": "class PostDeployMonitor:\n def __init__(self, metrics, alerting):\n self.metrics = metrics\n self.alerting = alerting\n \n async def monitor(self, agent_name: str, deploy_time: str) -> dict:\n checks = []\n \n error_rate = await self.metrics.query(\n f'rate(agent_errors_total{{agent=\"{agent_name}\"}}[5m])'\n )\n checks.append({\n \"name\": \"error_rate\", \"value\": error_rate,\n \"status\": \"pass\" if error_rate < 0.05 else \"fail\",\n })\n \n p95 = await self.metrics.query(\n f'histogram_quantile(0.95, rate(agent_latency_seconds_bucket{{agent=\"{agent_name}\"}}[5m]))'\n )\n checks.append({\"name\": \"p95_latency\", \"value\": p95, \"status\": \"pass\" if p95 < 30 else \"warn\"})\n \n failures = [c for c in checks if c[\"status\"] == \"fail\"]\n return {\n \"overall\": \"auto_rollback\" if len(failures) >= 2 else (\n \"investigate\" if failures else \"healthy\"\n ),\n \"checks\": checks,\n }\n\nclass AutoRollbackManager:\n def __init__(self, deployer, monitor):\n self.deployer = deployer\n self.monitor = monitor\n \n async def watch(self, agent_name: str, deploy_time: str):\n for _ in range(6):\n await asyncio.sleep(300)\n report = await self.monitor.monitor(agent_name, deploy_time)\n \n if report[\"overall\"] == \"auto_rollback\":\n await self.deployer.rollback(agent_name)\n await self.alerting.send(severity=\"critical\",\n message=f\"{agent_name} 自动回滚\")\n return {\"status\": \"rolled_back\"}\n \n return {\"status\": \"stable\"}", "section_ref": "40.7", "runnable": true, "dependencies": [] } ], "tables": [ { "headers": [ "变更类型", "影响范围", "回归风险", "测试方法" ], "data": [ [ "代码变更", "工具逻辑、Agent框架", "中", "单元测试 + 集成测试" ], [ "Prompt变更", "输出质量、行为模式", "高", "自动评估 + 人工抽检" ], [ "工具配置", "Agent能力范围", "高", "工具可用性测试" ], [ "模型变更", "推理质量、成本", "极高", "A/B测试 + 金丝雀" ], [ "知识库变更", "RAG准确性", "中", "检索质量评估" ] ] } ], "key_takeaways": [], "common_pitfalls": [], "related_chapters": [ "ch36" ] }