Compare commits

..

17 Commits
dev ... master

Author SHA1 Message Date
zaochen cc76394a64 [feat] 新增两个 Claude Code Skill:icpc-problem-summary + icpc-solution-writer 2026-07-08 10:42:10 +08:00
zaochen f70307f49e [feat] CLI/MCP ingest 支持 --filename/-n 自定义文件名(含文档更新) 2026-07-05 14:08:34 +08:00
zaochen ff10dbe250 [docs] 新增性能测试报告 PERFORMANCE_REPORT.md(9 次 RAG 查询 + 5 次标签提取,P50 26.7s,成功率 100%) 2026-07-05 13:31:23 +08:00
zaochen 00a996c491 [docs] Layer 5: CONFIG.md 新增性能监控章节(双通道日志/PerfTimer 覆盖/perf CLI/LangSmith) 2026-07-05 09:15:55 +08:00
zaochen 9b679ed419 [feat] Layer 4: 新增 algonotes perf 子命令(解析 perf.log,按 call_type 聚合延迟统计) 2026-07-05 09:14:50 +08:00
zaochen 299987c752 [feat] Layer 3: cleaner/tagger/ask_question 加 PerfTimer 流水线埋点 2026-07-05 09:12:26 +08:00
zaochen e68d1aced6 [fix] perf.log 通道隔离:algonotes.perf 用独立 logger + propagate=False,防止操作日志污染 2026-07-05 09:09:29 +08:00
zaochen bf4be9f286 [fix] Layer 2: 删除 reranker body 全量日志 + PerfTimer 埋点(延迟/分数分布) 2026-07-05 09:07:41 +08:00
zaochen f62e346d15 [docs] 同步 CONFIG.md 与 .env.example:补充 perf_logging/langsmith 配置段 2026-07-05 09:06:38 +08:00
zaochen e0c5d04d52 [feat] Layer 1: 重写 OpJsonFormatter/PerfJsonFormatter + log_perf() + 双通道 setup_logger() + PerfTimer 2026-07-05 09:05:16 +08:00
zaochen cc04ac73a1 [config] Layer 0: 新增 PerfLoggingConfig + LangSmithConfig + env sync 桥接
变更:

  - src/config.py: +45 行 — PerfLoggingConfig(4 字段) + LangSmithConfig(4 字段)

  - config.toml:  +12 行 — [perf_logging] + [langsmith] 配置段

  - .env.example: 精简 LangSmith 段,只保留 LANGSMITH_API_KEY secret

关键: _sync_langsmith_env() 在模块加载时将 config.toml 值写回

os.environ,确保 config.toml [langsmith].enabled 真正控制 LangSmith 开关。

.env 职责收缩为纯 secret store。
2026-07-05 08:54:16 +08:00
zaochen 549f43b588 [docs] 明确 api_key 安全规范:必须通过 ${VAR} 引用 .env,禁止硬编码 2026-07-04 16:52:33 +08:00
zaochen 1efe4ea9e0 [docs] 新增性能收集方案与日志系统设计文档
- docs/PERFORMANCE.md: 基于 LangSmith + PerfTimer 的混合埋点方案
  - 指标全景: 检索/重排/LLM生成/Agent/摄入六阶段 30+ 项指标
  - TTFT/TPOT/ITL 延迟指标详细定义
  - Phase 1 自动采集(延迟/成本/可靠性) + Phase 2 评估体系(质量)
  - 预计 ~180 行新代码,相比纯回调方案节约 90 行

- docs/LOGGER.md: 日志系统 v1.0 基线分析 → v1.1 目标架构
  - 诊断 10 个问题(结构化/安全/运维三级)
  - 双通道设计: app.log 操作日志 + perf.log 性能日志
  - OpJsonFormatter 模板化扩展 + PerfJsonFormatter trace_id 注入
  - log_perf() 类型安全便捷方法
2026-07-04 16:46:54 +08:00
zaochen cbf1f8f44f [fix] MCP --host/--port 被单例忽略,改为列表注册模式 2026-07-02 08:40:01 +08:00
zaochen 5972b933f1 统一环境变量名为GITEE_API_KEY,同步默认模型配置
- .env.example: OPENAI_API_KEY → GITEE_API_KEY
- config.py: 三处 api_key 默认值修正,LLM/Embedding Pydantic 默认模型同步为 config.toml 实际值
- config.toml: 去除 Qwen/ 前缀(embedding),reranker 从 8B 降为 4B
- CONFIG.md: 全面同步 model/api_key/max_tokens 到实际配置值
- README.md: 快速开始示例中的模型名和 api_key 过时修正
2026-07-02 07:03:13 +08:00
zaochen 4dd46d5fe3 [fix] Rich console.print 吞掉用户内容中的 [n]/[i] 等方括号
Rich 默认启用 markup 解析,将 [n]、[i]、[f] 等不认识的
标签静默删除。在打印用户内容的 console.print() 上添加
markup=False 关闭 markup 解析。

影响范围:
- _handle_query_show: 笔记原文的数组索引全部丢失
- _handle_query_search: 搜索预览中的方括号内容丢失
2026-07-01 13:10:15 +08:00
zaochen dd9e391b30 [refactor] 重构CLI为统一handler+dispatch风格,修复8项代码审查问题
- 提取 _build_parser() + 6 个 _add_*_parser() 分离参数定义
- 每个子命令独立 _handle_* 函数,用 dispatch dict 替代 if-elif 链
- 新增 _output() 统一 JSON/console 输出,--json 不再抑制 console
- _output() 自动创建父目录 (fix)
- _display(data) 闭包统一使用 data 参数而非闭包捕获 (fix)
- _ensure_utf8_stdout 异常时输出 stderr 警告而非静默吞异常 (fix)
- 移除 export handler 中的裸 except Exception (fix)
- is_dir() 结果缓存复用 (fix)
- delete 确认中止恢复 sys.exit(0) (fix)
- dispatch 表使用 [] 直接访问(required=True 保证键存在)(fix)
- 全量 handler 添加 docstring
2026-07-01 12:58:27 +08:00
24 changed files with 2924 additions and 438 deletions

View File

@ -1,6 +1,8 @@
OPENAI_API_KEY = "your-api-key"
# ── Gitee.AI ───────────────────────────────────────────
# 必填。从 https://ai.gitee.com/ 获取
GITEE_API_KEY = "your-api-key"
LANGSMITH_TRACING = true
LANGSMITH_ENDPOINT = https://api.smith.langchain.com
LANGSMITH_API_KEY = ""
LANGSMITH_PROJECT = ""
# ── LangSmith (可观测性) ───────────────────────────────
# 必填。从 https://smith.langchain.com/ 获取
# enabled / project / endpoint 在 config.toml [langsmith] 中配置
LANGSMITH_API_KEY = "your-langsmith-api-key"

View File

@ -95,14 +95,14 @@ uv sync
```toml
[llm]
model = "Qwen2.5-72B-Instruct" # 或 deepseek-V3,需要支持 function_call 的 llm
model = "Qwen3-Next-80B-A3B-Instruct" # 或 DeepSeek-R1,需要支持 function_call 的 llm
base_url = "https://ai.gitee.com/v1"
api_key = "your_api_key" # 不建议硬编码,建议引用 .env 中的变量
api_key = "${GITEE_API_KEY}" # 不建议硬编码,建议引用 .env 中的变量
[embedding]
model = "Qwen/Qwen3-Embedding-8B"
model = "Qwen3-Embedding-4B"
base_url = "https://ai.gitee.com/v1"
api_key = "your_api_key"
api_key = "${GITEE_API_KEY}"
```
> 💡 敏感信息不建议硬编码,可以在 `config.toml` 中使用 `${VAR}` 语法动态替换。

View File

@ -7,12 +7,12 @@ max_tokens = 50000
timeout = 60.0
[embedding]
model = "Qwen/Qwen3-Embedding-4B"
model = "Qwen3-Embedding-4B"
base_url = "https://ai.gitee.com/v1"
api_key = "${GITEE_API_KEY}"
[reranker]
model = "Qwen/Qwen3-Reranker-8B"
model = "Qwen3-Reranker-4B"
base_url = "https://ai.gitee.com/v1"
api_key = "${GITEE_API_KEY}"
top_n = 3
@ -23,6 +23,18 @@ file = "logs/app.log"
max_bytes = 10485760
backup_count = 5
[perf_logging]
enabled = true
file = "logs/perf.log"
max_bytes = 10485760
backup_count = 3
[langsmith]
enabled = true
api_key = "${LANGSMITH_API_KEY}"
project = "algonotes-rag"
endpoint = "https://api.smith.langchain.com"
[store]
chroma_dir = "data/chroma_db"
files_dir = "data/files"

View File

@ -15,7 +15,8 @@ config.toml → 业务配置(提交 Git${VAR} 引用环境变量)
```bash
# .env
OPENAI_API_KEY = "your-api-key-here"
GITEE_API_KEY = "your-gitee-api-key"
LANGSMITH_API_KEY = "your-langsmith-api-key"
```
## config.toml 说明
@ -25,22 +26,22 @@ OPENAI_API_KEY = "your-api-key-here"
[llm]
model = "Qwen3-Next-80B-A3B-Instruct"
base_url = "https://ai.gitee.com/v1"
api_key = "${OPENAI_API_KEY}"
api_key = "${GITEE_API_KEY}"
temperature = 0.7
max_tokens = 4096
max_tokens = 50000
timeout = 60.0
# ── Embedding Model ──
[embedding]
model = "Qwen/Qwen3-Embedding-4B"
model = "Qwen3-Embedding-4B"
base_url = "https://ai.gitee.com/v1"
api_key = "${OPENAI_API_KEY}"
api_key = "${GITEE_API_KEY}"
# ── Reranker Model ──
[reranker]
model = "Qwen3-Reranker-4B"
base_url = "https://ai.gitee.com/v1"
api_key = "${OPENAI_API_KEY}"
api_key = "${GITEE_API_KEY}"
top_n = 3
# ── Logging ──
@ -50,6 +51,20 @@ file = "logs/app.log"
max_bytes = 10485760
backup_count = 5
# ── Performance Logging ──
[perf_logging]
enabled = true
file = "logs/perf.log"
max_bytes = 10485760
backup_count = 3
# ── LangSmith Tracing ──
[langsmith]
enabled = true
api_key = "${LANGSMITH_API_KEY}"
project = "algonotes-rag"
endpoint = "https://api.smith.langchain.com"
# ── Storage Paths ──
[store]
chroma_dir = "data/chroma_db"
@ -69,7 +84,7 @@ model = "DeepSeek-R1"
base_url = "https://ai.gitee.com/v1"
[embedding]
model = "Qwen/Qwen3-Embedding-4B"
model = "Qwen3-Embedding-4B"
base_url = "https://ai.gitee.com/v1"
```
@ -81,7 +96,7 @@ model = "deepseek-ai/DeepSeek-V4-Flash"
base_url = "https://api.siliconflow.cn/v1"
[embedding]
model = "Qwen/Qwen3-Embedding-4B"
model = "Qwen3-Embedding-4B"
base_url = "https://api.siliconflow.cn/v1"
```
@ -120,7 +135,7 @@ logger = setup_logger()
1. 如果 `config.toml` 不存在,使用 Pydantic 模型中的硬编码默认值
2. 如果 `config.toml` 存在,以其为准
3. `${OPENAI_API_KEY}` 等占位符在加载时被 `.env` 中的实际值替换
3. `${GITEE_API_KEY}` 等占位符在加载时被 `.env` 中的实际值替换
## 公共题库CPGraph
@ -134,3 +149,39 @@ url = "https://mcp.cpgraph.top/mcp" # CPGraph MCP 服务地址
- `enabled = false`:仅查询个人笔记(默认)
- `enabled = true`:同时查询公共题库,需要 CPGraph MCP 服务可用
## 性能监控
### 双通道日志
系统输出两条独立的日志流:
| 通道 | 文件 | 内容 |
|:---|:---|:---|
| 操作日志 | `logs/app.log` | 所有模块的 INFO/DEBUG/WARNING/ERROR 日志 |
| 性能日志 | `logs/perf.log` | 仅性能埋点PerfTimer`latency_ms`/`trace_id`/`score_range` 等 |
两条通道通过 `algonotes.perf` 独立 logger`propagate = False`)隔离,互不污染。
### 性能埋点覆盖
| call_type | 文件 | 采集方式 |
|:---|:---|:---|
| `reranker` | `src/api/reranker_client.py` | `PerfTimer` 上下文管理器 |
| `pipeline` | `scripts/rag.py` | `PerfTimer` 上下文管理器 |
| `text_cleaning` | `src/ingestion/cleaner.py` | `PerfTimer` 上下文管理器 |
| `tag_extraction` | `src/ingestion/tagger.py` | `PerfTimer` 上下文管理器 |
### 查看性能报告
```bash
algonotes perf # 按 call_type 聚合的延迟统计表
algonotes perf --detail # 额外展示每条记录的详细信息
algonotes perf --json report.json # 导出 JSON 结构化数据
```
### LangSmith 全链路追踪
`config.toml``[langsmith]` 段控制 LangSmith 集成。设置 `enabled = true` 后,
LangChain 自动为所有 LLM / Tool / Retriever 调用上报 Trace 到 LangSmith Cloud。
本地 `perf.log``trace_id` 与 LangSmith Trace 的 `run_id` 一致,可交叉查询。

460
docs/LOGGER.md Normal file
View File

@ -0,0 +1,460 @@
# 📋 日志系统设计
> 本文档描述 AlgoNotes RAG 的日志系统架构包括双通道设计、JSON 格式化器、性能日志集成、
> 配置模型与改进计划。当前版本为 v1.0 基线分析v1.1 目标架构在此基础上扩展。
---
## 📊 当前架构v1.0 基线)
### 总览
```mermaid
graph TD
subgraph entry["入口"]
CLI["scripts/cli.py<br/>setup_logger console=True/False"]
end
subgraph core["src/logger.py"]
ROOT["root logger 'algonotes'<br/>level: INFO (config)"]
FILE["RotatingFileHandler<br/>logs/app.log<br/>10MB × 5, DEBUG"]
STDERR["StreamHandler stderr<br/>plain text, INFO<br/>仅 console=True"]
FMT["JsonFormatter<br/>固定 6 字段"]
end
subgraph modules["~20 个模块"]
SRC["src/**/*.py<br/>logging.getLogger('algonotes.xxx')"]
SCRIPTS["scripts/*.py<br/>setup_logger('algonotes.cli.xxx')"]
end
CLI --> ROOT
ROOT --> FILE
ROOT --> STDERR
FILE --> FMT
SRC --> ROOT
SCRIPTS --> ROOT
```
### 模块清单
| 模块 | 文件 | 行数 | 职责 |
|:---|:---|:---:|:---|
| **setup_logger()** | `src/logger.py` | 49 | 创建 root logger + 两个 Handler可重入 |
| **JsonFormatter** | `src/logger.py` | 12 | 每条日志格式化为一行 JSON固定 6 字段 |
| **LoggingConfig** | `src/config.py` | 6 | Pydantic 模型:`level` / `file` / `max_bytes` / `backup_count` |
| **config.toml** | `config.toml` | 5 | `[logging]`INFO 级别10MB 滚动5 个备份 |
### 日志层级
```
algonotes # rootINFO 级别
├── algonotes.api.llm_client # LLM 客户端初始化
├── algonotes.api.embedding_client # Embedding 客户端初始化
├── algonotes.api.reranker_client # Reranker 请求/响应
├── algonotes.store.file_store # 文件 CRUD
├── algonotes.store.sql_store # SQL 操作
├── algonotes.store.vector_store # Chroma CRUD + 搜索
├── algonotes.ingestion.loader # 文档加载
├── algonotes.ingestion.cleaner # LLM 文本清洗(含重试)
├── algonotes.ingestion.splitter # 分块统计
├── algonotes.ingestion.tagger # LLM 标签提取
├── algonotes.rag.agent # Agent 创建
├── algonotes.rag.retriever # 检索工具调用
├── algonotes.rag.generation # 重排序工具调用
├── algonotes.cli.ingest # CLI 摄入入口
├── algonotes.cli.update # CLI 更新入口
├── algonotes.cli.delete # CLI 删除入口
└── algonotes.cli.query # CLI 查询入口
```
### JsonFormatterv1.0
```python
class JsonFormatter(logging.Formatter):
def format(self, record):
log_entry = {
"timestamp": self.formatTime(record),
"level": record.levelname,
"logger": record.name,
"message": record.getMessage(),
"latency_ms": getattr(record, "latency_ms", None), # 始终为 None
"tokens": getattr(record, "tokens", None), # 始终为 None
}
return json.dumps(log_entry, ensure_ascii=False)
```
固定 6 字段,扩展性为零。`latency_ms` 和 `tokens` 字段预留了接口但从未有代码写入。
---
## 🔍 v1.0 问题诊断
### 结构性问题
| # | 问题 | 严重度 | 影响 |
|:---:|:---|:---:|:---|
| 1 | **JsonFormatter 字段硬编码** | 🔴 高 | 无法追加 `call_type` / `model` / `doc_count` 等性能字段;任何新增字段都需改源码 |
| 2 | **`latency_ms` / `tokens` 永为 None** | 🔴 高 | README 宣称有性能日志,实际零数据产出 |
| 3 | **无独立性能日志通道** | 🟡 中 | 性能数据与操作日志混在同一文件,无法按需过滤或单独轮转 |
| 4 | **无 Trace 关联** | 🟡 中 | 本地日志与 LangSmith Trace 之间无法关联——同一条查询在两边的时间戳可能相差秒级 |
| 5 | **`setup_logger` 被 scripts 层重复调用** | 🟢 低 | 每个 `scripts/*.py` 都调用 `setup_logger()`虽因可重入设计无害但语义混乱src 层用 `getLogger`scripts 层用 `setup_logger` |
### 安全隐患
| # | 问题 | 严重度 | 影响 |
|:---:|:---|:---:|:---|
| 6 | **Reranker 日志打印完整请求体** | 🔴 高 | `reranker_client.py:123` `logger.info(f"body: {payload}")` — 若 API Key 误写入 payload 中则会以明文泄露到日志文件 |
| 7 | **URL 未脱敏** | 🟡 中 | 多处日志打印 `base_url`,虽当前不含敏感参数,但缺乏防御性 |
### 运维问题
| # | 问题 | 严重度 | 影响 |
|:---:|:---|:---:|:---|
| 8 | **仅按大小轮转,无时间维度** | 🟢 低 | 无法按"保留最近 7 天"策略清理旧日志 |
| 9 | **无日志级别动态切换** | 🟢 低 | 生产环境需 DEBUG 排查时,必须改 `config.toml` 重启 |
| 10 | **`console=True` 无第三方日志控制** | 🟢 低 | 只控制了 `algonotes` 命名空间的日志LangChain/httpx 等第三方库的 DEBUG 日志无法开关 |
---
## 🏗️ v1.1 目标架构
### 设计原则
1. **双通道分离** — 操作日志 (`app.log`) 与性能日志 (`perf.log`) 独立文件、独立轮转、独立格式
2. **模板化 JSON** — Formatter 接受 `extra` 中任意字段,不再硬编码字段列表
3. **LangSmith 关联** — 每条性能日志携带 `trace_id``run_id`,与 LangSmith Trace 一一对应
4. **安全默认** — 请求体默认不记录需显式开启URL 自动脱敏
5. **类型安全**`log_perf()` 方法通过参数类型约束性能字段,避免拼写错误
6. **零侵入** — 现有 20+ 模块的 `logger.info(...)` 调用无需改动
### 架构图
```mermaid
graph TD
subgraph config["配置层"]
LOG_CFG["[logging] 操作日志<br/>level / file / rotation"]
PERF_CFG["[logging.perf] 性能日志<br/>enabled / file / rotation"]
end
subgraph core["src/logger.py"]
ROOT["root 'algonotes'"]
OP_FILE["RotatingFileHandler<br/>logs/app.log, JSON"]
PERF_FILE["RotatingFileHandler<br/>logs/perf.log, JSON"]
CONSOLE["StreamHandler stderr<br/>plain text, INFO"]
OP_FMT["OpJsonFormatter<br/>模板化: extra 字段全量输出"]
PERF_FMT["PerfJsonFormatter<br/>模板化 + trace_id 注入"]
end
subgraph perf["src/performance.py"]
MONITOR["log_perf()<br/>类型安全便捷方法"]
TIMER["PerfTimer<br/>自动调用 log_perf()"]
end
subgraph modules["~20 个模块 (零改动)"]
EXISTING["现有 logger.info/warning/error"]
end
LOG_CFG --> OP_FILE
PERF_CFG --> PERF_FILE
ROOT --> OP_FILE
ROOT --> PERF_FILE
ROOT --> CONSOLE
OP_FILE --> OP_FMT
PERF_FILE --> PERF_FMT
PERF_FILE -->|"注入"| LS["LangSmith trace_id"]
MONITOR -->|"logger.info(..., extra={...})"| PERF_FILE
TIMER --> MONITOR
EXISTING --> ROOT
```
### 改动清单
| # | 文件 | 操作 | 行数 | 说明 |
|:---:|:---|:---|:---:|:---|
| 1 | `src/logger.py` | **重写** | ~120 | 双 Handler、`OpJsonFormatter`、`PerfJsonFormatter`、`log_perf()` |
| 2 | `src/config.py` | 修改 | +15 | `LoggingConfig` 扩展 `perf` 子配置;新增 `LangSmithConfig` |
| 3 | `config.toml` | 修改 | +10 | 新增 `[logging.perf]``[langsmith]` 配置段 |
| 4 | `src/api/reranker_client.py` | 修改 | -2 | 删除 `logger.info(f"body: {payload}")` |
| 5 | `scripts/*.py` (5 文件) | 修改 | -5 | 删除冗余 `setup_logger()` 调用,改用 `getLogger()` |
| 6 | `src/performance.py` | 新增 | ~80 | PerfTimer + `_trace_to_langsmith()`(来自 PERFORMANCE.md |
---
## 📦 核心模块设计
### OpJsonFormatter — 操作日志格式化器
```python
class OpJsonFormatter(logging.Formatter):
"""操作日志 JSON 格式化器。
输出规则:
- 固定字段: timestamp, level, logger, message
- 扩展字段: 自动展开 logging.LogRecord 的 extra 字典
- 排除: args, exc_info, 等 logging 内部字段
"""
_FIXED_FIELDS = {"timestamp", "level", "logger", "message"}
def format(self, record: logging.LogRecord) -> str:
entry = {
"timestamp": self.formatTime(record),
"level": record.levelname,
"logger": record.name,
"message": record.getMessage(),
}
# 自动展开 extra 中的自定义字段
for key, value in record.__dict__.items():
if key not in self._FIXED_FIELDS and not key.startswith("_"):
if key not in logging.LogRecord.__dict__:
entry[key] = value
return json.dumps(entry, ensure_ascii=False, default=str)
```
> **关键改进**:不再硬编码字段列表。调用方通过 `logger.info("msg", extra={"custom_field": val})` 即可向日志注入任意结构化字段,无需修改 Formatter。
### PerfJsonFormatter — 性能日志格式化器
```python
class PerfJsonFormatter(OpJsonFormatter):
"""性能日志 JSON 格式化器。
在操作日志基础上:
- 强制注入 trace_id从 contextvars 读取 LangSmith run_id
- 可配置是否美化输出(开发环境)
"""
def format(self, record: logging.LogRecord) -> str:
# 注入当前 LangSmith trace context
if not hasattr(record, "trace_id"):
record.trace_id = _get_current_trace_id() # 从 contextvars 读取
return super().format(record)
```
### log_perf() — 性能日志便捷方法
```python
# 定义在 src/logger.py供 src/performance.py 调用
def log_perf(
call_type: str,
latency_ms: float,
*,
model: str | None = None,
doc_count: int | None = None,
chunk_count: int | None = None,
token_count: int | None = None,
success: bool = True,
error: str | None = None,
extra: dict | None = None,
) -> None:
"""写入一条性能日志到 logs/perf.log。
内部实现: 将所有参数打包进 extra调用
perf_logger.info(f"{call_type} call completed", extra={...})
Args 类型约束确保性能字段不会拼写错误。
"""
_perf_logger = logging.getLogger("algonotes.perf")
perf_data = {
"call_type": call_type,
"latency_ms": round(latency_ms, 2),
"model": model,
"doc_count": doc_count,
"chunk_count": chunk_count,
"token_count": token_count,
"success": success,
"error": error,
}
# 合并调用方追加的额外字段(如 score_range, finish_reason 等)
if extra:
perf_data.update(extra)
# 过滤 None 值,保持日志简洁
perf_data = {k: v for k, v in perf_data.items() if v is not None}
_perf_logger.info(f"{call_type} completed in {latency_ms:.1f}ms", extra=perf_data)
```
### setup_logger() — 入口函数
```python
def setup_logger(name: str = "algonotes", *, console: bool = False) -> logging.Logger:
"""初始化日志系统 — 可重入,后续调用为 no-op。
创建 Handler:
1. 操作日志 Handler — 写入 logs/app.log
- Formatter: OpJsonFormatter
- Level: DEBUG (config.logging.level 控制消息级别)
- Rotation: 按大小 (max_bytes) + 备份数 (backup_count)
2. 性能日志 Handler — 写入 logs/perf.log
- Formatter: PerfJsonFormatter
- Level: INFO
- Rotation: 独立配置 (perf_max_bytes / perf_backup_count)
- enabled=False 时不创建此 Handler
3. Console Handler (可选) — 写入 stderr
- Formatter: plain text
- Level: INFO
- 仅 console=True 时创建
"""
# ... 创建 root logger + 三个 Handler
# 静默第三方库的 DEBUG 噪音
logging.getLogger("httpx").setLevel(logging.WARNING)
logging.getLogger("chromadb").setLevel(logging.WARNING)
logging.getLogger("urllib3").setLevel(logging.WARNING)
```
### 配置模型(`src/config.py` 扩展)
```python
class PerfLoggingConfig(BaseModel):
"""性能日志独立配置"""
enabled: bool = True
file: str = "logs/perf.log"
max_bytes: int = 10 * 1024 * 1024 # 10MB
backup_count: int = 3
class LoggingConfig(BaseModel):
"""操作日志 + 性能日志配置"""
level: str = "INFO"
file: str = "logs/app.log"
max_bytes: int = 10 * 1024 * 1024
backup_count: int = 5
perf: PerfLoggingConfig = PerfLoggingConfig() # ← 新增
class LangSmithConfig(BaseModel):
"""LangSmith 可观测性配置(原仅环境变量,现正式化)"""
enabled: bool = True
api_key: str = ""
project: str = "algonotes-rag"
endpoint: str = "https://api.smith.langchain.com"
```
对应的 `config.toml`
> ⚠️ `[langsmith]``[llm]` 等段中的 `api_key` 必须使用 `${VAR}` 引用环境变量,**禁止**在 `config.toml` 中硬编码真实 Key。这一模式已在项目现有的 `[llm]` / `[embedding]` / `[reranker]` 配置中统一采用。
```toml
[logging]
level = "INFO"
file = "logs/app.log"
max_bytes = 10485760
backup_count = 5
[logging.perf]
enabled = true
file = "logs/perf.log"
max_bytes = 10485760
backup_count = 3
[langsmith]
enabled = true
api_key = "${LANGSMITH_API_KEY}"
project = "${LANGSMITH_PROJECT}"
endpoint = "https://api.smith.langchain.com"
```
---
## 📝 日志格式示例
### 操作日志(`logs/app.log`
```json
{"timestamp":"2026-07-04 15:30:00,123","level":"INFO","logger":"algonotes.api.llm_client","message":"Initializing chat model: Qwen3-Next-80B-A3B-Instruct"}
{"timestamp":"2026-07-04 15:30:00,456","level":"DEBUG","logger":"algonotes.ingestion.splitter","message":"Header split produced 5 chunks"}
{"timestamp":"2026-07-04 15:30:01,789","level":"INFO","logger":"algonotes.rag.retriever","message":"search_notes: query='树状数组', top_k=5"}
{"timestamp":"2026-07-04 15:30:01,890","level":"WARNING","logger":"algonotes.rag.retriever","message":"get_file_content: file not found 'unknown.md'"}
{"timestamp":"2026-07-04 15:30:03,456","level":"ERROR","logger":"algonotes.api.reranker_client","message":"Rerank API error: 500 Internal Server Error"}
```
### 性能日志(`logs/perf.log`
```json
{"timestamp":"2026-07-04 15:30:01,234","level":"INFO","logger":"algonotes.perf","message":"reranker completed in 500.1ms","call_type":"reranker","latency_ms":500.12,"model":"Qwen3-Reranker-4B","doc_count_input":15,"doc_count_output":3,"score_range":{"min":0.32,"max":0.95,"mean":0.67},"success":true,"trace_id":"run-abc123"}
{"timestamp":"2026-07-04 15:30:04,567","level":"INFO","logger":"algonotes.perf","message":"pipeline completed in 3200.8ms","call_type":"pipeline","latency_ms":3200.80,"agent_steps":4,"tool_calls":{"search_notes":1,"get_file_content":1,"rerank_results":1},"context_truncated":false,"finish_reason":"stop","success":true,"trace_id":"run-abc123"}
{"timestamp":"2026-07-04 16:00:10,123","level":"INFO","logger":"algonotes.perf","message":"ingest completed in 4500.0ms","call_type":"ingest","latency_ms":4500.00,"filename":"fenwick.md","file_size_bytes":3200,"chunk_count":5,"chunk_truncated":0,"tags":"树状数组, 数据结构","success":true}
```
> **`trace_id` 字段**:与 LangSmith Trace 的 `run_id` 一致,可直接在 LangSmith Web UI 搜索跳转。
---
## 🔒 安全与运维改进
### 问题Reranker 日志膨胀
`reranker_client.py:121-123` 当前代码:
```python
logger.info(f"POST {url}")
logger.info(f"body: {payload}") # ← 打印完整请求体,含 documents 全文
```
API Key 在 `Authorization` header 中,**body 日志不会泄露 Key**。但 `documents` 是待重排序的候选文本列表,在 Agent 检索场景下每条可能数百字符、总计数千字符——这会导致单条日志行极长(>5000 chars日志文件快速膨胀。
**修复**
```python
# 替换为摘要日志(保护 Key 的 header 本就未打印,此处只控制 body 长度)
logger.info(
f"Rerank request: model={self._model}, "
f"docs={len(documents)}, top_n={top_n}"
)
# 完整 payload 仅在 DEBUG 级别 + 显式开启时记录
if logger.isEnabledFor(logging.DEBUG):
logger.debug(
f"Rerank payload: query='{query[:100]}...', "
f"docs_count={len(documents)}, "
f"total_chars={sum(len(d) for d in documents)}"
)
```
---
## 🧪 验证方式
```bash
# 1. 确认双通道正常
ls -la logs/
# 预期: app.log (操作日志) + perf.log (性能日志)
# 2. 操作日志包含 DEBUG 级别的分块/检索细节
uv run algonotes query ask "树状数组" --verbose
tail -5 logs/app.log
# 预期看到: splitter, retriever, agent 的 INFO/DEBUG 日志
# 3. 性能日志包含 latency_ms / trace_id
tail -3 logs/perf.log | python -m json.tool
# 预期: call_type, latency_ms, trace_id 字段均非 null
# 4. 验证 trace_id 可关联 LangSmith
# 从 perf.log 取一条 trace_id → LangSmith Web UI 搜索 → 应跳转到对应 Trace
# 5. 验证敏感信息不泄露
grep -i "api_key\|token\|secret" logs/app.log logs/perf.log
# 预期: 无匹配 (除非显式配置了 DEBUG payload logging)
# 6. 验证轮转正常 (可选,需要触发)
# 写入超过 10MB 后检查 logs/app.log.1 是否生成
```
---
## ⚠️ 注意事项
1. **现有 logger.info() 调用零改动**`OpJsonFormatter` 自动展开 `extra` 字段,不传 `extra` 时输出与 v1.0 完全兼容(仅多了文件头一致的时间戳格式)。
2. **性能日志默认分离**`logs/perf.log` 独立轮转,不因性能数据高频写入而挤占操作日志的保留空间。
3. **`trace_id` 自动注入**PerfJsonFormatter 通过 `contextvars` 读取当前 LangChain/LangSmith 的 `run_id`,调用方无需手动传入。若上下文无 trace如非 LangChain 调用路径),则 `trace_id``null`
4. **PerfTimer 与 log_perf() 的关系**`PerfTimer.__exit__` 内部调用 `log_perf()`,应用代码不应同时使用两者记录同一段逻辑。
5. **日志级别建议**:操作日志 `INFO`(生产)/ `DEBUG`(开发排查);性能日志始终 `INFO`(每条都是一次 API 调用)。
6. **第三方库噪音控制**`setup_logger()` 自动将 `httpx`、`chromadb`、`urllib3` 的日志级别设为 `WARNING`,避免 DEBUG 模式下日志被 HTTP 请求头刷屏。
7. **`console=True` 不输出性能日志**stderr handler 仅绑到 root `algonotes` 的操作日志通道,`algonotes.perf` 日志仅写入文件(避免干扰终端交互体验)。

626
docs/PERFORMANCE.md Normal file
View File

@ -0,0 +1,626 @@
# 📈 性能收集方案设计
> 本文档描述 AlgoNotes RAG 的性能数据收集方案,基于 LangSmith 全链路追踪 + 本地 PerfTimer 补充埋点。
> 目标:自动采集 LLM / Embedding / Reranker / Tool 各环节的延迟与 Token 消耗,产出本地 JSON Lines 日志
> 与 `docs/PERFORMANCE_REPORT.md`,同时保留 LangSmith Web 端的交互式 Trace 探索能力。
---
## 📊 总览
```mermaid
graph TD
subgraph auto["LangSmith 自动追踪 — 零代码"]
LLM["LLM 调用"]
EMB["Embedding 调用"]
TOOL["Tool 调用"]
RETR["Retriever 调用"]
end
subgraph manual["PerfTimer 补充埋点"]
RERANK["Reranker 调用 — raw httpx"]
PIPE["整体流水线 — 端到端计时"]
end
LLM -->|"自动上报"| LS["LangSmith Cloud — Web UI 交互探索"]
EMB -->|"自动上报"| LS
TOOL -->|"自动上报"| LS
RETR -->|"自动上报"| LS
RERANK -->|"PerfTimer"| LOCAL["本地 JSON Lines 日志"]
PIPE -->|"PerfTimer"| LOCAL
LS -->|"langsmith.Client.list_runs"| REPORT["docs/PERFORMANCE_REPORT.md"]
LOCAL -->|"补充非 LangChain 调用"| REPORT
```
| 阶段 | 采集方式 | 代码量 | 说明 |
|:---|:---|:---:|:---|
| **LLM 调用** | LangSmith 自动追踪 | 0 行 | 环境变量 `LANGSMITH_TRACING=true` 即可启用 |
| **Embedding 调用** | LangSmith 自动追踪 | 0 行 | 同上LangChain 自动区分 LLM / Embedding |
| **Tool 调用** | LangSmith 自动追踪 | 0 行 | 含 `search_notes` / `get_file_content` / `search_by_tags` / `rerank_results` |
| **Retriever 调用** | LangSmith 自动追踪 | 0 行 | Chroma 语义检索的延迟与文档数 |
| **Reranker 调用** | `PerfTimer` 上下文管理器 | ~8 行 | raw httpx不走 LangChain需手动埋点 |
| **整体流水线** | `PerfTimer` 上下文管理器 | ~5 行×4 | 单次 RAG 问答端到端、笔记摄入端到端 |
---
## 🎯 设计选型
### 为什么用 LangSmith 替代自定义 CallbackHandler
| 维度 | 自定义 CallbackHandler | LangSmith 自动追踪 |
|:---|:---|:---|
| **LLM 埋点** | 需实现 `on_llm_start/end`~60 行 | 环境变量一行启用 |
| **Tool 埋点** | 需实现 `on_tool_start/end`~30 行 | 自动覆盖 |
| **Retriever 埋点** | 需实现 `on_retriever_end`~20 行 | 自动覆盖 |
| **contextvars 隔离** | 需手动维护 `run_id → t0` 映射 | 框架层面已处理 |
| **线程安全** | 需加锁保护聚合器 | 无需关心 |
| **Token 用量提取** | 需解析 `response.llm_output` | 自动,且支持流式 |
| **嵌套 Trace 树** | 需手动关联 parent_run_id | 自动构建完整调用树 |
| **可视化** | 无 | Web UI 免费提供 |
| **数据查询** | 需自己实现聚合逻辑 | `langsmith.Client` SDK 直接查询 |
| **总代码量** | ~120 行 | **0 行** |
**结论**LangSmith 已经做到了自定义 CallbackHandler 想做的一切,而且做得更好。我们只需要:
1. 在 `config.toml` 中正式化 LangSmith 配置(环境变量 → 配置模型)
2. 用 `PerfTimer` 覆盖 LangSmith 管不到的 Reranker 和整体流水线
3. 用 `langsmith.Client` 查询数据生成离线报告
---
## 🏗️ 架构设计
### 模块职责
| 模块 | 文件 | 职责 |
|:---|:---|:---|
| **LangSmith 自动追踪** | 无需代码 | 设置 `LANGSMITH_TRACING=true`LangChain 自动为所有 LLM/Tool/Retriever 调用上报 Trace |
| **LangSmithConfig** | `src/config.py` | Pydantic 配置模型,从 `config.toml` `[langsmith]` 段读取,替代裸环境变量 |
| **PerfTimer** | `src/performance.py` | 上下文管理器,计时任意代码块,写入本地 JSON Lines + 同步上报 LangSmith |
| **log_perf()** | `src/logger.py` | 便捷方法,写入带 `latency_ms` / `tokens` / `call_type` 的 JSON Lines 日志 |
| **perf CLI** | `scripts/perf_report.py` | CLI 子命令:从 LangSmith API + 本地日志聚合数据,生成 Markdown 报告 |
### 数据流
```mermaid
graph TD
subgraph zero["零代码 — 环境变量启用"]
ENV["LANGSMITH_TRACING = true"]
end
subgraph langsmith["LangSmith Cloud"]
TRACE["自动追踪<br/>LLM / Tool / Retriever<br/>完整嵌套 Trace 树"]
UI["Web UI 交互探索"]
SDK["langsmith.Client 编程查询"]
end
subgraph perf["PerfTimer 手动埋点"]
RERANK["reranker_client.py"]
CLEAN["cleaner.py"]
TAG["tagger.py"]
RAG["rag.py"]
CHAT["chat.py"]
end
subgraph local["本地存储"]
LOG["logs/app.log<br/>JSON Lines<br/>latency_ms / tokens / call_type"]
end
subgraph report["报告生成"]
CLI["scripts/perf_report.py<br/>algonotes perf"]
MD["docs/PERFORMANCE_REPORT.md"]
end
ENV -->|"零配置自动生效"| TRACE
TRACE --> UI
TRACE --> SDK
RERANK -->|"__exit__ 写入"| LOG
CLEAN -->|"__exit__ 写入"| LOG
TAG -->|"__exit__ 写入"| LOG
RAG -->|"__exit__ 写入"| LOG
CHAT -->|"__exit__ 写入"| LOG
LOG -->|"回读解析"| CLI
SDK -->|"list_runs 查询"| CLI
CLI -->|"聚合输出"| MD
```
---
## 📊 指标全景
RAG 管线的性能指标分为四个维度,覆盖从查询理解到答案生成的每个阶段:
| 维度 | 说明 | 采集难度 |
|:---|:---|:---:|
| **⏱️ 延迟** | 各阶段耗时,定位瓶颈 | 🟢 自动 |
| **🔢 吞吐与成本** | Token 消耗、调用次数、单次查询成本 | 🟢 自动 |
| **🛡️ 可靠性** | 空召回、截断、异常退出、finish_reason | 🟢 自动 |
| **🎯 质量** | 召回精度、答案忠实度、引用正确性 | 🟡 需标注集 / LLM-as-Judge |
### 按阶段展开
#### 1. 检索阶段Retrieval
| 指标 | 类型 | 定义 | 采集方式 |
|:---|:---|:---|:---|
| `retrieval_latency_ms` | 延迟 | 向量检索耗时(`search_notes` 内 Chroma 相似度搜索) | LangSmith 自动(`on_retriever_end` |
| `retrieval_result_count` | 吞吐 | 实际返回的文档数(≤ top_k | LangSmith 自动 |
| `retrieval_empty_result` | 可靠性 | 是否空召回(`result_count == 0` | LangSmith 自动,**>5% 告警** |
| `retrieval_top_k` | 配置 | 请求的 top_k 值 | LangSmith 自动 |
| `retrieval_scores` | 质量 | 返回文档的相似度分数分布 | LangSmith 自动Chroma distance → similarity |
| `recall_at_k` | 质量 | Top-k 结果中包含相关文档的比例 | 🟡 需标注集 |
| `precision_at_k` | 质量 | Top-k 结果中相关文档的比例 | 🟡 需标注集 |
| `mrr` | 质量 | Mean Reciprocal Rank — 第一个相关文档的排名倒数 | 🟡 需标注集 |
> **空召回**是 RAG 系统最危险的静默故障——检索返回空结果时Agent 要么编造答案,要么回复"未找到",两者都损害用户体验。竞赛场景下需重点展示**空召回率**和处理策略。
#### 2. 重排序阶段Reranker
| 指标 | 类型 | 定义 | 采集方式 |
|:---|:---|:---|:---|
| `rerank_latency_ms` | 延迟 | Gitee.AI `/v1/rerank` 接口耗时 | PerfTimer 手动 |
| `rerank_input_count` | 吞吐 | 送入重排序的候选文档数 | PerfTimer 手动 |
| `rerank_output_count` | 吞吐 | 重排序后保留的文档数top_n | PerfTimer 手动 |
| `rerank_score_range` | 质量 | 重排序分数的 min / max / mean | PerfTimer 手动 |
| `rerank_delta` | 质量 | 重排序前后 Top-3 集合变化率(判断重排序是否做了有用功) | PerfTimer + LangSmith 对比 |
> **`rerank_delta`** 是关键判断指标:如果重排序前后的 Top-3 几乎不变,说明向量检索已足够精确,重排序可能是多余的延迟开销。
#### 3. LLM 生成阶段Generation
| 指标 | 类型 | 定义 | 采集方式 |
|:---|:---|:---|:---|
| **`ttft_ms`** | 延迟 | **Time To First Token** — 首个 Token 产出耗时。= Prefill 时间 + 检索时间 + 重排序时间 | LangSmith 自动(首次 `on_llm_end` 或流式首个 chunk |
| **`tpot_ms`** | 延迟 | **Time Per Output Token** — 平均每个输出 Token 耗时。= `(total_latency - ttft) / (completion_tokens - 1)` | LangSmith 自动(计算得出) |
| **`itl_ms`** | 延迟 | **Inter-Token Latency** — Token 间间隔。反映 decode 阶段 GPU 批处理争用,比 TPOT 更精细 | LangSmith 自动(需流式追踪) |
| `generation_latency_ms` | 延迟 | LLM 调用总耗时(含 Prefill + Decode | LangSmith 自动 |
| `prompt_tokens` | 成本 | 输入 Token 数System Prompt + 检索上下文 + 用户问题) | LangSmith 自动 |
| `completion_tokens` | 成本 | 输出 Token 数 | LangSmith 自动 |
| `total_tokens` | 成本 | 总 Token 消耗 | LangSmith 自动 |
| `tokens_per_second` | 吞吐 | 生成速度(`completion_tokens / generation_latency_s` | LangSmith 计算 |
| `finish_reason` | 可靠性 | 生成终止原因。`"stop"` 正常 / `"length"` 截断 | LangSmith 自动,**length 率 >2% 告警** |
| `context_token_count` | 成本 | 注入 LLM 的检索上下文 Token 数(反映 Chunk 利用率) | PerfTimer 手动 |
| `context_truncated` | 可靠性 | 检索上下文是否因超长被截断(`bool` | PerfTimer 手动 |
> **TTFT vs TPOT 的权衡**TTFT 主要受 Prefill 计算和检索延迟影响TPOT/ITL 主要受 Decode 阶段 GPU 批处理争用影响。小模型 TTFT 低但 ITL 高(更多并发请求争用),大模型反之。详见 [NVIDIA RAG Benchmark](https://docs.nvidia.com/rag/2.6.0/performance-benchmarking.html)。
#### 4. Agent 执行阶段
| 指标 | 类型 | 定义 | 采集方式 |
|:---|:---|:---|:---|
| `agent_step_count` | 吞吐 | Agent 执行步数Tool 调用 + LLM 推理轮次) | LangSmith 自动(统计 on_tool_start 次数) |
| `tool_call_breakdown` | 吞吐 | 各 Tool 调用次数分布(`search_notes` / `search_by_tags` / `get_file_content` / `rerank_results` | LangSmith 自动 |
| `tool_call_latency_by_name` | 延迟 | 每种 Tool 的平均/P50/P95 延迟 | LangSmith 自动 |
| `reretrieval_rate` | 可靠性 | 触发二次检索的比例Agent 循环检测) | LangSmith 自动(同 Tool 同参数重复调用) |
| `agent_timeout_rate` | 可靠性 | Agent 执行超时的比例(无有效答案产出) | LangSmith 自动 |
> **Agent 步数过多**>6 步)通常意味着:检索结果不够好导致反复重试,或 Agent Prompt 缺少退出条件。应在 System Prompt 中设定明确的"够了就停"信号。
#### 5. 摄入阶段Ingestion
| 指标 | 类型 | 定义 | 采集方式 |
|:---|:---|:---|:---|
| `ingest_latency_ms` | 延迟 | 单篇笔记摄入总耗时Load → Split → Tag → Store | PerfTimer 手动 |
| `chunk_count` | 吞吐 | 分块数量 | SQLStore 记录 |
| `chunk_truncation_rate` | 可靠性 | 超大 Chunk 被二次分割的比例 | Splitter 内计数 |
| `chunk_size_distribution` | 质量 | 分块大小分布min / max / mean / median | Splitter 内统计 |
| `ingest_file_size` | 吞吐 | 源文件大小bytes | FileStore 记录 |
| `tag_extraction_latency_ms` | 延迟 | LLM 标签提取耗时 | PerfTimer 手动 |
> **Chunk 截断率**反映分块策略(当前 700 chars/chunkoverlap 120是否合理。高截断率说明大部分笔记超过单 chunk 上限,应考虑调整 `chunk_size` 或按语义边界切分。
#### 6. 端到端E2E
| 指标 | 类型 | 定义 | 采集方式 |
|:---|:---|:---|:---|
| `e2e_latency_ms` | 延迟 | 用户提问 → 完整答案的总耗时 | PerfTimer 手动 |
| `e2e_latency_breakdown` | 延迟 | 各阶段耗时占比(检索 / 重排 / LLM / 其他) | 综合计算 |
| `cost_per_query` | 成本 | 单次查询 Token 费用prompt_tokens × prompt_price + completion_tokens × completion_price | 计算得出 |
| `query_success_rate` | 可靠性 | 成功产出答案的查询比例 | PerfTimer 统计 |
---
### 分阶段实施计划
考虑到比赛交付周期,指标分两批落地:
```mermaid
graph LR
subgraph phase1["Phase 1 — 比赛交付 (自动采集)"]
L1["延迟指标<br/>TTFT / TPOT / ITL / E2E"]
C1["成本指标<br/>Token / 调用次数"]
R1["可靠性指标<br/>空召回率 / finish_reason / 截断"]
end
subgraph phase2["Phase 2 — 比赛后 (评估体系)"]
Q2["质量指标<br/>Recall / Precision / MRR"]
F2["忠实度指标<br/>Faithfulness / 引用正确性"]
E2["Eval 数据集<br/>50-100 标注问题"]
end
L1 --> REPORT["docs/PERFORMANCE_REPORT.md"]
C1 --> REPORT
R1 --> REPORT
Q2 -.->|"需标注集"| REPORT
F2 -.->|"需 LLM-as-Judge"| REPORT
```
**Phase 1**当前实现聚焦自动采集的延迟、成本、可靠性三类指标——这些零人工标注成本LangSmith + PerfTimer 全自动产出,满足比赛"真实调用验证 + 性能数据"的提交要求。
**Phase 2**(比赛后规划)引入质量评估:构建 50-100 道标注问题(已知正确答案和关联笔记),用 `recall@k`、`precision@k` 和 LLM-as-Judge 的忠实度评估来量化 RAG 回答质量。这部分需要人工标注投入,不适合在比赛周期内完成。
---
## 📦 核心类型
### PerfTimer`src/performance.py`~60 行)
```python
import time
from contextlib import ContextDecorator
from typing import Literal
class PerfTimer(ContextDecorator):
"""上下文管理器 + 装饰器 — 计时任意代码块,收集多维度指标。
用法(上下文管理器):
with PerfTimer("reranker", model="Qwen3-Reranker-4B",
doc_count=10, extra={"top_n": 3}) as t:
results = reranker.rerank(query, docs)
t.set_extra("score_range", {"min": 0.3, "max": 0.95})
用法(装饰器):
@PerfTimer("pipeline")
def ask_question(question): ...
__exit__ 时自动:
1. 写入本地 JSON Lines 日志 (via log_perf)
2. 同步上报 LangSmith (via langsmith.run_helpers.trace)
3. 异常时标记 success=False 并记录 error
"""
# 支持的调用类型
CallType = Literal[
"reranker", # 重排序调用
"pipeline", # 端到端流水线
"ingest", # 笔记摄入
"tag_extraction", # LLM 标签提取
"text_cleaning", # LLM 文本清洗
]
def __init__(
self,
call_type: CallType,
*,
model: str | None = None,
doc_count: int | None = None,
chunk_count: int | None = None,
token_count: int | None = None,
extra: dict | None = None,
):
self.call_type = call_type
self.model = model
self.doc_count = doc_count
self.chunk_count = chunk_count
self.token_count = token_count
self.extra = extra or {}
self._start: float | None = None
def set_extra(self, key: str, value) -> None:
"""在上下文内追加指标(如重排序分数分布)。"""
self.extra[key] = value
@property
def elapsed_ms(self) -> float:
"""当前已用时间(可在上下文内随时读取)。"""
if self._start is None:
return 0.0
return (time.perf_counter() - self._start) * 1000
def __enter__(self):
self._start = time.perf_counter()
return self # 返回 self 允许 with...as t 语法
def __exit__(self, exc_type, exc_val, exc_tb):
elapsed = self.elapsed_ms
success = exc_type is None
# 1. 本地 JSON Lines 日志
log_perf(
call_type=self.call_type,
latency_ms=elapsed,
model=self.model,
doc_count=self.doc_count,
chunk_count=self.chunk_count,
token_count=self.token_count,
success=success,
error=str(exc_val) if exc_val else None,
extra=self.extra,
)
# 2. 同步上报 LangSmith (可选,由 config 控制)
if _langsmith_enabled():
_trace_to_langsmith(
name=self.call_type,
latency_ms=elapsed,
model=self.model,
doc_count=self.doc_count,
extra=self.extra,
error=exc_val,
)
return False # 不吞异常
```
> **设计要点**
> - 同时继承 `ContextDecorator`,既可用 `with` 也可用 `@` 装饰器。
> - `__enter__` 返回 `self`,允许在 `with` 块内通过 `t.set_extra()` 动态追加指标。
> - `elapsed_ms` 属性允许代码块内部实时读取已用时间(如记录中间检查点)。
> - `CallType` 用 Literal 约束,防止拼写错误。
### LangSmithConfig`src/config.py` 新增,~10 行)
```python
class LangSmithConfig(BaseModel):
"""LangSmith 可观测性配置。
环境变量 (.env) 中已有的 LANGSMITH_* 变量将被此配置模型统一管理,
在 config.toml 中以 ${VAR} 语法引用。也可以通过 config.toml 直接写值。
"""
enabled: bool = True
api_key: str = ""
project: str = "algonotes-rag"
endpoint: str = "https://api.smith.langchain.com"
```
对应的 `config.toml`
> ⚠️ `api_key` 必须使用 `${LANGSMITH_API_KEY}` 引用环境变量,**禁止**在 `config.toml` 中硬编码真实 Key。
```toml
[langsmith]
enabled = true
api_key = "${LANGSMITH_API_KEY}"
project = "${LANGSMITH_PROJECT}"
endpoint = "https://api.smith.langchain.com"
```
---
## 📝 数据格式
### 本地 JSON Lines 日志示例
```json
// LLM 调用 (LangSmith 自动上报,不在本地日志中)
// → 数据存储于 LangSmith Cloud通过 langsmith.Client.list_runs() 查询
// Reranker 调用 (PerfTimer 手动)
{
"timestamp": "2026-07-04 15:30:01,234",
"level": "INFO",
"logger": "algonotes.perf",
"call_type": "reranker",
"latency_ms": 500.12,
"model": "Qwen3-Reranker-4B",
"doc_count_input": 15,
"doc_count_output": 3,
"score_range": {"min": 0.32, "max": 0.95, "mean": 0.67},
"success": true
}
// Pipeline 端到端 (PerfTimer 手动)
{
"timestamp": "2026-07-04 15:30:04,567",
"level": "INFO",
"logger": "algonotes.perf",
"call_type": "pipeline",
"latency_ms": 3200.80,
"agent_steps": 4,
"tool_calls": {"search_notes": 1, "get_file_content": 1, "rerank_results": 1},
"context_truncated": false,
"finish_reason": "stop",
"success": true
}
// 摄入笔记 (PerfTimer 手动)
{
"timestamp": "2026-07-04 16:00:10,123",
"level": "INFO",
"logger": "algonotes.perf",
"call_type": "ingest",
"latency_ms": 4500.00,
"filename": "fenwick.md",
"file_size_bytes": 3200,
"chunk_count": 5,
"chunk_truncated": 0,
"tags": "树状数组, 数据结构",
"success": true
}
```
> **设计原则**LangSmith 自动上报 LLM / Tool / Retriever 数据(在云端),本地日志仅记录 PerfTimer 覆盖的 Reranker / Pipeline / Ingest 调用。两者互补,`algonotes perf --export` 时会合并查询。
### 聚合报告(由 `algonotes perf --export` 生成)
```markdown
## 📈 性能测试数据
> 测试时间2026-07-04 15:30 ~ 16:00
> 测试环境:沐曦 GPU (Gitee.AI)
> LLMQwen3-Next-80B-A3B-Instruct
> EmbeddingQwen3-Embedding-4B | RerankerQwen3-Reranker-4B
> 数据来源LangSmith Trace + 本地日志
### 延迟指标
| 指标 | 调用次数 | 平均 | P50 | P95 | P99 |
|:---|:---:|:---:|:---:|:---:|:---:|
| **TTFT** (首 Token 延迟) | 50 | 1850 ms | 1700 ms | 3200 ms | 4500 ms |
| **TPOT** (每 Token 耗时) | 50 | 85 ms | 75 ms | 150 ms | 220 ms |
| **ITL** (Token 间间隔) | 50 | 82 ms | 72 ms | 145 ms | 210 ms |
| E2E RAG 问答 | 50 | 3200 ms | 2900 ms | 5200 ms | 6800 ms |
| 向量检索 | 50 | 45 ms | 40 ms | 80 ms | 120 ms |
| 重排序 | 50 | 500 ms | 450 ms | 780 ms | 950 ms |
| 文件读取 | 20 | 2 ms | 1 ms | 5 ms | 10 ms |
### E2E 延迟分解
| 阶段 | 占比 | 平均耗时 |
|:---|:---:|:---:|
| 向量检索 (search_notes) | 1.4% | 45 ms |
| 文件读取 (get_file_content) | <0.1% | 2 ms |
| 重排序 (rerank_results) | 15.6% | 500 ms |
| LLM Prefill (TTFT 检索 重排) | 40.6% | 1300 ms |
| LLM Decode (TPOT × output_tokens) | 42.4% | 1353 ms |
| **合计** | **100%** | **3200 ms** |
### 吞吐与成本
| 指标 | 值 |
|:---|:---|
| 总查询次数 | 50 |
| 总 Token 消耗 | 125,000 |
| 平均 Prompt Token / 查询 | 520 |
| 平均 Completion Token / 查询 | 180 |
| 平均 Token / 秒 (生成速度) | 11.8 tok/s |
| 平均每查询成本 | — (待填入 Gitee.AI 定价) |
| 上下文平均 Token 数 | 420 |
| Chunk 利用率 (被引用的上下文比例) | 65% |
### 可靠性
| 指标 | 次数 | 占比 | 告警阈值 |
|:---|:---:|:---:|:---:|
| 空召回 (检索返回 0 条) | 1 / 50 | 2% | >5% |
| 生成截断 (finish_reason="length") | 0 / 50 | 0% | >2% |
| 上下文截断 | 0 / 50 | 0% | >5% |
| Agent 超步数 (>6 steps) | 2 / 50 | 4% | — |
| 查询失败 (异常退出) | 0 / 50 | 0% | >1% |
### Agent 行为统计
| 指标 | 平均 | P50 | P95 |
|:---|:---:|:---:|:---:|
| Agent 步数 / 查询 | 3.2 | 3 | 5 |
| Tool 调用 / 查询 | 2.8 | 3 | 4 |
| search_notes 调用 / 查询 | 1.0 | 1 | 1 |
| get_file_content 调用 / 查询 | 0.4 | 0 | 1 |
| rerank_results 调用 / 查询 | 0.9 | 1 | 1 |
| 二次检索率 | 8% | — | — |
### 摄入性能
| 指标 | 值 |
|:---|:---|
| 已摄入笔记数 | 7 |
| 平均摄入耗时 / 篇 | 4500 ms |
| 平均 Chunk 数 / 篇 | 5.3 |
| Chunk 截断率 | 0% |
| LLM 标签提取耗时 | 1200 ms |
```
---
## 🔗 各文件改动清单
| # | 文件 | 改动类型 | 行数 | 说明 |
|:---:|:---|:---|:---:|:---|
| 1 | `src/performance.py` | **新增** | ~80 | PerfTimer + `_trace_to_langsmith()` + 模块初始化 |
| 2 | `src/config.py` | 修改 | +10 | 新增 `LangSmithConfig``AppConfig` 加 `langsmith` 字段 |
| 3 | `config.toml` | 修改 | +5 | 新增 `[langsmith]` 配置段 |
| 4 | `src/logger.py` | 修改 | +15 | 新增 `log_perf()` 便捷方法 |
| 5 | `src/api/reranker_client.py` | 修改 | +8 | `PerfTimer("reranker", ...)` 包装 `rerank()` |
| 6 | `src/ingestion/cleaner.py` | 修改 | +3 | `@PerfTimer("pipeline")` 装饰器 |
| 7 | `src/ingestion/tagger.py` | 修改 | +3 | 同上 |
| 8 | `scripts/rag.py` | 修改 | +3 | `@PerfTimer("pipeline")` 装饰器 |
| 9 | `scripts/chat.py` | 修改 | +3 | 同上(装饰 `chat_loop` 中的单次问答) |
| 10 | `scripts/perf_report.py` | **新增** | ~60 | `algonotes perf` CLILangSmith API + 本地日志聚合 |
**删减项**(相比 v1 方案):
| 删减内容 | 省去代码 |
|:---|:---:|
| `PerformanceMonitor(BaseCallbackHandler)` | ~120 行 |
| `SessionStats` 聚合器 | ~30 行 |
| `CallRecord` dataclass | ~15 行 |
| `llm_client.py``with_config()` 注入 | ~4 行 |
| `embedding_client.py``with_config()` 注入 | ~4 行 |
| `contextvars` 线程隔离逻辑 | ~10 行 |
总计:**~180 行新代码 + ~50 行修改**,相比 v1 方案节省约 90 行。
---
## 🧪 验证方式
### 前置配置
```bash
# .env 中确保 LangSmith 已配置(与现有 .env.example 一致)
LANGSMITH_TRACING=true
LANGSMITH_ENDPOINT=https://api.smith.langchain.com
LANGSMITH_API_KEY=lsv2_pt_xxxxx
LANGSMITH_PROJECT=algonotes-rag
```
或在新的 `config.toml` `[langsmith]` 段中统一管理。
### 验证步骤
```bash
# 1. 运行一次完整的 RAG 问答
uv run algonotes query ask "什么是树状数组"
# 2. 在 LangSmith Web UI 查看 Trace
# 打开 https://smith.langchain.com → 选择 algonotes-rag 项目
# 应能看到完整调用树: Agent → search_notes → rerank_results → LLM
# 3. 查看本地性能日志(仅 Reranker + Pipeline 部分)
tail -n 10 logs/app.log | grep '"logger":"algonotes.perf"'
# 4. 打印会话统计(聚合 LangSmith + 本地日志)
uv run algonotes perf
# 5. 导出性能报告
uv run algonotes perf --export
# → 生成 docs/PERFORMANCE_REPORT.md
```
### 预期本地日志片段
```text
// Reranker
{"timestamp":"...","level":"INFO","logger":"algonotes.perf","call_type":"reranker","latency_ms":500.12,"model":"Qwen3-Reranker-4B","doc_count_input":15,"doc_count_output":3,"score_range":{"min":0.32,"max":0.95,"mean":0.67},"success":true}
// Pipeline (含 Agent 行为摘要)
{"timestamp":"...","level":"INFO","logger":"algonotes.perf","call_type":"pipeline","latency_ms":3200.80,"agent_steps":4,"tool_calls":{"search_notes":1,"get_file_content":1,"rerank_results":1},"context_truncated":false,"finish_reason":"stop","success":true}
// Ingest
{"timestamp":"...","level":"INFO","logger":"algonotes.perf","call_type":"ingest","latency_ms":4500.00,"filename":"fenwick.md","file_size_bytes":3200,"chunk_count":5,"chunk_truncated":0,"tags":"树状数组, 数据结构","success":true}
```
### 预期 LangSmith 交互
打开 LangSmith Web UI 的某条 Trace能看到
```
📊 algonotes-rag / default
└─ AgentExecutor (3.2s)
├─ search_notes (45ms) → 返回 5 篇文档
├─ get_file_content (2ms) → 读取 fenwick.md
├─ rerank_results (500ms) → 重排序 → top 3
└─ ChatOpenAI (2.6s) → prompt: 520 tokens, completion: 180 tokens
```
---
## ⚠️ 注意事项
1. **LangSmith 是数据主源**LLM / Tool / Retriever 的延迟和 Token 数据以 LangSmith Trace 为准,本地日志仅记录 Reranker 和 Pipeline 级别数据。
2. **无 LangSmith 时的降级**:通过 `LangSmithConfig.enabled = false` 关闭 LangSmith 上报。此时本地 JSON Lines 日志仍然正常工作,`algonotes perf` 将提示"LangSmith 不可用,仅展示本地数据"。
3. **PerfTimer 同时上报 LangSmith**`_trace_to_langsmith()` 使用 `langsmith.run_helpers.trace()` 将手动计时的 Reranker/Pipeline 数据也同步到 LangSmith使 Trace 树完整(可关闭)。
4. **LangSmith 免费额度**LangSmith 提供免费 Personal 计划(每月 3000 traces 免费),超过后按量计费。比赛使用量(开发+测试阶段)应在免费额度内。
5. **流式 LLM**LangSmith 对 `model.stream()``model.invoke()` 一视同仁——`on_llm_end` 在流式完成后同样能拿到完整 Token 用量。
6. **本地日志不冗余**LangChain 自动上报的 LLM/Tool 调用不写入本地日志(避免 10 倍日志膨胀)。如需本地完整记录,设置 `LANGSMITH_TRACING=false` 后自行添加回调。
7. **数据一致性**`algonotes perf --export` 生成报告时LLM/Tool/Retriever 数据来自 `langsmith.Client.list_runs()` APIReranker/Pipeline 数据来自本地 JSON Lines 日志。两者按时间窗口对齐合并。

285
docs/PERFORMANCE_REPORT.md Normal file
View File

@ -0,0 +1,285 @@
# 📊 性能测试报告
> **项目**AlgoNotes RAG — 个人算法竞赛笔记智能助手
> **测试时间**2026-07-05 09:15 ~ 13:25CST
> **测试环境**:沐曦 GPU 算力Gitee.AI
> **LLM**Qwen3-Next-80B-A3B-Instruct
> **Embedding**Qwen3-Embedding-4B | **Reranker**Qwen3-Reranker-4B
> **数据来源**:本地 `logs/perf.log`JSON Lines + LangSmith Trace已启用项目 `algonotes-rag`
---
## 一、测试方法
### 1.1 测试场景
运行 8 次不同的 RAG 问答查询 + 1 次批量笔记摄入5 篇),覆盖以下操作类型:
| 操作类型 | 采集方式 | 说明 |
|:---|:---|:---|
| `pipeline` | `PerfTimer` 上下文管理器 | 端到端 RAG 问答(`scripts/rag.py → ask_question()` |
| `tag_extraction` | `PerfTimer` 上下文管理器 | LLM 标签提取(`src/ingestion/tagger.py → extract_tags()` |
| `reranker` | `PerfTimer` 上下文管理器 | 重排序 API 调用(`src/api/reranker_client.py → rerank()` |
| `text_cleaning` | `PerfTimer` 上下文管理器 | LLM 文本清洗(`src/ingestion/cleaner.py → clean_text()` |
### 1.2 查询列表
| # | 查询内容 | Agent 步数 |
|:---:|:---|:---:|
| 1 | "树状数组是什么?" | 1 |
| 2 | "线段树和树状数组有什么区别?" | 5 |
| 3 | "如何实现并查集的路径压缩?" | 5 |
| 4 | "二分查找的模板怎么写?给出代码示例" | 7 |
| 5 | "图的DFS和BFS有什么区别分别适用于什么场景" | 5 |
| 6 | "什么是差分数组?树状数组如何维护差分?" | 6 |
| 7 | "树状数组如何求逆序对?给出算法思想和复杂度分析" | 6 |
| 8 | "带权并查集是什么?如何维护节点到根节点的距离?" | 5 |
### 1.3 笔记库规模
测试时共 5 篇笔记7 月 5 日批量导入):
| ID | 笔记 | 标签数 |
|:---:|:---|:---:|
| 1 | 图论-图的概念存储和遍历.md | 5 |
| 2 | 基础算法-二分模板.md | 5 |
| 3 | 数据结构-并查集.md | 5 |
| 4 | 数据结构-树状数组.md | 5 |
| 5 | 数据结构-线段树(一).md | 5 |
---
## 二、延迟统计
### 2.1 总体汇总
数据来源:`algonotes perf` 聚合 `logs/perf.log`14 条记录)。
| call_type | 次数 | avg | P50 | P95 | P99 | min | max | 成功率 |
|:---|:---:|:---:|:---:|:---:|:---:|:---:|:---:|:---:|
| **pipeline** | 9 | 31,496ms | 26,677ms | 73,382ms | 95,770ms | 8,016ms | 101,367ms | 100.0% |
| **tag_extraction** | 5 | 1,172ms | 1,118ms | 1,514ms | 1,576ms | 949ms | 1,591ms | 100.0% |
> **注**:本次测试中 `reranker``text_cleaning` 无数据。Reranker 未被触发是因为 Agent 的标签检索精准匹配到 1-2 篇笔记无需重排序Text Cleaning 仅在对网页内容导入时触发,本次未导入网页笔记。
### 2.2 Pipeline 延迟分布
| 延迟区间 | 次数 | 占比 |
|:---|:---:|:---:|
| < 10s | 1 | 11% |
| 10s - 20s | 1 | 11% |
| 20s - 30s | 4 | 44% |
| 30s - 40s | 2 | 22% |
| > 100s | 1 | 11% |
**分析**
- 大部分查询56%)在 10-30s 间完成P50 为 26.7s
- 最慢的查询101s查询 #4 "二分查找模板"Agent 执行了 7 步,多次调用 `search_by_tags` 尝试不同关键词
- 最快的查询8s查询 #1 "树状数组是什么")仅 1 步即命中目标笔记
---
## 三、端到端 Pipeline 分析
### 3.1 Pipeline 阶段分解
`logs/app.log` 中提取一次典型 RAG 查询的执行链路(查询 #8 "带权并查集",总延迟 31,317ms
| 阶段 | 操作 | 耗时(估算) | 占比 |
|:---|:---|:---:|:---:|
| 初始化 | Embedding model + VectorStore + SQLStore + Agent | ~2,000ms | 6% |
| 检索 | `search_by_tags("带权并查集")` | ~1,000ms | 3% |
| 文件读取 | `get_file_content` (9,785 chars) | ~1,000ms | 3% |
| LLM 生成 | Qwen3-Next-80B-A3B-Instruct含 System Prompt + 检索上下文 + 用户问题) | ~27,000ms | 86% |
| 输出 | 流式输出到终端 | < 500ms | 2% |
| **总计** | | **~31,500ms** | **100%** |
> **关键发现**LLM 生成耗时占端到端延迟的 **86%**,是绝对瓶颈。检索和文件读取合计仅占 6%,效率令人满意。
### 3.2 Agent 步数统计
| Agent 步数 | 次数 | 占比 |
|:---:|:---:|:---:|
| 1 | 1 | 11% |
| 5 | 4 | 44% |
| 6 | 2 | 22% |
| 7 | 1 | 11% |
**分析**
- 平均步数 5.0,中位数 5
- 步数 5 是"标准流程"`search_by_tags` → `get_file_content` → 生成 → 质量评估
- 步数 7查询 #4)多出 2 次重试Agent 先用 `search_notes` 向量检索未找到满意结果,转而用 `search_by_tags`
- 步数 1查询 #1)说明标签直击目标时可跳过多次工具调用
### 3.3 Tool 调用模式
所有 8 次查询中 Agent 的工具调用模式高度一致:
```
search_by_tags → get_file_content → (生成答案) → (质量评估)
```
- `search_by_tags` 命中率8/8100%),平均每次搜索命中 1-2 篇笔记
- `search_notes`(向量检索):仅在查询 #4 中被调用Agent 用标签搜索失败后降级尝试)
- `rerank_results`:未被调用(单笔记场景下无需重排序)
- `get_file_content`:每次查询均调用,平均加载 8,000 chars
---
## 四、Token 消耗估算
> **说明**Token 精确数据需从 LangSmith Trace 中提取。以下基于典型查询的估算值。
### 4.1 单次查询 Token 分布(估算)
| 组成部分 | Token 估算 | 说明 |
|:---|:---:|:---|
| System Prompt | ~2,000 | Agent 系统提示词(含工具定义、检索策略指导) |
| 检索上下文 | ~2,500 | 平均加载 8,000 chars ≈ 2,500 tokens中英文混合 |
| 用户问题 | ~50 | 简短的中文问题 |
| **Prompt Tokens 合计** | **~4,500** | |
| 生成回答 | ~800 | 含引用来源和 RAG 效果评价 |
| 工具调用中间步骤 | ~2,000 | 5 步 Agent 推理 × ~400 tokens/步 |
| **Completion Tokens 合计** | **~2,800** | |
### 4.2 成本估算
Gitee.AI 当前为免费 Beta 阶段,以下按参考定价估算:
| 模型 | 输入价格 | 输出价格 | 单次查询成本 |
|:---|:---:|:---:|:---:|
| Qwen3-Next-80B-A3B-Instruct | ¥0.5/M tokens | ¥1.0/M tokens | ~¥0.0054,500 × 0.5 + 2,800 × 1.0 / 1M |
> **结论**:单次 RAG 查询的 Token 成本极低(< ¥0.01),主要开销在延迟而非金钱。
---
## 五、摄入阶段性能
### 5.1 Tag Extraction 性能
5 篇笔记的 LLM 标签提取(`Qwen3-Next-80B-A3B-Instruct`
| 指标 | 值 |
|:---|:---:|
| 次数 | 5 |
| 平均延迟 | 1,172ms |
| P50 | 1,118ms |
| P95 | 1,514ms |
| 成功率 | 100% |
**分析**LLM 标签提取耗时稳定在 950ms~1,600ms远快于 RAG 问答(不含检索步骤的纯推理时间)。这是因为标签提取的 Prompt 短仅含笔记内容和标签格式要求Prefill 阶段计算量小。
### 5.2 摄入 Pipeline 全流程(估算)
一次完整 ingest 的端到端耗时(不含整体 PerfTimer基于日志拼接
| 阶段 | 操作 | 典型耗时 |
|:---|:---|:---:|
| 加载 | FileStore 保存 | < 10ms |
| 分块 | MarkdownHeaderTextSplitter | < 50ms |
| 标签提取 | LLM `extract_tags()` | ~1,200ms |
| 存储 | SQL insert + Chroma embed | ~500ms含 embedding API 调用) |
| **总计** | | **~1,750ms / 篇** |
---
## 六、成功率与可靠性
### 6.1 总体成功率
| 操作类型 | 总次数 | 成功 | 失败 | 成功率 |
|:---|:---:|:---:|:---:|:---:|
| pipeline | 9 | 9 | 0 | **100.0%** |
| tag_extraction | 5 | 5 | 0 | **100.0%** |
| **合计** | **14** | **14** | **0** | **100.0%** |
### 6.2 可靠性分析
| 指标 | 观测值 | 状态 |
|:---|:---|:---:|
| 空召回率(检索返回 0 篇) | 1/8 次(查询 #6 首次 `search_by_tags("差分数组")` 返回空Agent 自动切换为 `search_by_tags("树状数组")` 后命中) | ✅ 有自愈能力 |
| 生成截断(`finish_reason="length"` | 未观测到 | ✅ |
| 异常退出 | 0 次 | ✅ |
| Agent 步数 > 6 | 1 次(查询 #47 步)| ⚠️ 建议关注 |
> **空召回处理**:查询 #6 中 Agent 用 `search_by_tags("差分数组")` 返回空结果后,自动尝试 `search_by_tags("树状数组")` 命中——Agent 的自主纠错机制有效。
---
## 七、日志样本
以下为 `logs/perf.log` 中的典型性能日志条目JSON Lines 格式):
```jsonl
{"timestamp": "2026-07-05 13:21:06,694", "level": "INFO", "logger": "algonotes.perf", "message": "pipeline completed in 26676.7ms", "call_type": "pipeline", "latency_ms": 26676.71, "success": true, "agent_steps": 6}
{"timestamp": "2026-07-05 13:06:13,834", "level": "INFO", "logger": "algonotes.perf", "message": "tag_extraction completed in 1590.9ms", "call_type": "tag_extraction", "latency_ms": 1590.95, "model": "Qwen3-Next-80B-A3B-Instruct", "success": true}
{"timestamp": "2026-07-05 13:15:52,309", "level": "INFO", "logger": "algonotes.perf", "message": "pipeline completed in 31405.1ms", "call_type": "pipeline", "latency_ms": 31405.12, "success": true, "agent_steps": 5}
```
相应操作日志(`logs/app.log`)样本:
```jsonl
{"timestamp": "2026-07-05 13:21:24,430", "level": "INFO", "logger": "algonotes.rag.retriever", "message": "search_by_tags: keyword='差分数组'"}
{"timestamp": "2026-07-05 13:21:24,431", "level": "INFO", "logger": "algonotes.rag.retriever", "message": "search_by_tags: found 0 files"}
{"timestamp": "2026-07-05 13:21:24,433", "level": "INFO", "logger": "algonotes.rag.retriever", "message": "search_by_tags: keyword='树状数组'"}
{"timestamp": "2026-07-05 13:21:24,434", "level": "INFO", "logger": "algonotes.rag.retriever", "message": "search_by_tags: found 1 files"}
{"timestamp": "2026-07-05 13:21:25,326", "level": "INFO", "logger": "algonotes.rag.retriever", "message": "get_file_content: loaded 7399 chars"}
```
---
## 八、LangSmith Trace
LangSmith 已启用(项目:`algonotes-rag`Endpoint`https://api.smith.langchain.com`),自动追踪以下调用:
| 追踪对象 | 采集指标 |
|:---|:---|
| LLM 调用 | TTFT、TPOT、ITL、prompt_tokens、completion_tokens、finish_reason |
| Tool 调用 | 各 Tool 延迟、输入参数、返回值 |
| Retriever 调用 | Chroma 向量检索延迟、返回文档数、相似度分数 |
| Embedding 调用 | 嵌入延迟、batch 大小 |
> **截图指引**:登录 [smith.langchain.com](https://smith.langchain.com) → 选择项目 `algonotes-rag` → 点击任意 Trace → 截图完整的 Trace 树(含 LLM / Tool / Retriever 节点和延迟标注)。
---
## 九、结论与优化建议
### 9.1 当前性能总结
| 维度 | 评估 | 说明 |
|:---|:---:|:---|
| 检索效率 | 🟢 优秀 | `search_by_tags` 标签搜索精准命中,平均 < 1s |
| LLM 生成 | 🟡 可接受 | P50 26.7s,受 Gitee.AI 免费 API 并发限制影响 |
| 可靠性 | 🟢 优秀 | 100% 成功率,空召回有自愈机制 |
| Agent 效率 | 🟡 良好 | 平均 5 步,少数查询存在过度搜索 |
| Token 成本 | 🟢 极低 | 单次查询 < ¥0.01 |
### 9.2 优化方向
1. **减少 Agent 步数**:当前 System Prompt 有时导致 Agent 过度验证(多轮 `get_file_content` 重读)。可在 Prompt 中增加"单次读取足够"的指引。
2. **Reranker 利用不足**当前标签检索足够精准Reranker 未被触发。随着笔记库增长(> 20 篇),`search_notes` 语义检索 + `rerank_results` 重排序将成为常规路径——届时需要补充 Reranker 性能数据。
3. **嵌入缓存**:摄入阶段每次 embed 调用需访问 Gitee.AI API~500ms/篇),大规模导入时可考虑本地 batch + 去重优化。
4. **LLM 延迟波动**P9573s与 P5027s差距 2.7 倍,反映 Gitee.AI 免费 API 的并发排队效应。生产环境可考虑付费 Tier 降低长尾延迟。
5. **补充 LangSmith 数据**:当前 `algonotes perf` 只读取本地 `perf.log`。后续可扩展 `perf_report.py` 通过 `langsmith.Client.list_runs()` 拉取 LLM/Tool/Retriever 的精确 Token 和 TTFT 数据,替代当前的手工估算。
### 9.3 与比赛评分对照
| 评分维度 | 分值 | 证据 |
|:---|:---:|:---|
| 国产算力集成 | 15 | Gitee.AI 三模型LLM/Embedding/Reranker配置分离 |
| MCP 工具 | 30 | 9 个 MCP 工具SSE 传输 |
| **真实调用验证** | **30** | **本报告(延迟 P50/P95/P99 + 成功率 + 日志样本)** |
| 开源质量 | 25 | 17 个 docs/ + README + ARCHITECTURE + 本报告 |
---
> **报告生成命令**`uv run algonotes perf --json data/perf_report.json`
> **原始数据**`logs/perf.log`14 条)、`data/perf_report.json`(结构化导出)

View File

@ -15,6 +15,9 @@
| `--path` | `-i` | 本地文件或目录路径 | 与 `--url` 二选一 |
| `--url` | `-u` | 网页 URL 地址 | 与 `--path` 二选一 |
| `--no-tag` | - | 跳过打 tag | `False` |
| `--filename` | `-n` | 自定义文件名 | 源文件名或 URL 派生名 |
| `--type` | - | 笔记类型note/solution/template | `note` |
| `--author` | - | 笔记作者 | 无 |
| `--verbose` | `-v` | 详细输出 | `False` |
| `--json` | - | 输出到 JSON 文件 | stdout使用时须指定文件路径 |
@ -27,6 +30,9 @@ algonotes ingest -i ./my_notes/
# 从 URL 导入(默认自动清洗)
algonotes ingest -u https://cnblogs.com/xxx
# 自定义文件名
algonotes ingest -i ./my_notes/fenwick.md -n "树状数组笔记.md"
# 跳过打 tag
algonotes ingest -i ./my_notes/ --no-tag
@ -150,7 +156,7 @@ graph TD
## 注意事项
1. **文件命名**导入后文件名即唯一标识,平铺存储在 `data/files/`
1. **文件命名**默认使用源文件 basename 或 URL 派生名作为唯一标识;可通过 `-n/--filename` 自定义,平铺存储在 `data/files/`
2. **文件名冲突**:同名文件会自动添加时间戳后缀(如 `fenwick_20260616_143022.md`
3. **网页清洗**:从 URL 导入时,默认自动执行 LLM 清洗以去除网页噪声;本地导入默认不启用
4. **打 tag**:需要调用 LLM会增加处理时间可用 `--no-tag` 跳过

View File

@ -17,6 +17,7 @@
| `tag` | bool | 提取标签 | `true` |
| `type` | string | 笔记类型note/solution/template | `note` |
| `author` | string \| null | 笔记作者 | 无 |
| `filename` | string \| null | 自定义文件名 | 源文件名或 URL 派生名 |
## 使用示例
@ -32,6 +33,9 @@
// 跳过打 tag
{"path": "./my_notes/", "tag": false}
// 自定义文件名
{"path": "./my_notes/fenwick.md", "filename": "树状数组笔记.md"}
```
## 返回格式
@ -88,6 +92,6 @@
## 注意事项
1. **互斥参数**`path` 和 `url` 必须且只能提供一个
2. **文件命名**导入后文件名即唯一标识,平铺存储在 `data/files/`
2. **文件命名**默认使用源文件 basename 或 URL 派生名作为唯一标识;可通过 `filename` 参数自定义,平铺存储在 `data/files/`
3. **网页清洗**:从 URL 导入时自动执行 LLM 清洗;本地导入默认不清洗
4. **打 tag**:需要调用 LLM会增加处理时间可设 `tag: false` 跳过

View File

@ -1,78 +1,131 @@
# scripts/cli.py
# Unified CLI entry point for `algonotes` command.
#
# Architecture:
# 1. Parser builders (_add_*_parser) — define arguments only
# 2. Handler functions (_handle_*) — validate args → delegate → output
# 3. Dispatch tables — map command strings to handlers
# 4. main() — build parser, parse, dispatch
import argparse
import json
import sys
from pathlib import Path
from typing import Any, Callable
def _setup_cli_logging():
"""Configure logging for CLI use.
# ═══════════════════════════════════════════════════════════
# Setup
# ═══════════════════════════════════════════════════════════
Must be called before any module with logger calls is imported.
Parses sys.argv to check for ``--verbose`` before argparse (which is
fine because we only need the flag's presence, not its value).
def _setup_cli_logging() -> None:
"""Configure logging before any module with loggers is imported.
Checks sys.argv for ``-v`` / ``--verbose`` before argparse runs,
so the flag only needs to be *present* not fully parsed.
When present, enables human-readable stderr output at INFO level
alongside the default JSON-lines file logger.
"""
console = "-v" in sys.argv or "--verbose" in sys.argv
from src.logger import setup_logger
setup_logger("algonotes", console=console)
def _ensure_utf8_stdout():
"""Reconfigure stdout encoding to UTF-8 if running on Windows with GBK.
def _ensure_utf8_stdout() -> None:
"""Reconfigure stdout to UTF-8 on Windows (e.g. GBK/cp936 → UTF-8).
Agent responses may contain emoji or other characters not representable
in the system's default codepage (e.g. cp936/GBK on Chinese Windows).
Agent responses may contain emoji or characters outside the system
codepage. When ``reconfigure`` fails (e.g. stdout is piped to a
non-reconfigurable stream), a warning is printed to stderr instead
of silently swallowing the error.
"""
try:
if sys.stdout.encoding and sys.stdout.encoding.lower() not in (
"utf-8", "utf8",
):
sys.stdout.reconfigure(encoding="utf-8")
except Exception:
pass
except (OSError, AttributeError):
print("⚠ 无法将 stdout 重配置为 UTF-8可能被管道占用",
file=sys.stderr)
def main():
_setup_cli_logging()
_ensure_utf8_stdout()
parser = argparse.ArgumentParser(
prog="algonotes",
description="AlgoNotes RAG CLI — 算法竞赛笔记管理工具",
)
sub = parser.add_subparsers(dest="command", required=True)
# ═══════════════════════════════════════════════════════════
# Output helper
# ═══════════════════════════════════════════════════════════
# ── ingest ──
p_ingest = sub.add_parser("ingest", help="导入笔记")
source = p_ingest.add_mutually_exclusive_group(required=True)
source.add_argument("-i", "--path", type=Path,
help="本地文件或目录路径")
source.add_argument("-u", "--url",
help="网页 URL 地址")
p_ingest.add_argument("--no-tag", action="store_true",
help="跳过打 tag")
p_ingest.add_argument("--type", choices=["note", "solution", "template"],
default="note", help="笔记类型")
p_ingest.add_argument("--author", help="笔记作者")
p_ingest.add_argument("-v", "--verbose", action="store_true",
help="详细输出")
p_ingest.add_argument("--json", type=Path, metavar="FILE",
help="输出结果到 JSON 文件")
def _output(data: Any, json_path: Path | None, *,
display: Callable[[Any], None] | None = None) -> None:
"""Unified output: write JSON and/or call ``display(data)``.
# ── update ──
p_update = sub.add_parser("update", help="更新笔记")
u_sub = p_update.add_subparsers(dest="update_command", required=True)
Eliminates the repeated ``if args.json: else: print()`` pattern
that was duplicated across every handler.
When *json_path* is given the data is written as UTF-8 JSON (parent
directories are created automatically). When *display* is given it
is called with *data* for console output. Both can fire together
they are independent; ``--json`` does not suppress console output.
Args:
data: Result to output. For JSON mode this is serialised with
``json.dumps(ensure_ascii=False, indent=2)``. For console
mode it is passed directly to *display*.
json_path: Path from the ``--json`` flag, or ``None`` to skip.
display: Callable that renders *data* to stdout, or ``None``.
If both *json_path* and *display* are ``None`` the call is
a silent no-op.
"""
if json_path:
json_path.parent.mkdir(parents=True, exist_ok=True)
json_path.write_text(
json.dumps(data, ensure_ascii=False, indent=2),
encoding="utf-8",
)
if display:
display(data)
# ═══════════════════════════════════════════════════════════
# Parser builders
# ═══════════════════════════════════════════════════════════
def _add_ingest_parser(sub: argparse._SubParsersAction) -> None:
"""Register the ``ingest`` subcommand and its arguments.
Adds ``algonotes ingest`` with mutually exclusive ``-i/--path``
(local file or directory) and ``-u/--url`` (web page) sources,
plus optional ``--no-tag``, ``--type``, ``--author``, ``-v``,
and ``--json`` flags.
"""
p = sub.add_parser("ingest", help="导入笔记")
src = p.add_mutually_exclusive_group(required=True)
src.add_argument("-i", "--path", type=Path, help="本地文件或目录路径")
src.add_argument("-u", "--url", help="网页 URL 地址")
p.add_argument("--no-tag", action="store_true", help="跳过打 tag")
p.add_argument("--type", choices=["note", "solution", "template"],
default="note", help="笔记类型")
p.add_argument("--author", help="笔记作者")
p.add_argument("-n", "--filename", help="自定义文件名默认使用源文件名或URL派生名")
p.add_argument("-v", "--verbose", action="store_true", help="详细输出")
p.add_argument("--json", type=Path, metavar="FILE", help="输出结果到 JSON 文件")
def _add_update_parser(sub: argparse._SubParsersAction) -> None:
"""Register the ``update`` subcommand with ``content`` and ``metadata`` sub-subcommands.
``algonotes update content`` re-processes a note's file content
through the full ingestion pipeline (split tag re-index).
``algonotes update metadata`` edits tags/title/author/type in-place
without re-embedding.
"""
p = sub.add_parser("update", help="更新笔记")
u_sub = p.add_subparsers(dest="update_command", required=True)
p_content = u_sub.add_parser("content", help="更新笔记内容(重新处理文件)")
p_content.add_argument("filename", help="笔记文件名")
p_content.add_argument("--file", type=Path, help="新的本地文件路径")
p_content.add_argument("--no-tag", action="store_true",
help="跳过打 tag")
p_content.add_argument("-v", "--verbose", action="store_true",
help="详细输出")
p_content.add_argument("--json", type=Path, metavar="FILE",
help="输出结果到 JSON 文件")
p_content.add_argument("--no-tag", action="store_true", help="跳过打 tag")
p_content.add_argument("-v", "--verbose", action="store_true", help="详细输出")
p_content.add_argument("--json", type=Path, metavar="FILE", help="输出结果到 JSON 文件")
p_meta = u_sub.add_parser("metadata", help="更新笔记元数据(不修改内容)")
p_meta.add_argument("filename", help="笔记文件名")
@ -81,139 +134,186 @@ def main():
p_meta.add_argument("--author", help="新作者")
p_meta.add_argument("--type", choices=["note", "solution", "template"],
help="笔记类型")
p_meta.add_argument("--json", type=Path, metavar="FILE",
help="输出结果到 JSON 文件")
p_meta.add_argument("--json", type=Path, metavar="FILE", help="输出结果到 JSON 文件")
# ── delete ──
p_delete = sub.add_parser("delete", help="删除笔记")
p_delete.add_argument("filename", help="笔记文件名")
p_delete.add_argument("--force", action="store_true",
help="跳过确认提示")
p_delete.add_argument("--json", type=Path, metavar="FILE",
help="输出结果到 JSON 文件")
# ── query ──
p_query = sub.add_parser("query", help="查询笔记")
q_sub = p_query.add_subparsers(dest="query_command", required=True)
def _add_delete_parser(sub: argparse._SubParsersAction) -> None:
"""Register the ``delete`` subcommand and its arguments.
Adds ``algonotes delete <filename>`` with optional ``--force``
(skip confirmation prompt) and ``--json`` flags.
"""
p = sub.add_parser("delete", help="删除笔记")
p.add_argument("filename", help="笔记文件名")
p.add_argument("--force", action="store_true", help="跳过确认提示")
p.add_argument("--json", type=Path, metavar="FILE", help="输出结果到 JSON 文件")
def _add_query_parser(sub: argparse._SubParsersAction) -> None:
"""Register the ``query`` subcommand and its six sub-subcommands.
Adds ``algonotes query {show, list, info, search, export, ask}``
with their respective arguments. This is the most complex parser
group because it covers all read-oriented operations.
"""
p = sub.add_parser("query", help="查询笔记")
q_sub = p.add_subparsers(dest="query_command", required=True)
p_show = q_sub.add_parser("show", help="查看笔记原文")
p_show.add_argument("filename", help="笔记文件名")
p_show.add_argument("--lines", type=int, help="只显示前 N 行")
p_show.add_argument("--json", type=Path, metavar="FILE",
help="输出到 JSON 文件")
p_show.add_argument("--json", type=Path, metavar="FILE", help="输出到 JSON 文件")
p_list = q_sub.add_parser("list", help="列出笔记")
p_list.add_argument("--tag", help="按标签筛选")
p_list.add_argument("-v", "--verbose", action="store_true",
help="显示详细信息")
p_list.add_argument("--json", type=Path, metavar="FILE",
help="输出到 JSON 文件")
p_list.add_argument("-v", "--verbose", action="store_true", help="显示详细信息")
p_list.add_argument("--json", type=Path, metavar="FILE", help="输出到 JSON 文件")
p_info = q_sub.add_parser("info", help="查看笔记详情")
p_info.add_argument("filename", nargs="?", help="笔记文件名")
p_info.add_argument("--id", type=int, help="笔记 ID")
p_info.add_argument("--json", type=Path, metavar="FILE",
help="输出到 JSON 文件")
p_info.add_argument("--json", type=Path, metavar="FILE", help="输出到 JSON 文件")
p_search = q_sub.add_parser("search", help="语义搜索")
p_search.add_argument("query", help="搜索关键词")
p_search.add_argument("--top-k", type=int, default=5,
help="返回结果数量")
p_search.add_argument("--json", type=Path, metavar="FILE",
help="输出到 JSON 文件")
p_search.add_argument("--top-k", type=int, default=5, help="返回结果数量")
p_search.add_argument("--json", type=Path, metavar="FILE", help="输出到 JSON 文件")
p_export = q_sub.add_parser("export",
help="导出笔记YAML frontmatter兼容 Obsidian")
p_export.add_argument("filename", nargs="?", help="笔记文件名")
p_export.add_argument("--all", action="store_true", help="导出所有笔记")
p_export.add_argument("-o", "--output", type=Path,
default=Path("./export"), help="输出目录")
p_export.add_argument("--json", type=Path, metavar="FILE",
help="输出到 JSON 文件")
p_export.add_argument("-o", "--output", type=Path, default=Path("./export"),
help="输出目录")
p_export.add_argument("--json", type=Path, metavar="FILE", help="输出到 JSON 文件")
p_ask = q_sub.add_parser("ask", help="RAG 问答(单次,直接输出答案)")
p_ask.add_argument("question", help="问题")
p_ask.add_argument("--top-k", type=int, default=5,
help="检索结果数量(默认 5")
p_ask.add_argument("--stream", action="store_true",
help="流式输出答案")
p_ask.add_argument("--json", type=Path, metavar="FILE",
help="输出到 JSON 文件")
p_ask.add_argument("--top-k", type=int, default=5, help="检索结果数量(默认 5")
p_ask.add_argument("--stream", action="store_true", help="流式输出答案")
p_ask.add_argument("--json", type=Path, metavar="FILE", help="输出到 JSON 文件")
# ── chat ──
p_chat = sub.add_parser("chat", help="交互式问答")
p_chat.add_argument("--thread-id", default="default",
help="会话标识符,用于区分不同对话")
# ── mcp ──
p_mcp = sub.add_parser("mcp", help="启动 MCP 服务器")
p_mcp.add_argument("--host", default="127.0.0.1",
help="监听地址(默认 127.0.0.1")
p_mcp.add_argument("--port", type=int, default=8000,
help="监听端口(默认 8000")
def _add_chat_parser(sub: argparse._SubParsersAction) -> None:
"""Register the ``chat`` subcommand for interactive RAG sessions.
# ── dispatch ──
args = parser.parse_args()
Adds ``algonotes chat`` with an optional ``--thread-id`` argument
that isolates conversation history (default: ``"default"``).
"""
p = sub.add_parser("chat", help="交互式问答")
p.add_argument("--thread-id", default="default",
help="会话标识符,用于区分不同对话")
if args.command == "ingest":
_run_ingest(args)
elif args.command == "update":
_run_update(args)
elif args.command == "delete":
_run_delete(args)
elif args.command == "query":
_run_query(args)
elif args.command == "chat":
_run_chat(args)
elif args.command == "mcp":
_run_mcp(args)
def _run_ingest(args):
def _add_mcp_parser(sub: argparse._SubParsersAction) -> None:
"""Register the ``mcp`` subcommand to start the MCP server.
Adds ``algonotes mcp`` with optional ``--host`` (default ``127.0.0.1``)
and ``--port`` (default ``8000``) to control the SSE transport binding.
"""
p = sub.add_parser("mcp", help="启动 MCP 服务器")
p.add_argument("--host", default="127.0.0.1", help="监听地址(默认 127.0.0.1")
p.add_argument("--port", type=int, default=8000, help="监听端口(默认 8000")
def _add_perf_parser(sub: argparse._SubParsersAction) -> None:
"""Register the ``perf`` subcommand for performance reporting.
Adds ``algonotes perf`` with optional ``--detail`` (show per-entry
breakdown) and ``--json`` flags.
"""
p = sub.add_parser("perf", help="性能报告")
p.add_argument("--detail", action="store_true", help="显示每条记录的详细信息")
p.add_argument("--json", type=Path, metavar="FILE", help="输出结果到 JSON 文件")
def _build_parser() -> argparse.ArgumentParser:
"""Construct the full argument parser with all subcommands.
Returns an ``ArgumentParser`` that is ready for ``parse_args()``.
Each subcommand group is delegated to its own ``_add_*_parser``
helper so parser construction stays modular.
"""
parser = argparse.ArgumentParser(
prog="algonotes",
description="AlgoNotes RAG CLI — 算法竞赛笔记管理工具",
)
sub = parser.add_subparsers(dest="command", required=True)
_add_ingest_parser(sub)
_add_update_parser(sub)
_add_delete_parser(sub)
_add_query_parser(sub)
_add_chat_parser(sub)
_add_mcp_parser(sub)
_add_perf_parser(sub)
return parser
# ═══════════════════════════════════════════════════════════
# Handler: ingest
# ═══════════════════════════════════════════════════════════
def _handle_ingest(args: argparse.Namespace) -> None:
"""Handle ``algonotes ingest`` — import notes from local files or web URLs.
Dispatches to:
- ``ingest_local`` for ``-i <file>`` (single .md file)
- ``ingest_locals`` for ``-i <dir>`` (batch, all *.md in directory)
- ``ingest_web`` for ``-u <url>`` (web page with LLM cleaning)
Outputs a summary line for single imports; batch imports print
per-file progress from within ``ingest_locals`` and stay silent here.
Supports ``--json`` for structured output.
"""
from scripts.ingest import ingest_local, ingest_locals, ingest_web
tag = not args.no_tag
note_type = args.type
author = args.author
is_dir = args.path and args.path.is_dir()
if args.path:
if args.path.is_file():
result = ingest_local(args.path,
tag=tag, verbose=args.verbose,
type=note_type, author=author)
elif args.path.is_dir():
results = ingest_locals(args.path,
tag=tag, verbose=args.verbose,
type=note_type, author=author)
result = ingest_local(args.path, tag=tag, verbose=args.verbose,
type=args.type, author=args.author,
filename=args.filename)
elif is_dir:
results = ingest_locals(args.path, tag=tag, verbose=args.verbose,
type=args.type, author=args.author)
else:
print(f"❌ 路径不存在: {args.path}")
sys.exit(1)
elif args.url:
result = ingest_web(args.url,
tag=tag, verbose=args.verbose,
type=note_type, author=author)
result = ingest_web(args.url, tag=tag, verbose=args.verbose,
type=args.type, author=args.author,
filename=args.filename)
if args.path and args.path.is_dir():
if is_dir:
output_data = [r.__dict__ for r in results]
else:
output_data = result.__dict__
if args.json:
args.json.write_text(
json.dumps(output_data, ensure_ascii=False, indent=2),
encoding="utf-8",
)
elif not (args.path and args.path.is_dir()):
print(
f"{result.file_name} ({result.chunk_count} chunks, tags: {result.tags})")
def _display(data):
# Directory ingest is silent here — ingest_locals already
# prints per-file progress lines during iteration.
if not is_dir:
print(f"{data['file_name']} ({data['chunk_count']} chunks, tags: {data['tags']})")
_output(output_data, args.json, display=_display)
def _run_update(args):
if args.update_command == "content":
_run_update_content(args)
elif args.update_command == "metadata":
_run_update_metadata(args)
# ═══════════════════════════════════════════════════════════
# Handlers: update
# ═══════════════════════════════════════════════════════════
def _handle_update_content(args: argparse.Namespace) -> None:
"""Handle ``algonotes update content <filename>`` — reprocess a note's file.
def _run_update_content(args):
Validates the note exists in SQLStore and that a content source
is available (``--file`` or stored ``source_url``), then runs the
full ingestion pipeline: load split tag re-index in both
SQLStore and VectorStore (old chunks are deleted first).
Exits with code 1 if the note is not found or no source is available.
"""
from scripts.update import update_note
from src.store.sql_store import get_sql_store
@ -229,52 +329,68 @@ def _run_update_content(args):
print(f"❌ 笔记 {args.filename} 无 source_url请使用 --file 指定新文件")
sys.exit(1)
result = update_note(
args.filename,
file_path=args.file,
tag=tag,
verbose=args.verbose,
)
result = update_note(args.filename, file_path=args.file, tag=tag,
verbose=args.verbose)
if result.success:
print(f"Update success: {result.file_name} ({result.chunk_count} chunks)")
else:
print(f"Update failed: {result.file_name}")
def _display(data):
if data["success"]:
print(f"Update success: {data['file_name']} ({data['chunk_count']} chunks)")
else:
print(f"Update failed: {data['file_name']}")
if args.json:
args.json.write_text(
json.dumps(result.__dict__, ensure_ascii=False, indent=2),
encoding="utf-8",
)
_output(result.__dict__, args.json, display=_display)
def _run_update_metadata(args):
def _handle_update_metadata(args: argparse.Namespace) -> None:
"""Handle ``algonotes update metadata <filename>`` — edit metadata in-place.
Updates title, tags, author, and/or type in SQLStore without
re-processing file content. If tags are changed the VectorStore
chunk metadata is updated to match (no re-embedding).
At least one of ``--title``, ``--tags``, ``--author``, ``--type``
should be provided; the handler itself does not enforce this
``update_metadata`` simply returns with ``fields_updated=[]``.
"""
from scripts.update import update_metadata
result = update_metadata(
args.filename,
title=args.title,
tags=args.tags,
author=args.author,
type=args.type,
)
result = update_metadata(args.filename, title=args.title, tags=args.tags,
author=args.author, type=args.type)
if result.success:
print(f"Metadata updated: {result.filename} ({', '.join(result.fields_updated)})")
else:
print(f"Metadata update failed: {result.filename}")
def _display(data):
if data["success"]:
print(f"Metadata updated: {data['filename']} "
f"({', '.join(data['fields_updated'])})")
else:
print(f"Metadata update failed: {data['filename']}")
if args.json:
args.json.write_text(
json.dumps({"filename": result.filename,
"success": result.success,
"fields_updated": result.fields_updated},
ensure_ascii=False, indent=2),
encoding="utf-8",
)
_output({"filename": result.filename, "success": result.success,
"fields_updated": result.fields_updated}, args.json, display=_display)
def _run_delete(args):
def _handle_update(args: argparse.Namespace) -> None:
"""Dispatch ``algonotes update {content, metadata}`` to the correct sub-handler.
``args.update_command`` is guaranteed to exist because the sub-subparser
was created with ``required=True``.
"""
_UPDATE_HANDLERS[args.update_command](args)
# ═══════════════════════════════════════════════════════════
# Handler: delete
# ═══════════════════════════════════════════════════════════
def _handle_delete(args: argparse.Namespace) -> None:
"""Handle ``algonotes delete <filename>`` — remove a note from all three stores.
Unless ``--force`` is passed, prompts the user to type the filename
for confirmation (type ``"stop"`` to abort). Deletion order is
VectorStore FileStore SQLStore (see ``scripts/delete.py``).
Does NOT exit with error if the file doesn't exist; prints a
warning and returns normally.
"""
from scripts.delete import delete_note
if not args.force:
@ -284,137 +400,166 @@ def _run_delete(args):
result = delete_note(args.filename)
if result.success:
print(f"成功删除 {result.file_name}")
def _display(data):
if data["success"]:
print(f"成功删除 {data['file_name']}")
else:
print(f"{data['file_name']} 文件不存在,无法进行删除")
_output(result.__dict__, args.json, display=_display)
# ═══════════════════════════════════════════════════════════
# Handlers: query
# ═══════════════════════════════════════════════════════════
def _handle_query_show(args: argparse.Namespace) -> None:
"""Handle ``algonotes query show <filename>`` — print raw note content.
Reads the file from FileStore and prints it to the terminal via
``rich.console.Console``. Use ``--lines N`` to limit output to
the first N lines. Supports ``--json`` for structured output.
"""
from scripts.query import show_raw, console
content = show_raw(args.filename, args.lines)
if content is None:
console.print(f"[red]笔记不存在: {args.filename}[/red]")
return
def _display(data):
console.print(data["content"], markup=False)
_output({"filename": args.filename, "content": content,
"line_count": len(content.splitlines())},
args.json, display=_display)
def _handle_query_list(args: argparse.Namespace) -> None:
"""Handle ``algonotes query list`` — list all notes as a table.
Optionally filter by tag with ``--tag <keyword>`` (SQL LIKE on
the comma-separated tags field). Renders a ``rich`` table with
columns: ID, filename, tags, ingestion date. Supports ``--json``.
"""
from scripts.query import list_notes, print_list, NoteInfo
notes = list_notes(args.tag)
def _display(data):
print_list([NoteInfo.from_dict(d) for d in data])
_output([n.__dict__ for n in notes], args.json, display=_display)
def _handle_query_info(args: argparse.Namespace) -> None:
"""Handle ``algonotes query info`` — show detailed metadata for one note.
Looks up the note by ``<filename>`` or ``--id <n>`` (at least one
required). Prints each metadata field as a key-value pair via
``rich`` console. Supports ``--json``.
"""
from scripts.query import show_info, console
from src.store.sql_store import get_sql_store
if not args.filename and not args.id:
console.print("[red]须指定 filename 或 --id[/red]")
return
if args.id:
record = get_sql_store().get(args.id)
note = show_info(record["filename"]) if record and record.get("filename") else None
else:
print(f"{result.file_name} 文件不存在,无法进行删除")
note = show_info(args.filename)
if args.json:
args.json.write_text(
json.dumps(result.__dict__, ensure_ascii=False, indent=2),
encoding="utf-8",
)
if note is None:
console.print(f"[red]笔记不存在: {args.filename or f'id={args.id}'}[/red]")
return
def _display(data):
for k, v in data.items():
console.print(f" [cyan]{k}:[/cyan] {v}")
_output(note.__dict__, args.json, display=_display)
def _run_query(args):
from scripts.query import (show_raw, list_notes, show_info,
semantic_search, print_list, console)
import json as _json
def _handle_query_search(args: argparse.Namespace) -> None:
"""Handle ``algonotes query search <query>`` — semantic vector search.
if args.query_command == "show":
content = show_raw(args.filename, args.lines)
if content is None:
console.print(f"[red]笔记不存在: {args.filename}[/red]")
elif args.json:
line_count = len(content.splitlines())
args.json.write_text(
_json.dumps({"filename": args.filename, "content": content,
"line_count": line_count},
ensure_ascii=False, indent=2),
encoding="utf-8",
)
else:
console.print(content)
Searches the Chroma vector store for chunks semantically similar
to *query*. Prints top ``--top-k`` (default 5) results with source
filename and a 120-character preview. Supports ``--json``.
"""
from scripts.query import semantic_search, console
elif args.query_command == "list":
notes = list_notes(args.tag)
if args.json:
args.json.write_text(
_json.dumps([n.__dict__ for n in notes],
ensure_ascii=False, indent=2),
encoding="utf-8",
)
else:
print_list(notes)
docs = semantic_search(args.query, args.top_k)
if not docs:
console.print("[yellow]无匹配结果[/yellow]")
return
elif args.query_command == "info":
if not args.filename and not args.id:
console.print("[red]须指定 filename 或 --id[/red]")
else:
if args.id:
from src.store.sql_store import get_sql_store
record = get_sql_store().get(args.id)
if record:
note = show_info(record["filename"]) if record.get("filename") else None
else:
note = None
else:
note = show_info(args.filename)
if note is None:
console.print(f"[red]笔记不存在: {args.filename or f'id={args.id}'}[/red]")
elif args.json:
args.json.write_text(
_json.dumps(note.__dict__, ensure_ascii=False, indent=2),
encoding="utf-8",
)
else:
for k, v in note.__dict__.items():
console.print(f" [cyan]{k}:[/cyan] {v}")
def _display(data):
results = data["results"]
console.print(f"找到 {len(results)} 条相关结果:\n")
for i, r in enumerate(results, 1):
console.print(
f"[cyan][{i}][/cyan] 来自: {r.get('source', '?')}")
console.print(f" {r['content'][:120]}...\n", markup=False)
elif args.query_command == "search":
docs = semantic_search(args.query, args.top_k)
if not docs:
console.print("[yellow]无匹配结果[/yellow]")
elif args.json:
args.json.write_text(
_json.dumps({"query": args.query,
"results": [{"source": d.metadata.get("source"),
"content": d.page_content,
"metadata": {k: v for k, v in d.metadata.items()
if k != "source"}}
for d in docs]},
ensure_ascii=False, indent=2),
encoding="utf-8",
)
else:
console.print(f"找到 {len(docs)} 条相关结果:\n")
for i, doc in enumerate(docs, 1):
console.print(
f"[cyan][{i}][/cyan] 来自: {doc.metadata.get('source', '?')}")
console.print(f" {doc.page_content[:120]}...\n")
elif args.query_command == "export":
_run_query_export(args)
elif args.query_command == "ask":
_run_query_ask(args)
_output({"query": args.query,
"results": [{"source": d.metadata.get("source"),
"content": d.page_content,
"metadata": {k: v for k, v in d.metadata.items()
if k != "source"}}
for d in docs]},
args.json, display=_display)
def _run_query_export(args):
def _handle_query_export(args: argparse.Namespace) -> None:
"""Handle ``algonotes query export`` — export notes with YAML frontmatter.
Supports two modes:
- ``export <filename>`` single note
- ``export --all`` every note in the store
Output files are written to ``-o/--output`` (default ``./export``)
and are compatible with Obsidian. Catches ``FileNotFoundError``
for missing notes gracefully. Supports ``--json``.
"""
from scripts.query import export_note, export_all
import json as _json
try:
if args.all:
results = export_all(args.output)
if args.json:
args.json.write_text(
_json.dumps(results, ensure_ascii=False, indent=2),
encoding="utf-8",
)
else:
print(f"Exported {len(results)} notes to {args.output}")
def _display(data):
print(f"Exported {len(data)} notes to {args.output}")
_output(results, args.json, display=_display)
elif args.filename:
path = export_note(args.filename, args.output)
if args.json:
args.json.write_text(
_json.dumps({"filename": args.filename,
"path": str(path)},
ensure_ascii=False, indent=2),
encoding="utf-8",
)
else:
print(f"Exported {args.filename} to {path}")
def _display(data):
print(f"Exported {data['filename']} to {data['path']}")
_output({"filename": args.filename, "path": str(path)},
args.json, display=_display)
else:
print("请指定 filename 或 --all")
except FileNotFoundError as e:
print(f"❌ 笔记不存在: {e}")
except Exception as e:
print(f"❌ 导出失败: {e}")
def _run_query_ask(args):
def _handle_query_ask(args: argparse.Namespace) -> None:
"""Handle ``algonotes query ask <question>`` — single-shot RAG question.
Runs the full RAG pipeline (retrieve rerank generate) via
``scripts.rag.ask_question`` and prints the answer. Use ``--stream``
for token-by-token output (the handler itself is silent in stream
mode because ``ask_question`` already prints). Supports ``--json``.
Exits with code 1 if the RAG pipeline returns ``success=False``.
"""
from scripts.rag import ask_question
import json as _json
result = ask_question(args.question, stream=args.stream)
@ -422,24 +567,125 @@ def _run_query_ask(args):
print(f"❌ 问答失败: {result.error}")
sys.exit(1)
if args.json:
args.json.write_text(
_json.dumps(result.dict, ensure_ascii=False, indent=2),
encoding="utf-8",
)
elif not args.stream:
print(result.answer)
def _display(data):
if not args.stream:
print(data["answer"])
_output(result.dict, args.json, display=_display)
def _run_chat(args):
def _handle_query(args: argparse.Namespace) -> None:
"""Dispatch ``algonotes query {show, list, info, search, export, ask}``.
``args.query_command`` is guaranteed to exist because the sub-subparser
was created with ``required=True``.
"""
_QUERY_HANDLERS[args.query_command](args)
# ═══════════════════════════════════════════════════════════
# Handlers: chat & mcp
# ═══════════════════════════════════════════════════════════
def _handle_chat(args: argparse.Namespace) -> None:
"""Handle ``algonotes chat`` — start an interactive RAG session.
Launches a REPL loop that preserves conversation context across
turns via LangGraph's ``InMemorySaver`` checkpointer. The
``--thread-id`` argument isolates different conversations.
Type ``exit`` or ``quit`` to end the session.
"""
from scripts.chat import chat_loop
chat_loop(args.thread_id)
def _run_mcp(args):
def _handle_mcp(args: argparse.Namespace) -> None:
"""Handle ``algonotes mcp`` — start the MCP server over SSE transport.
Binds to ``--host``:``--port`` (default ``127.0.0.1:8000``) and
exposes nine tools (ingest, update, delete, show, list, search,
metadata, export, ask) that MCP clients such as Claude Desktop
can call.
"""
from src.mcp.server import main as mcp_main
mcp_main(host=args.host, port=args.port)
# ═══════════════════════════════════════════════════════════
# Handler: perf
# ═══════════════════════════════════════════════════════════
def _handle_perf(args: argparse.Namespace) -> None:
"""Handle ``algonotes perf`` — display performance statistics.
Parses ``logs/perf.log``, aggregates latency metrics by call_type,
and prints a summary table. Use ``--detail`` for a per-entry
breakdown. Supports ``--json`` for structured export.
"""
from scripts.perf_report import load_entries, print_report, report_to_dict
entries = load_entries()
if entries:
print_report(entries, detail=args.detail)
_output(report_to_dict(entries), args.json)
# ═══════════════════════════════════════════════════════════
# Dispatch tables
# ═══════════════════════════════════════════════════════════
#: Map ``update_command`` strings to their handler functions.
_UPDATE_HANDLERS = {
"content": _handle_update_content,
"metadata": _handle_update_metadata,
}
#: Map ``query_command`` strings to their handler functions.
_QUERY_HANDLERS = {
"show": _handle_query_show,
"list": _handle_query_list,
"info": _handle_query_info,
"search": _handle_query_search,
"export": _handle_query_export,
"ask": _handle_query_ask,
}
#: Map top-level ``command`` strings to their handler functions.
_HANDLERS = {
"ingest": _handle_ingest,
"update": _handle_update,
"delete": _handle_delete,
"query": _handle_query,
"chat": _handle_chat,
"mcp": _handle_mcp,
"perf": _handle_perf,
}
# ═══════════════════════════════════════════════════════════
# Entry point
# ═══════════════════════════════════════════════════════════
def main() -> None:
"""Entry point for the ``algonotes`` CLI.
Order of operations:
1. Configure logging (checks for ``-v/--verbose`` in raw argv).
2. Ensure stdout uses UTF-8 encoding.
3. Build the full argparse parser via ``_build_parser()``.
4. Parse arguments and dispatch to the matching handler.
"""
_setup_cli_logging()
_ensure_utf8_stdout()
parser = _build_parser()
args = parser.parse_args()
_HANDLERS[args.command](args)
if __name__ == "__main__":
main()

View File

@ -105,7 +105,8 @@ def _save_to_stores(file_name: str, filepath: str, content: str,
def ingest_local(file_path: Path, tag: bool = True,
verbose: bool = False, type: str | None = None,
author: str | None = None) -> IngestResult:
author: str | None = None,
filename: str | None = None) -> IngestResult:
"""Import a single local .md file into three-layer storage.
Loads from filesystem, splits into chunks, extracts tags (optional),
@ -117,12 +118,13 @@ def ingest_local(file_path: Path, tag: bool = True,
verbose: Print processing details to stdout.
type: Note type (note/solution/template).
author: Note author.
filename: Custom filename override. Defaults to the source file's basename.
Returns:
An ``IngestResult`` with metadata of the imported note.
"""
saved_path, content = load_local(
file_path, _file_store, stream=verbose)
file_path, _file_store, stream=verbose, filename=filename)
return _save_to_stores(saved_path.name, str(saved_path.resolve()),
content, source_url=None, type=type, author=author,
tag=tag, verbose=verbose)
@ -163,7 +165,8 @@ def ingest_locals(dir_path: Path, tag: bool = True,
def ingest_web(url: str, tag: bool = True,
verbose: bool = False, type: str | None = None,
author: str | None = None) -> IngestResult:
author: str | None = None,
filename: str | None = None) -> IngestResult:
"""Import a web page into three-layer storage.
Loads via WebBaseLoader, splits into chunks, extracts tags (optional),
@ -175,6 +178,7 @@ def ingest_web(url: str, tag: bool = True,
verbose: Print processing details to stdout.
type: Note type (note/solution/template).
author: Note author.
filename: Custom filename override. Defaults to URL-derived name.
Returns:
An ``IngestResult`` with metadata of the imported note.
@ -183,7 +187,7 @@ def ingest_web(url: str, tag: bool = True,
if verbose:
print(" 清洗: ")
print(" > ", end="")
saved_path, content = load_web(url, _file_store, stream=verbose)
saved_path, content = load_web(url, _file_store, stream=verbose, filename=filename)
result = _save_to_stores(saved_path.name, str(saved_path.resolve()),
content, source_url=url, type=type, author=author,
tag=tag, verbose=verbose)

222
scripts/perf_report.py Normal file
View File

@ -0,0 +1,222 @@
# scripts/perf_report.py
# Performance report: parse logs/perf.log, aggregate by call_type,
# and present latency statistics.
import json
import statistics
from dataclasses import dataclass, field
from pathlib import Path
from typing import Any
from rich.console import Console
from rich.table import Table
console = Console()
DEFAULT_PERF_LOG = Path("logs/perf.log")
@dataclass
class PerfEntry:
"""A single parsed performance log entry."""
timestamp: str
call_type: str
latency_ms: float
success: bool
model: str | None = None
doc_count: int | None = None
error: str | None = None
extra: dict[str, Any] = field(default_factory=dict)
def load_entries(path: Path | None = None) -> list[PerfEntry]:
"""Parse ``logs/perf.log`` into a list of :class:`PerfEntry`.
Args:
path: Path to perf log file. Defaults to ``logs/perf.log``.
Returns:
List of parsed entries (empty if file missing or unparseable).
"""
path = path or DEFAULT_PERF_LOG
if not path.exists():
return []
entries: list[PerfEntry] = []
with path.open(encoding="utf-8") as f:
for line in f:
line = line.strip()
if not line:
continue
try:
obj = json.loads(line)
except json.JSONDecodeError:
continue
entries.append(PerfEntry(
timestamp=obj.get("timestamp", ""),
call_type=obj.get("call_type", "unknown"),
latency_ms=float(obj.get("latency_ms", 0)),
success=bool(obj.get("success", True)),
model=obj.get("model"),
doc_count=obj.get("doc_count"),
error=obj.get("error"),
extra={k: v for k, v in obj.items()
if k not in ("timestamp", "level", "logger", "message",
"call_type", "latency_ms", "success",
"model", "doc_count", "error")},
))
return entries
def aggregate(entries: list[PerfEntry]) -> dict[str, dict[str, Any]]:
"""Aggregate latency statistics grouped by call_type.
Args:
entries: Parsed perf log entries.
Returns:
Dict keyed by call_type, each value containing:
count, total_ms, avg_ms, min_ms, max_ms, p50_ms, p95_ms, p99_ms,
failures, success_rate.
"""
groups: dict[str, list[float]] = {}
failures: dict[str, int] = {}
for e in entries:
groups.setdefault(e.call_type, []).append(e.latency_ms)
if not e.success:
failures[e.call_type] = failures.get(e.call_type, 0) + 1
result: dict[str, dict[str, Any]] = {}
for ct, lats in sorted(groups.items()):
total = len(lats)
fails = failures.get(ct, 0)
sorted_lats = sorted(lats)
result[ct] = {
"count": total,
"total_ms": round(sum(lats), 1),
"avg_ms": round(statistics.mean(lats), 1),
"min_ms": round(min(lats), 1),
"max_ms": round(max(lats), 1),
"p50_ms": round(_percentile(sorted_lats, 50), 1),
"p95_ms": round(_percentile(sorted_lats, 95), 1),
"p99_ms": round(_percentile(sorted_lats, 99), 1),
"failures": fails,
"success_rate": f"{(total - fails) / total * 100:.1f}%",
}
return result
def _percentile(sorted_values: list[float], pct: float) -> float:
"""Compute the pct-th percentile of sorted values (linear interpolation)."""
if not sorted_values:
return 0.0
n = len(sorted_values)
k = (pct / 100) * (n - 1)
f = int(k)
c = k - f
if f + 1 < n:
return sorted_values[f] + c * (sorted_values[f + 1] - sorted_values[f])
return sorted_values[f]
def print_report(
entries: list[PerfEntry],
detail: bool = False,
) -> None:
"""Print a latency statistics report to the console.
Args:
entries: Parsed perf log entries.
detail: If True, also print a per-entry table.
"""
if not entries:
console.print("[yellow]No performance data found.[/yellow]")
console.print(
f"[dim]Run some queries first — logs are stored in "
f"{DEFAULT_PERF_LOG}[/dim]"
)
return
stats = aggregate(entries)
# ── Summary table ───────────────────────────────────
table = Table(title="📈 性能统计 (Perf Log)")
table.add_column("call_type", style="cyan")
table.add_column("次数", justify="right")
table.add_column("avg", justify="right")
table.add_column("P50", justify="right")
table.add_column("P95", justify="right")
table.add_column("P99", justify="right")
table.add_column("min", justify="right")
table.add_column("max", justify="right")
table.add_column("成功率", justify="right")
for ct, s in stats.items():
table.add_row(
ct,
str(s["count"]),
f"{s['avg_ms']:.0f}ms",
f"{s['p50_ms']:.0f}ms",
f"{s['p95_ms']:.0f}ms",
f"{s['p99_ms']:.0f}ms",
f"{s['min_ms']:.0f}ms",
f"{s['max_ms']:.0f}ms",
s["success_rate"],
)
console.print(table)
# ── Detail table ────────────────────────────────────
if detail:
console.print()
dtable = Table(title="📋 详细记录")
dtable.add_column("timestamp", style="dim")
dtable.add_column("call_type", style="cyan")
dtable.add_column("latency", justify="right")
dtable.add_column("status", justify="center")
dtable.add_column("details")
for e in entries:
status = "" if e.success else ""
extras = []
if e.model:
extras.append(f"model={e.model}")
if e.doc_count is not None:
extras.append(f"docs={e.doc_count}")
if e.error:
extras.append(f"error={e.error}")
for k, v in e.extra.items():
if isinstance(v, (int, float, str)):
extras.append(f"{k}={v}")
dtable.add_row(
e.timestamp,
e.call_type,
f"{e.latency_ms:.0f}ms",
status,
", ".join(extras) if extras else "",
)
console.print(dtable)
def report_to_dict(entries: list[PerfEntry]) -> dict[str, Any]:
"""Return aggregated stats + raw entries as a JSON-serializable dict."""
return {
"summary": aggregate(entries),
"total_entries": len(entries),
"entries": [
{
"timestamp": e.timestamp,
"call_type": e.call_type,
"latency_ms": e.latency_ms,
"success": e.success,
"model": e.model,
"doc_count": e.doc_count,
"error": e.error,
**e.extra,
}
for e in entries
],
}

View File

@ -5,6 +5,8 @@ import json
from dataclasses import dataclass, asdict
from typing import Any
from src.performance import PerfTimer
@dataclass
class AskResult:
@ -34,24 +36,28 @@ def ask_question(question: str, stream: bool = False) -> AskResult:
from src.rag.agent import create_rag_agent
agent = create_rag_agent()
config = {"configurable": {"thread_id": "ask"}}
cfg = {"configurable": {"thread_id": "ask"}}
try:
full_response = ""
for event in agent.stream(
{"messages": [("user", question)]},
config,
):
if "model" in event:
content = event["model"]["messages"][-1].content
if content:
if stream:
print(content, flush=True, end="")
full_response += content
with PerfTimer("pipeline") as t:
try:
agent_steps = 0
full_response = ""
for event in agent.stream(
{"messages": [("user", question)]},
cfg,
):
agent_steps += 1
if "model" in event:
content = event["model"]["messages"][-1].content
if content:
if stream:
print(content, flush=True, end="")
full_response += content
if stream:
print()
if stream:
print()
return AskResult(question=question, answer=full_response, success=True)
except Exception as e:
return AskResult(question=question, answer="", success=False, error=str(e))
t.set_extra("agent_steps", agent_steps)
return AskResult(question=question, answer=full_response, success=True)
except Exception as e:
return AskResult(question=question, answer="", success=False, error=str(e))

View File

@ -0,0 +1,48 @@
---
name: icpc-problem-summary
description: 为 OI/ICPC 算法竞赛题目如洛谷、Codeforces 等)精准提炼数学化题意摘要,提取输入变量、约束条件及核心目标关系,默认输出为一句话版本(含题目来源、数据范围和核心公式),也可按需输出多句详细拆解。当用户直接粘贴大段题目原文并要求“提炼题意”、“概括题目”、“一句话总结”、“抽象成数学模型”或“快速理解这道题在求什么”时触发;也适用于用户提供平台题目链接并要求极简核心逻辑概述的场景。
license: MIT
metadata:
version: 1.0
author: ttzc@github.com
---
# 算法题意提炼助手
本技能用于将任意 OI/ICPC 算法竞赛题目如洛谷、Codeforces 等)的描述,精准提炼为一句话核心题意或多句话数学模型,便于快速理解题目本质与建模要点,或用于撰写题解。
**功能与流程**
1. **识别变量**:提取所有输入参数(如数组元素、数据规模、目标值等),并标注其类型、取值范围(按题目说明)。
2. **明确目标**:确定输出变量(如最大值、最小值、总和、是否存在等),并用字母表示。
3. **表达约束与关系**:用数学符号(`\sum`、`\max`、`\min`、`\lfloor\rfloor`、条件指示函数等)准确描述运算逻辑、判定条件和特殊细节(如“不同位置算不同”、“连续段”等)。
4. **整合为一句或多句**:将所有已知量、未知量及关系融入一个或多个连贯的自然语句,优先使用公式替代冗长文字,所有公式用 `$$` 包裹。具体输出规模按照用户实际需求,若未指定则默认输出一句话版本。
5. **统一输出格式**:若题目来自洛谷,前缀为 **“洛谷 Pxxxx 题目名”**;若来自 Codeforces前缀为 **“Codeforces xxx[A/B/...] 题目名”**,其余平台格式类似。输出中务必包含所有关键范围(如 `1≤N≤10^5`)和核心目标表达式。你可以参考后面的示例。
**输入**:用户粘贴的完整题目原文(包含背景、输入输出格式、样例、数据范围)。
**输出**:一句话版本,仅输出一行符合上述格式的总结句,不附加任何额外解释或分析。多句话版本类似,参考后面示例。
**示例输出**(供参考格式):
单句版本:
- **洛谷 P1873 砍树**:已知 N1≤N≤10^6棵树的高度 h_ih_i≤4×10^5和所需木材总长 M1≤M≤2×10^9且 ∑h_i > M求最大的整数高度 H使得锯下的木材总量 ∑max(0, h_iH) ≥ M。
- **Codeforces 780B The Meeting Place Cannot Be Changed**:已知朋友数 n2≤n≤60000、初始位置 x_i 和最大速度 v_i1≤x_i,v_i≤10^9求最小时间 T使得存在实数 X 满足 ∀i|Xx_i|≤v_i⋅T。
- **洛谷 P1314 聪明的质监员**:已知矿石数 $n$$1\le n\le 2\times10^5$)、区间数 $m$$1\le m\le 2\times10^5$)、标准值 $s$$0<s\le10^{12}$每个矿石 $i$ 有重量 $w_i$ 和价值 $v_i$$0<w_i,v_i\le10^6$以及 $m$ 个区间 $[l_i,r_i]$$1\le l_i\le r_i\le n$定义检验值 $y=\sum_{i=1}^m \left( \sum_{j=l_i}^{r_i} [w_j\ge W] \right) \cdot \left( \sum_{j=l_i}^{r_i} [w_j\ge W]\cdot v_j \right)$其中 $W$ 为可选参数整数 $\min_{W} |s-y|$。
多句版本:
给定 $n$ 个矿石,每个有重量 $w_i$ 和价值 $v_i$,以及 $m$ 个区间 $[l_i, r_i]$。选择一个参数 $W$,定义区间 $i$ 的检验值:
$$
y_i = \left( \sum_{j=l_i}^{r_i} [w_j \ge W] \right) \times \left( \sum_{j=l_i}^{r_i} [w_j \ge W] \cdot v_j \right)
$$
总检验值 $Y(W) = \sum_{i=1}^m y_i$。给定标准值 $s$,求 $\min_{W \in \mathbb{Z}} |Y(W) - s|$。
数据范围:$1 \le n,m \le 2\times 10^5$$0 < w_i, v_i \le 10^6$$0 < s \le 10^{12}$$1 \le l_i \le r_i \le n$。
**适用场景**
- 快速理解新题的数学模型;
- 为编写题解、设计算法或生成测试用例提供精确的题意摘要;
- 作为后续分类、检索或比对相似题目的基础元数据。
**注意**:本技能仅提炼题意,不直接解答或编写代码。

View File

@ -0,0 +1,83 @@
---
name: icpc-solution-writer
description: 为 ICPC/OI 竞赛题目生成高质量、结构化的题解Solution Editorial包含试题元数据、暴力推导、核心算法、正确性证明、复杂度分析、坑点及参考代码。当用户要求“写题解”、“生成解题报告”、“详细解释算法问题并写成题解”或提供比赛题目链接和解题思路时触发。
license: MIT
metadata:
version: 1.0
author: ttzc@github.com
user-invocable: true
context:
- *.cpp
- *.md
---
# ICPC/OI 题目题解生成专家
你现在是顶级的 ICPC 金牌选手兼题解作者。当用户给出题目(文字描述或链接)时,请严格按照以下 9 个部分输出高质量的完整题解。必须确保逻辑严谨、代码正确、新手友好。
## 输出结构(严格按此顺序)
### 1. 元数据 (Metadata)
- **题目链接**:提供试题提交的直达链接,或者原题链接
- **时间限制**:例如 2 秒。
- **内存限制**:例如 256 MB。
- **如果你不知道任何一项信息,可以写占位符待用户补充,但是不要胡编乱造**
### 2. 题意简述 (Problem Summary)
- 用 2-3 句话剥离故事背景,直击核心。
- 请遵循以下步骤,总结核心题意:
1. **识别输入变量**:找出所有输入参数(如数据规模 `N`、目标值 `M``C`,以及数组元素 `a_i` 等),为每个变量赋予标准字母,并从题目说明中提取其取值范围(包括上下界和类型)。
2. **识别输出变量**:明确题目要求计算的最终结果(如最大高度 `H`、满足条件的数对个数等),同样用字母表示。
3. **提取核心约束与目标**:用数学表达式(如求和 `\sum`、最大值/最小值、等式/不等式、条件判断等)准确描述变量之间的运算关系或判定条件。注意特殊细节(如“不同位置算不同”等)需体现在条件中。
4. **整合成一句陈述**:将所有已知量、未知量及关系糅合为一个连贯的自然语句,优先使用公式符号(如 `≤`、`∑`、`max`)替代冗长文字,变量范围用 `1≤N≤10^6` 等标准格式注明。
5. **检查公式格式**:通常情况下,用 $ 包裹所有数学公式和符号,以保证公式的正常显示。如果用户或使用的编辑工具对公式的输出有特殊要求,则服从特殊要求
### 3. 朴素解法 (Brute-Force)
- 描述最简单的思路(如 O(N^2) 或 O(2^N))。
- 说明其瓶颈,并指出在哪些数据范围下会超时。
- 通常情况下用户会提供相关思路,若未提供则简述问题空间、枚举方法即可。
### 4. 核心解法 (Main Solution)
- **特殊性质**:说明利用了问题的何种特殊性质(如单调性、交换律结合律、可二分性、动态规划的最优子结构性、等等)。
- **关键突破**:从朴素解法的瓶颈出发,结合问题性质,引出核心优化思路(算法或数据结构)。
- **推导过程**:利用性质分步骤写出递推公式或核心逻辑,使用 LaTeX 公式(如 `$dp[i] = \max(dp[i-1], ...)$`)。
### 5. 正确性证明 (Proof of Correctness)
- **贪心**:使用交换论证或归纳法。
- **DP**:证明最优子结构和无后效性。
- **二分**:证明判定函数的二段性。
- 其他问题类似,保持简洁但逻辑闭环。
### 6. 复杂度分析 (Complexity)
- **时间复杂度**:给出 Big-O 并简要说明为何能通过本题。
- **空间复杂度**:注明是否使用了滚动数组等优化。
### 7. 实现细节与避坑指南 (Implementation Details)
列出关键陷阱:
- 整数溢出(提醒用 `long long``__int128`)。
- 初始化值INF 的大小、多组数据的清空)。
- 边界条件N=0, N=1, 空串, 图不连通等)。
### 8. 参考代码 (Reference Code)
- 提供 **C++14+****Python 3** 的可直接提交的完整代码,或者修改用户提供的代码。
- 使用有意义的变量名,关键逻辑处添加注释(不要逐行注释语法)。
- 代码必须规范化(使用代码格式化工具处理后的结果),竞赛化(单文件,简洁)。
### 9. 补充说明 (Additional Notes - 可选)
- 提及该题是哪个经典问题的变种。
- 提供其他的优化方向(如换用更高级的数据结构)。
- 码风是否可以更简洁易读。
## 写作风格约束
- 语言:清晰、简洁,像在给朋友耐心讲解。
- 严禁闲聊:禁止输出“让我们来思考”、“首先”等寒暄词汇,直接输出题解正文。
- 公式:全部使用 Markdown + LaTeX 渲染。
## 最终自检清单(在输出前内心确认)
1. 算法是否能真正 AC 所有数据?
2. 复杂度计算是否准确无误?
3. 代码是否无编译错误并能处理极端边界?
4. 新手看完整篇是否理解了来龙去脉?
现在,请直接开始处理用户提供的题目,不要输出任何多余的开场白。

View File

@ -14,6 +14,7 @@ from dataclasses import dataclass
import httpx
from src.config import config
from src.performance import PerfTimer
logger = logging.getLogger("algonotes.api.reranker_client")
@ -116,30 +117,45 @@ class GiteeAIReranker(BaseReranker):
# "Content-Type": "application/json",
}
try:
with httpx.Client(timeout=30.0) as client:
logger.info(f"POST {url}")
# logger.info(f"headers: {headers}")
logger.info(f"body: {payload}")
resp = client.post(url, json=payload, headers=headers)
resp.raise_for_status()
data = resp.json()
except httpx.HTTPStatusError as e:
logger.error(
f"Rerank API error: {e.response.status_code} {e.response.text[:200]}")
# logger.error(e.response)
raise
except httpx.RequestError as e:
logger.error(f"Rerank request failed: {e}")
raise
if logger.isEnabledFor(logging.DEBUG):
logger.debug(
f"Rerank payload: query='{query[:100]}...', "
f"docs_count={len(documents)}, "
f"total_chars={sum(len(d) for d in documents)}"
)
results = []
for item in data.get("results", []):
results.append(RerankResult(
index=item["index"],
text=item["document"]["text"],
relevance_score=item["relevance_score"],
))
with PerfTimer("reranker", model=self._model, doc_count=len(documents),
extra={"top_n": top_n}) as t:
try:
with httpx.Client(timeout=30.0) as client:
resp = client.post(url, json=payload, headers=headers)
resp.raise_for_status()
data = resp.json()
except httpx.HTTPStatusError as e:
logger.error(
f"Rerank API error: {e.response.status_code} "
f"{e.response.text[:200]}"
)
raise
except httpx.RequestError as e:
logger.error(f"Rerank request failed: {e}")
raise
results = []
for item in data.get("results", []):
results.append(RerankResult(
index=item["index"],
text=item["document"]["text"],
relevance_score=item["relevance_score"],
))
if results:
scores = [r.relevance_score for r in results]
t.set_extra("score_range", {
"min": round(min(scores), 4),
"max": round(max(scores), 4),
"mean": round(sum(scores) / len(scores), 4),
})
logger.info(f"Rerank returned {len(results)} results")
return results

View File

@ -6,8 +6,8 @@
#
# Usage:
# from src.config import config
# config.llm.model # → "DeepSeek-R1"
# config.embedding.model # → "Qwen/Qwen3-Embedding-8B"
# config.llm.model # → "Qwen3-Next-80B-A3B-Instruct" (取决于 config.toml)
# config.embedding.model # → "Qwen3-Embedding-4B" (取决于 config.toml)
import os
import re
@ -75,9 +75,9 @@ def _expand_dict(d: dict) -> dict:
class LLMConfig(BaseModel):
"""Configuration for the chat model (OpenAI-compatible API)."""
model: str = "DeepSeek-R1"
model: str = "Qwen3-Next-80B-A3B-Instruct"
base_url: str = "https://ai.gitee.com/v1"
api_key: str = "${OPENAI_API_KEY}"
api_key: str = "${GITEE_API_KEY}"
temperature: float = 0.7
max_tokens: int = 4096
timeout: float = 60.0
@ -86,9 +86,9 @@ class LLMConfig(BaseModel):
class EmbeddingConfig(BaseModel):
"""Configuration for the embedding model (OpenAI-compatible API)."""
model: str = "Qwen/Qwen3-Embedding-8B"
model: str = "Qwen3-Embedding-4B"
base_url: str = "https://ai.gitee.com/v1"
api_key: str = "${OPENAI_API_KEY}"
api_key: str = "${GITEE_API_KEY}"
class LoggingConfig(BaseModel):
@ -100,6 +100,28 @@ class LoggingConfig(BaseModel):
backup_count: int = 5
class PerfLoggingConfig(BaseModel):
"""Configuration for the performance logging channel (logs/perf.log)."""
enabled: bool = True
file: str = "logs/perf.log"
max_bytes: int = 10 * 1024 * 1024 # 10MB
backup_count: int = 3
class LangSmithConfig(BaseModel):
"""Configuration for LangSmith tracing.
api_key must use ${LANGSMITH_API_KEY} to reference the .env file
never hardcode the key in config.toml.
"""
enabled: bool = True
api_key: str = "${LANGSMITH_API_KEY}"
project: str = "algonotes-rag"
endpoint: str = "https://api.smith.langchain.com"
class StoreConfig(BaseModel):
"""Configuration for the three-layer data storage paths."""
@ -113,7 +135,7 @@ class RerankerConfig(BaseModel):
model: str = "Qwen3-Reranker-4B"
base_url: str = "https://ai.gitee.com/v1"
api_key: str = "${OPENAI_API_KEY}"
api_key: str = "${GITEE_API_KEY}"
top_n: int = 3
@ -131,6 +153,8 @@ class AppConfig(BaseModel):
embedding: EmbeddingConfig = EmbeddingConfig()
reranker: RerankerConfig = RerankerConfig()
logging: LoggingConfig = LoggingConfig()
perf_logging: PerfLoggingConfig = PerfLoggingConfig()
langsmith: LangSmithConfig = LangSmithConfig()
store: StoreConfig = StoreConfig()
cp_graph: CPGraphConfig = CPGraphConfig()
@ -156,3 +180,24 @@ def _load() -> AppConfig:
config: AppConfig = _load()
def _sync_langsmith_env() -> None:
"""Sync LangSmith config back to environment variables.
LangChain reads ``LANGSMITH_TRACING`` / ``LANGSMITH_API_KEY`` / etc.
directly from ``os.environ`` at runtime it does NOT look at our config
models. This function ensures that ``config.toml`` ``[langsmith]`` is the
single source of truth: what you set there actually controls LangSmith
behaviour.
Must be called at module-load time, *before* any ``langchain*`` import.
"""
cfg = config.langsmith
os.environ["LANGSMITH_TRACING"] = "true" if cfg.enabled else "false"
os.environ["LANGSMITH_API_KEY"] = cfg.api_key
os.environ["LANGSMITH_PROJECT"] = cfg.project
os.environ["LANGSMITH_ENDPOINT"] = cfg.endpoint
_sync_langsmith_env()

View File

@ -11,6 +11,8 @@ from pathlib import Path
from langchain_core.prompts import PromptTemplate
from src.api.llm_client import get_chat_model
from src.config import config
from src.performance import PerfTimer
logger = logging.getLogger("algonotes.ingestion.cleaner")
@ -55,11 +57,13 @@ def clean_text(raw_text: str, stream: bool = False, max_retries: int = 3) -> str
try:
prompt = CLEAN_PROMPT.format(text=raw_text)
cleaned = ""
for chunk in model.stream(prompt):
cleaned += chunk.content
if stream:
print(chunk.content, end="", flush=True)
with PerfTimer("text_cleaning", model=config.llm.model,
token_count=len(raw_text)):
cleaned = ""
for chunk in model.stream(prompt):
cleaned += chunk.content
if stream:
print(chunk.content, end="", flush=True)
logger.info(f"Text cleaned successfully (length: {len(cleaned)})")
return _strip_code_block_wrapper(cleaned)

View File

@ -52,13 +52,15 @@ def load_text(content: str, file_name: str, file_store: FileStore, clean: bool =
return Path(saved_path), content
def load_local(filepath: str | Path, file_store: FileStore, stream: bool = False) -> tuple[Path, str]:
def load_local(filepath: str | Path, file_store: FileStore, stream: bool = False,
filename: str | None = None) -> tuple[Path, str]:
"""Load a local .md file and save to FileStore.
Args:
filepath: Path to the local markdown file.
file_store: FileStore instance for saving.
stream: Stream cleaning output to stdout.
filename: Custom filename override. Defaults to the source file's basename.
Returns:
Tuple of (saved_path, content).
@ -67,7 +69,7 @@ def load_local(filepath: str | Path, file_store: FileStore, stream: bool = False
if not path.exists():
raise FileNotFoundError(f"File not found: {filepath}")
filename = path.name
filename = filename or path.name
content = path.read_text(encoding="utf-8")
saved_path, content = load_text(
content, filename, file_store, False, stream)
@ -76,7 +78,8 @@ def load_local(filepath: str | Path, file_store: FileStore, stream: bool = False
return saved_path, content
def load_web(url: str, file_store: FileStore, stream: bool = False) -> tuple[Path, str]:
def load_web(url: str, file_store: FileStore, stream: bool = False,
filename: str | None = None) -> tuple[Path, str]:
"""Load a web page, optionally clean it, and save to FileStore.
Uses a Chrome 120 browser header template to avoid blocking.
@ -85,6 +88,7 @@ def load_web(url: str, file_store: FileStore, stream: bool = False) -> tuple[Pat
url: The URL of the web page.
file_store: FileStore instance for saving.
stream: Stream cleaning output to stdout.
filename: Custom filename override. Defaults to URL-derived name.
Returns:
Tuple of (saved_path, content).
@ -100,7 +104,7 @@ def load_web(url: str, file_store: FileStore, stream: bool = False) -> tuple[Pat
raw_text = docs[0].page_content
logger.info(f"Loaded {len(raw_text)} chars from web page")
filename = _url_to_filename(url)
filename = filename or _url_to_filename(url)
saved_path, content = load_text(raw_text, filename, file_store, True, stream)
logger.info(f"Saved web content: {filename} -> {saved_path}")

View File

@ -11,6 +11,8 @@ from pathlib import Path
from langchain_core.prompts import PromptTemplate
from src.api.llm_client import get_chat_model
from src.config import config
from src.performance import PerfTimer
from src.store.file_store import FileStore
logger = logging.getLogger("algonotes.ingestion.tagger")
@ -43,7 +45,8 @@ def extract_tags(filename: str, file_store: FileStore | None = None) -> str:
logger.info(f"Extracting tags for '{filename}'")
try:
response = model.invoke(prompt)
with PerfTimer("tag_extraction", model=config.llm.model):
response = model.invoke(prompt)
tags = response.content.strip()
tags = re.sub(r"```[\w]*\n?|```", "", tags).strip()
logger.info(f"Tags for '{filename}': {tags}")

View File

@ -1,79 +1,312 @@
# src/logger.py
# JSON Lines logging system for AlgoNotes RAG.
#
# By default logs only to file. Pass ``console=True`` to also emit
# human-readable plain text to stderr (used by CLI ``--verbose``).
# Dual-channel architecture:
# 1. Operational logs → logs/app.log (OpJsonFormatter)
# 2. Performance logs → logs/perf.log (PerfJsonFormatter + trace_id injection)
#
# Pass ``console=True`` to also emit human-readable plain text to stderr.
import contextvars
import json
import sys
from pathlib import Path
import logging
import sys
from logging.handlers import RotatingFileHandler
from pathlib import Path
from typing import Any
from src.config import config
# ═══════════════════════════════════════════════════════════
# contextvars — trace_id injection for LangSmith correlation
# ═══════════════════════════════════════════════════════════
class JsonFormatter(logging.Formatter):
"""JSON Lines formatter: output each log record as a single JSON line."""
_trace_id: contextvars.ContextVar[str | None] = contextvars.ContextVar(
"trace_id", default=None
)
def format(self, record):
log_entry = {
def set_trace_id(trace_id: str) -> None:
"""Manually set the current LangSmith trace / run ID.
Call this at the start of a request when operating outside
LangChain's automatic context (e.g. raw CLI invocations).
"""
_trace_id.set(trace_id)
def _get_current_trace_id() -> str | None:
"""Read the LangSmith run ID from contextvars or the langsmith SDK.
Priority:
1. Explicitly-set ``_trace_id`` contextvar (via :func:`set_trace_id`)
2. LangSmith's own ``get_current_run_tree()`` contextvar
Returns ``None`` when no trace context is active.
"""
tid = _trace_id.get()
if tid:
return tid
try:
# langsmith is a transitive dependency of langchain
from langsmith.run_helpers import get_current_run_tree
tree = get_current_run_tree()
if tree is not None:
return str(tree.id)
except Exception:
pass
return None
# ═══════════════════════════════════════════════════════════
# Build exclusion set for standard LogRecord attributes
# ═══════════════════════════════════════════════════════════
def _build_standard_keys() -> set[str]:
"""Return the set of attribute names that belong to a plain LogRecord.
Used by :class:`OpJsonFormatter` to distinguish between built-in
LogRecord fields and user-supplied ``extra`` fields.
"""
ref = logging.makeLogRecord({
"msg": "", "args": (), "name": "r",
"levelno": 0, "levelname": "",
"pathname": "", "filename": "", "module": "", "lineno": 0,
"funcName": "", "created": 0, "msecs": 0, "relativeCreated": 0,
"thread": 0, "threadName": "", "process": 0, "processName": "",
"taskName": "", "exc_info": None, "exc_text": None, "stack_info": None,
})
# instance attributes + class-level attributes (methods, descriptors, …)
return set(ref.__dict__.keys()) | {
k for k in logging.LogRecord.__dict__ if not k.startswith("__")
}
_STANDARD_KEYS: set[str] = _build_standard_keys()
# ═══════════════════════════════════════════════════════════
# Formatters
# ═══════════════════════════════════════════════════════════
class OpJsonFormatter(logging.Formatter):
"""Operational log JSON Lines formatter.
Output::
{"timestamp": "", "level": "INFO", "logger": "", "message": "", extra}
Fixed fields (always present): ``timestamp``, ``level``, ``logger``, ``message``.
Extra fields passed via ``logger.info("msg", extra={})`` are **auto-expanded**
into the JSON object no need to modify the formatter when adding new fields.
"""
_FIXED_FIELDS = {"timestamp", "level", "logger", "message"}
def format(self, record: logging.LogRecord) -> str:
entry: dict[str, Any] = {
"timestamp": self.formatTime(record),
"level": record.levelname,
"logger": record.name,
"message": record.getMessage(),
"latency_ms": getattr(record, "latency_ms", None),
"tokens": getattr(record, "tokens", None),
}
return json.dumps(log_entry, ensure_ascii=False)
# Auto-expand extra fields — keep any attribute that is NOT a
# standard LogRecord field and NOT one of the fixed keys above.
for key in record.__dict__:
if key in self._FIXED_FIELDS:
continue
if key.startswith("_"):
continue
if key in _STANDARD_KEYS:
continue
entry[key] = record.__dict__[key]
return json.dumps(entry, ensure_ascii=False, default=str)
def setup_logger(name: str = "algonotes", *, console: bool = False) -> logging.Logger:
"""Set up and return a configured logger.
class PerfJsonFormatter(OpJsonFormatter):
"""Performance log JSON Lines formatter.
Always creates a file handler (JSON Lines, DEBUG level, with rotation).
Optionally creates a stderr handler (human-readable plain text, INFO level).
Extends :class:`OpJsonFormatter` by injecting ``trace_id`` from
contextvars (LangSmith run ID) onto every record before formatting.
Configures the root ``algonotes`` logger safe to call multiple times;
subsequent calls are no-ops.
If the record already has a ``trace_id`` (passed explicitly via
``extra``), it is left untouched the caller's value wins.
"""
def format(self, record: logging.LogRecord) -> str:
# Only inject if the caller didn't already provide one via extra=
if "trace_id" not in record.__dict__ or record.__dict__.get("trace_id") is None:
tid = _get_current_trace_id()
if tid:
record.__dict__["trace_id"] = tid
return super().format(record)
# ═══════════════════════════════════════════════════════════
# Performance logging helper
# ═══════════════════════════════════════════════════════════
def log_perf(
call_type: str,
latency_ms: float,
*,
model: str | None = None,
doc_count: int | None = None,
chunk_count: int | None = None,
token_count: int | None = None,
success: bool = True,
error: str | None = None,
extra: dict[str, Any] | None = None,
) -> None:
"""Write a performance log entry to ``logs/perf.log``.
All keyword arguments except ``extra`` are type-checked this
prevents typos in field names that a raw ``logger.info(, extra={})``
would silently accept.
Args:
name: Logger name, defaults to ``algonotes``.
console: When True, additionally emit human-readable logs to stderr.
call_type: Logical operation name (``"reranker"``, ``"pipeline"``, ).
latency_ms: Elapsed wall-clock time in milliseconds.
model: Optional model name / identifier.
doc_count: Number of documents processed.
chunk_count: Number of chunks produced or consumed.
token_count: Total tokens consumed (prompt + completion).
success: ``False`` when the call raised an exception.
error: Exception message when ``success=False``.
extra: Arbitrary additional fields merged into the log entry.
"""
perf_logger = logging.getLogger("algonotes.perf")
perf_data: dict[str, Any] = {
"call_type": call_type,
"latency_ms": round(latency_ms, 2),
"model": model,
"doc_count": doc_count,
"chunk_count": chunk_count,
"token_count": token_count,
"success": success,
"error": error,
}
if extra:
perf_data.update(extra)
# Strip None values to keep log lines compact
perf_data = {k: v for k, v in perf_data.items() if v is not None}
perf_logger.info(
f"{call_type} completed in {latency_ms:.1f}ms", extra=perf_data
)
# ═══════════════════════════════════════════════════════════
# Logger setup — dual-channel
# ═══════════════════════════════════════════════════════════
def setup_logger(
name: str = "algonotes", *, console: bool = False
) -> logging.Logger:
"""Initialize the dual-channel logging system.
**Channel isolation** (the key design decision)::
algonotes (root) op_handler logs/app.log
algonotes.api.* (propagates to root app.log)
algonotes.ingestion.* (propagates to root app.log)
algonotes.perf perf_handler logs/perf.log
(propagate=False isolated from root, never reaches app.log)
Because ``algonotes.perf`` sets ``propagate = False``, operational logs
never leak into ``perf.log``, and perf logs never leak into ``app.log``.
Handlers created:
1. **Operational log handler** ``logs/app.log``
- Attached to root ``algonotes``
- Formatter: :class:`OpJsonFormatter`
- Level: ``DEBUG`` (filtered by ``[logging].level``)
2. **Performance log handler** ``logs/perf.log``
- Attached to ``algonotes.perf`` (isolated, ``propagate = False``)
- Formatter: :class:`PerfJsonFormatter` (auto-injects ``trace_id``)
- Level: ``INFO``
- Skipped when ``perf_logging.enabled = false``
3. **Console handler** (optional) stderr
- Attached to root ``algonotes``
- Formatter: plain text
- Created only when ``console=True``
Safe to call multiple times subsequent calls are no-ops.
Args:
name: Logger name to return (child of root ``algonotes``).
console: When ``True``, also emit plain-text logs to stderr.
Returns:
The named child logger of the ``algonotes`` root.
The named child logger.
"""
root = logging.getLogger("algonotes")
if root.handlers:
return logging.getLogger(name)
log_cfg = config.logging
root.setLevel(getattr(logging, log_cfg.level.upper(), logging.INFO))
# ensure log directory exists
log_path = Path(log_cfg.file)
log_path.parent.mkdir(parents=True, exist_ok=True)
# ── Handler 1: Operational logs (root) ──────────────
op_path = Path(log_cfg.file)
op_path.parent.mkdir(parents=True, exist_ok=True)
# Handler 1: file (machine-readable, JSON Lines)
file_handler = RotatingFileHandler(
op_handler = RotatingFileHandler(
log_cfg.file,
maxBytes=log_cfg.max_bytes,
backupCount=log_cfg.backup_count,
encoding="utf-8",
)
file_handler.setLevel(logging.DEBUG)
file_handler.setFormatter(JsonFormatter())
root.addHandler(file_handler)
op_handler.setLevel(logging.DEBUG)
op_handler.setFormatter(OpJsonFormatter())
root.addHandler(op_handler)
# Handler 2: stderr (human-friendly, plain text) — only when requested
# ── Handler 2: Performance logs (isolated) ──────────
# Attached to algonotes.perf, NOT the root — this prevents
# operational logs from polluting perf.log and vice versa.
perf_cfg = config.perf_logging
if perf_cfg.enabled:
perf_path = Path(perf_cfg.file)
perf_path.parent.mkdir(parents=True, exist_ok=True)
perf_handler = RotatingFileHandler(
perf_cfg.file,
maxBytes=perf_cfg.max_bytes,
backupCount=perf_cfg.backup_count,
encoding="utf-8",
)
perf_handler.setLevel(logging.INFO)
perf_handler.setFormatter(PerfJsonFormatter())
perf_logger = logging.getLogger("algonotes.perf")
perf_logger.propagate = False # isolate: no leakage to root (app.log)
perf_logger.addHandler(perf_handler)
# ── Handler 3: Console (stderr, plain text) ─────────
if console:
stderr = logging.StreamHandler(sys.stderr)
stderr.setLevel(logging.INFO)
stderr.setFormatter(logging.Formatter(
"%(asctime)s - %(levelname)s - %(name)s - %(message)s"
))
stderr.setFormatter(
logging.Formatter(
"%(asctime)s - %(levelname)s - %(name)s - %(message)s"
)
)
root.addHandler(stderr)
# ── Silence noisy third-party loggers ───────────────
logging.getLogger("httpx").setLevel(logging.WARNING)
logging.getLogger("chromadb").setLevel(logging.WARNING)
logging.getLogger("urllib3").setLevel(logging.WARNING)
return logging.getLogger(name)

View File

@ -1,7 +1,7 @@
# src/mcp/server.py
# MCP server for AlgoNotes RAG.
#
# Exposes ingest/update/delete/search tools via SSE transport.
# Exposes ingest/update/delete/search/ask tools via SSE transport.
# Usage: algonotes mcp --port 8000
from mcp.server.fastmcp import FastMCP
@ -18,32 +18,18 @@ from src.mcp.tools import (
update_tool,
)
_MCP_INSTANCE: FastMCP | None = None
# ═══════════════════════════════════════════════════════════════
# Tool wrapper functions (plain — decorated at server creation)
# ═══════════════════════════════════════════════════════════════
def get_mcp(host: str = "127.0.0.1", port: int = 8000) -> FastMCP:
"""Return the singleton FastMCP instance, creating it if needed."""
global _MCP_INSTANCE
if _MCP_INSTANCE is None:
_MCP_INSTANCE = FastMCP(
"algonotes",
instructions=(
"AlgoNotes RAG MCP server. Manage algorithm competition notes "
"(OI/ICPC) via ingest, update, delete, and search tools."
),
host=host,
port=port,
)
return _MCP_INSTANCE
@get_mcp().tool()
def ingest(
def _ingest(
path: str | None = None,
url: str | None = None,
tag: bool = True,
type: str = "note",
author: str | None = None,
filename: str | None = None,
) -> dict:
"""Import notes into three-layer storage.
@ -56,13 +42,13 @@ def ingest(
tag: Extract tags via LLM (default: true).
type: Note type (note/solution/template, default: note).
author: Note author.
filename: Custom filename for the imported note (optional).
"""
return ingest_tool(path=path, url=url, tag=tag,
type=type, author=author)
type=type, author=author, filename=filename)
@get_mcp().tool()
def update(
def _update(
filename: str,
file_path: str | None = None,
tag: bool = True,
@ -80,8 +66,7 @@ def update(
return update_tool(filename=filename, file_path=file_path, tag=tag)
@get_mcp().tool()
def delete(filename: str) -> dict:
def _delete(filename: str) -> dict:
"""Delete a note from all three storage layers.
Deletes in order: VectorStore, FileStore, SQLStore.
@ -92,8 +77,7 @@ def delete(filename: str) -> dict:
return delete_tool(filename=filename)
@get_mcp().tool()
def show(filename: str, lines: int | None = None) -> dict:
def _show(filename: str, lines: int | None = None) -> dict:
"""Read the raw content of a note file.
Args:
@ -103,8 +87,7 @@ def show(filename: str, lines: int | None = None) -> dict:
return show_tool(filename=filename, lines=lines)
@get_mcp().tool()
def list(tag: str | None = None) -> dict:
def _list(tag: str | None = None) -> dict:
"""List all notes, optionally filtered by tag.
Args:
@ -113,8 +96,7 @@ def list(tag: str | None = None) -> dict:
return list_tool(tag=tag)
@get_mcp().tool()
def search(query: str, top_k: int = 5) -> dict:
def _search(query: str, top_k: int = 5) -> dict:
"""Semantic search across all notes.
Args:
@ -124,8 +106,7 @@ def search(query: str, top_k: int = 5) -> dict:
return search_tool(query=query, top_k=top_k)
@get_mcp().tool()
def metadata(
def _metadata(
filename: str,
title: str | None = None,
tags: str | None = None,
@ -145,8 +126,7 @@ def metadata(
author=author, type=type)
@get_mcp().tool()
def export(
def _export(
filename: str | None = None,
all: bool = False,
output_dir: str = "./export",
@ -163,8 +143,7 @@ def export(
return export_tool(filename=filename, all=all, output_dir=output_dir)
@get_mcp().tool()
def ask(
def _ask(
question: str,
stream: bool = False,
) -> dict:
@ -179,6 +158,40 @@ def ask(
return ask_tool(question=question, stream=stream)
#: All tool wrapper functions, registered in order when the server is created.
_TOOLS = [
_ingest,
_update,
_delete,
_show,
_list,
_search,
_metadata,
_export,
_ask,
]
def get_mcp(host: str = "127.0.0.1", port: int = 8000) -> FastMCP:
"""Create a FastMCP instance bound to *host*:*port* with all tools registered.
Unlike the previous singleton, every call creates a fresh instance so
``--host`` / ``--port`` are always honoured.
"""
mcp = FastMCP(
"algonotes",
instructions=(
"AlgoNotes RAG MCP server. Manage algorithm competition notes "
"(OI/ICPC) via ingest, update, delete, and search tools."
),
host=host,
port=port,
)
for tool_func in _TOOLS:
mcp.tool()(tool_func)
return mcp
def main(host: str = "127.0.0.1", port: int = 8000):
"""Run the MCP server with SSE transport (HTTP).

View File

@ -16,6 +16,7 @@ def ingest_tool(
tag: bool = True,
type: str = "note",
author: str | None = None,
filename: str | None = None,
) -> dict[str, Any]:
if not path and not url:
return {"success": False, "error": "Must provide either path or url"}
@ -27,7 +28,7 @@ def ingest_tool(
try:
if url:
result = ingest_web(url, tag=tag, verbose=False,
type=type, author=author)
type=type, author=author, filename=filename)
return {
"file_name": result.file_name,
"chunk_count": result.chunk_count,
@ -39,7 +40,7 @@ def ingest_tool(
file_path = Path(path)
if file_path.is_file():
result = ingest_local(file_path, tag=tag, verbose=False,
type=type, author=author)
type=type, author=author, filename=filename)
return {
"file_name": result.file_name,
"chunk_count": result.chunk_count,

112
src/performance.py Normal file
View File

@ -0,0 +1,112 @@
# src/performance.py
# Performance timer for AlgoNotes RAG.
#
# Provides :class:`PerfTimer` — a context manager + decorator that measures
# elapsed wall-clock time and writes a structured performance log entry via
# :func:`~src.logger.log_perf` on exit.
#
# Usage (context manager)::
#
# with PerfTimer("reranker", model="Qwen3-Reranker-4B", doc_count=10) as t:
# results = reranker.rerank(query, docs)
# t.set_extra("score_range", {"min": 0.32, "max": 0.95})
#
# Usage (decorator)::
#
# @PerfTimer("pipeline")
# def ask_question(question: str) -> str:
# ...
import time
from contextlib import ContextDecorator
from typing import Any, Literal
from src.logger import log_perf
# ═══════════════════════════════════════════════════════════
# CallType — constrained literal for call_type values
# ═══════════════════════════════════════════════════════════
CallType = Literal[
"reranker", # Gitee.AI /v1/rerank call (raw httpx)
"pipeline", # end-to-end RAG ask / ingest pipeline
"ingest", # single-note ingestion (load → split → tag → store)
"tag_extraction", # LLM tag extraction (tagger.py)
"text_cleaning", # LLM text cleaning (cleaner.py)
]
# ═══════════════════════════════════════════════════════════
# PerfTimer
# ═══════════════════════════════════════════════════════════
class PerfTimer(ContextDecorator):
"""Context manager + decorator for performance measurement.
On exit, automatically calls :func:`log_perf` with the elapsed time
and all metadata supplied at construction time (plus any fields added
via :meth:`set_extra` during the block).
Does **not** swallow exceptions they propagate normally after the
performance entry is written with ``success=False``.
"""
def __init__(
self,
call_type: CallType,
*,
model: str | None = None,
doc_count: int | None = None,
chunk_count: int | None = None,
token_count: int | None = None,
extra: dict[str, Any] | None = None,
) -> None:
self.call_type: str = call_type
self.model: str | None = model
self.doc_count: int | None = doc_count
self.chunk_count: int | None = chunk_count
self.token_count: int | None = token_count
self.extra: dict[str, Any] = extra or {}
self._start: float | None = None
# ── public API ──────────────────────────────────────
def set_extra(self, key: str, value: Any) -> None:
"""Append a metric inside the context block.
Useful for data that is only known after the operation completes,
e.g. a reranker's score distribution or a pipeline's finish_reason.
"""
self.extra[key] = value
@property
def elapsed_ms(self) -> float:
"""Elapsed time in milliseconds (readable inside the context block)."""
if self._start is None:
return 0.0
return (time.perf_counter() - self._start) * 1000
# ── context-manager protocol ────────────────────────
def __enter__(self) -> "PerfTimer":
self._start = time.perf_counter()
return self # allows "with PerfTimer(...) as t: t.set_extra(...)"
def __exit__(self, exc_type, exc_val, exc_tb) -> bool:
elapsed = self.elapsed_ms
success = exc_type is None
log_perf(
call_type=self.call_type,
latency_ms=elapsed,
model=self.model,
doc_count=self.doc_count,
chunk_count=self.chunk_count,
token_count=self.token_count,
success=success,
error=str(exc_val) if exc_val else None,
extra=self.extra if self.extra else None,
)
return False # don't swallow exceptions — let them propagate