feat(research): 新增 repro-audit 科研复现性审计（Skill + 工具 + 真实验证）
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
This commit is contained in:
farmyobutu5233
parent
9749a4c832
commit
45970f6013
|
|
@ -0,0 +1,56 @@
|
|||
# 科研项目复现性审计（repro-audit）
|
||||
|
||||
把 [`gitlink-repro-audit` Skill](../../../skills/gitlink-repro-audit/SKILL.md) 包成**可直接运行的科研辅助工具**（子赛题四：应用 GitLink 辅助科研）：
|
||||
|
||||
> **采集 → 评估 → 报告 → 回写**：用 `repo +tree` / `file +view --raw` / `release +list` 采集仓库结构、README 与发布信息，做八维**确定性复现性检查**（文档 / 许可证 / 引用 / 依赖固化 / 运行入口 / 数据说明 / 测试 / 版本固化），输出 0-100 评分卡 + A/B/C/D 等级 + 逐项修复建议，并（仅在 `--apply` 时）用 `issue +create` 把报告作为改进 tracking issue 回写。
|
||||
|
||||
## 科研赋能价值
|
||||
|
||||
- **课题组自查**：论文投稿/开源发布前对代码仓库做复现性体检，逐项补齐
|
||||
- **学术规范检查**：许可证、引用信息（CITATION.cff/BibTeX）、数据可得性一次性核验
|
||||
- **可进 CI**：退出码 `2` 表示存在明显复现缺口，可作为科研仓库的发布门禁
|
||||
|
||||
## 交付物
|
||||
|
||||
- `scripts/repro_audit.py`：完整审计闭环（纯标准库，Python ≥3.9，零第三方依赖）
|
||||
- `tests/test_audit.py`：确定性回归护栏（7/7 全绿，同输入同分同等级）
|
||||
- `docs/verification.md`：真实科研仓库验证证据
|
||||
- `examples/demo-outputs/`：两个真实仓库的审计报告
|
||||
|
||||
## 快速运行（默认 dry-run，不写远端）
|
||||
|
||||
```bash
|
||||
npm install -g @gitlink-ai/cli
|
||||
gitlink-cli auth login
|
||||
|
||||
python3 scripts/repro_audit.py --owner <owner> --repo <repo> --output-dir outputs
|
||||
# 回写改进 tracking issue（请先确认报告内容）：加 --apply
|
||||
```
|
||||
|
||||
## 已在真实科研仓库验证
|
||||
|
||||
见 [`docs/verification.md`](docs/verification.md)：
|
||||
|
||||
| 仓库 | 类型 | 得分 | 等级 |
|
||||
|------|------|------|------|
|
||||
| GengzhaoWang/UAV-Paper | 论文笔记仓库 | 8/100 | D（复现困难，符合预期） |
|
||||
| momomym/paper-summarizer | 科研工具代码仓库 | 58/100 | C（缺引用/数据说明/release，建议明确） |
|
||||
|
||||
两个仓库得分区分度显著，逐项证据均可在仓库页面人工复核。
|
||||
|
||||
## 评分标准
|
||||
|
||||
| 检查项 | 分值 | 判据 |
|
||||
|--------|------|------|
|
||||
| README 与运行说明 | 15 | README 存在且含运行/复现章节（仅 README 得 8） |
|
||||
| 开源许可证 | 15 | LICENSE / COPYING |
|
||||
| 引用信息 | 10 | CITATION.cff 或 README BibTeX |
|
||||
| 依赖清单 | 15 | requirements.txt / environment.yml / go.mod 等 |
|
||||
| 运行入口 | 10 | Makefile / run.sh / main.py / Dockerfile / scripts |
|
||||
| 数据可得性说明 | 10 | data 目录或 README 数据说明 |
|
||||
| 测试/验证代码 | 10 | tests 目录 |
|
||||
| 版本发布 | 15 | ≥1 个 release |
|
||||
|
||||
等级：≥85 A / ≥70 B / ≥50 C / <50 D。
|
||||
|
||||
> 依赖 `file` 快捷命令组（PR #330）；在其合并前可用 `--cli` 指向包含该命令的本地构建。
|
||||
|
|
@ -0,0 +1,35 @@
|
|||
# 真实平台验证记录
|
||||
|
||||
验证日期：2026-07-05；平台：生产环境 gitlink.org.cn；CLI：包含 `file` 命令组（PR #330）的本地构建。
|
||||
|
||||
## 1. 论文笔记仓库（预期低分）
|
||||
|
||||
```bash
|
||||
python3 scripts/repro_audit.py --owner GengzhaoWang --repo UAV-Paper
|
||||
```
|
||||
|
||||
- 结果：**8/100 — D（复现困难）**，退出码 `2`
|
||||
- 证据：仅有 README（无运行说明章节），无 LICENSE / 引用 / 依赖清单 / 入口 / 数据说明 / 测试 / release
|
||||
- 完整报告：[`../examples/demo-outputs/repro-audit-GengzhaoWang-UAV-Paper.md`](../examples/demo-outputs/repro-audit-GengzhaoWang-UAV-Paper.md)
|
||||
|
||||
## 2. 科研工具代码仓库（中等分，建议明确）
|
||||
|
||||
```bash
|
||||
python3 scripts/repro_audit.py --owner momomym --repo paper-summarizer
|
||||
```
|
||||
|
||||
- 结果：**58/100 — C（存在明显缺口）**，退出码 `2`
|
||||
- 检出：LICENSE ✅、requirements.txt ✅、scripts 入口 ✅、tests ✅
|
||||
- 缺口：README 无运行章节（部分分 8/15）、无引用信息、无数据说明、无 release
|
||||
- 完整报告：[`../examples/demo-outputs/repro-audit-momomym-paper-summarizer.md`](../examples/demo-outputs/repro-audit-momomym-paper-summarizer.md)
|
||||
|
||||
两个真实仓库得分区分度显著（8 vs 58），每项证据均可在仓库网页人工复核。
|
||||
|
||||
## 3. 确定性回归护栏
|
||||
|
||||
```bash
|
||||
python3 tests/test_audit.py
|
||||
# Ran 7 tests ... OK
|
||||
```
|
||||
|
||||
覆盖：满分样例（100/A）、空仓库（0/D）、README 无运行说明的部分分、README 内引用识别、scripts 目录作为入口、同输入同输出确定性、失败项必附修复建议。
|
||||
|
|
@ -0,0 +1,17 @@
|
|||
# 科研复现性审计报告：GengzhaoWang/UAV-Paper（ref: 默认分支）
|
||||
|
||||
**总分：8/100 — 等级 D（复现困难）**
|
||||
|
||||
| 检查项 | 得分 | 证据 | 建议 |
|
||||
|--------|------|------|------|
|
||||
| README 与运行说明 | 8/15 | README=README.md；运行/复现章节=未检出 | 补充 README 并加入 How to run / 复现步骤章节 |
|
||||
| 开源许可证 | 0/15 | 许可证文件=缺失 | 添加 LICENSE 文件（科研代码推荐 MIT/Apache-2.0/BSD） |
|
||||
| 引用信息 | 0/10 | CITATION 文件=无；README 引用章节=无 | 添加 CITATION.cff 或在 README 中给出 BibTeX |
|
||||
| 依赖清单（环境固化） | 0/15 | 依赖文件=缺失 | 添加 requirements.txt / environment.yml 等依赖清单并固定版本 |
|
||||
| 运行入口 | 0/10 | 入口=未检出 | 提供 Makefile / run.sh / main.py 等一键运行入口 |
|
||||
| 数据可得性说明 | 0/10 | 数据目录=无；README 数据说明=无 | 在 README 中说明数据集来源、获取方式与预处理步骤 |
|
||||
| 测试/验证代码 | 0/10 | 测试目录=无 | 添加 tests/ 验证关键结果可复算 |
|
||||
| 版本发布（成果固化） | 0/15 | release 数=0 | 为论文对应的代码状态打 tag 并创建 release |
|
||||
|
||||
---
|
||||
*由 repro-audit 生成（确定性检查，同输入同输出）。*
|
||||
|
|
@ -0,0 +1,17 @@
|
|||
# 科研复现性审计报告：momomym/paper-summarizer（ref: 默认分支）
|
||||
|
||||
**总分：58/100 — 等级 C（存在明显缺口）**
|
||||
|
||||
| 检查项 | 得分 | 证据 | 建议 |
|
||||
|--------|------|------|------|
|
||||
| README 与运行说明 | 8/15 | README=README.md；运行/复现章节=未检出 | 补充 README 并加入 How to run / 复现步骤章节 |
|
||||
| 开源许可证 | 15/15 | 许可证文件=LICENSE | — |
|
||||
| 引用信息 | 0/10 | CITATION 文件=无；README 引用章节=无 | 添加 CITATION.cff 或在 README 中给出 BibTeX |
|
||||
| 依赖清单（环境固化） | 15/15 | 依赖文件=requirements.txt | — |
|
||||
| 运行入口 | 10/10 | 入口=scripts | — |
|
||||
| 数据可得性说明 | 0/10 | 数据目录=无；README 数据说明=无 | 在 README 中说明数据集来源、获取方式与预处理步骤 |
|
||||
| 测试/验证代码 | 10/10 | 测试目录=tests | — |
|
||||
| 版本发布（成果固化） | 0/15 | release 数=0 | 为论文对应的代码状态打 tag 并创建 release |
|
||||
|
||||
---
|
||||
*由 repro-audit 生成（确定性检查，同输入同输出）。*
|
||||
Binary file not shown.
|
|
@ -0,0 +1,205 @@
|
|||
#!/usr/bin/env python3
|
||||
"""科研项目复现性审计（repro-audit）。
|
||||
|
||||
面向科研代码仓库的确定性复现性评估：
|
||||
1. 采集：`repo +tree` 获取仓库结构，`file +view --raw` 拉取 README，`release +list` 查发布
|
||||
2. 评估：八个维度的复现性检查项（文档/许可证/引用/依赖/入口/数据说明/测试/版本固化）
|
||||
3. 报告：0-100 评分卡 + A/B/C/D 等级 + 逐项修复建议
|
||||
4. 回写：--apply 时用 `issue +create` 把报告作为改进 tracking issue 提交
|
||||
|
||||
纯标准库实现（Python >= 3.9），gitlink-cli 为唯一外部依赖。默认 dry-run。
|
||||
"""
|
||||
|
||||
import argparse
|
||||
import json
|
||||
import re
|
||||
import subprocess
|
||||
import sys
|
||||
from dataclasses import dataclass
|
||||
from pathlib import Path
|
||||
|
||||
|
||||
@dataclass
|
||||
class CheckResult:
|
||||
name: str
|
||||
weight: int
|
||||
score: int # 0..weight
|
||||
evidence: str
|
||||
advice: str
|
||||
|
||||
|
||||
def run_cli(args, cli="gitlink-cli"):
|
||||
result = subprocess.run([cli, *args], capture_output=True, text=True, check=False)
|
||||
if result.returncode != 0:
|
||||
raise RuntimeError(
|
||||
f"gitlink-cli {' '.join(args)} 失败: {result.stderr.strip() or result.stdout.strip()}"
|
||||
)
|
||||
return result.stdout
|
||||
|
||||
|
||||
def get_json(args, cli="gitlink-cli"):
|
||||
payload = json.loads(run_cli([*args, "--format", "json"], cli=cli))
|
||||
return payload.get("data", payload)
|
||||
|
||||
|
||||
def list_entries(owner, repo, ref, path="", cli="gitlink-cli"):
|
||||
args = ["repo", "+tree", "--owner", owner, "--repo", repo]
|
||||
if ref:
|
||||
args += ["--ref", ref]
|
||||
if path:
|
||||
args += ["--path", path]
|
||||
data = get_json(args, cli=cli)
|
||||
entries = data.get("entries", data) if isinstance(data, dict) else data
|
||||
out = []
|
||||
if isinstance(entries, list):
|
||||
for e in entries:
|
||||
if isinstance(e, dict) and e.get("name"):
|
||||
out.append((e["name"], e.get("type", "")))
|
||||
return out
|
||||
|
||||
|
||||
def fetch_readme(owner, repo, ref, names, cli="gitlink-cli"):
|
||||
for name in names:
|
||||
if re.match(r"(?i)^readme(\.|$)", name):
|
||||
args = ["file", "+view", "--owner", owner, "--repo", repo, "--path", name, "--raw"]
|
||||
if ref:
|
||||
args += ["--ref", ref]
|
||||
try:
|
||||
return name, run_cli(args, cli=cli)
|
||||
except RuntimeError:
|
||||
continue
|
||||
return None, ""
|
||||
|
||||
|
||||
def count_releases(owner, repo, cli="gitlink-cli"):
|
||||
try:
|
||||
data = get_json(["release", "+list", "--owner", owner, "--repo", repo], cli=cli)
|
||||
except RuntimeError:
|
||||
return 0
|
||||
releases = data.get("releases", data) if isinstance(data, dict) else data
|
||||
return len(releases) if isinstance(releases, list) else 0
|
||||
|
||||
|
||||
DEP_FILES = [
|
||||
"requirements.txt", "environment.yml", "environment.yaml", "Pipfile",
|
||||
"pyproject.toml", "setup.py", "go.mod", "package.json", "Cargo.toml",
|
||||
"pom.xml", "build.gradle", "CMakeLists.txt", "DESCRIPTION", "renv.lock",
|
||||
]
|
||||
ENTRY_FILES = ["Makefile", "makefile", "run.sh", "train.sh", "main.py", "run.py", "train.py", "Dockerfile", "docker-compose.yml"]
|
||||
CITATION_FILES = ["CITATION.cff", "CITATION", "CITATION.bib", "citation.bib"]
|
||||
LICENSE_RE = re.compile(r"(?i)^(license|licence|copying)(\.|$)")
|
||||
DATA_HINT_RE = re.compile(r"(?i)\b(dataset|data set|数据集|数据来源|download.*data|data/)\b")
|
||||
REPRO_HINT_RE = re.compile(r"(?i)(reproduc|复现|实验设置|experiment setup|how to run|usage|快速开始|quick\s?start)")
|
||||
|
||||
|
||||
def audit(names, dirs, readme_text, release_count):
|
||||
"""确定性打分：names=顶层文件名，dirs=顶层目录名。返回 CheckResult 列表。"""
|
||||
nameset = set(names)
|
||||
results = []
|
||||
|
||||
def add(name, weight, ok, evidence, advice, partial=None):
|
||||
score = weight if ok else (partial if partial else 0)
|
||||
results.append(CheckResult(name, weight, score, evidence, "" if ok else advice))
|
||||
|
||||
readme_name = next((n for n in names if re.match(r"(?i)^readme(\.|$)", n)), None)
|
||||
has_usage = bool(readme_text and REPRO_HINT_RE.search(readme_text))
|
||||
add("README 与运行说明", 15, bool(readme_name and has_usage),
|
||||
f"README={readme_name or '缺失'}；运行/复现章节={'有' if has_usage else '未检出'}",
|
||||
"补充 README 并加入 How to run / 复现步骤章节",
|
||||
partial=8 if readme_name else 0)
|
||||
|
||||
lic = next((n for n in names if LICENSE_RE.match(n)), None)
|
||||
add("开源许可证", 15, bool(lic), f"许可证文件={lic or '缺失'}",
|
||||
"添加 LICENSE 文件（科研代码推荐 MIT/Apache-2.0/BSD）")
|
||||
|
||||
cit = next((n for n in names if n in CITATION_FILES), None)
|
||||
cite_in_readme = bool(readme_text and re.search(r"(?i)(citation|引用|bibtex|@(article|inproceedings))", readme_text))
|
||||
add("引用信息", 10, bool(cit or cite_in_readme),
|
||||
f"CITATION 文件={cit or '无'}；README 引用章节={'有' if cite_in_readme else '无'}",
|
||||
"添加 CITATION.cff 或在 README 中给出 BibTeX")
|
||||
|
||||
dep = next((n for n in names if n in DEP_FILES), None)
|
||||
add("依赖清单（环境固化）", 15, bool(dep), f"依赖文件={dep or '缺失'}",
|
||||
"添加 requirements.txt / environment.yml 等依赖清单并固定版本")
|
||||
|
||||
entry = next((n for n in names if n in ENTRY_FILES), None)
|
||||
script_dir = next((d for d in dirs if d in ("scripts", "bin")), None)
|
||||
add("运行入口", 10, bool(entry or script_dir),
|
||||
f"入口={entry or script_dir or '未检出'}",
|
||||
"提供 Makefile / run.sh / main.py 等一键运行入口")
|
||||
|
||||
data_dir = next((d for d in dirs if re.match(r"(?i)^(data|datasets?)$", d)), None)
|
||||
data_in_readme = bool(readme_text and DATA_HINT_RE.search(readme_text))
|
||||
add("数据可得性说明", 10, bool(data_dir or data_in_readme),
|
||||
f"数据目录={data_dir or '无'}；README 数据说明={'有' if data_in_readme else '无'}",
|
||||
"在 README 中说明数据集来源、获取方式与预处理步骤")
|
||||
|
||||
test_dir = next((d for d in dirs if re.match(r"(?i)^tests?$", d)), None)
|
||||
add("测试/验证代码", 10, bool(test_dir), f"测试目录={test_dir or '无'}",
|
||||
"添加 tests/ 验证关键结果可复算")
|
||||
|
||||
add("版本发布（成果固化）", 15, release_count > 0, f"release 数={release_count}",
|
||||
"为论文对应的代码状态打 tag 并创建 release")
|
||||
|
||||
return results
|
||||
|
||||
|
||||
GRADE = [(85, "A（可复现性良好）"), (70, "B（基本可复现）"), (50, "C（存在明显缺口）"), (0, "D（复现困难）")]
|
||||
|
||||
|
||||
def render_report(owner, repo, ref, results):
|
||||
total = sum(r.score for r in results)
|
||||
grade = next(g for t, g in GRADE if total >= t)
|
||||
lines = [
|
||||
f"# 科研复现性审计报告：{owner}/{repo}（ref: {ref or '默认分支'}）",
|
||||
"",
|
||||
f"**总分：{total}/100 — 等级 {grade}**",
|
||||
"",
|
||||
"| 检查项 | 得分 | 证据 | 建议 |",
|
||||
"|--------|------|------|------|",
|
||||
]
|
||||
for r in results:
|
||||
lines.append(f"| {r.name} | {r.score}/{r.weight} | {r.evidence} | {r.advice or '—'} |")
|
||||
lines += ["", "---", "*由 repro-audit 生成（确定性检查，同输入同输出）。*"]
|
||||
return "\n".join(lines), total
|
||||
|
||||
|
||||
def main():
|
||||
parser = argparse.ArgumentParser(description="科研项目复现性审计")
|
||||
parser.add_argument("--owner", required=True)
|
||||
parser.add_argument("--repo", required=True)
|
||||
parser.add_argument("--ref", default="")
|
||||
parser.add_argument("--apply", action="store_true", help="把报告作为 tracking issue 回写（默认 dry-run）")
|
||||
parser.add_argument("--output-dir", default="outputs")
|
||||
parser.add_argument("--cli", default="gitlink-cli")
|
||||
args = parser.parse_args()
|
||||
|
||||
entries = list_entries(args.owner, args.repo, args.ref, cli=args.cli)
|
||||
names = [n for n, t in entries if t != "dir"]
|
||||
dirs = [n for n, t in entries if t == "dir"]
|
||||
_, readme_text = fetch_readme(args.owner, args.repo, args.ref, names, cli=args.cli)
|
||||
releases = count_releases(args.owner, args.repo, cli=args.cli)
|
||||
|
||||
results = audit(names, dirs, readme_text, releases)
|
||||
report, total = render_report(args.owner, args.repo, args.ref, results)
|
||||
|
||||
out_dir = Path(args.output_dir)
|
||||
out_dir.mkdir(parents=True, exist_ok=True)
|
||||
path = out_dir / f"repro-audit-{args.owner}-{args.repo}.md"
|
||||
path.write_text(report, encoding="utf-8")
|
||||
print(report)
|
||||
print(f"\n报告已保存：{path}", file=sys.stderr)
|
||||
|
||||
if args.apply:
|
||||
run_cli([
|
||||
"issue", "+create", "--owner", args.owner, "--repo", args.repo,
|
||||
"--title", f"[repro-audit] 复现性审计报告（{total}/100）",
|
||||
"--body", report, "--format", "json",
|
||||
], cli=args.cli)
|
||||
print("已创建 tracking issue。", file=sys.stderr)
|
||||
|
||||
return 0 if total >= 70 else 2
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
sys.exit(main())
|
||||
|
|
@ -0,0 +1,73 @@
|
|||
"""确定性回归护栏：同输入 → 同分 → 同等级。"""
|
||||
|
||||
import sys
|
||||
import unittest
|
||||
from pathlib import Path
|
||||
|
||||
sys.path.insert(0, str(Path(__file__).resolve().parent.parent / "scripts"))
|
||||
|
||||
from repro_audit import audit, render_report # noqa: E402
|
||||
|
||||
GOOD_README = """# Project
|
||||
|
||||
## How to run
|
||||
|
||||
```bash
|
||||
make train
|
||||
```
|
||||
|
||||
## Dataset
|
||||
|
||||
Download the dataset from ...
|
||||
|
||||
## Citation
|
||||
|
||||
```bibtex
|
||||
@article{x2026}
|
||||
```
|
||||
"""
|
||||
|
||||
|
||||
class AuditTest(unittest.TestCase):
|
||||
def test_full_marks(self):
|
||||
names = ["README.md", "LICENSE", "CITATION.cff", "requirements.txt", "Makefile"]
|
||||
dirs = ["data", "tests", "src"]
|
||||
results = audit(names, dirs, GOOD_README, release_count=2)
|
||||
report, total = render_report("o", "r", "", results)
|
||||
self.assertEqual(total, 100)
|
||||
self.assertIn("等级 A", report)
|
||||
|
||||
def test_empty_repo(self):
|
||||
results = audit([], [], "", release_count=0)
|
||||
report, total = render_report("o", "r", "", results)
|
||||
self.assertEqual(total, 0)
|
||||
self.assertIn("等级 D", report)
|
||||
|
||||
def test_readme_without_usage_partial(self):
|
||||
results = audit(["README.md"], [], "# hi", release_count=0)
|
||||
readme = next(r for r in results if r.name == "README 与运行说明")
|
||||
self.assertEqual(readme.score, 8) # 有 README 无运行说明 → 部分分
|
||||
|
||||
def test_citation_in_readme_counts(self):
|
||||
results = audit(["README.md"], [], "## Citation\n@article{x}", release_count=0)
|
||||
cit = next(r for r in results if r.name == "引用信息")
|
||||
self.assertEqual(cit.score, cit.weight)
|
||||
|
||||
def test_scripts_dir_as_entry(self):
|
||||
results = audit([], ["scripts"], "", release_count=0)
|
||||
entry = next(r for r in results if r.name == "运行入口")
|
||||
self.assertEqual(entry.score, entry.weight)
|
||||
|
||||
def test_deterministic(self):
|
||||
a = audit(["README.md", "go.mod"], ["tests"], GOOD_README, 1)
|
||||
b = audit(["README.md", "go.mod"], ["tests"], GOOD_README, 1)
|
||||
self.assertEqual([vars(x) for x in a], [vars(y) for y in b])
|
||||
|
||||
def test_advice_present_for_failures(self):
|
||||
results = audit([], [], "", 0)
|
||||
for r in results:
|
||||
self.assertTrue(r.advice, f"{r.name} 应给出修复建议")
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
unittest.main()
|
||||
|
|
@ -0,0 +1,69 @@
|
|||
---
|
||||
name: gitlink-repro-audit
|
||||
version: 1.0.0
|
||||
description: "科研项目复现性审计：对 GitLink 上的科研代码仓库做八维复现性检查（文档/许可证/引用/依赖/入口/数据说明/测试/版本固化），生成 0-100 评分卡与修复建议，并可回写改进 tracking issue。当用户需要评估科研仓库可复现性、准备论文代码发布、或做学术规范检查时触发。"
|
||||
metadata:
|
||||
requires:
|
||||
bins: ["gitlink-cli"]
|
||||
cliHelp: "gitlink-cli repo --help"
|
||||
---
|
||||
|
||||
# gitlink-repro-audit（科研复现性审计）
|
||||
|
||||
**CRITICAL — 开始前必须先阅读 [`../gitlink-shared/SKILL.md`](../gitlink-shared/SKILL.md)，其中包含认证、权限处理和 API 注意事项。**
|
||||
**CRITICAL — 审计本身只读；`--apply` 回写 tracking issue 前必须先向用户展示报告并确认。**
|
||||
**CRITICAL — GitLink 操作只能用 `gitlink-cli`。**
|
||||
|
||||
> **前置条件：** 先阅读 [`../gitlink-shared/SKILL.md`](../gitlink-shared/SKILL.md)
|
||||
|
||||
---
|
||||
|
||||
## 功能概述
|
||||
|
||||
论文代码「跑不起来」是科研复现危机的核心。本技能对科研代码仓库做系统化复现性评估：
|
||||
|
||||
| 检查项 | 分值 | 判据 |
|
||||
|--------|------|------|
|
||||
| README 与运行说明 | 15 | 存在 README 且含 How to run / 复现步骤章节（仅有 README 得 8 分） |
|
||||
| 开源许可证 | 15 | LICENSE / COPYING 文件 |
|
||||
| 引用信息 | 10 | CITATION.cff 或 README 中的 BibTeX/引用章节 |
|
||||
| 依赖清单（环境固化） | 15 | requirements.txt / environment.yml / go.mod / package.json 等 |
|
||||
| 运行入口 | 10 | Makefile / run.sh / main.py / Dockerfile / scripts 目录 |
|
||||
| 数据可得性说明 | 10 | data 目录或 README 中的数据集来源说明 |
|
||||
| 测试/验证代码 | 10 | tests 目录 |
|
||||
| 版本发布（成果固化） | 15 | 至少一个 release |
|
||||
|
||||
总分映射等级：≥85 A（可复现性良好）/ ≥70 B / ≥50 C / <50 D（复现困难）。
|
||||
|
||||
## 使用方式
|
||||
|
||||
### 方式一：确定性脚本（推荐，可进 CI）
|
||||
|
||||
```bash
|
||||
python3 examples/research/repro-audit/scripts/repro_audit.py \
|
||||
--owner <owner> --repo <repo> [--ref <branch>] [--apply]
|
||||
```
|
||||
|
||||
- 默认 dry-run 只输出报告；`--apply` 用 `issue +create` 回写改进 tracking issue
|
||||
- 退出码：`0` 总分 ≥70（基本可复现），`2` 存在明显缺口
|
||||
|
||||
### 方式二：AI Agent 手工执行命令链
|
||||
|
||||
```bash
|
||||
# 1. 结构采集
|
||||
gitlink-cli repo +tree --owner <owner> --repo <repo> --format json
|
||||
# 2. README 内容
|
||||
gitlink-cli file +view --owner <owner> --repo <repo> --path README.md --raw
|
||||
# 3. 版本发布
|
||||
gitlink-cli release +list --owner <owner> --repo <repo> --format json
|
||||
# 4.（确认后）回写改进 issue
|
||||
gitlink-cli issue +create --owner <owner> --repo <repo> -t "[repro-audit] 复现性审计报告" -b "<报告>"
|
||||
```
|
||||
|
||||
AI 按上表逐项打分，报告须给出每项证据与修复建议，不允许笼统结论。
|
||||
|
||||
## 注意事项
|
||||
|
||||
- 判据基于仓库顶层结构与 README 文本，属于必要条件检查：高分不代表结果一定可复现，低分则一定存在工程缺口。
|
||||
- 对论文笔记类仓库（纯 Markdown）等级普遍偏低，属预期行为；报告建议仅在其确为「实验代码仓库」时回写。
|
||||
- 与 [`../gitlink-doc-sync/SKILL.md`](../gitlink-doc-sync/SKILL.md)（文档一致性）、[`../gitlink-license-compliance/SKILL.md`](../gitlink-license-compliance/SKILL.md)（许可证合规）互补：本技能聚焦复现性维度的整体评估。
|
||||
Loading…
Reference in New Issue