forked from ccf-ai-infra/TileOPs-Metax
ef5d248f2f
## Summary Extract a generic `ManifestBenchmark` base class and shared helpers (`roofline_vars`, `workloads_to_params`) into `benchmarks/benchmark.py`, so manifest-driven bench files only specify the op name and baseline function per test. This eliminates ~300 lines of duplication across 5 benchmark files. Closes #904 ## Changes - **benchmarks/benchmark.py** — Add `ManifestBenchmark(op_name)` subclass of `BenchmarkBase` with `roofline_vars(workload)` and `workloads_to_params(op_name)` public helpers - **benchmarks/ops/bench_{softmax,reduce,argreduce,logical_reduce,vector_norm}.py** — Refactor to use `ManifestBenchmark`, replacing 19 per-op benchmark classes - **scripts/validate_manifest.py** — Teach AST validator to recognize `ManifestBenchmark` subclasses as satisfying `eval_roofline` requirement - **tests/test_validate_manifest.py** — Add tests for indirect bench-validator AST paths and a regression test documenting the subclass override limitation ## Test plan - [x] AC-1: All 5 bench files use `ManifestBenchmark` instead of per-op classes - [x] AC-2: `_roofline_vars` and `_workloads_to_params` exist only in `benchmarks/benchmark.py`, not duplicated - [x] AC-3: `pytest --collect-only` passes for all 5 bench files (63 tests collected) - [x] AC-4: `validate_manifest.py --check-op` produces zero `[bench]` warnings for all 19 affected ops - [x] 89 validator tests pass, lint clean ## Follow-up - #917 — Add import alias and attribute access support to bench AST validator - #918 — Type-narrow roofline_vars parameter from WorkloadBase to typed Protocol --------- Co-authored-by: Ibuki 🍃 — a wind born from Claude Opus <Ibuki-wind@users.noreply.github.com> |
||
|---|---|---|
| .. | ||
| conftest_warmup.py | ||
| dep_hash.py | ||
| gpu_smoke_report.py | ||
| nightly_report.py | ||
| test_node_delta.py | ||
| validate_manifest.py | ||
| warmup_kernel_cache.py | ||