forked from ccf-ai-infra/TileOPs-Metax
b77f40c762
## Summary Upgrade the benchmark workload interface from hard-coded `WorkloadBase` composition to capability-based protocols. Closes #918 ### Changes - **Capability protocols**: `ShapeDtypeWorkload`, `InputGeneratingWorkload`, `BenchmarkWorkload` — structural types for workload capabilities - **Generic `BenchmarkBase[W]`**: base class parameterized by workload type, replacing the hard-coded `WorkloadBase` constructor - **`ManifestBenchmark(BenchmarkBase[ShapeDtypeWorkload])`**: aligned contracts, zero `type: ignore` in `benchmarks/benchmark.py` - **`WorkloadBase` preserved**: default in-repo implementation, not the public contract - **Docs updated**: `docs/testing.md`, `docs/trust-model.md`, `docs/architecture.md` - **7 tests**: duck-typed protocols, ManifestBenchmark contract, WorkloadBase compatibility ## Test plan - [x] **AC-1**: `roofline_vars` parameter typed to require `shape` and `dtype` via `ShapeDtypeWorkload` protocol - [x] **AC-2**: 1056 benchmarks collect, 103 benchmark ops pass (1 pre-existing failure in bench_argreduce, unmodified on this branch), 7 protocol tests pass - [x] **AC-3**: No new type errors — zero `type: ignore[arg-type]` in `benchmarks/benchmark.py` ## Verification evidence All acceptance criteria verified by automated reviewer (two-pass code review complete). Design direction confirmed by Ibuki-wind (Option B: full benchmark interface refactor). ## Follow-up - #936 — Add benchmarks/tests/ to pytest testpaths for CI coverage - #937 — Migrate 97 BenchmarkBase subclasses to explicit generic parameterization (depends on #936) - #938 — Archive benchmark-workload-interface.md plan document Suggestions: Update RooflineWorkload backward-compat alias removal once confirmed no external consumers depend on it --------- Co-authored-by: Ibuki 🍃 — a wind born from Claude Opus <Ibuki-wind@users.noreply.github.com> |
||
|---|---|---|
| .. | ||
| __init__.py | ||
| test_roofline_workload_protocol.py | ||