forked from ccf-ai-infra/TileOPs-Metax
b3003ff157
## Problem Several legacy binary benchmarks pass a short string such as `maximum`, `minimum`, or `cmp_eq` to `BenchmarkReport.record()` instead of passing the Op instance that was actually profiled. String records contain neither the canonical class identity nor `op_module`. The nightly report therefore treats a legacy alias and its manifest-driven benchmark as different operators. For example, the same `MaximumFwdOp` implementation can appear once as `MaximumFwdOp` and again as `maximum`. Downstream documentation then has to classify the alias from its name alone, which can also place `maximum` under Reduction because it contains `max`. This affects the standard binary arithmetic, comparison, logical, and bitwise benchmark groups. The measurements themselves are valid; the recorded operator identity is not canonical. ## Solution Pass the real Op instance to `BenchmarkReport.record()` for both the TileOPs measurement and its baseline: ```python op = op_cls(...) result = bm.profile(op, *inputs) BenchmarkReport.record(op, locals(), result, tag='tileops') result_bl = bm.profile(baseline_fn, *inputs) BenchmarkReport.record(op, locals(), result_bl, tag='torch') ``` `BenchmarkReport` then derives the identity consistently from: ```python op.__class__.__name__ op.__class__.__module__ ``` This consolidates legacy aliases under canonical identities such as `MaximumFwdOp`, `MinimumFwdOp`, and `EqFwdOp`, while preserving the benchmark parameters, measurements, and baseline tags. The change is intentionally limited to identity recording. It does not add display-category metadata or move taxonomy into TileOPs; benchmark page classification remains owned by TileOPs.github.io. ## Changes - record standard binary arithmetic benchmarks with their real Op instances - record comparison benchmarks with their real Op instances instead of `cmp_*` aliases - record logical and bitwise benchmarks with their real Op instances - preserve canonical class names and `op_module` metadata in nightly JUnit output ## Validation - `python -m py_compile benchmarks/ops/bench_binary_elementwise.py` - `git diff --check` GPU benchmark execution was not available in the local worktree. Related documentation taxonomy fix: tile-ai/TileOPs.github.io#13 |
||
|---|---|---|
| .. | ||
| hardware | ||
| kernels | ||
| ops | ||
| tests | ||
| __init__.py | ||
| benchmark_base.py | ||
| conftest.py | ||