TileOPs-Metax/benchmarks/ops
Ang Gao b3003ff157
[Fix][Bench] Record binary ops by canonical identity (#1788)
## Problem

Several legacy binary benchmarks pass a short string such as `maximum`,
`minimum`, or `cmp_eq` to `BenchmarkReport.record()` instead of passing
the Op instance that was actually profiled.

String records contain neither the canonical class identity nor
`op_module`. The nightly report therefore treats a legacy alias and its
manifest-driven benchmark as different operators. For example, the same
`MaximumFwdOp` implementation can appear once as `MaximumFwdOp` and
again as `maximum`. Downstream documentation then has to classify the
alias from its name alone, which can also place `maximum` under
Reduction because it contains `max`.

This affects the standard binary arithmetic, comparison, logical, and
bitwise benchmark groups. The measurements themselves are valid; the
recorded operator identity is not canonical.

## Solution

Pass the real Op instance to `BenchmarkReport.record()` for both the
TileOPs measurement and its baseline:

```python
op = op_cls(...)
result = bm.profile(op, *inputs)
BenchmarkReport.record(op, locals(), result, tag='tileops')

result_bl = bm.profile(baseline_fn, *inputs)
BenchmarkReport.record(op, locals(), result_bl, tag='torch')
```

`BenchmarkReport` then derives the identity consistently from:

```python
op.__class__.__name__
op.__class__.__module__
```

This consolidates legacy aliases under canonical identities such as
`MaximumFwdOp`, `MinimumFwdOp`, and `EqFwdOp`, while preserving the
benchmark parameters, measurements, and baseline tags.

The change is intentionally limited to identity recording. It does not
add display-category metadata or move taxonomy into TileOPs; benchmark
page classification remains owned by TileOPs.github.io.

## Changes

- record standard binary arithmetic benchmarks with their real Op
instances
- record comparison benchmarks with their real Op instances instead of
`cmp_*` aliases
- record logical and bitwise benchmarks with their real Op instances
- preserve canonical class names and `op_module` metadata in nightly
JUnit output

## Validation

- `python -m py_compile benchmarks/ops/bench_binary_elementwise.py`
- `git diff --check`

GPU benchmark execution was not available in the local worktree.

Related documentation taxonomy fix: tile-ai/TileOPs.github.io#13
2026-07-27 11:28:08 +08:00
..
attention [Refactor][POOL] strip over-design and scaffolding from the merged cleanup PRs (#1779) 2026-07-27 10:05:38 +08:00
__init__.py [Refactor] Separate benchmark profiling from unit tests (#224) 2026-02-27 16:46:35 +08:00
bench_ada_layer_norm.py [Chore][Cleanup] repo slimming: dead code, duplicated tests/benches, compat shims, file defragmentation (#1764) 2026-07-26 12:59:48 +08:00
bench_argreduce.py [Chore][Cleanup] repo slimming: dead code, duplicated tests/benches, compat shims, file defragmentation (#1764) 2026-07-26 12:59:48 +08:00
bench_batch_norm.py [Chore][Cleanup] repo slimming: dead code, duplicated tests/benches, compat shims, file defragmentation (#1764) 2026-07-26 12:59:48 +08:00
bench_binary_elementwise.py [Fix][Bench] Record binary ops by canonical identity (#1788) 2026-07-27 11:28:08 +08:00
bench_bmm.py [Chore][Cleanup] repo slimming: dead code, duplicated tests/benches, compat shims, file defragmentation (#1764) 2026-07-26 12:59:48 +08:00
bench_convolution.py [Refactor][POOL] strip over-design and scaffolding from the merged cleanup PRs (#1779) 2026-07-27 10:05:38 +08:00
bench_cumulative.py [Chore][Cleanup] repo slimming: dead code, duplicated tests/benches, compat shims, file defragmentation (#1764) 2026-07-26 12:59:48 +08:00
bench_deltanet.py [Refactor][POOL] strip over-design and scaffolding from the merged cleanup PRs (#1779) 2026-07-27 10:05:38 +08:00
bench_deltanet_recurrence.py [Chore][Cleanup] repo slimming: dead code, duplicated tests/benches, compat shims, file defragmentation (#1764) 2026-07-26 12:59:48 +08:00
bench_dropout.py [Refactor][Ops] Infer static metadata from forward inputs (#1734) 2026-07-25 12:05:17 +08:00
bench_elementwise_manifest.py [Fix][Benchmark] stop losing results on native failures; derive workload keys from signature (#1745) 2026-07-25 22:11:21 +08:00
bench_engram.py [Refactor][POOL] strip over-design and scaffolding from the merged cleanup PRs (#1779) 2026-07-27 10:05:38 +08:00
bench_fft.py [Refactor][Ops] Infer static metadata from forward inputs (#1734) 2026-07-25 12:05:17 +08:00
bench_fp8_lightning_indexer.py [Refactor][POOL] strip over-design and scaffolding from the merged cleanup PRs (#1779) 2026-07-27 10:05:38 +08:00
bench_fp8_quant.py [Refactor][POOL] strip over-design and scaffolding from the merged cleanup PRs (#1779) 2026-07-27 10:05:38 +08:00
bench_fused_moe_experts.py [Chore][Cleanup] repo slimming: dead code, duplicated tests/benches, compat shims, file defragmentation (#1764) 2026-07-26 12:59:48 +08:00
bench_gated_deltanet.py [Refactor][POOL] strip over-design and scaffolding from the merged cleanup PRs (#1779) 2026-07-27 10:05:38 +08:00
bench_gated_deltanet_prefill.py [Chore][Cleanup] repo slimming: dead code, duplicated tests/benches, compat shims, file defragmentation (#1764) 2026-07-26 12:59:48 +08:00
bench_gated_deltanet_recurrence.py [Chore][Cleanup] repo slimming: dead code, duplicated tests/benches, compat shims, file defragmentation (#1764) 2026-07-26 12:59:48 +08:00
bench_gemm.py [Chore][Cleanup] repo slimming: dead code, duplicated tests/benches, compat shims, file defragmentation (#1764) 2026-07-26 12:59:48 +08:00
bench_gla_chunkwise.py [Refactor][POOL] strip over-design and scaffolding from the merged cleanup PRs (#1779) 2026-07-27 10:05:38 +08:00
bench_gla_recurrence.py [Chore][Cleanup] repo slimming: dead code, duplicated tests/benches, compat shims, file defragmentation (#1764) 2026-07-26 12:59:48 +08:00
bench_group_norm.py [Chore][Cleanup] repo slimming: dead code, duplicated tests/benches, compat shims, file defragmentation (#1764) 2026-07-26 12:59:48 +08:00
bench_grouped_gemm.py [Refactor][POOL] strip over-design and scaffolding from the merged cleanup PRs (#1779) 2026-07-27 10:05:38 +08:00
bench_grouped_gemm_baselines.py [Chore][Cleanup] repo slimming: dead code, duplicated tests/benches, compat shims, file defragmentation (#1764) 2026-07-26 12:59:48 +08:00
bench_independent_elementwise.py [Refactor][POOL] strip over-design and scaffolding from the merged cleanup PRs (#1779) 2026-07-27 10:05:38 +08:00
bench_instance_norm.py [Chore][Cleanup] repo slimming: dead code, duplicated tests/benches, compat shims, file defragmentation (#1764) 2026-07-26 12:59:48 +08:00
bench_logical_reduce.py [Refactor][POOL] strip over-design and scaffolding from the merged cleanup PRs (#1779) 2026-07-27 10:05:38 +08:00
bench_mamba.py [Refactor][POOL] strip over-design and scaffolding from the merged cleanup PRs (#1779) 2026-07-27 10:05:38 +08:00
bench_mamba2_e2e.py [Chore][Cleanup] repo slimming: dead code, duplicated tests/benches, compat shims, file defragmentation (#1764) 2026-07-26 12:59:48 +08:00
bench_mhc.py [Refactor][POOL] strip over-design and scaffolding from the merged cleanup PRs (#1779) 2026-07-27 10:05:38 +08:00
bench_moe_fused_activation.py [Chore][Cleanup] repo slimming: dead code, duplicated tests/benches, compat shims, file defragmentation (#1764) 2026-07-26 12:59:48 +08:00
bench_moe_fused_moe.py [Feat][Ops] Infer shape metadata from op inputs (#1651) 2026-07-06 17:51:47 +08:00
bench_moe_fused_topk.py [Refactor][POOL] strip over-design and scaffolding from the merged cleanup PRs (#1779) 2026-07-27 10:05:38 +08:00
bench_moe_grouped_gemm_nopad.py [Chore][Cleanup] repo slimming: dead code, duplicated tests/benches, compat shims, file defragmentation (#1764) 2026-07-26 12:59:48 +08:00
bench_moe_permute_align.py [Chore][Cleanup] repo slimming: dead code, duplicated tests/benches, compat shims, file defragmentation (#1764) 2026-07-26 12:59:48 +08:00
bench_moe_permute_nopad.py [Chore][Cleanup] repo slimming: dead code, duplicated tests/benches, compat shims, file defragmentation (#1764) 2026-07-26 12:59:48 +08:00
bench_moe_shared_fused_moe.py [Chore][Cleanup] repo slimming: dead code, duplicated tests/benches, compat shims, file defragmentation (#1764) 2026-07-26 12:59:48 +08:00
bench_moe_unpermute.py [Refactor][POOL] strip over-design and scaffolding from the merged cleanup PRs (#1779) 2026-07-27 10:05:38 +08:00
bench_norm.py [Chore][Cleanup] repo slimming: dead code, duplicated tests/benches, compat shims, file defragmentation (#1764) 2026-07-26 12:59:48 +08:00
bench_pool.py [Refactor][POOL] strip over-design and scaffolding from the merged cleanup PRs (#1779) 2026-07-27 10:05:38 +08:00
bench_reduce.py [Refactor][POOL] strip over-design and scaffolding from the merged cleanup PRs (#1779) 2026-07-27 10:05:38 +08:00
bench_rope.py [Refactor][POOL] strip over-design and scaffolding from the merged cleanup PRs (#1779) 2026-07-27 10:05:38 +08:00
bench_softmax.py [Refactor][POOL] strip over-design and scaffolding from the merged cleanup PRs (#1779) 2026-07-27 10:05:38 +08:00
bench_topk_selector.py [Refactor][POOL] strip over-design and scaffolding from the merged cleanup PRs (#1779) 2026-07-27 10:05:38 +08:00
bench_unary_elementwise.py [Refactor][POOL] strip over-design and scaffolding from the merged cleanup PRs (#1779) 2026-07-27 10:05:38 +08:00
bench_vector_norm.py [Refactor][POOL] strip over-design and scaffolding from the merged cleanup PRs (#1779) 2026-07-27 10:05:38 +08:00