# Test Evidence

## Script Tests

Command run on 2026-07-20:

```bash
python -m unittest \
  code/tests/model_executor/offloader/test_acceptance_scripts.py \
  code/tests/model_executor/offloader/test_accuracy_repro.py
```

Result (final rerun on 2026-07-21):

```text
Ran 35 tests in 0.341s
OK
```

The complete remote offloader suite was then run under the CANN 8.5.1 environment:

```bash
python -m pytest tests/model_executor/offloader -q
```

```text
69 passed, 1 warning, 21 subtests passed in 11.05s
```

The warning is the upstream `pkg_resources` deprecation warning from `patch_weight_utils.py`.

## Remote Runtime

Verified under `/data/vllm-ascend-v0.18.0/.venv`:

```text
vllm 0.18.0+empty
vllm-ascend 0.1.dev1+g72dc68973.d20260720
torch 2.9.0+cpu
torch-npu 2.9.0
triton-ascend 3.2.0.dev20260322
npu_available True
npu_device_count 1
```

The rebuilt `AddRmsNormBias` custom operator loaded and produced the expected values:

```text
custom_enabled True
result (2, 16) (2, 1) 1.0
residual 2.0
```

## Formal Performance Evidence

Command settings: 3 repeats, 1 warmup round, 3 measured rounds, 8 prompts,
128 input tokens, and 64 output tokens. Nine raw runs were aggregated by median.

| Case | Output tok/s | Change | Saved weight storage | Available KV cache |
| --- | ---: | ---: | ---: | ---: |
| `baseline_0_1` | 742.77 | 0.00% | 0.0000 GB | 27.27 GiB |
| `group_8_num_1` | 678.85 | -8.61% | 0.2615 GB | 27.53 GiB |
| `group_8_num_2` | 383.49 | -48.37% | 0.5230 GB | 27.79 GiB |

Remote artifacts: `/data/v018-performance-formal-20260721`.
Local copy: `materials/results/v018-performance-formal-20260721`.

## Full Accuracy Evidence

`ACCURACY_LIMIT=0`, temperature 0, seed 0:

| Dataset | Rows | Baseline | `group_8_num_1` | `group_8_num_2` |
| --- | ---: | ---: | ---: | ---: |
| GSM8K | 1319 | 51 (3.8666%) | 51 (3.8666%) | 51 (3.8666%) |
| GPQA Diamond | 198 | 0 (0.0000%) | 0 (0.0000%) | 0 (0.0000%) |

The verifier returned `all_passed: true` with `MAX_ACCURACY_DROP=0`. The
example JSONL hashes are identical across all three cases for each dataset, so
the claimed parameter cases reproduce every baseline prediction byte-for-byte.

Remote artifacts: `/data/v018-accuracy-formal-20260721`.
Local copy: `materials/results/v018-accuracy-formal-20260721`.

## Final Device Health

After the cloud instance restart, logical runtime device 0 maps to physical NPU
0/chip 0. Before and after the formal runs:

```text
Health Status : OK
Error Code : NA
Error Information : NA
```

The optional `msserviceprofiler` entry point emits a non-fatal missing-module
message. It does not affect Ascend activation, inference, summaries, or clean
engine shutdown.
