# vLLM Ascend v0.18.0 Deployment and Reproduction

## Scope

This submission adapts and tests these two parameters on vLLM Ascend v0.18.0:

- `--offload-group-size`
- `--offload-num-in-group`

The comparison matrix is fixed to three cases:

| Case | group size | num in group |
| --- | ---: | ---: |
| `baseline_0_1` | 0 | 1 |
| `group_8_num_1` | 8 | 1 |
| `group_8_num_2` | 8 | 2 |

The v0.18.0 base scan did not find an existing implementation of these parameters, so the additional requirement to outperform an upstream implementation is not applicable. Performance scripts still quantify throughput, latency, process RSS, and NPU HBM trade-offs against the baseline.

## Verified Software Matrix

| Component | Version |
| --- | --- |
| vLLM | `0.18.0+empty` |
| vLLM Ascend source base | `72dc68973bd7e6ceef9a122de316a7ee9c9a84aa` |
| CANN | `8.5.1` |
| torch | `2.9.0+cpu` |
| torch-npu | `2.9.0` |
| triton-ascend | `3.2.0.dev20260322` |
| Python | `3.11` |

Remote paths:

```text
/data/vllm-ascend-v0.18.0
/data/vllm-source-v0.18.0
/data/vllm-ascend-v0.18.0/.venv
/usr/local/Ascend/cann-8.5.1/cann-8.5.1/set_env.sh
/data/models/Qwen2.5-1.5B-Instruct
```

## Runtime Installation

Install the CANN 8.5.1 toolkit, 910B operator package, and NNAL package first. Then rebuild every CANN-bound plugin artifact:

```bash
cd /data/vllm-ascend-v0.18.0
bash scripts/install_v018_runtime_compat.sh
```

The installer pins the Python packages, downloads the exact Triton wheel, clears only generated build artifacts under the guarded repository path, and rebuilds the editable vLLM Ascend package.

## Device Mapping and Health

The runtime uses logical device 0. The reproduction scripts derive the physical
NPU index from `npu-smi info -m` before running the health query. After the
cloud instance restart, logical device 0 maps to physical NPU 0/chip 0 and the
device reports `Health Status : OK`:

```bash
export ASCEND_RT_VISIBLE_DEVICES=0
bash scripts/check_npu_health.sh
```

Set `NPU_INDEX` and `NPU_CHIP_INDEX` only when an explicit override is needed.
Do not bypass the health check for formal runs. `SKIP_NPU_HEALTH_CHECK=1`
exists only for environments whose driver does not implement the health query.

## Dataset Evidence

Materialized datasets:

| Dataset | Rows | SHA-256 |
| --- | ---: | --- |
| GSM8K test | 1319 | `32c548f08195e19e33408b844dd7be6aa4bcae457d957bc05909ff4bd4a00595` |
| GPQA Diamond | 198 | `0c16aaa98af019ca691f98eeec9f99ca4cabfb9c920c9e19e40ebedf8d6a0174` |

Files are under `/data/accuracy-gsm8k-gpqa`. Each accuracy result records the dataset source, revision, row count, and hash.

## Performance Reproduction

```bash
cd /data/vllm-ascend-v0.18.0
MODEL_PATH=/data/models/Qwen2.5-1.5B-Instruct \
RESULTS_DIR=/data/v018-performance-results \
REPEATS=3 WARMUP_ROUNDS=1 MEASURED_ROUNDS=3 \
PROMPT_COUNT=8 INPUT_TOKENS=128 MAX_TOKENS=64 MAX_MODEL_LEN=512 \
bash scripts/run_benchmark_all.sh
```

Artifacts:

```text
performance-runs.jsonl
performance-summary.json
performance-summary.csv
performance-summary.md
performance-run.log
```

### Formal performance result

The formal run at `/data/v018-performance-formal-20260721` contains nine raw
runs: three independent model loads for each case. The table reports medians.

| Case | Selected layers | Weight storage saved | KV cache available | Output tok/s | Change | P50 round latency |
| --- | --- | ---: | ---: | ---: | ---: | ---: |
| `baseline_0_1` | none | 0.0000 GB | 27.27 GiB | 742.77 | 0.00% | 0.6927 s |
| `group_8_num_1` | 7, 15, 23 | 0.2615 GB | 27.53 GiB | 678.85 | -8.61% | 0.7501 s |
| `group_8_num_2` | 6, 7, 14, 15, 22, 23 | 0.5230 GB | 27.79 GiB | 383.49 | -48.37% | 1.3351 s |

The two parameters provide a quantifiable memory/throughput trade-off. Do not
interpret the post-generation `npu-smi` delta as weight storage: vLLM consumes
newly available HBM for a larger KV cache. The model-loader logs provide the
weight-storage and KV-capacity evidence shown above.

## Accuracy Reproduction

Full GSM8K and GPQA Diamond:

```bash
cd /data/vllm-ascend-v0.18.0
MODEL_PATH=/data/models/Qwen2.5-1.5B-Instruct \
DATA_DIR=/data/accuracy-gsm8k-gpqa \
RESULTS_DIR=/data/v018-accuracy-results \
ACCURACY_LIMIT=0 \
bash scripts/run_accuracy_gsm8k_gpqa.sh
```

For a plumbing smoke test, set `ACCURACY_LIMIT=2`; limited results are not submission accuracy evidence.

After all six dataset/case runs, `verify_accuracy_matrix.py` checks dataset hashes, sample counts, claimed parameter settings, and accuracy against the baseline. The default `MAX_ACCURACY_DROP=0` requires zero accuracy loss. It writes:

```text
accuracy-summary.json
accuracy-summary.csv
accuracy-summary.md
```

### Formal accuracy result

The full run at `/data/v018-accuracy-formal-20260721` used temperature 0 and
seed 0. No sample limit was applied.

| Dataset | Samples | Baseline | `group_8_num_1` | `group_8_num_2` | Accuracy change |
| --- | ---: | ---: | ---: | ---: | ---: |
| GSM8K test | 1319 | 51/1319 (3.8666%) | 51/1319 (3.8666%) | 51/1319 (3.8666%) | 0.0000 |
| GPQA Diamond | 198 | 0/198 (0.0000%) | 0/198 (0.0000%) | 0/198 (0.0000%) | 0.0000 |

`accuracy-summary.json` reports `all_passed: true`. In addition, the three
GSM8K example JSONL files have the same SHA-256
`e6c13ad281308e2dc103279f6820b9212a9895ecb80ec5e705be4697deb2eead`,
and the three GPQA example files have the same SHA-256
`baa5a967b1f4c4d96f6be810f7176019b037617141436255959f2b7b046c2623`.
Thus both claimed parameter cases reproduce the baseline output byte-for-byte.
The GPQA number is the zero-shot baseline of this 1.5B model and prompt, not a
claim of high task accuracy.

## Unified Reproduction

```bash
cd /data/vllm-ascend-v0.18.0
MODEL_PATH=/data/models/Qwen2.5-1.5B-Instruct \
DATA_DIR=/data/accuracy-gsm8k-gpqa \
REPRO_RESULTS_DIR=/data/v018-reproduction-results \
bash scripts/run_reproduce_all.sh
```

The unified script runs performance before accuracy and writes `reproduction-manifest.json` with commands, package versions, source commit, model path, and artifact locations.

## Verified Server State

After the instance restart, the mapped NPU changed from physical ID 7 to ID 0.
The final performance run, full accuracy run, and final test suite all completed
with the health check reporting `Health Status : OK`, `Error Code : NA`.

The environment contains an unrelated optional `msserviceprofiler` entry point
whose module is absent. vLLM logs that plugin-load error and continues; Ascend
platform activation, model execution, result generation, and shutdown all
complete successfully.
