forked from metax-maca/op_optimization
fused_moe教程修改
This commit is contained in:
parent
6e6419fdb4
commit
925a3de34a
|
|
@ -59,6 +59,7 @@
|
|||
|
||||
* 已进入赛事专属镜像环境。
|
||||
|
||||
|
||||
**工具准备：**
|
||||
|
||||
* 已准备 Agent 工具；
|
||||
|
|
@ -173,127 +174,127 @@ PYTHON_BIN=/path/to/python bash scripts/build_fused_moe_i8_tn_pybind.sh
|
|||
|
||||
且成功生成 `fused_moe/standalone/fused_moe_i8_tn/build/fused_moe_i8_tn_pybind.cpython-310-x86_64-linux-gnu.so` 文件
|
||||
|
||||
**常见问题：**
|
||||
**常见问题：**
|
||||
|
||||
| 报错 | 原因 | 解决办法 |
|
||||
| --- | --- | --- |
|
||||
| `Python.h: No such file or directory` | Python 头文件路径未找到 | 确认 `PYTHON_BIN` 路径正确，脚本自动探测 `sysconfig.get_path('include')` |
|
||||
| `libpython3.x.so: cannot find` | 链接时找不到 Python 库 | 1、执行 `find $CONDA_PREFIX -name "libpython3*.so*"`查找绝对路径<br>2、将该路径赋值给 `LIBPYTHON_PATH` |
|
||||
| `recompile with -fPIC` | 编译未开启位置无关代码 | 确保 `mxcc`/ `g++`编译参数中有 `-fPIC` |
|
||||
| `permission denied` | 无脚本执行权限 | `chmod +x scripts/*.sh` |
|
||||
| `undefined reference to Py_...` | Python 版本不匹配 | 确认编译脚本中`PYTHON_BIN`路径与当前运行的 Python 环境完全一致 |
|
||||
| 报错 | 原因 | 解决办法 |
|
||||
| --- | --- | --- |
|
||||
| `Python.h: No such file or directory` | Python 头文件路径未找到 | 确认 `PYTHON_BIN` 路径正确，脚本自动探测 `sysconfig.get_path('include')` |
|
||||
| `libpython3.x.so: cannot find` | 链接时找不到 Python 库 | 1、执行 `find $CONDA_PREFIX -name "libpython3*.so*"`查找绝对路径<br>2、将该路径赋值给 `LIBPYTHON_PATH` |
|
||||
| `recompile with -fPIC` | 编译未开启位置无关代码 | 确保 `mxcc`/ `g++`编译参数中有 `-fPIC` |
|
||||
| `permission denied` | 无脚本执行权限 | `chmod +x scripts/*.sh` |
|
||||
| `undefined reference to Py_...` | Python 版本不匹配 | 确认编译脚本中`PYTHON_BIN`路径与当前运行的 Python 环境完全一致 |
|
||||
|
||||
### 步骤 4：正确性测试
|
||||
### 步骤 4：正确性测试
|
||||
|
||||
**目标：**验证 reference 计算、pybind 计算、Triton 计算这三种方式计算结果的数值是否完全一致。
|
||||
**目标：**验证 reference 计算、pybind 计算、Triton 计算这三种方式计算结果的数值是否完全一致。
|
||||
|
||||
**操作：**运行 `fused_moe/scripts/run_fused_moe_i8_tn_pybind_test.sh` 脚本
|
||||
**操作：**运行 `fused_moe/scripts/run_fused_moe_i8_tn_pybind_test.sh` 脚本
|
||||
|
||||
**命令示例：**
|
||||
**命令示例：**
|
||||
|
||||
```apl
|
||||
bash scripts/run_fused_moe_i8_tn_pybind_test.sh --backend all # 运行全部计算方式
|
||||
```apl
|
||||
bash scripts/run_fused_moe_i8_tn_pybind_test.sh --backend all # 运行全部计算方式
|
||||
|
||||
# --backend：选择计算方式
|
||||
# 只测 pybind：
|
||||
bash scripts/run_fused_moe_i8_tn_pybind_test.sh --backend pybind
|
||||
# 只测 triton：
|
||||
bash scripts/run_fused_moe_i8_tn_pybind_test.sh --backend triton
|
||||
# 只测 reference:
|
||||
bash scripts/run_fused_moe_i8_tn_pybind_test.sh --backend reference
|
||||
```
|
||||
# --backend：选择计算方式
|
||||
# 只测 pybind：
|
||||
bash scripts/run_fused_moe_i8_tn_pybind_test.sh --backend pybind
|
||||
# 只测 triton：
|
||||
bash scripts/run_fused_moe_i8_tn_pybind_test.sh --backend triton
|
||||
# 只测 reference:
|
||||
bash scripts/run_fused_moe_i8_tn_pybind_test.sh --backend reference
|
||||
```
|
||||
|
||||
**预期结果：**
|
||||
**预期结果：**
|
||||
|
||||
编译成功无报错，输出示例如下：
|
||||
编译成功无报错，输出示例如下：
|
||||
|
||||
> pybind:fused\_moe\_i8\_tn\_topk1 passed: rows=256, cols=128, sample C\[0\]=0.69531, C\[last\]=-0.44531
|
||||
> pybind:fused\_moe\_i8\_tn\_topk1 passed: rows=256, cols=128, sample C\[0\]=0.69531, C\[last\]=-0.44531
|
||||
|
||||
> pybind:fused\_moe\_i8\_tn\_topk2 passed: rows=512, cols=128, sample C\[0\]=-0.57813, C\[last\]=-0.49805
|
||||
> pybind:fused\_moe\_i8\_tn\_topk2 passed: rows=512, cols=128, sample C\[0\]=-0.57813, C\[last\]=-0.49805
|
||||
|
||||
> pybind:fused\_moe\_i8\_tn\_topk3 passed: rows=384, cols=128, sample C\[0\]=-1.08594, C\[last\]=-0.33594
|
||||
> pybind:fused\_moe\_i8\_tn\_topk3 passed: rows=384, cols=128, sample C\[0\]=-1.08594, C\[last\]=-0.33594
|
||||
|
||||
> reference:fused\_moe\_i8\_tn\_topk1 passed: rows=256, cols=128, sample C\[0\]=0.6934, C\[last\]=-0.4451
|
||||
> reference:fused\_moe\_i8\_tn\_topk1 passed: rows=256, cols=128, sample C\[0\]=0.6934, C\[last\]=-0.4451
|
||||
|
||||
> reference:fused\_moe\_i8\_tn\_topk2 passed: rows=512, cols=128, sample C\[0\]=-0.5768, C\[last\]=-0.4975
|
||||
> reference:fused\_moe\_i8\_tn\_topk2 passed: rows=512, cols=128, sample C\[0\]=-0.5768, C\[last\]=-0.4975
|
||||
|
||||
> reference:fused\_moe\_i8\_tn\_topk3 passed: rows=384, cols=128, sample C\[0\]=-1.0875, C\[last\]=-0.3362
|
||||
> reference:fused\_moe\_i8\_tn\_topk3 passed: rows=384, cols=128, sample C\[0\]=-1.0875, C\[last\]=-0.3362
|
||||
|
||||
> triton:fused\_moe\_i8\_tn\_topk1 passed: rows=256, cols=128, sample C\[0\]=0.69337, C\[last\]=-0.44513
|
||||
> triton:fused\_moe\_i8\_tn\_topk1 passed: rows=256, cols=128, sample C\[0\]=0.69337, C\[last\]=-0.44513
|
||||
|
||||
> triton:fused\_moe\_i8\_tn\_topk2 passed: rows=512, cols=128, sample C\[0\]=-0.57678, C\[last\]=-0.49749
|
||||
> triton:fused\_moe\_i8\_tn\_topk2 passed: rows=512, cols=128, sample C\[0\]=-0.57678, C\[last\]=-0.49749
|
||||
|
||||
> triton:fused\_moe\_i8\_tn\_topk3 passed: rows=384, cols=128, sample C\[0\]=-1.08748, C\[last\]=-0.33618
|
||||
> triton:fused\_moe\_i8\_tn\_topk3 passed: rows=384, cols=128, sample C\[0\]=-1.08748, C\[last\]=-0.33618
|
||||
|
||||
**结果解释：**
|
||||
**结果解释：**
|
||||
|
||||
* “pybind/reference/Triton”：三种计算方式；
|
||||
* “pybind/reference/Triton”：三种计算方式；
|
||||
|
||||
* “fused\_moe\_i8\_tn\_topk1/2/3 passed”：测试算子通过数值校验，数值误差在允许范围内且无明显异常，否则会报错 FAILED；
|
||||
* “fused\_moe\_i8\_tn\_topk1/2/3 passed”：测试算子通过数值校验，数值误差在允许范围内且无明显异常，否则会报错 FAILED；
|
||||
|
||||
* ”rows=... , cols=...“：输出 Tensor 的行和列的大小；
|
||||
* ”rows=... , cols=...“：输出 Tensor 的行和列的大小；
|
||||
|
||||
* ”sample C\[0\]=... , C\[last\]=...“：首尾采样值，用于辅助定位数值偏差，不作为精度判定依据。
|
||||
* ”sample C\[0\]=... , C\[last\]=...“：首尾采样值，用于辅助定位数值偏差，不作为精度判定依据。
|
||||
|
||||
|
||||
**常见问题：**
|
||||
**常见问题：**
|
||||
|
||||
| 报错 | 原因 | 解决办法 |
|
||||
| --- | --- | --- |
|
||||
| `ModuleNotFoundError: fused_moe_i8_tn_pybind` | pybind 模块未编译或未加入 `PYTHONPATH` | 回到步骤 3，确认 `.so`已生成；执行 `export PYTHONPATH=/root/Project/fused_moe:$PYTHONPATH` |
|
||||
| `FAILED: max abs diff too large` | 数值误差超过阈值 | 检查 scale 是否应用位置错误；确认 TopK 索引与权重是否一致 |
|
||||
| `FAILED: shape mismatch` | 输出张量形状不一致 | 检查 Token Permute / Unpermute 逻辑；确认 expert 维度对齐 |
|
||||
| `FAILED: NaN or Inf detected` | 溢出或未初始化内存 | 检查 INT8 乘加是否溢出；确认 GEMM 输出是否反量化 |
|
||||
| 终端长时间无输出 | Kernel 死锁或 Launch 失败 | 减小测试 shape；检查是否触发 MACA 硬件限制 |
|
||||
| 报错 | 原因 | 解决办法 |
|
||||
| --- | --- | --- |
|
||||
| `ModuleNotFoundError: fused_moe_i8_tn_pybind` | pybind 模块未编译或未加入 `PYTHONPATH` | 回到步骤 3，确认 `.so`已生成；执行 `export PYTHONPATH=/root/Project/fused_moe:$PYTHONPATH` |
|
||||
| `FAILED: max abs diff too large` | 数值误差超过阈值 | 检查 scale 是否应用位置错误；确认 TopK 索引与权重是否一致 |
|
||||
| `FAILED: shape mismatch` | 输出张量形状不一致 | 检查 Token Permute / Unpermute 逻辑；确认 expert 维度对齐 |
|
||||
| `FAILED: NaN or Inf detected` | 溢出或未初始化内存 | 检查 INT8 乘加是否溢出；确认 GEMM 输出是否反量化 |
|
||||
| 终端长时间无输出 | Kernel 死锁或 Launch 失败 | 减小测试 shape；检查是否触发 MACA 硬件限制 |
|
||||
|
||||
### 步骤5：性能测试
|
||||
### 步骤5：性能测试
|
||||
|
||||
**目标：**输出 benchmark 结果对比表
|
||||
**目标：**输出 benchmark 结果对比表
|
||||
|
||||
**操作：**运行 `fused_moe/scripts/run_fused_moe_i8_tn_benchmark.sh` 脚本
|
||||
**操作：**运行 `fused_moe/scripts/run_fused_moe_i8_tn_benchmark.sh` 脚本
|
||||
|
||||
**命令示例：**
|
||||
**命令示例：**
|
||||
|
||||
```apl
|
||||
bash scripts/run_fused_moe_i8_tn_benchmark.sh --backend all --warmup 5 --iters 20
|
||||
# --backend：选择计算方式
|
||||
# --warmup：设置预热次数
|
||||
# --iters：设置迭代次数
|
||||
```
|
||||
```apl
|
||||
bash scripts/run_fused_moe_i8_tn_benchmark.sh --backend all --warmup 5 --iters 20
|
||||
# --backend：选择计算方式
|
||||
# --warmup：设置预热次数
|
||||
# --iters：设置迭代次数
|
||||
```
|
||||
|
||||
**预期结果：**
|
||||
**预期结果：**
|
||||
|
||||
编译成功无报错，输出示例如下：
|
||||
编译成功无报错，输出示例如下：
|
||||
|
||||
> pybind:fused\_moe\_i8\_tn\_topk1 benchmark: avg\_ms=0.308978, TOPS=0.027149, warmup=5, iters=20
|
||||
> pybind:fused\_moe\_i8\_tn\_topk1 benchmark: avg\_ms=0.308978, TOPS=0.027149, warmup=5, iters=20
|
||||
|
||||
> pybind:fused\_moe\_i8\_tn\_topk2 benchmark: avg\_ms=0.304500, TOPS=0.055098, warmup=5, iters=20
|
||||
> pybind:fused\_moe\_i8\_tn\_topk2 benchmark: avg\_ms=0.304500, TOPS=0.055098, warmup=5, iters=20
|
||||
|
||||
> pybind:fused\_moe\_i8\_tn\_topk3 benchmark: avg\_ms=0.297775, TOPS=0.042256, warmup=5, iters=20
|
||||
> pybind:fused\_moe\_i8\_tn\_topk3 benchmark: avg\_ms=0.297775, TOPS=0.042256, warmup=5, iters=20
|
||||
|
||||
> reference:fused\_moe\_i8\_tn\_topk1 benchmark: avg\_ms=1685.43, TOPS=0.000005, warmup=5, iters=20
|
||||
> reference:fused\_moe\_i8\_tn\_topk1 benchmark: avg\_ms=1685.43, TOPS=0.000005, warmup=5, iters=20
|
||||
|
||||
> reference:fused\_moe\_i8\_tn\_topk2 benchmark: avg\_ms=3384.52, TOPS=0.000005, warmup=5, iters=20
|
||||
> reference:fused\_moe\_i8\_tn\_topk2 benchmark: avg\_ms=3384.52, TOPS=0.000005, warmup=5, iters=20
|
||||
|
||||
> reference:fused\_moe\_i8\_tn\_topk3 benchmark: avg\_ms=2532.14, TOPS=0.000005, warmup=5, iters=20
|
||||
> reference:fused\_moe\_i8\_tn\_topk3 benchmark: avg\_ms=2532.14, TOPS=0.000005, warmup=5, iters=20
|
||||
|
||||
> triton:fused\_moe\_i8\_tn\_topk1 benchmark: avg\_ms=19.013421, TOPS=0.000441, warmup=5, iters=20
|
||||
> triton:fused\_moe\_i8\_tn\_topk1 benchmark: avg\_ms=19.013421, TOPS=0.000441, warmup=5, iters=20
|
||||
|
||||
> triton:fused\_moe\_i8\_tn\_topk2 benchmark: avg\_ms=16.745914, TOPS=0.001002, warmup=5, iters=20
|
||||
> triton:fused\_moe\_i8\_tn\_topk2 benchmark: avg\_ms=16.745914, TOPS=0.001002, warmup=5, iters=20
|
||||
|
||||
> triton:fused\_moe\_i8\_tn\_topk3 benchmark: avg\_ms=19.630328, TOPS=0.000641, warmup=5, iters=20
|
||||
> triton:fused\_moe\_i8\_tn\_topk3 benchmark: avg\_ms=19.630328, TOPS=0.000641, warmup=5, iters=20
|
||||
|
||||
**结果解释：**
|
||||
**结果解释：**
|
||||
|
||||
* “pybind/reference/Triton”：三种计算方式；
|
||||
* “pybind/reference/Triton”：三种计算方式；
|
||||
|
||||
* “fused\_moe\_i8\_tn\_topk1/2/3”：分别对应选择前 1 / 2 / 3 个专家场景下的 MoE 算子；
|
||||
* “fused\_moe\_i8\_tn\_topk1/2/3”：分别对应选择前 1 / 2 / 3 个专家场景下的 MoE 算子；
|
||||
|
||||
* “avg\_ms”：平均算子执行耗时（毫秒），这里不计算预热时间，只计算正式迭代的<E4BBA3><E79A84><EFBFBD>间<EFBFBD><E997B4><EFBFBD>
|
||||
* “avg\_ms”：平均算子执行耗时（毫秒），这里不计算预热时间，只计算正式迭代的<E4BBA3><E79A84><EFBFBD>间<EFBFBD><E997B4><EFBFBD>
|
||||
|
||||
* “TOPS”：Tera Operations Per Second，本次 MoE 算子的总运算量 / 实际耗时；
|
||||
* “TOPS”：Tera Operations Per Second，本次 MoE 算子的总运算量 / 实际耗时；
|
||||
|
||||
* “warmup=5, iters=20”：预热轮数和正式迭代数。
|
||||
* “warmup=5, iters=20”：预热轮数和正式迭代数。
|
||||
|
||||
|
||||
**常见错误：**
|
||||
|
|
@ -456,7 +457,7 @@ PYTHON_BIN=/path/to/python bash scripts/build_fused_moe_i8_tn_pybind.sh
|
|||
* 顶部：任务名称、创建/更新时间、适配硬件、当前轮次进度
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
3. 获取优化结果：当前任务状态为【已完成】时，可在详情页查看优化结果：
|
||||
|
||||
|
|
|
|||
Loading…
Reference in New Issue