chenyu520
/
Mooncake
forked from mooncake-track/Mooncake
0
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

fix wrong command in XpYd (#357) 

Signed-off-by: zibai <zibai.gj@alibaba-inc.com>
This commit is contained in:
gujing 2025-05-13 20:31:58 +08:00 committed by GitHub
parent feb26e7802
commit 32a76aa3eb
No known key found for this signature in database
GPG Key ID: B5690EEEBB952194
1 changed files with 1 additions and 1 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

2
doc/en/sglang-integration-v1.md
View File

@ -97,5 +97,5 @@ Note:
- TP is supported but not required, you can remove `--tp-size 2` if you want.
- The `--disaggregation-ib-device` is the device to be used for data transmission, it is optional since we will detect this config automatically. Or you can still explicitly specify devices if needed. If multiple NIC devices are used, they can be separated by commas, such as "erdma_0,erdma_1". Please note that there are no spaces between them.
- XpYd is supported. It is ok to run one prefill and multiple decode instances on the same node, however, multiple prefill instances on the same node are not supported due to the port conflict of bootstrap server.
- e.g., `python3 -m sglang.srt.disaggregation.mini_lb --prefill http://192.168.0.137:30000,http://192.168.0.140:30000 --decode http://192.168.0.137:30001,http://192.168.0.140:30001 --host 0.0.0.0 --port 8000`
- e.g., `python3 -m sglang.srt.disaggregation.mini_lb --prefill http://192.168.0.137:30000 http://192.168.0.140:30000 --decode http://192.168.0.137:30001 http://192.168.0.140:30001 --host 0.0.0.0 --port 8000`
- HuggingFace timeout can be addressed by `export SGLANG_USE_MODELSCOPE=true`