* feat: expose batch_replica_clear in Python binding
Add batch_replica_clear(keys, segment_name) to PyClient/RealClient/DummyClient.
Allows explicit deletion of replicas for lease-expired keys.
Also adds Client::getClientId() public getter.
---------
Signed-off-by: hnts03-moreh <geonwoo.choi@moreh.io>
* Add ROCm HIP support to the Mooncake Python package
Expose HIP as the Python-facing AMD GPU transport, wire HIP transport into the
build and runtime selection paths, and fix sticky peer-access errors so
repeated connector initialization works reliably. Update wheel packaging and
container validation to cover ROCm HIP usage in vllm-omni.
Signed-off-by: Zejian Wang <zejianwang@sjtu.edu.cn>
* [Bugfix] Decouple HIP transport from NVLink branch
HIP IPC is intra-node only and should coexist with RDMA, not replace it.
Signed-off-by: Zejian Wang <zejianwang.sjtu.edu.cn>
Signed-off-by: Zejian Wang <zejianwang@sjtu.edu.cn>
* Abandon manual transport
Signed-off-by: Zejian Wang <zejianwang.sjtu.edu.cn>
Signed-off-by: Zejian Wang <zejianwang@sjtu.edu.cn>
* style: apply clang-format to changed files
Signed-off-by: Zejian Wang <zejianwang@sjtu.edu.cn>
Made-with: Cursor
Signed-off-by: Zejian Wang <zejianwang@sjtu.edu.cn>
---------
Signed-off-by: Zejian Wang <zejianwang@sjtu.edu.cn>
Signed-off-by: Zejian Wang <zejianwang.sjtu.edu.cn>
Co-authored-by: Zejian Wang <zejianwang@sjtu.edu.cn>
* [Store] add get_into_range and src_offset plumbing
Add single-key range read support to Mooncake Store:
- get_into_range: read [src_offset, src_offset+size) from an object
into (buffer + dst_offset)
- submitRangeRead in TransferSubmitter with src_offset support
- Get() overload and TransferReadRange in Client
- DummyClient RPC path via get_into_range_dummy_helper
- PyClient virtual interface for get_into_range
Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
* Add dedicated peer liveness probe for PG recovery
* [PG]: add elastic test for GPU mode
* [feat]: code format
* [TEST] rm time delay for mock fault rank
* [TENT]: rm resubmit in get transfer status
During Elastic EP rank recovery, the recovered rank must re-run expensive initialization (especially CUDA graph capture).
Previously, this required the recovered and healthy ranks to enter the same communication phase, which paused healthy-rank inference and increased recovery disruption.
This change enables **asynchronous recovered-rank initialization** in Mooncake PG: recovered ranks can initialize in isolation first, then join the live process group after local recovery work is finished.
* [Store] feat: CXL storage full features.
* fix(ci): resolve cxl test failure and code format
* fix(ci): add CXL protocol support and fix code format issues
* [Store] feat: CXL storage full features, reset and rm extern/pybind
* [TE] Support TCP fallback in EFA build and improve EFA documentation (#1523)
When building with USE_EFA=ON, auto_discover is disabled to prevent
RDMA transport installation (QP creation fails on EFA devices). This
means TCP transport is also not installed automatically. Add explicit
TCP transport installation for non-EFA protocols in the EFA build path.
Documentation changes:
- build.md: Add USE_EFA option and clarify USE_CUDA default/purpose
- supported-protocols.md: Add EFA as a supported protocol
- efa_transport.md: Add USE_CUDA=ON to build command, document GPU
memory requirement
Co-authored-by: whn09 <whn09@users.noreply.github.com>
Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
Co-authored-by: Teng Ma <teng-ma@linux.alibaba.com>
* [Store] Optimize BucketStorageBackend for reduced lock contention and add delete safety (#1456)
* fix(ci): resolve cxl test failure and code format
* [Store] Add Local Cache Mechanism for Mooncake Store Client (#1226)
* feat(Store): add local hot cache for client
* feat(Store): add client local hot cache log to show performance
* fix: local hot cache initialize bug
* fix(Store): Mooncake put slice is max 16MB， so make local hot cache block 16MB
* feat(Store): move local hot cache initialization to Client::Create
* feat(Store): local hot cache remove unused small block implementation
* feat(Store): add client local hot cache unit test
* fix(Store): modify client local hot cache suit with v0.3.7
* feat(Store): change local hot cache unit tes
* fix: initialize local hot cache with negative value
* feat: use in process master and metadata fro local hot cache unit test.
* feat: update local hot cache to one replica one slice version
* fix: local hot cache unit test use in process master service
* fix: code style fix
* fix: fix dirty read when client wants to read a previously hitted hot block but the hot block is modified by incoming put actions
* fix: local hot cache unit test use in process master service
* fix: code format fix
* fix: fix comment problems for
* feat: add local hot asynchronous queue size limit
* fix: local hot cache task involves the block so that there is no memcpy operation when inserting local hot cache
* fix: code check fix
* fix: update block in_use prop to reference count
---------
Co-authored-by: shichangzhang064 <zhangshichang@h-partners.com>
* fix(ci): add CXL protocol support and fix code format issues
* fix: address comments from code review
* fix(ci): resolve cxl test failure
* fix(ci): resolve ci error
---------
Co-authored-by: 王鹤男 <wanghenan09@gmail.com>
Co-authored-by: whn09 <whn09@users.noreply.github.com>
Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
Co-authored-by: Teng Ma <teng-ma@linux.alibaba.com>
Co-authored-by: Mahesh Bapatu <153306023+maheshrbapatu@users.noreply.github.com>
Co-authored-by: Shichang Zhang <77728761+Shichang-Zhang@users.noreply.github.com>
Co-authored-by: shichangzhang064 <zhangshichang@h-partners.com>
* feat: use setuptools_scm for more elegant version and support MOONCAKE_LOCAL_VERSION env
* feat: more compatible
* feat: add fetch-tags for build ci
* feat: manually add Fetch git tags actions
* feat: use fetch-depth: 0 and filter: tree:0
---------
Co-authored-by: huangweixiao <huangweixiao@moonshot.cn>
* [Store] Retrieve the actual glibc version during the build process
Signed-off-by: Cruz Zhao <CruzZhao@linux.alibaba.com>
* [build] avoid use hardcode libc.so.6 and avoid use python in shell script
Signed-off-by: Cruz Zhao <CruzZhao@linux.alibaba.com>
---------
Signed-off-by: Cruz Zhao <CruzZhao@linux.alibaba.com>
* Add a CI test for Mooncake EP Backend (CPU only)
* Cleanup disk space
* Update mooncake_backend.cpp
* Do not call cuda APIs for the mooncake-cpu backend
* Change transport protocol from RDMA to TCP
* Fix
* Fix memory leak in MooncakeBackend destructor (#1164)
* Initial plan
* Add destructor to fix memory leak in MooncakeBackend
Co-authored-by: UNIDY2002 <41641284+UNIDY2002@users.noreply.github.com>
* Final update: Memory leak fix completed
Co-authored-by: UNIDY2002 <41641284+UNIDY2002@users.noreply.github.com>
* Delete _codeql_detected_source_root
---------
Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com>
Co-authored-by: UNIDY2002 <41641284+UNIDY2002@users.noreply.github.com>
Co-authored-by: Xun Sun <UNIDY2002@outlook.com>
* Update mooncake-ep/src/mooncake_worker.cu
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
* Use `kWildcardLocation` when registering GPU memory
---------
Co-authored-by: Copilot <198982749+Copilot@users.noreply.github.com>
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
* [Store]: Add independent deployment implementation for Client
Signed-off-by: Xingrui Yi <yixingrui@linux.alibaba.com>
* [Doc]: mooncake-store: add introduce for client standalone mode
Signed-off-by: Xingrui Yi <yixingrui@linux.alibaba.com>
* [CI]: Add dummy client test
Signed-off-by: Xingrui Yi <yixingrui@linux.alibaba.com>
* [Store]: Change dummy client setup into a new func
Signed-off-by: Xingrui Yi <yixingrui@linux.alibaba.com>
* [Store]: Add more log for dummy client
Signed-off-by: Xingrui Yi <yixingrui@linux.alibaba.com>
---------
Signed-off-by: Xingrui Yi <yixingrui@linux.alibaba.com>
* Implement a fallback impl of EP's dispatch & combine when IBGDA is not available
* Fix format
* Fix
* Fix
* Add fault-tolerance support for the fallback path
* Fix
* Fix
* Exit from `__init__` early, if using fallback impl of EP
* Fix
* Fix
* Bump version to 0.3.7 in pyproject.toml
* Update build guide with CUDA installation note
Added note about CUDA requirement for mooncake-transfer-engine.
* Update build.md
Geonwoo Choi