* [Store] Add native registered buffer pool
Add a native Python registered buffer pool for reusable zero-copy scratch buffers, with lease lifetime checks and docs/tests for the public API.
---------
Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
* Initial plan
* Add MC_RDMA_BIND_ADDRESS support for dual-NIC P2PHANDSHAKE setups
In dual-NIC environments where TCP and RDMA use separate interfaces,
P2PHANDSHAKE mode previously required using a single IP for both
TCP handshake and RDMA NIC paths, causing conflicts.
This change adds MC_RDMA_BIND_ADDRESS env var support:
- When set, RDMA NIC paths use the RDMA-reachable IP
- TCP P2P routing continues using the local_server_name IP
- Segment descriptors carry rdma_server_name for consistent NIC
path construction on both sides
- P2P metadata exchange caches RDMA->TCP address mapping so
subsequent handshakes resolve to TCP-routable addresses
* Changes before error encountered
Agent-Logs-Url: https://github.com/kvcache-ai/Mooncake/sessions/fc2826eb-a0ae-450f-b1f4-4ab94269d97a
* Apply dual-NIC (MC_RDMA_BIND_ADDRESS) support to TENT transport and update Chinese docs
---------
Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com>
Exposes the existing TransferMetadata::sendProbe C++ method through the
TransferEngine pybind module as engine.send_probe(peer_server_name).
This enables SGLang's MooncakeKVManager to issue lightweight JSON-RPC
probes against peers, used to test whether a previously-blacklisted
mooncake_session_id has become reachable again so it can be removed
from the failed_sessions set.
Returns 0 on success, non-zero on failure (matching the C++ contract).
No behavior change for existing engine.* methods.
Tested:
- New Python unit tests in transfer_engine_initiator_test.py covering
both the reachable-peer and unknown-peer cases.
- Manually validated end-to-end against SGLang's MooncakeKVManager.
* [Store] add tensor object metadata and TP upsert APIs
Introduce explicit tensor object metadata for tensor read/write paths, add TP-aware tensor upsert wrappers, and update tests/docs for the new serialized layout.
---------
Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
* [Store] add get_into_range and src_offset plumbing
Add single-key range read support to Mooncake Store:
- get_into_range: read [src_offset, src_offset+size) from an object
into (buffer + dst_offset)
- submitRangeRead in TransferSubmitter with src_offset support
- Get() overload and TransferReadRange in Client
- DummyClient RPC path via get_into_range_dummy_helper
- PyClient virtual interface for get_into_range
Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
* fix: correct TP zero-copy put semantics
* fix: apply clang-format to store binding
* fix: default batch TP-from results to errors
* fix: return success for batch TP put-from
Ensure batch_put_tensor_with_tp_impl reports zero on fully successful shard writes so TP zero-copy Python tests don't fail with false INVALID_PARAMS results.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* tests: align TP zero-copy benchmark with get API signature
Keep the TP zero-copy benchmark focused on put-from semantics by removing an unsupported split_dim argument from batch_get_tensor_with_tp_into.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* trigger ci
---------
Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
Co-authored-by: Xuchun Shang <xuchun.shang@linux.alibaba.com>
* [Store]: add task executor feature with unit and executor test
Signed-off-by: Vincent Gao <vincentbo@linux.alibaba.com>
* [Store]: add some optimizations to task executor
Refactor the task_executor into the client_service and add a
task structure on the client side. Additionally, remove the
existence check in the execute function;
only retrying should be performed if replica allocation fails.
Signed-off-by: Vincent Gao <vincentbo@linux.alibaba.com>
* [Store]: get source replica from copyStart or moveStart api
* [Store]: call move or copy end if the target replica already exist to complete the replication task
* [Store]: use the max_retry_attempts in master side
* [Store]: add client integration test and set default max_retry_attempts to 10
* [Doc] update task api introduction
Signed-off-by: Vincent Gao <vincentbo@linux.alibaba.com>
* [Store] Set copy and move as private methods
Signed-off-by: Vincent Gao <vincentbo@linux.alibaba.com>
* [Doc]: change the default task max_retry_attempts to 10
* [Doc]: fix some description error
* [Store]: allocate the buffer size to be a multiple of 16MB
* [Store]: validate the replica is in local and directly construct slices from replica buffer address instead of copy the data to local buffer
* [Store]: change the validate logic to directly use transfer engine endpoint or local_hostname_.
* [Store] add e2e ci test for copy and move api
* [Store] refactor the client move and copy function
* [Store] fix the e2e test
* [Store] remove unused code
* [Store] add source field when build replica copy payload in the task_manager_test
* [Store] rename back to snake case for split_into_slices function and also remove hard code for client poll count
* [Store] revert mis deleted field when resolve conflicts
* [Store] change test to validate the real behaviour
* [Store] change the default task fetch size to 16
* [Doc]: change the replica copy/move sequence diagram
* [Store] add new split_into_slice method
* [Store] change the real client to use split_to_slice with buffer handle parameters
---------
Signed-off-by: Vincent Gao <vincentbo@linux.alibaba.com>
Co-authored-by: Vincent Gao <vincentbo@linux.alibaba.com>
* feat(metrics): add task completion latency tracking and detailed metrics reporting
- Add histogram metric for tracking transfer task completion latency with
configurable buckets
- Record task start time in submitTransfer and calculate latency upon completion
- Enhance metrics reporting thread to include latency distribution statistics
- Add skip_metrics parameter to getBatchTransferStatus to avoid double counting
- Update task structure to include start_time field for latency calculation
Signed-off-by: staryxchen <staryxchen@tencent.com>
* feat: add metrics support for submitTransferWithNotify
Signed-off-by: staryxchen <staryxchen@tencent.com>
* refactor(metrics): improve metrics recording logic with early returns
- Replace nested if statements with early returns using goto for cleaner flow
- Add boundary checks for task_id and validate start_time before recording
- Only record metrics for COMPLETED status and reset start_time to prevent
duplicates
Signed-off-by: staryxchen <staryxchen@tencent.com>
* refactor(metrics): simplify bucket count tracking logic
- Remove conditional checks for vector bounds by ensuring prev_bucket_counts_
size matches bucket_counts
- Consolidate previous snapshot update into a single assignment
Signed-off-by: staryxchen <staryxchen@tencent.com>
* fix(metrics): conditionally enable metrics collection
- Add metrics_enabled_ flag check before collecting metrics
- Wrap metrics collection logic with flag condition to avoid unnecessary
operations
- Ensure metrics are only recorded when explicitly enabled
Signed-off-by: staryxchen <staryxchen@tencent.com>
* docs: clarify Transfer Engine metrics limitation with TENT
- Add note about MC_TE_METRIC not being supported when using Transfer Engine
TENT in deployment guide
- Add same note to Python API reference for MC_TE_METRIC environment variable
Signed-off-by: staryxchen <staryxchen@tencent.com>
---------
Signed-off-by: staryxchen <staryxchen@tencent.com>
* feat(store): disable auto discovery by default, require rdma_devices for RDMA
* feat(client): support optional auto-discovery with default for RDMA
* update comment
* update doc
* update hello world
* feat(allocator): Add transport endpoint support for segment transfers
* refactor(Client): Replace segment_name with transport_endpoint in buffer descriptors and tests
* refactor(test): switch to in-proc master and remove external service management
* style: format code and adjust transfer task timeout
* feat(Client::MountSegment): set te_endpoint conditionally based on metadata_connstring for P2P mode
* Allow configuring default KV lease TTL via environment variable
Read DEFAULT_KV_LEASE_TTL environment variable at server startup and
parse as uint64_t value. This enables CI tests to override default
configuration without code changes.
Signed-off-by: xiaguan <751080330@qq.com>
* style: format CI config and add [[nodiscard]] to methods
Signed-off-by: xiaguan <751080330@qq.com>
* refactor(tests): remove local_hostname parameter and improve test helpers
The commit removes the unused local_hostname parameter from TransferSubmitter constructor, updates the associated initialization logic, and simplifies test code by introducing helper functions for segment creation. The changes improve code maintainability while preserving functionality.
* refactor(TransferSubmitter): Remove unused local_hostname_ member and validation check
* docs(store): update setup examples for p2p
---------
Signed-off-by: xiaguan <751080330@qq.com>
Cruz Zhao