- Introduce`USE_MLX5DV`CMake option and link against`libmlx5`when enabled
- Add`MC_MLX5_QP_UDP_SPORTS`
environment variable to specify comma-separated UDP source ports for ECMP/LAG
path diversification
- Add`MC_MLX5_QP_LAG_PORT_BALANCE`
environment variable to enable automatic QP distribution across bonded LAG ports
- Update`RdmaContext`to query and expose the number of LAG ports via
`mlx5dv_query_device`
- Implement QP modification logic in`RdmaEndPoint::doSetupConnection`
to apply UDP source port and LAG port pinning
- Extend configuration parsing and logging to support the new mlx5-specific
options
- Document the new runtime options in the design documentation
Signed-off-by: staryxchen <staryxchen@tencent.com>
* [TENT][Sunrise] Add sunrise_link transport, platform support, and UT coverage
Integrate Sunrise platform/transport wiring across TENT runtime and examples, add SunriseLink end-to-end unit tests, and fix RDMA error logging pointer formatting to avoid crash during registration failure paths.
Made-with: Cursor
* [CI] Update pre-commit hook versions
Bump pre-commit hook revisions to current releases so local checks and CI use newer lint/format toolchains consistently.
Made-with: Cursor
* [TENT][Sunrise] Resolve review issues and drop stale bench target
Address review feedback in SunriseLink transport/platform paths (stream/device context, registration map synchronization, safer probe/allocator handling, and cache-refresh strategy), and remove the obsolete transfer_engine_sunrise_bench CMake target now that its source no longer exists.
* [TENT][sunrise] add engnish doc for sunrise_link
* [TENT][sunrise] restore the memory free logic in the bench, and restore config files
---------
Co-authored-by: liujialai <liujialai@sunrise-ai.com>
* [transfer_engine] feat: make RDMA QP pkey_index configurable via MC_PKEY_INDEX
Previously the QP attr.pkey_index was hardcoded to 0 during connection
setup, which prevented use of non-default partition keys. Add a
pkey_index field to GlobalConfig (default 0) that can be overridden
through the MC_PKEY_INDEX environment variable, and apply it when
transitioning the QP to INIT state.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* patch
* add test and also update doc
* Use try catch blocks instead
---------
Co-authored-by: Yan Huang <yan.huang@Yan-Huangs-MacBook-Pro.local>
Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* [transfer_engine] fix: add periodic endpoint reclaim from monitorWorker
reclaimEndpoint() is currently invoked only from RdmaContext::endpoint()
after a new insertion. Under healthy load, insertions and evictions are
1:1 so this works. Under failure load -- many error completions trigger
deleteEndpoint(), but new-insertion traffic stalls because the dead peer
isn't generating new connection paths -- waiting_list_ grows without
bound and QPs never get destroyed.
Add a 1Hz reclaimEndpoints() call from monitorWorker on the existing
1-second context heartbeat. This decouples reclaim cadence from
insertion traffic.
See issue #1845.
* [transfer_engine] test: endpoint_store reclaim coverage for #1845
Adds unit + integration coverage for the periodic reclaim fix.
endpoint_store_test (5 tests, no RDMA device, runs under ctest):
- reclaim drains quiescent entries on its own
- reclaim leaves active entries alone (gate preserved)
- reclaim is idempotent when empty
- leak manifests without reclaim call (1118-eviction mirror of reporter)
- reclaim works without active map (guard against insert/reclaim coupling)
endpoint_store_integration_test (requires RDMA device, not auto-registered):
- Verifies WorkerPool::monitorWorker actually calls reclaimEndpoints at
~1 Hz by constructing a real RdmaContext and waiting for the tick to
drain injected entries. Confirms the end-to-end fix wiring.
Supporting changes:
- EndpointStore::waitingListSize() accessor (diagnostics + tests)
- SIEVEEndpointStore::testOnlyInsertWaiting() for test injection
- RdmaContext::endpointStore() accessor (diagnostics + tests)
* [docs] note periodic reclaim behavior and #1845 symptom
- design/transfer-engine: add a sentence to Endpoint Management explaining
that waiting_list_ drains both on insertion and on the monitorWorker
heartbeat, so accumulated reclaim does not stall under failure load.
- troubleshooting: extend the "Failed to create QP: Cannot allocate
memory" entry with a bullet pointing at issue #1845 so operators
seeing the symptom find the cause and the fix.
* [transfer_engine] fix: guard FIFOEndpointStore::waitingListSize with atomic counter
Per PR #1952 review: FIFO variant returned waiting_list_.size() on
std::unordered_set without holding endpoint_map_lock_, racing
concurrent modification. Mirror the SIEVE pattern with an atomic
waiting_list_len_ incremented in delete/evict, decremented in reclaim.
* [transfer_engine] test: suppress intentional RdmaTransport leak under LSAN
CI build (3.10/3.12) runs with -DENABLE_ASAN=ON and LSAN flagged the
5 × 288 byte allocation the test fixture intentionally leaks
(~RdmaTransport dereferences a null metadata_ unless install() ran).
Gate on __SANITIZE_ADDRESS__ / __has_feature and mark the pointer with
__lsan_ignore_object so real leaks are still caught.
* [transfer_engine] fix: widen waiting_list_len_ atomic to size_t
waitingListSize() returns size_t but the underlying counter was atomic<int>,
which quietly narrowed on load. Promote to atomic<size_t> in both FIFO and
SIEVE so the getter is a clean pass-through with no implicit conversion.
* [transfer_engine] docs: pin reclaimEndpoint lock contract on base interface
monitorWorker now calls reclaimEndpoint() via RdmaContext; it already
acquired endpoint_map_lock_ internally, but nothing declared that. Codify
the precondition on the base so future callers know not to hold the lock.
RWSpinlock is non-reentrant, so recursive acquisition would deadlock.
* [transfer_engine] refactor: narrow RdmaContext endpoint store test surface
Previously exposed a raw EndpointStore* via RdmaContext::endpointStore()
for the integration test. A raw pointer is easy to misuse outside of
tests and couples the caller to the concrete store via dynamic_cast.
Replace with two narrow methods on RdmaContext: waitingListSize() (value
return) and testOnlyInsertWaiting(shared_ptr<RdmaEndPoint>). The latter
is lifted onto the EndpointStore base interface and implemented on both
FIFO and SIEVE, so the integration test no longer downcasts.
* [transfer_engine] test: register endpoint_store_integration_test with ctest
Integration test was previously unregistered and invoked manually. Now
self-skips via GTEST_SKIP when no RDMA device is present, so it runs
cleanly on CI runners without RDMA (skips) and on rxe/mlx5 hosts
(executes). Labeled "rdma" for ctest -L filtering.
* [transfer_engine] perf: short-circuit FIFO reclaim when waiting list is empty
monitorWorker now drives reclaim at ~1 Hz regardless of activity. On FIFO
this grabbed endpoint_map_lock_ as WriteGuard every tick even in the
common steady-state case where waiting_list_ is empty. Add the same
counter-check short-circuit SIEVE already has.
* [transfer_engine] test: skip integration test when RdmaContext::construct fails
GHA ubuntu-22.04 runners enumerate a phantom mlx5_0 via ibv_get_device_list
without a working port/GID, so pickRdmaDevice() returns a non-empty name
and the earlier GTEST_SKIP on empty device list doesn't fire. Then
construct() fails with ERR_CONTEXT and the hard ASSERT_EQ fails the test.
Convert the assertion to a GTEST_SKIP on construct failure. Matches the
"attempt setup, skip on failure" convention used elsewhere in the repo
(e.g., client_local_hot_cache_test.cpp:794-799).
* [TE] Add AWS EFA transport using libfabric
Add EfaTransport as a new transport backend for AWS Elastic Fabric
Adapter (EFA) devices. EFA exposes RDMA-like NICs but does not support
the full ibverbs QP API, so this transport uses libfabric's FI_EP_RDM
(Reliable Datagram Message) endpoint type instead.
Architecture (per EFA device):
EfaTransport → EfaContext → EfaEndPoint
- EfaContext: owns fabric/domain/AV/CQ/MR resources
- EfaEndPoint: one RDM endpoint per peer, with address-vector addressing
- Dedicated CQ poller thread per device for responsive completion draining
Key design decisions:
- FI_THREAD_SAFE requested from provider; per-endpoint spinlock on
fi_write as safety net for concurrent submission threads
- Atomic CAS reservation of CQ and WR capacity before posting fi_write
to prevent CQ overflow under high concurrency
- CQ error path drains all queued errors (fi_cq_readerr loop) before
returning, per libfabric semantics
- Retry-with-backoff on CQ/WR full instead of immediate slice failure
- Thread-safe endpoint creation via atomic getOrInsert to prevent
duplicate endpoints for the same peer
- Handshake exchanges EFA endpoint addresses via dedicated efa_addr
field in HandShakeDesc
Build: cmake -DUSE_EFA=ON (requires libfabric from AWS EFA installer)
Tested on p6-b200.48xlarge (8 EFA devices, 8×400 Gbps): 59.72 GB/s
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* [TE] Add EFA unit tests and bench tool support
Add efa_transport_test with 5 test cases:
- InstallTransport: verify EFA transport installation
- LoopbackWrite: basic loopback write operation
- WriteAndRead: write then read with data integrity check
- MultiWrite: batch write (16 requests)
- StressMultipleBatches: stress test (20 batches × 8 requests)
Add --protocol=efa support to transfer_engine_bench with manual
topology discovery (EFA needs explicit discover() since
TransferEngine(false) skips auto-discovery).
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* [Docs] Add EFA transport documentation
Add comprehensive EFA transport documentation covering:
- Prerequisites and build instructions
- Usage with vLLM (prefill/decode disaggregation)
- Unit test descriptions and environment variables
- Benchmark results on p6-b200.48xlarge: 59.72 GB/s (EFA) vs
9.5 GB/s (TCP iperf3) vs 0.11 GB/s (Mooncake TCP transport)
- EFA vs RoCE RDMA comparison table
- Thread safety design notes
- Troubleshooting guide
Add EfaTransport to the transfer-engine index toctree and supported
transport lists.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* [TE] Address PR review: ifdef EFA fields, use find_package for libfabric
- Wrap efa_addr in HandShakeDesc with #ifdef USE_EFA in transfer_metadata.h
- Wrap efa_addr serialization/deserialization with #ifdef USE_EFA in transfer_metadata.cpp
- Replace hardcoded /opt/amazon/efa paths with find_path/find_library in common.cmake
- Remove redundant hardcoded EFA paths from all CMakeLists.txt files
- Fix git clone URL in efa-transport.md to use official kvcache-ai/Mooncake repo
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* [Docs] Update EFA benchmark results with tuned parameters (170 GB/s)
Update benchmark documentation with comprehensive parameter tuning results
from cross-machine testing on p6-b200.48xlarge instances. Key finding:
MC_SLICE_SIZE=262144 nearly doubles EFA throughput from ~70 to ~170 GB/s,
reaching 88% of RoCE RDMA performance.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* Fix clang-format violation in transfer_metadata.h
Remove extra space before comment on efa_addr field to satisfy
clang-format-20 style check.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* [Docs] Add EFA latency benchmark script, rename efa doc to underscore
- Add efa_latency_bench.py: automated benchmark script that measures
EFA throughput for tuned/default configs via SSH and plots
Latency vs Cache Size chart
- Add efa_latency_bench.png: benchmark results chart
- Rename efa-transport.md -> efa_transport.md to match naming
convention of other transport docs (ascend_transport.md, etc.)
- Update toctree reference in index.md
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
---------
Co-authored-by: Ubuntu <ubuntu@ip-172-31-25-79.us-east-2.compute.internal>
Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
Co-authored-by: Ubuntu <ubuntu@ip-172-31-22-204.us-east-2.compute.internal>
Add MC_HANDSHAKE_MAX_LENGTH environment variable to configure the maximum
handshake message length in P2P mode.
## Problem
When using P2P handshake mode with a single RDMA instance that registers
many memory buffers (>10,000), the serialized segment metadata JSON can
exceed the hardcoded 1MB limit, causing handshake failures with error:
"readString: too large length from socket: <length>"
Each registered buffer adds ~96 bytes to the JSON payload:
- 1,000 buffers ≈ 94KB
- 5,000 buffers ≈ 469KB
- 10,000 buffers ≈ 938KB (near 1MB limit)
- 15,000 buffers ≈ 1.37MB (exceeds limit)
## Solution
- Add getHandshakeMaxLength() function that reads MC_HANDSHAKE_MAX_LENGTH
- Value is in bytes, valid range: 1MB to 128MB
- Default remains 1MB (1048576 bytes) for backward compatibility
- Logs custom value when set, warns on invalid values
## Usage
```bash
# Set to 4MB to support ~40,000 buffers
export MC_HANDSHAKE_MAX_LENGTH=4194304
```
* Add early mem backend detection method in NVLINK_allocator
Add early detection method for sglang NVLINK_allocator to avoid CuMemCreate
Use enumerate type to indicate mem backend type
format check use pre-commit
* Change enumerate type in allocator.py for different mem backend
* Isolate nvlink intraNode transport from nvlink_transport and modify corresponding transfer_engine_bench
* IntraNode transport isolation to be compatible with SGlang
* Modify the code style using pre-commit check
* isolate intraNode nvlink from previous nvlink_transport
* [TE] Revert to previous nvlink_transport
* Delete extra log and refine the code format
* Discard revert due to new related PR has been submitted
* Change the Code format to align with main branch
* Change to clang-format
* Modify transfer_engine_bench to be compatible with protocol --nvlink_intra
* Delete useless function in intranode_nvlink.cpp
* Used for rerun CI
* Modify the intraNode isolation to be compatible with transfer_bench and transfer_engine_impl
* isolate intraNode header file from mnnvl.h
* Delete specific instaince type for intraNode
---------
Co-authored-by: 百麒 <yaozhong.lyz@alibaba-inc.com>
* add MC_FORCE_HCA environment variable to force use rdma
* Updater for better readability
Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
* fix clang-format problem
---------
Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
* [TE]: Add HIP transport for AMD GPUs support
Forked from nvlink_transport and adapted for HIP/AMD GPUs.
* [TE/HIP] Addressed review comments
* [TE] Move NVLINK and HIP common functions to common files
* [TE] Fix incorrect length assignment in relocateSharedMemoryAddress
Use entry.length instead of length parameter when storing OpenedShmEntry.
The length parameter represents the requested transfer length, while
entry.length represents the actual buffer's full length, which is the
correct value to store and is consistent with openShareableHandle usage.
* add PCIe Relaxed Ordering (RO) support.
* fix: add env variable to control Relaxed Ordering (RO)
* refactor: simplify logic in mooncake-transfer-engine/src/transport/rdma_transport/rdma_transport.cpp
Co-authored-by: Teng Ma <teng-ma@linux.alibaba.com>
* fix(RO): set default value to 0
* fix(ci): resolve compilation error in PR build
* Supplement guidance documents
* refactor: remove dlopen, use dlsym alone for symbol checking
---------
Co-authored-by: Teng Ma <teng-ma@linux.alibaba.com>
Aoi