* [Store] Add lock-free MmapArena allocator for buffer mmap path
Replace per-allocation mmap() syscalls in allocate_buffer_mmap_memory()
with a lock-free atomic bump allocator (MmapArena). Pre-allocates a
configurable pool (default 64GB) and serves allocations via CAS loop,
reducing allocation latency from ~1us (mmap syscall) to ~50ns (atomic).
Allocation lifecycle is static: all callers (ClientBufferAllocator,
global segments in RealClient::setup_internal) allocate at startup and
free at shutdown. The arena outlives all allocations, so the bump-only
(no individual free) design is correct for this usage pattern.
Feature-flagged via gflags:
--use_mmap_arena_allocator (default: true)
--mmap_arena_pool_size (default: 64GB)
Falls back to direct mmap() when arena is disabled, fails to init,
or is exhausted.
Cherry-picked from flow-ipc-poc branch (utils.cpp perf path only).
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* [Store] Fix three correctness issues in MmapArena
1. Honor caller's alignment contract: allocate() now accepts a
per-call alignment parameter and uses max(arena default, caller
request). allocate_buffer_mmap_memory() forwards its alignment
argument to the arena. Previously, the caller's alignment was
silently ignored — the arena always used 64-byte alignment
regardless of what the caller requested.
2. Remove MAP_POPULATE from arena pool mmap: the default pool is
64GB but callers typically use only a fraction (e.g. 4GB of
segments). MAP_POPULATE would pre-fault all 64GB of pages upfront,
causing seconds of startup delay and potentially triggering OOM
on machines with less physical memory. Pages now fault on demand.
3. Make alignment_ atomic and store it BEFORE the CAS on pool_base_:
previously alignment_ was a plain size_t written AFTER the release
CAS, so the store was not in the happens-before relationship
established by the acquire-release pair on pool_base_. Now both
alignment_ and pool_size_ are stored before the CAS with the
release fence guaranteeing their visibility to readers.
---------
Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
* feat(tent): improve RailMonitor recovery with exponential backoff and
configurable parameters
- Introduce exponential backoff cooldown with a configurable upper bound
(kMaxCooldown)
- Support dynamic configuration of error threshold, error window, and cooldown
via Config
- Call markRecovered on successful transfer completion to un-pause rails
promptly
- Reset error count and cooldown on recovery to prevent accumulated doubling
- Cache target_machine_id in RdmaSlice to avoid segment lookup on hot path
- Add unit tests for recovery behavior, cooldown reset, and best-device mapping
Signed-off-by: staryxchen <staryxchen@tencent.com>
* test(engine): add end-to-end failover tests and test-only transport swap hook
- Add`swapTransportForTest`method to`TransferEngineImpl`
for test-only transport injection.
- Introduce`engine_failover_e2e_test.cpp`
with scenarios: status corruption failover, budget exhaustion, mixed faults,
per-task independence, and boundary conditions for
`max_failover_attempts`.
- Register new test target`tent_engine_failover_e2e_test`in CMakeLists.
Signed-off-by: staryxchen <staryxchen@tencent.com>
* docs(tent): add failover design doc
Describe TENT's two-layer failure handling: cross-transport failover in
TransferEngineImpl and per-rail cooldown recovery in RailMonitor.
Covers fault model, state machines (with code refs), config knobs,
observability, testing, and known gaps (submit-stage failover, cooldown
reset on recovery, no CI coverage).
Signed-off-by: staryxchen <staryxchen@tencent.com>
* refactor(tent): cache RailMonitor pointer on slice to avoid hot-path string lookup
Each RdmaSlice carried the target machine_id as std::string and
asyncPollCq did a hash+strcmp lookup on worker.rails per completion.
Resolve the RailMonitor once during generatePostPath and stash the
pointer on the slice; the completion path becomes a single deref.
WorkerContext::rails now stores values via unique_ptr so a rehash
only moves the pointer slot and does not invalidate pointers already
held by in-flight slices.
disableEndpoint no longer needs a segment lookup to mark the rail
failed either -- it uses the cached pointer.
Addresses gemini-code-assist review on PR #1984.
Signed-off-by: staryxchen <staryxchen@tencent.com>
---------
Signed-off-by: staryxchen <staryxchen@tencent.com>
* [TENT][Sunrise] Add sunrise_link transport, platform support, and UT coverage
Integrate Sunrise platform/transport wiring across TENT runtime and examples, add SunriseLink end-to-end unit tests, and fix RDMA error logging pointer formatting to avoid crash during registration failure paths.
Made-with: Cursor
* [CI] Update pre-commit hook versions
Bump pre-commit hook revisions to current releases so local checks and CI use newer lint/format toolchains consistently.
Made-with: Cursor
* [TENT][Sunrise] Resolve review issues and drop stale bench target
Address review feedback in SunriseLink transport/platform paths (stream/device context, registration map synchronization, safer probe/allocator handling, and cache-refresh strategy), and remove the obsolete transfer_engine_sunrise_bench CMake target now that its source no longer exists.
* [TENT][sunrise] add engnish doc for sunrise_link
* [TENT][sunrise] restore the memory free logic in the bench, and restore config files
---------
Co-authored-by: liujialai <liujialai@sunrise-ai.com>
* [Store] add tensor object metadata and TP upsert APIs
Introduce explicit tensor object metadata for tensor read/write paths, add TP-aware tensor upsert wrappers, and update tests/docs for the new serialized layout.
---------
Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
* [transfer_engine] feat: make RDMA QP pkey_index configurable via MC_PKEY_INDEX
Previously the QP attr.pkey_index was hardcoded to 0 during connection
setup, which prevented use of non-default partition keys. Add a
pkey_index field to GlobalConfig (default 0) that can be overridden
through the MC_PKEY_INDEX environment variable, and apply it when
transitioning the QP to INIT state.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* patch
* add test and also update doc
* Use try catch blocks instead
---------
Co-authored-by: Yan Huang <yan.huang@Yan-Huangs-MacBook-Pro.local>
Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* [transfer_engine] fix: add periodic endpoint reclaim from monitorWorker
reclaimEndpoint() is currently invoked only from RdmaContext::endpoint()
after a new insertion. Under healthy load, insertions and evictions are
1:1 so this works. Under failure load -- many error completions trigger
deleteEndpoint(), but new-insertion traffic stalls because the dead peer
isn't generating new connection paths -- waiting_list_ grows without
bound and QPs never get destroyed.
Add a 1Hz reclaimEndpoints() call from monitorWorker on the existing
1-second context heartbeat. This decouples reclaim cadence from
insertion traffic.
See issue #1845.
* [transfer_engine] test: endpoint_store reclaim coverage for #1845
Adds unit + integration coverage for the periodic reclaim fix.
endpoint_store_test (5 tests, no RDMA device, runs under ctest):
- reclaim drains quiescent entries on its own
- reclaim leaves active entries alone (gate preserved)
- reclaim is idempotent when empty
- leak manifests without reclaim call (1118-eviction mirror of reporter)
- reclaim works without active map (guard against insert/reclaim coupling)
endpoint_store_integration_test (requires RDMA device, not auto-registered):
- Verifies WorkerPool::monitorWorker actually calls reclaimEndpoints at
~1 Hz by constructing a real RdmaContext and waiting for the tick to
drain injected entries. Confirms the end-to-end fix wiring.
Supporting changes:
- EndpointStore::waitingListSize() accessor (diagnostics + tests)
- SIEVEEndpointStore::testOnlyInsertWaiting() for test injection
- RdmaContext::endpointStore() accessor (diagnostics + tests)
* [docs] note periodic reclaim behavior and #1845 symptom
- design/transfer-engine: add a sentence to Endpoint Management explaining
that waiting_list_ drains both on insertion and on the monitorWorker
heartbeat, so accumulated reclaim does not stall under failure load.
- troubleshooting: extend the "Failed to create QP: Cannot allocate
memory" entry with a bullet pointing at issue #1845 so operators
seeing the symptom find the cause and the fix.
* [transfer_engine] fix: guard FIFOEndpointStore::waitingListSize with atomic counter
Per PR #1952 review: FIFO variant returned waiting_list_.size() on
std::unordered_set without holding endpoint_map_lock_, racing
concurrent modification. Mirror the SIEVE pattern with an atomic
waiting_list_len_ incremented in delete/evict, decremented in reclaim.
* [transfer_engine] test: suppress intentional RdmaTransport leak under LSAN
CI build (3.10/3.12) runs with -DENABLE_ASAN=ON and LSAN flagged the
5 × 288 byte allocation the test fixture intentionally leaks
(~RdmaTransport dereferences a null metadata_ unless install() ran).
Gate on __SANITIZE_ADDRESS__ / __has_feature and mark the pointer with
__lsan_ignore_object so real leaks are still caught.
* [transfer_engine] fix: widen waiting_list_len_ atomic to size_t
waitingListSize() returns size_t but the underlying counter was atomic<int>,
which quietly narrowed on load. Promote to atomic<size_t> in both FIFO and
SIEVE so the getter is a clean pass-through with no implicit conversion.
* [transfer_engine] docs: pin reclaimEndpoint lock contract on base interface
monitorWorker now calls reclaimEndpoint() via RdmaContext; it already
acquired endpoint_map_lock_ internally, but nothing declared that. Codify
the precondition on the base so future callers know not to hold the lock.
RWSpinlock is non-reentrant, so recursive acquisition would deadlock.
* [transfer_engine] refactor: narrow RdmaContext endpoint store test surface
Previously exposed a raw EndpointStore* via RdmaContext::endpointStore()
for the integration test. A raw pointer is easy to misuse outside of
tests and couples the caller to the concrete store via dynamic_cast.
Replace with two narrow methods on RdmaContext: waitingListSize() (value
return) and testOnlyInsertWaiting(shared_ptr<RdmaEndPoint>). The latter
is lifted onto the EndpointStore base interface and implemented on both
FIFO and SIEVE, so the integration test no longer downcasts.
* [transfer_engine] test: register endpoint_store_integration_test with ctest
Integration test was previously unregistered and invoked manually. Now
self-skips via GTEST_SKIP when no RDMA device is present, so it runs
cleanly on CI runners without RDMA (skips) and on rxe/mlx5 hosts
(executes). Labeled "rdma" for ctest -L filtering.
* [transfer_engine] perf: short-circuit FIFO reclaim when waiting list is empty
monitorWorker now drives reclaim at ~1 Hz regardless of activity. On FIFO
this grabbed endpoint_map_lock_ as WriteGuard every tick even in the
common steady-state case where waiting_list_ is empty. Add the same
counter-check short-circuit SIEVE already has.
* [transfer_engine] test: skip integration test when RdmaContext::construct fails
GHA ubuntu-22.04 runners enumerate a phantom mlx5_0 via ibv_get_device_list
without a working port/GID, so pickRdmaDevice() returns a non-empty name
and the earlier GTEST_SKIP on empty device list doesn't fire. Then
construct() fails with ERR_CONTEXT and the hard ASSERT_EQ fails the test.
Convert the assertion to a GTEST_SKIP on construct failure. Matches the
"attempt setup, skip on failure" convention used elsewhere in the repo
(e.g., client_local_hot_cache_test.cpp:794-799).
* feat(efa): idempotent warmupSegment for repeated calls
Adds EfaContext::peekEndpoint() — a non-creating, normalized-key lookup
— and uses it in EfaTransport::warmupSegment() to short-circuit when
every (local_ctx, peer_nic) pair is already connected. Repeated warmup
calls on the same segment now return immediately instead of firing a
256-thread std::async fan-out each time.
Verified on p6-B300 (16 NICs × 16 peer NICs = 256 endpoints):
iter 0: 7.95s (full handshake)
iter 1: 0.000s (short-circuit hit)
iter 2: 0.000s (short-circuit hit)
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
* feat(efa): evict-on-ENOMEM + drop-on-failure for endpoint store
Recover from QP exhaustion (fi_enable returns -FI_ENOMEM at the device
cap, 768/device on p6-B300) and prevent dead endpoints from piling up
under callers that drift endpoint keys (e.g. keys carrying a timestamp).
Three places:
* EfaContext::endpoint (slow path): on construct failure, evict stale
endpoints once and retry. Eviction only scans when we actually ran
out of room, so normal traffic pays nothing.
* EfaTransport::warmupSegment: if setupConnectionsByActive fails, drop
the endpoint via deleteEndpoint(normalizedKey) instead of leaving a
dead fid_ep holding its QP slot.
* EfaContext::submitPostSend: if submitPostSend returns non-zero and
the endpoint is no longer connected, drop it for the same reason.
Reproduced on B300: without the fix, injecting ":drift<iter>" into the
peer_nic_path exhausts fi_enable at iter 48 (768/16 peer NICs). With
the fix, 325 iterations complete with RSS bounded at ~6 GB.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
* feat(efa): SRD shared-endpoint refactor — drop per-peer fid_ep model
Under SRD (FI_EP_RDM), a single fid_ep per local NIC can address any number
of peers via fi_av_insert. The old model created 16 per-peer endpoints per
connection (one per local NIC), consuming 16 QPs each — hitting the 768 QP
cap after only 48 peers and forcing an LRU + evict-on-ENOMEM dance.
This change:
- EfaContext owns a single shared fid_ep + peer address vector (`peer_map_`)
- EfaEndPoint becomes a thin per-peer fi_addr_t holder (no fi_endpoint,
no fi_enable) — handshake is RPC + fi_av_insert only
- submitPostSend delegates to EfaContext::submitSlicesOnPeer, using the
peer's AV index against the shared endpoint
- setPeerNicPath detaches stale AV slot on peer reconnect so drift scenarios
re-handshake cleanly
- Deletes EfaEndpointStore + LRU/ENOMEM eviction machinery — no longer needed
- warmupSegment short-circuit updated for the shared-endpoint flow
P5EN drift stress (1 target ↔ 1 initiator, customer shape 328 GB, 16 NIC,
180s target lifetime):
- iter 0 (fresh peer) first_batch: 33.6 ms (was ~8.95 s → 266× faster)
- iter 1–3 (drift) first_batch: 260–340 ms (was ~8.95 s → ~26× faster)
- steady: ~215–220 GB/s, 0 failures across 204 M completed ops
- QP growth per added peer: 0 (was 16) — no more 48-peer cliff
See mooncake-transfer-engine/example/efa_srd_refactor_validation/ for raw
CSV and target-teardown timings.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
* feat(efa): expose warmup_efa_segment in Python binding
The C API (warmupEfaSegment) and Rust FFI (warmup_efa_segment) already
pre-connect every (local_ctx, peer_nic) pair so the first submitTransfer
does not stall on handshake RPC + fi_av_insert. The pybind TransferEngine
did not expose this, so Python-driven integrations (vLLM / SGLang) could
not benefit from the first-call latency improvement from the SRD shared-
endpoint refactor (iter 0 first_batch 33.6 ms vs 8.95 s baseline).
Mirror the C API shape: thin pass-through that dynamic_casts the installed
"efa" transport and calls EfaTransport::warmupSegment. No-op on non-EFA
builds or when the EFA transport is not installed.
Tested: built with USE_EFA=ON on p5en, imported engine.TransferEngine, and
confirmed `warmup_efa_segment(segment_name: str) -> int` is bound.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
* chore(efa): apply code_format.sh (clang-format-20)
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
* chore(efa): drop internal drift_stress/validation dirs + stale gflags doc note
- Remove mooncake-transfer-engine/example/drift_stress/ — depends on
the out-of-tree customer_pattern Rust binary and isn't usable by
upstream contributors.
- Remove mooncake-transfer-engine/example/efa_srd_refactor_validation/
— validation artifacts for the SRD refactor; the headline numbers
live in the PR description instead.
- Remove the libgflags-dev note from docs/.../efa_transport.md — it
is pulled in transitively by dependencies.sh and no longer needs a
manual install step.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
* fix(efa): close shared endpoint before dropping peer map in teardown
Calling fi_av_remove() on peer addresses after the shared fid_ep was
still open — or calling it at all during context teardown when the AV
itself is about to be fi_av_close()d — trips a libfabric EFA-provider
assertion, causing the efa_transport_test unit tests to segfault on
engine destruction.
Reorder EfaContext::deconstruct() to close shared_ep_ first, then
detach every peer handle via a new markDetachedForTeardown() that just
resets the AV-slot field without calling fi_av_remove(). fi_av_close()
below invalidates every slot in one shot, so no per-slot removal is
required on the teardown path.
Verified: all 5 EFATransportTest cases (InstallTransport, LoopbackWrite,
WriteAndRead, MultiWrite, StressMultipleBatches) now pass on P5EN.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
* test(efa): add coverage for warmupSegment, batch register, large xfer, re-open
Five additional EFATransportTest cases, filling gaps in the existing
suite (which only exercised install + 4 single-buffer write/read paths):
- WarmupSegmentLoopback — covers EfaTransport::warmupSegment() and
its idempotent short-circuit on the second call, plus the empty /
self-name no-op path.
- WarmupSegmentNotFound — warmupSegment must fail cleanly (not hang)
for an unknown segment name.
- RegisterMemoryBatch — registerLocalMemoryBatch +
unregisterLocalMemoryBatch round-trip.
- LargeTransfer — 128 MB buffer, 64 x 1 MB slices; exercises
WR / CQ pacing in EfaContext::submitSlicesOnPeer well past the
16 x 64 KB MultiWrite ceiling.
- RepeatedOpenSegment — openSegment() on the same peer repeatedly
must keep transferring correctly; guards the setPeerNicPath detach
path that target-restart drift depends on.
docs/.../efa_transport.md's Unit Tests table updated to match.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
* docs(efa): restrict CTest invocation to the efa tests
Running the full ctest suite on an EFA-only host trips etcd /
mooncake_master requirements in the TCP / metadata / store tests, which
can read as "EFA broke the build". The EFA doc should not recommend
that invocation — filter to the two EFA binaries and call out the
reason in a note.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
* docs(efa): drop redundant MC_METADATA_SERVER / MC_LOCAL_SERVER_NAME block
The defaults (P2PHANDSHAKE and 127.0.0.1:12345) are what the loopback
unit tests actually want, and nothing in the current test suite needs
an override, so documenting these "env var exports" next to the test
invocation just makes readers wonder whether they are mandatory.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
* docs(efa): drop obsolete CUDA_VISIBLE_DEVICES tip for CPU-to-CPU runs
The tip existed because libfabric's EFA provider used to dlopen
libcudart at fi_getinfo time, creating a CUDA context even for
CPU-only benchmarks. EfaContext::construct now sets FI_HMEM=system
when the build is not GPU-enabled, which already prevents that
context from being created — the CUDA_VISIBLE_DEVICES="" workaround
is no longer needed.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
* docs(efa): lead with a bench command that hits line rate
The previous "Initiator Node" snippet used block_size=65536 /
threads=8 / buf=1GB, which reaches only ~50 GB/s on an 8x400G host
and reads as "EFA is broken" to anyone copy-pasting it. Replace with
block=1MB / threads=32 / buf=4GB — the same values that produce the
GB/s numbers quoted in the Benchmark Results tables — and add a short
sentence telling readers exactly why block_size matters. Also clarify
that the MC_SLICE_SIZE no-effect note covers both CPU-to-CPU and
GPU-to-GPU paths (verified: no EFA code reads globalConfig().slice_size,
only rdma_transport and kunpeng_transport do).
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
* docs(efa): drop MC_SLICE_SIZE references, sync target --buffer_size
EFA transport never reads globalConfig().slice_size — the only readers
are rdma_transport and kunpeng_transport — so the env-var table row,
the "MC_SLICE_SIZE has no effect" tuning-tip bullet, and the
pre-optimization tuning-history details block were all documentation
debt rather than user guidance. Drop all three.
Also: the target command defaulted to --buffer_size=1GB (the flag's
default), but the recommended initiator command now uses 4GB. Since
the target allocates and registers exactly FLAGS_buffer_size bytes and
the initiator writes into offsets [0, buffer_size), they have to
match. Add --buffer_size=4294967296 to the target snippet and a short
line calling out the constraint.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
* fix(efa): skip preTouchMemory for GPU buffers, clarify target flags
preTouchMemory() does a CPU-side store per page, which was gated only
by "chunk >= 4 GiB". That trips a segfault the moment a user registers
a >=4 GiB cudaMalloc buffer — exactly what the recommended benchmark
config does. Restrict the pre-touch to host (cpu:*) memory so VRAM
registration stays a pure libfabric call.
Also amend the benchmark docs:
- call out that --use_vram / --gpu_id apply to the target too, and
that GPU-to-GPU runs need --gpu_id=-1 on both sides
- mention --use_vram=false as the alternative to a -DUSE_CUDA=OFF
build for CPU-to-CPU runs
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
* docs(efa): default benchmark snippet to gpu_id=-1 + threads=16
Align the benchmark quick-start with what actually reaches line rate
on a p5en.48xlarge pair (≈350 GB/s write, verified). The previous
snippet recommended --threads=32 and no --gpu_id override, which:
- ran on one GPU (default gpu_id=0), so buffers sat on one NUMA
node and half the NICs were cross-NUMA — settled at ≈56 GB/s
- ran into the SRD shared-endpoint WR cap (16 × 256 = 4096 WRs
vs 32 × 128 = 4096 outstanding slices, zero headroom) and
triggered "timed out waiting for CQ drain"
Rework the section: lead with the two knobs that matter (--gpu_id=-1
on both sides, --block_size=1048576), add --gpu_id=-1 to the target
snippet for symmetry, switch the initiator to --threads=16, and
explain in a note why 16 is the ceiling (so readers know where the
limit comes from, not just that we picked it).
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
* docs(efa): complete the Key Parameters table
The table was missing several flags that the bench snippet in the
same section actually uses (--mode, --protocol, --metadata_server,
--segment_id, --use_vram), and some of the flags it did list had
defaults / descriptions that did not match the binary (e.g. default
operation is read, not write). Sync every row to DEFINE_* in
transfer_engine_bench.cpp and add explicit "what this is for on EFA"
hints next to the two flags (--block_size, --gpu_id) that most
determine whether the run hits line rate.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
* docs(efa): reorder benchmark sections by peak throughput
Lead with p6-b300 (752 GB/s peak), then p5en (347 GB/s), then
p6-b200 (313 GB/s). Readers skim the first table to decide whether
EFA transport is fast enough for them; putting the best-performing
platform first avoids undersell.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
* docs(efa): reorder benchmarks by hardware generation, not raw GB/s
Previous reorder put p5en (H200) above p6-b200 (B200) because the
p5en rows happen to show a higher headline number, but B200 is a
newer generation than H200 and the p6-b200 numbers there are an
older snapshot. Fix the ordering to reflect hardware recency:
p6-b300 → p6-b200 → p5en. Same change applied to the
Cross-Transport Comparison table.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
* docs(efa): refresh p5en bench results (366 GB/s write / 304 GB/s read)
Re-swept on P5EN (H200, 16×200G) with the SRD shared-endpoint build
and --buffer_size=4GB per GPU. New peaks:
- Write: 365.66 GB/s @ threads=16, batch=128 (up from 347 GB/s)
- Read: 303.78 GB/s @ threads=16, batch=32 (up from 308 GB/s,
but now with a clear optimal config)
The new table preserves enough rows to show the two interesting
axes — write wants big batches, read wants small ones — and the
Cross-Transport Comparison is updated to 366 GB/s (~91% of the
400 GB/s line rate). B200/B300 rows are unchanged (no hardware
available to re-sweep).
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
* docs(efa): mark B200/B300 bench rows stale, refresh p5en CPU, drop cross-transport table
- p5en CPU-to-CPU re-swept on current SRD shared-endpoint build:
peak write 213.57 GB/s @ threads=48/batch=32, peak read 212.18 GB/s
@ threads=16/batch=32 (vs old 192 GB/s / 182 GB/s). Table shows a
few representative points — throughput is essentially flat across
the parameter grid because the run is DRAM-bound.
- B200 and B300 sections tagged as predating the SRD refactor and the
recent tuning work; a re-sweep is pending hardware availability.
- Drop the "Cross-Transport Comparison" subsection and the
"EFA vs RoCE RDMA" paragraph — both were derived numbers that
quickly rot and were already redundant with the per-platform
tables above them.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
* docs(efa): add block_size + buffer_size sweep tables as evidence
Two sweeps on p5en GPU-to-GPU (SRD shared-endpoint build, --gpu_id=-1)
at the peak config (threads=16, batch=128), backing up the tuning
claims elsewhere in this file:
- block_size: 64 KB default reaches only 26% of peak. Write
throughput climbs steeply through 512 KB, plateaus between 1 MB
and 2 MB. 2 MB is MC_EFA_STRIPING_THRESHOLD (a different code
path). 1 MB is the recommended value — within 4% of the 2 MB
peak and safely below the striping boundary.
- buffer_size: only needs to clear block × batch × threads (2 GB
in the peak config). 2 GB vs 4 GB differs by ~3% on write, read
is flat within noise. The example commands use 4 GB purely as
a generous default, not because smaller fails.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
* docs(efa): rewrite Tuning Tips around the SRD shared-endpoint model
Two stale claims under Tuning Tips were written for the old per-peer
endpoint code:
- "Increase threads to 32-48": under the shared endpoint each NIC
has a 256 WR cap, and threads × batch past 16×256 = 4096 trips
the CQ-drain timeout. The sweep on p5en shows 16 threads is the
ceiling, not the floor.
- "Write peaks at threads=32, read at threads=16 / buffer=2GB max
per GPU": both taken from the pre-refactor B300 snapshot.
Current p5en peaks are both at threads=16; the 2GB/GPU limit
was the VRAM preTouch segfault that is now fixed.
Rewrite the bullets to describe what the SRD shared-endpoint
actually constrains: block_size sweet spot, threads × batch WR
cap, the write-vs-read batch-size split, and --gpu_id=-1 on both
sides. Add an explicit line that buffer_size only needs to clear
block × batch × threads (pointing readers at the sweep table).
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
* refactor(efa): remove broken per-request striping across NICs
submitTransferTask had a branch that, when request.length exceeded
MC_EFA_STRIPING_THRESHOLD (default 2 MB), split the request into
num_nics slices and dispatched one per NIC. The intent was to speed
up large single transfers by paralleling across all NICs.
A sweep on p5en (SRD shared-endpoint build, 16 NICs x 200G) shows
the branch is a ~20x negative optimization in every bench config:
block=1 MB, threads=16, batch=128: ON 16 GB/s vs OFF 366 GB/s
block=8 MB, threads=16, batch=16: ON 18 GB/s vs OFF 355 GB/s
block=32 MB, threads=16, batch=4: ON 18 GB/s vs OFF 328 GB/s
Even the designed-for scenario (threads=1, batch=1, single huge
request) only wins by 1.2x (27 vs 23 GB/s) — far below the 16x
expected from parallel dispatch, because the per-slice post_lock /
peer lookup still serialize the dispatch. And that "win" is only
reachable with threads=1 — any realistic caller with multiple
threads can parallelize on its own without this mechanism.
Since the branch silently degrades every common bench / workload
configuration above the 2 MB threshold and delivers no real win
even in its best case, delete it:
- efa_transport.cpp: drop the LARGE TRANSFER if-branch and the
kStripingThreshold read.
- config.h / config.cpp: drop the efa_striping_threshold field
and the MC_EFA_STRIPING_THRESHOLD env var.
- docs/.../efa_transport.md: drop the Environment Variable row,
the "Note on EFA slicing" striping explanation, and the
"striping off" callouts in the benchmark tables and tuning
tips.
Verified: all 10 EFATransportTest cases pass, and the peak p5en
bench run is unchanged (352.81 GB/s write @ threads=16, batch=128).
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
* docs(efa): number the sequential subsections for easier skimming
Four groups of level-3 / level-4 headings in this doc read as
step-by-step instructions (do A first, then B), but only two of
them were numbered while the others relied on order alone. Result:
readers had to count positions to tell "how far through" they were.
Number every such group consistently:
- Performance Benchmark: Target Node (1) / Initiator Node (2)
- Usage with vLLM: Prefill Instance (1) / Decode Instance (2)
- Benchmark Results: p6-b300 (1) / p6-b200 (2) / p5en (3)
Reference-style sections (Key Parameters, Tuning Tips, Warmup,
Technical Details, Troubleshooting) stay unnumbered — those aren't
ordered and numbering them would imply a reading path that
doesn't exist.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
* test(efa): add efa_first_submit_probe for warmup / first-submit timing
A standalone cross-node probe that measures, separately:
- warmupSegment() latency, and
- per-submit latency for the first N single-transfer submits
so we can isolate the handshake / av_insert cost (warmup=OFF makes
it land on submit #0) and compare pre-refactor vs post-refactor
code without having to squint at 10s throughput averages.
Not enabled by default ctest — only built when USE_EFA=ON. Meant
for manual two-host runs alongside transfer_engine_bench.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
* docs(efa): replace warmup numbers with a cross-branch p5en measurement
The old "Eager endpoint warmup" numbers (4 s stall / 13.5 ms warmed)
were from a B300 snapshot that predated the SRD shared-endpoint
refactor, so they described the OLD code. Replace with a fresh
cross-node measurement on p5en that compares both branches head to
head using efa_first_submit_probe:
- Warmup itself: 1.1 s (SRD shared endpoint) vs 17 s (per-peer
fid_ep on upstream main) — ~15x faster and much more stable
(SRD runs were all 1.13-1.14 s; old code swung 9-17 s across
three reps).
- Cold first submit with NO warmup: 26 ms (SRD) vs 99 ms (old) —
~4x faster, because the shared endpoint removes the per-peer
fi_endpoint / fi_enable step that used to dominate the first
send.
The prior text's "first-batch stall" framing also reads as if the
whole first submitTransfer always pays the full handshake cost.
That's not quite how it works — the initiator round-robins across
local NICs, so only a handful of pairs warm on each call, and the
stall amortizes across the first several submits. Fix the wording.
Add a pointer to the probe source (example/efa_first_submit_probe.cpp)
so the measurement is reproducible.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
* docs(efa): reframe "first-request latency" around two audiences
The previous version led with a long explanation of warmupSegment and
its internals, which made it sound like callers had to opt in to get
any first-request speedup from this PR. That isn't the case: the SRD
shared-endpoint refactor cuts the cold submit from 99 ms to 26 ms
automatically, and vLLM / SGLang (which don't call warmupSegment) see
that 4x win without any integration work on their side. The explicit
warmupSegment API is an additional 15x improvement on top of that,
but it matters mainly to direct Mooncake callers.
Restructure the section accordingly:
- Lead with a 2x2 table showing both numbers side by side.
- Frame the ~4x cold-submit win as the default, no-code-change
benefit everyone gets.
- Frame warmupSegment as the opt-in path for callers that want
sub-10 ms first-request latency.
- Explicitly note that vLLM / SGLang currently don't call it.
- Keep the peer handshake-daemon bottleneck note so readers
understand why the warmup scales linearly.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
* docs(efa): reference #1944 explicitly instead of "this PR"
"This PR" dangles once the doc lands on main — it's fine in review
context, less fine afterwards. Use the concrete issue number so
readers (including future us) can follow the link.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
* docs(efa): unwrap soft-wrapped paragraphs, let the renderer flow them
Several blockquote notes, tuning-tip bullets, and the "first-request
latency" section used ~70-column soft wraps, which look fine rendered
but make the raw markdown look ragged — the line after 'Peer
addressing resolves lazily:' used to stop before the line was
actually full, which reads as a formatting bug in plain text. Let
each logical paragraph / bullet be a single line and defer wrapping
to the browser / sphinx renderer. No rendered change.
Also update the PR body on #1944 the same way.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
* docs(efa): fix post-refactor stale references (architecture, thread safety, comparison)
Four places still described the pre-#1944 code:
1. docs/.../efa_transport.md "EFA Transport Architecture" diagram —
showed fid_ep inside EfaEndpoint, which is wrong under the
shared-endpoint model. fid_ep lives on EfaContext (shared_ep_);
EfaEndPoint only holds the peer's fi_addr_t AV slot. Redraw.
2. docs/.../efa_transport.md "Thread Safety" — said the transport
"adds per-endpoint spinlocks" (plural, per-endpoint). In the
new code there's exactly one post_lock_ per EfaContext guarding
the shared fid_ep. Reword.
3. docs/.../efa_transport.md "EFA vs RoCE RDMA" comparison rows —
used the pre-refresh p5en GPU number (347 GB/s) and the stale
B200 row (313 GB/s). Replace with the current p5en peak from
the benchmark tables above (365 GB/s GPU, 213 GB/s CPU), drop
the row pointing at a now-absent B200 re-sweep.
4. efa_context.cpp:589 — "Fast path: peer info pre-resolved by
submitTransferTask's striping path". The striping path was
removed in an earlier commit in this PR; the fast path now
just catches slices whose peer_nic_path the caller already
filled in. Reword without referencing the removed mechanism.
No behavioral change.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
* refactor(efa): address pr review — drop unused params, sysconf once, drop friend-class, expand probe comment
Addresses trivial review feedback on #1944:
- EfaContext::construct() no longer takes num_comp_channels / port /
gid_index. These three parameters were plumbed through from the
original RDMA-era signature but the EFA/libfabric path never read
them; the previous commit just silenced warnings with (void) casts.
Remove them from the signature and the one caller in
EfaTransport::installSglangTransport.
- preTouchMemory() was calling sysconf(_SC_PAGESIZE) twice (once
for the >0 check and once for the value). Cache the result in a
local.
- `friend class EfaEndPoint` in efa_context.h was unnecessary —
EfaEndPoint only touches public methods on EfaContext
(nicPath, localEpAddr, engine, insertPeerAddr, removePeerAddr,
submitSlicesOnPeer). Drop the friend decl.
- Expand the header comment on efa_first_submit_probe.cpp to say
what problem it exists to measure and why transfer_engine_bench
alone isn't sufficient — reviewer asked what the example is for.
No behavioral change.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
* perf(efa): faster hex<->binary AV address handling
Reviewer pointed out two sources of overhead in the handshake path:
1. localEpAddr() built the hex string via std::ostringstream with
std::hex / std::setw / std::setfill, which pulls in stream
formatting for every byte. Replace with a direct table-driven
encoder writing into a pre-sized std::string.
2. insertPeerAddr() decoded the hex using substr() + strtol() for
every byte (a temporary std::string and a libc base-16 parse per
pair). Replace with a single pass using a hexNibble() helper and
a pre-sized buffer.
3. Loopback called insertPeerAddr(localEpAddr()), encoding the
bytes to hex and then immediately decoding back. Expose the raw
bytes via localEpAddrBytes() and add an insertPeerAddrBytes()
overload; loopback now skips the round-trip entirely.
Handshake path is not the dominant cost in steady-state serving, but
warmupSegment() fires 256 of these on a fresh 16x16 topology and
every one of them was doing the extra work. The new path also avoids
the per-byte std::string allocation (strtol path).
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
* refactor(efa): use std::atomic for wr_depth / cq outstanding, fix O(N^2) erase
Replace `volatile int` + __sync_* builtins with std::atomic<int> on the
shared-endpoint pacing counters (EfaContext::wr_depth_ and
EfaCq::outstanding). `volatile` never implied atomicity under the C++
memory model, and mixing it with __sync builtins was technically UB; the
atomic version is both portable and more explicit about the intended
memory ordering (acq_rel on the mutating ops, relaxed on the speculative
load used for CAS).
Also fix an O(N^2) batch consumption pattern in submitSlicesOnPeer:
`slice_list.erase(begin, begin + batch_count)` shifted the tail on every
iteration. Consume via a moving cursor instead, and splice retry slices
in place when -FI_EAGAIN forces a partial post.
Behavior is unchanged on the happy path; the retry path now preserves
the original slice order at the cursor, matching the previous insert-at-
front semantics.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
* docs(efa): drop stale efa_latency_bench.{py,png}, document efa_first_submit_probe
The `efa_latency_bench.py` script and its `efa_latency_bench.png` output
shipped with the original EFA transport PR (#1509) two years ago. No
markdown doc references them, the script hard-codes old private IPs as
defaults, and nothing guarantees the chart is still representative.
Remove both.
Expand the First-request latency section of efa_transport.md to actually
document `efa_first_submit_probe`: what it measures (cold-submit +
eager-warmup cost, which `transfer_engine_bench`'s 10s average hides),
how to run it (target / initiator commands), the expected output layout,
all flags, and the two situations where it is worth running (deciding
whether your app needs `warmupSegment()`, and comparing PR branches on
the same hardware). Marked it explicitly as EFA-specific — RDMA/TCP
transports do not have an equivalent warmup concept, so folding this into
`transfer_engine_bench` would muddy the generic benchmark for no gain.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 4.7 <noreply@anthropic.com>
Co-authored-by: EC2 Default User <ec2-user@ip-172-31-8-212.us-west-1.compute.internal>
* feat(efa): auto-split large MR registrations exceeding max_mr_size
Buffers larger than the EFA device's max_mr_size are now transparently
split into chunks, each registered as a separate MR. This fixes the
silent truncation bug where only the first max_mr_size bytes were
registered, causing transfers to unregistered regions to fail at runtime.
Key changes:
- Query EFA device max_mr_size via ibverbs during init (libfabric does
not expose this) and clamp globalConfig accordingly
- Auto-split buffers > (max_mr_size - 1GB) into chunks in
registerLocalMemoryInternal, each with its own BufferDesc metadata
- Track chunk mappings for proper cleanup in unregisterLocalMemory
- Change lkey()/rkey() from exact-match to range lookup (matching
mrDesc() pattern) so key lookups work for any address within a chunk
- Replace silent truncation in EfaContext with a hard error as safety net
- Remove preTouchMemory truncation to touch the full buffer
Tested on P5EN (16 EFA NIC, max_mr_size=192GB): 191GB single-chunk
registration succeeds. Auto-split triggers correctly for larger buffers
(200GB splits into 191GB + 9GB). Total registerable size per buffer is
bounded by system pinned_vm limits (~191GB on 16-NIC P5EN).
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* feat(efa): per-NIC partition for large buffer MR registration
Each EFA NIC can only register up to max_mr_size total. The previous
auto-split approach registered every chunk on ALL NICs, hitting the
per-NIC limit for buffers > max_mr_size. This change assigns each
chunk to a disjoint subset of NICs, enabling registration of buffers
up to max_mr_size × num_NICs (e.g. ~1.5TB on P5EN with 16 NICs).
Key changes:
- chunk_limit = max_mr_size / 2 (was max_mr_size - 1GB) for headroom
- NIC assignment: chunks distributed evenly across available NICs
- selectDevice(): skips NICs with rkey=0 (unassigned for that chunk)
- Striping path: filters by lkey!=0 to avoid unregistered NICs
- Unregister: only deregisters from assigned NICs per chunk
- ChunkRegistration struct tracks per-chunk NIC assignments
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* feat(efa): PTE-aware auto-split replaces conservative max_mr_size/2 threshold
The previous chunk_limit of max_mr_size/2 (~96GB on P5EN) caused unnecessary
buffer splitting even when hugepages were available. This detects the actual
backing page size via /proc/self/smaps and computes the PTE-based limit:
- 4KB pages: 22M PTEs × 4KB = 88GB (genuine hardware constraint)
- 2MB hugepages: 22M PTEs × 2MB = 44TB (effectively max_mr_size)
Verified on P5EN (H200, 16 EFA): 100GB pool with hugepages no longer splits,
restoring full 16-NIC throughput (108 GB/s vs 46 GB/s with the old threshold).
MR registration time dropped from 302s to 1.6s.
Adds MC_EFA_MAX_PTE_ENTRIES env var for override (default 22M).
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* feat(efa): add KV cache prefix transfer benchmark script
Python benchmark to measure EFA transfer performance for LLM prefix
cache hit scenarios. Tests different pool sizes (10GB-500GB) and prefix
lengths (4K-32K tokens) to evaluate per-NIC partition impact on
transfer latency and throughput.
Default KV bytes/token matches GLM-5.1 (754B MoE, MLA attention):
(kv_lora_rank=512 + qk_rope_head_dim=64) * 2 * 78 layers = 88KB/token
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* fix(bench): add connection warmup before benchmark loop
The first prefix size's measurements were skewed by EFA connection
establishment (openSegment, endpoint creation). Add 3 small transfers
before entering the benchmark loop to warm up the connection.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* fix(bench): per-offset warmup and p50 throughput reporting
Warmup now exercises all offsets the benchmark will measure, eliminating
first-access TLB/page-fault outliers (4K token p99 dropped from 80ms to
3.5ms). Throughput is reported from p50 latency instead of avg for more
stable numbers.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* feat(bench): add --threads option for concurrent prefix transfer
Adds optional multi-threaded transfer support. Each thread transfers a
chunk of the prefix in parallel via separate transfer_sync_read calls.
Default is 1 (single transfer, same as before). Testing shows threads=2
matches single-thread throughput (~108 GB/s), while higher values add
scheduling overhead with no benefit since EFA already stripes internally.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* feat(efa): full NIC coverage for multi-chunk MR registration
When a buffer exceeds max_mr_size and must be split into multiple chunks,
register every chunk on ALL NICs instead of disjoint per-NIC partition,
as long as total PTE usage per NIC fits within the PTE budget.
With hugepages (2MB), 500GB buffer uses only 250K PTE/NIC (budget: 24M),
so all 16 NICs cover every address. Falls back to disjoint partition when
PTE budget is exceeded (e.g. 4KB pages with large buffers).
500GB pool throughput: 35 GB/s (disjoint, 5-6 NIC) → 108 GB/s (full, 16 NIC).
Registration time unchanged (~9.5s) due to parallel MR registration.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* feat(bench): add batch memory registration benchmark
Test script for registering multiple independent memory blocks
(e.g. multi-tenant KV cache pools). Supports both per-block
register_memory and batch_register_memory APIs. Target mode
allocates and registers N blocks; initiator mode transfers
data and measures throughput.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* style: apply clang-format to EFA transport files
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* feat(efa): expose discoverTopology C API and Rust bindings
Add discoverTopology() to the C API and discover_topology()/install_transport()
to the Rust bindings, enabling EFA transport initialization from C/Rust callers.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* test(efa): add C API test for discoverTopology and EFA transport
Verify that the new discoverTopology() C API correctly populates the
device list, enabling installTransport("efa") and memory registration
via the pure C interface (used by Rust/Go bindings).
All 4 tests passed on p5en.48xlarge (16 EFA NICs).
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* fix(efa): O(log n) MR lookup and chunk registration rollback
- Replace unordered_map with std::map for mr_map_ and use upper_bound
for O(log n) range lookups in rkey/lkey/mrDesc instead of O(n) scan
- Add rollbackChunks lambda to unregister already-registered chunks on
failure in registerLocalMemoryInternal, preventing MR leaks
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* style: apply clang-format to efa_c_api_test.cpp
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* fix: rename thr_tag to thread_tag to pass typos spell check
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* fix(efa): round-robin multi-chunk MR and large MR registration test
When a buffer splits into more chunks than available NICs, round-robin
assign chunks across NICs with per-NIC PTE budget validation, instead
of hard-failing with "Buffer requires N chunks but only M NICs".
Add efa_single_nic_large_mr_test: tests single-NIC and all-NIC large
MR registration with hugepages. Supports --chunk_gb for multi-buffer
mode (e.g. 200×2GB).
Verified on P5EN (16 EFA NIC, 2MB hugepages):
- 1 NIC × 200×2GB (400GB): 53.7s
- 16 NICs × 200×2GB (400GB): 64.8s
- 1 NIC × 200GB single buffer: 2.7s
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* feat(efa): add cross-node transfer test for multi-buffer MR registration
Tests 200x2GB buffer registration on all NICs with actual data transfer
between two P5EN nodes. Supports target/initiator modes with single-read
and multi-block batch benchmarks.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* fix(efa): avoid CUDA context leak when built without GPU support
libfabric 2.4's EFA provider dlopens libcudart/libcuda during
fi_getinfo/fi_domain to detect HMEM support, even though the caller
never touches GPU memory. This creates a CUDA primary context on
GPU 0 and permanently holds ~616 MiB of device memory.
When Mooncake is built with USE_CUDA=OFF (and USE_HIP=OFF), set
FI_HMEM=system before fi_getinfo and drop FI_MR_HMEM from the domain
hints so the provider skips GPU hmem initialization entirely.
* feat(efa): multi-thread initiator and wildcard location for 16-NIC coverage
Register target buffers with wildcard location "*" instead of "cpu:0"
so initiator-side remote NIC selection distributes evenly across all 16
NICs (both NUMA nodes). With "cpu:0", selectDevice only picked NUMA-0's
8 NICs, leaving NUMA-1 idle — throughput capped at ~107 GB/s instead of
~149 GB/s on P5EN (16×EFA 200Gbps).
Also convert the initiator from single-threaded to multi-threaded
(--threads flag) and print the actual P2P handshake address.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* docs(efa): add p6-b300 bandwidth results (752 GB/s GPU, 230 GB/s CPU)
- GPU-to-GPU peak 752 GB/s write / 713 GB/s read at ~94% line rate
(16×400 Gbps = 800 GB/s theoretical)
- CPU-to-CPU peak 230 GB/s, DRAM-limited on Xeon 8559C
(NUMA-0 NICs 90 Gbps vs NUMA-1 NICs 53 Gbps)
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
* feat(rust): customer_pattern sample for per-NUMA multi-MR registration
Adds a Rust binary mirroring the customer's EFA usage pattern: target
registers many MRs per NUMA (e.g. 16 NICs x N x 10GB via cpu:<numa>
locations); initiator reads/writes a specific (numa, buffer_index)
tuple, using --source-numa to pick which local NIC set is exercised.
Also makes rust/build.rs robust against non-default transfer_engine
build configurations: etcd-cpp-api is opt-in via MOONCAKE_WITH_ETCD=1,
CUDA linking is opt-in via MOONCAKE_WITH_CUDA=1 with CUDART_LIB_DIR
override, and libbase.a + libasio.so + libfabric paths are picked up
whether the repo uses the standalone or top-level CMake layout.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
* chore(rust): drop customer_pattern sample from upstream
Revert Cargo.toml [[bin]] additions and remove the bench demo source so
the public tree no longer ships it. Keep it gitignored locally so it can
be iterated on without accidental re-adds.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
* feat(efa): eager endpoint warmup for segments to remove first-batch stall
libfabric FI_EP_RDM endpoints resolve peer addresses lazily — the first
submitTransfer() to a new segment serializes fi_av_insert + handshake over
every (local_ctx, peer_nic) pair. On 16-NIC instances this is a ~4 s stall
on the first 100 × 0.5 MB batch (measured on p6-B300).
Add EfaTransport::warmupSegment(name) that pre-connects all pairs
concurrently via std::async, plus C wrapper warmupEfaSegment() (guarded by
USE_EFA) and Rust binding TransferEngine::warmup_efa_segment(). Idempotent,
safe to re-run after metadata changes. RDMA/TCP paths untouched.
Measured on p6-B300 (16 × 16 endpoints, dual-NUMA initiator):
first-batch: 4043 ms -> 13.5 ms (~300x)
steady-state: 141 GB/s -> 230 GB/s
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
* fix(efa): enable auto-split when max_mr_size is not configured
When MC_EFA_MAX_MR_SIZE is unset, max_mr was 0 and chunk_limit collapsed
to 0, bypassing the PTE-aware split entirely. Large 4KB-paged buffers
then hit the per-NIC PTE ceiling at registration time.
Fall back to pte_limit so splitting kicks in based on the PTE budget
alone when no explicit max_mr_size is provided.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
* style: apply clang-format to EFA transport/tests and PEP8 to kvcache bench
Pure formatting changes to satisfy CI format hook (clang-format-20) and
address Gemini review comment on kvcache_prefix_bench.py indentation.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
Co-authored-by: Ubuntu <ubuntu@ip-172-31-13-185.us-east-2.compute.internal>
Co-authored-by: whn09 <whn09@github.com>
* fix(efa): add fi_read support and endpoint eviction to EFA transport
1. EFA submitPostSend() now branches on slice->opcode to call fi_read
or fi_write, matching RDMA transport behavior. Previously only
fi_write was implemented, breaking remote-read workloads on EFA.
2. EfaEndpointStore now tracks endpoint activity and evicts stale
endpoints when approaching AV capacity. This prevents AV exhaustion
in long-running services communicating with many transient peers.
- Endpoints marked active on access, inactive on set_active(false)
- Configurable inactive timeout (default 5 min)
- evictStale() removes endpoints inactive beyond timeout
- removeDisconnected() cleans up broken connections
- Eviction triggered automatically in getOrInsert() at capacity
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* perf(efa): eliminate per-slice overhead with NIC-count-based striping
Replace fixed 64KB slicing with NIC-count-based chunking in EFA
submitTransferTask. For large transfers (>128KB), creates one slice
per active NIC instead of thousands of small slices, dramatically
reducing spinlock, atomic, and allocation overhead.
- Large transfers: stripe across all active NICs (1 chunk per NIC)
- Small transfers: single slice on one NIC, no sub-slicing
- Fallback: per-slice retry for edge cases (unregistered memory)
Example: 240MB transfer on 32 NICs now creates 32 slices vs 3840,
matching NIXL approach of one fi_write per rail per descriptor.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* fix(efa): add FI_MR_HMEM to mr_mode hints for GPU memory registration
EFA provider requires FI_MR_HMEM in domain mr_mode hints to support
heterogeneous (GPU) memory registration via fi_mr_reg. Without this
flag, fi_mr_reg fails with EFAULT (Bad address) when registering
CUDA device memory buffers.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* style(efa): apply clang-format-20 formatting
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* fix(efa): restore lost changes from merge — endpoint eviction, batched WR, docs
Restores 5 commits that were lost during the upstream merge:
- AV entry removal on disconnect to prevent target degradation
- Endpoint reuse for same physical peer (normalizeNicPath)
- Batched WR submission in submitPostSend
- Pre-resolved remote peer info for striped transfers
- Documentation: GPU-to-GPU (313 GB/s), CPU-to-CPU (222 GB/s) benchmarks,
CPU-only build instructions, B300/P5 instance types
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* docs(efa): use dependencies.sh for build deps, add EFA-specific extras
Replace manual package list with dependencies.sh script reference.
Add libgflags-dev as EFA-specific extra dependency.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* docs(efa): add tip to use CUDA_VISIBLE_DEVICES for CPU-to-CPU benchmarks
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* docs(efa): add DLAMI troubleshooting for CUDA env and Python activation
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* docs(efa): add LIBRARY_PATH for CUDA libs, remove unneeded CUDAToolkit_ROOT
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* docs(efa): add block_size=1MB to all benchmark configurations
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* docs(efa): merge clone step into build dependencies section
dependencies.sh already runs git submodule update --init --recursive,
so the separate clone section was redundant. Move git clone into the
Prerequisites section and renumber build steps.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* docs(efa): add SGLang usage, Docker troubleshooting, reorder sections
- Move Usage with vLLM/SGLang after benchmark results (was too early)
- Add Usage with SGLang section (EFA patch, env vars, Docker example)
- Add libfabric version mismatch in Docker troubleshooting
- Reorder p5en results: GPU-to-GPU before CPU-to-CPU (consistent with B200)
- Update p5en single-instance CPU results: Write 179 GB/s, Read 185 GB/s
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* feat(efa): use GlobalConfig max_ep_per_ctx (65536) as default, round-robin CQ assignment
- Change EfaEndpointStore and EfaContext::construct default max_endpoints
from 256 to 65536, consistent with GlobalConfig.max_ep_per_ctx
- Add round-robin CQ assignment (nextCq) for new endpoints instead of
always using cq_list_[0], distributing load across CQs
- Pass CQ outstanding counter directly to endpoint construct
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* style(efa): clang-format, move striping threshold to GlobalConfig
- Run clang-format on all EFA transport files and common.h
- Move MC_EFA_STRIPING_THRESHOLD from inline std::getenv to GlobalConfig
mechanism (loaded centrally in loadGlobalConfig), per reviewer feedback
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* ci: add 'wqs' to typos ignore list (EFA API function name)
efadv_query_qp_wqs is a libfabric EFA API symbol, not a typo.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* feat(efa): add per-transfer latency benchmark script
Automated script that measures single-transfer latency across block sizes
by SSHing into target/initiator nodes and running transfer_engine_bench
with threads=1, batch_size=1.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* feat(efa): add --threads and --batch_size args to latency bench script
Allow configurable threads/batch_size for multi-thread bandwidth scaling tests.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* feat(efa): add --env flag to pass env vars to remote bench via SSH
Allows setting MC_EFA_STRIPING_THRESHOLD and other env vars on the
remote initiator without manual SSH config.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* ci: fix typos check by adding wqs to extend-words
The extend-ignore-words key is not recognized by typos v1.30.2 used
in CI. Add wqs (EFA API function name efadv_query_qp_wqs) to the
[default.extend-words] section which is the correct format.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* style: revert formatting-only changes in common.h
Revert pointer/reference style changes (`&`/`*` placement) that were
introduced by clang-format but are unnecessary for this PR. Keep only
the normalizeNicPath() function addition.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* docs(efa): clarify that MC_SLICE_SIZE does not apply to EFA transport
EFA transport uses NIC-count-based striping (since commit 84d7dc5),
not fixed-size slicing. block_size directly determines per fi_write/
fi_read size for transfers below the striping threshold. Update docs
to reflect this and correct outdated tuning advice.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* fix(efa): address review issues - nextCq guard, error path double-free, GPU MR registration
1. nextCq(): add empty guard to prevent division-by-zero when cq_list_ is empty
2. submitTransferTask error paths: replace getSliceCache().deallocate(s) with
s->markFailed() to prevent double-free (TransferTask destructor also deallocates)
and ensure tasks complete with FAILED status instead of hanging
3. registerMemoryRegionInternal(): use fi_mr_regattr() with explicit FI_HMEM_CUDA
iface and device ordinal for GPU memory, since EFA provider fi_mr_reg()
hardcodes iface=FI_HMEM_SYSTEM
4. Make total_bytes updates atomic (__sync_fetch_and_add) for thread safety
Tested: unit tests (5/5), cross-node GPU benchmark (write 250 GB/s, read 269 GB/s)
on p5en.48xlarge.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* style(efa): apply clang-format to efa_context.cpp
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
Co-authored-by: EC2 Default User <ec2-user@ip-172-31-8-212.us-west-1.compute.internal>
Co-authored-by: whn09 <whn09@github.com>
* [Store] Add hard pin mechanism for eviction-protected objects
Objects created with ReplicateConfig.with_hard_pin=true are never
evicted by the eviction policy, providing guaranteed persistence
for model weights in RL and model management workloads.
Changes:
- ReplicateConfig: add with_hard_pin field (default false)
- ObjectMetadata: add hard_pinned boolean, set at creation via PutStart
- BatchEvict: skip hard-pinned objects in all eviction passes
- Serialization: persist hard_pinned in snapshots (backward compatible
with old format that lacks the field)
- Tests: verify hard-pinned objects survive eviction, coexist with
soft pin, and can still be explicitly removed
When building with USE_EFA=ON, auto_discover is disabled to prevent
RDMA transport installation (QP creation fails on EFA devices). This
means TCP transport is also not installed automatically. Add explicit
TCP transport installation for non-EFA protocols in the EFA build path.
Documentation changes:
- build.md: Add USE_EFA option and clarify USE_CUDA default/purpose
- supported-protocols.md: Add EFA as a supported protocol
- efa_transport.md: Add USE_CUDA=ON to build command, document GPU
memory requirement
Co-authored-by: whn09 <whn09@users.noreply.github.com>
Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
Co-authored-by: Teng Ma <teng-ma@linux.alibaba.com>
* add indexer api design doc
* remove some api & add explain info for tiered storage
* rename tired storage level name
* add overview architecture
* define the input of indexer kvevent
* add example for indexer api output
* fix some issue from code reviews
* add explain for different medium
* delete the confusing note
* add more Introduction for indexer output value
* change some code based on the code review
* use medium name directly
* fix some format problem
* support multi-tenant & cache salt and rename engine_name
* [TE] Add AWS EFA transport using libfabric
Add EfaTransport as a new transport backend for AWS Elastic Fabric
Adapter (EFA) devices. EFA exposes RDMA-like NICs but does not support
the full ibverbs QP API, so this transport uses libfabric's FI_EP_RDM
(Reliable Datagram Message) endpoint type instead.
Architecture (per EFA device):
EfaTransport → EfaContext → EfaEndPoint
- EfaContext: owns fabric/domain/AV/CQ/MR resources
- EfaEndPoint: one RDM endpoint per peer, with address-vector addressing
- Dedicated CQ poller thread per device for responsive completion draining
Key design decisions:
- FI_THREAD_SAFE requested from provider; per-endpoint spinlock on
fi_write as safety net for concurrent submission threads
- Atomic CAS reservation of CQ and WR capacity before posting fi_write
to prevent CQ overflow under high concurrency
- CQ error path drains all queued errors (fi_cq_readerr loop) before
returning, per libfabric semantics
- Retry-with-backoff on CQ/WR full instead of immediate slice failure
- Thread-safe endpoint creation via atomic getOrInsert to prevent
duplicate endpoints for the same peer
- Handshake exchanges EFA endpoint addresses via dedicated efa_addr
field in HandShakeDesc
Build: cmake -DUSE_EFA=ON (requires libfabric from AWS EFA installer)
Tested on p6-b200.48xlarge (8 EFA devices, 8×400 Gbps): 59.72 GB/s
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* [TE] Add EFA unit tests and bench tool support
Add efa_transport_test with 5 test cases:
- InstallTransport: verify EFA transport installation
- LoopbackWrite: basic loopback write operation
- WriteAndRead: write then read with data integrity check
- MultiWrite: batch write (16 requests)
- StressMultipleBatches: stress test (20 batches × 8 requests)
Add --protocol=efa support to transfer_engine_bench with manual
topology discovery (EFA needs explicit discover() since
TransferEngine(false) skips auto-discovery).
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* [Docs] Add EFA transport documentation
Add comprehensive EFA transport documentation covering:
- Prerequisites and build instructions
- Usage with vLLM (prefill/decode disaggregation)
- Unit test descriptions and environment variables
- Benchmark results on p6-b200.48xlarge: 59.72 GB/s (EFA) vs
9.5 GB/s (TCP iperf3) vs 0.11 GB/s (Mooncake TCP transport)
- EFA vs RoCE RDMA comparison table
- Thread safety design notes
- Troubleshooting guide
Add EfaTransport to the transfer-engine index toctree and supported
transport lists.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* [TE] Address PR review: ifdef EFA fields, use find_package for libfabric
- Wrap efa_addr in HandShakeDesc with #ifdef USE_EFA in transfer_metadata.h
- Wrap efa_addr serialization/deserialization with #ifdef USE_EFA in transfer_metadata.cpp
- Replace hardcoded /opt/amazon/efa paths with find_path/find_library in common.cmake
- Remove redundant hardcoded EFA paths from all CMakeLists.txt files
- Fix git clone URL in efa-transport.md to use official kvcache-ai/Mooncake repo
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* [Docs] Update EFA benchmark results with tuned parameters (170 GB/s)
Update benchmark documentation with comprehensive parameter tuning results
from cross-machine testing on p6-b200.48xlarge instances. Key finding:
MC_SLICE_SIZE=262144 nearly doubles EFA throughput from ~70 to ~170 GB/s,
reaching 88% of RoCE RDMA performance.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* Fix clang-format violation in transfer_metadata.h
Remove extra space before comment on efa_addr field to satisfy
clang-format-20 style check.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* [Docs] Add EFA latency benchmark script, rename efa doc to underscore
- Add efa_latency_bench.py: automated benchmark script that measures
EFA throughput for tuned/default configs via SSH and plots
Latency vs Cache Size chart
- Add efa_latency_bench.png: benchmark results chart
- Rename efa-transport.md -> efa_transport.md to match naming
convention of other transport docs (ascend_transport.md, etc.)
- Update toctree reference in index.md
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
---------
Co-authored-by: Ubuntu <ubuntu@ip-172-31-25-79.us-east-2.compute.internal>
Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
Co-authored-by: Ubuntu <ubuntu@ip-172-31-22-204.us-east-2.compute.internal>
* [Store]: add task executor feature with unit and executor test
Signed-off-by: Vincent Gao <vincentbo@linux.alibaba.com>
* [Store]: add some optimizations to task executor
Refactor the task_executor into the client_service and add a
task structure on the client side. Additionally, remove the
existence check in the execute function;
only retrying should be performed if replica allocation fails.
Signed-off-by: Vincent Gao <vincentbo@linux.alibaba.com>
* [Store]: get source replica from copyStart or moveStart api
* [Store]: call move or copy end if the target replica already exist to complete the replication task
* [Store]: use the max_retry_attempts in master side
* [Store]: add client integration test and set default max_retry_attempts to 10
* [Doc] update task api introduction
Signed-off-by: Vincent Gao <vincentbo@linux.alibaba.com>
* [Store] Set copy and move as private methods
Signed-off-by: Vincent Gao <vincentbo@linux.alibaba.com>
* [Doc]: change the default task max_retry_attempts to 10
* [Doc]: fix some description error
* [Store]: allocate the buffer size to be a multiple of 16MB
* [Store]: validate the replica is in local and directly construct slices from replica buffer address instead of copy the data to local buffer
* [Store]: change the validate logic to directly use transfer engine endpoint or local_hostname_.
* [Store] add e2e ci test for copy and move api
* [Store] refactor the client move and copy function
* [Store] fix the e2e test
* [Store] remove unused code
* [Store] add source field when build replica copy payload in the task_manager_test
* [Store] rename back to snake case for split_into_slices function and also remove hard code for client poll count
* [Store] revert mis deleted field when resolve conflicts
* [Store] change test to validate the real behaviour
* [Store] change the default task fetch size to 16
* [Doc]: change the replica copy/move sequence diagram
* [Store] add new split_into_slice method
* [Store] change the real client to use split_to_slice with buffer handle parameters
---------
Signed-off-by: Vincent Gao <vincentbo@linux.alibaba.com>
Co-authored-by: Vincent Gao <vincentbo@linux.alibaba.com>
Add MC_HANDSHAKE_MAX_LENGTH environment variable to configure the maximum
handshake message length in P2P mode.
## Problem
When using P2P handshake mode with a single RDMA instance that registers
many memory buffers (>10,000), the serialized segment metadata JSON can
exceed the hardcoded 1MB limit, causing handshake failures with error:
"readString: too large length from socket: <length>"
Each registered buffer adds ~96 bytes to the JSON payload:
- 1,000 buffers ≈ 94KB
- 5,000 buffers ≈ 469KB
- 10,000 buffers ≈ 938KB (near 1MB limit)
- 15,000 buffers ≈ 1.37MB (exceeds limit)
## Solution
- Add getHandshakeMaxLength() function that reads MC_HANDSHAKE_MAX_LENGTH
- Value is in bytes, valid range: 1MB to 128MB
- Default remains 1MB (1048576 bytes) for backward compatibility
- Logs custom value when set, warns on invalid values
## Usage
```bash
# Set to 4MB to support ~40,000 buffers
export MC_HANDSHAKE_MAX_LENGTH=4194304
```
* Add early mem backend detection method in NVLINK_allocator
Add early detection method for sglang NVLINK_allocator to avoid CuMemCreate
Use enumerate type to indicate mem backend type
format check use pre-commit
* Change enumerate type in allocator.py for different mem backend
* Isolate nvlink intraNode transport from nvlink_transport and modify corresponding transfer_engine_bench
* IntraNode transport isolation to be compatible with SGlang
* Modify the code style using pre-commit check
* isolate intraNode nvlink from previous nvlink_transport
* [TE] Revert to previous nvlink_transport
* Delete extra log and refine the code format
* Discard revert due to new related PR has been submitted
* Change the Code format to align with main branch
* Change to clang-format
* Modify transfer_engine_bench to be compatible with protocol --nvlink_intra
* Delete useless function in intranode_nvlink.cpp
* Used for rerun CI
* Modify the intraNode isolation to be compatible with transfer_bench and transfer_engine_impl
* isolate intraNode header file from mnnvl.h
* Delete specific instaince type for intraNode
---------
Co-authored-by: 百麒 <yaozhong.lyz@alibaba-inc.com>
* feat(metrics): add TENT metrics system with HTTP server and Prometheus
integration
- Add comprehensive metrics system based on yalantinglibs for monitoring data
transfer performance
- Implement HTTP server with endpoints for Prometheus, JSON, and human-readable
metrics
- Add compile-time and runtime performance optimization with zero-overhead when
disabled
- Integrate metrics into TransferEngine with automatic latency tracking
- Add configuration loader supporting config files and environment variables
- Include example application demonstrating metrics usage
- Add documentation for metrics system configuration and usage
Signed-off-by: staryxchen <staryxchen@tencent.com>
* Update docs/source/design/tent/metrics.md
Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
* refactor(metrics): simplify config loading with explicit priority
- Replace indirect environment config loading with direct parsing
- Implement clear priority: file config > environment variables > defaults
- Add validation for environment variable values
- Remove redundant default value comparisons
Signed-off-by: staryxchen <staryxchen@tencent.com>
* refactor(transfer_engine): extract metrics recording logic into dedicated method
- Add recordTaskCompletionMetrics method to TransferEngineImpl class
- Replace duplicate metrics recording code in getTransferStatus methods with
calls to new method
- Centralize task completion metrics logic for better maintainability
Signed-off-by: staryxchen <staryxchen@tencent.com>
* build(metrics): improve yalantinglibs dependency handling
- Change warning to fatal error when TENT_METRICS_ENABLED is ON but
yalantinglibs is missing
- Provide clearer warning message when metrics are disabled
Signed-off-by: staryxchen <staryxchen@tencent.com>
* refactor(metrics): replace manual JSON construction with nlohmann/json library
- Use nlohmann/json for cleaner and more maintainable JSON serialization
- Remove manual string stream manipulation and formatting
- Improve code readability and reduce error-prone manual concatenation
Signed-off-by: staryxchen <staryxchen@tencent.com>
* style: reformat code with clang-format
Signed-off-by: staryxchen <staryxchen@tencent.com>
* refactor(config): centralize parsing utilities in ConfigHelper
- Move parsing functions from MetricsConfigLoader to ConfigHelper
- Add applyEnvironmentOverrides method to reduce code duplication
- Update includes and comments to reflect new structure
Signed-off-by: staryxchen <staryxchen@tencent.com>
* test: add unit tests for metrics config loader and reorganize test structure
- Move examples directory to tests directory in CMakeLists.txt
- Add comprehensive unit tests for MetricsConfigLoader functionality
- Include tests for config parsing, environment variable loading, and validation
- Rename and relocate tent_metrics_example.cpp to tests directory
Signed-off-by: staryxchen <staryxchen@tencent.com>
* style: reformat code lines for better readability
Signed-off-by: staryxchen <staryxchen@tencent.com>
* fix(build): remove redundant Asio dependency from metrics CMakeLists
- Remove Asio dependency search and linking as yalantinglibs bundles it
internally
- Add clarifying comment about bundled Asio in yalantinglibs
Signed-off-by: staryxchen <staryxchen@tencent.com>
---------
Signed-off-by: staryxchen <staryxchen@tencent.com>
Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
* [Store] add HugePage support
Co-authored-by: Teng Ma <sima.mt@alibaba-inc.com>
Signed-off-by: Xingrui Yi <yixingrui@linux.alibaba.com>
* doc: add huge page env introduction
Signed-off-by: Xingrui Yi <yixingrui@linux.alibaba.com>
---------
Signed-off-by: Xingrui Yi <yixingrui@linux.alibaba.com>
Co-authored-by: Teng Ma <sima.mt@alibaba-inc.com>
* add MC_FORCE_HCA environment variable to force use rdma
* Updater for better readability
Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
* fix clang-format problem
---------
Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
* [TE]: Add HIP transport for AMD GPUs support
Forked from nvlink_transport and adapted for HIP/AMD GPUs.
* [TE/HIP] Addressed review comments
* [TE] Move NVLINK and HIP common functions to common files
* [TE] Fix incorrect length assignment in relocateSharedMemoryAddress
Use entry.length instead of length parameter when storing OpenedShmEntry.
The length parameter represents the requested transfer length, while
entry.length represents the actual buffer's full length, which is the
correct value to store and is consistent with openShareableHandle usage.
* add PCIe Relaxed Ordering (RO) support.
* fix: add env variable to control Relaxed Ordering (RO)
* refactor: simplify logic in mooncake-transfer-engine/src/transport/rdma_transport/rdma_transport.cpp
Co-authored-by: Teng Ma <teng-ma@linux.alibaba.com>
* fix(RO): set default value to 0
* fix(ci): resolve compilation error in PR build
* Supplement guidance documents
* refactor: remove dlopen, use dlsym alone for symbol checking
---------
Co-authored-by: Teng Ma <teng-ma@linux.alibaba.com>
* [Store] feat: Add BatchReplicaClear API for manual cache cleanup
Implement the `BatchReplicaClear` API to allow clients to manually clear
cached object replicas. This feature enables explicit storage management
by deleting replicas for specified keys. The implementation is end-to-end,
from the client to the master service.
Signed-off-by: Vincent Gao <vincentbo@linux.alibaba.com>
* [Store] feat: Fix BatchReplicaClear monitor issues
Fixed issues with monitoring metrics not being updated and function naming.
Fix the issue of unclear descriptions in the file.
Signed-off-by: Vincent Gao <vincentbo@linux.alibaba.com>
---------
Signed-off-by: Vincent Gao <vincentbo@linux.alibaba.com>
Dmitry Barsukoff