forked from mooncake-track/Mooncake
a6cbc1a417
* fix(efa): add fi_read support and endpoint eviction to EFA transport
1. EFA submitPostSend() now branches on slice->opcode to call fi_read
or fi_write, matching RDMA transport behavior. Previously only
fi_write was implemented, breaking remote-read workloads on EFA.
2. EfaEndpointStore now tracks endpoint activity and evicts stale
endpoints when approaching AV capacity. This prevents AV exhaustion
in long-running services communicating with many transient peers.
- Endpoints marked active on access, inactive on set_active(false)
- Configurable inactive timeout (default 5 min)
- evictStale() removes endpoints inactive beyond timeout
- removeDisconnected() cleans up broken connections
- Eviction triggered automatically in getOrInsert() at capacity
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* perf(efa): eliminate per-slice overhead with NIC-count-based striping
Replace fixed 64KB slicing with NIC-count-based chunking in EFA
submitTransferTask. For large transfers (>128KB), creates one slice
per active NIC instead of thousands of small slices, dramatically
reducing spinlock, atomic, and allocation overhead.
- Large transfers: stripe across all active NICs (1 chunk per NIC)
- Small transfers: single slice on one NIC, no sub-slicing
- Fallback: per-slice retry for edge cases (unregistered memory)
Example: 240MB transfer on 32 NICs now creates 32 slices vs 3840,
matching NIXL approach of one fi_write per rail per descriptor.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* fix(efa): add FI_MR_HMEM to mr_mode hints for GPU memory registration
EFA provider requires FI_MR_HMEM in domain mr_mode hints to support
heterogeneous (GPU) memory registration via fi_mr_reg. Without this
flag, fi_mr_reg fails with EFAULT (Bad address) when registering
CUDA device memory buffers.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* style(efa): apply clang-format-20 formatting
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* fix(efa): restore lost changes from merge — endpoint eviction, batched WR, docs
Restores 5 commits that were lost during the upstream merge:
- AV entry removal on disconnect to prevent target degradation
- Endpoint reuse for same physical peer (normalizeNicPath)
- Batched WR submission in submitPostSend
- Pre-resolved remote peer info for striped transfers
- Documentation: GPU-to-GPU (313 GB/s), CPU-to-CPU (222 GB/s) benchmarks,
CPU-only build instructions, B300/P5 instance types
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* docs(efa): use dependencies.sh for build deps, add EFA-specific extras
Replace manual package list with dependencies.sh script reference.
Add libgflags-dev as EFA-specific extra dependency.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* docs(efa): add tip to use CUDA_VISIBLE_DEVICES for CPU-to-CPU benchmarks
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* docs(efa): add DLAMI troubleshooting for CUDA env and Python activation
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* docs(efa): add LIBRARY_PATH for CUDA libs, remove unneeded CUDAToolkit_ROOT
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* docs(efa): add block_size=1MB to all benchmark configurations
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* docs(efa): merge clone step into build dependencies section
dependencies.sh already runs git submodule update --init --recursive,
so the separate clone section was redundant. Move git clone into the
Prerequisites section and renumber build steps.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* docs(efa): add SGLang usage, Docker troubleshooting, reorder sections
- Move Usage with vLLM/SGLang after benchmark results (was too early)
- Add Usage with SGLang section (EFA patch, env vars, Docker example)
- Add libfabric version mismatch in Docker troubleshooting
- Reorder p5en results: GPU-to-GPU before CPU-to-CPU (consistent with B200)
- Update p5en single-instance CPU results: Write 179 GB/s, Read 185 GB/s
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* feat(efa): use GlobalConfig max_ep_per_ctx (65536) as default, round-robin CQ assignment
- Change EfaEndpointStore and EfaContext::construct default max_endpoints
from 256 to 65536, consistent with GlobalConfig.max_ep_per_ctx
- Add round-robin CQ assignment (nextCq) for new endpoints instead of
always using cq_list_[0], distributing load across CQs
- Pass CQ outstanding counter directly to endpoint construct
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* style(efa): clang-format, move striping threshold to GlobalConfig
- Run clang-format on all EFA transport files and common.h
- Move MC_EFA_STRIPING_THRESHOLD from inline std::getenv to GlobalConfig
mechanism (loaded centrally in loadGlobalConfig), per reviewer feedback
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* ci: add 'wqs' to typos ignore list (EFA API function name)
efadv_query_qp_wqs is a libfabric EFA API symbol, not a typo.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* feat(efa): add per-transfer latency benchmark script
Automated script that measures single-transfer latency across block sizes
by SSHing into target/initiator nodes and running transfer_engine_bench
with threads=1, batch_size=1.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* feat(efa): add --threads and --batch_size args to latency bench script
Allow configurable threads/batch_size for multi-thread bandwidth scaling tests.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* feat(efa): add --env flag to pass env vars to remote bench via SSH
Allows setting MC_EFA_STRIPING_THRESHOLD and other env vars on the
remote initiator without manual SSH config.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* ci: fix typos check by adding wqs to extend-words
The extend-ignore-words key is not recognized by typos v1.30.2 used
in CI. Add wqs (EFA API function name efadv_query_qp_wqs) to the
[default.extend-words] section which is the correct format.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* style: revert formatting-only changes in common.h
Revert pointer/reference style changes (`&`/`*` placement) that were
introduced by clang-format but are unnecessary for this PR. Keep only
the normalizeNicPath() function addition.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* docs(efa): clarify that MC_SLICE_SIZE does not apply to EFA transport
EFA transport uses NIC-count-based striping (since commit
|
||
|---|---|---|
| .. | ||
| source | ||
| Makefile | ||
| README.md | ||
| make.bat | ||
README.md
Mooncake documents
Build the docs
- Make sure in
docsdirectory
cd docs
- Install the dependencies:
pip install -r ../requirements_docs.txt
- Clean the previous build (optional but recommended):
make clean
- Generate the HTML documentation:
make html
Be sure to set locale right before building.
Open the docs with your browser
- Serve the documentation locally:
python -m http.server -d build/html/
This will start a local server at http://localhost:8000. You can now open your browser and view the documentation.
If port 8000 is already in use, you can specify a different port, for example:
python -m http.server 3000 -d build/html/