Commit Graph

535 Commits

Author SHA1 Message Date
Ze Jiang dfd445acbd
chore(issue): Fix typo in issue template (#858) 2025-09-18 15:24:35 +08:00
JinYan Su dafaaad076
feat(store): add client heartbeat support for non ha mode (#845)
* feat(store): add client heartbeat support for non ha mode

This change extends client functionality to handle master reconnection and segment remounting in non-HA mode, adds corresponding test infrastructure, and improves utility functions for HTTP operations and port allocation.

Signed-off-by: xiaguan <751080330@qq.com>

* refactor(tests): reorganize test entries in CMakeLists.txt

* update docs

---------

Signed-off-by: xiaguan <751080330@qq.com>
Co-authored-by: Ke Yang <oneday117@qq.com>
2025-09-17 14:54:48 +08:00
joss bf88c3dc4f
[Store] Add Master Json Config Example 2025-09-17 11:54:22 +08:00
ascend-direct-dev c0f83b1ab3
add ascend direct transport to mooncake store (#835)
Co-authored-by: youxiao <youxiao@huawei.com>
2025-09-17 09:25:19 +08:00
JinYan Su 785e939523
refactor(store): use dedicated thread for signal handling (#840)
* refactor(store): use dedicated thread for signal handling and atomic cleanup

* refactor(store): switch PyClient to shared_ptr and improve resource management

* refactor(ResourceTracker): use leaked heap object to avoid static destruction order issues

* test: increase unittest verbosity to show running tests

* try fix ci

* fix setup after close

* fix comments
2025-09-16 14:41:19 +08:00
JinYan Su 51687e916e
docs(deployment): Add Mooncake Store deployment guide with configuration and metrics (#825) 2025-09-16 10:02:55 +08:00
JinYan Su 42a9d64a66
feat(store): add NUMA node binding support via bind_to_numa_node method (#823)
* feat(store): add NUMA node binding support via bind_to_numa_node method

* feat(store): Add NUMA node binding with validation and error logging

* refactor(store): move NUMA binding to module-level function

* fmt
2025-09-15 20:40:38 +08:00
ykwd 9b8426beac
[Docs] Fix Broken Trace Link (#841) 2025-09-15 16:34:03 +08:00
ykwd 8bdb472c19
[Doc] SGLang HiCache Intergration (#833) 2025-09-15 14:05:51 +08:00
ykwd 82b90fc1fe
[Build] Install Python Files Patch (#839)
This is a patch for this pr: #836

This PR makes the following changes:

- Only install Python files that depend on Store when the WITH_STORE flag is enabled.

- Update the permissions of the site-packages/mooncake directory from 766 to 767 (i.e., chmod o+x) so that non-root users can access the files inside. With 766, since the files are installed via sudo make install, regular users cannot read or execute them.
2025-09-15 12:52:17 +08:00
Houjiang Chen a099ced0d3
[TransferEngine] Make ascend TE to be released successfully and support fast recovery from failures through retry (#827)
* make hccl transfer engine to be delete successfully

* reinit transport mem when a data transfer timeout occurs. This usually happens when the remote target is restarted.

* support retry

* fix compile
2025-09-14 10:18:51 +08:00
ykwd 6d6de583db
[Build] Install Python Files (#836)
* Install python files

* Remove test message
2025-09-13 23:50:46 +08:00
doujiang24 e44d78b862
[TransferEngine]: remove SO_REUSEADDR in findAvailableTcpPort (#830)
Signed-off-by: doujiang24 <doujiang24@gmail.com>
2025-09-12 09:54:23 +08:00
ykwd 63db31c592
[Doc] Store Integrated to SGLang HiCache (#829)
* Update readme

* Add benchmark

* Minor update

* Add news
2025-09-11 22:49:16 +08:00
Peng Guanwen e2632c5afa
[CI/Build] gate store test behind BUILD_UNIT_TESTS option (#821) 2025-09-11 13:47:57 +08:00
ykwd 728fb6437f
[Store] Change Default Value of eviction_high_watermark_ratio and eviction_ratio (#820) 2025-09-10 17:13:59 +08:00
Xuchun Shang 06cc217504
[Store] skip null buffer (#812)
* Skip null buffer

Signed-off-by: Xuchun Shang <xuchun.shang@linux.alibaba.com>

* fix lint

Signed-off-by: Xuchun Shang <xuchun.shang@linux.alibaba.com>

* update

Signed-off-by: Xuchun Shang <xuchun.shang@linux.alibaba.com>

---------

Signed-off-by: Xuchun Shang <xuchun.shang@linux.alibaba.com>
2025-09-10 16:00:47 +08:00
Shangming Cai be894977d9
chore: bump version to 0.3.6 in pyproject.toml (#819) 2025-09-10 15:45:37 +08:00
JinYan Su d8f1ee24b2
[Store] feat(store): add duplicate rpc_meta key check and CI integration (#818)
* feat(store): add duplicate rpc_meta key check and CI integration

* fix(KVBootstrapServer): disallow duplicate rpc_meta keys in PUT requests

* revert ci changes

* fix ci
2025-09-10 12:10:32 +08:00
Sgt.Pepper cc2c9c19af
Fix(store): Fix hf3fs_file.cpp log compile problem (#811) 2025-09-10 10:48:50 +08:00
JinYan Su ba22ad9096
Refactor(store): Enable the transfer engine to autonomously detect (#817)
* Refactor(store): Enable the transfer engine to autonomously detect topology

* feat(client): support manual RDMA device configuration without auto-discovery
2025-09-10 10:05:06 +08:00
JinYan Su 5bc114cba9
fix(transfer_engine): use thread-local CURL handles for thread safety (#815)
* fix(transfer_engine): use thread-local CURL handles and improve error handling

* Update mooncake-transfer-engine/src/transfer_metadata_plugin.cpp

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

* fix fmt

---------

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
2025-09-09 09:52:49 +08:00
licunlong 3554015b0e
[p2pstore] allow topologyMatrix to be empty (#807)
when install tcp transport, nicPriorityMatrix is "". set args[0] to nil so that tcp transport can be installed
2025-09-09 09:48:12 +08:00
Letian Ruan e711b6fcc3
[RL] Add dummy example of RL training on mooncake store (#810)
* feat: init rl training using store for rollout

* chore: update structure

* fix: object store setup and training
2025-09-08 22:56:05 +08:00
JinYan Su 4fd859b525
refactor(AutoPortBinder): remove SO_REUSEADDR setting (#816) 2025-09-08 17:41:36 +08:00
Ruoyu Qin dedfbde5da
[Doc] Update docs for a better quick start (#814)
* add transfer engine python api

* add quick start

* add lmcache v1 integration

* update trace

* fix

* modify segment size
2025-09-06 11:16:42 +08:00
ykwd cd717c7091
[Chores] Offset Allocator Test Fix & Docs Fix (#806)
* Fix unit test

* fix docs
2025-09-05 10:38:19 +08:00
Mumupika a7e1f1c360
[CI] Fix Release build_wheel.sh to make python 3.8 auditwheel happy (#801)
* Prepare test action env.

* [Trial] fix wildcard.

* [Trial] fix libcuda.so.1 has been modified after cuda-12.8 installed in alternative install.

* Revert to previous release.yml.

* Try to add repo for prevent publish to pypi failed in fork repo.
2025-09-05 10:37:28 +08:00
Haifeng Chen 1c7246c5c7
Remove unnecessary register buffer from put_tensor (#803) 2025-09-05 09:57:24 +08:00
JinYan Su 9971eda79c
feat(build): add CI-specific build option with --ci-build flag (#808) 2025-09-04 20:49:49 +08:00
JinYan Su 03a26184ea
fix(store): add mutex locks for thread-safe metrics retrieval (#804)
* fix(store): add mutex locks for thread-safe metrics retrieval and comparisons

* refactor(get_metrics_internal): require caller to hold m_mutex instead of internal locking
2025-09-04 16:54:11 +08:00
zuochunwei 0cc51a60ac
[TransferEngine] heterogeneous_ascend support kv-cache transfer between npu and gpu (#759)
* heterogeneous_ascend
Co-authored-by: AscendTransport<ascend_transport@yeah.net>

* update desc for USE_ASCEND_HETEROGENEOUS option

* format

* fix bug

* format

* lock_guard

* a new fix

---------

Co-authored-by: zuochunwei <zuochunwei@meituan.com>
Co-authored-by: ascend_transport <ascend_transport@yeah.net>
Co-authored-by: AscendTransport <ascendtransport@yeah.net>
2025-09-03 09:41:48 +08:00
Stary 5eb8948425
[TE][RDMA Transport]: Simplify Transfer Submission Logic (#772)
* opt(rdma): optimize device selection logic in transfer submission

Pre-select device for entire request to avoid per-slice selection overhead.

Signed-off-by: staryxchen <staryxchen@tencent.com>

* refactor(rdma): simplify transfer submission by delegating to helper method

Replace inline slice processing logic with call to submitTransferTask method to reduce code duplication and improve maintainability.

Signed-off-by: staryxchen <staryxchen@tencent.com>

* fix(rdma): add explicit casts for size comparisons to prevent signed/unsigned mismatch

Signed-off-by: staryxchen <staryxchen@tencent.com>

---------

Signed-off-by: staryxchen <staryxchen@tencent.com>
2025-09-03 09:38:53 +08:00
Shaofeng Ding 206d52e1f6
[TransferEngine] Fix SIEVE eviction algorithm for RDMA endpoint store (#767) 2025-09-02 11:17:53 +08:00
qicosmos 185c5d229b
[coro_rpc] use client pool and enable rdma (#789) 2025-09-02 00:40:21 +08:00
ascend-direct-dev 506231b615
change get memory type (#786)
Co-authored-by: youxiao <youxiao@huawei.com>
2025-09-01 15:32:19 +08:00
Haifeng Chen e475a369fe
[Store] GIL release for put_tensor and get_tenor (#783) 2025-09-01 14:56:06 +08:00
dependabot[bot] 897728ddbf
chore(deps): bump tracing-subscriber in /mooncake-transfer-engine/rust (#794)
Bumps [tracing-subscriber](https://github.com/tokio-rs/tracing) from 0.3.18 to 0.3.20.
- [Release notes](https://github.com/tokio-rs/tracing/releases)
- [Commits](https://github.com/tokio-rs/tracing/compare/tracing-subscriber-0.3.18...tracing-subscriber-0.3.20)

---
updated-dependencies:
- dependency-name: tracing-subscriber
  dependency-version: 0.3.20
  dependency-type: direct:production
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2025-08-30 15:57:52 +08:00
Chaos Dong d944902104
[Bench] fix: Update stress_cluster_benchmark (#791)
Co-authored-by: 春明 <chunming.dc@alibaba-inc.com>
2025-08-29 14:46:42 +08:00
Stary fc04a31cae
[TE] fix(transfer): correct handshake daemon initialization order (#765)
Fix incorrect initialization sequence in startHandshakeDaemon by moving handshake_plugin_->startDaemon() call after callback registrations to ensure proper setup before daemon starts.

Signed-off-by: staryxchen <staryxchen@tencent.com>
2025-08-28 19:54:18 +08:00
JinYan Su 6fbb85b37c
feat(store): Add largest free region filtering for allocation (#785)
The commit introduces two main features:
1. A thread-safe retry counter to track allocation attempts in RandomAllocationStrategy
2. A getLargestFreeRegion() method in BufferAllocatorBase interface that allows filtering allocators by available space, with specific implementations for both Cachelib and Offset allocators

These changes improve allocation reliability and efficiency by enabling better failure tracking and space-aware allocation decisions.

Signed-off-by: Jinyang Su <751080330@qq.com>
2025-08-28 19:16:12 +08:00
haobayuxi 9e8adb55cd
fix nixl bench bug (#788) 2025-08-28 15:55:13 +08:00
Feng Ren 555f67f90e
[TE] Fix notifs problems in C wrapper (#779)
* [TE] Fix notifs problems in C wrapper

* update alloc function

* Fix memory alloc issue

* format
2025-08-28 14:39:51 +08:00
Alex Nails a726295136
[Bugfix] YAML CPP has inline impl in header file which will cause linking error (#784) 2025-08-28 10:29:06 +08:00
ascend-direct-dev 562ed4475b
change adxl error code, compatible to CANN 8.2.RC1 (#764)
Co-authored-by: youxiao <youxiao@huawei.com>
2025-08-28 10:14:29 +08:00
Sgt.Pepper dcdf1c784b
[Doc] Add description for CONTRIBUTING.md (#773) 2025-08-26 17:46:22 +08:00
Teng Ma 0ea4d8c439
[DOC] Add all badges (#781)
* [DOC] add badges

* format
2025-08-26 14:25:11 +08:00
ykwd e6da3013c3
[Store] Break Circular Dependency Between type.h And Other Files (#771) 2025-08-26 14:16:09 +08:00
Vladislav Nosivskoy aa9d4471a1
[Store] Add replication guarantees (#744)
* add replication guarantees

Signed-off-by: Vladislav Nosivskoy <vladnosiv@gmail.com>

* refactor implementation

Signed-off-by: Vladislav Nosivskoy <vladnosiv@gmail.com>

* fix merge aftermath

Signed-off-by: Vladislav Nosivskoy <vladnosiv@gmail.com>

* less copies

Signed-off-by: Vladislav Nosivskoy <vladnosiv@gmail.com>

* try fix wheel tests

Signed-off-by: Vladislav Nosivskoy <vladnosiv@gmail.com>

* fix wheel tests

Signed-off-by: Vladislav Nosivskoy <vladnosiv@gmail.com>

* remove old comment

Signed-off-by: Vladislav Nosivskoy <vladnosiv@gmail.com>

* fix wheel tests and add replication fault tolerance wheel test

Signed-off-by: Vladislav Nosivskoy <vladnosiv@gmail.com>

* split wheel tests

Signed-off-by: Vladislav Nosivskoy <vladnosiv@gmail.com>

* simplify code

Signed-off-by: Vladislav Nosivskoy <vladnosiv@gmail.com>

* replica allocation is best effort operation

Signed-off-by: Vladislav Nosivskoy <vladnosiv@gmail.com>

* fix tests for new best-effort behaviour

Signed-off-by: Vladislav Nosivskoy <vladnosiv@gmail.com>

* fix

Signed-off-by: Vladislav Nosivskoy <vladnosiv@gmail.com>

* update docs

Signed-off-by: Vladislav Nosivskoy <vladnosiv@gmail.com>

* update another docs

Signed-off-by: Vladislav Nosivskoy <vladnosiv@gmail.com>

---------

Signed-off-by: Vladislav Nosivskoy <vladnosiv@gmail.com>
2025-08-25 20:24:41 +08:00
Teng Ma f76c92295b
[Store] add c++ http metadata server in mooncake master (#766)
* [Store] config: start http server from master

Signed-off-by: Teng Ma <sima.mt@alibaba-inc.com>

* [Refactor] Use coro http server for metadata

* fix cmake

* fix merge

* add doc and test

* clear

---------

Signed-off-by: Teng Ma <sima.mt@alibaba-inc.com>
2025-08-25 13:56:37 +08:00