* feat(tone_tests): add E2E test cases and bilingual documentation
Signed-off-by: lukotong-7 <shicanwei.scw@alibaba-inc.com>
* update T-One job link to show real-time logs
Signed-off-by: lukotong-7 <shicanwei.scw@alibaba-inc.com>
* Add description for test_hicache_storage_mooncake_backend.py
Signed-off-by: lukotong-7 <shicanwei.scw@alibaba-inc.com>
* Add parse and cleanup command options to test_1p1d_erdma.sh
Signed-off-by: lukotong-7 <shicanwei.scw@alibaba-inc.com>
* Ensure that cleanup and parse are always executed, and replace exit with return.
Signed-off-by: lukotong-7 <shicanwei.scw@alibaba-inc.com>
---------
Signed-off-by: lukotong-7 <shicanwei.scw@alibaba-inc.com>
* [Store]: Add independent deployment implementation for Client
Signed-off-by: Xingrui Yi <yixingrui@linux.alibaba.com>
* [Doc]: mooncake-store: add introduce for client standalone mode
Signed-off-by: Xingrui Yi <yixingrui@linux.alibaba.com>
* [CI]: Add dummy client test
Signed-off-by: Xingrui Yi <yixingrui@linux.alibaba.com>
* [Store]: Change dummy client setup into a new func
Signed-off-by: Xingrui Yi <yixingrui@linux.alibaba.com>
* [Store]: Add more log for dummy client
Signed-off-by: Xingrui Yi <yixingrui@linux.alibaba.com>
---------
Signed-off-by: Xingrui Yi <yixingrui@linux.alibaba.com>
* ci: add non-CUDA release workflow and update documentation
- Add release-non-cuda.yaml workflow for building non-CUDA version
- Modify build_wheel.sh to support dynamic package name modification
- Update README.md and docs with installation instructions for both versions
- CUDA version includes Mooncake-EP and GPU topology detection
- Non-CUDA version for environments without CUDA dependencies
* Update README.md
Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
---------
Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
* Always build with EP in CI
* Add missing LIBRARY_PATH
* Do not build tests and examples for the final steps to save up disk space
* Remove the requirement of USE_CUDA=ON for transfer-engine (without EP)
* Initialize a mooncake backend
* Add pybind
* Fix incorrect backend registration
* Fix wheel building of mooncake_ep
* Add a fake allreduce implementation
* Introduce transfer_engine to mooncake_backend
* Add a basic CPU proxy execution framework
* Implement a seemingly working allgather
* Remove mooncake_ep's dependency on etcd
* Implement `_allgather_base`
* Implement `allreduce`
* Implement `alltoall`
* Use an even-odd pattern for data transfer
* Add a `set_host_ip` method
* Switch to an extended-API implementation of the Mooncake backend
* Implement `broadcast`
* Implement `barrier`
* Extend Mooncake backend to CPU
* Support more operations for reduction
* Fix the backend-worker coordination logic
* Optimize CPU worker with a callback pattern
* Add a timeout-based broken-ranks detection
* Merge EP module into Mooncake's build system
* Share transfer buffer across all worker instances
* Switch to a more robust approach to detect broken ranks
* Specify CUDA device for test_mooncake_backend.py
* Explicitly stop mooncake worker
* Use transfer engine's notifications to implement collective signals
* Remove the unused `all_reduce_without` API
* Switch to mooncake backend for test_mooncake_ep.py
* Support both IB and RoCE
* Fix EP unit test
* Pass the auto-detected nic_id to EP Buffer
* Fix CMake conditional branches when `PYTORCH_CMAKE_PATH` is not set
* Fix ibgda syncing for RoCE
* Revert "Share transfer buffer across all worker instances"
This reverts commit 964e0a96
* Implement `_reduce_scatter_base`
* Make CPU backends aware of broken ranks
* Fix .typos.toml
* Add a perf test for mooncake backend
* Support more dtypes for reduction
* Revert "Use transfer engine's notifications to implement collective signals"
This reverts commit f20ffb21
* Share worker thread among all process groups
* Share transfer engine among all process groups
* Fix unit tests
* Add a warmup phase for transfer engine
* Fix transfer engine buffer locations
* Fix incorrect calculation of mooncake ep buffer
* Do not use timeout detection in mooncake_ep tests
* Update mooncake backend perf test
* Demangle per-group buffer offset from the shared taskId
* Stop allocating the useless `cuda_counter_buffer` and `cuda_data_buffer`
* Split the task list into a CPU region and a CUDA region
* Add a warmup for test_mooncake_backend_perf.py
* Switch from raw cudaEvent to `torch::Event`
* Fix MooncakeWorkCuda::wait() to make it compatible with cuda graphs
* Add doc
* Fix perf test
* Implement all-gather for perf test
* Move impl of `MooncakeEpBuffer`'s member functions to .cpp
* Change `gathered_experts` to `broken_nodes` to make the API more consistent
* `broken_nodes` should be `broken_ranks`
* API rename
* Fix format
* Enable WITH_EP option in CI
* Try installing torch in advance in CI
* Set `TORCH_CUDA_ARCH_LIST` in CMakeLists.txt
* Install required dependencies in the CI CUDA environment
* [CI] Add the matching PyTorch
* [CI] Add a workaround for missing `CUDA::nvToolsExt`
* Remove unused pybind base class declaration of `MooncakeBackendOptions`
* Support `set_device_filter`
* Remove unused headers for ep_py.cpp
* Build the EP-wheel with setuptools on CI
* [CI] Add the build-with-ep process to release.yaml
* Minor format fix
* Update build guide
* Fix docs
* Only build EP wheel with torch==2.8.0
* Add a torch version assertion for Mooncake Backend
* Fix some python typing
* Use the correct group for EP's initial data sharing
* API: invert `broken_ranks` and change into `active_ranks`
* Followup fix for inverting the API
* Fix format
* Bug-fix in mooncake_ep_kernel.cu
* Mooncake EP has to be built with USE_CUDA on
* Fixed some issues according to the review
* Fix bug
* Prepare test action env.
* [Trial] fix wildcard.
* [Trial] fix libcuda.so.1 has been modified after cuda-12.8 installed in alternative install.
* Revert to previous release.yml.
* Try to add repo for prevent publish to pypi failed in fork repo.
* [Store] config: start http server from master
Signed-off-by: Teng Ma <sima.mt@alibaba-inc.com>
* [Refactor] Use coro http server for metadata
* fix cmake
* fix merge
* add doc and test
* clear
---------
Signed-off-by: Teng Ma <sima.mt@alibaba-inc.com>
* [Store] feat: add put/get tensor API
* add new paramter for get
* add random test
* resolve conflicts
* add tensor api test to ci
* fix ci
* modify position
* use put_from
* opt
* install torch for ci
* reduce the scope of pybind11 torch
* pass ci
* add dereg
* fix get tensor
---------
Co-authored-by: qicosmos <qicosmos@linux.alibaba.com>
* refactor: introduce expected pattern for error handling in master service
- Replace ErrorCode return types with tl::expected<T, ErrorCode> pattern
- Improve error handling clarity by separating success values from error codes
- Update MasterService methods to return expected<void, ErrorCode> or expected<T, ErrorCode>
- Modify RPC service interfaces to support expected pattern
- Update all related tests to handle new expected return types
- Add necessary includes for ylt/util/expected.hpp
This change makes error handling more explicit and type-safe:
- Success cases can be accessed via .value()
- Error cases can be accessed via .error()
- Eliminates ambiguity between success and error states
Future work:
- Extend expected pattern to RPC response types
- Enhance error code system for more comprehensive error handling
* Refactor error handling to use expected<T,E> pattern instead of ErrorCode
- Updated MasterService methods to replace ylt::expected with tl::expected for better error handling.
- Modified BatchGetReplicaList, BatchPutStart, BatchPutEnd, and other methods to return tl::expected types.
- Enhanced ClientIntegrationTest to handle expected results from Put, Get, Remove, and other operations using tl::expected.
- Adjusted error handling in clientctl and master_metrics_test to utilize the new expected type.
- Improved overall error reporting in tests to provide clearer feedback on operation failures.
* fix test compile
* refactor: update client implementation and remove master.proto
- Enhanced client.h and client.cpp with new functionality
- Removed obsolete master.proto file
- Updated master_client.cpp and transfer_task.cpp
- Improved integration and stress tests
* fix: update Python integration to work with new batch API
- Replace BatchObjectInfo with vector<vector<Replica::Descriptor>>
- Update BatchPut to handle new return type vector<tl::expected<void, ErrorCode>>
- Fix BatchQuery API usage to work with new expected pattern
- Convert unordered_map to vector format for BatchPut parameter compatibility
* refine master log and metric
* fix(ci): should alloc first
* merge main
* fix tests
* enable client ssd offload and storage persistence
* add storage_root_path in all tests setup() initialization and add related description in doc
* clean up headers and improve code readability - Added consistent Doxygen-style comments to all header files - Removed redundant code and outdated comments - Optimized function execution logic in LocalFile
* Revert "add storage_root_path in all tests setup() initialization and add related description in doc"
This reverts commit 159442d44e.
revert old high-level api test and doc modification
* Restore the high-level API to its original state and modify it to introduce the storage path through environment variables.
* add local_file_test and thread_pool_test
* feat(client_ssd_offload): implement async writes and fix locking bugs
- Refactor write operations to use thread pool for async file I/O
- Fix potential double-unlock bug by adding atomic is_locked_ flag
- Add corrupted file cleanup on write failure:
- Auto-delete files with failed writes in destructor
- Prevent subsequent reads of corrupted data
* add support for remove , remove_all , isexist interface etc.
* feat(kvcache): implement cluster isolation with session IDs
* Remove precompilation parameters to simplify build configuration
* Add session ID mechanism for cluster isolation:
- Master node now generates unique session IDs on initialization
- All persistent operations are scoped under session-specific subdirectories
* edit two parameters client get, add persisitence path in client rather than store_py.cpp
* add support for batch api conflict , refactor replica.descriptor to support file and memory type
* add test branch
* add ci ssd
* change python test
* add log for fail
* change querykey return value type
* fix bug
* fix bug
* add sleep for removefile
* fix sleep
* edit ci.yml and fix delete before write problem
* add comment for storage_backend
* spell check
* fix name problem and decrease errorcode for file
* add pytest for ssd offload
* edit test
* edit test
* fix test
* fix bug
* fix test
* Modify the thread pool value capture to reference capture to fix the issue of significant performance degradation when writing files with put.
* add async getfrom file in batchget transfertask. delete file_storage_backend
* add support for HA in cluster_id subdirectory, change session_id to fsdir
* add persistence in batchput
* add disk allocate for get_into py interface
* fix bug
* temp
* fix bug in submit fileread task for std:move(slices)
* edit querykey to return optional<descriptor>, add interface batchquerykey for storagebackend
* fix confict in batchget, add batchget/batchput test
* fix bug
* fix bug
* comment batch test
* fix conflict and add batch_get_into file test
* fix test bug
* fix test bug
* fix conflict
* feat(store): add zero-copy operations for python binding
* test: rename dict fuzz e2e test to run last
* test: remove obsolete test_multicards.py from repository
* chore(tests): remove multicards test execution from script
* [Build] Optimize store build control for wheel and local build
Signed-off-by: Shangming Cai <caishangming@linux.alibaba.com>
* fix typo
Signed-off-by: Shangming Cai <caishangming@linux.alibaba.com>
* fix rm
Signed-off-by: Shangming Cai <caishangming@linux.alibaba.com>
---------
Signed-off-by: Shangming Cai <caishangming@linux.alibaba.com>
* chore: automate build output directory and update scripts
* feat(ci): enable CUDA support in workflows
Signed-off-by: Jinyang Su <751080330@qq.com>
* chore(build): switch to nvcc for nvlink hook compilation
Signed-off-by: Jinyang Su <751080330@qq.com>
---------
Signed-off-by: Jinyang Su <751080330@qq.com>
* leave endpoint status unchanged when delete endpoint reference to avoid endpoint deconstruction before CQ being generated
Signed-off-by: jiafu.zhang <jiafu.zhang@intel.com>
* leave endpoint status unchanged when delete endpoint reference to avoid endpoint deconstruction before CQ being generated
Signed-off-by: jiafu.zhang <jiafu.zhang@intel.com>
* leave endpoint status unchanged when delete endpoint reference to avoid endpoint deconstruction before CQ being generated
Signed-off-by: jiafu.zhang <jiafu.zhang@intel.com>
---------
Signed-off-by: jiafu.zhang <jiafu.zhang@intel.com>
* feat: use pyproject.toml and simplify setup.py
* feat: add apache license and update pyproject.toml
* chore: use string quotes for python version in CI matrix
* feat: build wheels for python 3.10 and 3.12
* chore: update wheel build process in CI/CD
* ci: refactor ci and release workflows for matrix builds
* chore: remove unused workflow and fix license path
* feat: initialize git submodules in dependencies.sh
Feng Ren