* Prepare test action env.
* [Trial] fix wildcard.
* [Trial] fix libcuda.so.1 has been modified after cuda-12.8 installed in alternative install.
* Revert to previous release.yml.
* Try to add repo for prevent publish to pypi failed in fork repo.
* [Store] config: start http server from master
Signed-off-by: Teng Ma <sima.mt@alibaba-inc.com>
* [Refactor] Use coro http server for metadata
* fix cmake
* fix merge
* add doc and test
* clear
---------
Signed-off-by: Teng Ma <sima.mt@alibaba-inc.com>
* [Store] feat: add put/get tensor API
* add new paramter for get
* add random test
* resolve conflicts
* add tensor api test to ci
* fix ci
* modify position
* use put_from
* opt
* install torch for ci
* reduce the scope of pybind11 torch
* pass ci
* add dereg
* fix get tensor
---------
Co-authored-by: qicosmos <qicosmos@linux.alibaba.com>
* refactor: introduce expected pattern for error handling in master service
- Replace ErrorCode return types with tl::expected<T, ErrorCode> pattern
- Improve error handling clarity by separating success values from error codes
- Update MasterService methods to return expected<void, ErrorCode> or expected<T, ErrorCode>
- Modify RPC service interfaces to support expected pattern
- Update all related tests to handle new expected return types
- Add necessary includes for ylt/util/expected.hpp
This change makes error handling more explicit and type-safe:
- Success cases can be accessed via .value()
- Error cases can be accessed via .error()
- Eliminates ambiguity between success and error states
Future work:
- Extend expected pattern to RPC response types
- Enhance error code system for more comprehensive error handling
* Refactor error handling to use expected<T,E> pattern instead of ErrorCode
- Updated MasterService methods to replace ylt::expected with tl::expected for better error handling.
- Modified BatchGetReplicaList, BatchPutStart, BatchPutEnd, and other methods to return tl::expected types.
- Enhanced ClientIntegrationTest to handle expected results from Put, Get, Remove, and other operations using tl::expected.
- Adjusted error handling in clientctl and master_metrics_test to utilize the new expected type.
- Improved overall error reporting in tests to provide clearer feedback on operation failures.
* fix test compile
* refactor: update client implementation and remove master.proto
- Enhanced client.h and client.cpp with new functionality
- Removed obsolete master.proto file
- Updated master_client.cpp and transfer_task.cpp
- Improved integration and stress tests
* fix: update Python integration to work with new batch API
- Replace BatchObjectInfo with vector<vector<Replica::Descriptor>>
- Update BatchPut to handle new return type vector<tl::expected<void, ErrorCode>>
- Fix BatchQuery API usage to work with new expected pattern
- Convert unordered_map to vector format for BatchPut parameter compatibility
* refine master log and metric
* fix(ci): should alloc first
* merge main
* fix tests
* enable client ssd offload and storage persistence
* add storage_root_path in all tests setup() initialization and add related description in doc
* clean up headers and improve code readability - Added consistent Doxygen-style comments to all header files - Removed redundant code and outdated comments - Optimized function execution logic in LocalFile
* Revert "add storage_root_path in all tests setup() initialization and add related description in doc"
This reverts commit 159442d44e.
revert old high-level api test and doc modification
* Restore the high-level API to its original state and modify it to introduce the storage path through environment variables.
* add local_file_test and thread_pool_test
* feat(client_ssd_offload): implement async writes and fix locking bugs
- Refactor write operations to use thread pool for async file I/O
- Fix potential double-unlock bug by adding atomic is_locked_ flag
- Add corrupted file cleanup on write failure:
- Auto-delete files with failed writes in destructor
- Prevent subsequent reads of corrupted data
* add support for remove , remove_all , isexist interface etc.
* feat(kvcache): implement cluster isolation with session IDs
* Remove precompilation parameters to simplify build configuration
* Add session ID mechanism for cluster isolation:
- Master node now generates unique session IDs on initialization
- All persistent operations are scoped under session-specific subdirectories
* edit two parameters client get, add persisitence path in client rather than store_py.cpp
* add support for batch api conflict , refactor replica.descriptor to support file and memory type
* add test branch
* add ci ssd
* change python test
* add log for fail
* change querykey return value type
* fix bug
* fix bug
* add sleep for removefile
* fix sleep
* edit ci.yml and fix delete before write problem
* add comment for storage_backend
* spell check
* fix name problem and decrease errorcode for file
* add pytest for ssd offload
* edit test
* edit test
* fix test
* fix bug
* fix test
* Modify the thread pool value capture to reference capture to fix the issue of significant performance degradation when writing files with put.
* add async getfrom file in batchget transfertask. delete file_storage_backend
* add support for HA in cluster_id subdirectory, change session_id to fsdir
* add persistence in batchput
* add disk allocate for get_into py interface
* fix bug
* temp
* fix bug in submit fileread task for std:move(slices)
* edit querykey to return optional<descriptor>, add interface batchquerykey for storagebackend
* fix confict in batchget, add batchget/batchput test
* fix bug
* fix bug
* comment batch test
* fix conflict and add batch_get_into file test
* fix test bug
* fix test bug
* fix conflict
* feat(store): add zero-copy operations for python binding
* test: rename dict fuzz e2e test to run last
* test: remove obsolete test_multicards.py from repository
* chore(tests): remove multicards test execution from script
* [Build] Optimize store build control for wheel and local build
Signed-off-by: Shangming Cai <caishangming@linux.alibaba.com>
* fix typo
Signed-off-by: Shangming Cai <caishangming@linux.alibaba.com>
* fix rm
Signed-off-by: Shangming Cai <caishangming@linux.alibaba.com>
---------
Signed-off-by: Shangming Cai <caishangming@linux.alibaba.com>
* chore: automate build output directory and update scripts
* feat(ci): enable CUDA support in workflows
Signed-off-by: Jinyang Su <751080330@qq.com>
* chore(build): switch to nvcc for nvlink hook compilation
Signed-off-by: Jinyang Su <751080330@qq.com>
---------
Signed-off-by: Jinyang Su <751080330@qq.com>
* leave endpoint status unchanged when delete endpoint reference to avoid endpoint deconstruction before CQ being generated
Signed-off-by: jiafu.zhang <jiafu.zhang@intel.com>
* leave endpoint status unchanged when delete endpoint reference to avoid endpoint deconstruction before CQ being generated
Signed-off-by: jiafu.zhang <jiafu.zhang@intel.com>
* leave endpoint status unchanged when delete endpoint reference to avoid endpoint deconstruction before CQ being generated
Signed-off-by: jiafu.zhang <jiafu.zhang@intel.com>
---------
Signed-off-by: jiafu.zhang <jiafu.zhang@intel.com>
* feat: use pyproject.toml and simplify setup.py
* feat: add apache license and update pyproject.toml
* chore: use string quotes for python version in CI matrix
* feat: build wheels for python 3.10 and 3.12
* chore: update wheel build process in CI/CD
* ci: refactor ci and release workflows for matrix builds
* chore: remove unused workflow and fix license path
* feat: initialize git submodules in dependencies.sh
* chore(ci): integrate auditwheel into wheel build process
* remove copy .so
* use a pure ubuntu to test
* run tests
* chore(ci): add Golang setup and start metadata server in CI workflow
* ci: add matrix strategy for testing on multiple ubuntu versions
* build: enable debug mode and specify platform for auditwheel repair
* test: improve test_installation script with dependency check
* refactor(test_installation): simplify import verification steps
* chore(scripts): remove unused system packages from test script
* Refactor dependencies.sh for improved installation process and error handling
* [CI] Add Python build dependencies to CI workflow
* fix ci
* use another port
ascend-direct-dev