* [Store]: add task executor feature with unit and executor test
Signed-off-by: Vincent Gao <vincentbo@linux.alibaba.com>
* [Store]: add some optimizations to task executor
Refactor the task_executor into the client_service and add a
task structure on the client side. Additionally, remove the
existence check in the execute function;
only retrying should be performed if replica allocation fails.
Signed-off-by: Vincent Gao <vincentbo@linux.alibaba.com>
* [Store]: get source replica from copyStart or moveStart api
* [Store]: call move or copy end if the target replica already exist to complete the replication task
* [Store]: use the max_retry_attempts in master side
* [Store]: add client integration test and set default max_retry_attempts to 10
* [Doc] update task api introduction
Signed-off-by: Vincent Gao <vincentbo@linux.alibaba.com>
* [Store] Set copy and move as private methods
Signed-off-by: Vincent Gao <vincentbo@linux.alibaba.com>
* [Doc]: change the default task max_retry_attempts to 10
* [Doc]: fix some description error
* [Store]: allocate the buffer size to be a multiple of 16MB
* [Store]: validate the replica is in local and directly construct slices from replica buffer address instead of copy the data to local buffer
* [Store]: change the validate logic to directly use transfer engine endpoint or local_hostname_.
* [Store] add e2e ci test for copy and move api
* [Store] refactor the client move and copy function
* [Store] fix the e2e test
* [Store] remove unused code
* [Store] add source field when build replica copy payload in the task_manager_test
* [Store] rename back to snake case for split_into_slices function and also remove hard code for client poll count
* [Store] revert mis deleted field when resolve conflicts
* [Store] change test to validate the real behaviour
* [Store] change the default task fetch size to 16
* [Doc]: change the replica copy/move sequence diagram
* [Store] add new split_into_slice method
* [Store] change the real client to use split_to_slice with buffer handle parameters
---------
Signed-off-by: Vincent Gao <vincentbo@linux.alibaba.com>
Co-authored-by: Vincent Gao <vincentbo@linux.alibaba.com>
* feat(store): disable auto discovery by default, require rdma_devices for RDMA
* feat(client): support optional auto-discovery with default for RDMA
* update comment
* update doc
* update hello world
* feat(allocator): Add transport endpoint support for segment transfers
* refactor(Client): Replace segment_name with transport_endpoint in buffer descriptors and tests
* refactor(test): switch to in-proc master and remove external service management
* style: format code and adjust transfer task timeout
* feat(Client::MountSegment): set te_endpoint conditionally based on metadata_connstring for P2P mode
* Allow configuring default KV lease TTL via environment variable
Read DEFAULT_KV_LEASE_TTL environment variable at server startup and
parse as uint64_t value. This enables CI tests to override default
configuration without code changes.
Signed-off-by: xiaguan <751080330@qq.com>
* style: format CI config and add [[nodiscard]] to methods
Signed-off-by: xiaguan <751080330@qq.com>
* refactor(tests): remove local_hostname parameter and improve test helpers
The commit removes the unused local_hostname parameter from TransferSubmitter constructor, updates the associated initialization logic, and simplifies test code by introducing helper functions for segment creation. The changes improve code maintainability while preserving functionality.
* refactor(TransferSubmitter): Remove unused local_hostname_ member and validation check
* docs(store): update setup examples for p2p
---------
Signed-off-by: xiaguan <751080330@qq.com>
Cruz Zhao