* adjust hearbeat init order
* base version
* add test file
* optimize async metadata notifier and test
* rename master_reachable to master_reconnected for clarity
* rewrite HA integration test: two clients, manual heartbeat, fault scenarios
* refactor ha recovery manager test and fix bugs
* modify the comment
* clang format
---------
Co-authored-by: wanyue.wy <wanyue.wy@oceanbase.com>
* [Store] Make tiered scheduler incremental
Rework the tiered backend scheduler to avoid a shared global stats lock
and full keyspace scans in the background loop.
Key changes:
- shard stats collection and maintain ordered incremental indexes for
Simple and LRU snapshots
- split recent heat and recency rank semantics in the scheduler policy
interface
- build policy input from snapshot candidates plus fast-tier residents
using scheduler-side metadata cache
- hook scheduler cache maintenance into TieredBackend commit/delete paths
- extend integration tests for concurrent stats, bounded snapshots, and
wall-clock decay behavior
Signed-off-by: Xingrui Yi <yixingrui@linux.alibaba.com>
* [Store] Make bucket file eviction async
Signed-off-by: Xingrui Yi <yixingrui@linux.alibaba.com>
* [Store] Guard bucket offload with physical space
Signed-off-by: Xingrui Yi <yixingrui@linux.alibaba.com>
* [Store] Unify SSD backend space management
Signed-off-by: Xingrui Yi <yixingrui@linux.alibaba.com>
* [Store] Preallocate storage tier staging pool
Signed-off-by: Xingrui Yi <yixingrui@linux.alibaba.com>
* [Store] Make allocation handles own tier lifetime
Signed-off-by: Xingrui Yi <yixingrui@linux.alibaba.com>
* [Store] Add reclaim planner to tiered scheduler
Signed-off-by: Xingrui Yi <yixingrui@linux.alibaba.com>
* [Store] Unify tiered backend test init
Fold repeated TieredBackend::Init test setup into a shared helper and guard storage-tier bucket eviction with a safe runtime cast so file-per-key tests no longer crash when capacity is exceeded.
Signed-off-by: Xingrui Yi <yixingrui@linux.alibaba.com>
* [Store] Surface scheduler policy errors
Signed-off-by: Xingrui Yi <yixingrui@linux.alibaba.com>
* [Store] Fix tensor API CI mode selection
Signed-off-by: Xingrui Yi <yixingrui@linux.alibaba.com>
* [Store] Fix bucket offload pre-init capacity check
Signed-off-by: Xingrui Yi <yixingrui@linux.alibaba.com>
* [Store] Fix storage path build warnings
Signed-off-by: Xingrui Yi <yixingrui@linux.alibaba.com>
* [Store] Unify tiered storage accounting semantics
Clarify storage-tier live-byte accounting versus backend physical-byte accounting, reclaim per-key physical bytes on delete, and make per-key metadata scans idempotent without holding the accounting lock across the full directory walk.
Signed-off-by: Xingrui Yi <yixingrui@linux.alibaba.com>
* [Store] Shard scheduler key cache and surface alloc errors
Shard the scheduler key cache to reduce lock contention, preserve allocation error codes across tier fallback, and tighten storage-tier tests around explicit overflow and capacity failures.
Signed-off-by: Xingrui Yi <yixingrui@linux.alibaba.com>
---------
Signed-off-by: Xingrui Yi <yixingrui@linux.alibaba.com>
* 1. update client name to client service
2. clang format
* 1. implement ClientConfig for build client service
2. split client service based on architecture
* 1. add graceful shutdown mechanism for store components
2. fix some ut bugs about param or return value type
3. fix signal ignore in seperatly deployment mode
* modify stress workload test to adapt p2p mode
* 1. rename some class and add some comment
2. move dummy_client_monitor start logic to real_client and add stop logic for it
* add in-flight request check
* 1. optimize GetLocal() function
2. rename some var and refactor some code format
* add stream output for replica descriptor
* fix ut bug
* replace dynamic_cast with static_cast
* fix typo
* fix format
* fix ci
* 1. fix error code typo of p2p ut
2. fix destory bug of centralized client service
---------
Co-authored-by: wanyue.wy <wanyue.wy@oceanbase.com>
* [Store]: Add ssd tier support
Signed-off-by: Xingrui Yi <yixingrui@linux.alibaba.com>
* [Store]: add client scheduler support
Signed-off-by: Xingrui Yi <yixingrui@linux.alibaba.com>
* [Store]: Add CAS support for tiered backend
Signed-off-by: Xingrui Yi <yixingrui@linux.alibaba.com>
* [Store] Add LRU policy and enhance stats collector
- Add LRUPolicy: watermark-based promotion and eviction
- Promotion limited by capacity budget (up to low_watermark)
- Eviction triggered when usage exceeds high_watermark
- Support EVICT (delete replica) and MIGRATE (move to slow tier)
- Add LRUStatsCollector: maintains LRU ordered key list
- O(1) access recording using list + hashmap
- MRU keys at front, LRU keys at back
- Enhance StatsCollector interface:
- Add RemoveKey() for cleanup on key deletion
- Add decay factor to SimpleStatsCollector (default 0.5)
- Prevents history loss between scheduler cycles
- Add size_bytes field to KeyContext for capacity calculations
Signed-off-by: Xingrui Yi <yixingrui@linux.alibaba.com>
* [Store] Integrate LRU scheduler with TieredBackend
- ClientScheduler enhancements:
- Support JSON config for policy selection (SIMPLE/LRU)
- Add OnDelete() callback for LRU list cleanup
- Implement two-phase action execution (EVICT first, then MIGRATE)
- Handle NO_AVAILABLE_HANDLE error for insufficient space
- TieredBackend integration:
- Pass config to ClientScheduler constructor
- Add OnAccess() call in Commit()
- Add OnDelete() call in Delete()
- Add capacity pre-check in Transfer()
- Add integration tests:
- TestLRUCacheThrashing: end-to-end hot/cold data handling
- TestLRUPromotionBudget: verify capacity-limited promotion
- TestLRUEviction: verify cold data eviction
Signed-off-by: Xingrui Yi <yixingrui@linux.alibaba.com>
* [Store] Improve StorageTier capacity tracking and thread safety
- Add capacity parameter to StorageTier constructor
- GetCapacity() now uses configured capacity or falls back to storage backend config
- Add persisted_size_ tracking for data written to disk
- GetUsage() returns pending_batch_size_ + persisted_size_
- Make pending_batch_size_ atomic for thread-safe GetUsage()
- Add IsPersisted() method to StorageBuffer
- Free() now properly updates persisted_size_ when freeing persisted data
Signed-off-by: Xingrui Yi <yixingrui@linux.alibaba.com>
* [Store] Reorganize tier files into tiers/ subdirectory
- Move all tier-related files to tiered_cache/tiers/ subdirectory:
- cache_tier.h (base class)
- dram_tier.h/cpp
- storage_tier.h/cpp
- ascend_tier.h/cpp
- Merge disk_buffer.h into storage_tier.h (StorageBuffer class)
- Update all include paths to use new tiers/ location
- Update CMakeLists.txt for new file locations
This improves code organization by grouping all tier implementations
in a dedicated subdirectory.
Signed-off-by: Xingrui Yi <yixingrui@linux.alibaba.com>
* [Store] Make StorageBuffer::is_on_disk_ atomic for thread safety
Change is_on_disk_ from plain bool to std::atomic<bool> with proper
memory ordering (acquire/release) to prevent data races between
Persist() and concurrent read operations like ReadTo().
Signed-off-by: Xingrui Yi <yixingrui@linux.alibaba.com>
* [Store] Refactor LRU policy to use global heat ranking
Redesign LRU scheduling algorithm:
- Sort all keys by heat score globally (not per-tier)
- Select hottest keys to fill fast tier up to low_watermark (70%)
- Generate evict/promote actions by comparing current vs target state
This ensures fast tier always contains the globally hottest keys,
and usage stabilizes at low_watermark after scheduling.
Update tests to verify:
- All hot keys are retained in DRAM
- Total keys in DRAM matches expected slots
- Cold keys only fill remaining slots after hot keys
Signed-off-by: Xingrui Yi <yixingrui@linux.alibaba.com>
* [Store] Fix use-after-free in StorageTier::FlushInternal
Add is_flushing_ flag to StorageBuffer to prevent Free() from
destroying a buffer while FlushInternal() is using it.
Race condition fixed:
1. FlushInternal() snapshots buffers and clears pending_batch_
2. FlushInternal() unlocks and starts IO
3. Free() called - buffer not in pending_batch_, not yet persisted
4. Free() destroys buffer -> FlushInternal() crashes on Persist()
Fix:
- FlushInternal() sets is_flushing_=true before unlock
- Free() calls WaitForFlushComplete() to wait for flush
- FlushInternal() sets is_flushing_=false after Persist()
Signed-off-by: Xingrui Yi <yixingrui@linux.alibaba.com>
* [Store] Fix data race in StorageBuffer between Persist and ReadTo
Add data_mutex_ to protect data_ vector access. Without this lock,
Persist() and ReadTo() can race:
- ReadTo() checks is_on_disk_=false, starts reading data_
- Persist() clears data_ and sets is_on_disk_=true
- ReadTo() accesses cleared data_ -> undefined behavior
The TieredBackend per-key lock doesn't prevent this because
FlushInternal() operates at StorageTier level without acquiring
TieredBackend locks.
Signed-off-by: Xingrui Yi <yixingrui@linux.alibaba.com>
* [Store] fix UAF race, CAS ordering, flush recovery, copier TOCTOU
- StorageTier: replace spin-wait WaitForFlushComplete with
condition_variable under batch_mutex_, eliminating the TOCTOU
window between Free() and FlushInternal()
- StorageTier: FlushInternal restores pending_batch_ on IO failure
instead of silently dropping entries
- TieredBackend::Commit: validate CAS version before tier->Commit()
and metadata_sync_callback() to prevent side effects on stale writes,
with re-check under entry write lock for concurrent race safety
- TieredBackend::Get: remove redundant out_version write
- DRAM->NVME copier: snapshot dst.buffer->data() once to avoid
TOCTOU with concurrent Persist()
- Add regression tests: CASFailureNoSideEffects,
CASFailureNoCallbackInvoked, ConcurrentFlushDeleteStress
Signed-off-by: Xingrui Yi <yixingrui@linux.alibaba.com>
* [Store] Add sync eviction mode and strict allocation with single-tier support
Implement configurable eviction modes (SYNC/ASYNC) and strict allocation
parameter to trigger immediate eviction on allocation failure. Fix single-tier
configuration to properly evict keys via DELETE action when no slow tier exists.
Key changes:
- Add EvictionMode enum (SYNC/ASYNC) to ClientScheduler with JSON config support
- Add strict parameter to TieredBackend::Allocate for tier-specific allocation
- Implement TriggerSyncEviction to immediately free space on allocation failure
- Fix LRU policy to generate DELETE actions for single-tier eviction scenarios
- Add comprehensive tests for capacity limits, sync eviction, and single-tier setup
Signed-off-by: Xingrui Yi <yixingrui@linux.alibaba.com>
* [Store] Add bucket-level eviction with fragmentation tracking
Implement bucket eviction functionality to reclaim space from fragmented
or old buckets in the storage tier. The eviction strategy prioritizes
buckets with >50% fragmentation, falling back to LRU for non-fragmented
buckets.
Key changes:
- Add SelectBucketForEviction() and EvictBucket() methods to BucketStorageBackend
- Track valid key count per bucket for fragmentation calculation
- Implement MarkKeyDeleted() to update fragmentation metrics on key deletion
- Add TriggerBucketEviction() to StorageTier for manual eviction
- Move Storage Tier tests to dedicated storage_tier_test.cpp file
- Add BucketEviction test to verify eviction functionality
Signed-off-by: Xingrui Yi <yixingrui@linux.alibaba.com>
* [Store] Improve bucket eviction with target-size and auto-eviction
Enhance bucket eviction to support target-based eviction and automatic
triggering when storage capacity is exceeded.
Key improvements:
- EvictBucket() now returns freed space size instead of void
- TriggerBucketEviction() accepts target_free_size parameter
- Loop eviction until target size is met (max 10 attempts)
- Auto-trigger eviction in Allocate() when capacity exceeded
- Update persisted_size_ after eviction to reflect freed space
- Add AutoEvictionOnCapacityExceeded test
This addresses the issue where single bucket eviction may not free
enough space, and enables automatic space reclamation when the
scheduler encounters capacity constraints.
Signed-off-by: Xingrui Yi <yixingrui@linux.alibaba.com>
---------
Signed-off-by: Xingrui Yi <yixingrui@linux.alibaba.com>
* Implement TransferDataToRemote and TransferDataFromRemote in DataManager
* resolve reviews
* parallel transfer request submit
* loopback
* [Store] Implement PeerClient async RPC and RDMA performance tests
Implement AsyncReadRemoteData/AsyncWriteRemoteData coroutine interfaces
for PeerClient, enabling concurrent RPC calls via collectAllPara. Add
comprehensive performance benchmarks comparing async vs sync at varying
concurrency levels, plus RDMA-enabled tests that measure real data
transfer throughput across different buffer sizes (4KB-1MB).
Key changes:
- peer_client.h/cpp: Add async single-key RPC interfaces using
async_simple::coro::Lazy, implement Connect with client pool,
wire sync/batch methods to delegate to async+syncAwait
- peer_client_perf_test.cpp: RPC-only benchmarks (PeerClientPerfTest)
and RDMA data transfer benchmarks (PeerClientRdmaPerfTest) with
MC_RDMA_DEVICE env var for single-NIC loopback filtering
- peer_client_test.cpp: Unit tests for PeerClient interfaces
- CMakeLists.txt: Add new test targets, temporarily exclude
master_client.cpp due to yalantinglibs v0.5.6 incompatibility
Generated with [Claude Code](https://claude.ai/code)
via [Happy](https://happy.engineering)
Co-Authored-By: Claude <noreply@anthropic.com>
Co-Authored-By: Happy <yesreply@happy.engineering>
* [CI/Build] Re-enable master_client.cpp and centralized_master_client.cpp
These files were temporarily excluded due to a compilation error with
GCC 10 where yalantinglibs' util::is_invocable cannot handle abstract
class types (WrappedMasterService). The issue does not occur with
GCC 11+ which is the target build environment.
Generated with [Claude Code](https://claude.ai/code)
via [Happy](https://happy.engineering)
Co-Authored-By: Claude <noreply@anthropic.com>
Co-Authored-By: Happy <yesreply@happy.engineering>
* [Store] Enhance RDMA perf tests: warmup, windowed concurrency, batch comparison
- Add warmup rounds to all RDMA tests to eliminate cold-start effects
(QP connection setup, path resolution) that skewed measurements
- Add window parameter to RunRdmaAsyncReads/Writes for windowed
concurrency via collectAllWindowedPara
- Add RdmaWindowedConcurrency test comparing window sizes {5,10,25,50,ALL}
- Add batch RPC helpers and RdmaSyncAsyncBatchComparison test for
three-way sync vs async vs batch comparison
- Change minimum test data size from 4KB to 32KB to focus on
meaningful transfer sizes where async consistently outperforms sync
Generated with [Claude Code](https://claude.ai/code)
via [Happy](https://happy.engineering)
Co-Authored-By: Claude <noreply@anthropic.com>
Co-Authored-By: Happy <yesreply@happy.engineering>
* [Store] Use median-of-5 sampling in RDMA perf tests for stability
Add MedianOf helper that runs each benchmark point 5 times and takes
the median, filtering out sporadic outliers caused by RDMA QP
contention, coroutine scheduling jitter, or shared pod noise. This
eliminates the occasional 0.03x-0.27x anomalies that appeared
randomly across different (size, N) combinations.
Generated with [Claude Code](https://claude.ai/code)
via [Happy](https://happy.engineering)
Co-Authored-By: Claude <noreply@anthropic.com>
Co-Authored-By: Happy <yesreply@happy.engineering>
* add errlog and remove batch api
---------
Co-authored-by: chenwenxiao <chenwenxiaolive@gmail.com>
Co-authored-by: qinwenzh <zhengqinwen4@huawei.com>
Co-authored-by: Claude <noreply@anthropic.com>
Co-authored-by: Happy <yesreply@happy.engineering>
- Fix AclMemcpyWithDevice to use dst.buffer->size() for dst_size parameter
(should be max allocated size, not copy size per ACL API spec)
- Change AscendBuffer::data() to return device_ptr instead of
AscendUnifiedPointer address for standard BufferBase semantics
- Update documentation to reflect data() behavior change
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
- Change AscendBuffer to accept std::unique_ptr<AscendUnifiedPointer>
- Change unified_ptr_ member from raw pointer to std::unique_ptr
- Update AllocateDeviceMemory to return std::unique_ptr
- Simplify move semantics using std::move
- Use unified_ptr_.reset() instead of manual delete
- Update tests to use std::make_unique
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
Wan