forked from mooncake-track/Mooncake
62ef96e53b
* [Store] Make tiered scheduler incremental Rework the tiered backend scheduler to avoid a shared global stats lock and full keyspace scans in the background loop. Key changes: - shard stats collection and maintain ordered incremental indexes for Simple and LRU snapshots - split recent heat and recency rank semantics in the scheduler policy interface - build policy input from snapshot candidates plus fast-tier residents using scheduler-side metadata cache - hook scheduler cache maintenance into TieredBackend commit/delete paths - extend integration tests for concurrent stats, bounded snapshots, and wall-clock decay behavior Signed-off-by: Xingrui Yi <yixingrui@linux.alibaba.com> * [Store] Make bucket file eviction async Signed-off-by: Xingrui Yi <yixingrui@linux.alibaba.com> * [Store] Guard bucket offload with physical space Signed-off-by: Xingrui Yi <yixingrui@linux.alibaba.com> * [Store] Unify SSD backend space management Signed-off-by: Xingrui Yi <yixingrui@linux.alibaba.com> * [Store] Preallocate storage tier staging pool Signed-off-by: Xingrui Yi <yixingrui@linux.alibaba.com> * [Store] Make allocation handles own tier lifetime Signed-off-by: Xingrui Yi <yixingrui@linux.alibaba.com> * [Store] Add reclaim planner to tiered scheduler Signed-off-by: Xingrui Yi <yixingrui@linux.alibaba.com> * [Store] Unify tiered backend test init Fold repeated TieredBackend::Init test setup into a shared helper and guard storage-tier bucket eviction with a safe runtime cast so file-per-key tests no longer crash when capacity is exceeded. Signed-off-by: Xingrui Yi <yixingrui@linux.alibaba.com> * [Store] Surface scheduler policy errors Signed-off-by: Xingrui Yi <yixingrui@linux.alibaba.com> * [Store] Fix tensor API CI mode selection Signed-off-by: Xingrui Yi <yixingrui@linux.alibaba.com> * [Store] Fix bucket offload pre-init capacity check Signed-off-by: Xingrui Yi <yixingrui@linux.alibaba.com> * [Store] Fix storage path build warnings Signed-off-by: Xingrui Yi <yixingrui@linux.alibaba.com> * [Store] Unify tiered storage accounting semantics Clarify storage-tier live-byte accounting versus backend physical-byte accounting, reclaim per-key physical bytes on delete, and make per-key metadata scans idempotent without holding the accounting lock across the full directory walk. Signed-off-by: Xingrui Yi <yixingrui@linux.alibaba.com> * [Store] Shard scheduler key cache and surface alloc errors Shard the scheduler key cache to reduce lock contention, preserve allocation error codes across tier fallback, and tighten storage-tier tests around explicit overflow and capacity failures. Signed-off-by: Xingrui Yi <yixingrui@linux.alibaba.com> --------- Signed-off-by: Xingrui Yi <yixingrui@linux.alibaba.com> |
||
|---|---|---|
| .. | ||
| ascend_tier.h | ||
| cache_tier.h | ||
| dram_tier.h | ||
| storage_tier.h | ||