Commit Graph

9 Commits

Author SHA1 Message Date
Colors-111 788c1c737e
[Store] Fix: Auto-recovery for SSD Offload after Master Restart (#2077)
Co-authored-by: ruanzhao <ruanzhao@kingsoft.com>
2026-05-25 11:44:31 +08:00
Yaoming Zhan da70ac9f0c
[Store] L2->L1 promotion-on-hit (#2071) 2026-05-21 15:55:50 +08:00
lujh 3e7c78de9b
[Store] Fix segfault in disk-replica/offload paths when handling GPU pointers (#1892)
PutToLocalFile and FileStorage::OffloadObjects crash with SIGSEGV when
slice.ptr points to GPU device memory, because CPU memcpy cannot access
GPU virtual addresses. The RDMA memory-replica path is unaffected.

Add synchronous Device-to-Host staging via PinnedBufferPool before data
reaches the disk-write paths:

- New gpu_staging_utils.h: shared IsDevicePointer/CopyDeviceToHost/
  SetDevice helpers with cross-platform support (CUDA/HIP/MUSA/MACA/
  Ascend CANN)
- New PinnedBufferPool: thread-safe pinned host memory pool with
  max capacity limit (default 32) and O(1) swap-pop acquire
- PutToLocalFile: sync D2H on calling thread, PutRevoke on failure
- OffloadObjects: D2H staging before BatchOffload; on per-slice failure
  the entire object is skipped to prevent partial/corrupt data
- CMakeLists: auto-detect CUDAToolkit/HIP/Ascend independently of
  global USE_CUDA flag, with explicit PRIVATE compile definitions
- CI: add -lcudart to Go test CGO_LDFLAGS when CUDA is present

Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-20 11:51:53 +08:00
lujh c2573405bb
[Store] Support SSD Metrics (#1879) 2026-04-15 12:01:18 +08:00
zhangzuo21 e7f6bad7fa
[Store]Optimize uring file support for SSD offloading (#1562)
Co-authored-by: zhuxinjie-nz <240190801+zhuxinjie-nz@users.noreply.github.com>
2026-03-12 12:34:44 +08:00
zhangzuo21 4ff4dc1e27
[Store] Implement a file interface based on io uring to optimize storage backend. (#1500)
Co-authored-by: zhuxinjie-nz <240190801+zhuxinjie-nz@users.noreply.github.com>
2026-02-25 14:18:22 +08:00
zhuxinjie-nz 24c632f84a
feat(Store): Support get local ssd object (#1203)
Co-authored-by: zhang zuoyuan <zhangzuo21@mails.tsinghua.edu.cn>
2026-01-22 21:15:50 +08:00
zhangzuo21 926c63aec8
[Store] feat: Implement a unified storage interface to simplify integration and extension (#1185) 2025-12-17 22:33:05 +08:00
zhuxinjie-nz 455e11a1dc
[Store] feat: Implement a FileStorage component to manage the lifecycle of key-value data (#1031) 2025-12-09 15:00:57 +08:00