zbtrs2 56efccf73c ollama-kvcache: add Mooncake store proxy ... A long-lived gRPC service that owns one warm MooncakeDistributedStore handle and exposes it to the Go sidecar, which cannot link the Python bindings directly. Using the official store client keeps the integration on the supported API surface. KV snapshots are multi-GiB, so the proxy moves them by file path: PutFile reads a slot file and stores it, GetFile materialises an object straight into the slot directory, and the blob is copied at most once. Objects are striped into <=64 MiB chunks transferred in parallel with batch_put_from / batch_get_into, because single-object RDMA degrades for large objects while striped batches sustain full bandwidth. A pool of pre-registered (pinned) staging buffers amortises RDMA memory registration across operations. Eviction or lease-expiry races on read are reported as not-found so the caller falls back to recompute rather than failing. A filesystem backend (no master required) backs the local baseline, and a selftest verifies put/get round-trips and existence checks over both backends.		2026-06-29 01:51:08 +08:00
..
gen	ollama-kvcache: add Mooncake store proxy	2026-06-29 01:51:08 +08:00
store_proxy.py	ollama-kvcache: add Mooncake store proxy	2026-06-29 01:51:08 +08:00