mooncake-track
/
Mooncake
0
0
Fork 12
Code Issues Pull Requests 13 Actions Packages Projects Releases Wiki Activity

Add Ollama integration: cross-process KV-cache reuse via the Mooncake Store #5

Open
zbtrs2 wants to merge 9 commits from zbtrs2/Mooncake:feature/ollama-kvcache-bus into main
pull from: zbtrs2/Mooncake:feature/ollama-kvcache-bus
merge into: mooncake-track:main
Conversation 0 Commits 9 Files Changed 77 +10774 -2
zbtrs2 commented 2026-07-08 16:11:26 +08:00
First-time contributor
Copy Link

Summary

This adds an integration under mooncake-integration/ollama/ that lets multiple Ollama / llama.cpp processes share one long-prefix KV cache through the Mooncake Store. The first process to prefill a long context publishes its KV to the store; other processes - in a different process, on a different GPU, or on a different node - restore that KV instead of re-prefilling it. The integration is a sidecar plus a thin store proxy and two small, env-gated Ollama patches, so an unconfigured Ollama behaves exactly as before. All changes are confined to mooncake-integration/ollama/ (76 files); no existing Mooncake files are modified.

Motivation

Ollama is a common entry point for local models and multi-agent tooling, but it has no KV-cache sharing of its own: prefix reuse is confined to a single process's RAM and dies with the request. Mooncake already pools and shares KV for vLLM / SGLang / TRT-LLM. This integration extends that reuse to the local/edge multi-agent case (multiple coding agents sharing one repo context) and to Ollama, with a design that needs almost no change to Ollama itself.

What's included

All paths are relative to mooncake-integration/ollama/.

  • bridge/ - Go sidecar. Serves a gRPC KVCacheBus (Lookup/Prepare/Commit) and an HTTP/JSON gateway with a Prometheus /metrics endpoint. Pure Go (bridge/go.mod, module ollama-mooncake-bridge, Go 1.23).
    • bridge/internal/cachekey/ - GGUF metadata parser and chained per-block content hashing.
    • bridge/internal/prefixindex/ - cross-process radix tree of KV prefixes.
    • bridge/internal/arbiter/ - restore-vs-recompute cost arbiter with online learning.
    • bridge/internal/orchestrator/ - 3-stage Lookup -> Prepare(restore) -> Commit(save).
    • bridge/internal/store/ - pluggable backend: Mooncake via proxy or local file.
    • bridge/internal/llamabridge/ - Stage-1 llama.cpp /slots HTTP client.
    • bridge/internal/seqstate/ - Stage-2 cgo binding for llama_state_seq_*_ext / ON_DEVICE, behind the cgo_stage2 build tag.
  • store-proxy/ - Python gRPC wrapper that owns a single warm MooncakeDistributedStore handle. KV blobs are passed as file paths.
  • ollama-patches/ - two additive, env-gated diffs against ollama/ollama plus notes.
  • benchmarks/, deploy/, scripts/, docs/ - benchmark matrix, compose/Prometheus/Grafana, lifecycle scripts, REPORT.md + result JSONs.

Design

Three independently deployable processes: agents/patched Ollama -> bridge (Go sidecar: key + radix index + arbiter, 3-stage Lookup/Prepare/Commit) -> store-proxy (one warm store client, striped batch RDMA) -> Mooncake Store.

  • Content-addressed keys. Model digest, tokenizer hash, RoPE hash, KV dtype, KV layout, context length, block size fold into the key space. Prompt tokens are hashed per fixed-size block with a forward chain h_i = H(h_{i-1} || block_i). KV is never mixed across models/tokenizers; favors a miss over a mis-hit.
  • Radix prefix index. Cross-process analogue of RadixAttention referencing KV snapshots in the store; O(blocks) longest-prefix lookup, per-prefix hotness, LRU eviction.
  • Cost arbiter. Break-even bandwidth B* = (KV bytes/token) x (prefill tokens/s); learns effective restore bandwidth and prefill rate per model online (EWMA), restores only when cheaper, else recomputes.
  • Transport. KV split into <=64 MiB chunks via batch_put_from/batch_get_into over pre-registered pinned staging buffers. Slot files on /dev/shm.
  • Two stages. Stage 1 drives unmodified llama.cpp via /slots; Stage 2 is a cgo path keeping KV in device buffers (targets GPUDirect RDMA).

Results

Quoted from the submitted technical report and result artifacts; measured on 2x NVIDIA A100-80GB SXM (NVSwitch), 200 Gb/s InfiniBand, CUDA 12.x, CUDA llama.cpp, 7B-class GGUF models, and q8_0 KV unless noted designed. A 1.5B model is used only for the "restore is not worth it" arbiter check.

Scaling, 7B / 30k ctx, fresh namespace per point (measured):

agents prefill tokens (no share) prefill tokens (Mooncake) swarm throughput
1 28,942 28,942 0.36x
2 57,884 28,953 0.93x
4 115,768 28,975 1.00x
6 173,653 28,998 1.36x
8 231,535 29,018 1.51x

Single uncontended cross-GPU reuser: honest TTFT 5.99 s (cold) -> 1.83 s (restore), -69%. This isolates the restore path from multi-agent contention and shows the upper bound before the single store client / NIC starts serializing concurrent restores.

Cold-start end-to-end demo (6 agents, 30k context): no-sharing prefill 173,653 tokens -> 28,998 tokens with Mooncake, saving 144,655 tokens; total prefill drops by 83% and redundant prefill is eliminated. Mean TTFT improves from 4,279 ms to 2,934 ms, swarm throughput is 1.26x, and 5 of 6 agents hit the shared KV because agent 0 is the cold cache builder.

Warm RDMA store (same 6-agent shape, public prefix already committed): mean TTFT improves from 4.03 s to 2.88 s (-29%) and swarm throughput reaches 1.74x. This is the steady-state number for repeated work over the same repository context; the cold-start number above is the first-round cost that includes cache creation.

Stage-2 KV-state export microbench: 7B/16k, 875 MiB KV, /slots file path 417 ms vs in-process llama_state_seq_get/set_data_ext 91 ms; 1.5B/8k, 219 MiB KV, 105 ms vs 19 ms. Both sizes pass round-trip correctness by continuing decode and matching the next-token argmax. On-device GPUDirect transfer through the Transfer Engine is designed, not yet wired (needs C++ headers absent from the Python wheel).

How to test

Go unit tests run anywhere (no GPU/RDMA) and pass:

cd mooncake-integration/ollama/bridge && go test ./...
# ok internal/arbiter ; ok internal/cachekey ; ok internal/prefixindex

Full stack + benchmarks need the A100 + InfiniBand setup above, CUDA llama.cpp, the model artifacts, and a reachable Mooncake Store; scripts keep state under $OMB_STATE:
bash scripts/setup_*.sh, bash scripts/demo.sh 7b 6, python benchmarks/run_matrix.py ..., bash scripts/stack_down.sh.

Notes / limitations

  • Only the Go unit tests run without a GPU. Demo / matrix / Stage-2 microbench require the measured A100 + InfiniBand setup, CUDA llama.cpp, and RDMA-capable Mooncake Store; numbers above are from that setup.
  • Stage-1 is bounded by llama.cpp's file restore (not GPUDirect); Stage 2 targets this gap (on-device GPUDirect path designed, not wired - missing wheel headers).
  • A single store client / NIC serializes concurrent restores, so swarm-mean TTFT trails the single-reuser number until the store is scaled.
  • On a tiny model the file path can be slower than a local prefill; the arbiter detects this and declines.
  • Cross-model sharing is out of scope by construction.
  • Ollama patch 0002 (env-gated --slot-save-path) is a clean standalone upstream candidate; 0001 adds a self-contained options.mooncake.* client without modifying upstream symbols.
## Summary This adds an integration under `mooncake-integration/ollama/` that lets multiple Ollama / llama.cpp processes share one long-prefix KV cache through the Mooncake Store. The first process to prefill a long context publishes its KV to the store; other processes - in a different process, on a different GPU, or on a different node - restore that KV instead of re-prefilling it. The integration is a sidecar plus a thin store proxy and two small, env-gated Ollama patches, so an unconfigured Ollama behaves exactly as before. All changes are confined to `mooncake-integration/ollama/` (76 files); no existing Mooncake files are modified. ## Motivation Ollama is a common entry point for local models and multi-agent tooling, but it has no KV-cache sharing of its own: prefix reuse is confined to a single process's RAM and dies with the request. Mooncake already pools and shares KV for vLLM / SGLang / TRT-LLM. This integration extends that reuse to the local/edge multi-agent case (multiple coding agents sharing one repo context) and to Ollama, with a design that needs almost no change to Ollama itself. ## What's included All paths are relative to `mooncake-integration/ollama/`. - **`bridge/`** - Go sidecar. Serves a gRPC `KVCacheBus` (Lookup/Prepare/Commit) and an HTTP/JSON gateway with a Prometheus `/metrics` endpoint. Pure Go (`bridge/go.mod`, module `ollama-mooncake-bridge`, Go 1.23). - `bridge/internal/cachekey/` - GGUF metadata parser and chained per-block content hashing. - `bridge/internal/prefixindex/` - cross-process radix tree of KV prefixes. - `bridge/internal/arbiter/` - restore-vs-recompute cost arbiter with online learning. - `bridge/internal/orchestrator/` - 3-stage Lookup -> Prepare(restore) -> Commit(save). - `bridge/internal/store/` - pluggable backend: Mooncake via proxy or local file. - `bridge/internal/llamabridge/` - Stage-1 llama.cpp `/slots` HTTP client. - `bridge/internal/seqstate/` - Stage-2 cgo binding for `llama_state_seq_*_ext` / `ON_DEVICE`, behind the `cgo_stage2` build tag. - **`store-proxy/`** - Python gRPC wrapper that owns a single warm `MooncakeDistributedStore` handle. KV blobs are passed as file paths. - **`ollama-patches/`** - two additive, env-gated diffs against `ollama/ollama` plus notes. - **`benchmarks/`**, **`deploy/`**, **`scripts/`**, **`docs/`** - benchmark matrix, compose/Prometheus/Grafana, lifecycle scripts, REPORT.md + result JSONs. ## Design Three independently deployable processes: agents/patched Ollama -> bridge (Go sidecar: key + radix index + arbiter, 3-stage Lookup/Prepare/Commit) -> store-proxy (one warm store client, striped batch RDMA) -> Mooncake Store. - **Content-addressed keys.** Model digest, tokenizer hash, RoPE hash, KV dtype, KV layout, context length, block size fold into the key space. Prompt tokens are hashed per fixed-size block with a forward chain `h_i = H(h_{i-1} || block_i)`. KV is never mixed across models/tokenizers; favors a miss over a mis-hit. - **Radix prefix index.** Cross-process analogue of RadixAttention referencing KV snapshots in the store; O(blocks) longest-prefix lookup, per-prefix hotness, LRU eviction. - **Cost arbiter.** Break-even bandwidth `B* = (KV bytes/token) x (prefill tokens/s)`; learns effective restore bandwidth and prefill rate per model online (EWMA), restores only when cheaper, else recomputes. - **Transport.** KV split into <=64 MiB chunks via `batch_put_from`/`batch_get_into` over pre-registered pinned staging buffers. Slot files on `/dev/shm`. - **Two stages.** Stage 1 drives unmodified llama.cpp via `/slots`; Stage 2 is a cgo path keeping KV in device buffers (targets GPUDirect RDMA). ## Results Quoted from the submitted technical report and result artifacts; **measured** on 2x NVIDIA A100-80GB SXM (NVSwitch), 200 Gb/s InfiniBand, CUDA 12.x, CUDA llama.cpp, 7B-class GGUF models, and q8_0 KV unless noted **designed**. A 1.5B model is used only for the "restore is not worth it" arbiter check. Scaling, 7B / 30k ctx, fresh namespace per point (measured): | agents | prefill tokens (no share) | prefill tokens (Mooncake) | swarm throughput | |---|---|---|---| | 1 | 28,942 | 28,942 | 0.36x | | 2 | 57,884 | 28,953 | 0.93x | | 4 | 115,768 | 28,975 | 1.00x | | 6 | 173,653 | 28,998 | 1.36x | | 8 | 231,535 | 29,018 | 1.51x | Single uncontended cross-GPU reuser: honest TTFT 5.99 s (cold) -> 1.83 s (restore), -69%. This isolates the restore path from multi-agent contention and shows the upper bound before the single store client / NIC starts serializing concurrent restores. Cold-start end-to-end demo (6 agents, 30k context): no-sharing prefill 173,653 tokens -> 28,998 tokens with Mooncake, saving 144,655 tokens; total prefill drops by 83% and redundant prefill is eliminated. Mean TTFT improves from 4,279 ms to 2,934 ms, swarm throughput is 1.26x, and 5 of 6 agents hit the shared KV because agent 0 is the cold cache builder. Warm RDMA store (same 6-agent shape, public prefix already committed): mean TTFT improves from 4.03 s to 2.88 s (-29%) and swarm throughput reaches 1.74x. This is the steady-state number for repeated work over the same repository context; the cold-start number above is the first-round cost that includes cache creation. Stage-2 KV-state export microbench: 7B/16k, 875 MiB KV, `/slots` file path 417 ms vs in-process `llama_state_seq_get/set_data_ext` 91 ms; 1.5B/8k, 219 MiB KV, 105 ms vs 19 ms. Both sizes pass round-trip correctness by continuing decode and matching the next-token argmax. On-device GPUDirect transfer through the Transfer Engine is **designed**, not yet wired (needs C++ headers absent from the Python wheel). ## How to test Go unit tests run anywhere (no GPU/RDMA) and pass: ``` cd mooncake-integration/ollama/bridge && go test ./... # ok internal/arbiter ; ok internal/cachekey ; ok internal/prefixindex ``` Full stack + benchmarks need the A100 + InfiniBand setup above, CUDA llama.cpp, the model artifacts, and a reachable Mooncake Store; scripts keep state under `$OMB_STATE`: `bash scripts/setup_*.sh`, `bash scripts/demo.sh 7b 6`, `python benchmarks/run_matrix.py ...`, `bash scripts/stack_down.sh`. ## Notes / limitations - Only the Go unit tests run without a GPU. Demo / matrix / Stage-2 microbench require the measured A100 + InfiniBand setup, CUDA llama.cpp, and RDMA-capable Mooncake Store; numbers above are from that setup. - Stage-1 is bounded by llama.cpp's file restore (not GPUDirect); Stage 2 targets this gap (on-device GPUDirect path designed, not wired - missing wheel headers). - A single store client / NIC serializes concurrent restores, so swarm-mean TTFT trails the single-reuser number until the store is scaled. - On a tiny model the file path can be slower than a local prefill; the arbiter detects this and declines. - Cross-model sharing is out of scope by construction. - Ollama patch `0002` (env-gated `--slot-save-path`) is a clean standalone upstream candidate; `0001` adds a self-contained `options.mooncake.*` client without modifying upstream symbols.
zbtrs2 added 9 commits 2026-07-08 16:11:27 +08:00
2febfc17e6 ollama-kvcache: add cache-key, prefix index and cost arbiter 
Introduce the algorithmic core of the Ollama KVCache Bus, an integration
that lets independent Ollama/llama.cpp processes share prompt KV through
the Mooncake Store.

cachekey derives a content-addressed key from a model fingerprint (model
digest, tokenizer/RoPE hashes parsed from GGUF metadata, KV dtype/layout,
context length, block size) and a forward-chained per-block hash of the
prompt tokens, where h_i = H(h_{i-1} || block_i). Chaining makes
longest-prefix matching exact: two prompts agree on h_i iff they share
every token of blocks 0..i. KV produced under different model parameters
lands in a disjoint key space and is never reused.

prefixindex maintains an in-memory radix tree over those chained hashes,
the cross-process analogue of a RadixAttention tree whose nodes reference
remote KV snapshots. It answers longest-prefix lookups in O(blocks) and
tracks per-prefix fan-out (hotness) and recency for replication and
eviction decisions.

arbiter decides restore-vs-recompute per request. A cached prefix only
helps when the store can deliver its KV faster than the GPU regenerates
it; the arbiter estimates both from rates it learns online (EWMA, seeded
on first observation) per model, and restores only when it is cheaper.
This keeps KV sharing loss-free across hardware regimes.

Includes unit tests for hash determinism, prefix consistency, model
separation, longest-prefix matching, eviction ordering and the arbiter's
adapt-to-recompute behaviour.
4927b5a694 ollama-kvcache: add gRPC contracts and store backends 
Define the two service contracts and the object-store abstraction the
orchestrator builds on.

bridge.proto is the agent-facing KVCacheBus: Lookup (read-only longest
prefix match plus arbiter decision), Prepare (Lookup plus restore of the
matched KV into a llama.cpp slot) and Commit (save the slot KV back to
the store). storeproxy.proto is the sidecar-to-proxy contract; KV blobs
are addressed by file path so a multi-GiB snapshot is copied at most once
and never streams through the Go process.

The store package exposes a small Backend interface with two
implementations: a Mooncake backend that forwards file-path Put/Get and
batched existence checks to the Python store proxy over gRPC, and a
local filesystem backend used as the cross-process baseline and a
dependency-free fallback. Generated stubs are checked in so the module
builds without protoc.
0b208be4e6 ollama-kvcache: add sidecar orchestrator, llama.cpp client and server 
Wire the pieces into the bridged daemon.

orchestrator implements the three-stage reuse flow. Lookup builds the
per-block keys and finds the longest stored prefix with a single batched
existence query (correct across nodes). Prepare additionally restores the
matched KV into a target llama.cpp slot when the arbiter approves, so the
server only prefills the uncached tail. Commit saves the slot KV and
stores it under the block key for the largest block-aligned prefix, with
skip-if-exists giving single-writer dedup when concurrent agents share a
prefix. It feeds the arbiter the full restore wall time and the observed
prefill rate so estimates self-calibrate, and tracks per-model KV
bytes/token for transfer sizing.

llamabridge drives the stock llama.cpp server over HTTP: /tokenize,
/completion (reading timings.prompt_n to count avoided prefill),
/slots save|restore|erase and /props. No llama.cpp source change is
needed for this path.

metrics exposes Prometheus instrumentation; the headline series is the
cumulative count of prompt tokens not re-prefilled thanks to reuse.
server adapts the orchestrator onto the gRPC KVCacheBus and an HTTP/JSON
gateway, and cmd/bridged is the daemon entrypoint serving gRPC (TCP and
unix socket), the JSON gateway and /metrics.
56efccf73c ollama-kvcache: add Mooncake store proxy 
A long-lived gRPC service that owns one warm MooncakeDistributedStore
handle and exposes it to the Go sidecar, which cannot link the Python
bindings directly. Using the official store client keeps the integration
on the supported API surface.

KV snapshots are multi-GiB, so the proxy moves them by file path: PutFile
reads a slot file and stores it, GetFile materialises an object straight
into the slot directory, and the blob is copied at most once. Objects are
striped into <=64 MiB chunks transferred in parallel with
batch_put_from / batch_get_into, because single-object RDMA degrades for
large objects while striped batches sustain full bandwidth. A pool of
pre-registered (pinned) staging buffers amortises RDMA memory
registration across operations. Eviction or lease-expiry races on read
are reported as not-found so the caller falls back to recompute rather
than failing.

A filesystem backend (no master required) backs the local baseline, and
a selftest verifies put/get round-trips and existence checks over both
backends.
0b18e65193 ollama-kvcache: add Stage-2 on-device KV state path 
The Stage-1 path serialises a sequence's KV through host memory and a
file (llama.cpp /slots save), which is several times slower than a raw
in-process copy because of the double copy described in llama.cpp issue
8915. Stage 2 targets the high-performance route.

seqstate is a cgo binding (build tag cgo_stage2) for
llama_state_seq_get/set_data_ext. With LLAMA_STATE_SEQ_FLAGS_ON_DEVICE
the KV tensors stay in device buffers, ready to be registered with the
Mooncake Transfer Engine for GPUDirect RDMA with no host copy. It is
gated behind a build tag because it links libllama and only applies to
the embedded deployment; the default HTTP path needs none of it.

omb_kvbench is an in-process microbenchmark against libllama that
compares the file-save path, the raw host export and the on-device size,
and verifies that a sequence's KV round-trips correctly (export seq 0,
import into seq 1, identical next-token argmax). It quantifies the
file-path overhead the on-device route removes.
e31c2de9b8 ollama-kvcache: add Ollama integration patches 
Two small, additive patches that make upstream Ollama a participant of
the KVCache Bus, verified against ollama/ollama main at 1abd56b.

0001 adds a self-contained HTTP client for the sidecar's prepare/commit
endpoints plus parsing of the options.mooncake.* request extension
(enable, namespace, read, write, block_size, transport, replica_num,
soft_pin), without touching any existing symbol.

0002 injects --slot-save-path into the bundled llama-server command line
when OLLAMA_MOONCAKE_SLOT_SAVE_PATH is set, exposing the /slots
save|restore endpoints the sidecar drives. Both are gated, so an
unconfigured Ollama behaves exactly as before. The README documents how
to apply them and the one call-site hook left as a reference, and notes
0002 as a clean upstream-PR candidate aligned with Ollama issue 14872.
897a11bd34 ollama-kvcache: add benchmarks and orchestration scripts 
agent_swarm drives the core scenario: several agents share a long repo
context and differ only in a short instruction. It runs the workload with
sharing off (baseline) and on, erasing llama's own per-slot cache between
runs so the comparison isolates cross-process reuse, and reports honest
end-to-end TTFT, recomputed prefill tokens, hit rate and throughput.
smoke_e2e is a minimal two-agent cross-GPU reuse check; microbench_store
and microbench_striped measure raw and striped store bandwidth;
run_matrix sweeps a config and plot renders the figures.

scripts/env.sh keeps every toolchain, cache and runtime directory under
an out-of-tree state root so the source stays clean and nothing is
written under /. The setup_* scripts install a local Go toolchain, a
venv with mooncake-transfer-engine, a CUDA build of llama.cpp and the
GGUF models; the *_start/stop and demo scripts bring the stack up and
down, scoped to this integration's own ports and paths.
fd7bbf720a ollama-kvcache: add deployment, observability and docs 
deploy packages the topology as docker-compose (master, store proxy,
sidecar, patched Ollama, Prometheus, Grafana) with the RDMA/GPUDirect
host passthrough it needs, a Prometheus scrape config and a Grafana
dashboard whose primary panel integrates the live count of avoided
prefill tokens.

docs/REPORT.md is the full write-up: design, what is real versus
simulated, the measured store bandwidth, the multi-agent results, the
adaptive-arbiter behaviour, the Stage-2 microbenchmark and the honest
limitations. The README is the entry point and quickstart, and the
figures and result JSONs are the measured data behind the report.
Auto Label PRs / triage (pull_request) Failing after 1m19s Details
Build & Test (Linux) / build (3.10) (pull_request) Has been cancelled Details
Build & Test (Linux) / build (3.12) (pull_request) Has been cancelled Details
Build & Test (Linux) / build-musa (pull_request) Has been cancelled Details
Build & Test (Linux) / test-wheel-ubuntu (3.10, ubuntu-22.04) (pull_request) Has been cancelled Details
Build & Test (Linux) / test-wheel-ubuntu (3.10, ubuntu-24.04) (pull_request) Has been cancelled Details
Build & Test (Linux) / test-wheel-ubuntu (3.12, ubuntu-22.04) (pull_request) Has been cancelled Details
Build & Test (Linux) / test-wheel-ubuntu (3.12, ubuntu-24.04) (pull_request) Has been cancelled Details
Build & Test (Linux) / build-flags (3.10) (pull_request) Has been cancelled Details
Build & Test (Linux) / build-flags (3.12) (pull_request) Has been cancelled Details
Build & Test (Linux) / Build Docker Image (pull_request) Has been cancelled Details
Build & Test (Linux) / Spell Check with Typos (pull_request) Has been cancelled Details
Build & Test (Linux) / Check code format (pull_request) Has been cancelled Details
Build & Test (Linux) / check-paths (pull_request) Has been cancelled Details
Build & Test (Linux) / build-wheel-cu13 (pull_request) Has been cancelled Details
Build & Test (Linux) / ascend-test (pull_request) Has been cancelled Details
Build & Test (Linux) / integration-test (pull_request) Has been cancelled Details
Build & Test (Linux) / CI Gate (pull_request) Has been cancelled Details
ca8674f51b test(store): normalize allocator test headings
Some checks failed
Auto Label PRs / triage (pull_request) Failing after 1m19s
Details
Build & Test (Linux) / build (3.10) (pull_request) Has been cancelled
Details
Build & Test (Linux) / build (3.12) (pull_request) Has been cancelled
Details
Build & Test (Linux) / build-musa (pull_request) Has been cancelled
Details
Build & Test (Linux) / test-wheel-ubuntu (3.10, ubuntu-22.04) (pull_request) Has been cancelled
Details
Build & Test (Linux) / test-wheel-ubuntu (3.10, ubuntu-24.04) (pull_request) Has been cancelled
Details
Build & Test (Linux) / test-wheel-ubuntu (3.12, ubuntu-22.04) (pull_request) Has been cancelled
Details
Build & Test (Linux) / test-wheel-ubuntu (3.12, ubuntu-24.04) (pull_request) Has been cancelled
Details
Build & Test (Linux) / build-flags (3.10) (pull_request) Has been cancelled
Details
Build & Test (Linux) / build-flags (3.12) (pull_request) Has been cancelled
Details
Build & Test (Linux) / Build Docker Image (pull_request) Has been cancelled
Details
Build & Test (Linux) / Spell Check with Typos (pull_request) Has been cancelled
Details
Build & Test (Linux) / Check code format (pull_request) Has been cancelled
Details
Build & Test (Linux) / check-paths (pull_request) Has been cancelled
Details
Build & Test (Linux) / build-wheel-cu13 (pull_request) Has been cancelled
Details
Build & Test (Linux) / ascend-test (pull_request) Has been cancelled
Details
Build & Test (Linux) / integration-test (pull_request) Has been cancelled
Details
Build & Test (Linux) / CI Gate (pull_request) Has been cancelled
Details
This pull request can be merged automatically.
You are not authorized to merge this pull request.
You can also view command line instructions.

Step 1:

From your project repository, check out a new branch and test the changes.
git checkout -b zbtrs2-feature/ollama-kvcache-bus main
git pull feature/ollama-kvcache-bus

Step 2:

Merge the changes and update on Gitea.
git checkout main
git merge --no-ff zbtrs2-feature/ollama-kvcache-bus
git push origin main
Sign in to join this conversation.
Reviewers
No reviewers
Labels
Clear labels
No items
No Label
Milestone
Clear milestone
No items
No Milestone
Projects
Clear projects
No project
Assignees
Clear assignees
No Assignees
1 Participants
Notifications
Due Date
The due date is invalid or out of range. Please use the format 'yyyy-mm-dd'.

No due date set.

Dependencies

No dependencies set.

Reference: mooncake-track/Mooncake#5
No description provided.
Delete Branch "zbtrs2/Mooncake:feature/ollama-kvcache-bus"

Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?