forked from mooncake-track/Mooncake
15 lines
601 B
Plaintext
15 lines
601 B
Plaintext
$ bash scripts/demo.sh 7b 6
|
|
...
|
|
================================================================
|
|
SCOREBOARD
|
|
================================================================
|
|
agents sharing context : 6
|
|
prefill tokens, no sharing : 173,653
|
|
prefill tokens, Mooncake : 28,998 (144,655 saved)
|
|
total prefill work avoided : 83%
|
|
redundant prefill killed : 100% (only the pioneer must prefill)
|
|
mean TTFT : 4279 ms -> 2934 ms
|
|
swarm throughput : 1.26x
|
|
cache hits : 5/6
|
|
================================================================
|