mooncake-track
/
dynamo
0
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity
3,250 Commits 2 Branches 0 Tags 76 MiB
Commit Graph

43 Commits

Author SHA1 Message Date
Yan Ru Pei 6783bdcaa9
chore: enable local indexers by default, and use normal event plane by default (not jetstream) (#5941) 
Signed-off-by: PeaBrane <yanrpei@gmail.com>
 2026-02-08 20:48:02 +00:00
Jacky 1ffa489ea1
refactor: Move --migration-limit flag from backend to frontend (#5918) 
Signed-off-by: Jacky <18255193+kthui@users.noreply.github.com>
 2026-02-06 20:50:55 +00:00
Keiven C cb7ebdd7e8
chore: remove unused NIM specific code (part 2) (#5893) 
Signed-off-by: Keiven Chang <keivenchang@users.noreply.github.com>
Co-authored-by: Keiven Chang <keivenchang@users.noreply.github.com>
 2026-02-05 12:21:10 -08:00
Yan Ru Pei 91fb78cddf
chore: make mocker it's own crate (#5958) 
Signed-off-by: PeaBrane <yanrpei@gmail.com>
 2026-02-05 19:35:42 +00:00
Yan Ru Pei a337113aa9
feat: prefill tokens threshold based on max num batched tokens frac (#5867) 
Signed-off-by: PeaBrane <yanrpei@gmail.com>
 2026-02-03 03:08:25 +00:00
Yan Ru Pei 050906b593
feat: track output tokens / blocks in the Router (optional) (#5452) 
Signed-off-by: PeaBrane <yanrpei@gmail.com>
 2026-01-18 09:30:25 +08:00
Yan Ru Pei 7fdc742e17
feat: allow router not assuming decode kv reuse (#5350) 
Signed-off-by: PeaBrane <yanrpei@gmail.com>
 2026-01-13 07:33:47 +08:00
Tushar Sharma cf433e6825
chore: update all copyright headers in repo to 2026 (#5130) 
Signed-off-by: Tushar Sharma <tusharma@nvidia.com>
 2026-01-02 22:08:23 +00:00
Yan Ru Pei 0b33c1dfa0
fix: sglang disagg routing fixes and optimizations [DYN-1692] (#5106) 
Signed-off-by: PeaBrane <yanrpei@gmail.com>
Co-authored-by: Ishan Dhanani <ishandhanani@gmail.com>
Co-authored-by: Sean SH Choi <sechoi@nvidia.com>
Co-authored-by: ishandhanani <82981111+ishandhanani@users.noreply.github.com>
 2025-12-31 10:55:44 -08:00
Keiven C c22280ccf4
refactor: frontend/grpc tests to use dynamic ports (#4992) 
Signed-off-by: Keiven Chang <keivenchang@users.noreply.github.com>
Co-authored-by: Keiven Chang <keivenchang@users.noreply.github.com>
 2025-12-18 16:21:28 -08:00
Graham King da0f2fb84a
feat(frontend): First part of Python request handling (#4999) 
Signed-off-by: Graham King <grahamk@nvidia.com>
 2025-12-17 20:03:39 -05:00
Yan Ru Pei 10b01b4590
feat: early rejection based on active prefill tokens (#4837) 
Signed-off-by: PeaBrane <yanrpei@gmail.com>
 2025-12-11 23:16:02 +00:00
Graham King 0fc5273c11
refactor(llm): Rename EngineConfig::Static to InProcess (#4585) 
Signed-off-by: Graham King <grahamk@nvidia.com>
 2025-11-25 12:22:09 -05:00
Yan Ru Pei c61e0dd3cc
chore: merge KvIndexer and ApproxKvIndexer (#4500) 
Signed-off-by: PeaBrane <yanrpei@gmail.com>
 2025-11-21 09:09:11 -08:00
Yan Ru Pei ce83398366
chore: better error handling in prefill router (#4286) 
Signed-off-by: PeaBrane <yanrpei@gmail.com>
 2025-11-13 22:29:26 +00:00
Graham King e1af3af6ee
chore: Remove static mode (#4235) 
Signed-off-by: Graham King <grahamk@nvidia.com>
 2025-11-11 19:25:12 +00:00
Graham King 794c0a4431
feat(keyvalue): Filesystem backed KeyValueStore (#4138) 
Signed-off-by: Graham King <grahamk@nvidia.com>
 2025-11-07 21:11:33 +00:00
Yan Ru Pei 41ff394f51
chore: restructure mocker cli args handling, to include prefill/decode (#3847) 
Signed-off-by: PeaBrane <yanrpei@gmail.com>
 2025-10-23 10:41:23 -07:00
Yan Ru Pei ae4e96a2bc
fix: mocker engines should ignore downloading weights from hf (again) (#3664) 
Signed-off-by: PeaBrane <yanrpei@gmail.com>
 2025-10-16 02:48:40 +00:00
Graham King ab0da5821e
feat: Python binding to download a model. (#3593) 
Signed-off-by: Graham King <grahamk@nvidia.com>
 2025-10-15 12:26:02 -04:00
Keiven C 65cc533702
feat: implement custom backend metrics for NIM (#3266) 
Signed-off-by: Keiven Chang <keivenchang@users.noreply.github.com>
 2025-10-10 20:28:58 -07:00
Graham King 0e0218ff02
chore: Remove model_config from LocalModel (#3558) 
Signed-off-by: Graham King <grahamk@nvidia.com>
 2025-10-10 22:01:30 +00:00
Yan Ru Pei 3aa3077808
fix: more fixes for stable router benchmarking (#3264) 
Signed-off-by: PeaBrane <yanrpei@gmail.com>
 2025-09-29 23:08:56 +00:00
Yan Ru Pei 031590fc14
feat: vllm prefill router (#3155) 
Signed-off-by: PeaBrane <yanrpei@gmail.com>
 2025-09-22 16:38:56 -07:00
Graham King 7a5a0bd6cd
chore: Upgrade Rust to 1.90 (#3147) 
Signed-off-by: Graham King <grahamk@nvidia.com>
 2025-09-19 18:54:29 -04:00
Yan Ru Pei 5b19a39bab
feat: allow router to not track active blocks (prefill), and to not track cached blocks (decode) (#3135) 
Signed-off-by: PeaBrane <yanrpei@gmail.com>
 2025-09-19 20:33:07 +00:00
Graham King 87e6e0529d
fix: Interactive inputs actually stops, does not ignore stop token (#3057) 
Signed-off-by: Graham King <grahamk@nvidia.com>
 2025-09-16 14:39:05 -04:00
Biswa Panda c6becbc859
feat: dynamo namespace isolation (#2394) 
Signed-off-by: Biswa Panda <biswa.panda@gmail.com>
 2025-09-03 15:47:06 +00:00
Yan Ru Pei 7fabe7bfe2
fix: do not delete KV events jetstream (#2800) 
Signed-off-by: PeaBrane <yanrpei@gmail.com>
 2025-09-01 21:25:49 +00:00
Yan Ru Pei 488c87095c
feat: Router warm restarts via durable KV event consumers and radix snapshotting (#2756) 
Signed-off-by: PeaBrane <yanrpei@gmail.com>
 2025-08-30 23:42:57 +00:00
Yan Ru Pei f08729ae15
feat: python bindings for the entire KvPushRouter + per-request router configs (#2658) 2025-08-25 22:27:10 +00:00
Graham King 6a358f7c8c
chore(llm): Rename protocols::Endpoint to EndpointId (#2615) 2025-08-22 15:07:34 +00:00
suzu c5d9d26703
feat(frontend): support setting HTTP host via CLI (--http-host) (#2523) 2025-08-19 09:01:04 -04:00
Yan Ru Pei 85d8310806
feat: router-level request rejection (#2465) 2025-08-19 01:51:00 -07:00
Graham King a4bbe49228
feat(http): TLS support (#2492) 2025-08-18 16:06:29 -04:00
Jorge António d0a6363584
feat: add RuntimeConfig to ModelEntry (#2311) 
Co-authored-by: Yan Ru Pei <yanrpei@gmail.com>
 2025-08-14 10:19:53 -07:00
Yan Ru Pei 5166a3dd44
feat: Router replicas with state-sharing (#2264) 2025-08-07 22:57:57 +00:00
Graham King 6a1a801c2d
feat: Support static workers, run without etcd. (#2281) 2025-08-06 09:51:43 -04:00
Yan Ru Pei bae25dc6d4
feat: skip downloading model weights if using mocker (only tokenizer) (#2213) 2025-07-31 18:15:57 +00:00
Graham King fc12436048
feat(frontend): router-mode settings (#2001) 2025-07-18 18:52:57 +00:00
Graham King 182d3b5dc7
chore(bindings): Remove mistralrs / llama.cpp (#1970) 2025-07-16 16:12:40 -04:00
Yan Ru Pei f31732a22d
feat: integrate mocker with dynamo-run and python cli (#1927) 2025-07-16 18:22:15 +00:00
Graham King 2bf27924a1
feat(python): Python bindings for the Dynamo CLI tools (#1799) 2025-07-08 21:40:36 +00:00