mooncake-track
/
dynamo
0
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity
3,795 Commits 2 Branches 0 Tags 76 MiB
Commit Graph

275 Commits

Author SHA1 Message Date
Yan Ru Pei 2e29620df1
chore(kv-router): move benches to lib/bench to break circular dep [OPS-3752] (#7013) 
Signed-off-by: PeaBrane <yanrpei@gmail.com>
 2026-03-06 23:24:03 +00:00
Graham King abc02c689f
fix: llm/mocker: Remove the llm -> mocker crate dependency, move config (#6998) 
Signed-off-by: Graham King <grahamk@nvidia.com>
Signed-off-by: PeaBrane <yanrpei@gmail.com>
 2026-03-06 09:41:39 -08:00
Yan Ru Pei f934744fb9
fix: restore --enforce-disagg to reject requests before prefill router activates (#6957) 
Signed-off-by: PeaBrane <yanrpei@gmail.com>
 2026-03-06 04:39:28 +00:00
GuanLuo 90d7463721
fix: properly setup and register vLLM worker for external / hybrid load balancing. Update launch script (#6695) 
Signed-off-by: Guan Luo <41310872+GuanLuo@users.noreply.github.com>
 2026-03-03 14:00:52 -08:00
Yan Ru Pei 8055b7dd2c
fix(kv-router): plumb expected_osl through scheduler queue path (#6812) 
Signed-off-by: PeaBrane <yanrpei@gmail.com>
 2026-03-03 10:57:42 -08:00
Yan Ru Pei 07742cc27f
feat: move standalone KV indexer into kv-router crate with HTTP integration tests (#6569) 
Signed-off-by: PeaBrane <yanrpei@gmail.com>
Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
 2026-03-02 10:34:21 -08:00
Michael Feil 713c96d26d
feat: dynamic batching of client side events (#6733) 
Signed-off-by: michaelfeil <63565275+michaelfeil@users.noreply.github.com>
 2026-02-28 18:27:57 +00:00
Yan Ru Pei 55c0a769b4
feat: default router_event_threads to 4 (#6672) 
Signed-off-by: PeaBrane <yanrpei@gmail.com>
 2026-02-27 22:14:47 +00:00
ishandhanani f1d5c95a4e
feat: wire nvext.cache_control TTL-based pinning through Dynamo router (#6213) 
Co-authored-by: Claude <noreply@anthropic.com>
 2026-02-27 01:03:27 +00:00
Yan Ru Pei ebaf048d16
chore: plumb allowed_worker_ids through RoutingHints (#6580) 
Signed-off-by: PeaBrane <yanrpei@gmail.com>
 2026-02-25 12:12:53 -08:00
atchernych c916cd42ff
feat: Support epp's "pods" interface in Dynamo fixes [DEP-424] (#6302) 
Signed-off-by: Anna Tchernych <atchernych@nvidia.com>
 2026-02-25 01:54:38 +00:00
Biswa Panda 75fea78797
feat: make KV cache events and routing LoRA-aware (#6517) 2026-02-24 18:35:08 +00:00
Yan Ru Pei c9ff623538
chore: replace --enforce-disagg with --decode-fallback, default to enforcing disagg (#6515) 
Signed-off-by: PeaBrane <yanrpei@gmail.com>
 2026-02-24 09:24:11 -08:00
Jacky c8276cd28b
feat: Standardized Dynamo Error Type (#6303) 
Signed-off-by: Jacky <18255193+kthui@users.noreply.github.com>
 2026-02-23 20:23:58 -08:00
Tzu-Ling Kan 80cac7c14b
feat: Remove Component from public (#6403) 
Signed-off-by: tzulingk@nvidia.com <tzulingk@nvidia.com>
 2026-02-23 22:58:38 +00:00
Thomas Montfort cb55766c47
feat(runtime): add hierarchical Model/WorkerSet architecture for multi-namespace support (#6054) 
Signed-off-by: tmontfort <tmontfort@nvidia.com>
 2026-02-23 22:44:38 +00:00
Yan Ru Pei 038b50d2b6
feat: add standalone KV indexer with query endpoint [DYN-2164] (#6446) 
Signed-off-by: PeaBrane <yanrpei@gmail.com>
Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
 2026-02-23 12:12:25 -08:00
Tzu-Ling Kan 42d6980546
feat: Remove public uses of CancellationToken (#6405) 
Signed-off-by: tzulingk@nvidia.com <tzulingk@nvidia.com>
 2026-02-20 21:01:13 -08:00
Tzu-Ling Kan 0ce3461a9e
feat: Add runtime.endpoint() method to eliminate namespace chaining (#6386) 
Signed-off-by: tzulingk@nvidia.com <tzulingk@nvidia.com>
 2026-02-19 16:02:37 -05:00
Yan Ru Pei fc229004b5
chore: Remove ZmqKvEventListener binding and rework standalone TRT-LLM example to use native Python ZMQ (#6164) 
Signed-off-by: PeaBrane <yanrpei@gmail.com>
 2026-02-19 04:15:13 +00:00
Tzu-Ling Kan bc8f1170ce
feat: Move ModelDeploymentCard to _internal (#6378) 
Signed-off-by: tzulingk@nvidia.com <tzulingk@nvidia.com>
 2026-02-18 23:12:18 +00:00
Ayush Agarwal 2ace5a4a3d
chore: multimodal endpoint registration via cli modality (#6270) 
Signed-off-by: ayushag <ayushag@nvidia.com>
 2026-02-18 07:59:39 +00:00
zhongdaor-nv 815b129126
feat: mm aware routing for vllm (#6235) 
Signed-off-by: zhongdaor <zhongdaor@nvidia.com>
Signed-off-by: zhongdaor-nv <zhongdaor@nvidia.com>
 2026-02-13 21:16:06 -08:00
Tzu-Ling Kan 5624d14481
Rename fetch_llm to fetch_model (#6268) 
Signed-off-by: tzulingk@nvidia.com <tzulingk@nvidia.com>
 2026-02-13 23:22:35 +00:00
mohammedabdulwahhab a289695c37
fix: consolidate dyn_discovery_backend and dyn_kv_store (#6167) 
Signed-off-by: mohammedabdulwahhab <furkhan324@berkeley.edu>
 2026-02-13 19:07:11 +00:00
ishandhanani 2be83be2d8
feat: add video generation support (T2V) (#5793) 2026-02-13 09:09:56 +00:00
Yan Ru Pei 14eceb43df
chore: rename KvPushRouter to KvRouter in python + more bindings removal (#6238) 
Signed-off-by: PeaBrane <yanrpei@gmail.com>
 2026-02-12 23:44:52 -08:00
atchernych 5227176057
feat: Decomposed pipeline for EPP integration [DEP-730] (#5446) 
Signed-off-by: Anna Tchernych <atchernych@nvidia.com>
 2026-02-13 06:18:29 +00:00
Yan Ru Pei bc514fbee6
feat: router priority queue (#6010) 
Signed-off-by: PeaBrane <yanrpei@gmail.com>
Co-authored-by: ishandhanani <82981111+ishandhanani@users.noreply.github.com>
 2026-02-13 02:43:17 +00:00
Karen Chung cd6984b951
feat: use RNG when dp routing targets are tied; override no-assume-kv-reuse for decode requests (#6253) 
Signed-off-by: Karen Chung <karenc@nvidia.com>
 2026-02-12 15:59:37 -08:00
Yan Ru Pei 937398cf61
feat: Flash Indexer (#5785) 
Signed-off-by: PeaBrane <yanrpei@gmail.com>
Signed-off-by: jthomson04 <jothomson@nvidia.com>
Signed-off-by: jthomson04 <jwillthomson19@gmail.com>
Signed-off-by: Janelle Cai <jcai18@mit.edu>
Co-authored-by: jthomson04 <jwillthomson19@gmail.com>
Co-authored-by: Janelle Cai <jcai18@mit.edu>
 2026-02-11 22:26:56 -08:00
Yan Ru Pei 6a728d1044
chore: remove kv indexers bindings (#6159) 
Signed-off-by: PeaBrane <yanrpei@gmail.com>
 2026-02-11 20:23:48 +00:00
Graham King 73dc3be81a
chore: Merge bindings client and client2 functions (#6158) 
Signed-off-by: Graham King <grahamk@nvidia.com>
 2026-02-11 11:51:01 -05:00
Yan Ru Pei f46720c996
feat: add router-level Prometheus metrics and centralize request tracking (#6146) 
Signed-off-by: PeaBrane <yanrpei@gmail.com>
Co-authored-by: Cursor <cursoragent@cursor.com>
 2026-02-11 07:36:08 +00:00
Graham King 4f99451bb0
feat(frontend): Use vllm for pre and post processing (#5544) 
Signed-off-by: Graham King <grahamk@nvidia.com>
 2026-02-10 22:39:11 +00:00
Ayush Agarwal 9f76d0606c
feat: text to image vLLM Omni (#5912) 
Signed-off-by: ayushag <ayushag@nvidia.com>
 2026-02-09 18:19:27 -05:00
Yan Ru Pei 6783bdcaa9
chore: enable local indexers by default, and use normal event plane by default (not jetstream) (#5941) 
Signed-off-by: PeaBrane <yanrpei@gmail.com>
 2026-02-08 20:48:02 +00:00
Jacky 1ffa489ea1
refactor: Move --migration-limit flag from backend to frontend (#5918) 
Signed-off-by: Jacky <18255193+kthui@users.noreply.github.com>
 2026-02-06 20:50:55 +00:00
Yan Ru Pei 3e41702211
chore: remove and unify bindings in kv.rs (#6016) 
Signed-off-by: PeaBrane <yanrpei@gmail.com>
 2026-02-06 18:44:46 +00:00
Biswa Panda 953e5d7beb
feat(lora): implement HRW-based LoRA allocation system (#5992) 2026-02-05 14:56:15 -08:00
Keiven C cb7ebdd7e8
chore: remove unused NIM specific code (part 2) (#5893) 
Signed-off-by: Keiven Chang <keivenchang@users.noreply.github.com>
Co-authored-by: Keiven Chang <keivenchang@users.noreply.github.com>
 2026-02-05 12:21:10 -08:00
Yan Ru Pei 91fb78cddf
chore: make mocker it's own crate (#5958) 
Signed-off-by: PeaBrane <yanrpei@gmail.com>
 2026-02-05 19:35:42 +00:00
Hongkuan Zhou 0c0336e6ea
feat: Add per-worker Prometheus metrics for router load monitoring (#5842) 
Signed-off-by: hongkuanz <hongkuanz@nvidia.com>
 2026-02-04 11:51:43 -08:00
Konrad Nowicki 7e970d44c9
feat: image diffusion with SGLang diffusion (#5609) 
Signed-off-by: Konrad Nowicki <knowicki@nvidia.com>
Co-authored-by: dagil-nvidia <dagil@nvidia.com>
 2026-02-03 21:19:21 +00:00
Yan Ru Pei a337113aa9
feat: prefill tokens threshold based on max num batched tokens frac (#5867) 
Signed-off-by: PeaBrane <yanrpei@gmail.com>
 2026-02-03 03:08:25 +00:00
Yan Ru Pei 902eabd9ec
feat: per dp rank gap detection (#5873) 
Signed-off-by: PeaBrane <yanrpei@gmail.com>
 2026-02-03 03:07:57 +00:00
Biswa Panda 373e76c1ae
feat(lora): Add lora_name tracking to scheduling and sequence management (#5875) 2026-02-03 00:59:09 +00:00
Janelle Cai 18d9d1fa4f
feat(mocker): pre-fetch model and staggered launches (#5871) 2026-02-03 00:46:23 +00:00
ishandhanani c5f5ab6056
fix: enable DP attention KV events for multi-node deployments (#5589) 
Co-authored-by: Claude Opus 4.5 <noreply@anthropic.com>
Co-authored-by: huitianbai <huitianbai@gmail.com>
Co-authored-by: Huitianqi Bai <huitianbai@users.noreply.github.com>
 2026-02-02 07:20:29 +00:00
Biswa Panda 1174c819cb
feat: remove deprecated events api (#5778) 2026-01-29 05:48:16 +00:00