Yan Ru Pei
|
bddaaa2658
|
feat(kv-router): pluggable scheduling policy for router queue [DYN-2454] (#7260)
Signed-off-by: PeaBrane <yanrpei@gmail.com>
|
2026-03-13 04:16:49 +00:00 |
|
Yan Ru Pei
|
f80ad6926a
|
feat(kv-router): package dynamo-kv-indexer binary via maturin [LLM-126] (#7194)
Signed-off-by: PeaBrane <yanrpei@gmail.com>
|
2026-03-12 21:33:11 +00:00 |
|
Hongkuan Zhou
|
cd4773fb31
|
feat: ForwardPassMetrics dynamo event plane integration (#7250)
Signed-off-by: hongkuanz <hongkuanz@nvidia.com>
|
2026-03-12 20:58:01 +00:00 |
|
Michael Feil
|
fca0a801dc
|
feat: follow up of optinal, batching-of-client-side-events (#6741)
Signed-off-by: michaelfeil <63565275+michaelfeil@users.noreply.github.com>
Signed-off-by: Michael Feil <michaelfeil@users.noreply.github.com>
Signed-off-by: Michael Feil <63565275+michaelfeil@users.noreply.github.com>
|
2026-03-11 23:30:51 +00:00 |
|
Yan Ru Pei
|
2e29620df1
|
chore(kv-router): move benches to lib/bench to break circular dep [OPS-3752] (#7013)
Signed-off-by: PeaBrane <yanrpei@gmail.com>
|
2026-03-06 23:24:03 +00:00 |
|
Graham King
|
abc02c689f
|
fix: llm/mocker: Remove the llm -> mocker crate dependency, move config (#6998)
Signed-off-by: Graham King <grahamk@nvidia.com>
Signed-off-by: PeaBrane <yanrpei@gmail.com>
|
2026-03-06 09:41:39 -08:00 |
|
Yan Ru Pei
|
f934744fb9
|
fix: restore --enforce-disagg to reject requests before prefill router activates (#6957)
Signed-off-by: PeaBrane <yanrpei@gmail.com>
|
2026-03-06 04:39:28 +00:00 |
|
GuanLuo
|
90d7463721
|
fix: properly setup and register vLLM worker for external / hybrid load balancing. Update launch script (#6695)
Signed-off-by: Guan Luo <41310872+GuanLuo@users.noreply.github.com>
|
2026-03-03 14:00:52 -08:00 |
|
Yan Ru Pei
|
8055b7dd2c
|
fix(kv-router): plumb expected_osl through scheduler queue path (#6812)
Signed-off-by: PeaBrane <yanrpei@gmail.com>
|
2026-03-03 10:57:42 -08:00 |
|
Yan Ru Pei
|
07742cc27f
|
feat: move standalone KV indexer into kv-router crate with HTTP integration tests (#6569)
Signed-off-by: PeaBrane <yanrpei@gmail.com>
Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
|
2026-03-02 10:34:21 -08:00 |
|
Michael Feil
|
713c96d26d
|
feat: dynamic batching of client side events (#6733)
Signed-off-by: michaelfeil <63565275+michaelfeil@users.noreply.github.com>
|
2026-02-28 18:27:57 +00:00 |
|
Yan Ru Pei
|
55c0a769b4
|
feat: default router_event_threads to 4 (#6672)
Signed-off-by: PeaBrane <yanrpei@gmail.com>
|
2026-02-27 22:14:47 +00:00 |
|
ishandhanani
|
f1d5c95a4e
|
feat: wire nvext.cache_control TTL-based pinning through Dynamo router (#6213)
Co-authored-by: Claude <noreply@anthropic.com>
|
2026-02-27 01:03:27 +00:00 |
|
Yan Ru Pei
|
ebaf048d16
|
chore: plumb allowed_worker_ids through RoutingHints (#6580)
Signed-off-by: PeaBrane <yanrpei@gmail.com>
|
2026-02-25 12:12:53 -08:00 |
|
atchernych
|
c916cd42ff
|
feat: Support epp's "pods" interface in Dynamo fixes [DEP-424] (#6302)
Signed-off-by: Anna Tchernych <atchernych@nvidia.com>
|
2026-02-25 01:54:38 +00:00 |
|
Biswa Panda
|
75fea78797
|
feat: make KV cache events and routing LoRA-aware (#6517)
|
2026-02-24 18:35:08 +00:00 |
|
Yan Ru Pei
|
c9ff623538
|
chore: replace --enforce-disagg with --decode-fallback, default to enforcing disagg (#6515)
Signed-off-by: PeaBrane <yanrpei@gmail.com>
|
2026-02-24 09:24:11 -08:00 |
|
Jacky
|
c8276cd28b
|
feat: Standardized Dynamo Error Type (#6303)
Signed-off-by: Jacky <18255193+kthui@users.noreply.github.com>
|
2026-02-23 20:23:58 -08:00 |
|
Tzu-Ling Kan
|
80cac7c14b
|
feat: Remove Component from public (#6403)
Signed-off-by: tzulingk@nvidia.com <tzulingk@nvidia.com>
|
2026-02-23 22:58:38 +00:00 |
|
Thomas Montfort
|
cb55766c47
|
feat(runtime): add hierarchical Model/WorkerSet architecture for multi-namespace support (#6054)
Signed-off-by: tmontfort <tmontfort@nvidia.com>
|
2026-02-23 22:44:38 +00:00 |
|
Yan Ru Pei
|
038b50d2b6
|
feat: add standalone KV indexer with query endpoint [DYN-2164] (#6446)
Signed-off-by: PeaBrane <yanrpei@gmail.com>
Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
|
2026-02-23 12:12:25 -08:00 |
|
Tzu-Ling Kan
|
42d6980546
|
feat: Remove public uses of CancellationToken (#6405)
Signed-off-by: tzulingk@nvidia.com <tzulingk@nvidia.com>
|
2026-02-20 21:01:13 -08:00 |
|
Tzu-Ling Kan
|
0ce3461a9e
|
feat: Add runtime.endpoint() method to eliminate namespace chaining (#6386)
Signed-off-by: tzulingk@nvidia.com <tzulingk@nvidia.com>
|
2026-02-19 16:02:37 -05:00 |
|
Yan Ru Pei
|
fc229004b5
|
chore: Remove ZmqKvEventListener binding and rework standalone TRT-LLM example to use native Python ZMQ (#6164)
Signed-off-by: PeaBrane <yanrpei@gmail.com>
|
2026-02-19 04:15:13 +00:00 |
|
Tzu-Ling Kan
|
bc8f1170ce
|
feat: Move ModelDeploymentCard to _internal (#6378)
Signed-off-by: tzulingk@nvidia.com <tzulingk@nvidia.com>
|
2026-02-18 23:12:18 +00:00 |
|
Ayush Agarwal
|
2ace5a4a3d
|
chore: multimodal endpoint registration via cli modality (#6270)
Signed-off-by: ayushag <ayushag@nvidia.com>
|
2026-02-18 07:59:39 +00:00 |
|
zhongdaor-nv
|
815b129126
|
feat: mm aware routing for vllm (#6235)
Signed-off-by: zhongdaor <zhongdaor@nvidia.com>
Signed-off-by: zhongdaor-nv <zhongdaor@nvidia.com>
|
2026-02-13 21:16:06 -08:00 |
|
Tzu-Ling Kan
|
5624d14481
|
Rename fetch_llm to fetch_model (#6268)
Signed-off-by: tzulingk@nvidia.com <tzulingk@nvidia.com>
|
2026-02-13 23:22:35 +00:00 |
|
mohammedabdulwahhab
|
a289695c37
|
fix: consolidate dyn_discovery_backend and dyn_kv_store (#6167)
Signed-off-by: mohammedabdulwahhab <furkhan324@berkeley.edu>
|
2026-02-13 19:07:11 +00:00 |
|
ishandhanani
|
2be83be2d8
|
feat: add video generation support (T2V) (#5793)
|
2026-02-13 09:09:56 +00:00 |
|
Yan Ru Pei
|
14eceb43df
|
chore: rename KvPushRouter to KvRouter in python + more bindings removal (#6238)
Signed-off-by: PeaBrane <yanrpei@gmail.com>
|
2026-02-12 23:44:52 -08:00 |
|
atchernych
|
5227176057
|
feat: Decomposed pipeline for EPP integration [DEP-730] (#5446)
Signed-off-by: Anna Tchernych <atchernych@nvidia.com>
|
2026-02-13 06:18:29 +00:00 |
|
Yan Ru Pei
|
bc514fbee6
|
feat: router priority queue (#6010)
Signed-off-by: PeaBrane <yanrpei@gmail.com>
Co-authored-by: ishandhanani <82981111+ishandhanani@users.noreply.github.com>
|
2026-02-13 02:43:17 +00:00 |
|
Karen Chung
|
cd6984b951
|
feat: use RNG when dp routing targets are tied; override no-assume-kv-reuse for decode requests (#6253)
Signed-off-by: Karen Chung <karenc@nvidia.com>
|
2026-02-12 15:59:37 -08:00 |
|
Yan Ru Pei
|
937398cf61
|
feat: Flash Indexer (#5785)
Signed-off-by: PeaBrane <yanrpei@gmail.com>
Signed-off-by: jthomson04 <jothomson@nvidia.com>
Signed-off-by: jthomson04 <jwillthomson19@gmail.com>
Signed-off-by: Janelle Cai <jcai18@mit.edu>
Co-authored-by: jthomson04 <jwillthomson19@gmail.com>
Co-authored-by: Janelle Cai <jcai18@mit.edu>
|
2026-02-11 22:26:56 -08:00 |
|
Yan Ru Pei
|
6a728d1044
|
chore: remove kv indexers bindings (#6159)
Signed-off-by: PeaBrane <yanrpei@gmail.com>
|
2026-02-11 20:23:48 +00:00 |
|
Graham King
|
73dc3be81a
|
chore: Merge bindings client and client2 functions (#6158)
Signed-off-by: Graham King <grahamk@nvidia.com>
|
2026-02-11 11:51:01 -05:00 |
|
Yan Ru Pei
|
f46720c996
|
feat: add router-level Prometheus metrics and centralize request tracking (#6146)
Signed-off-by: PeaBrane <yanrpei@gmail.com>
Co-authored-by: Cursor <cursoragent@cursor.com>
|
2026-02-11 07:36:08 +00:00 |
|
Graham King
|
4f99451bb0
|
feat(frontend): Use vllm for pre and post processing (#5544)
Signed-off-by: Graham King <grahamk@nvidia.com>
|
2026-02-10 22:39:11 +00:00 |
|
Ayush Agarwal
|
9f76d0606c
|
feat: text to image vLLM Omni (#5912)
Signed-off-by: ayushag <ayushag@nvidia.com>
|
2026-02-09 18:19:27 -05:00 |
|
Yan Ru Pei
|
6783bdcaa9
|
chore: enable local indexers by default, and use normal event plane by default (not jetstream) (#5941)
Signed-off-by: PeaBrane <yanrpei@gmail.com>
|
2026-02-08 20:48:02 +00:00 |
|
Jacky
|
1ffa489ea1
|
refactor: Move --migration-limit flag from backend to frontend (#5918)
Signed-off-by: Jacky <18255193+kthui@users.noreply.github.com>
|
2026-02-06 20:50:55 +00:00 |
|
Yan Ru Pei
|
3e41702211
|
chore: remove and unify bindings in kv.rs (#6016)
Signed-off-by: PeaBrane <yanrpei@gmail.com>
|
2026-02-06 18:44:46 +00:00 |
|
Biswa Panda
|
953e5d7beb
|
feat(lora): implement HRW-based LoRA allocation system (#5992)
|
2026-02-05 14:56:15 -08:00 |
|
Keiven C
|
cb7ebdd7e8
|
chore: remove unused NIM specific code (part 2) (#5893)
Signed-off-by: Keiven Chang <keivenchang@users.noreply.github.com>
Co-authored-by: Keiven Chang <keivenchang@users.noreply.github.com>
|
2026-02-05 12:21:10 -08:00 |
|
Yan Ru Pei
|
91fb78cddf
|
chore: make mocker it's own crate (#5958)
Signed-off-by: PeaBrane <yanrpei@gmail.com>
|
2026-02-05 19:35:42 +00:00 |
|
Hongkuan Zhou
|
0c0336e6ea
|
feat: Add per-worker Prometheus metrics for router load monitoring (#5842)
Signed-off-by: hongkuanz <hongkuanz@nvidia.com>
|
2026-02-04 11:51:43 -08:00 |
|
Konrad Nowicki
|
7e970d44c9
|
feat: image diffusion with SGLang diffusion (#5609)
Signed-off-by: Konrad Nowicki <knowicki@nvidia.com>
Co-authored-by: dagil-nvidia <dagil@nvidia.com>
|
2026-02-03 21:19:21 +00:00 |
|
Yan Ru Pei
|
a337113aa9
|
feat: prefill tokens threshold based on max num batched tokens frac (#5867)
Signed-off-by: PeaBrane <yanrpei@gmail.com>
|
2026-02-03 03:08:25 +00:00 |
|
Yan Ru Pei
|
902eabd9ec
|
feat: per dp rank gap detection (#5873)
Signed-off-by: PeaBrane <yanrpei@gmail.com>
|
2026-02-03 03:07:57 +00:00 |