mooncake-track
/
dynamo
0
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity
3,250 Commits 2 Branches 0 Tags 76 MiB
Commit Graph

404 Commits

Author SHA1 Message Date
Keiven C 027d2653a5
feat: expose Python Prometheus metric via DynamoComponentMetrics (#5817) 
Signed-off-by: Keiven Chang <keivenchang@users.noreply.github.com>
 2026-02-09 17:10:37 -08:00
Ayush Agarwal 9f76d0606c
feat: text to image vLLM Omni (#5912) 
Signed-off-by: ayushag <ayushag@nvidia.com>
 2026-02-09 18:19:27 -05:00
Yan Ru Pei 6783bdcaa9
chore: enable local indexers by default, and use normal event plane by default (not jetstream) (#5941) 
Signed-off-by: PeaBrane <yanrpei@gmail.com>
 2026-02-08 20:48:02 +00:00
Yongming Ding 7c25f70291
feat(mocker): add optional KV cache allocation/eviction trace (#6052) 
Signed-off-by: Yongming Ding <yongmingd@nvidia.com>
Co-authored-by: Yan Ru Pei <yanrpei@gmail.com>
 2026-02-07 03:02:28 +00:00
Jacky 1ffa489ea1
refactor: Move --migration-limit flag from backend to frontend (#5918) 
Signed-off-by: Jacky <18255193+kthui@users.noreply.github.com>
 2026-02-06 20:50:55 +00:00
Anant Sharma 3842b24479
fix: update bytes crate version to latest (#6041) 
Signed-off-by: Anant Sharma <anants@nvidia.com>
 2026-02-06 15:11:04 -05:00
Yan Ru Pei 3e41702211
chore: remove and unify bindings in kv.rs (#6016) 
Signed-off-by: PeaBrane <yanrpei@gmail.com>
 2026-02-06 18:44:46 +00:00
Biswa Panda 953e5d7beb
feat(lora): implement HRW-based LoRA allocation system (#5992) 2026-02-05 14:56:15 -08:00
Keiven C cb7ebdd7e8
chore: remove unused NIM specific code (part 2) (#5893) 
Signed-off-by: Keiven Chang <keivenchang@users.noreply.github.com>
Co-authored-by: Keiven Chang <keivenchang@users.noreply.github.com>
 2026-02-05 12:21:10 -08:00
Yan Ru Pei 91fb78cddf
chore: make mocker it's own crate (#5958) 
Signed-off-by: PeaBrane <yanrpei@gmail.com>
 2026-02-05 19:35:42 +00:00
Hongkuan Zhou 0c0336e6ea
feat: Add per-worker Prometheus metrics for router load monitoring (#5842) 
Signed-off-by: hongkuanz <hongkuanz@nvidia.com>
 2026-02-04 11:51:43 -08:00
Konrad Nowicki 7e970d44c9
feat: image diffusion with SGLang diffusion (#5609) 
Signed-off-by: Konrad Nowicki <knowicki@nvidia.com>
Co-authored-by: dagil-nvidia <dagil@nvidia.com>
 2026-02-03 21:19:21 +00:00
milesial 9bff03f237
feat: vLLM backend with frontend media decoding (#5781) 
Signed-off-by: Alexandre Milesi <milesial@users.noreply.github.com>
 2026-02-03 09:19:26 -08:00
Yan Ru Pei a337113aa9
feat: prefill tokens threshold based on max num batched tokens frac (#5867) 
Signed-off-by: PeaBrane <yanrpei@gmail.com>
 2026-02-03 03:08:25 +00:00
Yan Ru Pei 902eabd9ec
feat: per dp rank gap detection (#5873) 
Signed-off-by: PeaBrane <yanrpei@gmail.com>
 2026-02-03 03:07:57 +00:00
Biswa Panda 373e76c1ae
feat(lora): Add lora_name tracking to scheduling and sequence management (#5875) 2026-02-03 00:59:09 +00:00
Janelle Cai 18d9d1fa4f
feat(mocker): pre-fetch model and staggered launches (#5871) 2026-02-03 00:46:23 +00:00
ishandhanani c5f5ab6056
fix: enable DP attention KV events for multi-node deployments (#5589) 
Co-authored-by: Claude Opus 4.5 <noreply@anthropic.com>
Co-authored-by: huitianbai <huitianbai@gmail.com>
Co-authored-by: Huitianqi Bai <huitianbai@users.noreply.github.com>
 2026-02-02 07:20:29 +00:00
Biswa Panda 1174c819cb
feat: remove deprecated events api (#5778) 2026-01-29 05:48:16 +00:00
Yan Ru Pei fc92fc187b
chore: clean ups in kv-router (#5771) 
Signed-off-by: PeaBrane <yanrpei@gmail.com>
 2026-01-29 03:20:04 +00:00
Keiven C 5a00a7d643
fix: prevent duplicate Prometheus metrics from Python expfmt callbacks (#5761) 
Signed-off-by: Keiven Chang <keivenchang@users.noreply.github.com>
 2026-01-28 23:34:33 +00:00
Keiven C 8e72fb69cf
docs: DYN-1967 update metrics docs after kvstats removal (#5704) 
Signed-off-by: Keiven Chang <keivenchang@users.noreply.github.com>
Co-authored-by: Keiven Chang <keivenchang@users.noreply.github.com>
 2026-01-27 17:24:12 -08:00
Dmitry Tokarev 2d6babe5fc
chore: Bump rust lib oneshot to 0.1.13 (#5694) 
Signed-off-by: Dmitry Tokarev <dtokarev@nvidia.com>
 2026-01-27 17:26:11 -05:00
Pavithra Vijayakrishnan 66c369964a
chore: version update for 0.9.0 (#5661) 
Signed-off-by: pvijayakrish <pvijayakrish@nvidia.com>
 2026-01-26 17:18:05 -08:00
Qi Wang 89e135b9ce
chore: remove legacy KVBM pythong static type checks and test (#5486) 2026-01-26 12:46:49 -08:00
Abhishek Gupta 7de80960dc
feat: add __all__ exports and __repr__ methods for better debugging (#5606) 2026-01-23 12:44:03 -08:00
Keiven C 7361de4208
refactor: reduce Python API to only Prometheus Exposition Format callback (#5594) 
Signed-off-by: Keiven Chang <keivenchang@users.noreply.github.com>
Co-authored-by: Keiven Chang <keivenchang@users.noreply.github.com>
 2026-01-23 10:06:06 -08:00
Yan Ru Pei feb6d272ac
chore: nuke ForwardPassMetrics (#5531) 
Signed-off-by: PeaBrane <yanrpei@gmail.com>
 2026-01-23 16:26:01 +00:00
Nate Mailhot 6dba119d7a
chore: update nixl to 0.9.0 (#5528) 2026-01-22 15:25:03 -08:00
Graham King ca31b3aa6b
chore: Remove binding OAIChatPreprocessor (#5516) 
Signed-off-by: Graham King <grahamk@nvidia.com>
 2026-01-22 08:43:02 +08:00
Michael Feil fa66f65437
feat: typing - Add py.typed support for typing (#4913) 
Signed-off-by: Michael Feil <63565275+michaelfeil@users.noreply.github.com>
Co-authored-by: Anant Sharma <anants@nvidia.com>
 2026-01-20 15:28:44 +00:00
Schwinn Saereesitthipitak d953f9d038
feat: add endpoint instance registration/unregistration for sleep/wake (#5203) 
Signed-off-by: Schwinn Saereesitthipitak <schwinns@nvidia.com>
Co-authored-by: Claude Opus 4.5 <noreply@anthropic.com>
 2026-01-20 12:17:58 +00:00
Yan Ru Pei 050906b593
feat: track output tokens / blocks in the Router (optional) (#5452) 
Signed-off-by: PeaBrane <yanrpei@gmail.com>
 2026-01-18 09:30:25 +08:00
Graham King a6e7348400
chore(bindings): Remove 'Backend' binding (#5458) 
Signed-off-by: Graham King <grahamk@nvidia.com>
 2026-01-16 12:29:50 -05:00
Graham King c5458b9954
chore(router): Remove deprecated best_worker_id and metrics_labels (#5412) 
Signed-off-by: Graham King <grahamk@nvidia.com>
 2026-01-14 16:20:42 +00:00
Neal Vaidya a936ba5ad2
fix: read store_kv arg from env when using decorator (#5421) 
Signed-off-by: Neal Vaidya <nealv@nvidia.com>
 2026-01-14 03:27:34 +00:00
Yan Ru Pei 7fdc742e17
feat: allow router not assuming decode kv reuse (#5350) 
Signed-off-by: PeaBrane <yanrpei@gmail.com>
 2026-01-13 07:33:47 +08:00
Graham King f6ae58e349
fix(llm): Send HF URLs to frontend not full paths (#5290) 
Signed-off-by: Graham King <grahamk@nvidia.com>
 2026-01-12 12:05:49 -05:00
Biswa Panda 0d3ff44000
fix: enable toggling kv events pub/sub (currently nats based) with --no-kv-events flag (#5237) 2026-01-09 06:36:51 +00:00
Neelay Shah d22d9e761e
fix: support truthy values for OTEL_EXPORT_ENABLED in bindings layer [DYN-1690][DYN-1646] (#5129) 
Co-authored-by: Claude <noreply@anthropic.com>
 2026-01-02 23:39:09 +00:00
Tushar Sharma cf433e6825
chore: update all copyright headers in repo to 2026 (#5130) 
Signed-off-by: Tushar Sharma <tusharma@nvidia.com>
 2026-01-02 22:08:23 +00:00
Yan Ru Pei 0b33c1dfa0
fix: sglang disagg routing fixes and optimizations [DYN-1692] (#5106) 
Signed-off-by: PeaBrane <yanrpei@gmail.com>
Co-authored-by: Ishan Dhanani <ishandhanani@gmail.com>
Co-authored-by: Sean SH Choi <sechoi@nvidia.com>
Co-authored-by: ishandhanani <82981111+ishandhanani@users.noreply.github.com>
 2025-12-31 10:55:44 -08:00
Phillippe Siclait ac8c9023e4
fix: Truncate HttpError message to first 8192 characters (#5020) 
Signed-off-by: Phillippe Siclait <546332+siclait@users.noreply.github.com>
 2025-12-22 12:23:35 -08:00
Yan Ru Pei f4245c9929
chore: clean up PreprocessedRequest (#5040) 
Signed-off-by: PeaBrane <yanrpei@gmail.com>
 2025-12-19 23:15:59 +00:00
Jacky e6a6a1f244
feat: Request Migration Metrics (#5029) 
Signed-off-by: Jacky <18255193+kthui@users.noreply.github.com>
 2025-12-19 12:22:36 -08:00
Nate Mailhot 910b587db1
chore: update nixl to 0.8.0 and ucx 1.20 (#5007) 
Signed-off-by: Nate Mailhot <nmailhot@nvidia.com>
 2025-12-19 10:55:40 -08:00
Anant Sharma 7436e3bc03
chore: update versions for 0.8.0 release (#5031) 
Signed-off-by: Anant Sharma <anants@nvidia.com>
 2025-12-19 10:21:52 -08:00
milesial d2faf0e6f5
feat: Runtime media decoder config (#5011) 
Signed-off-by: Alexandre Milesi <milesial@users.noreply.github.com>
 2025-12-19 05:34:09 +00:00
zhongdaor-nv 481dc6363b
feat: add multimodal support to KV router with standalone trtllm example (#4577) 
Signed-off-by: zhongdaor <zhongdaor@nvidia.com>
Signed-off-by: zhongdaor-nv <zhongdaor@nvidia.com>
 2025-12-18 21:01:45 -08:00
Keiven C c22280ccf4
refactor: frontend/grpc tests to use dynamic ports (#4992) 
Signed-off-by: Keiven Chang <keivenchang@users.noreply.github.com>
Co-authored-by: Keiven Chang <keivenchang@users.noreply.github.com>
 2025-12-18 16:21:28 -08:00