mooncake-track
/
dynamo
0
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity
3,250 Commits 2 Branches 0 Tags 76 MiB
Commit Graph

1006 Commits

Author SHA1 Message Date
Keiven C f51ec24d6f
fix: test_router_decisions async fix race condition (#6084) 
Signed-off-by: Keiven Chang <keivenchang@users.noreply.github.com>
 2026-02-09 17:12:31 -08:00
Keiven C 027d2653a5
feat: expose Python Prometheus metric via DynamoComponentMetrics (#5817) 
Signed-off-by: Keiven Chang <keivenchang@users.noreply.github.com>
 2026-02-09 17:10:37 -08:00
Ayush Agarwal 9f76d0606c
feat: text to image vLLM Omni (#5912) 
Signed-off-by: ayushag <ayushag@nvidia.com>
 2026-02-09 18:19:27 -05:00
dagil-nvidia 0ef41ffe03
refactor: use tempfile module instead of hardcoded /tmp paths (#5789) 
Signed-off-by: Dan Gil <dagil@nvidia.com>
Co-authored-by: Cursor <cursoragent@cursor.com>
 2026-02-09 21:44:03 +00:00
Vladislav Nosivskoy 7202ced7b4
fix: DeepSeek V3.2 chat template FC and SO support (#6034) 
Signed-off-by: Vladislav Nosivskoy <vladnosiv@gmail.com>
 2026-02-09 10:57:47 -08:00
Yan Ru Pei 6783bdcaa9
chore: enable local indexers by default, and use normal event plane by default (not jetstream) (#5941) 
Signed-off-by: PeaBrane <yanrpei@gmail.com>
 2026-02-08 20:48:02 +00:00
Konstantin Korolev d59b9d72ed
fix: reduce NATS consumer inactive_threshold from 1h to 2min (#5861) 
Signed-off-by: advpropsys <korolev.konstantin.v@gmail.com>
Co-authored-by: Yan Ru Pei <yanrpei@gmail.com>
 2026-02-07 03:02:45 +00:00
Yongming Ding 7c25f70291
feat(mocker): add optional KV cache allocation/eviction trace (#6052) 
Signed-off-by: Yongming Ding <yongmingd@nvidia.com>
Co-authored-by: Yan Ru Pei <yanrpei@gmail.com>
 2026-02-07 03:02:28 +00:00
Jacky 1ffa489ea1
refactor: Move --migration-limit flag from backend to frontend (#5918) 
Signed-off-by: Jacky <18255193+kthui@users.noreply.github.com>
 2026-02-06 20:50:55 +00:00
Anant Sharma 3842b24479
fix: update bytes crate version to latest (#6041) 
Signed-off-by: Anant Sharma <anants@nvidia.com>
 2026-02-06 15:11:04 -05:00
Yan Ru Pei 9e33e3fa66
chore: clean ups in kv_router.rs (#6028) 
Signed-off-by: PeaBrane <yanrpei@gmail.com>
 2026-02-06 19:27:44 +00:00
Yan Ru Pei 3e41702211
chore: remove and unify bindings in kv.rs (#6016) 
Signed-off-by: PeaBrane <yanrpei@gmail.com>
 2026-02-06 18:44:46 +00:00
dagil-nvidia b19de4ed77
docs: cleanup of docs refactor for components, integrations, and features (#6019) 
Signed-off-by: Dan Gil <dagil@nvidia.com>
Co-authored-by: Cursor <cursoragent@cursor.com>
 2026-02-05 19:50:17 -08:00
akshatha-k 4c3eba2aeb
docs: restructure KVBM documentation into three-tier format (#5905) 
Signed-off-by: akshatha-k <akshutk@gmail.com>
Signed-off-by: Dan Gil <dagil@nvidia.com>
Signed-off-by: dagil-nvidia <dagil@nvidia.com>
Co-authored-by: dagil-nvidia <dagil@nvidia.com>
Co-authored-by: Cursor <cursoragent@cursor.com>
Co-authored-by: Kyle McGill <101670481+nv-kmcgill53@users.noreply.github.com>
 2026-02-05 18:02:05 -06:00
Biswa Panda 953e5d7beb
feat(lora): implement HRW-based LoRA allocation system (#5992) 2026-02-05 14:56:15 -08:00
Keiven C cb7ebdd7e8
chore: remove unused NIM specific code (part 2) (#5893) 
Signed-off-by: Keiven Chang <keivenchang@users.noreply.github.com>
Co-authored-by: Keiven Chang <keivenchang@users.noreply.github.com>
 2026-02-05 12:21:10 -08:00
Yan Ru Pei 91fb78cddf
chore: make mocker it's own crate (#5958) 
Signed-off-by: PeaBrane <yanrpei@gmail.com>
 2026-02-05 19:35:42 +00:00
Indrajit Bhosale 8c8680b179
feat: Make grpc startup dynamic for high ISL/OSL in GRPC frontend (#5536) 
Signed-off-by: Indrajit Bhosale <iamindrajitb@gmail.com>
 2026-02-05 10:11:20 -08:00
Kyle McGill 3a2885c822
fix: fix for how vllm disseminates cached requests to kvbm (#5851) 
Signed-off-by: Kyle McGill <kmcgill@nvidia.com>
 2026-02-04 15:22:06 -08:00
Janelle Cai 039d35ffd0
test: indexer and full router benchmarks (#5784) 2026-02-04 20:40:27 +00:00
Hongkuan Zhou 0c0336e6ea
feat: Add per-worker Prometheus metrics for router load monitoring (#5842) 
Signed-off-by: hongkuanz <hongkuanz@nvidia.com>
 2026-02-04 11:51:43 -08:00
Yan Ru Pei 67df0065d2
chore: linux gate lib/memory (#5942) 
Signed-off-by: PeaBrane <yanrpei@gmail.com>
 2026-02-04 10:38:00 -08:00
jthomson04 d22ca52310
feat: Various Mocker Perf improvements + fixes (#5808) 
Signed-off-by: jthomson04 <jothomson@nvidia.com>
 2026-02-04 10:22:45 -08:00
Janelle Cai 7beb5f1ad0
fix: fuse unfold streams to prevent panics from poll after termination (#5872) 2026-02-04 09:01:21 -08:00
Janelle Cai c7f6f6d980
fix: race between worker discovery and runtimeconfig discovery in KV router (#5924) 
Signed-off-by: Janelle Cai <jcai18@mit.edu>
Signed-off-by: PeaBrane <yanrpei@gmail.com>
Co-authored-by: Yan Ru Pei <yanrpei@gmail.com>
 2026-02-04 04:16:09 +00:00
Biswa Panda 72a12869e6
feat(lora): add lora load estimator (#5880) 2026-02-04 02:28:57 +00:00
Biswa Panda 4f9a190ca2
docs: add overview doc for GPU Memory Service (GMS) (#5920) 
Co-authored-by: Schwinn Saereesitthipitak <schwinns@nvidia.com>
 2026-02-03 17:22:01 -08:00
milesial 8daacbd7c0
feat: default with lib/memory, media-nixl and kvbm (#5602) 
Signed-off-by: Alexandre Milesi <milesial@users.noreply.github.com>
 2026-02-04 00:47:47 +00:00
Ameen Patel da0cbf6856
fix(runtime): return 500 on LoRA load/unload errors (#5626) 
Signed-off-by: AmeenP <ameenp360@gmail.com>
Co-authored-by: Biswa Panda <biswa.panda@gmail.com>
 2026-02-03 16:10:29 -08:00
Konrad Nowicki 7e970d44c9
feat: image diffusion with SGLang diffusion (#5609) 
Signed-off-by: Konrad Nowicki <knowicki@nvidia.com>
Co-authored-by: dagil-nvidia <dagil@nvidia.com>
 2026-02-03 21:19:21 +00:00
Julien Mancuso f3aa1e0129
feat: introducing ChReK (Checkpoint Restore in K8s) (#4978) 
Signed-off-by: Julien Mancuso <jmancuso@nvidia.com>
 2026-02-03 13:34:14 -07:00
atchernych 2d602853ed
feat: Add EPP startup probe and adjust recipe - fixes [DEP-749] (#5770) 
Signed-off-by: Julien Mancuso <jmancuso@nvidia.com>
Signed-off-by: Anna Tchernych <atchernych@nvidia.com>
Co-authored-by: Julien Mancuso <jmancuso@nvidia.com>
 2026-02-03 19:27:36 +00:00
milesial 9bff03f237
feat: vLLM backend with frontend media decoding (#5781) 
Signed-off-by: Alexandre Milesi <milesial@users.noreply.github.com>
 2026-02-03 09:19:26 -08:00
Yan Ru Pei a337113aa9
feat: prefill tokens threshold based on max num batched tokens frac (#5867) 
Signed-off-by: PeaBrane <yanrpei@gmail.com>
 2026-02-03 03:08:25 +00:00
Yan Ru Pei 902eabd9ec
feat: per dp rank gap detection (#5873) 
Signed-off-by: PeaBrane <yanrpei@gmail.com>
 2026-02-03 03:07:57 +00:00
Biswa Panda 373e76c1ae
feat(lora): Add lora_name tracking to scheduling and sequence management (#5875) 2026-02-03 00:59:09 +00:00
Janelle Cai 18d9d1fa4f
feat(mocker): pre-fetch model and staggered launches (#5871) 2026-02-03 00:46:23 +00:00
Yan Ru Pei df4cd1914a
chore: remove dead AllWorkersBusy handling in KvScheduler (#5869) 
Signed-off-by: PeaBrane <yanrpei@gmail.com>
 2026-02-02 11:10:42 -08:00
Yan Ru Pei 8dd1fc0705
chore: in discovery, use dashmap in place of mutex hashmap (#5868) 
Signed-off-by: PeaBrane <yanrpei@gmail.com>
 2026-02-02 10:39:36 -08:00
ishandhanani c5f5ab6056
fix: enable DP attention KV events for multi-node deployments (#5589) 
Co-authored-by: Claude Opus 4.5 <noreply@anthropic.com>
Co-authored-by: huitianbai <huitianbai@gmail.com>
Co-authored-by: Huitianqi Bai <huitianbai@users.noreply.github.com>
 2026-02-02 07:20:29 +00:00
Janelle Cai b88cb59bf9
chore: move HF config download and prevent duplicate concurrent model registration (#5767) 2026-01-31 00:04:06 +00:00
Yan Ru Pei aa4ac947c8
chore: atomic file write (#5809) 
Signed-off-by: PeaBrane <yanrpei@gmail.com>
 2026-01-30 01:52:10 +00:00
Schwinn Saereesitthipitak 69fdc9dd87
fix: CUDA synchronization to prevent sleep/wake race conditions (#5759) 
Signed-off-by: Schwinn Saereesitthipitak <schwinns@nvidia.com>
 2026-01-29 16:13:43 -08:00
Yuewei Na eef5e64523
refactor: read offload priority env var once at init (#5798) 
Signed-off-by: Yuewei Na <nv-yna@users.noreply.github.com>
Co-authored-by: Yuewei Na <nv-yna@users.noreply.github.com>
 2026-01-29 16:04:11 -08:00
Julien Mancuso 9e2a2cc9e6
feat: add epp component (#5611) 
Signed-off-by: Julien Mancuso <jmancuso@nvidia.com>
 2026-01-29 22:43:01 +00:00
Yuewei Na 6271a31f0d
feat: Implement priority-based KV cache offload filtering (#5563) 
Signed-off-by: Yuewei Na <248773860+nv-yna@users.noreply.github.com>
Signed-off-by: Yuewei Na <nv-yna@users.noreply.github.com>
Signed-off-by: yna <nv-yna@users.noreply.github.com>
Co-authored-by: Yuewei Na <nv-yna@users.noreply.github.com>
 2026-01-29 21:42:37 +00:00
Schwinn Saereesitthipitak a2fbda3e1c
feat(sglang): integration with GPU Memory Service (#5664) 
Signed-off-by: Schwinn Saereesitthipitak <17022745+galletas1712@users.noreply.github.com>
 2026-01-29 09:19:59 -08:00
Yan Ru Pei 584020f4b3
chore: consistently used the arc shared workers configs + CI flake fixes (#5707) 
Signed-off-by: PeaBrane <yanrpei@gmail.com>
 2026-01-29 09:29:24 +00:00
Biswa Panda 1174c819cb
feat: remove deprecated events api (#5778) 2026-01-29 05:48:16 +00:00
Yan Ru Pei fc92fc187b
chore: clean ups in kv-router (#5771) 
Signed-off-by: PeaBrane <yanrpei@gmail.com>
 2026-01-29 03:20:04 +00:00