mooncake-track
/
dynamo
0
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity
3,557 Commits 2 Branches 0 Tags 76 MiB
Commit Graph

668 Commits

Author SHA1 Message Date
Alec e10787894d
fix: add periodic vLLM engine stats logging and fix log routing (#6566) 
Signed-off-by: alec-flowers <aflowers@nvidia.com>
Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
 2026-02-25 23:45:34 +00:00
Schwinn Saereesitthipitak 5a7ead2bdc
feat(sglang): add checkpoint/restore support for chrek (#6594) 
Co-authored-by: Hannah Zhang <hannahz@nvidia.com>
 2026-02-25 23:42:16 +00:00
zhongdaor-nv 49eca14b69
fix: optimize uuid calculation (#6596) 
Signed-off-by: zhongdaor <zhongdaor@nvidia.com>
 2026-02-25 15:25:48 -08:00
daiyaanarfeen 651ef5b506
feat: throughput-metrics-source for SLA planner + GlobalPlanner disagg scaling (#6500) 
Signed-off-by: Daiyaan <darfeen@nvidia.com>
Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com>
 2026-02-25 22:58:24 +00:00
Kris Hung 80955ef463
perf: Keep embeddings on GPU Embedding Sender in EPD pipeline + minor fixes (#6535) 2026-02-25 07:51:20 -08:00
Yongming Ding 5d9586234e
feat(mocker): add KV cache transfer latency for disagg serving (#6504) 
Signed-off-by: Yongming Ding <yongmingd@nvidia.com>
 2026-02-24 23:53:28 -08:00
Yan Ru Pei 65dc451db9
feat: add ZMQ KV event publishing to mocker [DYN-2221] (#6528) 
Signed-off-by: Ru Pei <rupei@nvidia.com>
Signed-off-by: PeaBrane <yanrpei@gmail.com>
Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
 2026-02-24 18:57:28 -08:00
KrishnanPrash 8e2363758e
fix: restore E/P/D multimodal disagg serving and add Qwen3-VL-30B-A3B support (#6533) 
Signed-off-by: Krishnan Prashanth <kprashanth@nvidia.com>
Signed-off-by: KrishnanPrash <140860868+KrishnanPrash@users.noreply.github.com>
 2026-02-24 14:13:10 -08:00
ishandhanani 6642e23e0f
feat: sglang to 0.5.9 + updated docs (#6518) 
Co-authored-by: baihuitian <baihuitian.bht@gmail.com>
Co-authored-by: Cursor <cursoragent@cursor.com>
 2026-02-24 21:48:25 +00:00
Ayush Agarwal e94f464760
chore: install vllm-omni in vllm container (#6458) 
Signed-off-by: ayushag <ayushag@nvidia.com>
 2026-02-24 21:03:53 +00:00
Raghav Potluri 35128b144b
fix: frontend crash when using TRTLLM runtime image (#6481) 
Signed-off-by: Raghav Potluri <raghav.potluri21@gmail.com>
 2026-02-24 19:52:04 +00:00
Yan Ru Pei bc00ef3879
chore: split JetStream subscriber into dedicated module and deprecate durable_kv_events [DYN-2203] (#6477) 
Signed-off-by: PeaBrane <yanrpei@gmail.com>
 2026-02-24 11:30:38 -08:00
Biswa Panda 75fea78797
feat: make KV cache events and routing LoRA-aware (#6517) 2026-02-24 18:35:08 +00:00
Alec 6d3b92f04e
feat: remove --connector flag for vLLM backend (LLM-90) (#6450) 
Signed-off-by: alec-flowers <aflowers@nvidia.com>
Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
 2026-02-24 17:49:00 +00:00
Yan Ru Pei c9ff623538
chore: replace --enforce-disagg with --decode-fallback, default to enforcing disagg (#6515) 
Signed-off-by: PeaBrane <yanrpei@gmail.com>
 2026-02-24 09:24:11 -08:00
hhzhang16 659122ed07
feat: add Planner schema to DGDR and Profiler input (#6463) 
Signed-off-by: Hannah Zhang <hannahz@nvidia.com>
Co-authored-by: Sertaç Özercan <852750+sozercan@users.noreply.github.com>
 2026-02-24 10:14:57 -05:00
Qi Wang 33b3cb8aba
feat: embedding cache in agg vLLM node (#6153) 2026-02-23 21:59:53 -08:00
Indrajit Bhosale eb76a8b5c5
chore: Expose new kv_cache metrics from trtllm backend (#6469) 2026-02-23 18:44:52 -08:00
Alec 7893f2684e
feat: add --disaggregation-mode enum to vLLM backend (#6483) 
Signed-off-by: alec-flowers <aflowers@nvidia.com>
Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
 2026-02-23 19:34:30 -07:00
Tzu-Ling Kan 80cac7c14b
feat: Remove Component from public (#6403) 
Signed-off-by: tzulingk@nvidia.com <tzulingk@nvidia.com>
 2026-02-23 22:58:38 +00:00
Thomas Montfort cb55766c47
feat(runtime): add hierarchical Model/WorkerSet architecture for multi-namespace support (#6054) 
Signed-off-by: tmontfort <tmontfort@nvidia.com>
 2026-02-23 22:44:38 +00:00
jh-nv c40f352530
chore: use config.modality directly for multimodal init and remove incorrect fallback value (#6471) 2026-02-23 12:19:37 -08:00
Alec 43ce8608fd
fix: prefix omni/diffusion CLI flags with --omni- to avoid vLLM collisions (#6476) 
Signed-off-by: alec-flowers <aflowers@nvidia.com>
Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
 2026-02-23 10:43:20 -08:00
Schwinn Saereesitthipitak 673822ea93
fix(chrek): fix CI errors and replace SIGUSR2 with SIGKILL on checkpoint failure (#6478) 2026-02-23 10:23:52 -05:00
jh-nv ea86df2985
feat: Backend accept new requests during shutdown grace period (#6093) 
Signed-off-by: Jacky <18255193+kthui@users.noreply.github.com>
Co-authored-by: Jacky <18255193+kthui@users.noreply.github.com>
 2026-02-23 03:18:46 +00:00
Alec 4ebb244b2a
feat: add --headless mode for multi-node TP/PP in dynamo.vllm (#6204) 
Signed-off-by: alec-flowers <aflowers@nvidia.com>
Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
 2026-02-22 22:38:17 +00:00
Alec a95684559d
fix: add pre_merge markers to vLLM unit tests (#6482) 
Signed-off-by: alec-flowers <aflowers@nvidia.com>
Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
 2026-02-22 19:41:50 +00:00
William Zhang 8d30cd4587
fix: Multimodal flag was ignored for TRTLLM (#6468) 
* Why?

Commit `5a67b246d` refactored configs for the TRTLLM backend, breaking
`--modality multimodal`.

* What?

This commit fixes this bug, and adds a unit test verified to fail
without it.
 2026-02-21 07:59:54 -07:00
Schwinn Saereesitthipitak bb8fc8a4a9
feat(chrek): external restore, signal-based IPC, and package refactor (#6286) 
Co-authored-by: Dan Feigin <dfeigin@nvidia.com>
 2026-02-21 01:31:32 +00:00
Graham King f91b42b909
feat(frontend): Reduce Python-side overhead in the vLLM chat path (#6437) 
Signed-off-by: Graham King <grahamk@nvidia.com>
 2026-02-20 17:37:26 -05:00
Jacky 41d7d5490f
feat: Configurable Request Cancellation abort passage to TRT-LLM (#6445) 
Signed-off-by: Jacky <18255193+kthui@users.noreply.github.com>
 2026-02-20 10:36:18 -08:00
Ayush Agarwal 7409bd3a36
fix: vllm omni image perf fix (#6451) 
Signed-off-by: ayushag <ayushag@nvidia.com>
 2026-02-20 09:48:21 -08:00
Qi Wang c82fe88847
feat: add embedding cache to pd worker (#6061) 2026-02-20 08:27:44 -08:00
hhzhang16 ebc61637d8
feat: Add v1beta1 DGDR API with conversion framework (#6352) 
Signed-off-by: Jont828 <jt572@cornell.edu>
Signed-off-by: Hongkuan Zhou <hongkuanz@nvidia.com>
Signed-off-by: Hannah Zhang <hannahz@nvidia.com>
Co-authored-by: Jont828 <jt572@cornell.edu>
 2026-02-19 23:09:02 -08:00
Alec 7bbacce196
feat: default kv-events-config to empty (align with vLLM defaults) (#6404) 
Signed-off-by: alec-flowers <aflowers@nvidia.com>
Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
 2026-02-20 06:44:51 +00:00
Ayush Agarwal d6c49779de
chore: unify filesystem usage across all frameworks and workers (#6391) 
Signed-off-by: ayushag <ayushag@nvidia.com>
 2026-02-20 05:56:31 +00:00
Ayush Agarwal 6dd3ce2ea0
feat: vllm omni text to video generation pipeline (#6104) 
Signed-off-by: ayushag <ayushag@nvidia.com>
 2026-02-20 04:23:44 +00:00
zhongdaor-nv 23de4e86ae
feat: e2e mm aware kv cache routing support for trtllm backend (#5480) 
Signed-off-by: zhongdaor <zhongdaor@nvidia.com>
Signed-off-by: zhongdaor-nv <zhongdaor@nvidia.com>
Signed-off-by: Zhongdao Ren <zhongdaor@zhongdaor-mlt.client.nvidia.com>
Co-authored-by: Claude Sonnet 4.5 <noreply@anthropic.com>
Co-authored-by: Zhongdao Ren <zhongdaor@zhongdaor-mlt.client.nvidia.com>
 2026-02-19 20:52:00 -07:00
Michal Guzek 8a098a6650
feat: TRT-LLM multimodal preprocessor with backend media decoding (#5910) 
Signed-off-by: Michal Guzek <mguzek@nvidia.com>
 2026-02-20 01:39:21 +00:00
Biswa Panda 026f361dd4
feat: resolve lora request for multimodal workers (#6399) 2026-02-19 23:18:32 +00:00
Tzu-Ling Kan 0ce3461a9e
feat: Add runtime.endpoint() method to eliminate namespace chaining (#6386) 
Signed-off-by: tzulingk@nvidia.com <tzulingk@nvidia.com>
 2026-02-19 16:02:37 -05:00
Biswa Panda 4ba48b7993
feat: add model field to multimodal protocol for LoRA identity (#6382) 2026-02-19 12:48:00 -08:00
GuanLuo a2a6917fa6
feat: use embedding transfer classes for EPD (#6223) 
Signed-off-by: Guan Luo <41310872+GuanLuo@users.noreply.github.com>
 2026-02-19 12:27:01 -08:00
jh-nv 44a76f96b3
refactor: update frontend kv-router flags to be consistent with router (#6361) 2026-02-19 00:27:15 +00:00
Hongkuan Zhou b2075619c9
feat: planner argparse CLI -> config file (#6356) 
Signed-off-by: hongkuanz <hongkuanz@nvidia.com>
 2026-02-18 23:17:36 +00:00
Tzu-Ling Kan bc8f1170ce
feat: Move ModelDeploymentCard to _internal (#6378) 
Signed-off-by: tzulingk@nvidia.com <tzulingk@nvidia.com>
 2026-02-18 23:12:18 +00:00
Qi Wang 4d0380d54a
refactor: introduce worker factory in vLLM multimodal (#6060) 2026-02-18 12:21:21 -08:00
Schwinn Saereesitthipitak a55b2433f7
fix: remove unnecessary cuda synchronize calls in GMS adapters (#6362) 2026-02-18 09:30:55 -08:00
jh-nv 5a67b246d0
feat: Migrate trtllm configuration (#6297) 2026-02-18 15:52:06 +00:00
Alec 9352da7a24
feat: add --endpoint flag support to dynamo.vllm (#6360) 
Signed-off-by: alec-flowers <aflowers@nvidia.com>
Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
 2026-02-18 09:25:09 -05:00