mooncake-track
/
dynamo
0
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity
3,557 Commits 2 Branches 0 Tags 76 MiB
Commit Graph

136 Commits

Author SHA1 Message Date
daiyaanarfeen 651ef5b506
feat: throughput-metrics-source for SLA planner + GlobalPlanner disagg scaling (#6500) 
Signed-off-by: Daiyaan <darfeen@nvidia.com>
Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com>
 2026-02-25 22:58:24 +00:00
jh-nv 42cde26489
fix: migrate example and document to the latest endpoint API (#6542) 2026-02-25 12:00:04 -08:00
Qi Wang d5add7fffa
docs: embedding cache in vLLM and TRT-LLM (#6555) 2026-02-25 00:04:46 -08:00
Alec 35323da20c
fix: vllm launch script errors for disagg and spec decoding (#6562) 
Signed-off-by: alec-flowers <aflowers@nvidia.com>
Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
 2026-02-24 20:41:45 -08:00
Tushar Sharma e3dc179304
fix: add explicit --kv-transfer-config to disagg prefill workers (#6554) 
Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
 2026-02-25 02:33:15 +00:00
Alec eac94322f0
feat: add --kv-transfer-config NixlConnector to disagg scripts and recipes (#6560) 
Signed-off-by: alec-flowers <aflowers@nvidia.com>
 2026-02-25 02:26:47 +00:00
atchernych c916cd42ff
feat: Support epp's "pods" interface in Dynamo fixes [DEP-424] (#6302) 
Signed-off-by: Anna Tchernych <atchernych@nvidia.com>
 2026-02-25 01:54:38 +00:00
KrishnanPrash 8e2363758e
fix: restore E/P/D multimodal disagg serving and add Qwen3-VL-30B-A3B support (#6533) 
Signed-off-by: Krishnan Prashanth <kprashanth@nvidia.com>
Signed-off-by: KrishnanPrash <140860868+KrishnanPrash@users.noreply.github.com>
 2026-02-24 14:13:10 -08:00
ishandhanani 6642e23e0f
feat: sglang to 0.5.9 + updated docs (#6518) 
Co-authored-by: baihuitian <baihuitian.bht@gmail.com>
Co-authored-by: Cursor <cursoragent@cursor.com>
 2026-02-24 21:48:25 +00:00
Alec 6d3b92f04e
feat: remove --connector flag for vLLM backend (LLM-90) (#6450) 
Signed-off-by: alec-flowers <aflowers@nvidia.com>
Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
 2026-02-24 17:49:00 +00:00
Alec 7893f2684e
feat: add --disaggregation-mode enum to vLLM backend (#6483) 
Signed-off-by: alec-flowers <aflowers@nvidia.com>
Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
 2026-02-23 19:34:30 -07:00
Julien Mancuso 8dd6369e55
feat: use vllm multiprocessing in multinode scenario (#6191) 
Signed-off-by: Julien Mancuso <jmancuso@nvidia.com>
 2026-02-23 22:30:34 +00:00
Alec 4ebb244b2a
feat: add --headless mode for multi-node TP/PP in dynamo.vllm (#6204) 
Signed-off-by: alec-flowers <aflowers@nvidia.com>
Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
 2026-02-22 22:38:17 +00:00
Qi Wang c82fe88847
feat: add embedding cache to pd worker (#6061) 2026-02-20 08:27:44 -08:00
Alec 7bbacce196
feat: default kv-events-config to empty (align with vLLM defaults) (#6404) 
Signed-off-by: alec-flowers <aflowers@nvidia.com>
Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
 2026-02-20 06:44:51 +00:00
Ayush Agarwal d6c49779de
chore: unify filesystem usage across all frameworks and workers (#6391) 
Signed-off-by: ayushag <ayushag@nvidia.com>
 2026-02-20 05:56:31 +00:00
Ayush Agarwal 6dd3ce2ea0
feat: vllm omni text to video generation pipeline (#6104) 
Signed-off-by: ayushag <ayushag@nvidia.com>
 2026-02-20 04:23:44 +00:00
zhongdaor-nv 23de4e86ae
feat: e2e mm aware kv cache routing support for trtllm backend (#5480) 
Signed-off-by: zhongdaor <zhongdaor@nvidia.com>
Signed-off-by: zhongdaor-nv <zhongdaor@nvidia.com>
Signed-off-by: Zhongdao Ren <zhongdaor@zhongdaor-mlt.client.nvidia.com>
Co-authored-by: Claude Sonnet 4.5 <noreply@anthropic.com>
Co-authored-by: Zhongdao Ren <zhongdaor@zhongdaor-mlt.client.nvidia.com>
 2026-02-19 20:52:00 -07:00
Tzu-Ling Kan 0ce3461a9e
feat: Add runtime.endpoint() method to eliminate namespace chaining (#6386) 
Signed-off-by: tzulingk@nvidia.com <tzulingk@nvidia.com>
 2026-02-19 16:02:37 -05:00
Qi Wang 4d0380d54a
refactor: introduce worker factory in vLLM multimodal (#6060) 2026-02-18 12:21:21 -08:00
Qi Wang d2a5783938
refactor: delete handlers and disagg EC producer/consumer (#6051) 2026-02-17 18:05:54 -08:00
Dmitry Tokarev ea02149e48
fix: Fixed vllm runtime image tags. Improved docker image tag selection logic (#6290) 
Signed-off-by: Dmitry Tokarev <dtokarev@nvidia.com>
Co-authored-by: Dillon Cullinan <dcullinan@nvidia.com>
 2026-02-17 13:55:49 -05:00
zhongdaor-nv 815b129126
feat: mm aware routing for vllm (#6235) 
Signed-off-by: zhongdaor <zhongdaor@nvidia.com>
Signed-off-by: zhongdaor-nv <zhongdaor@nvidia.com>
 2026-02-13 21:16:06 -08:00
Tzu-Ling Kan 5624d14481
Rename fetch_llm to fetch_model (#6268) 
Signed-off-by: tzulingk@nvidia.com <tzulingk@nvidia.com>
 2026-02-13 23:22:35 +00:00
mohammedabdulwahhab a289695c37
fix: consolidate dyn_discovery_backend and dyn_kv_store (#6167) 
Signed-off-by: mohammedabdulwahhab <furkhan324@berkeley.edu>
 2026-02-13 19:07:11 +00:00
ishandhanani 2be83be2d8
feat: add video generation support (T2V) (#5793) 2026-02-13 09:09:56 +00:00
MatejKosec 8cb47d04d2
feat: responses API compliance with upstream type alignment (#6089) 
Signed-off-by: Matej Kosec <mkosec@nvidia.com>
Co-authored-by: Ishan Dhanani <ishandhanani@gmail.com>
 2026-02-12 01:18:28 +00:00
Jonathan Tong 39d645e586
docs: migrate Fern docs from fern/ into docs/ (#6206) 
Signed-off-by: Jont828 <jt572@cornell.edu>
 2026-02-11 16:22:27 -08:00
Dillon Cullinan 3188c70a3b
chore: Templating Feedback Followup (#6125) 
Signed-off-by: Dillon Cullinan <dcullinan@nvidia.com>
 2026-02-11 13:09:27 -05:00
Indrajit Bhosale 120ae64916
fix: llama4 vllm agg multimodal script (#6103) 2026-02-10 09:53:34 -08:00
Qi Wang df8fd92b18
chore: consistent name -- MultimodalEmbeddingCache (#5962) 2026-02-10 09:53:13 -08:00
Dillon Cullinan ac0206293c
feat: Dockerfile templating (#5633) 
Signed-off-by: Dillon Cullinan <dcullinan@nvidia.com>
 2026-02-10 08:56:17 -05:00
Ayush Agarwal 9f76d0606c
feat: text to image vLLM Omni (#5912) 
Signed-off-by: ayushag <ayushag@nvidia.com>
 2026-02-09 18:19:27 -05:00
Yan Ru Pei 6783bdcaa9
chore: enable local indexers by default, and use normal event plane by default (not jetstream) (#5941) 
Signed-off-by: PeaBrane <yanrpei@gmail.com>
 2026-02-08 20:48:02 +00:00
Qi Wang 00ea11ff2c
feat: EC E/PD workflow in TRT-LLM (#5815) 2026-02-06 15:32:52 -08:00
dagil-nvidia b19de4ed77
docs: cleanup of docs refactor for components, integrations, and features (#6019) 
Signed-off-by: Dan Gil <dagil@nvidia.com>
Co-authored-by: Cursor <cursoragent@cursor.com>
 2026-02-05 19:50:17 -08:00
akshatha-k 80e7bafd37
docs: Migrate router documentation to three-tier structure (#5979) 
Signed-off-by: akshatha-k <akshutk@gmail.com>
Signed-off-by: dagil-nvidia <dagil@nvidia.com>
Signed-off-by: Dan Gil <dagil@nvidia.com>
Co-authored-by: dagil-nvidia <dagil@nvidia.com>
Co-authored-by: Cursor <cursoragent@cursor.com>
 2026-02-06 01:55:50 +00:00
Indrajit Bhosale a72f41f604
fix: Update vllm multimodal PD Disagg launch script (#5951) 2026-02-05 19:33:04 +00:00
Ayush Agarwal 76e0e2076b
feat: basic vllm omni pipeline support (#5608) 
Signed-off-by: ayushag <ayushag@nvidia.com>
 2026-02-04 18:35:41 +00:00
Sean SH Choi 3b9b3f31c0
fix: Add --is-decode-worker flag to vllm examples (#5899) 
Signed-off-by: Sean Choi <sechoi@nvidia.com>
 2026-02-03 07:25:14 +00:00
Julien Mancuso 9e2a2cc9e6
feat: add epp component (#5611) 
Signed-off-by: Julien Mancuso <jmancuso@nvidia.com>
 2026-01-29 22:43:01 +00:00
Neal Vaidya c5bb8df685
fix: specifiy triton worker dynamo base [DYN-1984] (#5794) 
Signed-off-by: Neal Vaidya <nealv@nvidia.com>
 2026-01-29 11:51:44 -08:00
Indrajit Bhosale 842f0f15ec
fix: VLLM Multimodal minor fixes (#5748) 
Signed-off-by: Indrajit Bhosale <iamindrajitb@gmail.com>
 2026-01-29 02:45:41 +00:00
GuanLuo 77aadb722f
chore: adjust gpu-memory-utilization to accommodate vLLM's runtime GPU memory requirement (#5755) 
Signed-off-by: Guan Luo <gluo@nvidia.com>
 2026-01-28 23:48:48 +00:00
GuanLuo 903f818498
chore: add multimodal image benchmark scripts for performance evaluation (#5509) 
Signed-off-by: Guan Luo <41310872+GuanLuo@users.noreply.github.com>
 2026-01-27 17:39:58 -08:00
Abhishek Gupta 912a4d4b7e
docs: add NIXL backend configuration and fix multiple typos (#5564) 
Signed-off-by: Abhishek Kumar Gupta (AbhiOnGithub)
Signed-off-by: Abhishek Kumar Gupta <mail2abhishekgupta@gmail.com>
Co-authored-by: dagil-nvidia <dagil@nvidia.com>
 2026-01-23 16:43:46 -08:00
Indrajit Bhosale 66963b7040
feat: Standalone encoder in dynamo trtllm (#4668) 
Signed-off-by: Indrajit Bhosale <iamindrajitb@gmail.com>
Signed-off-by: Krishnan Prashanth <kprashanth@nvidia.com>
Signed-off-by: KrishnanPrash <140860868+KrishnanPrash@users.noreply.github.com>
Co-authored-by: Krishnan Prashanth <kprashanth@nvidia.com>
Co-authored-by: KrishnanPrash <140860868+KrishnanPrash@users.noreply.github.com>
 2026-01-22 23:49:19 +00:00
Ayush Agarwal 91ddf4189b
feat: v0 diffusion handler support (#5533) 
Signed-off-by: ayushag <ayushag@nvidia.com>
 2026-01-21 22:25:54 +00:00
Neal Vaidya 971c30692e
feat: triton worker example (#4971) 
Signed-off-by: Neal Vaidya <nealv@nvidia.com>
 2026-01-16 12:53:09 -08:00
Ayush Agarwal f9918f615f
chore: support for muliple MM requests for vLLM Encoder + Rust Based Preprocessor (#5463) 
Signed-off-by: ayushag <ayushag@nvidia.com>
 2026-01-16 20:48:49 +00:00