mooncake-track
/
dynamo
0
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity
3,250 Commits 2 Branches 0 Tags 76 MiB
Commit Graph

570 Commits

Author SHA1 Message Date
Keiven C 027d2653a5
feat: expose Python Prometheus metric via DynamoComponentMetrics (#5817) 
Signed-off-by: Keiven Chang <keivenchang@users.noreply.github.com>
 2026-02-09 17:10:37 -08:00
William Arnold 56a1b6e314
feat: Add SGLang /engine weight update endpoints (#6094) 2026-02-10 01:03:45 +00:00
Ayush Agarwal 9f76d0606c
feat: text to image vLLM Omni (#5912) 
Signed-off-by: ayushag <ayushag@nvidia.com>
 2026-02-09 18:19:27 -05:00
dagil-nvidia 0ef41ffe03
refactor: use tempfile module instead of hardcoded /tmp paths (#5789) 
Signed-off-by: Dan Gil <dagil@nvidia.com>
Co-authored-by: Cursor <cursoragent@cursor.com>
 2026-02-09 21:44:03 +00:00
Yan Ru Pei 6783bdcaa9
chore: enable local indexers by default, and use normal event plane by default (not jetstream) (#5941) 
Signed-off-by: PeaBrane <yanrpei@gmail.com>
 2026-02-08 20:48:02 +00:00
Neal Vaidya 3d7182b80e
fix: read block size from vllm at runtime (#5713) 
Signed-off-by: Neal Vaidya <nealv@nvidia.com>
 2026-02-07 22:36:25 +00:00
Yongming Ding 7c25f70291
feat(mocker): add optional KV cache allocation/eviction trace (#6052) 
Signed-off-by: Yongming Ding <yongmingd@nvidia.com>
Co-authored-by: Yan Ru Pei <yanrpei@gmail.com>
 2026-02-07 03:02:28 +00:00
Karen Chung 2bcbda19e4
fix: Router + SGLang DP testing (#6057) 2026-02-06 18:39:28 -08:00
Qi Wang 00ea11ff2c
feat: EC E/PD workflow in TRT-LLM (#5815) 2026-02-06 15:32:52 -08:00
Jacky 1ffa489ea1
refactor: Move --migration-limit flag from backend to frontend (#5918) 
Signed-off-by: Jacky <18255193+kthui@users.noreply.github.com>
 2026-02-06 20:50:55 +00:00
GuanLuo ac50dccf95
feat: batch process images in encode worker. Add qwen3 to supported models (#6021) 
Signed-off-by: Guan Luo <41310872+GuanLuo@users.noreply.github.com>
 2026-02-06 10:51:37 -08:00
Yan Ru Pei 3e41702211
chore: remove and unify bindings in kv.rs (#6016) 
Signed-off-by: PeaBrane <yanrpei@gmail.com>
 2026-02-06 18:44:46 +00:00
Karen Chung 3a41825489
fix: Correctly pass DP rank from Dynamo router into vLLM engine (#6014) 2026-02-06 09:48:53 -08:00
dagil-nvidia b19de4ed77
docs: cleanup of docs refactor for components, integrations, and features (#6019) 
Signed-off-by: Dan Gil <dagil@nvidia.com>
Co-authored-by: Cursor <cursoragent@cursor.com>
 2026-02-05 19:50:17 -08:00
akshatha-k 80e7bafd37
docs: Migrate router documentation to three-tier structure (#5979) 
Signed-off-by: akshatha-k <akshutk@gmail.com>
Signed-off-by: dagil-nvidia <dagil@nvidia.com>
Signed-off-by: Dan Gil <dagil@nvidia.com>
Co-authored-by: dagil-nvidia <dagil@nvidia.com>
Co-authored-by: Cursor <cursoragent@cursor.com>
 2026-02-06 01:55:50 +00:00
Karen Chung b5c0db63c3
feat: TRTLLM DP Rank Routing (#5936) 2026-02-05 17:43:04 -08:00
dagil-nvidia 400126d2bc
docs: migrate Frontend docs to three-tier structure (#6002) 
Signed-off-by: Dan Gil <dagil@nvidia.com>
Co-authored-by: Cursor <cursoragent@cursor.com>
 2026-02-06 01:00:52 +00:00
Anish 7752ce219e
docs: planner 3-tier documentation restructure (#5876) 
Signed-off-by: athreesh <anish.maddipoti@utexas.edu>
Signed-off-by: dagil-nvidia <dagil@nvidia.com>
Signed-off-by: Dan Gil <dagil@nvidia.com>
Co-authored-by: Claude Opus 4.5 <noreply@anthropic.com>
Co-authored-by: dagil-nvidia <dagil@nvidia.com>
Co-authored-by: Hongkuan Zhou <tedzhouhk@gmail.com>
Co-authored-by: Cursor <cursoragent@cursor.com>
 2026-02-05 17:45:19 -06:00
Keiven C cb7ebdd7e8
chore: remove unused NIM specific code (part 2) (#5893) 
Signed-off-by: Keiven Chang <keivenchang@users.noreply.github.com>
Co-authored-by: Keiven Chang <keivenchang@users.noreply.github.com>
 2026-02-05 12:21:10 -08:00
KrishnanPrash 6b70d845d4
fix: multimodal worker internal-only to prevent registration collision (#5986) 
Signed-off-by: Krishnan Prashanth <kprashanth@nvidia.com>
 2026-02-05 11:20:52 -08:00
KrishnanPrash cfba042e99
fix: mm_item keys for SGLang API (#5981) 
Signed-off-by: Krishnan Prashanth <kprashanth@nvidia.com>
 2026-02-05 11:20:42 -08:00
Qi Wang a78a426065
feat: use encoder cache in TRT-LLM EPD workflow (#5780) 2026-02-04 14:34:41 -08:00
Thomas Montfort b12e671039
feat: read prefill/decode worker counts from DGD status (#5934) 2026-02-04 14:24:29 -08:00
KrishnanPrash cd3f9bbd0c
fix: treat `--tool-call-parser` and `--dyn-tool-call-parser` independently for SGLang (#5849) 
Signed-off-by: Krishnan Prashanth <kprashanth@nvidia.com>
 2026-02-04 13:07:21 -08:00
Qi Wang 6cb76b967a
feat: introduce cuda_ipc for TRT-LLM PrefillHandler (#5773) 2026-02-04 12:43:21 -08:00
Indrajit Bhosale eff08aed05
fix: Wrap default_multimodal_input_loader in asyncio.to_thread (#5945) 2026-02-04 11:31:17 -08:00
Ayush Agarwal 76e0e2076b
feat: basic vllm omni pipeline support (#5608) 
Signed-off-by: ayushag <ayushag@nvidia.com>
 2026-02-04 18:35:41 +00:00
jthomson04 d22ca52310
feat: Various Mocker Perf improvements + fixes (#5808) 
Signed-off-by: jthomson04 <jothomson@nvidia.com>
 2026-02-04 10:22:45 -08:00
Daniel Socek 763264ffb3
fix: Multimodal disaggregation improvements (#5895) 
Signed-off-by: Daniel Socek <daniel.socek@intel.com>
 2026-02-04 09:41:52 +00:00
Qi Wang b82b45a1ca
feat: add EncoderCacheManager to TRT-LLM PrefillHandler (#5714) 2026-02-04 02:06:29 +00:00
Thomas Montfort 2176c43193
feat(planner): Derive prefill/decode GPU counts from DGD (#5919) 2026-02-04 01:19:06 +00:00
milesial 8daacbd7c0
feat: default with lib/memory, media-nixl and kvbm (#5602) 
Signed-off-by: Alexandre Milesi <milesial@users.noreply.github.com>
 2026-02-04 00:47:47 +00:00
Konrad Nowicki 7e970d44c9
feat: image diffusion with SGLang diffusion (#5609) 
Signed-off-by: Konrad Nowicki <knowicki@nvidia.com>
Co-authored-by: dagil-nvidia <dagil@nvidia.com>
 2026-02-03 21:19:21 +00:00
Julien Mancuso f3aa1e0129
feat: introducing ChReK (Checkpoint Restore in K8s) (#4978) 
Signed-off-by: Julien Mancuso <jmancuso@nvidia.com>
 2026-02-03 13:34:14 -07:00
milesial 9bff03f237
feat: vLLM backend with frontend media decoding (#5781) 
Signed-off-by: Alexandre Milesi <milesial@users.noreply.github.com>
 2026-02-03 09:19:26 -08:00
Yan Ru Pei a337113aa9
feat: prefill tokens threshold based on max num batched tokens frac (#5867) 
Signed-off-by: PeaBrane <yanrpei@gmail.com>
 2026-02-03 03:08:25 +00:00
Yan Ru Pei 902eabd9ec
feat: per dp rank gap detection (#5873) 
Signed-off-by: PeaBrane <yanrpei@gmail.com>
 2026-02-03 03:07:57 +00:00
Janelle Cai 18d9d1fa4f
feat(mocker): pre-fetch model and staggered launches (#5871) 2026-02-03 00:46:23 +00:00
ishandhanani c5f5ab6056
fix: enable DP attention KV events for multi-node deployments (#5589) 
Co-authored-by: Claude Opus 4.5 <noreply@anthropic.com>
Co-authored-by: huitianbai <huitianbai@gmail.com>
Co-authored-by: Huitianqi Bai <huitianbai@users.noreply.github.com>
 2026-02-02 07:20:29 +00:00
William Zhang 95383fd61b
ci: Add missing pytest marker (#5837) 2026-01-30 15:52:18 -08:00
Jacky 934d03c545
fix: Disable calls to abort() at TRT-LLM backend - temporary (#5827) 
Signed-off-by: Jacky <18255193+kthui@users.noreply.github.com>
 2026-01-30 15:03:30 -08:00
William Zhang 2f666b73a2
fix: Properly forward sampling params from the request (#5797) 2026-01-30 12:38:40 -08:00
jh-nv 704c1dad84
fix: fix vllm graceful shutdown (#5818) 2026-01-30 15:13:49 -05:00
Indrajit Bhosale 454c28abc0
fix: sampling params parsing in vllm EPD flow (#5813) 
Signed-off-by: Indrajit Bhosale <iamindrajitb@gmail.com>
 2026-01-30 19:12:04 +00:00
Jacky b10d103d27
fix: Temporary disable cancellation at TRT-LLM decode worker (#5764) 
Signed-off-by: Jacky <18255193+kthui@users.noreply.github.com>
 2026-01-29 18:18:20 -08:00
Tanmay Verma ba711cc1ac
chore: Upgrade to Tensorrt-LLM 1.3.0rc1 (#5700) 
Co-authored-by: Pavithra Vijayakrishnan <160681768+pvijayakrish@users.noreply.github.com>
 2026-01-29 15:20:58 -08:00
Schwinn Saereesitthipitak a2fbda3e1c
feat(sglang): integration with GPU Memory Service (#5664) 
Signed-off-by: Schwinn Saereesitthipitak <17022745+galletas1712@users.noreply.github.com>
 2026-01-29 09:19:59 -08:00
Hongkuan Zhou e2f1e04f1c
refactor: separate planner into prefill/decode planner (#5622) 
Signed-off-by: hongkuanz <hongkuanz@nvidia.com>
 2026-01-28 16:08:06 -08:00
Qi Wang 165f326dd1
feat: add multimodal hasher to TRT-LLM (#5715) 2026-01-28 13:05:23 -08:00
Qi Wang 838ba140d6
feat: async encoder cache impl (#5676) 2026-01-28 13:05:13 -08:00