KrishnanPrash
|
da783ce0d3
|
fix: prevent DYN_SYSTEM_PORT collisions in TRT-LLM EPD launch scripts (#7366)
Signed-off-by: Krishnan Prashanth <kprashanth@nvidia.com>
|
2026-03-13 17:37:37 -07:00 |
|
KrishnanPrash
|
be2f1dc11b
|
ci: fit sglang multimodal EPD test on 1 GPU (#7046)
Signed-off-by: Krishnan Prashanth <kprashanth@nvidia.com>
|
2026-03-13 14:47:58 -07:00 |
|
Ayush Agarwal
|
1182e2071d
|
feat: vllm omni image to video support (#6530)
Signed-off-by: ayushag <ayushag@nvidia.com>
|
2026-03-12 17:30:46 +00:00 |
|
Keiven C
|
8d524d03d3
|
docs: add consistent startup banners to SGLang and TRT-LLM launch scripts (#7252)
Signed-off-by: Keiven Chang <keivenchang@users.noreply.github.com>
|
2026-03-11 19:21:01 -07:00 |
|
daiyaanarfeen
|
5d5fd243da
|
feat: GlobalPlanner --max-total-gpus for cluster-wide GPU budget (#7103)
Signed-off-by: Daiyaan <darfeen@nvidia.com>
Signed-off-by: athreesh <anish.maddipoti@utexas.edu>
Signed-off-by: Anish <80174047+athreesh@users.noreply.github.com>
Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com>
Co-authored-by: athreesh <anish.maddipoti@utexas.edu>
Co-authored-by: Anish <80174047+athreesh@users.noreply.github.com>
|
2026-03-11 21:54:55 +00:00 |
|
Keiven C
|
34ccc0b15f
|
feat: refactor launch scripts with shared launch_utils.sh for consistent failure handling (#7008)
Signed-off-by: Keiven Chang <keivenchang@users.noreply.github.com>
|
2026-03-11 13:21:39 -07:00 |
|
dagil-nvidia
|
fa474d360e
|
docs: fix disaggregated deployment example in tracing.md (#6999)
Signed-off-by: Dan Gil <dagil@nvidia.com>
|
2026-03-10 16:41:10 -05:00 |
|
Kris Hung
|
c619ce36b8
|
docs: Add numbers to showcase MM router perf (#7166)
|
2026-03-10 20:55:46 +00:00 |
|
zhongdaor-nv
|
6634f33f27
|
fix(perf): Skip duplicate image downloads and unnecessary image processing in MM Router (vLLM) (#7080)
Signed-off-by: zhongdaor <zhongdaor@nvidia.com>
Signed-off-by: zhongdaor-nv <zhongdaor@nvidia.com>
Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
|
2026-03-10 12:19:36 -07:00 |
|
jthomson04
|
864de4be5c
|
fix: Reduce max_num_tokens in wide_ep dsr1 prefill worker (#7133)
Signed-off-by: jthomson04 <jwillthomson19@gmail.com>
|
2026-03-10 11:08:25 -07:00 |
|
Keiven C
|
423e4b45aa
|
feat: add an auto GPU VRAM estimator for disagg-same-GPU (#6868)
Signed-off-by: Keiven Chang <keivenchang@users.noreply.github.com>
|
2026-03-10 07:35:01 -07:00 |
|
Indrajit Bhosale
|
f4e208105b
|
fix: Update free_gpu_memory_fraction for llama4 Maverick to avoid OOM in multinode setup (#7090)
|
2026-03-10 00:50:32 +00:00 |
|
Neal Vaidya
|
e14be96a1b
|
fix: [DYN-2335] correct docker tag for tritonserver build (#7037)
Signed-off-by: Neal Vaidya <nealv@nvidia.com>
|
2026-03-06 15:41:50 -08:00 |
|
Hongkuan Zhou
|
fb60cdc56f
|
fix: profiling data format in disagg_planner.yaml (#7019)
Signed-off-by: hongkuanz <hongkuanz@nvidia.com>
|
2026-03-06 18:37:14 -05:00 |
|
Yan Ru Pei
|
f934744fb9
|
fix: restore --enforce-disagg to reject requests before prefill router activates (#6957)
Signed-off-by: PeaBrane <yanrpei@gmail.com>
|
2026-03-06 04:39:28 +00:00 |
|
Indrajit Bhosale
|
7e55a73481
|
chore: Remove llava-hf recipies (#6954)
|
2026-03-05 22:23:30 +00:00 |
|
Tushar Sharma
|
876c976191
|
chore: add ephemeral-storage requests to deploy profiles (#6723)
Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
Co-authored-by: Dmitry Tokarev <dtokarev@nvidia.com>
|
2026-03-05 09:07:25 -05:00 |
|
Qi Wang
|
9780bf3ae3
|
perf: multimodal benchmark sweep (#6795)
|
2026-03-04 19:32:03 -08:00 |
|
Indrajit Bhosale
|
f0bfda1e0e
|
fix: Skip Encoder llm creation for unsupported models in trtllm (#6866)
Signed-off-by: Indrajit Bhosale <iamindrajitb@gmail.com>
|
2026-03-04 18:21:56 -08:00 |
|
zhongdaor-nv
|
d993f9d3db
|
fix: support TRT-LLM 1.3 apply_mm_hashes API (#6810)
Signed-off-by: zhongdaor <zhongdaor@nvidia.com>
|
2026-03-04 22:49:52 +00:00 |
|
atchernych
|
73a4ab3172
|
chore: Optimizations for epp interface complience (#6581)
Signed-off-by: Anna Tchernych <atchernych@nvidia.com>
|
2026-03-04 03:46:04 +00:00 |
|
GuanLuo
|
90d7463721
|
fix: properly setup and register vLLM worker for external / hybrid load balancing. Update launch script (#6695)
Signed-off-by: Guan Luo <41310872+GuanLuo@users.noreply.github.com>
|
2026-03-03 14:00:52 -08:00 |
|
Qi Wang
|
552ae18631
|
docs: delete script and add instructions (#6763)
|
2026-03-03 17:49:48 +00:00 |
|
Alec
|
9fe03dd864
|
docs: vLLM README container instructions and KV offloading page (#6793)
Signed-off-by: alec-flowers <aflowers@nvidia.com>
Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
|
2026-03-03 05:07:41 +00:00 |
|
Yue Yu
|
4d7c98458b
|
feat: FlexKV integration in Dynamo (#5858)
Signed-off-by: zhuofan1123 <zhuofanl@nvidia.com>
Co-authored-by: zhuofan1123 <zhuofanl@nvidia.com>
|
2026-03-02 20:18:39 -08:00 |
|
KrishnanPrash
|
146eb3b44f
|
fix: pass through extra args in trtllm agg.sh launch script (#6787)
Signed-off-by: Krishnan Prashanth <kprashanth@nvidia.com>
|
2026-03-02 17:50:56 -08:00 |
|
Tanmay Verma
|
1bb28d6ff3
|
docs: Refactor TensorRT-LLM backend docs (#6782)
|
2026-03-03 01:28:45 +00:00 |
|
Alec
|
75bf1e0993
|
docs: restructure vLLM docs and add startup banners to launch scripts (#6698)
Signed-off-by: alec-flowers <aflowers@nvidia.com>
Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
Co-authored-by: ishandhanani <ishandhanani@gmail.com>
|
2026-03-03 00:37:55 +00:00 |
|
Indrajit Bhosale
|
47ed1227fb
|
fix: Fix E + PD Multimodal Flow in trtllm (#6726)
Signed-off-by: Indrajit Bhosale <iamindrajitb@gmail.com>
|
2026-03-02 16:37:34 -08:00 |
|
Biswa Panda
|
1bc913ef6b
|
fix: update container image to standard vllm-runtime tag (#6781)
|
2026-03-02 23:32:24 +00:00 |
|
atchernych
|
ee33acf3c3
|
fix: Add weight: 1 to the EPP config plugins (#6756)
Signed-off-by: Anna Tchernych <atchernych@nvidia.com>
|
2026-03-02 22:13:44 +00:00 |
|
Hongkuan Zhou
|
a3cf35c3c8
|
fix: disagg_planner.yaml using new planner CLI (#6760)
Signed-off-by: hongkuanz <hongkuanz@nvidia.com>
|
2026-03-02 13:50:22 -08:00 |
|
GuanLuo
|
2cab0f7fcf
|
fix(perf): add embedding transfer implementation with NIXL WRITE initiation (#6651)
Signed-off-by: Guan Luo <41310872+GuanLuo@users.noreply.github.com>
|
2026-03-02 13:41:46 -08:00 |
|
Neal Vaidya
|
ece08dc926
|
docs: restructure docs directory and move fern config to fern/ (#6700)
Signed-off-by: Neal Vaidya <nealv@nvidia.com>
Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
|
2026-03-01 19:58:22 -08:00 |
|
Indrajit Bhosale
|
4ffa1082cd
|
fix: Phase out llava and make EPD single GPU (#6674)
Signed-off-by: Indrajit Bhosale <iamindrajitb@gmail.com>
|
2026-02-27 15:02:02 -08:00 |
|
KrishnanPrash
|
b1930a615f
|
chore: migrate vllm e/p/d test from gpu_2 -> gpu_1 (#6638)
Signed-off-by: Krishnan Prashanth <kprashanth@nvidia.com>
|
2026-02-27 09:15:45 +00:00 |
|
zhongdaor-nv
|
31642b963b
|
chore: add docs for vllm mm router (#6568)
Signed-off-by: zhongdaor <zhongdaor@nvidia.com>
Signed-off-by: zhongdaor-nv <zhongdaor@nvidia.com>
Co-authored-by: Ryan McCormick <rmccormick@nvidia.com>
|
2026-02-26 00:16:11 +00:00 |
|
William Arnold
|
4960dbb364
|
feat: enable returning routed experts info up through sglang (#6137)
Signed-off-by: William Arnold <7565007+Aphoh@users.noreply.github.com>
|
2026-02-25 16:07:48 -08:00 |
|
daiyaanarfeen
|
651ef5b506
|
feat: throughput-metrics-source for SLA planner + GlobalPlanner disagg scaling (#6500)
Signed-off-by: Daiyaan <darfeen@nvidia.com>
Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com>
|
2026-02-25 22:58:24 +00:00 |
|
jh-nv
|
42cde26489
|
fix: migrate example and document to the latest endpoint API (#6542)
|
2026-02-25 12:00:04 -08:00 |
|
Qi Wang
|
d5add7fffa
|
docs: embedding cache in vLLM and TRT-LLM (#6555)
|
2026-02-25 00:04:46 -08:00 |
|
Alec
|
35323da20c
|
fix: vllm launch script errors for disagg and spec decoding (#6562)
Signed-off-by: alec-flowers <aflowers@nvidia.com>
Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
|
2026-02-24 20:41:45 -08:00 |
|
Tushar Sharma
|
e3dc179304
|
fix: add explicit --kv-transfer-config to disagg prefill workers (#6554)
Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
|
2026-02-25 02:33:15 +00:00 |
|
Alec
|
eac94322f0
|
feat: add --kv-transfer-config NixlConnector to disagg scripts and recipes (#6560)
Signed-off-by: alec-flowers <aflowers@nvidia.com>
|
2026-02-25 02:26:47 +00:00 |
|
atchernych
|
c916cd42ff
|
feat: Support epp's "pods" interface in Dynamo fixes [DEP-424] (#6302)
Signed-off-by: Anna Tchernych <atchernych@nvidia.com>
|
2026-02-25 01:54:38 +00:00 |
|
KrishnanPrash
|
8e2363758e
|
fix: restore E/P/D multimodal disagg serving and add Qwen3-VL-30B-A3B support (#6533)
Signed-off-by: Krishnan Prashanth <kprashanth@nvidia.com>
Signed-off-by: KrishnanPrash <140860868+KrishnanPrash@users.noreply.github.com>
|
2026-02-24 14:13:10 -08:00 |
|
ishandhanani
|
6642e23e0f
|
feat: sglang to 0.5.9 + updated docs (#6518)
Co-authored-by: baihuitian <baihuitian.bht@gmail.com>
Co-authored-by: Cursor <cursoragent@cursor.com>
|
2026-02-24 21:48:25 +00:00 |
|
Alec
|
6d3b92f04e
|
feat: remove --connector flag for vLLM backend (LLM-90) (#6450)
Signed-off-by: alec-flowers <aflowers@nvidia.com>
Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
|
2026-02-24 17:49:00 +00:00 |
|
Alec
|
7893f2684e
|
feat: add --disaggregation-mode enum to vLLM backend (#6483)
Signed-off-by: alec-flowers <aflowers@nvidia.com>
Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
|
2026-02-23 19:34:30 -07:00 |
|
Julien Mancuso
|
8dd6369e55
|
feat: use vllm multiprocessing in multinode scenario (#6191)
Signed-off-by: Julien Mancuso <jmancuso@nvidia.com>
|
2026-02-23 22:30:34 +00:00 |