80cac7c14b
feat: Remove Component from public ( #6403 )
...
Signed-off-by: tzulingk@nvidia.com <tzulingk@nvidia.com>
2026-02-23 22:58:38 +00:00
0ce3461a9e
feat: Add runtime.endpoint() method to eliminate namespace chaining ( #6386 )
...
Signed-off-by: tzulingk@nvidia.com <tzulingk@nvidia.com>
2026-02-19 16:02:37 -05:00
4ba48b7993
feat: add model field to multimodal protocol for LoRA identity ( #6382 )
2026-02-19 12:48:00 -08:00
5624d14481
Rename fetch_llm to fetch_model ( #6268 )
...
Signed-off-by: tzulingk@nvidia.com <tzulingk@nvidia.com>
2026-02-13 23:22:35 +00:00
14eceb43df
chore: rename KvPushRouter to KvRouter in python + more bindings removal ( #6238 )
...
Signed-off-by: PeaBrane <yanrpei@gmail.com>
2026-02-12 23:44:52 -08:00
3e41702211
chore: remove and unify bindings in kv.rs ( #6016 )
...
Signed-off-by: PeaBrane <yanrpei@gmail.com>
2026-02-06 18:44:46 +00:00
feb6d272ac
chore: nuke ForwardPassMetrics ( #5531 )
...
Signed-off-by: PeaBrane <yanrpei@gmail.com>
2026-01-23 16:26:01 +00:00
03162161e4
chore: default python hash seed to zero always for the engines ( #5583 )
...
Signed-off-by: PeaBrane <yanrpei@gmail.com>
2026-01-22 17:20:24 -08:00
c5458b9954
chore(router): Remove deprecated best_worker_id and metrics_labels ( #5412 )
...
Signed-off-by: Graham King <grahamk@nvidia.com>
2026-01-14 16:20:42 +00:00
e22bb0376f
fix: Properly handle multiple text components from request ( #5196 )
2026-01-14 11:13:22 -05:00
a1333a8ddb
fix: update vLLM to 0.13.0 with API compatibility fixes ( #5222 )
...
Signed-off-by: Vasilis Vagias <vvagias@nvidia.com>
Signed-off-by: alec-flowers <aflowers@nvidia.com>
Co-authored-by: Vasilis Vagias <vvagias@nvidia.com>
2026-01-06 15:08:08 -08:00
2b157d89c2
fix: Allow requests to include audio contents before text ( #5143 )
2026-01-05 19:38:25 +00:00
cf433e6825
chore: update all copyright headers in repo to 2026 ( #5130 )
...
Signed-off-by: Tushar Sharma <tusharma@nvidia.com>
2026-01-02 22:08:23 +00:00
0173d5e6fb
fix: Fix multimodal EPD examples for vllm version bump ( #4849 )
2025-12-10 20:24:32 +00:00
96fe63fe2e
feat: nixl_connect: Improve Concurrency Support ( #4433 )
...
Signed-off-by: J Wyman <jwyman@nvidia.com>
2025-12-09 17:29:22 -05:00
db5687f535
fix: adjust usage of vLLM for deprecation of 'disable_log_requests' ( #4659 )
...
Co-authored-by: Ryan McCormick <rmccormick@nvidia.com>
2025-12-03 08:52:41 +00:00
69797b5ab3
feat: Only monitor NATS metrics if using NATS request plane ( #4442 )
...
Signed-off-by: Graham King <grahamk@nvidia.com>
2025-11-19 17:48:49 +00:00
0f5dd2b780
feat: Add vLLM multimodal audio support ( #2760 )
...
Signed-off-by: Yuekai Zhang <zhangyuekai@foxmail.com>
Signed-off-by: krishung5 <krish@nvidia.com>
Co-authored-by: Kris Hung <krish@nvidia.com>
2025-11-13 13:55:48 -08:00
e1af3af6ee
chore: Remove static mode ( #4235 )
...
Signed-off-by: Graham King <grahamk@nvidia.com>
2025-11-11 19:25:12 +00:00
6f8fd8655c
refactor: align multimodal example port allocation with vLLM components ( #4163 )
...
Co-authored-by: Claude <noreply@anthropic.com>
2025-11-07 13:24:10 -08:00
c78b590134
chore(runtime): Do not expose etcd lease ID ( #3915 )
...
Signed-off-by: Graham King <grahamk@nvidia.com>
2025-10-28 16:20:25 +00:00
069434b418
fix: Fix vllm multimodal tests ( #3361 )
...
Signed-off-by: krishung5 <krish@nvidia.com>
2025-10-06 10:49:41 -07:00
3865a94148
feat: Port vllm port allocator to Rust in bindings ( #3125 )
...
Signed-off-by: Graham King <grahamk@nvidia.com>
2025-09-19 18:34:52 +00:00
b6595e2484
chore(bindings): Provide a binding to clear etcd namespace ( #3094 )
...
Signed-off-by: Graham King <grahamk@nvidia.com>
2025-09-18 11:49:08 -04:00
960dc8967e
fix: replace hard coded dynamo namespace with env var ( #3048 )
...
Signed-off-by: Biswa Panda <biswa.panda@gmail.com>
2025-09-16 14:56:51 -07:00
27fad26faf
refactor: Split ModelType to ModelInput for request and response type; ModelType for the supported workloads ( #2714 )
...
Signed-off-by: Guan Luo <gluo@nvidia.com>
Signed-off-by: GuanLuo <41310872+GuanLuo@users.noreply.github.com>
Co-authored-by: Guan Luo <gluo@nvidia.com>
Co-authored-by: GuanLuo <41310872+GuanLuo@users.noreply.github.com>
2025-09-03 22:22:37 +00:00
dea5f88756
feat: Metrics labels for multimodal. ( #2835 )
...
Signed-off-by: tzulingk@nvidia.com <tzulingk@nvidia.com>
2025-09-03 19:18:42 +00:00
78a1107441
feat: Add vLLM multimodal video support ( #2738 )
...
Signed-off-by: krishung5 <krish@nvidia.com>
2025-08-28 03:41:56 +00:00
50cd81f320
feat: Add vllm multimodal qwen aggregated support ( #2694 )
2025-08-26 14:18:39 -07:00
9b9f2ce4f6
fix: pytest robustness and parsing error ( #2676 )
2025-08-24 10:13:41 -07:00
0a71aea62b
feat: Remove Duplicate Multimodel Nixl Connect Example ( #2622 )
2025-08-22 10:33:41 -07:00
ae4fb58f5c
fix: Remove metric_labels from multimodal codes. ( #2608 )
2025-08-21 19:07:57 +00:00
57728909cf
feat: Add model label for vllm backend metrics ( #2474 )
...
Co-authored-by: Keiven Chang <keivenchang@users.noreply.github.com>
2025-08-21 17:32:34 +00:00
844f8819a1
docs: Bring back some missed release/0.4.0 doc changes, fix broken links, add lychee link checker github action ( #2482 )
2025-08-18 10:26:15 -07:00
4dc529a1cc
chore: remove vLLM v0 multimodal example ( #2099 )
2025-07-25 15:22:33 -07:00
e0a51940d1
feat: Connect Library ( #1478 )
2025-07-23 14:16:34 -04:00
40d40ddabc
chore(multi-modal): Rename frontend.py to web.py ( #1951 )
2025-07-15 13:02:19 -07:00
e2716073c4
fix: Handle model not found error for multimodal example ( #1545 )
2025-06-17 08:54:14 -07:00
75503dae0b
feat: Video support with Dynamo ( #1443 )
...
Added support in examples for multimodal video with aggregated and disaggregated architecture using LLaVA-NeXT-Video-7B
2025-06-13 12:33:48 -07:00
e924a7c7c0
feat: generalize VLM embedding extraction ( #1388 )
...
Signed-off-by: hhzhang16 <54051230+hhzhang16@users.noreply.github.com>
Co-authored-by: Kris Hung <krish@nvidia.com>
2025-06-11 14:42:14 -07:00
b80482a4b2
feat: set model specific prompt templates in the multimodal config files, add documentation for multimodal example deployment ( #1366 )
2025-06-04 11:47:20 -07:00
68dc220c1e
docs: Add README for Connect Library ( #1303 )
...
Creates a README.md file for Connect.
The README contains and overview, examples w/ diagrams, and documents the important classes.
The README is not intended to be comprehensive.
Instead it's meant to be more of a "getting started" or "learn the basics".
More comprehensive information / documentation is available from the inline comments / documentation.
Additionally, updates the Multimodal Example:
Moves the remote and local prefill code from the generate method into remote_prefill and local_prefill respectively.
Code changes made.
Replaces reference to "agent" with "worker" for consistency reasons throughout the inline documentation.
Only comments updated. No code changes made.
The intention of this change is improve readability of the example code and to provide clearer examples to reference from within documentation.
DIS-101
2025-06-03 12:28:40 -04:00
92e33b86d8
perf: Create default sampling params only once during initialization ( #1294 )
2025-05-30 15:02:18 -07:00
5a30923f08
feat: Support OAI frontend format and add async image handing for multimodal ( #1214 )
...
Co-authored-by: J Wyman <jwyman@nvidia.com>
2025-05-28 16:38:01 -07:00
edc6fdea25
fix: Fix async_on_start syntax ( #1243 )
2025-05-28 11:37:33 -07:00
75e774d431
feat: NIXL Based RDMA Support w/ Multimodal Example ( #1060 )
2025-05-27 12:29:53 -07:00
6d46288c0c
feat: rename dynamo decorator ( #1133 )
2025-05-21 06:43:05 -07:00
d675d2218e
feat: decouple dynamo sdk to support mutiple deployment targets ( #905 )
2025-05-09 06:44:06 +00:00
10e912647a
feat: Add multimodal example with disaggregated serving ( #811 )
2025-05-07 16:21:44 -07:00
58df5aca4c
feat: Add multimodal example with aggregated serving ( #709 )
2025-05-02 12:45:31 -07:00
Tzu-Ling Kan