06b0ebef88
feat: transport agnostic request plane for dynamo - natless ( #4246 )
2025-11-13 17:54:54 +00:00
38242c8d7b
chore: use notify instead of inotify ( #4250 )
...
Signed-off-by: PeaBrane <yanrpei@gmail.com>
2025-11-12 16:53:39 +00:00
fbad28602e
chore: Remove the python bindings for port allocation ( #4237 )
...
Signed-off-by: Graham King <grahamk@nvidia.com>
2025-11-11 20:35:36 +00:00
7afb5431ff
fix: add kube impl for discovery and add metadata endpoint ( #4136 )
...
Signed-off-by: mohammedabdulwahhab <furkhan324@berkeley.edu>
Co-authored-by: tmontfort <tmontfort@nvidia.com>
2025-11-10 22:42:04 +00:00
23660bc5c3
chore: update versions for 0.7.0 release ( #4217 )
...
Signed-off-by: Anant Sharma <anants@nvidia.com>
2025-11-10 21:40:06 +00:00
794c0a4431
feat(keyvalue): Filesystem backed KeyValueStore ( #4138 )
...
Signed-off-by: Graham King <grahamk@nvidia.com>
2025-11-07 21:11:33 +00:00
3d036fc4a4
feat: Reject unsupported parameters with 400 Bad Request (`/v1/completions`) ( #4140 )
...
Signed-off-by: Krishnan Prashanth <kprashanth@nvidia.com>
2025-11-05 17:32:50 -08:00
b73c571f0f
feat: Add base64 and HTTP image URL support to vLLM workers ( #4114 )
...
Signed-off-by: Krishnan Prashanth <kprashanth@nvidia.com>
2025-11-05 16:47:06 -08:00
6afa679c5d
chore: KVBM pip wheel ( #3826 )
...
Signed-off-by: Anant Sharma <anants@nvidia.com>
Co-authored-by: Anant Sharma <anants@nvidia.com>
2025-10-31 14:19:30 -07:00
1da9d70ab1
chore: bump nixl verison to 0.7.0 ( #3936 )
...
Signed-off-by: Tushar Sharma <tusharma@nvidia.com>
2025-10-30 10:28:17 -07:00
3998fdcb28
feat: KVBM V2 Initial Migration ( #3861 )
...
Signed-off-by: jthomson04 <jwillthomson19@gmail.com>
2025-10-28 16:08:14 -07:00
9bb1af3376
chore: Update version to 0.6.1 ( #3916 )
...
Signed-off-by: Tushar Sharma <tusharma@nvidia.com>
2025-10-27 17:17:26 -04:00
94aa2a7bbb
refactor: kvbm modularity DIS-657 Eliminate ETCD from the leader-worker initialization ( #3202 )
...
Signed-off-by: richardhuo-nv <rihuo@nvidia.com>
2025-10-23 00:10:09 -07:00
9ae98ed7f9
refactor(runtime): Replace std::sync::Mutex with parking_lot::Mutex ( #3740 )
...
Signed-off-by: Graham King <grahamk@nvidia.com>
2025-10-21 12:38:51 +00:00
cd2389baef
chore: pre-0.6.0 activities ( #3592 )
...
Signed-off-by: Harrison Saturley-Hall <hsaturleyhal@nvidia.com>
2025-10-13 17:45:19 -04:00
1f92dd547e
feat: OTEL Exporter and Tempo Visualization ( #3307 )
...
Signed-off-by: mohammedabdulwahhab <furkhan324@berkeley.edu>
2025-10-13 18:39:09 +00:00
5c69c11930
chore: upgrade nixl to 0.6.0 ( #3550 )
...
Signed-off-by: richardhuo-nv <rihuo@nvidia.com>
2025-10-10 13:18:21 -07:00
ef04a0869e
feat: Add OpenAPI routes to serve HTTP server API documentation ( #3480 )
2025-10-08 13:52:19 -07:00
1b1265e6fc
chore: Remove GGUF support ( #3488 )
...
Signed-off-by: Graham King <grahamk@nvidia.com>
2025-10-08 11:42:24 -04:00
67e1f6ee11
feat: enable parallel tool calling and add testing ( #3188 )
...
Signed-off-by: Elyas Mehtabuddin <emehtabuddin@nvidia.com>
Co-authored-by: Ryan McCormick <rmccormick@nvidia.com>
2025-09-29 15:30:42 +00:00
c03e2f6bbe
chore: Migrate planner virtual_connector internals into bindings ( #3205 )
...
Signed-off-by: Graham King <grahamk@nvidia.com>
2025-09-25 18:04:58 -04:00
980727bba1
chore: bump versions ahead of 0.5.1 release ( #3209 )
...
Signed-off-by: Harrison Saturley-Hall <hsaturleyhal@nvidia.com>
2025-09-24 18:26:38 -04:00
c63cceaa77
feat: JailedStream ( #3034 )
...
Signed-off-by: ayushag <ayushag@nvidia.com>
2025-09-23 22:24:17 +00:00
3865a94148
feat: Port vllm port allocator to Rust in bindings ( #3125 )
...
Signed-off-by: Graham King <grahamk@nvidia.com>
2025-09-19 18:34:52 +00:00
31f5ed3ce7
feat: add finish reason = tool_calls for stream=False and phi-4 detect token start fix ( #3087 )
...
Signed-off-by: Elyas Mehtabuddin <emehtabuddin@nvidia.com>
2025-09-18 21:45:52 +00:00
f88d7dc74b
chore(bindings): Remove NatsQueue ( #3086 )
...
Signed-off-by: Graham King <grahamk@nvidia.com>
2025-09-17 12:46:54 -04:00
08cb08c1bc
feat: Canary Health Check. ( #2903 )
...
Signed-off-by: tzulingk@nvidia.com <tzulingk@nvidia.com>
2025-09-17 03:31:56 +00:00
87e6e0529d
fix: Interactive inputs actually stops, does not ignore stop token ( #3057 )
...
Signed-off-by: Graham King <grahamk@nvidia.com>
2025-09-16 14:39:05 -04:00
435803ea16
chore: add additional param support for multimodal models ( #3042 )
...
Signed-off-by: Ryan Lempka <rlempka@nvidia.com>
2025-09-16 16:26:39 +00:00
6f14e94159
feat: Add a checksum to ModelDeploymentCard fields ( #2934 )
...
Signed-off-by: Graham King <grahamk@nvidia.com>
2025-09-09 16:12:42 +00:00
27fad26faf
refactor: Split ModelType to ModelInput for request and response type; ModelType for the supported workloads ( #2714 )
...
Signed-off-by: Guan Luo <gluo@nvidia.com>
Signed-off-by: GuanLuo <41310872+GuanLuo@users.noreply.github.com>
Co-authored-by: Guan Luo <gluo@nvidia.com>
Co-authored-by: GuanLuo <41310872+GuanLuo@users.noreply.github.com>
2025-09-03 22:22:37 +00:00
c920cbd9dc
feat: Add --custom-jinja-template argument to pass a custom chat template for vLLM ( #2829 )
...
Signed-off-by: Krishnan Prashanth <kprashanth@nvidia.com>
2025-09-03 14:22:15 -07:00
561ecb98a2
chore: bump version numbers ahead of 0.5.0 release ( #2812 )
...
Signed-off-by: Harrison King Saturley-Hall <hsaturleyhal@nvidia.com>
2025-09-02 16:13:28 -04:00
95ce83d59a
feat: Integrate Model Express Client into Dynamo Model Downloads ( #2574 )
...
Signed-off-by: Kavin Krishnan <kavink@nvidia.com>
Co-authored-by: KavinKrishnan <kavin.krishnan@nvidia.com>
2025-08-28 16:09:04 -07:00
91a459c038
feat: KServe gRPC support ( #2638 )
2025-08-26 22:57:31 -07:00
f08729ae15
feat: python bindings for the entire KvPushRouter + per-request router configs ( #2658 )
2025-08-25 22:27:10 +00:00
3036e60b1e
feat: add gpt oss reasoning parser through harmony ( #2656 )
...
- couple of refactors
- added a new dependency, openai-harmony
- implemented the gpt oss parser
2025-08-25 17:13:38 +00:00
b658ba6139
feat: enable dynamo metrics on KVBM ( #2626 )
2025-08-22 19:58:05 +00:00
49958435eb
chore: Remove async-openai-macros ( #2554 )
2025-08-20 11:33:10 -07:00
d319abf3a3
feat: upload/download rust structs directly through NATs object store ( #2540 )
2025-08-20 17:17:21 +00:00
9a02188531
chore: Bumped Dynamo version to 0.4.1 ( #2545 )
2025-08-19 21:55:37 -04:00
07cfc3a11b
feat: kvbm + connector ( #2258 )
...
Signed-off-by: Ryan Olson <rolson@nvidia.com>
Co-authored-by: Olga Andreeva <oandreeva@nvidia.com>
Co-authored-by: Ziqi Fan <ziqif@nvidia.com>
Co-authored-by: John Thompson <jothomson@nvidia.com>
Co-authored-by: Richard Huo <rihuo@nvidia.com>
Co-authored-by: Zicheng Ma <zichengm@nvidia.com>
2025-08-19 12:36:53 -07:00
a33033b7f6
feat: task scheduler ( #2406 )
...
Signed-off-by: Ryan Olson <ryanolson@users.noreply.github.com>
2025-08-19 12:35:31 -06:00
a4bbe49228
feat(http): TLS support ( #2492 )
2025-08-18 16:06:29 -04:00
0444217339
fix: replace metrics callback with background scraping to prevent tim… ( #2480 )
...
Co-authored-by: Keiven Chang <keivenchang@users.noreply.github.com>
2025-08-18 12:38:50 -07:00
c12c25787f
fix: upgrade cudarc to 0.17.1 ( #2341 )
...
Signed-off-by: Dan Aloni <dan.aloni@vastdata.com>
Co-authored-by: Tushar Sharma <tusharma@nvidia.com>
2025-08-13 15:45:08 -04:00
5166a3dd44
feat: Router replicas with state-sharing ( #2264 )
2025-08-07 22:57:57 +00:00
bd4fe1a7d1
feat: cross process instrumentation ( #2243 )
...
Signed-off-by: Neelay Shah <neelays@nvidia.com>
Co-authored-by: coderabbitai[bot] <136622811+coderabbitai[bot]@users.noreply.github.com>
Co-authored-by: Olga Andreeva <124622579+oandreeva-nv@users.noreply.github.com>
2025-08-07 15:32:48 +00:00
dbe48a1d2b
chore: Bump mistral.rs, llama.cpp and tokenizers deps ( #2338 )
2025-08-06 16:23:54 -04:00
b2aa504b47
fix: upgrade axum to 0.8 and etcd-client to 0.16 ( #2317 )
...
Signed-off-by: Dan Aloni <dan.aloni@vastdata.com>
2025-08-06 18:25:49 +00:00
Biswa Panda