Biswa Panda
|
cec19d4db6
|
feat(perf): add request-plane, transport, and work-handler metric definitions (#6735)
|
2026-03-14 07:36:31 +00:00 |
|
jh-nv
|
c263a99ed2
|
feat: propagate OTEL trace context across E/P/D multimodal workers (#7239)
|
2026-03-14 01:20:41 +00:00 |
|
Biswa Panda
|
947939c727
|
fix: populate logprobs bytes and token fields in OpenAI-compatible responses (#6953)
|
2026-03-13 22:59:29 +00:00 |
|
Yan Ru Pei
|
cb9b1cd54c
|
test(kv-router): use flush-and-settle helper in indexer tests (#7347)
Signed-off-by: PeaBrane <yanrpei@gmail.com>
|
2026-03-13 13:55:51 -07:00 |
|
Yan Ru Pei
|
7e07495f6c
|
feat(mocker): add --decode-speedup-ratio for speculative decoding simulation (#7349)
Signed-off-by: PeaBrane <yanrpei@gmail.com>
|
2026-03-13 12:35:40 -07:00 |
|
Graham King
|
e6cb684709
|
fix: Upgrade futures-util to 0.3.32 (#7346)
Signed-off-by: Graham King <grahamk@nvidia.com>
|
2026-03-13 19:11:14 +00:00 |
|
Yan Ru Pei
|
5aa2f53fc2
|
feat(kv-router): add Prometheus metrics to standalone indexer (#7339)
Signed-off-by: PeaBrane <yanrpei@gmail.com>
|
2026-03-13 10:07:48 -07:00 |
|
Yan Ru Pei
|
bddaaa2658
|
feat(kv-router): pluggable scheduling policy for router queue [DYN-2454] (#7260)
Signed-off-by: PeaBrane <yanrpei@gmail.com>
|
2026-03-13 04:16:49 +00:00 |
|
Ryan Olson
|
17db1b6abd
|
fix: harden numa cpuset lookup (#7060)
Signed-off-by: Ryan Olson <rolson@nvidia.com>
|
2026-03-12 20:38:23 -07:00 |
|
Michael Feil
|
d170d56ae7
|
fix: cancellation kv commit router (#7178)
Signed-off-by: michaelfeil <63565275+michaelfeil@users.noreply.github.com>
|
2026-03-13 01:33:46 +00:00 |
|
Yan Ru Pei
|
dcbccbcd2e
|
feat(kv-router): add worker_type to router selection log (#7258)
Signed-off-by: PeaBrane <yanrpei@gmail.com>
|
2026-03-12 23:14:28 +00:00 |
|
Yan Ru Pei
|
f80ad6926a
|
feat(kv-router): package dynamo-kv-indexer binary via maturin [LLM-126] (#7194)
Signed-off-by: PeaBrane <yanrpei@gmail.com>
|
2026-03-12 21:33:11 +00:00 |
|
Hongkuan Zhou
|
cd4773fb31
|
feat: ForwardPassMetrics dynamo event plane integration (#7250)
Signed-off-by: hongkuanz <hongkuanz@nvidia.com>
|
2026-03-12 20:58:01 +00:00 |
|
dagil-nvidia
|
330f001d08
|
docs: update KVBM diagram from PNG to SVG (#7277)
Signed-off-by: akshatha-k <akshutk@gmail.com>
Signed-off-by: Dan Gil <dagil@nvidia.com>
Co-authored-by: akshatha-k <akshutk@gmail.com>
|
2026-03-12 15:26:05 -05:00 |
|
Anant Sharma
|
3f565053e0
|
chore: remove unused python dependencies (#7238)
Signed-off-by: Anant Sharma <anants@nvidia.com>
|
2026-03-12 15:28:57 -04:00 |
|
Graham King
|
87ac404eb7
|
test: Fix cancellation test (#7296)
Signed-off-by: Graham King <grahamk@nvidia.com>
|
2026-03-12 14:47:26 -04:00 |
|
Yan Ru Pei
|
bb07b2f455
|
feat(kv-router): ZMQ gap detection + replay for standalone indexer [LLM-126] (#7209)
Signed-off-by: PeaBrane <yanrpei@gmail.com>
|
2026-03-12 10:46:01 -07:00 |
|
MatejKosec
|
ba52c8ff5d
|
feat: support multimodal (vision) inputs in Anthropic Messages API (#7256)
Signed-off-by: Matej Kosec <mkosec@nvidia.com>
|
2026-03-12 17:34:07 +00:00 |
|
Ayush Agarwal
|
1182e2071d
|
feat: vllm omni image to video support (#6530)
Signed-off-by: ayushag <ayushag@nvidia.com>
|
2026-03-12 17:30:46 +00:00 |
|
brluo
|
745d7e4ca2
|
fix: respect per-request chat_template_kwargs for DeepSeek V3.2 thinking mode (#7286)
Co-authored-by: kangclzjc <kangz@nvidia.com>
|
2026-03-12 09:36:51 -04:00 |
|
Daniel Socek
|
387100c8d5
|
feat: replaces PersistentConnector monkey-patch with proper nixl_conn… (#6913)
Signed-off-by: Daniel Socek <daniel.socek@intel.com>
Co-authored-by: Ryan McCormick <rmccormick@nvidia.com>
|
2026-03-12 05:50:17 +00:00 |
|
Michael Feil
|
fca0a801dc
|
feat: follow up of optinal, batching-of-client-side-events (#6741)
Signed-off-by: michaelfeil <63565275+michaelfeil@users.noreply.github.com>
Signed-off-by: Michael Feil <michaelfeil@users.noreply.github.com>
Signed-off-by: Michael Feil <63565275+michaelfeil@users.noreply.github.com>
|
2026-03-11 23:30:51 +00:00 |
|
MatejKosec
|
5178a4a485
|
feat: streaming tool call and reasoning dispatch SSE events (#7114)
Signed-off-by: Matej Kosec <mkosec@nvidia.com>
|
2026-03-11 15:24:39 -07:00 |
|
daiyaanarfeen
|
5d5fd243da
|
feat: GlobalPlanner --max-total-gpus for cluster-wide GPU budget (#7103)
Signed-off-by: Daiyaan <darfeen@nvidia.com>
Signed-off-by: athreesh <anish.maddipoti@utexas.edu>
Signed-off-by: Anish <80174047+athreesh@users.noreply.github.com>
Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com>
Co-authored-by: athreesh <anish.maddipoti@utexas.edu>
Co-authored-by: Anish <80174047+athreesh@users.noreply.github.com>
|
2026-03-11 21:54:55 +00:00 |
|
Neal Vaidya
|
c95bfc2e59
|
fix: add force_nonempty_content for nemotron (#7225)
Signed-off-by: Neal Vaidya <nealv@nvidia.com>
|
2026-03-11 13:01:15 -07:00 |
|
Ryan Olson
|
ddf503596c
|
fix: removing reference to async-ucx (#7213)
Signed-off-by: Ryan Olson <rolson@nvidia.com>
|
2026-03-11 09:16:37 -06:00 |
|
Ryan Olson
|
eb99f293d9
|
chore: set kvbm subcrates to 0.1.0 (#7147)
|
2026-03-11 08:37:51 -06:00 |
|
jthomson04
|
761e848b5b
|
fix: Fix mocker+router segfaults + reliability fixes (#7162)
Signed-off-by: jthomson04 <jwillthomson19@gmail.com>
|
2026-03-10 22:39:06 -07:00 |
|
MatejKosec
|
012236ee4e
|
feat(anthropic): add thinking block support and preamble stripping to /v1/messages (#7137)
Signed-off-by: Matej Kosec <mkosec@nvidia.com>
|
2026-03-11 00:07:09 +00:00 |
|
Ryan Olson
|
72e849e780
|
chore: set velo crates to initial v0.1.0 (#7156)
Signed-off-by: Ryan Olson <rolson@nvidia.com>
Co-authored-by: Anant Sharma <anants@nvidia.com>
|
2026-03-10 17:57:24 -06:00 |
|
jh-nv
|
734c0b8b0b
|
feat: Add Loki log aggregation, and enable unified OTLP ingestion for both traces and logs through a single endpoint (#6974)
|
2026-03-10 15:20:33 -07:00 |
|
Biswa Panda
|
3c44b88e89
|
feat(perf): add frontend pipeline and tokio runtime perf metric definitions (#6731)
|
2026-03-10 21:04:02 +00:00 |
|
Anant Sharma
|
3cebc864f3
|
feat: split monolithic requirements.txt and remove test deps from runtime image (#6656)
Signed-off-by: Anant Sharma <anants@nvidia.com>
|
2026-03-10 20:05:12 +00:00 |
|
zhongdaor-nv
|
6634f33f27
|
fix(perf): Skip duplicate image downloads and unnecessary image processing in MM Router (vLLM) (#7080)
Signed-off-by: zhongdaor <zhongdaor@nvidia.com>
Signed-off-by: zhongdaor-nv <zhongdaor@nvidia.com>
Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
|
2026-03-10 12:19:36 -07:00 |
|
Graham King
|
5265787b5d
|
chore: Enable "multipart" feature of reqwest (#7084)
Signed-off-by: Graham King <grahamk@nvidia.com>
|
2026-03-10 13:47:01 +00:00 |
|
Ryan Olson
|
b685291960
|
fix: handle early arrivals (#7126)
Signed-off-by: Ryan Olson <rolson@nvidia.com>
|
2026-03-10 09:56:06 +00:00 |
|
Yan Ru Pei
|
236cb17d00
|
fix(mocker): align vLLM scheduler with v1 — drop watermark, LIFO preemption, retry loop (#7020)
Signed-off-by: PeaBrane <yanrpei@gmail.com>
Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
|
2026-03-10 09:53:50 +00:00 |
|
Biswa Panda
|
b407b41969
|
fix: resolve HTTP port collisions when multiple workers share a process (#7063)
|
2026-03-10 05:50:52 +00:00 |
|
Yan Ru Pei
|
48eb52e777
|
feat(kv-router): P2P recovery for standalone KV indexer [DYN-2367] (#6934)
Signed-off-by: PeaBrane <yanrpei@gmail.com>
Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
|
2026-03-09 23:52:48 +00:00 |
|
ishandhanani
|
fab07d1c5b
|
fix: reject guided_grammar with excessive nesting depth (#7091)
|
2026-03-09 23:38:39 +00:00 |
|
Ryan Olson
|
63d7c01c9c
|
feat: velo-backend (#6547)
Signed-off-by: Ryan Olson <rolson@nvidia.com>
|
2026-03-09 23:26:25 +00:00 |
|
Yan Ru Pei
|
e96f71bad8
|
fix(prefill-router): pass preselected_worker in sync prefill fallback path (#7055)
Signed-off-by: PeaBrane <yanrpei@gmail.com>
|
2026-03-09 22:48:05 +00:00 |
|
Yan Ru Pei
|
2e29620df1
|
chore(kv-router): move benches to lib/bench to break circular dep [OPS-3752] (#7013)
Signed-off-by: PeaBrane <yanrpei@gmail.com>
|
2026-03-06 23:24:03 +00:00 |
|
Yifan Jiang
|
100819299f
|
feat(trtllm): add additional metrics for dynamo-trtllm (#6668)
Signed-off-by: Yifan Jiang <19356972+yifjiang@users.noreply.github.com>
Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
|
2026-03-06 14:34:38 -08:00 |
|
jh-nv
|
2831bfecde
|
chore: add mypy typing to vllm (#6858)
|
2026-03-06 21:45:43 +00:00 |
|
Shounak Ray
|
64ae47fdc9
|
fix: use checked arithmetic in TwoPartCodec to prevent integer overflow (#6959)
Signed-off-by: ShounakRay <shounak@stanford.edu>
|
2026-03-06 19:53:44 +00:00 |
|
Schwinn Saereesitthipitak
|
a9064c5113
|
fix: hide inactive models from /v1/models (#6966)
|
2026-03-06 18:35:51 +00:00 |
|
Biswa Panda
|
d71c0c324a
|
fix: Kimi K2.5 - tiktoken incomplete multi-byte sequence handling with regression tests (#6996)
|
2026-03-06 18:27:16 +00:00 |
|
MatejKosec
|
9b2b44e39d
|
fix(responses): accept assistant output_text messages without id/status in input (#6599)
Signed-off-by: Marko Kosec <mkosec@nvidia.com>
Signed-off-by: Matej Kosec <mkosec@nvidia.com>
Signed-off-by: Vasilis Vagias <vvagias@nvidia.com>
Co-authored-by: vvagias <vasilis.n.vagias@gmail.com>
Co-authored-by: ishandhanani <82981111+ishandhanani@users.noreply.github.com>
|
2026-03-06 17:44:23 +00:00 |
|
Graham King
|
abc02c689f
|
fix: llm/mocker: Remove the llm -> mocker crate dependency, move config (#6998)
Signed-off-by: Graham King <grahamk@nvidia.com>
Signed-off-by: PeaBrane <yanrpei@gmail.com>
|
2026-03-06 09:41:39 -08:00 |