Neelay Shah
|
d22d9e761e
|
fix: support truthy values for OTEL_EXPORT_ENABLED in bindings layer [DYN-1690][DYN-1646] (#5129)
Co-authored-by: Claude <noreply@anthropic.com>
|
2026-01-02 23:39:09 +00:00 |
|
Tushar Sharma
|
cf433e6825
|
chore: update all copyright headers in repo to 2026 (#5130)
Signed-off-by: Tushar Sharma <tusharma@nvidia.com>
|
2026-01-02 22:08:23 +00:00 |
|
Yan Ru Pei
|
0b33c1dfa0
|
fix: sglang disagg routing fixes and optimizations [DYN-1692] (#5106)
Signed-off-by: PeaBrane <yanrpei@gmail.com>
Co-authored-by: Ishan Dhanani <ishandhanani@gmail.com>
Co-authored-by: Sean SH Choi <sechoi@nvidia.com>
Co-authored-by: ishandhanani <82981111+ishandhanani@users.noreply.github.com>
|
2025-12-31 10:55:44 -08:00 |
|
Phillippe Siclait
|
ac8c9023e4
|
fix: Truncate HttpError message to first 8192 characters (#5020)
Signed-off-by: Phillippe Siclait <546332+siclait@users.noreply.github.com>
|
2025-12-22 12:23:35 -08:00 |
|
Yan Ru Pei
|
f4245c9929
|
chore: clean up PreprocessedRequest (#5040)
Signed-off-by: PeaBrane <yanrpei@gmail.com>
|
2025-12-19 23:15:59 +00:00 |
|
Jacky
|
e6a6a1f244
|
feat: Request Migration Metrics (#5029)
Signed-off-by: Jacky <18255193+kthui@users.noreply.github.com>
|
2025-12-19 12:22:36 -08:00 |
|
Nate Mailhot
|
910b587db1
|
chore: update nixl to 0.8.0 and ucx 1.20 (#5007)
Signed-off-by: Nate Mailhot <nmailhot@nvidia.com>
|
2025-12-19 10:55:40 -08:00 |
|
Anant Sharma
|
7436e3bc03
|
chore: update versions for 0.8.0 release (#5031)
Signed-off-by: Anant Sharma <anants@nvidia.com>
|
2025-12-19 10:21:52 -08:00 |
|
milesial
|
d2faf0e6f5
|
feat: Runtime media decoder config (#5011)
Signed-off-by: Alexandre Milesi <milesial@users.noreply.github.com>
|
2025-12-19 05:34:09 +00:00 |
|
zhongdaor-nv
|
481dc6363b
|
feat: add multimodal support to KV router with standalone trtllm example (#4577)
Signed-off-by: zhongdaor <zhongdaor@nvidia.com>
Signed-off-by: zhongdaor-nv <zhongdaor@nvidia.com>
|
2025-12-18 21:01:45 -08:00 |
|
Keiven C
|
c22280ccf4
|
refactor: frontend/grpc tests to use dynamic ports (#4992)
Signed-off-by: Keiven Chang <keivenchang@users.noreply.github.com>
Co-authored-by: Keiven Chang <keivenchang@users.noreply.github.com>
|
2025-12-18 16:21:28 -08:00 |
|
Yan Ru Pei
|
85b5c808ac
|
feat: unify per-request metric tracking (#5004)
Signed-off-by: PeaBrane <yanrpei@gmail.com>
|
2025-12-18 23:59:25 +00:00 |
|
Biswa Panda
|
18b64e90b6
|
feat: switch to tcp as default request plane (#4845)
|
2025-12-18 23:05:38 +00:00 |
|
Anant Sharma
|
5b82b8b096
|
chore: update versions to match latest release (#4966)
Signed-off-by: Anant Sharma <anants@nvidia.com>
|
2025-12-18 08:45:44 -08:00 |
|
Graham King
|
da0f2fb84a
|
feat(frontend): First part of Python request handling (#4999)
Signed-off-by: Graham King <grahamk@nvidia.com>
|
2025-12-17 20:03:39 -05:00 |
|
Sean SH Choi
|
1c39e4835c
|
feat: add frontend based prefill request routing for sglang (#4635)
Signed-off-by: Sean Choi <sechoi@nvidia.com>
Signed-off-by: PeaBrane <yanrpei@gmail.com>
Co-authored-by: Yan Ru Pei <yanrpei@gmail.com>
Co-authored-by: ishandhanani <82981111+ishandhanani@users.noreply.github.com>
|
2025-12-17 09:51:46 -08:00 |
|
milesial
|
74fcd4a9b9
|
feat: video decoder in the frontend (#4719)
Signed-off-by: Alexandre Milesi <milesial@users.noreply.github.com>
|
2025-12-16 14:28:06 -08:00 |
|
KrishnanPrash
|
24ac6c1588
|
chore: bump minijinja to 2.14.0 (#4949)
Signed-off-by: Krishnan Prashanth <kprashanth@nvidia.com>
|
2025-12-15 09:58:52 -08:00 |
|
Yan Ru Pei
|
c2a29f8045
|
chore: enable local indexers for sglang and trtllm (#4932)
Signed-off-by: PeaBrane <yanrpei@gmail.com>
|
2025-12-12 11:41:37 -08:00 |
|
Karen Chung
|
332499458b
|
feat: worker-local KvIndexer in KvEventPublisher (#4519)
Co-authored-by: Yan Ru Pei <yanrpei@gmail.com>
|
2025-12-11 23:18:34 +00:00 |
|
Yan Ru Pei
|
10b01b4590
|
feat: early rejection based on active prefill tokens (#4837)
Signed-off-by: PeaBrane <yanrpei@gmail.com>
|
2025-12-11 23:16:02 +00:00 |
|
Dmitry Tokarev
|
2ef0f5fbab
|
fix: relax pydantic requirements (#4865)
Signed-off-by: Dmitry Tokarev <dtokarev@nvidia.com>
|
2025-12-11 19:32:19 +00:00 |
|
Vladislav Nosivskoy
|
f8bb53c0ed
|
feat: add cached tokens prometheus metric (#4534)
Signed-off-by: Vladislav Nosivskoy <vladnosiv@gmail.com>
Co-authored-by: Ryan McCormick <rmccormick@nvidia.com>
|
2025-12-11 08:03:12 +03:00 |
|
Jacky
|
00f8615e47
|
test: Ensure Request Cancellation, Migration, Rejection Work with TCP transport (#4875)
Signed-off-by: Jacky <18255193+kthui@users.noreply.github.com>
|
2025-12-11 02:21:17 +00:00 |
|
J Wyman
|
96fe63fe2e
|
feat: nixl_connect: Improve Concurrency Support (#4433)
Signed-off-by: J Wyman <jwyman@nvidia.com>
|
2025-12-09 17:29:22 -05:00 |
|
Keiven C
|
1e37c1033f
|
fix: add timeout to tests before lora to unblock (#4808)
Signed-off-by: Keiven Chang <keivenchang@users.noreply.github.com>
Co-authored-by: Keiven Chang <keivenchang@users.noreply.github.com>
|
2025-12-08 20:50:33 -08:00 |
|
Biswa Panda
|
5588d77074
|
test: add timeout for lora utils tests (#4781)
|
2025-12-06 02:23:20 +00:00 |
|
William Arnold
|
3bc129dc37
|
feat: Add python-configurable engine routes for sglang profiling (#4617)
Signed-off-by: William Arnold <7565007+Aphoh@users.noreply.github.com>
|
2025-12-05 10:28:36 -08:00 |
|
Neal Vaidya
|
118323f26e
|
fix: skip HuggingFace download for non-llms (#4686)
Signed-off-by: Neal Vaidya <nealv@nvidia.com>
|
2025-12-04 10:23:24 -08:00 |
|
Biswa Panda
|
71f94eda23
|
feat: lora - centralize lora cache key, restructure folders, s3 resiliency (#4644)
|
2025-12-02 08:57:25 +00:00 |
|
Biswa Panda
|
6a0e67ed78
|
feat: add LoRA common APIs and implementation for lora management (#4464)
|
2025-11-26 20:29:25 +00:00 |
|
Graham King
|
fcb91e4b3d
|
refactor(storage): Remove the stuttering from key_value_store. (#4604)
Signed-off-by: Graham King <grahamk@nvidia.com>
|
2025-11-25 22:59:29 +00:00 |
|
milesial
|
0a06591826
|
feat: decoded media via NIXL (#3988)
Signed-off-by: Alexandre Milesi <milesial@users.noreply.github.com>
Signed-off-by: Ayush Agarwal <ayushag@nvidia.com>
Signed-off-by: Krishnan Prashanth <kprashanth@nvidia.com>
Co-authored-by: Ayush Agarwal <ayushag@nvidia.com>
Co-authored-by: Krishnan Prashanth <kprashanth@nvidia.com>
Co-authored-by: KrishnanPrash <140860868+KrishnanPrash@users.noreply.github.com>
|
2025-11-25 19:15:04 +00:00 |
|
Graham King
|
0fc5273c11
|
refactor(llm): Rename EngineConfig::Static to InProcess (#4585)
Signed-off-by: Graham King <grahamk@nvidia.com>
|
2025-11-25 12:22:09 -05:00 |
|
Graham King
|
27904535fb
|
fix(storage): Correctly encoding FileStore keys (#4539)
Signed-off-by: Graham King <grahamk@nvidia.com>
|
2025-11-21 17:01:40 -05:00 |
|
Yan Ru Pei
|
c61e0dd3cc
|
chore: merge KvIndexer and ApproxKvIndexer (#4500)
Signed-off-by: PeaBrane <yanrpei@gmail.com>
|
2025-11-21 09:09:11 -08:00 |
|
Yan Ru Pei
|
d821a8b9f7
|
chore: parallelize planner profile tests + bindings test cleanup (#4532)
Signed-off-by: PeaBrane <yanrpei@gmail.com>
|
2025-11-21 09:21:43 +00:00 |
|
Kris Hung
|
2f18b23ee7
|
chore: Use general engine source for KVBM KV events (#4515)
|
2025-11-20 14:47:48 -08:00 |
|
Graham King
|
8a14f9edc1
|
chore: Remove DistributedRuntime::etcd_client (#4489)
Signed-off-by: Graham King <grahamk@nvidia.com>
|
2025-11-20 12:35:10 -05:00 |
|
Biswa Panda
|
781331c6f3
|
feat: unregister discovery instance (#4459)
|
2025-11-19 20:25:44 +00:00 |
|
Graham King
|
69797b5ab3
|
feat: Only monitor NATS metrics if using NATS request plane (#4442)
Signed-off-by: Graham King <grahamk@nvidia.com>
|
2025-11-19 17:48:49 +00:00 |
|
Biswa Panda
|
684107c461
|
feat: generate deterministic lora id from lora name (#4457)
|
2025-11-19 02:26:00 +00:00 |
|
Graham King
|
2f1778c13d
|
fix(storage): File store shouldn't notify on metdata changes (#4434)
Signed-off-by: Graham King <grahamk@nvidia.com>
|
2025-11-18 10:24:04 -08:00 |
|
J Wyman
|
0814653104
|
feat: nixl_connect: Change the way `bytes` are passed to NIXL (#4314)
Signed-off-by: J Wyman <jwyman@nvidia.com>
Co-authored-by: Ryan McCormick <rmccormick@nvidia.com>
|
2025-11-18 18:05:46 +00:00 |
|
Yan Ru Pei
|
fc2ad4eb1e
|
feat: mocker can use planner profile data (#4422)
Signed-off-by: PeaBrane <yanrpei@gmail.com>
Signed-off-by: Yan Ru Pei <yanrpei@gmail.com>
Co-authored-by: Hongkuan Zhou <tedzhouhk@gmail.com>
|
2025-11-18 08:06:50 +00:00 |
|
Keiven C
|
0e77d3442e
|
refactor: centralize environment variable constants (#4083)
Signed-off-by: Keiven Chang <keivenchang@users.noreply.github.com>
Co-authored-by: Keiven Chang <keivenchang@users.noreply.github.com>
|
2025-11-17 11:16:38 -08:00 |
|
Graham King
|
886506c12d
|
feat: Command line flag to set request plane mode: tcp, http or nats (#4365)
Signed-off-by: Graham King <grahamk@nvidia.com>
|
2025-11-17 09:02:36 -08:00 |
|
Waël Boukhobza
|
2cf427ed8f
|
fix(router): change prune channel to be mpsc instead of watch, increase max size (#4351)
Signed-off-by: Wael Boukhobza <wawa_wael@live.fr>
|
2025-11-14 18:19:20 -05:00 |
|
Yan Ru Pei
|
ce83398366
|
chore: better error handling in prefill router (#4286)
Signed-off-by: PeaBrane <yanrpei@gmail.com>
|
2025-11-13 22:29:26 +00:00 |
|
Biswa Panda
|
06b0ebef88
|
feat: transport agnostic request plane for dynamo - natless (#4246)
|
2025-11-13 17:54:54 +00:00 |