Biswa Panda
|
06b0ebef88
|
feat: transport agnostic request plane for dynamo - natless (#4246)
|
2025-11-13 17:54:54 +00:00 |
|
Yan Ru Pei
|
8379b0cd10
|
feat: kv router should route to available instances (#4225)
Signed-off-by: PeaBrane <yanrpei@gmail.com>
|
2025-11-13 02:34:58 +00:00 |
|
Yan Ru Pei
|
38242c8d7b
|
chore: use notify instead of inotify (#4250)
Signed-off-by: PeaBrane <yanrpei@gmail.com>
|
2025-11-12 16:53:39 +00:00 |
|
Waël Boukhobza
|
3ea22fcf39
|
feat(router): max tree size based pruning (#4057)
Signed-off-by: Wael Boukhobza <wawa_wael@live.fr>
|
2025-11-12 08:33:49 -08:00 |
|
Graham King
|
fbad28602e
|
chore: Remove the python bindings for port allocation (#4237)
Signed-off-by: Graham King <grahamk@nvidia.com>
|
2025-11-11 20:35:36 +00:00 |
|
Graham King
|
e1af3af6ee
|
chore: Remove static mode (#4235)
Signed-off-by: Graham King <grahamk@nvidia.com>
|
2025-11-11 19:25:12 +00:00 |
|
Aryan Bagade
|
8f868986e8
|
feat: Add output token counter to frontend metrics (#4202)
Signed-off-by: Aryan Bagade <aryan@aryanbagade.com>
|
2025-11-11 05:09:07 +00:00 |
|
mohammedabdulwahhab
|
7afb5431ff
|
fix: add kube impl for discovery and add metadata endpoint (#4136)
Signed-off-by: mohammedabdulwahhab <furkhan324@berkeley.edu>
Co-authored-by: tmontfort <tmontfort@nvidia.com>
|
2025-11-10 22:42:04 +00:00 |
|
zhongdaor-nv
|
ec7af93953
|
fix: Extend add_tensor_model so that ModelDeploymentCard can be correctly picked up (#4169)
Signed-off-by: zhongdaor <zhongdaor@nvidia.com>
|
2025-11-10 13:59:40 -08:00 |
|
Anant Sharma
|
23660bc5c3
|
chore: update versions for 0.7.0 release (#4217)
Signed-off-by: Anant Sharma <anants@nvidia.com>
|
2025-11-10 21:40:06 +00:00 |
|
Keiven C
|
afccc9d434
|
refactor: consolidate Observability files (e.g. OTEL docker-compose, md files) (#4173)
Signed-off-by: Keiven Chang <keivenchang@users.noreply.github.com>
Co-authored-by: Keiven Chang <keivenchang@users.noreply.github.com>
|
2025-11-10 20:19:34 +00:00 |
|
Graham King
|
cf630bf7b6
|
refactor: Make the Runtime and DistributedRuntime fields private (#4193)
Signed-off-by: Graham King <grahamk@nvidia.com>
|
2025-11-10 15:56:22 +00:00 |
|
mohammedabdulwahhab
|
09b26bf6b3
|
fix: refactor to use service discovery (#4092)
Signed-off-by: mohammedabdulwahhab <furkhan324@berkeley.edu>
|
2025-11-08 18:05:18 +00:00 |
|
milesial
|
14af074ecd
|
feat: Media decoder and fetcher options in the MDC (#4094)
Signed-off-by: Alexandre Milesi <milesial@users.noreply.github.com>
|
2025-11-08 02:17:39 +00:00 |
|
Keiven C
|
6bccf099f8
|
feat: deprecate DYN_SYSTEM_ENABLED in favor of DYN_SYSTEM_PORT (#4082)
Signed-off-by: Keiven Chang <keivenchang@users.noreply.github.com>
Co-authored-by: Keiven Chang <keivenchang@users.noreply.github.com>
|
2025-11-07 15:31:51 -08:00 |
|
Graham King
|
794c0a4431
|
feat(keyvalue): Filesystem backed KeyValueStore (#4138)
Signed-off-by: Graham King <grahamk@nvidia.com>
|
2025-11-07 21:11:33 +00:00 |
|
KrishnanPrash
|
3d036fc4a4
|
feat: Reject unsupported parameters with 400 Bad Request (`/v1/completions`) (#4140)
Signed-off-by: Krishnan Prashanth <kprashanth@nvidia.com>
|
2025-11-05 17:32:50 -08:00 |
|
Jacky
|
6e2b22eaef
|
feat: ETCD high availability client failover - lease watch resilience (#3950)
Signed-off-by: Jacky <18255193+kthui@users.noreply.github.com>
Co-authored-by: Ryan McCormick <rmccormick@nvidia.com>
|
2025-11-05 17:21:09 -08:00 |
|
KrishnanPrash
|
b73c571f0f
|
feat: Add base64 and HTTP image URL support to vLLM workers (#4114)
Signed-off-by: Krishnan Prashanth <kprashanth@nvidia.com>
|
2025-11-05 16:47:06 -08:00 |
|
Yan Ru Pei
|
427ca9aba4
|
chore: no need for explicit Python::with_gil (#4075)
Signed-off-by: PeaBrane <yanrpei@gmail.com>
|
2025-11-05 05:41:49 +00:00 |
|
Michael Feil
|
defe5de74b
|
feat: release gil for publisher (#3747)
Signed-off-by: michaelfeil <me@michaelfeil.eu>
|
2025-11-05 02:17:06 +00:00 |
|
Graham King
|
dadf0e2247
|
chore: Remove old DisaggregatedRouter, making etcd presence optional (#4011)
Signed-off-by: Graham King <grahamk@nvidia.com>
|
2025-11-03 10:10:13 -05:00 |
|
Yan Ru Pei
|
6ff49edbb7
|
feat: efficient serving of multiple mockers (#3997)
Signed-off-by: PeaBrane <yanrpei@gmail.com>
|
2025-11-01 01:24:55 +00:00 |
|
Richard Huo
|
6afa679c5d
|
chore: KVBM pip wheel (#3826)
Signed-off-by: Anant Sharma <anants@nvidia.com>
Co-authored-by: Anant Sharma <anants@nvidia.com>
|
2025-10-31 14:19:30 -07:00 |
|
Anant Sharma
|
8bd37c96d6
|
refactor: move backend deploy, launch and slurm files from components to examples (#3849)
Signed-off-by: Anant Sharma <anants@nvidia.com>
|
2025-10-31 13:09:46 -04:00 |
|
Tushar Sharma
|
1da9d70ab1
|
chore: bump nixl verison to 0.7.0 (#3936)
Signed-off-by: Tushar Sharma <tusharma@nvidia.com>
|
2025-10-30 10:28:17 -07:00 |
|
Kris Hung
|
1898601025
|
feat: Add KV event consolidator for KVBM (vllm) and router integration (#3725)
Signed-off-by: krishung5 <krish@nvidia.com>
|
2025-10-30 17:11:42 +00:00 |
|
milesial
|
da10299c19
|
fix: NIXL connect header strip (#3959)
Signed-off-by: Alexandre Milesi <milesial@users.noreply.github.com>
|
2025-10-29 16:11:35 -07:00 |
|
jthomson04
|
3998fdcb28
|
feat: KVBM V2 Initial Migration (#3861)
Signed-off-by: jthomson04 <jwillthomson19@gmail.com>
|
2025-10-28 16:08:14 -07:00 |
|
Yan Ru Pei
|
cc4c3516bb
|
feat: mocker disagg (#3833)
Signed-off-by: PeaBrane <yanrpei@gmail.com>
|
2025-10-28 20:36:02 +00:00 |
|
milesial
|
76f2ef9112
|
feat: b64 encoding for nixl_connect metadata (#3843)
|
2025-10-28 11:22:50 -07:00 |
|
Graham King
|
c78b590134
|
chore(runtime): Do not expose etcd lease ID (#3915)
Signed-off-by: Graham King <grahamk@nvidia.com>
|
2025-10-28 16:20:25 +00:00 |
|
Kyle McGill
|
0028cdf43d
|
feat: Use ForwardPassCallback api from TRTLLM to register end of forward pass callback to enable cuda graphs (#3297)
Signed-off-by: Kyle McGill <kmcgill@nvidia.com>
|
2025-10-27 15:12:04 -07:00 |
|
Tushar Sharma
|
9bb1af3376
|
chore: Update version to 0.6.1 (#3916)
Signed-off-by: Tushar Sharma <tusharma@nvidia.com>
|
2025-10-27 17:17:26 -04:00 |
|
Keiven C
|
cbe0b177ab
|
refactor: redesign the metrics API from Trait to composition to make the code cleaner and easier to understand (#3687)
Signed-off-by: Keiven Chang <keivenchang@users.noreply.github.com>
|
2025-10-24 14:34:29 -07:00 |
|
zhongdaor-nv
|
9e5407f20d
|
feat: python binding for kserve grpc frontend (#3739)
Signed-off-by: zhongdaor <zhongdaor@nvidia.com>
Signed-off-by: zhongdaor-nv <zhongdaor@nvidia.com>
Co-authored-by: coderabbitai[bot] <136622811+coderabbitai[bot]@users.noreply.github.com>
Co-authored-by: Ryan McCormick <rmccormick@nvidia.com>
|
2025-10-24 11:20:20 -07:00 |
|
Graham King
|
7731b0245c
|
chore: Use KeyValueStoreManager instead of etcd::Client (#3822)
Signed-off-by: Graham King <grahamk@nvidia.com>
|
2025-10-23 14:09:58 -04:00 |
|
Yan Ru Pei
|
41ff394f51
|
chore: restructure mocker cli args handling, to include prefill/decode (#3847)
Signed-off-by: PeaBrane <yanrpei@gmail.com>
|
2025-10-23 10:41:23 -07:00 |
|
Richard Huo
|
94aa2a7bbb
|
refactor: kvbm modularity DIS-657 Eliminate ETCD from the leader-worker initialization (#3202)
Signed-off-by: richardhuo-nv <rihuo@nvidia.com>
|
2025-10-23 00:10:09 -07:00 |
|
Keiven C
|
f93b619ad9
|
feat: add TensorRT-LLM Prometheus metrics support with prefixing + filtering (#3676)
Signed-off-by: Keiven Chang <keivenchang@users.noreply.github.com>
|
2025-10-22 11:25:13 -07:00 |
|
Michael Feil
|
681951d4e6
|
feat: python gil release for radix tree + dump_tree_as_events in python (#3748)
Signed-off-by: michaelfeil <me@michaelfeil.eu>
Co-authored-by: Yan Ru Pei <yanrpei@gmail.com>
|
2025-10-22 01:41:35 +00:00 |
|
Yan Ru Pei
|
e01c6e99bd
|
feat: bake prefill router into frontend, supporting vllm for now (#3762)
Signed-off-by: PeaBrane <yanrpei@gmail.com>
|
2025-10-21 15:43:02 -07:00 |
|
mohammedabdulwahhab
|
8a04d7b252
|
fix: revert placement of logging init if otel exports is disabled (#3797)
Signed-off-by: mohammedabdulwahhab <furkhan324@berkeley.edu>
|
2025-10-21 17:36:31 -04:00 |
|
Ziqi Fan
|
d5840cda65
|
feat: enable KVBM GPU offload to Disk bypassing CPU (#3510)
Signed-off-by: Ziqi Fan <ziqif@nvidia.com>
|
2025-10-21 08:49:54 -07:00 |
|
Graham King
|
9ae98ed7f9
|
refactor(runtime): Replace std::sync::Mutex with parking_lot::Mutex (#3740)
Signed-off-by: Graham King <grahamk@nvidia.com>
|
2025-10-21 12:38:51 +00:00 |
|
Graham King
|
f6ed01b14c
|
chore: Replace ServiceConfigBuilder with add_stats_service (#3736)
Signed-off-by: Graham King <grahamk@nvidia.com>
|
2025-10-20 16:47:05 -04:00 |
|
Michael Feil
|
af7b73c045
|
feat: python package build with abi (cross py3.10+) (#3571)
Signed-off-by: michaelfeil <me@michaelfeil.eu>
|
2025-10-20 12:04:38 -07:00 |
|
Yan Ru Pei
|
4b7a806c7a
|
feat: add prefill workers to discovery (#3709)
Signed-off-by: PeaBrane <yanrpei@gmail.com>
|
2025-10-18 00:59:32 +00:00 |
|
Yan Ru Pei
|
4c207e0c8c
|
chore: remove kv metrics scraping and aggregation (#3701)
Signed-off-by: PeaBrane <yanrpei@gmail.com>
|
2025-10-17 10:58:48 -07:00 |
|
Anish
|
598cbbb73b
|
docs: reorganizing documentation to make things clearer (#3658)
Signed-off-by: athreesh <anish.maddipoti@utexas.edu>
Co-authored-by: Claude <noreply@anthropic.com>
|
2025-10-16 23:59:59 +00:00 |