Yan Ru Pei
|
41ff394f51
|
chore: restructure mocker cli args handling, to include prefill/decode (#3847)
Signed-off-by: PeaBrane <yanrpei@gmail.com>
|
2025-10-23 10:41:23 -07:00 |
|
Richard Huo
|
94aa2a7bbb
|
refactor: kvbm modularity DIS-657 Eliminate ETCD from the leader-worker initialization (#3202)
Signed-off-by: richardhuo-nv <rihuo@nvidia.com>
|
2025-10-23 00:10:09 -07:00 |
|
Keiven C
|
f93b619ad9
|
feat: add TensorRT-LLM Prometheus metrics support with prefixing + filtering (#3676)
Signed-off-by: Keiven Chang <keivenchang@users.noreply.github.com>
|
2025-10-22 11:25:13 -07:00 |
|
Michael Feil
|
681951d4e6
|
feat: python gil release for radix tree + dump_tree_as_events in python (#3748)
Signed-off-by: michaelfeil <me@michaelfeil.eu>
Co-authored-by: Yan Ru Pei <yanrpei@gmail.com>
|
2025-10-22 01:41:35 +00:00 |
|
Yan Ru Pei
|
e01c6e99bd
|
feat: bake prefill router into frontend, supporting vllm for now (#3762)
Signed-off-by: PeaBrane <yanrpei@gmail.com>
|
2025-10-21 15:43:02 -07:00 |
|
mohammedabdulwahhab
|
8a04d7b252
|
fix: revert placement of logging init if otel exports is disabled (#3797)
Signed-off-by: mohammedabdulwahhab <furkhan324@berkeley.edu>
|
2025-10-21 17:36:31 -04:00 |
|
Ziqi Fan
|
d5840cda65
|
feat: enable KVBM GPU offload to Disk bypassing CPU (#3510)
Signed-off-by: Ziqi Fan <ziqif@nvidia.com>
|
2025-10-21 08:49:54 -07:00 |
|
Graham King
|
9ae98ed7f9
|
refactor(runtime): Replace std::sync::Mutex with parking_lot::Mutex (#3740)
Signed-off-by: Graham King <grahamk@nvidia.com>
|
2025-10-21 12:38:51 +00:00 |
|
Graham King
|
f6ed01b14c
|
chore: Replace ServiceConfigBuilder with add_stats_service (#3736)
Signed-off-by: Graham King <grahamk@nvidia.com>
|
2025-10-20 16:47:05 -04:00 |
|
Michael Feil
|
af7b73c045
|
feat: python package build with abi (cross py3.10+) (#3571)
Signed-off-by: michaelfeil <me@michaelfeil.eu>
|
2025-10-20 12:04:38 -07:00 |
|
Yan Ru Pei
|
4b7a806c7a
|
feat: add prefill workers to discovery (#3709)
Signed-off-by: PeaBrane <yanrpei@gmail.com>
|
2025-10-18 00:59:32 +00:00 |
|
Yan Ru Pei
|
4c207e0c8c
|
chore: remove kv metrics scraping and aggregation (#3701)
Signed-off-by: PeaBrane <yanrpei@gmail.com>
|
2025-10-17 10:58:48 -07:00 |
|
Anish
|
598cbbb73b
|
docs: reorganizing documentation to make things clearer (#3658)
Signed-off-by: athreesh <anish.maddipoti@utexas.edu>
Co-authored-by: Claude <noreply@anthropic.com>
|
2025-10-16 23:59:59 +00:00 |
|
Yan Ru Pei
|
ae4e96a2bc
|
fix: mocker engines should ignore downloading weights from hf (again) (#3664)
Signed-off-by: PeaBrane <yanrpei@gmail.com>
|
2025-10-16 02:48:40 +00:00 |
|
Jacky
|
734d2f87a8
|
feat: Enable cancellation during or before a stream is established (#3635)
Signed-off-by: Jacky <18255193+kthui@users.noreply.github.com>
Co-authored-by: Ryan McCormick <rmccormick@nvidia.com>
|
2025-10-15 19:22:55 -07:00 |
|
Yan Ru Pei
|
f978f4d1d6
|
feat: dp rank routing (#3597)
Signed-off-by: PeaBrane <yanrpei@gmail.com>
|
2025-10-16 01:48:27 +00:00 |
|
Ziqi Fan
|
f5854e1784
|
fix: avoid offload redundant prefill blocks | fix cuda graph hanging (#3632)
Signed-off-by: Ziqi Fan <ziqif@nvidia.com>
|
2025-10-15 10:38:35 -07:00 |
|
Graham King
|
ab0da5821e
|
feat: Python binding to download a model. (#3593)
Signed-off-by: Graham King <grahamk@nvidia.com>
|
2025-10-15 12:26:02 -04:00 |
|
Harrison Saturley-Hall
|
cd2389baef
|
chore: pre-0.6.0 activities (#3592)
Signed-off-by: Harrison Saturley-Hall <hsaturleyhal@nvidia.com>
|
2025-10-13 17:45:19 -04:00 |
|
Yan Ru Pei
|
b5e762b2b8
|
feat: remove stale workers on snapshot + some refactoring (#3589)
Signed-off-by: PeaBrane <yanrpei@gmail.com>
|
2025-10-13 21:36:09 +00:00 |
|
mohammedabdulwahhab
|
1f92dd547e
|
feat: OTEL Exporter and Tempo Visualization (#3307)
Signed-off-by: mohammedabdulwahhab <furkhan324@berkeley.edu>
|
2025-10-13 18:39:09 +00:00 |
|
Alec
|
90dc758938
|
chore: bump vllm to 0.11.0 (#3422)
Signed-off-by: alec-flowers <aflowers@nvidia.com>
|
2025-10-12 14:26:45 -07:00 |
|
Yan Ru Pei
|
c3fcfdd6e3
|
chore: consolidations of KvPushRouter bindings and usage examples (#3543)
Signed-off-by: PeaBrane <yanrpei@gmail.com>
Signed-off-by: Yan Ru Pei <yanrpei@gmail.com>
Co-authored-by: Ryan McCormick <rmccormick@nvidia.com>
|
2025-10-11 05:47:25 +00:00 |
|
Keiven C
|
65cc533702
|
feat: implement custom backend metrics for NIM (#3266)
Signed-off-by: Keiven Chang <keivenchang@users.noreply.github.com>
|
2025-10-10 20:28:58 -07:00 |
|
Ziqi Fan
|
4224e57da0
|
feat: rm the old KVBM metrics | update G2 to G3 metrics collection (#3561)
Signed-off-by: Ziqi Fan <ziqif@nvidia.com>
|
2025-10-11 01:48:09 +00:00 |
|
Keiven C
|
55e458d899
|
feat: add SGLang and vLLM passthrough metrics on Dynamo backend worker (#3539)
Signed-off-by: Keiven Chang <keivenchang@users.noreply.github.com>
Co-authored-by: Keiven Chang <keivenchang@users.noreply.github.com>
|
2025-10-10 17:59:35 -07:00 |
|
Keiven C
|
a4746ab65f
|
fix: callback registration, fix metric name access, ensure ordered vec, etc... (#3541)
Signed-off-by: Keiven Chang <keivenchang@users.noreply.github.com>
Co-authored-by: Keiven Chang <keivenchang@users.noreply.github.com>
|
2025-10-11 00:03:24 +00:00 |
|
Graham King
|
0e0218ff02
|
chore: Remove model_config from LocalModel (#3558)
Signed-off-by: Graham King <grahamk@nvidia.com>
|
2025-10-10 22:01:30 +00:00 |
|
Richard Huo
|
5c69c11930
|
chore: upgrade nixl to 0.6.0 (#3550)
Signed-off-by: richardhuo-nv <rihuo@nvidia.com>
|
2025-10-10 13:18:21 -07:00 |
|
Keiven C
|
f2ba58e568
|
feat: add Rust-to-Python const code generator for prometheus_names.py (#3425)
Signed-off-by: Keiven Chang <keivenchang@users.noreply.github.com>
|
2025-10-10 11:20:36 -07:00 |
|
Ziqi Fan
|
ca67409888
|
feat: add KVBM host to disk metrics | clean up dashboard (#3534)
Signed-off-by: Ziqi Fan <ziqif@nvidia.com>
|
2025-10-10 18:17:34 +00:00 |
|
Ziqi Fan
|
b94ecd1627
|
feat: add disk offloading filtering in KVBM (#3532)
Signed-off-by: Ziqi Fan <ziqif@nvidia.com>
Co-authored-by: coderabbitai[bot] <136622811+coderabbitai[bot]@users.noreply.github.com>
|
2025-10-10 09:51:55 -07:00 |
|
Graham King
|
e9a710095b
|
chore: Remove clear_namespace script (#3519)
Signed-off-by: Graham King <grahamk@nvidia.com>
|
2025-10-09 13:43:59 -04:00 |
|
Ryan McCormick
|
ef04a0869e
|
feat: Add OpenAPI routes to serve HTTP server API documentation (#3480)
|
2025-10-08 13:52:19 -07:00 |
|
Ziqi Fan
|
9975cb9ffe
|
feat: enable KVBM to support PD disagg in Dynamo vLLM (#3352)
Signed-off-by: Ziqi Fan <ziqif@nvidia.com>
|
2025-10-08 13:45:36 -07:00 |
|
Graham King
|
0aa0768f53
|
chore: Remove llama.cpp engine (#3499)
Signed-off-by: Graham King <grahamk@nvidia.com>
|
2025-10-08 16:44:53 -04:00 |
|
Richard Huo
|
cf83794ae0
|
feat: DIS-678 kvbm modularity: standalone metrics endpoint (#3433)
Signed-off-by: richardhuo-nv <rihuo@nvidia.com>
|
2025-10-08 10:18:33 -07:00 |
|
Graham King
|
1b1265e6fc
|
chore: Remove GGUF support (#3488)
Signed-off-by: Graham King <grahamk@nvidia.com>
|
2025-10-08 11:42:24 -04:00 |
|
Keiven C
|
0c4c4d1de3
|
feat: add Python MetricsRegistry Python metrics registration (#3341)
Signed-off-by: Keiven Chang <keivenchang@users.noreply.github.com>
|
2025-10-07 21:24:25 -07:00 |
|
Graham King
|
bdad6f1a50
|
fix: Make planner VirtualConnectorClient also use v1/ prefix. (#3468)
Signed-off-by: Graham King <grahamk@nvidia.com>
|
2025-10-07 22:34:30 +00:00 |
|
Tanmay Verma
|
68bbe8a9a5
|
feat: Connect DYN_LOG level to TLLM_LOG_LEVEL (#3451)
Signed-off-by: Tanmay Verma <tanmay2592@gmail.com>
Co-authored-by: coderabbitai[bot] <136622811+coderabbitai[bot]@users.noreply.github.com>
|
2025-10-07 21:02:35 +00:00 |
|
Graham King
|
a5371bfc50
|
feat(etcd): Version the etcd keys (#3458)
Signed-off-by: Graham King <grahamk@nvidia.com>
|
2025-10-07 16:48:26 +00:00 |
|
Graham King
|
81162dfeb9
|
chore(discovery): Watch/publish ModelDeploymentCard instead of ModelEntry (#3350)
Signed-off-by: Graham King <grahamk@nvidia.com>
|
2025-10-07 10:02:02 -04:00 |
|
Yan Ru Pei
|
30610e7371
|
feat: use KvPushRouter for prefill router (#3401)
Signed-off-by: PeaBrane <yanrpei@gmail.com>
|
2025-10-03 18:21:48 -07:00 |
|
Olga Andreeva
|
d2e3b66e65
|
feat: Transition to FullyContiguous Host and Disk layouts (#3090)
Signed-off-by: Olga Andreeva <oandreeva@nvidia.com>
Signed-off-by: Olga Andreeva <124622579+oandreeva-nv@users.noreply.github.com>
Co-authored-by: oandreeva-nv <oandreeva-nv@nvidia.com>
|
2025-10-01 16:26:39 -07:00 |
|
Richard Huo
|
713e9e481b
|
fix: DIS-706 skip offloading the G1 matched blocks during offloading (#3299)
Signed-off-by: richardhuo-nv <rihuo@nvidia.com>
|
2025-10-01 10:28:06 -07:00 |
|
Yan Ru Pei
|
9b9536d0d7
|
feat: make prefill router general (#3329)
Signed-off-by: PeaBrane <yanrpei@gmail.com>
|
2025-09-30 20:38:47 -07:00 |
|
Keiven C
|
f4a3a6b66a
|
refactor: standardize Prometheus metric naming conventions (part 1) (#3035)
Signed-off-by: Keiven Chang <keivenchang@users.noreply.github.com>
|
2025-09-30 19:01:39 -07:00 |
|
Keiven C
|
af4c0cadcd
|
feat: add Context class to Python type stubs (#3327)
Signed-off-by: Keiven Chang <keivenchang@users.noreply.github.com>
|
2025-09-30 17:10:02 -07:00 |
|
Michael Feil
|
5b457b70a6
|
feat: python add abi compatability for cross-platform builds + add a unit test to HttpServer (#3044)
Signed-off-by: michaelfeil <me@michaelfeil.eu>
Signed-off-by: Michael Feil <63565275+michaelfeil@users.noreply.github.com>
Signed-off-by: root <root@michaelfeil2-dev-pod-b200-0.michaelfeil2-dev-pod-b200.baseten.svc.cluster.local>
Signed-off-by: root <root@michaelfeildns-dev-pod-h100-0.michaelfeildns-dev-pod-h100.baseten.svc.cluster.local>
Co-authored-by: root <root@michaelfeil2-dev-pod-b200-0.michaelfeil2-dev-pod-b200.baseten.svc.cluster.local>
Co-authored-by: root <root@michaelfeildns-dev-pod-h100-0.michaelfeildns-dev-pod-h100.baseten.svc.cluster.local>
|
2025-09-30 14:42:42 -07:00 |