6a728d1044
chore: remove kv indexers bindings ( #6159 )
...
Signed-off-by: PeaBrane <yanrpei@gmail.com>
2026-02-11 20:23:48 +00:00
902eabd9ec
feat: per dp rank gap detection ( #5873 )
...
Signed-off-by: PeaBrane <yanrpei@gmail.com>
2026-02-03 03:07:57 +00:00
cf433e6825
chore: update all copyright headers in repo to 2026 ( #5130 )
...
Signed-off-by: Tushar Sharma <tusharma@nvidia.com>
2026-01-02 22:08:23 +00:00
1e37c1033f
fix: add timeout to tests before lora to unblock ( #4808 )
...
Signed-off-by: Keiven Chang <keivenchang@users.noreply.github.com>
Co-authored-by: Keiven Chang <keivenchang@users.noreply.github.com>
2025-12-08 20:50:33 -08:00
c61e0dd3cc
chore: merge KvIndexer and ApproxKvIndexer ( #4500 )
...
Signed-off-by: PeaBrane <yanrpei@gmail.com>
2025-11-21 09:09:11 -08:00
d821a8b9f7
chore: parallelize planner profile tests + bindings test cleanup ( #4532 )
...
Signed-off-by: PeaBrane <yanrpei@gmail.com>
2025-11-21 09:21:43 +00:00
69797b5ab3
feat: Only monitor NATS metrics if using NATS request plane ( #4442 )
...
Signed-off-by: Graham King <grahamk@nvidia.com>
2025-11-19 17:48:49 +00:00
886506c12d
feat: Command line flag to set request plane mode: tcp, http or nats ( #4365 )
...
Signed-off-by: Graham King <grahamk@nvidia.com>
2025-11-17 09:02:36 -08:00
e1af3af6ee
chore: Remove static mode ( #4235 )
...
Signed-off-by: Graham King <grahamk@nvidia.com>
2025-11-11 19:25:12 +00:00
794c0a4431
feat(keyvalue): Filesystem backed KeyValueStore ( #4138 )
...
Signed-off-by: Graham King <grahamk@nvidia.com>
2025-11-07 21:11:33 +00:00
681951d4e6
feat: python gil release for radix tree + dump_tree_as_events in python ( #3748 )
...
Signed-off-by: michaelfeil <me@michaelfeil.eu>
Co-authored-by: Yan Ru Pei <yanrpei@gmail.com>
2025-10-22 01:41:35 +00:00
4c207e0c8c
chore: remove kv metrics scraping and aggregation ( #3701 )
...
Signed-off-by: PeaBrane <yanrpei@gmail.com>
2025-10-17 10:58:48 -07:00
f978f4d1d6
feat: dp rank routing ( #3597 )
...
Signed-off-by: PeaBrane <yanrpei@gmail.com>
2025-10-16 01:48:27 +00:00
0c4c4d1de3
feat: add Python MetricsRegistry Python metrics registration ( #3341 )
...
Signed-off-by: Keiven Chang <keivenchang@users.noreply.github.com>
2025-10-07 21:24:25 -07:00
5b457b70a6
feat: python add abi compatability for cross-platform builds + add a unit test to HttpServer ( #3044 )
...
Signed-off-by: michaelfeil <me@michaelfeil.eu>
Signed-off-by: Michael Feil <63565275+michaelfeil@users.noreply.github.com>
Signed-off-by: root <root@michaelfeil2-dev-pod-b200-0.michaelfeil2-dev-pod-b200.baseten.svc.cluster.local>
Signed-off-by: root <root@michaelfeildns-dev-pod-h100-0.michaelfeildns-dev-pod-h100.baseten.svc.cluster.local>
Co-authored-by: root <root@michaelfeil2-dev-pod-b200-0.michaelfeil2-dev-pod-b200.baseten.svc.cluster.local>
Co-authored-by: root <root@michaelfeildns-dev-pod-h100-0.michaelfeildns-dev-pod-h100.baseten.svc.cluster.local>
2025-09-30 14:42:42 -07:00
c8ecc4025f
ci: fix completions race ( #3010 )
...
Signed-off-by: alec-flowers <aflowers@nvidia.com>
2025-09-11 19:46:59 +00:00
6c539fbdac
feat: FT Request Cancellation feature and test for 0.5.0 ( #2500 )
2025-09-02 08:26:21 -07:00
488c87095c
feat: Router warm restarts via durable KV event consumers and radix snapshotting ( #2756 )
...
Signed-off-by: PeaBrane <yanrpei@gmail.com>
2025-08-30 23:42:57 +00:00
9f2356cb9e
chore: Remove unused portion of kv bindings test ( #2052 )
2025-07-22 19:12:42 +00:00
439e977d9c
feat: vllm speculative decoding metrics ( #1549 )
...
Signed-off-by: jain-ria <riajain@NVIDIA.com>
Co-authored-by: Alec <35311602+alec-flowers@users.noreply.github.com>
2025-07-07 23:40:43 +00:00
aaf283bbb8
feat: Approximate KV Routing ( #1636 )
2025-06-30 20:34:08 -07:00
13a99b7f76
feat: Standalone Router ( #1409 )
...
Signed-off-by: PeaBrane <yanrpei@gmail.com>
Signed-off-by: Yan Ru Pei <yanrpei@gmail.com>
Signed-off-by: jain-ria <riajain@NVIDIA.com>
Co-authored-by: coderabbitai[bot] <136622811+coderabbitai[bot]@users.noreply.github.com>
Co-authored-by: jain-ria <riajain@NVIDIA.com>
2025-06-14 19:02:31 +00:00
2f8da9ad1d
refactor: rename KvMetricsPublisher to WorkerMetricsPublisher ( #1284 )
2025-05-30 17:23:32 +00:00
88ad3425c4
feat: Python decorator dynamo_worker takes optional `static` parameter without etcd ( #494 )
...
Adds `@dynamo_worker(static = True)` to create a static worker which has a predictable name and hence does not require discovery or `etcd` to be running. There can only be a single static worker per namespace / component / endpoint trio.
This contrasts with the default dynamic `dynamo_worker` endpoints we have now, which get a unique random name (based on namespace/component/endpoint), and are discovered by ingress components using etcd.
Also change the hello_world example to use `dynamo_worker(static = True)` so that it is exercised and demonstrated somewhere.
For NIM.
2025-04-04 09:26:52 -04:00
6e09681e0e
feat: expose Python binding for KVEventPublisher. Use event pub/sub trait for KV events ( #169 )
2025-03-17 00:17:33 -07:00
3f84cdadfa
feat: add new metrics and simple router cost fn ( #88 )
2025-03-11 11:53:21 -07:00
989bb3d59c
feat: make block_size input for indexer, router, publisher ( #66 )
2025-03-09 15:58:27 -07:00
6ba39b0932
chore: address comments for #35 ( #53 )
2025-03-09 07:44:45 -07:00
602352ce19
chore: rename dynamo ( #44 )
...
Co-authored-by: Biswa Panda <biswa.panda@gmail.com>
2025-03-08 01:42:40 -08:00
dcecc47de9
test: add tests for kv bindings ( #35 )
2025-03-07 16:01:28 -08:00
Yan Ru Pei