9f2356cb9e
chore: Remove unused portion of kv bindings test ( #2052 )
2025-07-22 19:12:42 +00:00
439e977d9c
feat: vllm speculative decoding metrics ( #1549 )
...
Signed-off-by: jain-ria <riajain@NVIDIA.com>
Co-authored-by: Alec <35311602+alec-flowers@users.noreply.github.com>
2025-07-07 23:40:43 +00:00
aaf283bbb8
feat: Approximate KV Routing ( #1636 )
2025-06-30 20:34:08 -07:00
13a99b7f76
feat: Standalone Router ( #1409 )
...
Signed-off-by: PeaBrane <yanrpei@gmail.com>
Signed-off-by: Yan Ru Pei <yanrpei@gmail.com>
Signed-off-by: jain-ria <riajain@NVIDIA.com>
Co-authored-by: coderabbitai[bot] <136622811+coderabbitai[bot]@users.noreply.github.com>
Co-authored-by: jain-ria <riajain@NVIDIA.com>
2025-06-14 19:02:31 +00:00
2f8da9ad1d
refactor: rename KvMetricsPublisher to WorkerMetricsPublisher ( #1284 )
2025-05-30 17:23:32 +00:00
7677f74f37
feat: KVBM async Python bindings and Layer class ( #1141 )
2025-05-29 17:49:05 +00:00
80256acf15
feat: adding outer dimension to isolate k/v blocks ( #1126 )
2025-05-20 08:51:41 -04:00
437cae0ad0
feat: KV Block Manager Python bindings ( #1022 )
2025-05-19 12:33:41 -04:00
88ad3425c4
feat: Python decorator dynamo_worker takes optional `static` parameter without etcd ( #494 )
...
Adds `@dynamo_worker(static = True)` to create a static worker which has a predictable name and hence does not require discovery or `etcd` to be running. There can only be a single static worker per namespace / component / endpoint trio.
This contrasts with the default dynamic `dynamo_worker` endpoints we have now, which get a unique random name (based on namespace/component/endpoint), and are discovered by ingress components using etcd.
Also change the hello_world example to use `dynamo_worker(static = True)` so that it is exercised and demonstrated somewhere.
For NIM.
2025-04-04 09:26:52 -04:00
6e09681e0e
feat: expose Python binding for KVEventPublisher. Use event pub/sub trait for KV events ( #169 )
2025-03-17 00:17:33 -07:00
3f84cdadfa
feat: add new metrics and simple router cost fn ( #88 )
2025-03-11 11:53:21 -07:00
989bb3d59c
feat: make block_size input for indexer, router, publisher ( #66 )
2025-03-09 15:58:27 -07:00
6ba39b0932
chore: address comments for #35 ( #53 )
2025-03-09 07:44:45 -07:00
602352ce19
chore: rename dynamo ( #44 )
...
Co-authored-by: Biswa Panda <biswa.panda@gmail.com>
2025-03-08 01:42:40 -08:00
dcecc47de9
test: add tests for kv bindings ( #35 )
2025-03-07 16:01:28 -08:00
1af7433bff
refactor: rename triton_distributed to dynemo ( #22 )
...
Co-authored-by: Graham King <grahamk@nvidia.com>
2025-03-05 09:15:46 -08:00
3a5fe17db9
feat: nixl metadata store and retrieved from etcd ( #6 )
...
Co-authored-by: hongkuanz <hongkuanz@nvidia.com>
Co-authored-by: Piotr Tarasiewicz <ptarasiewicz@nvidia.com>
Co-authored-by: Piotr Tarasiewicz Nvidia <ptarasiewicznv@Piotrs-MacBook-Pro.local>
Co-authored-by: Neelay Shah <neelays@ipp2-0493.ipp2u1.colossus.nvidia.com>
Co-authored-by: Neelay Shah <neelays@ipp1-1941.ipp1a1.colossus.nvidia.com>
Co-authored-by: ishandhanani <ishandhanani@gmail.com>
Co-authored-by: Neelay Shah <neelays@4u8g-gen-0078.ipp3a2.colossus.nvidia.com>
Co-authored-by: ptarasiewiczNV <104908264+ptarasiewiczNV@users.noreply.github.com>
2025-03-04 13:10:16 -08:00
08fcd7e93b
refactor: move libs to lib dir
...
Signed-off-by: Neelay Shah <neelays@nvidia.com>
Co-authored-by: Ryan McCormick <rmccormick@nvidia.com>
2025-02-24 18:21:02 -08:00
Ryan McCormick