7a6283e927
chore: cargo machete - remove unused dependencies ( #6453 )
...
Signed-off-by: Ryan Olson <rolson@nvidia.com>
2026-02-20 19:11:52 +00:00
cf51a0c4fd
chore: gate plotters dependency behind bench feature ( #6380 )
...
Co-authored-by: Cursor <cursoragent@cursor.com>
2026-02-18 15:27:19 -05:00
c5c6a55190
feat: more flash indexer optimizations ( #6305 )
...
Signed-off-by: PeaBrane <yanrpei@gmail.com>
Co-authored-by: Cursor <cursoragent@cursor.com>
2026-02-18 09:13:37 -08:00
94fad4799f
feat: kvbm-logical ( #6033 )
...
Signed-off-by: Ryan Olson <rolson@nvidia.com>
2026-02-14 02:24:26 +00:00
3842b24479
fix: update bytes crate version to latest ( #6041 )
...
Signed-off-by: Anant Sharma <anants@nvidia.com>
2026-02-06 15:11:04 -05:00
2ffd58475b
chore: Remove native-tls / openssl dependency ( #5703 )
...
Signed-off-by: Graham King <grahamk@nvidia.com>
2026-01-28 17:36:11 +00:00
66c369964a
chore: version update for 0.9.0 ( #5661 )
...
Signed-off-by: pvijayakrish <pvijayakrish@nvidia.com>
2026-01-26 17:18:05 -08:00
7436e3bc03
chore: update versions for 0.8.0 release ( #5031 )
...
Signed-off-by: Anant Sharma <anants@nvidia.com>
2025-12-19 10:21:52 -08:00
5b82b8b096
chore: update versions to match latest release ( #4966 )
...
Signed-off-by: Anant Sharma <anants@nvidia.com>
2025-12-18 08:45:44 -08:00
27904535fb
fix(storage): Correctly encoding FileStore keys ( #4539 )
...
Signed-off-by: Graham King <grahamk@nvidia.com>
2025-11-21 17:01:40 -05:00
2100f6aac5
feat(mistralrs): Upgrade to support CUDA 13 ( #4474 )
...
Signed-off-by: Graham King <grahamk@nvidia.com>
2025-11-19 14:12:17 -08:00
fbad28602e
chore: Remove the python bindings for port allocation ( #4237 )
...
Signed-off-by: Graham King <grahamk@nvidia.com>
2025-11-11 20:35:36 +00:00
23660bc5c3
chore: update versions for 0.7.0 release ( #4217 )
...
Signed-off-by: Anant Sharma <anants@nvidia.com>
2025-11-10 21:40:06 +00:00
cf630bf7b6
refactor: Make the Runtime and DistributedRuntime fields private ( #4193 )
...
Signed-off-by: Graham King <grahamk@nvidia.com>
2025-11-10 15:56:22 +00:00
794c0a4431
feat(keyvalue): Filesystem backed KeyValueStore ( #4138 )
...
Signed-off-by: Graham King <grahamk@nvidia.com>
2025-11-07 21:11:33 +00:00
9bb1af3376
chore: Update version to 0.6.1 ( #3916 )
...
Signed-off-by: Tushar Sharma <tusharma@nvidia.com>
2025-10-27 17:17:26 -04:00
9ae98ed7f9
refactor(runtime): Replace std::sync::Mutex with parking_lot::Mutex ( #3740 )
...
Signed-off-by: Graham King <grahamk@nvidia.com>
2025-10-21 12:38:51 +00:00
cd2389baef
chore: pre-0.6.0 activities ( #3592 )
...
Signed-off-by: Harrison Saturley-Hall <hsaturleyhal@nvidia.com>
2025-10-13 17:45:19 -04:00
6ffd20a81c
chore: Move model_input, model_type from ModelEntry to ModelDeploymentCard ( #3292 )
...
Signed-off-by: Graham King <grahamk@nvidia.com>
2025-09-30 09:35:15 -07:00
980727bba1
chore: bump versions ahead of 0.5.1 release ( #3209 )
...
Signed-off-by: Harrison Saturley-Hall <hsaturleyhal@nvidia.com>
2025-09-24 18:26:38 -04:00
cb5a657a6a
fix: Load the tokenizer JSON once for chat and completions. ( #2910 )
...
Signed-off-by: Graham King <grahamk@nvidia.com>
2025-09-05 16:48:16 -04:00
561ecb98a2
chore: bump version numbers ahead of 0.5.0 release ( #2812 )
...
Signed-off-by: Harrison King Saturley-Hall <hsaturleyhal@nvidia.com>
2025-09-02 16:13:28 -04:00
bce7458842
chore: Rust to 1.89 and edition 2024 ( #2659 )
2025-08-22 16:17:15 -07:00
9a02188531
chore: Bumped Dynamo version to 0.4.1 ( #2545 )
2025-08-19 21:55:37 -04:00
0444217339
fix: replace metrics callback with background scraping to prevent tim… ( #2480 )
...
Co-authored-by: Keiven Chang <keivenchang@users.noreply.github.com>
2025-08-18 12:38:50 -07:00
72ec5f5c7b
feat: Allow an endpoint to serve multiple models ( #2418 )
2025-08-13 10:48:42 -04:00
a3f7a39faf
chore(metrics): Remove the Arc ( #2357 )
2025-08-07 14:40:53 -04:00
4c90b1b924
chore: Version bump to 0.4.0 ( #2179 )
2025-07-30 02:07:31 +00:00
615580d8e6
feat: Base metrics: add generic ingress handler metrics ( #2090 )
...
Co-authored-by: Keiven Chang <keivenchang@users.noreply.github.com>
2025-07-28 10:35:56 -07:00
e5a8628fb7
feat: add a hierarchical Prometheus MetricsRegistry trait for DistributedRuntime, Namespace, Components, and Endpoint ( #2008 )
...
Co-authored-by: Keiven Chang <keivenchang@users.noreply.github.com>
Co-authored-by: Ryan Olson <rolson@nvidia.com>
2025-07-22 08:39:32 -07:00
aba60996f2
perf(router): Remove lock from router hot path ( #1963 )
2025-07-16 12:26:32 -04:00
ece76a6201
feat: Build DistributedRuntime-level HTTP server with /health /metrics ( #1656 )
2025-07-08 22:49:28 +00:00
c4935b3497
chore: update versions for 0.3.2 release ( #1793 )
2025-07-07 17:21:02 -04:00
4ab476174a
chore(engines): Upgrade mistralrs to 0.6.0 ( #1767 )
2025-07-03 12:45:58 -07:00
99e67e607a
chore: update dynamo and nixl versions for 0.3.1 ( #1517 )
2025-06-13 21:13:43 +00:00
9d9a1d9b74
chore: update dynamo and nixl versions for 0.3.0 ( #1240 )
2025-05-29 17:30:17 +00:00
e9cb035ac7
chore: bump versions and NIXL dependencies for 0.2.1 ( #1012 )
2025-05-09 15:35:01 +00:00
b2e401bcae
feat: allow adding auth to etcd ( #980 )
...
Allow both password or TLS auth, if none of these is provided fallback to no auth
Closes #657
2025-05-09 08:36:41 -04:00
0715d4691f
chore: bump NIXL version and package versions ( #836 )
...
Signed-off-by: Harrison Saturley-Hall <454891+saturley-hall@users.noreply.github.com>
2025-04-25 19:24:38 -04:00
d346782c7c
chore: Publish Model Deployment Card to NATS ( #799 )
...
This will allow an ingress-side pre-processor to see it without needing a model checkout.
Currently pre-processing is done in the worker, which has access to the model deployment card ("MDC") files (`config.json`, `tokenizer.json` and `tokenizer_config.json`) locally. We want to move the pre-processor to the ingress side to support KV routing. That requires ingress side (i.e the HTTP server), on a different machine than the worker to be able to see those three files.
To support that this PR makes the worker upload the contents of those files to the NATS object store, and publishes the MDC with those NATS urls to the key-value store.
The key-value store has an interface so any store (nats, etcd, redis, etc) can be supported. Implementations for memory and NATS are provided.
Fetching the MDC from the store, doing pre-processing ingress side, and publishing a card backed by a GGUF, are all for a later commit.
Part of #743
2025-04-25 08:52:59 -04:00
fa7ee14c2a
chore: update versions to 0.1.1 ( #552 )
2025-04-09 09:50:30 -04:00
de2905377c
refactor: prometheus upgrade ( #452 )
2025-03-31 13:03:36 -06:00
7c3fd5c94c
chore: Don't depend on openssl ( #292 )
...
This makes the Rust parts all use ring / rustls library instead of local install of openssl. It's a step on the journey to being statically linked.
Pieces:
- `tokenizers` and `mistralrs` now support rustls (mistralrs by default, tokenizers with feature flag).
- Move shared dependencies up into workspace
- New `rand` crate has some renames for future rust
- Ensure the dependency doesn't creep back in by enforcing it with cargo deny.
2025-03-19 14:16:19 -04:00
f04359cf69
feat: global kv block manager ( #45 )
2025-03-13 20:57:28 -06:00
3d2928510c
build: add top level rust workspace ( #137 )
2025-03-13 17:22:19 -04:00
e5db9e8634
refactor: Move rust binaries out of examples, update nixl dockerfile ( #89 )
...
Co-authored-by: Meenakshi Sharma <163925564+nvda-mesharma@users.noreply.github.com>
2025-03-11 17:52:57 -04:00
Ryan Olson