Jacky
|
1ffa489ea1
|
refactor: Move --migration-limit flag from backend to frontend (#5918)
Signed-off-by: Jacky <18255193+kthui@users.noreply.github.com>
|
2026-02-06 20:50:55 +00:00 |
|
Yan Ru Pei
|
9e33e3fa66
|
chore: clean ups in kv_router.rs (#6028)
Signed-off-by: PeaBrane <yanrpei@gmail.com>
|
2026-02-06 19:27:44 +00:00 |
|
Yan Ru Pei
|
91fb78cddf
|
chore: make mocker it's own crate (#5958)
Signed-off-by: PeaBrane <yanrpei@gmail.com>
|
2026-02-05 19:35:42 +00:00 |
|
Abhishek Gupta
|
491a21093f
|
fix: correct typos in help text and add missing docstrings (#5607)
Signed-off-by: Abhishek Kumar Gupta <mail2abhishekgupta@gmail.com>
|
2026-01-23 13:12:22 -08:00 |
|
Yan Ru Pei
|
050906b593
|
feat: track output tokens / blocks in the Router (optional) (#5452)
Signed-off-by: PeaBrane <yanrpei@gmail.com>
|
2026-01-18 09:30:25 +08:00 |
|
Yan Ru Pei
|
7fdc742e17
|
feat: allow router not assuming decode kv reuse (#5350)
Signed-off-by: PeaBrane <yanrpei@gmail.com>
|
2026-01-13 07:33:47 +08:00 |
|
Tushar Sharma
|
cf433e6825
|
chore: update all copyright headers in repo to 2026 (#5130)
Signed-off-by: Tushar Sharma <tusharma@nvidia.com>
|
2026-01-02 22:08:23 +00:00 |
|
Biswa Panda
|
18b64e90b6
|
feat: switch to tcp as default request plane (#4845)
|
2025-12-18 23:05:38 +00:00 |
|
Graham King
|
da0f2fb84a
|
feat(frontend): First part of Python request handling (#4999)
Signed-off-by: Graham King <grahamk@nvidia.com>
|
2025-12-17 20:03:39 -05:00 |
|
Graham King
|
fcb91e4b3d
|
refactor(storage): Remove the stuttering from key_value_store. (#4604)
Signed-off-by: Graham King <grahamk@nvidia.com>
|
2025-11-25 22:59:29 +00:00 |
|
Graham King
|
0fc5273c11
|
refactor(llm): Rename EngineConfig::Static to InProcess (#4585)
Signed-off-by: Graham King <grahamk@nvidia.com>
|
2025-11-25 12:22:09 -05:00 |
|
Graham King
|
e75bcf6704
|
fix(dynamo-run): Run without etcd/nats, HTTP port to 8000 (#4555)
Signed-off-by: Graham King <grahamk@nvidia.com>
|
2025-11-24 17:39:10 +00:00 |
|
Yan Ru Pei
|
c61e0dd3cc
|
chore: merge KvIndexer and ApproxKvIndexer (#4500)
Signed-off-by: PeaBrane <yanrpei@gmail.com>
|
2025-11-21 09:09:11 -08:00 |
|
Graham King
|
69797b5ab3
|
feat: Only monitor NATS metrics if using NATS request plane (#4442)
Signed-off-by: Graham King <grahamk@nvidia.com>
|
2025-11-19 17:48:49 +00:00 |
|
Keiven C
|
0e77d3442e
|
refactor: centralize environment variable constants (#4083)
Signed-off-by: Keiven Chang <keivenchang@users.noreply.github.com>
Co-authored-by: Keiven Chang <keivenchang@users.noreply.github.com>
|
2025-11-17 11:16:38 -08:00 |
|
Graham King
|
886506c12d
|
feat: Command line flag to set request plane mode: tcp, http or nats (#4365)
Signed-off-by: Graham King <grahamk@nvidia.com>
|
2025-11-17 09:02:36 -08:00 |
|
Graham King
|
e1af3af6ee
|
chore: Remove static mode (#4235)
Signed-off-by: Graham King <grahamk@nvidia.com>
|
2025-11-11 19:25:12 +00:00 |
|
Graham King
|
794c0a4431
|
feat(keyvalue): Filesystem backed KeyValueStore (#4138)
Signed-off-by: Graham King <grahamk@nvidia.com>
|
2025-11-07 21:11:33 +00:00 |
|
Anant Sharma
|
8bd37c96d6
|
refactor: move backend deploy, launch and slurm files from components to examples (#3849)
Signed-off-by: Anant Sharma <anants@nvidia.com>
|
2025-10-31 13:09:46 -04:00 |
|
Yan Ru Pei
|
a430bbb64c
|
fix: dynamo-run model name should default to remote path i.e. HFID (#3951)
Signed-off-by: PeaBrane <yanrpei@gmail.com>
|
2025-10-29 13:03:15 +00:00 |
|
Yan Ru Pei
|
41ff394f51
|
chore: restructure mocker cli args handling, to include prefill/decode (#3847)
Signed-off-by: PeaBrane <yanrpei@gmail.com>
|
2025-10-23 10:41:23 -07:00 |
|
Yan Ru Pei
|
ae4e96a2bc
|
fix: mocker engines should ignore downloading weights from hf (again) (#3664)
Signed-off-by: PeaBrane <yanrpei@gmail.com>
|
2025-10-16 02:48:40 +00:00 |
|
Graham King
|
ab0da5821e
|
feat: Python binding to download a model. (#3593)
Signed-off-by: Graham King <grahamk@nvidia.com>
|
2025-10-15 12:26:02 -04:00 |
|
Graham King
|
0e0218ff02
|
chore: Remove model_config from LocalModel (#3558)
Signed-off-by: Graham King <grahamk@nvidia.com>
|
2025-10-10 22:01:30 +00:00 |
|
Graham King
|
0aa0768f53
|
chore: Remove llama.cpp engine (#3499)
Signed-off-by: Graham King <grahamk@nvidia.com>
|
2025-10-08 16:44:53 -04:00 |
|
Graham King
|
1b1265e6fc
|
chore: Remove GGUF support (#3488)
Signed-off-by: Graham King <grahamk@nvidia.com>
|
2025-10-08 11:42:24 -04:00 |
|
Yan Ru Pei
|
031590fc14
|
feat: vllm prefill router (#3155)
Signed-off-by: PeaBrane <yanrpei@gmail.com>
|
2025-09-22 16:38:56 -07:00 |
|
Yan Ru Pei
|
5b19a39bab
|
feat: allow router to not track active blocks (prefill), and to not track cached blocks (decode) (#3135)
Signed-off-by: PeaBrane <yanrpei@gmail.com>
|
2025-09-19 20:33:07 +00:00 |
|
Graham King
|
723f2da74b
|
chore: Remove more extended Apache headers (#3063)
Signed-off-by: Graham King <grahamk@nvidia.com>
|
2025-09-16 15:48:51 -04:00 |
|
Graham King
|
87e6e0529d
|
fix: Interactive inputs actually stops, does not ignore stop token (#3057)
Signed-off-by: Graham King <grahamk@nvidia.com>
|
2025-09-16 14:39:05 -04:00 |
|
Yan Ru Pei
|
488c87095c
|
feat: Router warm restarts via durable KV event consumers and radix snapshotting (#2756)
Signed-off-by: PeaBrane <yanrpei@gmail.com>
|
2025-08-30 23:42:57 +00:00 |
|
GuanLuo
|
91a459c038
|
feat: KServe gRPC support (#2638)
|
2025-08-26 22:57:31 -07:00 |
|
Graham King
|
bce7458842
|
chore: Rust to 1.89 and edition 2024 (#2659)
|
2025-08-22 16:17:15 -07:00 |
|
nachiketb-nvidia
|
199b9a30f4
|
chore: Bring async-openai into repo as request starter (#2520)
Co-authored-by: Graham King <grahamk@nvidia.com>
|
2025-08-19 17:47:01 -04:00 |
|
Graham King
|
a4bbe49228
|
feat(http): TLS support (#2492)
|
2025-08-18 16:06:29 -04:00 |
|
Graham King
|
c443528fba
|
fix(preprocessor): Populate model ID in PreprocessedRequest (#2397)
|
2025-08-11 15:07:08 -04:00 |
|
Yan Ru Pei
|
5166a3dd44
|
feat: Router replicas with state-sharing (#2264)
|
2025-08-07 22:57:57 +00:00 |
|
Graham King
|
6be5c196f0
|
docs(dynamo-run): Remove vllm/sglang/trtllm engines from dynamo-run docs (#2332)
|
2025-08-06 16:26:16 +00:00 |
|
Graham King
|
6a1a801c2d
|
feat: Support static workers, run without etcd. (#2281)
|
2025-08-06 09:51:43 -04:00 |
|
Yan Ru Pei
|
bae25dc6d4
|
feat: skip downloading model weights if using mocker (only tokenizer) (#2213)
|
2025-07-31 18:15:57 +00:00 |
|
Jacky
|
fdcf611ffc
|
chore: Add Request Migration docs and minor enhancements (#2038)
|
2025-07-28 16:51:21 -07:00 |
|
Graham King
|
19a77ae7d5
|
chore(dynamo-run): Remove out=sglang|vllm|trtllm (#1920)
|
2025-07-23 17:22:31 -07:00 |
|
Jacky
|
1f07dab7bd
|
feat: Add migration to LLM requests (#1930)
|
2025-07-18 20:04:20 +00:00 |
|
Graham King
|
fc12436048
|
feat(frontend): router-mode settings (#2001)
|
2025-07-18 18:52:57 +00:00 |
|
Graham King
|
1eadc01340
|
feat(runtime): Support tokio-console (#1986)
|
2025-07-17 12:07:41 -07:00 |
|
Yan Ru Pei
|
f31732a22d
|
feat: integrate mocker with dynamo-run and python cli (#1927)
|
2025-07-16 18:22:15 +00:00 |
|
Yan Ru Pei
|
df91fce272
|
feat: prefill aware routing (#1895)
|
2025-07-14 19:24:04 +00:00 |
|
Graham King
|
ad8ad66b15
|
feat: Shrink the ai-dynamo wheel by 35 MiB (#1918)
Remove http and llmctl binaries. They have been unused for a while.
|
2025-07-14 19:20:35 +00:00 |
|
Alec
|
30e5c35bae
|
fix: add capi to pip and make it fallback (#1904)
|
2025-07-12 06:52:13 +00:00 |
|
Yan Ru Pei
|
13640e15d5
|
feat: allow using ApproxKvIndexer for routing via use_kv_events flag (#1869)
Signed-off-by: Yan Ru Pei <yanrpei@gmail.com>
Co-authored-by: Hongkuan Zhou <tedzhouhk@gmail.com>
|
2025-07-10 22:46:07 +00:00 |