mooncake-track
/
dynamo
0
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity
2,809 Commits 2 Branches 0 Tags 76 MiB
Commit Graph

108 Commits

Author SHA1 Message Date
Neelay Shah d22d9e761e
fix: support truthy values for OTEL_EXPORT_ENABLED in bindings layer [DYN-1690][DYN-1646] (#5129) 
Co-authored-by: Claude <noreply@anthropic.com>
 2026-01-02 23:39:09 +00:00
Tushar Sharma cf433e6825
chore: update all copyright headers in repo to 2026 (#5130) 
Signed-off-by: Tushar Sharma <tusharma@nvidia.com>
 2026-01-02 22:08:23 +00:00
Biswa Panda 18b64e90b6
feat: switch to tcp as default request plane (#4845) 2025-12-18 23:05:38 +00:00
William Arnold 3bc129dc37
feat: Add python-configurable engine routes for sglang profiling (#4617) 
Signed-off-by: William Arnold <7565007+Aphoh@users.noreply.github.com>
 2025-12-05 10:28:36 -08:00
Neal Vaidya 118323f26e
fix: skip HuggingFace download for non-llms (#4686) 
Signed-off-by: Neal Vaidya <nealv@nvidia.com>
 2025-12-04 10:23:24 -08:00
Biswa Panda 71f94eda23
feat: lora - centralize lora cache key, restructure folders, s3 resiliency (#4644) 2025-12-02 08:57:25 +00:00
Biswa Panda 6a0e67ed78
feat: add LoRA common APIs and implementation for lora management (#4464) 2025-11-26 20:29:25 +00:00
Graham King fcb91e4b3d
refactor(storage): Remove the stuttering from key_value_store. (#4604) 
Signed-off-by: Graham King <grahamk@nvidia.com>
 2025-11-25 22:59:29 +00:00
Biswa Panda 781331c6f3
feat: unregister discovery instance (#4459) 2025-11-19 20:25:44 +00:00
Graham King 69797b5ab3
feat: Only monitor NATS metrics if using NATS request plane (#4442) 
Signed-off-by: Graham King <grahamk@nvidia.com>
 2025-11-19 17:48:49 +00:00
Biswa Panda 684107c461
feat: generate deterministic lora id from lora name (#4457) 2025-11-19 02:26:00 +00:00
Keiven C 0e77d3442e
refactor: centralize environment variable constants (#4083) 
Signed-off-by: Keiven Chang <keivenchang@users.noreply.github.com>
Co-authored-by: Keiven Chang <keivenchang@users.noreply.github.com>
 2025-11-17 11:16:38 -08:00
Graham King 886506c12d
feat: Command line flag to set request plane mode: tcp, http or nats (#4365) 
Signed-off-by: Graham King <grahamk@nvidia.com>
 2025-11-17 09:02:36 -08:00
Graham King fbad28602e
chore: Remove the python bindings for port allocation (#4237) 
Signed-off-by: Graham King <grahamk@nvidia.com>
 2025-11-11 20:35:36 +00:00
Graham King e1af3af6ee
chore: Remove static mode (#4235) 
Signed-off-by: Graham King <grahamk@nvidia.com>
 2025-11-11 19:25:12 +00:00
Keiven C afccc9d434
refactor: consolidate Observability files (e.g. OTEL docker-compose, md files) (#4173) 
Signed-off-by: Keiven Chang <keivenchang@users.noreply.github.com>
Co-authored-by: Keiven Chang <keivenchang@users.noreply.github.com>
 2025-11-10 20:19:34 +00:00
Graham King cf630bf7b6
refactor: Make the Runtime and DistributedRuntime fields private (#4193) 
Signed-off-by: Graham King <grahamk@nvidia.com>
 2025-11-10 15:56:22 +00:00
milesial 14af074ecd
feat: Media decoder and fetcher options in the MDC (#4094) 
Signed-off-by: Alexandre Milesi <milesial@users.noreply.github.com>
 2025-11-08 02:17:39 +00:00
Graham King 794c0a4431
feat(keyvalue): Filesystem backed KeyValueStore (#4138) 
Signed-off-by: Graham King <grahamk@nvidia.com>
 2025-11-07 21:11:33 +00:00
Graham King dadf0e2247
chore: Remove old DisaggregatedRouter, making etcd presence optional (#4011) 
Signed-off-by: Graham King <grahamk@nvidia.com>
 2025-11-03 10:10:13 -05:00
Yan Ru Pei 6ff49edbb7
feat: efficient serving of multiple mockers (#3997) 
Signed-off-by: PeaBrane <yanrpei@gmail.com>
 2025-11-01 01:24:55 +00:00
Richard Huo 6afa679c5d
chore: KVBM pip wheel (#3826) 
Signed-off-by: Anant Sharma <anants@nvidia.com>
Co-authored-by: Anant Sharma <anants@nvidia.com>
 2025-10-31 14:19:30 -07:00
Graham King c78b590134
chore(runtime): Do not expose etcd lease ID (#3915) 
Signed-off-by: Graham King <grahamk@nvidia.com>
 2025-10-28 16:20:25 +00:00
zhongdaor-nv 9e5407f20d
feat: python binding for kserve grpc frontend (#3739) 
Signed-off-by: zhongdaor <zhongdaor@nvidia.com>
Signed-off-by: zhongdaor-nv <zhongdaor@nvidia.com>
Co-authored-by: coderabbitai[bot] <136622811+coderabbitai[bot]@users.noreply.github.com>
Co-authored-by: Ryan McCormick <rmccormick@nvidia.com>
 2025-10-24 11:20:20 -07:00
Graham King 7731b0245c
chore: Use KeyValueStoreManager instead of etcd::Client (#3822) 
Signed-off-by: Graham King <grahamk@nvidia.com>
 2025-10-23 14:09:58 -04:00
mohammedabdulwahhab 8a04d7b252
fix: revert placement of logging init if otel exports is disabled (#3797) 
Signed-off-by: mohammedabdulwahhab <furkhan324@berkeley.edu>
 2025-10-21 17:36:31 -04:00
Graham King f6ed01b14c
chore: Replace ServiceConfigBuilder with add_stats_service (#3736) 
Signed-off-by: Graham King <grahamk@nvidia.com>
 2025-10-20 16:47:05 -04:00
Yan Ru Pei 4b7a806c7a
feat: add prefill workers to discovery (#3709) 
Signed-off-by: PeaBrane <yanrpei@gmail.com>
 2025-10-18 00:59:32 +00:00
Yan Ru Pei 4c207e0c8c
chore: remove kv metrics scraping and aggregation (#3701) 
Signed-off-by: PeaBrane <yanrpei@gmail.com>
 2025-10-17 10:58:48 -07:00
Jacky 734d2f87a8
feat: Enable cancellation during or before a stream is established (#3635) 
Signed-off-by: Jacky <18255193+kthui@users.noreply.github.com>
Co-authored-by: Ryan McCormick <rmccormick@nvidia.com>
 2025-10-15 19:22:55 -07:00
Graham King ab0da5821e
feat: Python binding to download a model. (#3593) 
Signed-off-by: Graham King <grahamk@nvidia.com>
 2025-10-15 12:26:02 -04:00
mohammedabdulwahhab 1f92dd547e
feat: OTEL Exporter and Tempo Visualization (#3307) 
Signed-off-by: mohammedabdulwahhab <furkhan324@berkeley.edu>
 2025-10-13 18:39:09 +00:00
Keiven C a4746ab65f
fix: callback registration, fix metric name access, ensure ordered vec, etc... (#3541) 
Signed-off-by: Keiven Chang <keivenchang@users.noreply.github.com>
Co-authored-by: Keiven Chang <keivenchang@users.noreply.github.com>
 2025-10-11 00:03:24 +00:00
Keiven C f2ba58e568
feat: add Rust-to-Python const code generator for prometheus_names.py (#3425) 
Signed-off-by: Keiven Chang <keivenchang@users.noreply.github.com>
 2025-10-10 11:20:36 -07:00
Graham King e9a710095b
chore: Remove clear_namespace script (#3519) 
Signed-off-by: Graham King <grahamk@nvidia.com>
 2025-10-09 13:43:59 -04:00
Keiven C 0c4c4d1de3
feat: add Python MetricsRegistry Python metrics registration (#3341) 
Signed-off-by: Keiven Chang <keivenchang@users.noreply.github.com>
 2025-10-07 21:24:25 -07:00
Graham King a5371bfc50
feat(etcd): Version the etcd keys (#3458) 
Signed-off-by: Graham King <grahamk@nvidia.com>
 2025-10-07 16:48:26 +00:00
Yan Ru Pei 30610e7371
feat: use KvPushRouter for prefill router (#3401) 
Signed-off-by: PeaBrane <yanrpei@gmail.com>
 2025-10-03 18:21:48 -07:00
Michael Feil 5b457b70a6
feat: python add abi compatability for cross-platform builds + add a unit test to HttpServer (#3044) 
Signed-off-by: michaelfeil <me@michaelfeil.eu>
Signed-off-by: Michael Feil <63565275+michaelfeil@users.noreply.github.com>
Signed-off-by: root <root@michaelfeil2-dev-pod-b200-0.michaelfeil2-dev-pod-b200.baseten.svc.cluster.local>
Signed-off-by: root <root@michaelfeildns-dev-pod-h100-0.michaelfeildns-dev-pod-h100.baseten.svc.cluster.local>
Co-authored-by: root <root@michaelfeil2-dev-pod-b200-0.michaelfeil2-dev-pod-b200.baseten.svc.cluster.local>
Co-authored-by: root <root@michaelfeildns-dev-pod-h100-0.michaelfeildns-dev-pod-h100.baseten.svc.cluster.local>
 2025-09-30 14:42:42 -07:00
Keiven C cacac9b9f4
feat: add Python const for Prometheus metric names (#3244) 
Signed-off-by: Keiven Chang <keivenchang@users.noreply.github.com>
 2025-09-30 10:28:47 -07:00
Graham King 7ebbd001c0
chore: Remove etcd from python bindings (#3238) 
Signed-off-by: Graham King <grahamk@nvidia.com>
 2025-09-26 12:58:30 -04:00
Neelay Shah f2e2e935b2
feat: Add distributed tracing context support to Python bindings (#3160) 
Signed-off-by: nnshah1 <neelays@nvidia.com>
Signed-off-by: Jacky <18255193+kthui@users.noreply.github.com>
Co-authored-by: Jacky <18255193+kthui@users.noreply.github.com>
 2025-09-25 17:26:35 -07:00
Graham King c03e2f6bbe
chore: Migrate planner virtual_connector internals into bindings (#3205) 
Signed-off-by: Graham King <grahamk@nvidia.com>
 2025-09-25 18:04:58 -04:00
GuanLuo 6ba64c31f5
feat: tensor type for generic inference. (#2746) 
Signed-off-by: Guan Luo <gluo@nvidia.com>
Signed-off-by: GuanLuo <41310872+GuanLuo@users.noreply.github.com>
Co-authored-by: Olga Andreeva <124622579+oandreeva-nv@users.noreply.github.com>
Co-authored-by: Ryan McCormick <rmccormick@nvidia.com>
 2025-09-24 07:31:09 +00:00
Yan Ru Pei 031590fc14
feat: vllm prefill router (#3155) 
Signed-off-by: PeaBrane <yanrpei@gmail.com>
 2025-09-22 16:38:56 -07:00
Graham King 7a5a0bd6cd
chore: Upgrade Rust to 1.90 (#3147) 
Signed-off-by: Graham King <grahamk@nvidia.com>
 2025-09-19 18:54:29 -04:00
Graham King 3865a94148
feat: Port vllm port allocator to Rust in bindings (#3125) 
Signed-off-by: Graham King <grahamk@nvidia.com>
 2025-09-19 18:34:52 +00:00
Graham King b6595e2484
chore(bindings): Provide a binding to clear etcd namespace (#3094) 
Signed-off-by: Graham King <grahamk@nvidia.com>
 2025-09-18 11:49:08 -04:00
Graham King f88d7dc74b
chore(bindings): Remove NatsQueue (#3086) 
Signed-off-by: Graham King <grahamk@nvidia.com>
 2025-09-17 12:46:54 -04:00
Tzu-Ling Kan 08cb08c1bc
feat: Canary Health Check. (#2903) 
Signed-off-by: tzulingk@nvidia.com <tzulingk@nvidia.com>
 2025-09-17 03:31:56 +00:00