mooncake-track
/
dynamo
0
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity
3,557 Commits 2 Branches 0 Tags 76 MiB
Commit Graph

366 Commits

Author SHA1 Message Date
daiyaanarfeen 651ef5b506
feat: throughput-metrics-source for SLA planner + GlobalPlanner disagg scaling (#6500) 
Signed-off-by: Daiyaan <darfeen@nvidia.com>
Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com>
 2026-02-25 22:58:24 +00:00
Keiven C ff06b17e7f
fix: guarantee RouterRequestMetrics availability & documentation updates (#6558) 
Signed-off-by: Keiven Chang <keivenchang@users.noreply.github.com>
Co-authored-by: Keiven Chang <keivenchang@users.noreply.github.com>
Co-authored-by: Cursor <cursoragent@cursor.com>
 2026-02-25 10:18:01 -08:00
Ryan McCormick 4bd4c1657b
feat: Log discovery backend type (#6559) 2026-02-25 02:25:44 +00:00
Keiven C 1df620b4d0
feat: router metrics with dynamo_router_* {worker_id=...}. Update docs (#6227) 
Signed-off-by: Keiven Chang <keivenchang@users.noreply.github.com>
Co-authored-by: Keiven Chang <keivenchang@users.noreply.github.com>
Co-authored-by: Cursor <cursoragent@cursor.com>
 2026-02-24 21:34:14 +00:00
Vladislav Nosivskoy 432fae67b1
feat: add error_type label to reqs metric for fine-grained error classification (#5568) 
Signed-off-by: Vladislav Nosivskoy <vladnosiv@gmail.com>
Signed-off-by: Keiven Chang <keivenchang@users.noreply.github.com>
Co-authored-by: Keiven Chang <keivenchang@users.noreply.github.com>
Co-authored-by: Cursor <cursoragent@cursor.com>
 2026-02-24 10:31:47 -08:00
Jacky c8276cd28b
feat: Standardized Dynamo Error Type (#6303) 
Signed-off-by: Jacky <18255193+kthui@users.noreply.github.com>
 2026-02-23 20:23:58 -08:00
Tzu-Ling Kan 80cac7c14b
feat: Remove Component from public (#6403) 
Signed-off-by: tzulingk@nvidia.com <tzulingk@nvidia.com>
 2026-02-23 22:58:38 +00:00
Yan Ru Pei 038b50d2b6
feat: add standalone KV indexer with query endpoint [DYN-2164] (#6446) 
Signed-off-by: PeaBrane <yanrpei@gmail.com>
Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
 2026-02-23 12:12:25 -08:00
Ryan Olson 7a6283e927
chore: cargo machete - remove unused dependencies (#6453) 
Signed-off-by: Ryan Olson <rolson@nvidia.com>
 2026-02-20 19:11:52 +00:00
Yan Ru Pei cf51a0c4fd
chore: gate plotters dependency behind bench feature (#6380) 
Co-authored-by: Cursor <cursoragent@cursor.com>
 2026-02-18 15:27:19 -05:00
Yan Ru Pei c5c6a55190
feat: more flash indexer optimizations (#6305) 
Signed-off-by: PeaBrane <yanrpei@gmail.com>
Co-authored-by: Cursor <cursoragent@cursor.com>
 2026-02-18 09:13:37 -08:00
jh-nv 56379dd584
chore: standardizes Rust-side Dynamo environment variable names (#6358) 2026-02-18 11:39:03 -05:00
ishandhanani b6603d90d6
feat: add Anthropic Messages API endpoint (/v1/messages) (#6231) 
Signed-off-by: Matej Kosec <mkosec@nvidia.com>
Signed-off-by: Marko Kosec <mkosec@nvidia.com>
Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
Co-authored-by: Matej Kosec <mkosec@nvidia.com>
 2026-02-18 06:11:52 +00:00
Ryan Olson 94fad4799f
feat: kvbm-logical (#6033) 
Signed-off-by: Ryan Olson <rolson@nvidia.com>
 2026-02-14 02:24:26 +00:00
mohammedabdulwahhab a289695c37
fix: consolidate dyn_discovery_backend and dyn_kv_store (#6167) 
Signed-off-by: mohammedabdulwahhab <furkhan324@berkeley.edu>
 2026-02-13 19:07:11 +00:00
Yan Ru Pei f6c34e07de
chore: otel tracings for routing overheads and metrics (#6194) 
Signed-off-by: PeaBrane <yanrpei@gmail.com>
 2026-02-13 09:34:46 +00:00
atchernych 5227176057
feat: Decomposed pipeline for EPP integration [DEP-730] (#5446) 
Signed-off-by: Anna Tchernych <atchernych@nvidia.com>
 2026-02-13 06:18:29 +00:00
Graham King bbe82f182a
chore: Remove dynamo-run and mistral-rs engine (#6203) 
Signed-off-by: Graham King <grahamk@nvidia.com>
 2026-02-12 21:50:58 +00:00
Yan Ru Pei 7b16480ab4
chore: warn out request rejection (#6222) 
Signed-off-by: PeaBrane <yanrpei@gmail.com>
 2026-02-12 07:17:51 +00:00
Jonathan Tong 39d645e586
docs: migrate Fern docs from fern/ into docs/ (#6206) 
Signed-off-by: Jont828 <jt572@cornell.edu>
 2026-02-11 16:22:27 -08:00
Graham King adaf1a3989
feat: Metric for detokenization latency (#6160) 
Signed-off-by: Graham King <grahamk@nvidia.com>
 2026-02-11 20:06:13 +00:00
Yan Ru Pei f46720c996
feat: add router-level Prometheus metrics and centralize request tracking (#6146) 
Signed-off-by: PeaBrane <yanrpei@gmail.com>
Co-authored-by: Cursor <cursoragent@cursor.com>
 2026-02-11 07:36:08 +00:00
Keiven C e18840cef3
feat: add Prometheus auto and custom label injection for engine metrics (#5989) 
Signed-off-by: Keiven Chang <keivenchang@users.noreply.github.com>
 2026-02-10 21:22:53 -08:00
Graham King f1bcb17542
feat: Add metric tokenizer_latency_ms (#6092) 
Signed-off-by: Graham King <grahamk@nvidia.com>
 2026-02-10 08:52:31 -08:00
Keiven C f51ec24d6f
fix: test_router_decisions async fix race condition (#6084) 
Signed-off-by: Keiven Chang <keivenchang@users.noreply.github.com>
 2026-02-09 17:12:31 -08:00
Keiven C 027d2653a5
feat: expose Python Prometheus metric via DynamoComponentMetrics (#5817) 
Signed-off-by: Keiven Chang <keivenchang@users.noreply.github.com>
 2026-02-09 17:10:37 -08:00
Yan Ru Pei 6783bdcaa9
chore: enable local indexers by default, and use normal event plane by default (not jetstream) (#5941) 
Signed-off-by: PeaBrane <yanrpei@gmail.com>
 2026-02-08 20:48:02 +00:00
Konstantin Korolev d59b9d72ed
fix: reduce NATS consumer inactive_threshold from 1h to 2min (#5861) 
Signed-off-by: advpropsys <korolev.konstantin.v@gmail.com>
Co-authored-by: Yan Ru Pei <yanrpei@gmail.com>
 2026-02-07 03:02:45 +00:00
Yongming Ding 7c25f70291
feat(mocker): add optional KV cache allocation/eviction trace (#6052) 
Signed-off-by: Yongming Ding <yongmingd@nvidia.com>
Co-authored-by: Yan Ru Pei <yanrpei@gmail.com>
 2026-02-07 03:02:28 +00:00
Anant Sharma 3842b24479
fix: update bytes crate version to latest (#6041) 
Signed-off-by: Anant Sharma <anants@nvidia.com>
 2026-02-06 15:11:04 -05:00
Janelle Cai 039d35ffd0
test: indexer and full router benchmarks (#5784) 2026-02-04 20:40:27 +00:00
Hongkuan Zhou 0c0336e6ea
feat: Add per-worker Prometheus metrics for router load monitoring (#5842) 
Signed-off-by: hongkuanz <hongkuanz@nvidia.com>
 2026-02-04 11:51:43 -08:00
Ameen Patel da0cbf6856
fix(runtime): return 500 on LoRA load/unload errors (#5626) 
Signed-off-by: AmeenP <ameenp360@gmail.com>
Co-authored-by: Biswa Panda <biswa.panda@gmail.com>
 2026-02-03 16:10:29 -08:00
Julien Mancuso f3aa1e0129
feat: introducing ChReK (Checkpoint Restore in K8s) (#4978) 
Signed-off-by: Julien Mancuso <jmancuso@nvidia.com>
 2026-02-03 13:34:14 -07:00
Yan Ru Pei aa4ac947c8
chore: atomic file write (#5809) 
Signed-off-by: PeaBrane <yanrpei@gmail.com>
 2026-01-30 01:52:10 +00:00
Yan Ru Pei 584020f4b3
chore: consistently used the arc shared workers configs + CI flake fixes (#5707) 
Signed-off-by: PeaBrane <yanrpei@gmail.com>
 2026-01-29 09:29:24 +00:00
Biswa Panda 1174c819cb
feat: remove deprecated events api (#5778) 2026-01-29 05:48:16 +00:00
Keiven C 5a00a7d643
fix: prevent duplicate Prometheus metrics from Python expfmt callbacks (#5761) 
Signed-off-by: Keiven Chang <keivenchang@users.noreply.github.com>
 2026-01-28 23:34:33 +00:00
Graham King 2ffd58475b
chore: Remove native-tls / openssl dependency (#5703) 
Signed-off-by: Graham King <grahamk@nvidia.com>
 2026-01-28 17:36:11 +00:00
Keiven C 3d62cc7fd9
fix: avoid Prometheus collisions via multi-registry scrape (#5678) 
Signed-off-by: Keiven Chang <keivenchang@users.noreply.github.com>
Co-authored-by: Keiven Chang <keivenchang@users.noreply.github.com>
 2026-01-28 09:30:01 -08:00
Graham King 90b0759432
chore: Fix tests logging setup (#5693) 
Signed-off-by: Graham King <grahamk@nvidia.com>
 2026-01-27 22:06:34 +00:00
Yan Ru Pei 382a26f0ab
chore: type ZmqMessage (#5689) 
Signed-off-by: PeaBrane <yanrpei@gmail.com>
 2026-01-27 21:21:23 +00:00
Biswa Panda bc76247dff
fix: add recv timeout for zmq transport (#5685) 2026-01-27 18:55:50 +00:00
Biswa Panda 8fc70f20f3
feat: add ZMQ transport and its discovery (#5625) 2026-01-27 02:31:04 +00:00
Pavithra Vijayakrishnan 66c369964a
chore: version update for 0.9.0 (#5661) 
Signed-off-by: pvijayakrish <pvijayakrish@nvidia.com>
 2026-01-26 17:18:05 -08:00
Qi Wang 2a2ad756ff
test: unit test TCP client handle_reader [2/n] (#5056) 2026-01-26 05:01:49 +00:00
Biswa Panda ebde0f662c
feat: event plane apis and nats implementation (#5624) 2026-01-24 02:47:00 +00:00
Biswa Panda 3ee989252c
feat: add event plane discovery (#5614) 2026-01-23 13:59:43 -08:00
Yan Ru Pei feb6d272ac
chore: nuke ForwardPassMetrics (#5531) 
Signed-off-by: PeaBrane <yanrpei@gmail.com>
 2026-01-23 16:26:01 +00:00
Biswa Panda e2b12517c6
chore: reduce verbosity during dynamo start (#5518) 2026-01-21 19:11:07 +00:00