mooncake-track
/
dynamo
0
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity
3,944 Commits 2 Branches 0 Tags 76 MiB
Commit Graph

390 Commits

Author SHA1 Message Date
Biswa Panda cec19d4db6
feat(perf): add request-plane, transport, and work-handler metric definitions (#6735) 2026-03-14 07:36:31 +00:00
Yan Ru Pei 5aa2f53fc2
feat(kv-router): add Prometheus metrics to standalone indexer (#7339) 
Signed-off-by: PeaBrane <yanrpei@gmail.com>
 2026-03-13 10:07:48 -07:00
Yan Ru Pei bddaaa2658
feat(kv-router): pluggable scheduling policy for router queue [DYN-2454] (#7260) 
Signed-off-by: PeaBrane <yanrpei@gmail.com>
 2026-03-13 04:16:49 +00:00
Michael Feil fca0a801dc
feat: follow up of optinal, batching-of-client-side-events (#6741) 
Signed-off-by: michaelfeil <63565275+michaelfeil@users.noreply.github.com>
Signed-off-by: Michael Feil <michaelfeil@users.noreply.github.com>
Signed-off-by: Michael Feil <63565275+michaelfeil@users.noreply.github.com>
 2026-03-11 23:30:51 +00:00
MatejKosec 5178a4a485
feat: streaming tool call and reasoning dispatch SSE events (#7114) 
Signed-off-by: Matej Kosec <mkosec@nvidia.com>
 2026-03-11 15:24:39 -07:00
jthomson04 761e848b5b
fix: Fix mocker+router segfaults + reliability fixes (#7162) 
Signed-off-by: jthomson04 <jwillthomson19@gmail.com>
 2026-03-10 22:39:06 -07:00
MatejKosec 012236ee4e
feat(anthropic): add thinking block support and preamble stripping to /v1/messages (#7137) 
Signed-off-by: Matej Kosec <mkosec@nvidia.com>
 2026-03-11 00:07:09 +00:00
Ryan Olson 72e849e780
chore: set velo crates to initial v0.1.0 (#7156) 
Signed-off-by: Ryan Olson <rolson@nvidia.com>
Co-authored-by: Anant Sharma <anants@nvidia.com>
 2026-03-10 17:57:24 -06:00
jh-nv 734c0b8b0b
feat: Add Loki log aggregation, and enable unified OTLP ingestion for both traces and logs through a single endpoint (#6974) 2026-03-10 15:20:33 -07:00
Biswa Panda 3c44b88e89
feat(perf): add frontend pipeline and tokio runtime perf metric definitions (#6731) 2026-03-10 21:04:02 +00:00
Graham King 5265787b5d
chore: Enable "multipart" feature of reqwest (#7084) 
Signed-off-by: Graham King <grahamk@nvidia.com>
 2026-03-10 13:47:01 +00:00
Biswa Panda b407b41969
fix: resolve HTTP port collisions when multiple workers share a process (#7063) 2026-03-10 05:50:52 +00:00
Ryan Olson 63d7c01c9c
feat: velo-backend (#6547) 
Signed-off-by: Ryan Olson <rolson@nvidia.com>
 2026-03-09 23:26:25 +00:00
Yifan Jiang 100819299f
feat(trtllm): add additional metrics for dynamo-trtllm (#6668) 
Signed-off-by: Yifan Jiang <19356972+yifjiang@users.noreply.github.com>
Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
 2026-03-06 14:34:38 -08:00
Shounak Ray 64ae47fdc9
fix: use checked arithmetic in TwoPartCodec to prevent integer overflow (#6959) 
Signed-off-by: ShounakRay <shounak@stanford.edu>
 2026-03-06 19:53:44 +00:00
Ryan McCormick 2f15f334e6
chore(UX): Log full endpoint name when registering TCP endpoint (#6964) 2026-03-06 09:06:14 -05:00
Yuewei Na 124ecd9804
fix: reject prompts exceeding max_seq_len with HTTP 400 (#6635) 
Signed-off-by: Yuewei Na <nv-yna@users.noreply.github.com>
Co-authored-by: Yuewei Na <nv-yna@users.noreply.github.com>
 2026-03-06 00:08:18 +00:00
Keiven C c24882ffb2
fix: validate initial uptime metric parsing in integration test (#6943) 
Signed-off-by: Keiven Chang <keivenchang@users.noreply.github.com>
 2026-03-05 22:36:56 +00:00
Ryan Olson 57bdbdf1ee
feat: upgrade rust to v1.93.0 (#6802) 
Signed-off-by: Ryan Olson <rolson@nvidia.com>
 2026-03-03 17:08:35 +00:00
Ryan Olson 7604295938
feat(kvbm): add testing utilities to kvbm-logical and kvbm-physical (#6691) 
Signed-off-by: Ryan Olson <rolson@nvidia.com>
 2026-03-02 21:18:32 -07:00
Yan Ru Pei 073fb4371d
feat(kv-router): add router queue depth Prometheus metric and nvext field (#6786) 
Signed-off-by: PeaBrane <yanrpei@gmail.com>
 2026-03-02 17:04:51 -08:00
Ryan McCormick 9e6a84afa2
chore(style): Log with {var} style in lib/runtime (#6757) 2026-03-02 17:29:07 +00:00
Ryan Olson 9ab148dcb9
feat: kvbm-physical (#6490) 
Signed-off-by: Ryan Olson <rolson@nvidia.com>
 2026-03-01 09:43:03 +00:00
Harrison Saturley-Hall db88c9530b
chore: bump versions to 1.0.0 (#6613) 
Signed-off-by: Harrison Saturley-Hall <hsaturleyhal@nvidia.com>
 2026-02-25 22:42:33 -05:00
daiyaanarfeen 651ef5b506
feat: throughput-metrics-source for SLA planner + GlobalPlanner disagg scaling (#6500) 
Signed-off-by: Daiyaan <darfeen@nvidia.com>
Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com>
 2026-02-25 22:58:24 +00:00
Keiven C ff06b17e7f
fix: guarantee RouterRequestMetrics availability & documentation updates (#6558) 
Signed-off-by: Keiven Chang <keivenchang@users.noreply.github.com>
Co-authored-by: Keiven Chang <keivenchang@users.noreply.github.com>
Co-authored-by: Cursor <cursoragent@cursor.com>
 2026-02-25 10:18:01 -08:00
Ryan McCormick 4bd4c1657b
feat: Log discovery backend type (#6559) 2026-02-25 02:25:44 +00:00
Keiven C 1df620b4d0
feat: router metrics with dynamo_router_* {worker_id=...}. Update docs (#6227) 
Signed-off-by: Keiven Chang <keivenchang@users.noreply.github.com>
Co-authored-by: Keiven Chang <keivenchang@users.noreply.github.com>
Co-authored-by: Cursor <cursoragent@cursor.com>
 2026-02-24 21:34:14 +00:00
Vladislav Nosivskoy 432fae67b1
feat: add error_type label to reqs metric for fine-grained error classification (#5568) 
Signed-off-by: Vladislav Nosivskoy <vladnosiv@gmail.com>
Signed-off-by: Keiven Chang <keivenchang@users.noreply.github.com>
Co-authored-by: Keiven Chang <keivenchang@users.noreply.github.com>
Co-authored-by: Cursor <cursoragent@cursor.com>
 2026-02-24 10:31:47 -08:00
Jacky c8276cd28b
feat: Standardized Dynamo Error Type (#6303) 
Signed-off-by: Jacky <18255193+kthui@users.noreply.github.com>
 2026-02-23 20:23:58 -08:00
Tzu-Ling Kan 80cac7c14b
feat: Remove Component from public (#6403) 
Signed-off-by: tzulingk@nvidia.com <tzulingk@nvidia.com>
 2026-02-23 22:58:38 +00:00
Yan Ru Pei 038b50d2b6
feat: add standalone KV indexer with query endpoint [DYN-2164] (#6446) 
Signed-off-by: PeaBrane <yanrpei@gmail.com>
Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
 2026-02-23 12:12:25 -08:00
Ryan Olson 7a6283e927
chore: cargo machete - remove unused dependencies (#6453) 
Signed-off-by: Ryan Olson <rolson@nvidia.com>
 2026-02-20 19:11:52 +00:00
Yan Ru Pei cf51a0c4fd
chore: gate plotters dependency behind bench feature (#6380) 
Co-authored-by: Cursor <cursoragent@cursor.com>
 2026-02-18 15:27:19 -05:00
Yan Ru Pei c5c6a55190
feat: more flash indexer optimizations (#6305) 
Signed-off-by: PeaBrane <yanrpei@gmail.com>
Co-authored-by: Cursor <cursoragent@cursor.com>
 2026-02-18 09:13:37 -08:00
jh-nv 56379dd584
chore: standardizes Rust-side Dynamo environment variable names (#6358) 2026-02-18 11:39:03 -05:00
ishandhanani b6603d90d6
feat: add Anthropic Messages API endpoint (/v1/messages) (#6231) 
Signed-off-by: Matej Kosec <mkosec@nvidia.com>
Signed-off-by: Marko Kosec <mkosec@nvidia.com>
Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
Co-authored-by: Matej Kosec <mkosec@nvidia.com>
 2026-02-18 06:11:52 +00:00
Ryan Olson 94fad4799f
feat: kvbm-logical (#6033) 
Signed-off-by: Ryan Olson <rolson@nvidia.com>
 2026-02-14 02:24:26 +00:00
mohammedabdulwahhab a289695c37
fix: consolidate dyn_discovery_backend and dyn_kv_store (#6167) 
Signed-off-by: mohammedabdulwahhab <furkhan324@berkeley.edu>
 2026-02-13 19:07:11 +00:00
Yan Ru Pei f6c34e07de
chore: otel tracings for routing overheads and metrics (#6194) 
Signed-off-by: PeaBrane <yanrpei@gmail.com>
 2026-02-13 09:34:46 +00:00
atchernych 5227176057
feat: Decomposed pipeline for EPP integration [DEP-730] (#5446) 
Signed-off-by: Anna Tchernych <atchernych@nvidia.com>
 2026-02-13 06:18:29 +00:00
Graham King bbe82f182a
chore: Remove dynamo-run and mistral-rs engine (#6203) 
Signed-off-by: Graham King <grahamk@nvidia.com>
 2026-02-12 21:50:58 +00:00
Yan Ru Pei 7b16480ab4
chore: warn out request rejection (#6222) 
Signed-off-by: PeaBrane <yanrpei@gmail.com>
 2026-02-12 07:17:51 +00:00
Jonathan Tong 39d645e586
docs: migrate Fern docs from fern/ into docs/ (#6206) 
Signed-off-by: Jont828 <jt572@cornell.edu>
 2026-02-11 16:22:27 -08:00
Graham King adaf1a3989
feat: Metric for detokenization latency (#6160) 
Signed-off-by: Graham King <grahamk@nvidia.com>
 2026-02-11 20:06:13 +00:00
Yan Ru Pei f46720c996
feat: add router-level Prometheus metrics and centralize request tracking (#6146) 
Signed-off-by: PeaBrane <yanrpei@gmail.com>
Co-authored-by: Cursor <cursoragent@cursor.com>
 2026-02-11 07:36:08 +00:00
Keiven C e18840cef3
feat: add Prometheus auto and custom label injection for engine metrics (#5989) 
Signed-off-by: Keiven Chang <keivenchang@users.noreply.github.com>
 2026-02-10 21:22:53 -08:00
Graham King f1bcb17542
feat: Add metric tokenizer_latency_ms (#6092) 
Signed-off-by: Graham King <grahamk@nvidia.com>
 2026-02-10 08:52:31 -08:00
Keiven C f51ec24d6f
fix: test_router_decisions async fix race condition (#6084) 
Signed-off-by: Keiven Chang <keivenchang@users.noreply.github.com>
 2026-02-09 17:12:31 -08:00
Keiven C 027d2653a5
feat: expose Python Prometheus metric via DynamoComponentMetrics (#5817) 
Signed-off-by: Keiven Chang <keivenchang@users.noreply.github.com>
 2026-02-09 17:10:37 -08:00