mooncake-track
/
dynamo
0
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity
3,250 Commits 2 Branches 0 Tags 76 MiB
Commit Graph

342 Commits

Author SHA1 Message Date
Keiven C f51ec24d6f
fix: test_router_decisions async fix race condition (#6084) 
Signed-off-by: Keiven Chang <keivenchang@users.noreply.github.com>
 2026-02-09 17:12:31 -08:00
Keiven C 027d2653a5
feat: expose Python Prometheus metric via DynamoComponentMetrics (#5817) 
Signed-off-by: Keiven Chang <keivenchang@users.noreply.github.com>
 2026-02-09 17:10:37 -08:00
Yan Ru Pei 6783bdcaa9
chore: enable local indexers by default, and use normal event plane by default (not jetstream) (#5941) 
Signed-off-by: PeaBrane <yanrpei@gmail.com>
 2026-02-08 20:48:02 +00:00
Konstantin Korolev d59b9d72ed
fix: reduce NATS consumer inactive_threshold from 1h to 2min (#5861) 
Signed-off-by: advpropsys <korolev.konstantin.v@gmail.com>
Co-authored-by: Yan Ru Pei <yanrpei@gmail.com>
 2026-02-07 03:02:45 +00:00
Yongming Ding 7c25f70291
feat(mocker): add optional KV cache allocation/eviction trace (#6052) 
Signed-off-by: Yongming Ding <yongmingd@nvidia.com>
Co-authored-by: Yan Ru Pei <yanrpei@gmail.com>
 2026-02-07 03:02:28 +00:00
Anant Sharma 3842b24479
fix: update bytes crate version to latest (#6041) 
Signed-off-by: Anant Sharma <anants@nvidia.com>
 2026-02-06 15:11:04 -05:00
Janelle Cai 039d35ffd0
test: indexer and full router benchmarks (#5784) 2026-02-04 20:40:27 +00:00
Hongkuan Zhou 0c0336e6ea
feat: Add per-worker Prometheus metrics for router load monitoring (#5842) 
Signed-off-by: hongkuanz <hongkuanz@nvidia.com>
 2026-02-04 11:51:43 -08:00
Ameen Patel da0cbf6856
fix(runtime): return 500 on LoRA load/unload errors (#5626) 
Signed-off-by: AmeenP <ameenp360@gmail.com>
Co-authored-by: Biswa Panda <biswa.panda@gmail.com>
 2026-02-03 16:10:29 -08:00
Julien Mancuso f3aa1e0129
feat: introducing ChReK (Checkpoint Restore in K8s) (#4978) 
Signed-off-by: Julien Mancuso <jmancuso@nvidia.com>
 2026-02-03 13:34:14 -07:00
Yan Ru Pei aa4ac947c8
chore: atomic file write (#5809) 
Signed-off-by: PeaBrane <yanrpei@gmail.com>
 2026-01-30 01:52:10 +00:00
Yan Ru Pei 584020f4b3
chore: consistently used the arc shared workers configs + CI flake fixes (#5707) 
Signed-off-by: PeaBrane <yanrpei@gmail.com>
 2026-01-29 09:29:24 +00:00
Biswa Panda 1174c819cb
feat: remove deprecated events api (#5778) 2026-01-29 05:48:16 +00:00
Keiven C 5a00a7d643
fix: prevent duplicate Prometheus metrics from Python expfmt callbacks (#5761) 
Signed-off-by: Keiven Chang <keivenchang@users.noreply.github.com>
 2026-01-28 23:34:33 +00:00
Graham King 2ffd58475b
chore: Remove native-tls / openssl dependency (#5703) 
Signed-off-by: Graham King <grahamk@nvidia.com>
 2026-01-28 17:36:11 +00:00
Keiven C 3d62cc7fd9
fix: avoid Prometheus collisions via multi-registry scrape (#5678) 
Signed-off-by: Keiven Chang <keivenchang@users.noreply.github.com>
Co-authored-by: Keiven Chang <keivenchang@users.noreply.github.com>
 2026-01-28 09:30:01 -08:00
Graham King 90b0759432
chore: Fix tests logging setup (#5693) 
Signed-off-by: Graham King <grahamk@nvidia.com>
 2026-01-27 22:06:34 +00:00
Yan Ru Pei 382a26f0ab
chore: type ZmqMessage (#5689) 
Signed-off-by: PeaBrane <yanrpei@gmail.com>
 2026-01-27 21:21:23 +00:00
Biswa Panda bc76247dff
fix: add recv timeout for zmq transport (#5685) 2026-01-27 18:55:50 +00:00
Biswa Panda 8fc70f20f3
feat: add ZMQ transport and its discovery (#5625) 2026-01-27 02:31:04 +00:00
Pavithra Vijayakrishnan 66c369964a
chore: version update for 0.9.0 (#5661) 
Signed-off-by: pvijayakrish <pvijayakrish@nvidia.com>
 2026-01-26 17:18:05 -08:00
Qi Wang 2a2ad756ff
test: unit test TCP client handle_reader [2/n] (#5056) 2026-01-26 05:01:49 +00:00
Biswa Panda ebde0f662c
feat: event plane apis and nats implementation (#5624) 2026-01-24 02:47:00 +00:00
Biswa Panda 3ee989252c
feat: add event plane discovery (#5614) 2026-01-23 13:59:43 -08:00
Yan Ru Pei feb6d272ac
chore: nuke ForwardPassMetrics (#5531) 
Signed-off-by: PeaBrane <yanrpei@gmail.com>
 2026-01-23 16:26:01 +00:00
Biswa Panda e2b12517c6
chore: reduce verbosity during dynamo start (#5518) 2026-01-21 19:11:07 +00:00
Janelle Cai ca023c0c1f
fix: restrict pod hash for worker/instance id to <2^53 (#5471) 2026-01-21 06:32:01 +00:00
ishandhanani 9b8b35a887
fix: ipv6 support for sglang disaggregation (#5521) 
Co-authored-by: Claude Opus 4.5 <noreply@anthropic.com>
 2026-01-20 23:55:55 +00:00
Schwinn Saereesitthipitak d953f9d038
feat: add endpoint instance registration/unregistration for sleep/wake (#5203) 
Signed-off-by: Schwinn Saereesitthipitak <schwinns@nvidia.com>
Co-authored-by: Claude Opus 4.5 <noreply@anthropic.com>
 2026-01-20 12:17:58 +00:00
Neelay Shah 06939770b3
feat: add span event logging and fix trace context extraction (#5400) 
Co-authored-by: Claude Opus 4.5 <noreply@anthropic.com>
 2026-01-16 13:51:37 -08:00
Qi Wang 31189eeafa
fix: fix worker's push_handler non-blocking error upon stop word (#5157) 2026-01-14 11:29:33 -08:00
Biswa Panda bbb79afbfb
fix: use zero copy decoder for handling high concurrency / request bursts for tcp ingress (#5376) 2026-01-13 13:58:36 -08:00
Qi Wang e47457e4e7
test: unit test tcp/client.rs handle_writer [1/n] (#5055) 2026-01-13 17:06:17 +00:00
atchernych 1a8dcacd82
fix: enable K8s discovery in GAIE (#5303) 
Signed-off-by: Anna Tchernych <atchernych@nvidia.com>
 2026-01-12 10:43:55 -06:00
mohammedabdulwahhab 2f9812aad6
fix: fix bug in diffing logic in list_and_watch (#5318) 
Signed-off-by: mohammedabdulwahhab <furkhan324@berkeley.edu>
 2026-01-10 20:04:53 +00:00
jthomson04 b39aa54fdf
perf: Fix prefill router round robin (#5313) 
Signed-off-by: jthomson04 <jwillthomson19@gmail.com>
 2026-01-09 18:02:10 +00:00
Biswa Panda 0d3ff44000
fix: enable toggling kv events pub/sub (currently nats based) with --no-kv-events flag (#5237) 2026-01-09 06:36:51 +00:00
jh-nv ace35a8ebc
fix: distributed tracing propagation for TCP transport (#5283) 
Co-authored-by: Ishan Dhanani <ishandhanani@gmail.com>
 2026-01-08 19:12:24 -05:00
mohammedabdulwahhab f56483cbac
fix: prevent deadlock in etcd when flushing initial keys (#5091) 
Signed-off-by: mohammedabdulwahhab <furkhan324@berkeley.edu>
 2026-01-06 01:38:37 +00:00
Yan Ru Pei 0980b27f06
feat: mockers with bootstrap optimization (sglang testing) + CI test (#5121) 
Signed-off-by: PeaBrane <yanrpei@gmail.com>
 2026-01-02 16:05:04 -08:00
Tushar Sharma cf433e6825
chore: update all copyright headers in repo to 2026 (#5130) 
Signed-off-by: Tushar Sharma <tusharma@nvidia.com>
 2026-01-02 22:08:23 +00:00
Yan Ru Pei 0b33c1dfa0
fix: sglang disagg routing fixes and optimizations [DYN-1692] (#5106) 
Signed-off-by: PeaBrane <yanrpei@gmail.com>
Co-authored-by: Ishan Dhanani <ishandhanani@gmail.com>
Co-authored-by: Sean SH Choi <sechoi@nvidia.com>
Co-authored-by: ishandhanani <82981111+ishandhanani@users.noreply.github.com>
 2025-12-31 10:55:44 -08:00
Patrick 096699c4b6
feat: KVBM Object Support - add runtime vars for object (#5063) 
Signed-off-by: Patrick Riel <priel@nvidia.com>
 2025-12-23 16:34:36 -08:00
Thomas Montfort 3057af00b6
feat(dep-715): refactor helm directory and remove reference to cloud (#5042) 2025-12-23 12:08:19 -05:00
Jacky 88abc78758
fix: Client reconcile available instances from instance source periodically (#5043) 
Signed-off-by: Jacky <18255193+kthui@users.noreply.github.com>
 2025-12-22 10:42:44 -08:00
Jacky e6a6a1f244
feat: Request Migration Metrics (#5029) 
Signed-off-by: Jacky <18255193+kthui@users.noreply.github.com>
 2025-12-19 12:22:36 -08:00
Anant Sharma 7436e3bc03
chore: update versions for 0.8.0 release (#5031) 
Signed-off-by: Anant Sharma <anants@nvidia.com>
 2025-12-19 10:21:52 -08:00
mohammedabdulwahhab 4106b90f56
fix: support dynamic metadata updates in Kubernetes backed discovery (#4988) 
Signed-off-by: mohammedabdulwahhab <furkhan324@berkeley.edu>
 2025-12-19 08:33:14 -08:00
Biswa Panda 18b64e90b6
feat: switch to tcp as default request plane (#4845) 2025-12-18 23:05:38 +00:00
jh-nv b401a1d66f
feat: propagate trace context to vLLM engine (#4918) 
Signed-off-by: Jie Hao <jihao@nvidia.com>
 2025-12-18 16:50:29 -05:00