mooncake-track
/
dynamo
0
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity
3,557 Commits 2 Branches 0 Tags 76 MiB
Commit Graph

62 Commits

Author SHA1 Message Date
daiyaanarfeen 651ef5b506
feat: throughput-metrics-source for SLA planner + GlobalPlanner disagg scaling (#6500) 
Signed-off-by: Daiyaan <darfeen@nvidia.com>
Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com>
 2026-02-25 22:58:24 +00:00
Alec 7893f2684e
feat: add --disaggregation-mode enum to vLLM backend (#6483) 
Signed-off-by: alec-flowers <aflowers@nvidia.com>
Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
 2026-02-23 19:34:30 -07:00
Dmitry Tokarev 121d805020
fix: Fix async pytests - added missing marker (#6439) 
Signed-off-by: Dmitry Tokarev <dtokarev@nvidia.com>
 2026-02-19 19:06:13 -05:00
Tzu-Ling Kan 0ce3461a9e
feat: Add runtime.endpoint() method to eliminate namespace chaining (#6386) 
Signed-off-by: tzulingk@nvidia.com <tzulingk@nvidia.com>
 2026-02-19 16:02:37 -05:00
jh-nv 44a76f96b3
refactor: update frontend kv-router flags to be consistent with router (#6361) 2026-02-19 00:27:15 +00:00
Hongkuan Zhou b2075619c9
feat: planner argparse CLI -> config file (#6356) 
Signed-off-by: hongkuanz <hongkuanz@nvidia.com>
 2026-02-18 23:17:36 +00:00
daiyaanarfeen 0a26665303
feat: add GlobalPlanner component for centralized scaling (#5702) 
Signed-off-by: Daiyaan <darfeen@nvidia.com>
Signed-off-by: daiyaanarfeen <darfeen@nvidia.com>
Co-authored-by: Claude Sonnet 4.5 <noreply@anthropic.com>
 2026-02-17 23:12:10 +00:00
Hongkuan Zhou 359765d354
feat: load-based scaling in SLA Planner (#6145) 
Signed-off-by: hongkuanz <hongkuanz@nvidia.com>
 2026-02-14 09:43:02 -08:00
Jonathan Tong 39d645e586
docs: migrate Fern docs from fern/ into docs/ (#6206) 
Signed-off-by: Jont828 <jt572@cornell.edu>
 2026-02-11 16:22:27 -08:00
MatejKosec 45bc1b798c
fix: use tempfile.TemporaryDirectory in load generator test (#6196) 
Signed-off-by: Matej Kosec <mkosec@nvidia.com>
 2026-02-11 16:15:03 -05:00
Dmitry Tokarev 4220771fba
fix: Cleanup pytest markers, enable gpu_0 tests on trtllm arm, reduce log noise (#6124) 
Signed-off-by: Dmitry Tokarev <dtokarev@nvidia.com>
 2026-02-11 16:00:58 -05:00
Keiven C e18840cef3
feat: add Prometheus auto and custom label injection for engine metrics (#5989) 
Signed-off-by: Keiven Chang <keivenchang@users.noreply.github.com>
 2026-02-10 21:22:53 -08:00
MatejKosec 4ad739dd32
fix: prevent aiperf pipe hang in planner scaling test (#6099) 
Replace PIPE-based stdout/stderr capture with direct file output in LoadGenerator.generate_load() to prevent orphaned aiperf child processes from blocking communicate() indefinitely
Add start_new_session=True so os.killpg() can kill the entire process tree on timeout (not just the main process)
Add unit test validating process-group kill on timeout
Fixes DYN-2086
 2026-02-10 18:49:15 +00:00
dagil-nvidia b19de4ed77
docs: cleanup of docs refactor for components, integrations, and features (#6019) 
Signed-off-by: Dan Gil <dagil@nvidia.com>
Co-authored-by: Cursor <cursoragent@cursor.com>
 2026-02-05 19:50:17 -08:00
Thomas Montfort b12e671039
feat: read prefill/decode worker counts from DGD status (#5934) 2026-02-04 14:24:29 -08:00
Thomas Montfort 2176c43193
feat(planner): Derive prefill/decode GPU counts from DGD (#5919) 2026-02-04 01:19:06 +00:00
Hongkuan Zhou e2f1e04f1c
refactor: separate planner into prefill/decode planner (#5622) 
Signed-off-by: hongkuanz <hongkuanz@nvidia.com>
 2026-01-28 16:08:06 -08:00
Hongkuan Zhou 4557b6df5e
feat: predict log1p(y) instead of y in load predictor if ARIMA collapses (#5545) 
Signed-off-by: hongkuanz <hongkuanz@nvidia.com>
 2026-01-21 17:43:14 -08:00
hhzhang16 c8770464ab
feat: normalize dynamo namespace computation (#5231) 
Signed-off-by: Hannah Zhang <hannahz@nvidia.com>
 2026-01-12 19:09:43 +00:00
daiyaanarfeen daa1d740e5
revert: remove vLLM backend metrics from planner (#4134) (#5185) 
Signed-off-by: Daiyaan <darfeen@nvidia.com>
 2026-01-07 15:55:39 -08:00
Tushar Sharma cf433e6825
chore: update all copyright headers in repo to 2026 (#5130) 
Signed-off-by: Tushar Sharma <tusharma@nvidia.com>
 2026-01-02 22:08:23 +00:00
Hongkuan Zhou 2fa3627aa6
test: fix missing args in replica calculation test (#5113) 
Signed-off-by: hongkuanz <hongkuanz@nvidia.com>
 2025-12-30 16:32:03 -08:00
Tanmay Verma e1c685b105
fix: Remove debug logging from aiperf benchmarking (#5010) 2025-12-18 02:34:41 +00:00
Julien Mancuso 9573d34ca0
feat: make planner use DGD Scaling Adapters (#4825) 
Signed-off-by: Julien Mancuso <jmancuso@nvidia.com>
 2025-12-09 14:51:40 -07:00
daiyaanarfeen 197e0227a7
feat: migrate planner metrics to backend (1/3 vLLM) (#4134) 
Signed-off-by: Daiyaan <darfeen@nvidia.com>
Co-authored-by: tmontfort <tmontfort@nvidia.com>
 2025-12-09 00:28:35 +00:00
Biswa Panda 9fb5f03a6f
fix: pin sniffio dependency (#4665) 
Signed-off-by: PeaBrane <yanrpei@gmail.com>
Co-authored-by: PeaBrane <yanrpei@gmail.com>
 2025-12-02 06:58:57 +00:00
Pavithra Vijayakrishnan 0f6dca6e70
test: Add pytest markers (#4111) 
Signed-off-by: pvijayakrish <pvijayakrish@nvidia.com>
Signed-off-by: Pavithra Vijayakrishnan <160681768+pvijayakrish@users.noreply.github.com>
 2025-12-01 09:53:20 -08:00
Graham King 886506c12d
feat: Command line flag to set request plane mode: tcp, http or nats (#4365) 
Signed-off-by: Graham King <grahamk@nvidia.com>
 2025-11-17 09:02:36 -08:00
hhzhang16 9f0a3155e2
docs: add instructions in planner tests to remove volumes/volumeMounts (#4335) 
Signed-off-by: Hannah Zhang <hannahz@nvidia.com>
 2025-11-14 09:55:35 -08:00
hhzhang16 bb7d203f67
fix: append namespace to planner pod names in tests (#4273) 
Signed-off-by: Hannah Zhang <hannahz@nvidia.com>
 2025-11-12 22:45:04 +00:00
Graham King e1af3af6ee
chore: Remove static mode (#4235) 
Signed-off-by: Graham King <grahamk@nvidia.com>
 2025-11-11 19:25:12 +00:00
Keiven C 6bccf099f8
feat: deprecate DYN_SYSTEM_ENABLED in favor of DYN_SYSTEM_PORT (#4082) 
Signed-off-by: Keiven Chang <keivenchang@users.noreply.github.com>
Co-authored-by: Keiven Chang <keivenchang@users.noreply.github.com>
 2025-11-07 15:31:51 -08:00
Graham King 794c0a4431
feat(keyvalue): Filesystem backed KeyValueStore (#4138) 
Signed-off-by: Graham King <grahamk@nvidia.com>
 2025-11-07 21:11:33 +00:00
Anant Sharma 8bd37c96d6
refactor: move backend deploy, launch and slurm files from components to examples (#3849) 
Signed-off-by: Anant Sharma <anants@nvidia.com>
 2025-10-31 13:09:46 -04:00
hhzhang16 6a84ffd347
feat: turn profiling k8s jobs into sample DGDR requests (#3864) 
Signed-off-by: Hannah Zhang <hannahz@nvidia.com>
Signed-off-by: hongkuanz <hongkuanz@nvidia.com>
Signed-off-by: Hongkuan Zhou <tedzhouhk@gmail.com>
Co-authored-by: hongkuanz <hongkuanz@nvidia.com>
Co-authored-by: Hongkuan Zhou <tedzhouhk@gmail.com>
 2025-10-27 13:36:31 -07:00
Hongkuan Zhou fa45ff1f8a
fix: revert change to aiperf output format (#3837) 
Signed-off-by: hongkuanz <hongkuanz@nvidia.com>
 2025-10-22 23:57:44 +00:00
Anish c6b5904579
docs: address Harry/VDR feedback + fixing broken links across repository (#3802) 
Signed-off-by: Harry Kim <harry_kim@live.com>
Signed-off-by: athreesh <anish.maddipoti@utexas.edu>
Signed-off-by: akshatha-k <33278067+akshatha-k@users.noreply.github.com>
Signed-off-by: Harrison Saturley-Hall <hsaturleyhal@nvidia.com>
Signed-off-by: Harrison King Saturley-Hall <hsaturleyhal@nvidia.com>
Co-authored-by: Harry Kim <harry_kim@live.com>
Co-authored-by: Claude <noreply@anthropic.com>
Co-authored-by: akshatha-k <33278067+akshatha-k@users.noreply.github.com>
Co-authored-by: Harrison Saturley-Hall <hsaturleyhal@nvidia.com>
 2025-10-22 19:56:24 -04:00
hhzhang16 7b2f95e49f
fix: bug fixes for planner tests (#3821) 
Signed-off-by: Hannah Zhang <hannahz@nvidia.com>
 2025-10-22 15:01:00 -07:00
Hongkuan Zhou abbbc18727
fix: update aiperf commands for trace dataset (#3737) 
Signed-off-by: hongkuanz <hongkuanz@nvidia.com>
 2025-10-20 12:01:33 -07:00
William Arnold ee56782b98
fix: standardize all planner ttft/itl units to float ms and fix docs (#3673) 
Signed-off-by: William Arnold <7565007+Aphoh@users.noreply.github.com>
 2025-10-17 18:33:41 +00:00
Harrison Saturley-Hall 34fc9693f8
fix: git lfs support is erratic with uv so set env var to force it (#3671) 
Signed-off-by: Harrison Saturley-Hall <hsaturleyhal@nvidia.com>
 2025-10-16 18:49:42 -04:00
Anthony Casagrande 333ee9834f
fix: update invalid AIPerf scripts and parsing logic (#3675) 
Signed-off-by: Anthony Casagrande <acasagrande@nvidia.com>
 2025-10-16 13:30:11 -07:00
Harshini Komali 9f310225eb
feat: Replace genai-perf with aiperf (#3533) 
Signed-off-by: lkomali <lkomali@nvidia.com>
 2025-10-16 09:58:36 -07:00
Biswa Panda 552146f23c
fix: update dynamo namespace env var usage (#3477) 2025-10-08 12:24:03 -07:00
Hongkuan Zhou 83e259a7cc
feat: pre-deployment profiling automatically generates and deploys optimized DGD with planner (#3441) 
Signed-off-by: hongkuanz <hongkuanz@nvidia.com>
 2025-10-07 16:52:51 -07:00
Anant Sharma 836d74172f
feat: restructure source code for python packaging (#3201) 
Signed-off-by: Anant Sharma <anants@nvidia.com>
 2025-10-01 13:15:24 -04:00
Thomas Montfort 1a5016b035
feat: add subComponentType in DGD API and uptake in planner (#3200) 
Signed-off-by: tmontfort <tmontfort@nvidia.com>
Signed-off-by: hongkuanz <hongkuanz@nvidia.com>
Co-authored-by: hongkuanz <hongkuanz@nvidia.com>
 2025-09-29 09:40:58 -07:00
Harrison Saturley-Hall 9e8f67ed0f
fix: update the tags for consistency and remove 0.4.1 refs (#3058) 
Signed-off-by: Harrison King Saturley-Hall <hsaturleyhal@nvidia.com>
Signed-off-by: Harrison Saturley-Hall <hsaturleyhal@nvidia.com>
 2025-09-24 14:53:43 -04:00
Thomas Montfort 8bc2191edf
feat: update planner manifests for Grove support (#3203) 2025-09-24 16:53:45 +00:00
Hongkuan Zhou c34f945be1
fix: add prometheus_client to dep instead of transitive dependency (#3083) 
Signed-off-by: hongkuanz <hongkuanz@nvidia.com>
 2025-09-18 15:18:04 -07:00