mooncake-track
/
dynamo
0
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity
3,557 Commits 2 Branches 0 Tags 76 MiB
Commit Graph

68 Commits

Author SHA1 Message Date
ishandhanani b6603d90d6
feat: add Anthropic Messages API endpoint (/v1/messages) (#6231) 
Signed-off-by: Matej Kosec <mkosec@nvidia.com>
Signed-off-by: Marko Kosec <mkosec@nvidia.com>
Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
Co-authored-by: Matej Kosec <mkosec@nvidia.com>
 2026-02-18 06:11:52 +00:00
Alec d86937f955
test: relocate test output to /tmp to keep git working tree clean (#6289) 
Co-authored-by: Claude Sonnet 4.5 <noreply@anthropic.com>
 2026-02-17 21:21:01 -08:00
MatejKosec 8cb47d04d2
feat: responses API compliance with upstream type alignment (#6089) 
Signed-off-by: Matej Kosec <mkosec@nvidia.com>
Co-authored-by: Ishan Dhanani <ishandhanani@gmail.com>
 2026-02-12 01:18:28 +00:00
Keiven C e18840cef3
feat: add Prometheus auto and custom label injection for engine metrics (#5989) 
Signed-off-by: Keiven Chang <keivenchang@users.noreply.github.com>
 2026-02-10 21:22:53 -08:00
Keiven C 027d2653a5
feat: expose Python Prometheus metric via DynamoComponentMetrics (#5817) 
Signed-off-by: Keiven Chang <keivenchang@users.noreply.github.com>
 2026-02-09 17:10:37 -08:00
Tushar Sharma 6401e34daf
ci: Transition deploy tests to pytest framework (#5874) 
Signed-off-by: Tushar Sharma <tusharma@nvidia.com>
 2026-02-06 21:09:34 +00:00
Jacky 1ffa489ea1
refactor: Move --migration-limit flag from backend to frontend (#5918) 
Signed-off-by: Jacky <18255193+kthui@users.noreply.github.com>
 2026-02-06 20:50:55 +00:00
Keiven C e55ebec562
chore: rename terminate_existing to terminate_all_matching_processes (#5923) 
Signed-off-by: Keiven Chang <keivenchang@users.noreply.github.com>
Co-authored-by: Keiven Chang <keivenchang@users.noreply.github.com>
 2026-02-04 09:50:41 -08:00
jh-nv aea57caeb5
fix: fix test process termination logic to properly trigger graceful shutdown (#5775) 2026-01-31 02:51:53 +00:00
Yan Ru Pei 584020f4b3
chore: consistently used the arc shared workers configs + CI flake fixes (#5707) 
Signed-off-by: PeaBrane <yanrpei@gmail.com>
 2026-01-29 09:29:24 +00:00
Keiven C e5557803ef
fix: correct SGLang metrics prefix from sglang_ to sglang: (#5701) 
Signed-off-by: Keiven Chang <keivenchang@users.noreply.github.com>
Co-authored-by: Keiven Chang <keivenchang@users.noreply.github.com>
 2026-01-27 17:22:10 -08:00
Yan Ru Pei feb6d272ac
chore: nuke ForwardPassMetrics (#5531) 
Signed-off-by: PeaBrane <yanrpei@gmail.com>
 2026-01-23 16:26:01 +00:00
Biswa Panda 0d3ff44000
fix: enable toggling kv events pub/sub (currently nats based) with --no-kv-events flag (#5237) 2026-01-09 06:36:51 +00:00
ishandhanani 007c5b60a1
fix: sglang metrics and prefill router fix (#5147) 2026-01-04 20:46:45 +00:00
Tushar Sharma cf433e6825
chore: update all copyright headers in repo to 2026 (#5130) 
Signed-off-by: Tushar Sharma <tusharma@nvidia.com>
 2026-01-02 22:08:23 +00:00
Keiven C 2804479989
refactor: frontend/test_vllm.py to use shared dynamic ports (#5013) 
Signed-off-by: Keiven Chang <keivenchang@users.noreply.github.com>
Co-authored-by: Keiven Chang <keivenchang@users.noreply.github.com>
 2025-12-19 12:39:15 -08:00
Keiven C c22280ccf4
refactor: frontend/grpc tests to use dynamic ports (#4992) 
Signed-off-by: Keiven Chang <keivenchang@users.noreply.github.com>
Co-authored-by: Keiven Chang <keivenchang@users.noreply.github.com>
 2025-12-18 16:21:28 -08:00
Keiven C f10f59426d
refactor: consolidate fault_tolerance DynamoFrontendProcess, add timeouts (#5008) 
Signed-off-by: Keiven Chang <keivenchang@users.noreply.github.com>
Co-authored-by: Keiven Chang <keivenchang@users.noreply.github.com>
 2025-12-17 21:25:23 -08:00
Keiven C 1b8869f5e7
refactor: router tests to be pytest-xdist parallel safe (#5005) 
Signed-off-by: Keiven Chang <keivenchang@users.noreply.github.com>
Co-authored-by: Keiven Chang <keivenchang@users.noreply.github.com>
 2025-12-17 18:33:47 -08:00
Keiven C 6da23fcf79
test: make serve tests xdist-safe with dynamic ports (#4951) 
Signed-off-by: Keiven Chang <keivenchang@users.noreply.github.com>
Co-authored-by: Keiven Chang <keivenchang@users.noreply.github.com>
 2025-12-16 09:15:24 -08:00
Biswa Panda 3c2b72b016
fix: [lora] refactor test and clean up examples (#4884) 2025-12-11 10:05:00 -08:00
Keiven C c6b440e498
test: add dynamic port allocation for fault_tolerant test execution (#4835) 
Signed-off-by: Keiven Chang <keivenchang@users.noreply.github.com>
Co-authored-by: Keiven Chang <keivenchang@users.noreply.github.com>
 2025-12-10 23:11:09 -08:00
Biswa Panda acab636702
test: add ci tests for lora (agg and router) (#4817) 2025-12-10 11:35:43 -08:00
Aryan Bagade f57fd72f88
feat: Add logprobs support to vLLM backend (#4683) (#4697) 
Signed-off-by: Aryan Bagade <aryan@aryanbagade.com>
Signed-off-by: Aryan Bagade <73382554+AryanBagade@users.noreply.github.com>
Co-authored-by: Ryan McCormick <rmccormick@nvidia.com>
 2025-12-10 04:28:33 +00:00
Elijah Soba 01bfbea15f
feat: Add logprobs support to TRTLLM backend (#4759) 
Signed-off-by: Elijah Soba <esoba@nvidia.com>
 2025-12-09 21:40:04 +00:00
Thomas Montfort a473402fe0
feat: fault tolerance rolling upgrade test scenarios (#4558) 2025-12-09 10:04:10 -08:00
Vladislav Nosivskoy 3e4b480b95
fix: guided decoding params handling in vLLM (#4770) 
Signed-off-by: Vladislav Nosivskoy <vladnosiv@gmail.com>
Co-authored-by: Karen Chung <karenc@nvidia.com>
 2025-12-09 01:10:35 +00:00
Keiven C 2472aa4ac1
test: add timeout markers to pytest.mark.gpu_1 serve tests (#4768) 
Co-authored-by: Keiven Chang <keivenchang@users.noreply.github.com>
 2025-12-05 17:10:53 -08:00
Keiven C 00b64ae001
refactor: remove legacy NATS metrics and stats_handler (#4680) 
Signed-off-by: Keiven Chang <keivenchang@users.noreply.github.com>
 2025-12-04 09:59:39 -08:00
Indrajit Bhosale 0651a4fe44
test: Multimodal Tool Calling vLLM Test (#4663) 
Signed-off-by: Indrajit Bhosale <iamindrajitb@gmail.com>
Co-authored-by: Ryan McCormick <rmccormick@nvidia.com>
 2025-12-03 16:54:11 -08:00
KrishnanPrash e7544f1967
ci: Testing b64 multimodal support with real image (#4280) 
Signed-off-by: Krishnan Prashanth <kprashanth@nvidia.com>
Signed-off-by: KrishnanPrash <140860868+KrishnanPrash@users.noreply.github.com>
Co-authored-by: Harrison Saturley-Hall <hsaturleyhal@nvidia.com>
 2025-11-21 01:43:36 +00:00
Keiven C 75bf7c9b0f
fix: expose LMCache metrics via Dynamo metrics endpoint (#4461) 
Signed-off-by: Keiven Chang <keivenchang@users.noreply.github.com>
 2025-11-19 21:09:13 -08:00
Indrajit Bhosale a8e5328ed8
ci: TRT-LLM multimodal CI (#4118) 
Signed-off-by: Indrajit Bhosale <iamindrajitb@gmail.com>
 2025-11-19 09:48:17 -08:00
Keiven C 381c428c11
refactor: change TensorRT-LLM metrics prefix from trtllm: to trtllm_ (#4269) 
Signed-off-by: Keiven Chang <keivenchang@users.noreply.github.com>
Co-authored-by: Keiven Chang <keivenchang@users.noreply.github.com>
 2025-11-13 09:44:37 -08:00
Keiven C 6bccf099f8
feat: deprecate DYN_SYSTEM_ENABLED in favor of DYN_SYSTEM_PORT (#4082) 
Signed-off-by: Keiven Chang <keivenchang@users.noreply.github.com>
Co-authored-by: Keiven Chang <keivenchang@users.noreply.github.com>
 2025-11-07 15:31:51 -08:00
Indrajit Bhosale d1cf3c2cf6
ci: Add Fault Tolerance K8s test (#3801) 
Signed-off-by: Indrajit Bhosale <iamindrajitb@gmail.com>
 2025-11-07 15:06:02 -08:00
Keiven C 49ecfe601f
fix: add prefill metrics support for SGLang disaggregated mode (#3992) 
Signed-off-by: Keiven Chang <keivenchang@users.noreply.github.com>
 2025-11-04 11:17:54 -08:00
Anant Sharma 8bd37c96d6
refactor: move backend deploy, launch and slurm files from components to examples (#3849) 
Signed-off-by: Anant Sharma <anants@nvidia.com>
 2025-10-31 13:09:46 -04:00
Tzu-Ling Kan c4abe9bcf3
feat: Add prompt > seq_len k8 tests. (#3930) 
Signed-off-by: tzulingk@nvidia.com <tzulingk@nvidia.com>
 2025-10-30 21:20:16 -07:00
Kyle McGill 0028cdf43d
feat: Use ForwardPassCallback api from TRTLLM to register end of forward pass callback to enable cuda graphs (#3297) 
Signed-off-by: Kyle McGill <kmcgill@nvidia.com>
 2025-10-27 15:12:04 -07:00
Keiven C 07cf0744fa
refactor: remove hardcoded /workspace paths across tests (#3888) 
Signed-off-by: Keiven Chang <keivenchang@users.noreply.github.com>
 2025-10-27 10:53:08 -07:00
Keiven C 3e015595cc
feat: TensorRT-LLM and SGLang metrics validation (part 2/3 + 3/3) (#3842) 
Signed-off-by: Keiven Chang <keivenchang@users.noreply.github.com>
 2025-10-23 22:15:28 -07:00
Keiven C bdef0ec566
test: add backend-specific metrics validation for E2E tests (part 1/3, vLLM only) (#3804) 
Signed-off-by: Keiven Chang <keivenchang@users.noreply.github.com>
 2025-10-22 12:40:43 -07:00
Keiven C f2ba58e568
feat: add Rust-to-Python const code generator for prometheus_names.py (#3425) 
Signed-off-by: Keiven Chang <keivenchang@users.noreply.github.com>
 2025-10-10 11:20:36 -07:00
Tzu-Ling Kan cdfd70f0fe
feat: changing SIGKILL to SIGINT for graceful shutdown (#3513) 
Signed-off-by: tzulingk@nvidia.com <tzulingk@nvidia.com>
 2025-10-08 19:55:00 -07:00
zhongdaor-nv 89e7dab2ac
chore: add e2e test for reasoning_effort for gpt-oss model (#3421) 
Signed-off-by: zhongdaor <zhongdaor@nvidia.com>
Signed-off-by: zhongdaor-nv <zhongdaor@nvidia.com>
 2025-10-07 11:36:54 -07:00
Kris Hung d809906e62
feat: Add embedding support to sgl backend (#3427) 
Signed-off-by: krishung5 <krish@nvidia.com>
 2025-10-07 00:13:16 +00:00
Indrajit Bhosale 031dc58970
test: TRTLLM K8s FT tests (#3368) 
Signed-off-by: Indrajit Bhosale <iamindrajitb@gmail.com>
 2025-10-02 23:56:47 -07:00
Keiven C cacac9b9f4
feat: add Python const for Prometheus metric names (#3244) 
Signed-off-by: Keiven Chang <keivenchang@users.noreply.github.com>
 2025-09-30 10:28:47 -07:00
Tzu-Ling Kan ec50db0642
feat: Add sglang k8 FT tests (#3227) 
Signed-off-by: tzulingk@nvidia.com <tzulingk@nvidia.com>
 2025-09-26 20:42:31 +00:00