mooncake-track
/
dynamo
0
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity
3,557 Commits 2 Branches 0 Tags 76 MiB
Commit Graph

30 Commits

Author SHA1 Message Date
Alec 6d3b92f04e
feat: remove --connector flag for vLLM backend (LLM-90) (#6450) 
Signed-off-by: alec-flowers <aflowers@nvidia.com>
Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
 2026-02-24 17:49:00 +00:00
Tzu-Ling Kan 0ce3461a9e
feat: Add runtime.endpoint() method to eliminate namespace chaining (#6386) 
Signed-off-by: tzulingk@nvidia.com <tzulingk@nvidia.com>
 2026-02-19 16:02:37 -05:00
Tzu-Ling Kan 5624d14481
Rename fetch_llm to fetch_model (#6268) 
Signed-off-by: tzulingk@nvidia.com <tzulingk@nvidia.com>
 2026-02-13 23:22:35 +00:00
mohammedabdulwahhab a289695c37
fix: consolidate dyn_discovery_backend and dyn_kv_store (#6167) 
Signed-off-by: mohammedabdulwahhab <furkhan324@berkeley.edu>
 2026-02-13 19:07:11 +00:00
Keiven C e55ebec562
chore: rename terminate_existing to terminate_all_matching_processes (#5923) 
Signed-off-by: Keiven Chang <keivenchang@users.noreply.github.com>
Co-authored-by: Keiven Chang <keivenchang@users.noreply.github.com>
 2026-02-04 09:50:41 -08:00
Graham King e076f3a265
test: Auto-start services for prompt_embed_tests.py (#5491) 
Signed-off-by: Graham King <grahamk@nvidia.com>
 2026-01-20 09:16:52 -05:00
GuanLuo bb8eaa2305
fix: KServe propagate error to client in stream infer (#5263) 
Signed-off-by: Guan Luo <gluo@nvidia.com>
Signed-off-by: Guan Luo <41310872+GuanLuo@users.noreply.github.com>
 2026-01-08 22:31:32 -08:00
Wenqi Glantz 6f9619a210
feat(vllm): Add prompt embeds support for pre-computed inference inputs (#4739) 
Signed-off-by: Wenqi Glantz <wglantz@nvidia.com>
 2026-01-08 20:03:10 +00:00
Tushar Sharma cf433e6825
chore: update all copyright headers in repo to 2026 (#5130) 
Signed-off-by: Tushar Sharma <tusharma@nvidia.com>
 2026-01-02 22:08:23 +00:00
Keiven C 2804479989
refactor: frontend/test_vllm.py to use shared dynamic ports (#5013) 
Signed-off-by: Keiven Chang <keivenchang@users.noreply.github.com>
Co-authored-by: Keiven Chang <keivenchang@users.noreply.github.com>
 2025-12-19 12:39:15 -08:00
Keiven C c22280ccf4
refactor: frontend/grpc tests to use dynamic ports (#4992) 
Signed-off-by: Keiven Chang <keivenchang@users.noreply.github.com>
Co-authored-by: Keiven Chang <keivenchang@users.noreply.github.com>
 2025-12-18 16:21:28 -08:00
Yan Ru Pei d0e95c39df
test: remove pytest_runtestloop (#4886) 
Signed-off-by: PeaBrane <yanrpei@gmail.com>
 2025-12-11 21:29:51 +00:00
Neal Vaidya 118323f26e
fix: skip HuggingFace download for non-llms (#4686) 
Signed-off-by: Neal Vaidya <nealv@nvidia.com>
 2025-12-04 10:23:24 -08:00
GuanLuo d90dc39d4a
feat: add Kserve readiness endpoint support: ServerLive / ServerReady / ModelReady (#4708) 
Signed-off-by: Guan Luo <gluo@nvidia.com>
Signed-off-by: GuanLuo <41310872+GuanLuo@users.noreply.github.com>
Co-authored-by: Ryan McCormick <rmccormick@nvidia.com>
 2025-12-03 11:17:05 -08:00
Yan Ru Pei 7e499b5c46
test: bring back the framework 1 gpu pre-merge tests + clean up pytest markers (#4698) 
Signed-off-by: PeaBrane <yanrpei@gmail.com>
 2025-12-02 17:12:10 -08:00
Pavithra Vijayakrishnan 0f6dca6e70
test: Add pytest markers (#4111) 
Signed-off-by: pvijayakrish <pvijayakrish@nvidia.com>
Signed-off-by: Pavithra Vijayakrishnan <160681768+pvijayakrish@users.noreply.github.com>
 2025-12-01 09:53:20 -08:00
Karen Chung d461631b46
test: Router CI tests with vLLM engine (#3948) 
Signed-off-by: PeaBrane <yanrpei@gmail.com>
Co-authored-by: PeaBrane <yanrpei@gmail.com>
 2025-11-20 02:16:40 +00:00
Graham King 69797b5ab3
feat: Only monitor NATS metrics if using NATS request plane (#4442) 
Signed-off-by: Graham King <grahamk@nvidia.com>
 2025-11-19 17:48:49 +00:00
Graham King 58da7cfe36
test(key_value_store): Assert entries() works correctly (#4312) 
Signed-off-by: Graham King <grahamk@nvidia.com>
 2025-11-13 18:44:36 -05:00
Graham King e1af3af6ee
chore: Remove static mode (#4235) 
Signed-off-by: Graham King <grahamk@nvidia.com>
 2025-11-11 19:25:12 +00:00
zhongdaor-nv 93ada89909
feat: enable HTTP completion endpoint to accept arrays of prompts and generate multiple completions per prompt (#3953) 
Signed-off-by: zhongdaor <zhongdaor@nvidia.com>
 2025-11-07 15:33:44 -08:00
Keiven C 6bccf099f8
feat: deprecate DYN_SYSTEM_ENABLED in favor of DYN_SYSTEM_PORT (#4082) 
Signed-off-by: Keiven Chang <keivenchang@users.noreply.github.com>
Co-authored-by: Keiven Chang <keivenchang@users.noreply.github.com>
 2025-11-07 15:31:51 -08:00
GuanLuo eb3a486d01
feat: allow Triton model config specification in TensorModelConfig (#3874) 
Signed-off-by: Guan Luo <41310872+GuanLuo@users.noreply.github.com>
Signed-off-by: GuanLuo <41310872+GuanLuo@users.noreply.github.com>
 2025-10-30 11:19:20 -07:00
Richard Huo 0d07e2c365
test: fix frontend tests to make sure openai/gpt-oss-20b can be loaded on A100 (#3889) 2025-10-27 13:23:18 -07:00
KrishnanPrash 90caf3ea55
feat: Extend TensorRequest and TensorResponse to contains extra parameters (#3761) 
Signed-off-by: Krishnan Prashanth <kprashanth@nvidia.com>
Signed-off-by: KrishnanPrash <140860868+KrishnanPrash@users.noreply.github.com>
Co-authored-by: Ryan McCormick <rmccormick@nvidia.com>
 2025-10-24 10:22:10 -07:00
GuanLuo d68e4b8adf
fix: enhance gRPC frontend to return output in raw content field for Triton client compatibility (#3600) 
Signed-off-by: Guan Luo <gluo@nvidia.com>
Signed-off-by: Guan Luo <41310872+GuanLuo@users.noreply.github.com>
 2025-10-22 04:23:48 +00:00
zhongdaor-nv ea07d51fc0
test: add tool calling and reasoning tests for frontend on GPT-OSS (#3636) 
Signed-off-by: zhongdaor <zhongdaor@nvidia.com>
Signed-off-by: zhongdaor-nv <zhongdaor@nvidia.com>
Co-authored-by: coderabbitai[bot] <136622811+coderabbitai[bot]@users.noreply.github.com>
 2025-10-15 21:30:54 -07:00
zhongdaor-nv 3dc2512973
fix: Add support for single element arrays for chat and completions prompts (#3482) 
Signed-off-by: zhongdaor <zhongdaor@nvidia.com>
 2025-10-14 13:03:55 -07:00
Alec 90dc758938
chore: bump vllm to 0.11.0 (#3422) 
Signed-off-by: alec-flowers <aflowers@nvidia.com>
 2025-10-12 14:26:45 -07:00
zhongdaor-nv 89e7dab2ac
chore: add e2e test for reasoning_effort for gpt-oss model (#3421) 
Signed-off-by: zhongdaor <zhongdaor@nvidia.com>
Signed-off-by: zhongdaor-nv <zhongdaor@nvidia.com>
 2025-10-07 11:36:54 -07:00