mooncake-track
/
dynamo
0
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity
2,017 Commits 2 Branches 0 Tags 76 MiB
Commit Graph

87 Commits

Author SHA1 Message Date
Harrison Saturley-Hall cd2389baef
chore: pre-0.6.0 activities (#3592) 
Signed-off-by: Harrison Saturley-Hall <hsaturleyhal@nvidia.com>
 2025-10-13 17:45:19 -04:00
mohammedabdulwahhab 1f92dd547e
feat: OTEL Exporter and Tempo Visualization (#3307) 
Signed-off-by: mohammedabdulwahhab <furkhan324@berkeley.edu>
 2025-10-13 18:39:09 +00:00
Richard Huo 5c69c11930
chore: upgrade nixl to 0.6.0 (#3550) 
Signed-off-by: richardhuo-nv <rihuo@nvidia.com>
 2025-10-10 13:18:21 -07:00
Ryan McCormick ef04a0869e
feat: Add OpenAPI routes to serve HTTP server API documentation (#3480) 2025-10-08 13:52:19 -07:00
Graham King 1b1265e6fc
chore: Remove GGUF support (#3488) 
Signed-off-by: Graham King <grahamk@nvidia.com>
 2025-10-08 11:42:24 -04:00
Elyas Mehtabuddin 67e1f6ee11
feat: enable parallel tool calling and add testing (#3188) 
Signed-off-by: Elyas Mehtabuddin <emehtabuddin@nvidia.com>
Co-authored-by: Ryan McCormick <rmccormick@nvidia.com>
 2025-09-29 15:30:42 +00:00
Graham King c03e2f6bbe
chore: Migrate planner virtual_connector internals into bindings (#3205) 
Signed-off-by: Graham King <grahamk@nvidia.com>
 2025-09-25 18:04:58 -04:00
Harrison Saturley-Hall 980727bba1
chore: bump versions ahead of 0.5.1 release (#3209) 
Signed-off-by: Harrison Saturley-Hall <hsaturleyhal@nvidia.com>
 2025-09-24 18:26:38 -04:00
Ryan Olson c63cceaa77
feat: JailedStream (#3034) 
Signed-off-by: ayushag <ayushag@nvidia.com>
 2025-09-23 22:24:17 +00:00
Graham King 3865a94148
feat: Port vllm port allocator to Rust in bindings (#3125) 
Signed-off-by: Graham King <grahamk@nvidia.com>
 2025-09-19 18:34:52 +00:00
Elyas Mehtabuddin 31f5ed3ce7
feat: add finish reason = tool_calls for stream=False and phi-4 detect token start fix (#3087) 
Signed-off-by: Elyas Mehtabuddin <emehtabuddin@nvidia.com>
 2025-09-18 21:45:52 +00:00
Graham King f88d7dc74b
chore(bindings): Remove NatsQueue (#3086) 
Signed-off-by: Graham King <grahamk@nvidia.com>
 2025-09-17 12:46:54 -04:00
Tzu-Ling Kan 08cb08c1bc
feat: Canary Health Check. (#2903) 
Signed-off-by: tzulingk@nvidia.com <tzulingk@nvidia.com>
 2025-09-17 03:31:56 +00:00
Graham King 87e6e0529d
fix: Interactive inputs actually stops, does not ignore stop token (#3057) 
Signed-off-by: Graham King <grahamk@nvidia.com>
 2025-09-16 14:39:05 -04:00
ryan-lempka 435803ea16
chore: add additional param support for multimodal models (#3042) 
Signed-off-by: Ryan Lempka <rlempka@nvidia.com>
 2025-09-16 16:26:39 +00:00
Graham King 6f14e94159
feat: Add a checksum to ModelDeploymentCard fields (#2934) 
Signed-off-by: Graham King <grahamk@nvidia.com>
 2025-09-09 16:12:42 +00:00
Olga Andreeva 27fad26faf
refactor: Split ModelType to ModelInput for request and response type; ModelType for the supported workloads (#2714) 
Signed-off-by: Guan Luo <gluo@nvidia.com>
Signed-off-by: GuanLuo <41310872+GuanLuo@users.noreply.github.com>
Co-authored-by: Guan Luo <gluo@nvidia.com>
Co-authored-by: GuanLuo <41310872+GuanLuo@users.noreply.github.com>
 2025-09-03 22:22:37 +00:00
KrishnanPrash c920cbd9dc
feat: Add --custom-jinja-template argument to pass a custom chat template for vLLM (#2829) 
Signed-off-by: Krishnan Prashanth <kprashanth@nvidia.com>
 2025-09-03 14:22:15 -07:00
Harrison Saturley-Hall 561ecb98a2
chore: bump version numbers ahead of 0.5.0 release (#2812) 
Signed-off-by: Harrison King Saturley-Hall <hsaturleyhal@nvidia.com>
 2025-09-02 16:13:28 -04:00
KavinKrishnan 95ce83d59a
feat: Integrate Model Express Client into Dynamo Model Downloads (#2574) 
Signed-off-by: Kavin Krishnan <kavink@nvidia.com>
Co-authored-by: KavinKrishnan <kavin.krishnan@nvidia.com>
 2025-08-28 16:09:04 -07:00
GuanLuo 91a459c038
feat: KServe gRPC support (#2638) 2025-08-26 22:57:31 -07:00
Yan Ru Pei f08729ae15
feat: python bindings for the entire KvPushRouter + per-request router configs (#2658) 2025-08-25 22:27:10 +00:00
nachiketb-nvidia 3036e60b1e
feat: add gpt oss reasoning parser through harmony (#2656) 
- couple of refactors
- added a new dependency, openai-harmony
- implemented the gpt oss parser
 2025-08-25 17:13:38 +00:00
Ziqi Fan b658ba6139
feat: enable dynamo metrics on KVBM (#2626) 2025-08-22 19:58:05 +00:00
Graham King 49958435eb
chore: Remove async-openai-macros (#2554) 2025-08-20 11:33:10 -07:00
Yan Ru Pei d319abf3a3
feat: upload/download rust structs directly through NATs object store (#2540) 2025-08-20 17:17:21 +00:00
Dmitry Tokarev 9a02188531
chore: Bumped Dynamo version to 0.4.1 (#2545) 2025-08-19 21:55:37 -04:00
Ryan Olson 07cfc3a11b
feat: kvbm + connector (#2258) 
Signed-off-by: Ryan Olson <rolson@nvidia.com>
Co-authored-by: Olga Andreeva <oandreeva@nvidia.com>
Co-authored-by: Ziqi Fan <ziqif@nvidia.com>
Co-authored-by: John Thompson <jothomson@nvidia.com>
Co-authored-by: Richard Huo <rihuo@nvidia.com>
Co-authored-by: Zicheng Ma <zichengm@nvidia.com>
 2025-08-19 12:36:53 -07:00
Ryan Olson a33033b7f6
feat: task scheduler (#2406) 
Signed-off-by: Ryan Olson <ryanolson@users.noreply.github.com>
 2025-08-19 12:35:31 -06:00
Graham King a4bbe49228
feat(http): TLS support (#2492) 2025-08-18 16:06:29 -04:00
Keiven C 0444217339
fix: replace metrics callback with background scraping to prevent tim… (#2480) 
Co-authored-by: Keiven Chang <keivenchang@users.noreply.github.com>
 2025-08-18 12:38:50 -07:00
Dan Aloni c12c25787f
fix: upgrade cudarc to 0.17.1 (#2341) 
Signed-off-by: Dan Aloni <dan.aloni@vastdata.com>
Co-authored-by: Tushar Sharma <tusharma@nvidia.com>
 2025-08-13 15:45:08 -04:00
Yan Ru Pei 5166a3dd44
feat: Router replicas with state-sharing (#2264) 2025-08-07 22:57:57 +00:00
Neelay Shah bd4fe1a7d1
feat: cross process instrumentation (#2243) 
Signed-off-by: Neelay Shah <neelays@nvidia.com>
Co-authored-by: coderabbitai[bot] <136622811+coderabbitai[bot]@users.noreply.github.com>
Co-authored-by: Olga Andreeva <124622579+oandreeva-nv@users.noreply.github.com>
 2025-08-07 15:32:48 +00:00
Graham King dbe48a1d2b
chore: Bump mistral.rs, llama.cpp and tokenizers deps (#2338) 2025-08-06 16:23:54 -04:00
Dan Aloni b2aa504b47
fix: upgrade axum to 0.8 and etcd-client to 0.16 (#2317) 
Signed-off-by: Dan Aloni <dan.aloni@vastdata.com>
 2025-08-06 18:25:49 +00:00
Anant Sharma 625578c3bb
chore: update nixl version to 0.4.1 (#2221) 2025-07-31 17:46:21 -04:00
Dmitry Tokarev 4c90b1b924
chore: Version bump to 0.4.0 (#2179) 2025-07-30 02:07:31 +00:00
Paul Hendricks 66b7d2c75d
fix: updates versions and adds ahashmap to BPE (#2072) 2025-07-23 20:27:18 +00:00
Graham King 182d3b5dc7
chore(bindings): Remove mistralrs / llama.cpp (#1970) 2025-07-16 16:12:40 -04:00
Graham King aba60996f2
perf(router): Remove lock from router hot path (#1963) 2025-07-16 12:26:32 -04:00
Anant Sharma d975761b8b
chore: update nixl to 0.4.0 release (#1860) (#1886) 2025-07-11 12:45:44 -04:00
Tushar Sharma 1704b126f3
build: Revert "chore: update nixl to 0.4.0 release" (#1880) 2025-07-10 23:30:28 +00:00
Anant Sharma 5fa4cddab9
chore: update nixl to 0.4.0 release (#1860) 2025-07-10 17:08:34 -04:00
ZichengMa ece76a6201
feat: Build DistributedRuntime-level HTTP server with /health /metrics (#1656) 2025-07-08 22:49:28 +00:00
Graham King 2bf27924a1
feat(python): Python bindings for the Dynamo CLI tools (#1799) 2025-07-08 21:40:36 +00:00
Anant Sharma c4935b3497
chore: update versions for 0.3.2 release (#1793) 2025-07-07 17:21:02 -04:00
Anant Sharma a9241b61f0
chore: update nixl to latest 0.3.1 commit (#1762) 2025-07-03 15:31:47 -07:00
Graham King 4ab476174a
chore(engines): Upgrade mistralrs to 0.6.0 (#1767) 2025-07-03 12:45:58 -07:00
Graham King 92f06b0e7f
chore(dynamo-run): Refactor to library (#1687) 
Move much of what was in the `dynamo-run` crate into `dynamo-llm` so that everyone can use it.

Example usage:

1. Create a `LocalModel`:

```
    let local_model = LocalModelBuilder::default()
	.model_path("Qwen/Qwen3-0.6B")
	.http_port(8080)
	.build().await?;
```

2. Make an engine:

```
    let engine_config = EngineConfig::StaticFull {
	engine: dynamo_engine_mistralrs::make_engine(&local_model).await?,
	model: Box::new(local_model),
    };
```

3. Connect it to an input and run it

```
    dynamo_llm::entrypoint::input::run_input(Input::Http, runtime, engine_config).await?;
```

For https://github.com/ai-dynamo/dynamo/issues/1647

Code Rabbit summary, thanks:
  * Introduced a flexible builder pattern for local model configuration, allowing advanced customization and easier initialization.
  * Added new input modes and unified input handling, supporting interactive chat, HTTP server, batch file, and distributed endpoint modes.
  * Centralized engine configuration and routing, enabling more extensible and maintainable engine management.
  * Simplified and modularized the codebase by moving input and engine logic into dedicated modules.
  * Replaced direct model construction with an asynchronous builder for improved clarity and extensibility.
  * Streamlined configuration and validation for flags and router settings.
  * Added validation to prevent incompatible input and output combinations in endpoint and dynamic modes.
 2025-06-30 21:06:24 +00:00