abc02c689f
fix: llm/mocker: Remove the llm -> mocker crate dependency, move config ( #6998 )
...
Signed-off-by: Graham King <grahamk@nvidia.com>
Signed-off-by: PeaBrane <yanrpei@gmail.com>
2026-03-06 09:41:39 -08:00
90d7463721
fix: properly setup and register vLLM worker for external / hybrid load balancing. Update launch script ( #6695 )
...
Signed-off-by: Guan Luo <41310872+GuanLuo@users.noreply.github.com>
2026-03-03 14:00:52 -08:00
cf433e6825
chore: update all copyright headers in repo to 2026 ( #5130 )
...
Signed-off-by: Tushar Sharma <tusharma@nvidia.com>
2026-01-02 22:08:23 +00:00
1c39e4835c
feat: add frontend based prefill request routing for sglang ( #4635 )
...
Signed-off-by: Sean Choi <sechoi@nvidia.com>
Signed-off-by: PeaBrane <yanrpei@gmail.com>
Co-authored-by: Yan Ru Pei <yanrpei@gmail.com>
Co-authored-by: ishandhanani <82981111+ishandhanani@users.noreply.github.com>
2025-12-17 09:51:46 -08:00
332499458b
feat: worker-local KvIndexer in KvEventPublisher ( #4519 )
...
Co-authored-by: Yan Ru Pei <yanrpei@gmail.com>
2025-12-11 23:18:34 +00:00
f978f4d1d6
feat: dp rank routing ( #3597 )
...
Signed-off-by: PeaBrane <yanrpei@gmail.com>
2025-10-16 01:48:27 +00:00
6ba64c31f5
feat: tensor type for generic inference. ( #2746 )
...
Signed-off-by: Guan Luo <gluo@nvidia.com>
Signed-off-by: GuanLuo <41310872+GuanLuo@users.noreply.github.com>
Co-authored-by: Olga Andreeva <124622579+oandreeva-nv@users.noreply.github.com>
Co-authored-by: Ryan McCormick <rmccormick@nvidia.com>
2025-09-24 07:31:09 +00:00
cbe854fc5f
feat: [vLLM] implement cli args for tool and reasoning parsers ( #2619 )
2025-08-22 20:29:18 +00:00
d0a6363584
feat: add RuntimeConfig to ModelEntry ( #2311 )
...
Co-authored-by: Yan Ru Pei <yanrpei@gmail.com>
2025-08-14 10:19:53 -07:00
Graham King