feat: devcontainer.json to work for vLLM, SGLang, and TensorRT-LLM (#3228)

Signed-off-by: Keiven Chang <keivenchang@users.noreply.github.com>
2025-10-01 10:01:31 -07:00 · 2025-10-01 10:01:31 -07:00 · 4e38d62872
parent c142eacf91
commit 4e38d62872
6 changed files with 401 additions and 13 deletions
--- a/.devcontainer/README.md
+++ b/.devcontainer/README.md
@ -19,6 +19,51 @@ limitations under the License.

 > Warning: Dev Containers (aka `devcontainers`) is an evolving feature and we are not testing in CI. Please submit any problem/feedback using the issues on GitHub.

+## Framework-Specific Devcontainers
+
+This directory contains framework-specific devcontainer configurations generated from a Jinja2 template:
+
+- **`vllm/`** - Development environment for vLLM framework
+- **`sglang/`** - Development environment for SGLang framework
+- **`trtllm/`** - Development environment for TensorRT-LLM framework
+
+### Template System
+
+The devcontainer configurations are generated from:
+- **`devcontainer.json.j2`** - Jinja2 template with framework variables
+- **`gen_devcontainer_json.py`** - Python script to generate configs
+
+To regenerate the framework-specific configurations after making changes:
+```bash
+cd .devcontainer
+python3 gen_devcontainer_json.py
+```
+
+**Important**: Do not edit the generated `devcontainer.json` files directly. They contain auto-generated warnings and will be overwritten. Instead, edit the `devcontainer.json.j2` template and regenerate.
+
+#### Why We Use Templates Instead of a Single devcontainer.json
+
+The Dev Container Extension requires that each `devcontainer.json` file follow a specific directory convention, which results in significant duplication across framework-specific configurations. See https://code.visualstudio.com/remote/advancedcontainers/connect-multiple-containers
+
+```
+📁 project-root
+    📁 .git
+    📁 .devcontainer
+      📁 python-container
+        📄 devcontainer.json
+      📁 node-container
+        📄 devcontainer.json
+    📁 python-src
+        📄 hello.py
+    📁 node-src
+        📄 hello.js
+    📄 docker-compose.yml
+```
+
+Alternative approaches using undocumented methods (e.g., changing devcontainer.json name, custom configurations) were explored but proved unsuccessful. The template system was developed to minimize duplication while maintaining compatibility with the Dev Container Extension's directory requirements.
+
+Until the Microsoft Dev Container Extension adds new functionalities, this remains the recommended approach for managing multiple Dev Container configurations.
+
 ```mermaid
 graph TB
    subgraph "Developer Laptop"
@ -41,7 +86,7 @@ graph TB
            TOOLS["rust-analyzer<br/>cargo<br/>etc."]
        end

-        IMAGE["Docker Image<br/>dynamo:latest-vllm-local-dev"]
+        IMAGE["Docker Image<br/>dynamo:latest-{framework}-local-dev<br/>(vllm/sglang/trtllm)"]

        IMAGE -->|"docker run<br/>as ubuntu user"| CONTAINER
    end
@ -89,24 +134,41 @@ You must have the following path on your host.

 Follow these steps to get your NVIDIA Dynamo development environment up and running:

-### Step 1: Build the Development Container Image
+### Step 0: Build the Development Container Image

-Build `dynamo:latest-vllm-local-dev` from scratch from the source:
+Build the appropriate framework image (e.g., `dynamo:latest-vllm-local-dev`) from scratch from the source:

 ```bash
 # Single command approach (recommended)
-./container/build.sh --framework VLLM --target local-dev
-# Creates both dynamo:latest-vllm and dynamo:latest-vllm-local-dev
+export FRAMEWORK=VLLM         # Note: any of VLLM, SGLANG, TRTLLM can be used
+./container/build.sh --framework $FRAMEWORK --target local-dev

-# Alternatively, you can build a development container then local-dev
-./container/build.sh --framework VLLM
+# Now you've created both dynamo:latest-vllm and dynamo:latest-vllm-local-dev
+```
+
+Alternatively, you can build a development container, then build local-dev:
+
+```bash
+export FRAMEWORK=VLLM
+
+./container/build.sh --framework $FRAMEWORK
 # Now you have a development image dynamo:latest-vllm
-./container/build.sh --dev-image dynamo:latest-vllm --framework VLLM
+
+./container/build.sh --dev-image dynamo:latest-${FRAMEWORK,,}
 # Now you have a local-dev image dynamo:latest-vllm-local-dev
 ```

 The local-dev image will give you local user permissions matching your host user and includes extra developer utilities (debugging tools, text editors, system monitors, etc.).

+### Step 1: Choose Your Framework
+
+Select the appropriate devcontainer based on your framework:
+- Use `vllm/devcontainer.json` for vLLM development
+- Use `sglang/devcontainer.json` for SGLang development
+- Use `trtllm/devcontainer.json` for TensorRT-LLM development
+
+When opening the devcontainer in VS Code/Cursor, navigate to the specific framework directory (e.g., `.devcontainer/vllm/`) and open that devcontainer.json.
+
 ### Step 2: Install Dev Containers Extension

 **For Cursor:**
@ -362,9 +424,10 @@ If you see errors like "container is not running" or "An error occurred setting
   docker images | grep dynamo

   # If missing, build the dev image first, then build local-dev
-   ./container/build.sh --framework vllm
-   ./container/build.sh --dev-image dynamo:latest-vllm --framework vllm
-   # Output: dynamo:latest-vllm-local-dev
+   export FRAMEWORK=VLLM  # Replace with VLLM, SGLANG, or TRTLLM
+   ./container/build.sh --framework $FRAMEWORK
+   ./container/build.sh --dev-image dynamo:latest-${FRAMEWORK,,} --framework $FRAMEWORK
+   # Now you have dynamo:latest-vllm-local-dev
   ```

 2. **Container startup failure:**
--- a/.devcontainer/devcontainer.json.j2
+++ b/.devcontainer/devcontainer.json.j2
@ -0,0 +1,84 @@
+{
+    "$schema": "https://raw.githubusercontent.com/devcontainers/spec/main/schemas/devContainer.schema.json",
+    "copyright": [
+        "SPDX-FileCopyrightText: Copyright (c) {{ current_year }} NVIDIA CORPORATION & AFFILIATES. All rights reserved.",
+        "SPDX-License-Identifier: Apache-2.0"
+    ],
+    "name": "Dynamo {{ framework.upper() }} Dev Container",
+    "remoteUser": "ubuntu", // Matches our container user
+    "updateRemoteUserUID": true, // Updates the UID of the remote user to match the host user, avoids permission errors
+    "image": "dynamo:latest-{{ framework }}-local-dev", // Use the latest {{ framework.upper() }} dev image
+    "runArgs": [
+        "--gpus=all",
+        "--network=host",
+        "--ipc=host",
+        "--cap-add=SYS_PTRACE",
+        "--shm-size=10G",
+        "--ulimit=memlock=-1",
+        "--ulimit=stack=67108864",
+        "--ulimit=nofile=65536:65536"
+    ],
+    "customizations": {
+        "vscode": {
+            "extensions": [
+                "ms-python.python",
+                "ms-python.vscode-pylance",
+                "rust-lang.rust-analyzer"
+            ],
+            "settings": {
+                // Disable automatic copying of .gitconfig to avoid errors
+                "dev.containers.copyGitConfig": false,
+                "terminal.integrated.defaultProfile.linux": "bash",
+                "terminal.integrated.cwd": "/home/ubuntu/dynamo",
+
+                "python.defaultInterpreterPath": "/opt/dynamo/venv/bin/python",
+                "python.linting.enabled": true,
+
+                "rust-analyzer.memoryLimit": 4096, // larger memory limit to reduce latency
+                "rust-analyzer.checkOnSave.command": "clippy",
+                "rust-analyzer.checkOnSave.enable": true,
+                "rust-analyzer.cargo.buildScripts.enable": true,
+                "rust-analyzer.cargo.targetDir": "/home/ubuntu/dynamo/.build/target",
+                "rust-analyzer.procMacro.enable": true,
+                "rust-analyzer.completion.autoimport.enable": true,
+
+                // Enhanced rust-analyzer configuration
+                "rust-analyzer.linkedProjects": [
+                    "Cargo.toml",
+                    "lib/runtime/Cargo.toml",
+                    "lib/llm/Cargo.toml",
+                    "lib/tokens/Cargo.toml",
+                    "lib/bindings/python/Cargo.toml",
+                    "launch/dynamo-run/Cargo.toml"
+                ],
+                "files.trimTrailingWhitespace": true,
+                "files.insertFinalNewline": true
+            }
+        }
+    },
+    "workspaceFolder": "/home/ubuntu/dynamo",
+    "workspaceMount": "source=${localWorkspaceFolder},target=/home/ubuntu/dynamo,type=bind,consistency=cached",
+    "userEnvProbe": "interactiveShell",
+    "postCreateCommand": "/bin/bash /home/ubuntu/dynamo/.devcontainer/post-create.sh", // Runs cargo build and pip installs packages
+    "containerEnv": {
+        "DYNAMO_HOME": "/home/ubuntu/dynamo",
+        "CARGO_HOME": "/home/ubuntu/dynamo/.build/.cargo",
+        "RUSTUP_HOME": "/home/ubuntu/dynamo/.build/.rustup",
+        "CARGO_TARGET_DIR": "/home/ubuntu/dynamo/.build/target"
+    },
+    "remoteEnv": {
+        // Optional convenience tokens passed from host. SSH agent is forwarded via your IDE setting, not here by default.
+        "GITHUB_TOKEN": "${localEnv:GITHUB_TOKEN}",
+        "HF_TOKEN": "${localEnv:HF_TOKEN}"
+        // "SSH_AUTH_SOCK": "${env:SSH_AUTH_SOCK}" // Optional: only if you also bind-mount the socket path
+    },
+    "mounts": [
+        // These are for convenience, so that the history and pre-commit cache are persisted between sessions
+        "source=dynamo-bashhistory,target=/home/ubuntu/.commandhistory,type=volume",
+        "source=dynamo-precommit-cache,target=/home/ubuntu/.cache/pre-commit,type=volume",
+        // Default mounts
+        "source=/tmp/,target=/tmp/,type=bind"
+        // Uncomment this to reuse your Hugging Face cache
+        //"source=${localEnv:HOME}/.cache/huggingface,target=/home/ubuntu/.cache/huggingface,type=bind"
+    ]
+}
--- a/.devcontainer/gen_devcontainer_json.py
+++ b/.devcontainer/gen_devcontainer_json.py
@ -0,0 +1,64 @@
+#!/usr/bin/env python3
+# SPDX-FileCopyrightText: Copyright (c) 2025 NVIDIA CORPORATION & AFFILIATES. All rights reserved.
+# SPDX-License-Identifier: Apache-2.0
+
+"""
+Script to generate devcontainer.json files for different frameworks using Jinja2 template.
+"""
+
+from datetime import datetime
+from pathlib import Path
+
+from jinja2 import Environment, FileSystemLoader
+
+
+def main():
+    """Generate devcontainer.json files for different frameworks."""
+    # Define the frameworks to generate
+    frameworks = ["vllm", "sglang", "trtllm"]
+
+    # Get the current directory (where this script is located)
+    script_dir = Path(__file__).parent
+
+    # Setup Jinja2 environment
+    env = Environment(loader=FileSystemLoader(script_dir))
+    template = env.get_template("devcontainer.json.j2")
+
+    # Generate devcontainer.json for each framework
+    for framework in frameworks:
+        # Create the target directory
+        target_dir = script_dir / framework
+        target_dir.mkdir(exist_ok=True)
+
+        # Render the template with framework-specific values
+        current_year = datetime.now().year
+        rendered_content = template.render(
+            framework=framework, current_year=current_year
+        )
+
+        # Add auto-generated warning to the beginning of the file
+        warning_comment = """// AUTO-GENERATED FILE - DO NOT EDIT DIRECTLY
+// This file was generated from devcontainer.json.j2
+// To make changes, edit the .j2 template and run gen_devcontainer_json.py
+"""
+
+        # Insert warning after the opening brace
+        lines = rendered_content.split("\n")
+        if lines[0].strip() == "{":
+            lines.insert(1, warning_comment.rstrip())
+            rendered_content = "\n".join(lines)
+
+        # Write the rendered content to the target file
+        target_file = target_dir / "devcontainer.json"
+        with open(target_file, "w") as f:
+            f.write(rendered_content)
+
+        print(f"Generated {target_file}")
+
+    print(
+        f"Successfully generated devcontainer.json files for: {', '.join(frameworks)}"
+    )
+
+
+if __name__ == "__main__":
+    main()
--- a/.devcontainer/sglang/devcontainer.json
+++ b/.devcontainer/sglang/devcontainer.json
@ -0,0 +1,87 @@
+{
+// AUTO-GENERATED FILE - DO NOT EDIT DIRECTLY
+// This file was generated from devcontainer.json.j2
+// To make changes, edit the .j2 template and run gen_devcontainer_json.py
+    "$schema": "https://raw.githubusercontent.com/devcontainers/spec/main/schemas/devContainer.schema.json",
+    "copyright": [
+        "SPDX-FileCopyrightText: Copyright (c) 2025 NVIDIA CORPORATION & AFFILIATES. All rights reserved.",
+        "SPDX-License-Identifier: Apache-2.0"
+    ],
+    "name": "Dynamo SGLANG Dev Container",
+    "remoteUser": "ubuntu", // Matches our container user
+    "updateRemoteUserUID": true, // Updates the UID of the remote user to match the host user, avoids permission errors
+    "image": "dynamo:latest-sglang-local-dev", // Use the latest SGLANG dev image
+    "runArgs": [
+        "--gpus=all",
+        "--network=host",
+        "--ipc=host",
+        "--cap-add=SYS_PTRACE",
+        "--shm-size=10G",
+        "--ulimit=memlock=-1",
+        "--ulimit=stack=67108864",
+        "--ulimit=nofile=65536:65536"
+    ],
+    "customizations": {
+        "vscode": {
+            "extensions": [
+                "ms-python.python",
+                "ms-python.vscode-pylance",
+                "rust-lang.rust-analyzer"
+            ],
+            "settings": {
+                // Disable automatic copying of .gitconfig to avoid errors
+                "dev.containers.copyGitConfig": false,
+                "terminal.integrated.defaultProfile.linux": "bash",
+                "terminal.integrated.cwd": "/home/ubuntu/dynamo",
+
+                "python.defaultInterpreterPath": "/opt/dynamo/venv/bin/python",
+                "python.linting.enabled": true,
+
+                "rust-analyzer.memoryLimit": 4096, // larger memory limit to reduce latency
+                "rust-analyzer.checkOnSave.command": "clippy",
+                "rust-analyzer.checkOnSave.enable": true,
+                "rust-analyzer.cargo.buildScripts.enable": true,
+                "rust-analyzer.cargo.targetDir": "/home/ubuntu/dynamo/.build/target",
+                "rust-analyzer.procMacro.enable": true,
+                "rust-analyzer.completion.autoimport.enable": true,
+
+                // Enhanced rust-analyzer configuration
+                "rust-analyzer.linkedProjects": [
+                    "Cargo.toml",
+                    "lib/runtime/Cargo.toml",
+                    "lib/llm/Cargo.toml",
+                    "lib/tokens/Cargo.toml",
+                    "lib/bindings/python/Cargo.toml",
+                    "launch/dynamo-run/Cargo.toml"
+                ],
+                "files.trimTrailingWhitespace": true,
+                "files.insertFinalNewline": true
+            }
+        }
+    },
+    "workspaceFolder": "/home/ubuntu/dynamo",
+    "workspaceMount": "source=${localWorkspaceFolder},target=/home/ubuntu/dynamo,type=bind,consistency=cached",
+    "userEnvProbe": "interactiveShell",
+    "postCreateCommand": "/bin/bash /home/ubuntu/dynamo/.devcontainer/post-create.sh", // Runs cargo build and pip installs packages
+    "containerEnv": {
+        "DYNAMO_HOME": "/home/ubuntu/dynamo",
+        "CARGO_HOME": "/home/ubuntu/dynamo/.build/.cargo",
+        "RUSTUP_HOME": "/home/ubuntu/dynamo/.build/.rustup",
+        "CARGO_TARGET_DIR": "/home/ubuntu/dynamo/.build/target"
+    },
+    "remoteEnv": {
+        // Optional convenience tokens passed from host. SSH agent is forwarded via your IDE setting, not here by default.
+        "GITHUB_TOKEN": "${localEnv:GITHUB_TOKEN}",
+        "HF_TOKEN": "${localEnv:HF_TOKEN}"
+        // "SSH_AUTH_SOCK": "${env:SSH_AUTH_SOCK}" // Optional: only if you also bind-mount the socket path
+    },
+    "mounts": [
+        // These are for convenience, so that the history and pre-commit cache are persisted between sessions
+        "source=dynamo-bashhistory,target=/home/ubuntu/.commandhistory,type=volume",
+        "source=dynamo-precommit-cache,target=/home/ubuntu/.cache/pre-commit,type=volume",
+        // Default mounts
+        "source=/tmp/,target=/tmp/,type=bind"
+        // Uncomment this to reuse your Hugging Face cache
+        //"source=${localEnv:HOME}/.cache/huggingface,target=/home/ubuntu/.cache/huggingface,type=bind"
+    ]
+}
--- a/.devcontainer/trtllm/devcontainer.json
+++ b/.devcontainer/trtllm/devcontainer.json
@ -0,0 +1,87 @@
+{
+// AUTO-GENERATED FILE - DO NOT EDIT DIRECTLY
+// This file was generated from devcontainer.json.j2
+// To make changes, edit the .j2 template and run gen_devcontainer_json.py
+    "$schema": "https://raw.githubusercontent.com/devcontainers/spec/main/schemas/devContainer.schema.json",
+    "copyright": [
+        "SPDX-FileCopyrightText: Copyright (c) 2025 NVIDIA CORPORATION & AFFILIATES. All rights reserved.",
+        "SPDX-License-Identifier: Apache-2.0"
+    ],
+    "name": "Dynamo TRTLLM Dev Container",
+    "remoteUser": "ubuntu", // Matches our container user
+    "updateRemoteUserUID": true, // Updates the UID of the remote user to match the host user, avoids permission errors
+    "image": "dynamo:latest-trtllm-local-dev", // Use the latest TRTLLM dev image
+    "runArgs": [
+        "--gpus=all",
+        "--network=host",
+        "--ipc=host",
+        "--cap-add=SYS_PTRACE",
+        "--shm-size=10G",
+        "--ulimit=memlock=-1",
+        "--ulimit=stack=67108864",
+        "--ulimit=nofile=65536:65536"
+    ],
+    "customizations": {
+        "vscode": {
+            "extensions": [
+                "ms-python.python",
+                "ms-python.vscode-pylance",
+                "rust-lang.rust-analyzer"
+            ],
+            "settings": {
+                // Disable automatic copying of .gitconfig to avoid errors
+                "dev.containers.copyGitConfig": false,
+                "terminal.integrated.defaultProfile.linux": "bash",
+                "terminal.integrated.cwd": "/home/ubuntu/dynamo",
+
+                "python.defaultInterpreterPath": "/opt/dynamo/venv/bin/python",
+                "python.linting.enabled": true,
+
+                "rust-analyzer.memoryLimit": 4096, // larger memory limit to reduce latency
+                "rust-analyzer.checkOnSave.command": "clippy",
+                "rust-analyzer.checkOnSave.enable": true,
+                "rust-analyzer.cargo.buildScripts.enable": true,
+                "rust-analyzer.cargo.targetDir": "/home/ubuntu/dynamo/.build/target",
+                "rust-analyzer.procMacro.enable": true,
+                "rust-analyzer.completion.autoimport.enable": true,
+
+                // Enhanced rust-analyzer configuration
+                "rust-analyzer.linkedProjects": [
+                    "Cargo.toml",
+                    "lib/runtime/Cargo.toml",
+                    "lib/llm/Cargo.toml",
+                    "lib/tokens/Cargo.toml",
+                    "lib/bindings/python/Cargo.toml",
+                    "launch/dynamo-run/Cargo.toml"
+                ],
+                "files.trimTrailingWhitespace": true,
+                "files.insertFinalNewline": true
+            }
+        }
+    },
+    "workspaceFolder": "/home/ubuntu/dynamo",
+    "workspaceMount": "source=${localWorkspaceFolder},target=/home/ubuntu/dynamo,type=bind,consistency=cached",
+    "userEnvProbe": "interactiveShell",
+    "postCreateCommand": "/bin/bash /home/ubuntu/dynamo/.devcontainer/post-create.sh", // Runs cargo build and pip installs packages
+    "containerEnv": {
+        "DYNAMO_HOME": "/home/ubuntu/dynamo",
+        "CARGO_HOME": "/home/ubuntu/dynamo/.build/.cargo",
+        "RUSTUP_HOME": "/home/ubuntu/dynamo/.build/.rustup",
+        "CARGO_TARGET_DIR": "/home/ubuntu/dynamo/.build/target"
+    },
+    "remoteEnv": {
+        // Optional convenience tokens passed from host. SSH agent is forwarded via your IDE setting, not here by default.
+        "GITHUB_TOKEN": "${localEnv:GITHUB_TOKEN}",
+        "HF_TOKEN": "${localEnv:HF_TOKEN}"
+        // "SSH_AUTH_SOCK": "${env:SSH_AUTH_SOCK}" // Optional: only if you also bind-mount the socket path
+    },
+    "mounts": [
+        // These are for convenience, so that the history and pre-commit cache are persisted between sessions
+        "source=dynamo-bashhistory,target=/home/ubuntu/.commandhistory,type=volume",
+        "source=dynamo-precommit-cache,target=/home/ubuntu/.cache/pre-commit,type=volume",
+        // Default mounts
+        "source=/tmp/,target=/tmp/,type=bind"
+        // Uncomment this to reuse your Hugging Face cache
+        //"source=${localEnv:HOME}/.cache/huggingface,target=/home/ubuntu/.cache/huggingface,type=bind"
+    ]
+}
--- a/.devcontainer/vllm/devcontainer.json
+++ b/.devcontainer/vllm/devcontainer.json
@ -1,10 +1,13 @@
 {
+// AUTO-GENERATED FILE - DO NOT EDIT DIRECTLY
+// This file was generated from devcontainer.json.j2
+// To make changes, edit the .j2 template and run gen_devcontainer_json.py
    "$schema": "https://raw.githubusercontent.com/devcontainers/spec/main/schemas/devContainer.schema.json",
    "copyright": [
        "SPDX-FileCopyrightText: Copyright (c) 2025 NVIDIA CORPORATION & AFFILIATES. All rights reserved.",
        "SPDX-License-Identifier: Apache-2.0"
    ],
-    "name": "NVIDIA Dynamo Dev Container Development",
+    "name": "Dynamo VLLM Dev Container",
    "remoteUser": "ubuntu", // Matches our container user
    "updateRemoteUserUID": true, // Updates the UID of the remote user to match the host user, avoids permission errors
    "image": "dynamo:latest-vllm-local-dev", // Use the latest VLLM dev image
@ -81,4 +84,4 @@
        // Uncomment this to reuse your Hugging Face cache
        //"source=${localEnv:HOME}/.cache/huggingface,target=/home/ubuntu/.cache/huggingface,type=bind"
    ]
-}
+}