We use cookies to enhance your experience, serve personalized ads or content, and analyze our traffic. By clicking "Accept", you consent to our use of cookies. For more information, please see our privacy policy.
    Skip to content
    vLLM logo

    vLLM

    Run an OpenAI & Claude inference endpoint with vLLM
    OfficialAIInferenceLlmOpenAIVllm

    About vLLM

    Deploy an OpenAI & Claude compatible REST API endpoint using vLLM with GPU acceleration — running natively under systemd instead of Docker. Out of the box this template serves Qwen/Qwen3.8-27B-FP8 on a single FP8-native GPU with tool calling support and HuggingFace model caching.
    The endpoint is fully compatible with the OpenAI /v1/chat/completions and Claude /v1/messages, making it a drop-in replacement for any OpenAI compatible SDK client.
    Use a GPU server with NVIDIA drivers already installed and native FP8 support (Hopper or newer — e.g. Verda 1xH200, or scale to 2x/4x/8x for larger models). Dependencies are managed with uv from a native pyproject.toml, and uv auto-selects the PyTorch CUDA wheels matching the host driver (CUDA 12.9 / 13.0), so no CUDA toolkit install is required. Set HFTOKEN to your HuggingFace token to download gated models, and set MODELNAME and TENSORPARALLELSIZE to match the model and GPU count.
    This template is experimental — verify the deployment on your own GPU server before relying on it.
    DollarDeploy

    About DollarDeploy

    DollarDeploy deploys and manages apps on your own VPS — no SSH, no YAML, no lock-in. Launch vLLM in a few clicks, then get HTTPS, monitoring, logs and backups handled for you.