We use cookies to enhance your experience, serve personalized ads or content, and analyze our traffic. By clicking "Accept", you consent to our use of cookies. For more information, please see our privacy policy.
Skip to content
vLLM logo

vLLM

Run an OpenAI & Claude inference endpoint with vLLM
OfficialAIInferenceLlmOpenAIVllm

About vLLM

Deploy an OpenAI & Claude compatible REST API endpoint using vLLM with GPU acceleration — running natively under systemd instead of Docker. Out of the box this template serves Qwen/Qwen3.8-27B-FP8 on a single FP8-native GPU with tool calling support and HuggingFace model caching.
The endpoint is fully compatible with the OpenAI /v1/chat/completions and Claude /v1/messages, making it a drop-in replacement for any OpenAI compatible SDK client.
Use a GPU server with NVIDIA drivers already installed and native FP8 support (Hopper or newer — e.g. Verda 1xH200, or scale to 2x/4x/8x for larger models). Dependencies are managed with uv from a native pyproject.toml, and uv auto-selects the PyTorch CUDA wheels matching the host driver (CUDA 12.9 / 13.0), so no CUDA toolkit install is required. Set HF_TOKEN to your HuggingFace token to download gated models, and set MODEL_NAME and TENSOR_PARALLEL_SIZE to match the model and GPU count.
This template is experimental — verify the deployment on your own GPU server before relying on it.
DollarDeploy

About DollarDeploy

DollarDeploy deploys and manages apps on your own VPS — no SSH, no YAML, no lock-in. Launch vLLM in a few clicks, then get HTTPS, monitoring, logs and backups handled for you.