---
name: "vllm-high-throughput-inference"
description: "Serve open-source LLMs at maximum throughput with PagedAttention and vLLM engine."
metadata:
  author: "vLLM Team"
  version: "1.0.0"
  runtime_compatibility:
    - "codex"
    - "claude-code"
    - "cursor"
    - "windsurf"
triggers:
  - "deploy vllm server"
  - "serve llama 3 with vllm"
  - "configure pagedattention vllm"
permissions:
  fs_read: true
  fs_write: "workspace"
  shell_exec: false
  net_access: false
---

# vLLM High-Throughput PagedAttention Inference

Deploys vLLM server: PagedAttention KV-cache management, continuous batching, tensor parallelism across multiple GPUs, OpenAI-compatible REST server, and AWQ/FP8 quantization.

## When to Invoke
Invoke this skill whenever you need to:
- deploy vllm server
- serve llama 3 with vllm
- configure pagedattention vllm

## Execution SOP & Rules
1. Launch vLLM server: python3 -m vllm.entrypoints.openai.api_server --model meta-llama/Llama-3.3-70B-Instruct.
2. Tune --gpu-memory-utilization 0.95 and --max-model-len to maximize KV-cache capacity.
3. Enable tensor parallelism across GPUs using --tensor-parallel-size <num_gpus>.
4. Connect downstream applications using standard OpenAI client SDKs pointing to the vLLM endpoint.

## Verification Checklist
- Run local linting, syntax validation, and type-checks before reporting completion.
- Ensure all created or modified files adhere to workspace boundaries and avoid uncommitted secrets.
- Verify tests or build commands exit cleanly with return code 0.