---
name: "aphrodite-engine-inference-service"
description: "Serve large language models with diverse quantization formats using Aphrodite Engine."
metadata:
  author: "PygmalionAI"
  version: "1.0.0"
  runtime_compatibility:
    - "codex"
    - "claude-code"
    - "cursor"
    - "windsurf"
triggers:
  - "deploy aphrodite engine"
  - "serve gptq model aphrodite"
  - "run marlin kernel inference"
permissions:
  fs_read: true
  fs_write: "workspace"
  shell_exec: false
  net_access: false
---

# Aphrodite Engine Distributed LLM Serving

Deploys high-throughput LLM serving with Aphrodite Engine: PagedAttention, vLLM fork optimizations, wide quantization support (AWQ, GPTQ, GGUF, Marlin, FP8), and OpenAI-compatible endpoints.

## When to Invoke
Invoke this skill whenever you need to:
- deploy aphrodite engine
- serve gptq model aphrodite
- run marlin kernel inference

## Execution SOP & Rules
1. Launch server with aphrodite run <model_name> --quantization marlin.
2. Tune --max-num-batched-tokens and --max-num-seqs to match available GPU VRAM.
3. Connect standard OpenAI-compatible client SDKs directly to the Aphrodite HTTP port.
4. Monitor generation tokens-per-second using integrated health and metrics endpoints.

## Verification Checklist
- Run local linting, syntax validation, and type-checks before reporting completion.
- Ensure all created or modified files adhere to workspace boundaries and avoid uncommitted secrets.
- Verify tests or build commands exit cleanly with return code 0.