---
name: "tgi-text-generation-inference"
description: "Deploy enterprise LLM inference services with HuggingFace TGI container stack."
metadata:
  author: "HuggingFace"
  version: "1.0.0"
  runtime_compatibility:
    - "codex"
    - "claude-code"
    - "cursor"
    - "windsurf"
triggers:
  - "deploy huggingface tgi"
  - "configure flash-attention tgi"
  - "enable speculative decoding tgi"
permissions:
  fs_read: true
  fs_write: "workspace"
  shell_exec: false
  net_access: false
---

# HuggingFace Text Generation Inference (TGI)

Deploys HuggingFace Text Generation Inference: flash-attention 2, speculative decoding, dynamic batching, token streaming over gRPC/HTTP, and integration with HuggingFace Hub.

## When to Invoke
Invoke this skill whenever you need to:
- deploy huggingface tgi
- configure flash-attention tgi
- enable speculative decoding tgi

## Execution SOP & Rules
1. Run official Docker container: ghcr.io/huggingface/text-generation-inference:latest.
2. Specify model using --model-id <org/model_name> and set --max-batch-prefill-tokens.
3. Enable FlashAttention-2 with --sharded true on multi-GPU nodes.
4. Monitor inference metrics via Prometheus endpoint exposed at port 80/metrics.

## Verification Checklist
- Run local linting, syntax validation, and type-checks before reporting completion.
- Ensure all created or modified files adhere to workspace boundaries and avoid uncommitted secrets.
- Verify tests or build commands exit cleanly with return code 0.