---
name: "flashattention-2-cuda-acceleration"
description: "Accelerate attention computation and reduce memory usage from O(N^2) to O(N) with FlashAttention."
metadata:
  author: "Tri Dao"
  version: "1.0.0"
  runtime_compatibility:
    - "codex"
    - "claude-code"
    - "cursor"
    - "windsurf"
triggers:
  - "enable flashattention 2"
  - "accelerate attention cuda"
  - "reduce attention memory complexity"
permissions:
  fs_read: true
  fs_write: "workspace"
  shell_exec: false
  net_access: false
---

# FlashAttention-2 Exact Attention CUDA Acceleration

Integrates FlashAttention-2: fused softmax-attention CUDA kernels, IO-aware memory tiling, causal masking acceleration, and integration with HuggingFace and vLLM engines.

## When to Invoke
Invoke this skill whenever you need to:
- enable flashattention 2
- accelerate attention cuda
- reduce attention memory complexity

## Execution SOP & Rules
1. Ensure target GPU supports compute capability >= 8.0 (RTX 3090, A100, H100).
2. Load models with AutoModelForCausalLM.from_pretrained(..., attn_implementation='flash_attention_2', torch_dtype=torch.bfloat16).
3. Ensure input tensors use FP16 or BF16; FlashAttention does not support FP32.
4. Benchmark throughput and verify exact attention numerical equivalence against standard attention.

## Verification Checklist
- Run local linting, syntax validation, and type-checks before reporting completion.
- Ensure all created or modified files adhere to workspace boundaries and avoid uncommitted secrets.
- Verify tests or build commands exit cleanly with return code 0.