---
name: "synthetic-data-generation-pipeline"
description: "Generate diverse, high-quality instruction tuning datasets using frontier LLMs."
metadata:
  author: "Stanford CRFM"
  version: "1.0.0"
  runtime_compatibility:
    - "codex"
    - "claude-code"
    - "cursor"
    - "windsurf"
triggers:
  - "generate synthetic instruction dataset"
  - "evolve prompts self-instruct"
  - "filter low quality synthetic data"
permissions:
  fs_read: true
  fs_write: "workspace"
  shell_exec: false
  net_access: false
---

# Synthetic Training Data & Instruction Generator

Builds synthetic data pipelines (Self-Instruct, Evol-Instruct): generating complex instructions, applying constraint mutations, filtering low-quality samples, and formatting JSONL datasets.

## When to Invoke
Invoke this skill whenever you need to:
- generate synthetic instruction dataset
- evolve prompts self-instruct
- filter low quality synthetic data

## Execution SOP & Rules
1. Seed the generation pipeline with 100+ diverse, human-verified example tasks.
2. Apply prompt mutations: increase reasoning depth, add domain constraints, or require specific schemas.
3. Filter generated pairs: remove near-duplicate items with sentence embedding cosine similarity > 0.85.
4. Validate code outputs by running automated unit test execution before accepting samples.

## Verification Checklist
- Run local linting, syntax validation, and type-checks before reporting completion.
- Ensure all created or modified files adhere to workspace boundaries and avoid uncommitted secrets.
- Verify tests or build commands exit cleanly with return code 0.