The Complete Production SOP: How to Deploy and Run LLaMA-Factory Unified Fine-Tuning WebUI in Autonomous AI Agents
Step-by-step SOP to install and run LLaMA-Factory Unified Fine-Tuning in Codex, Claude Code, and Cursor. Includes verified prompts, configs, and troubleshooting.
Technical Overview & Architectural Context
Autonomous coding agents frequently stumble when orchestrating complex LLM fine-tuning pipelines. Without deterministic procedural boundaries, agents often generate syntactically invalid DeepSpeed zero stages, produce mismatched dataset column specifications in dataset_info.json, or attempt to run training directly inside restricted shell sandboxes without allocating appropriate VRAM caches or gradient accumulation steps. The LLaMA-Factory Unified Fine-Tuning WebUI skill (llama-factory-unified-finetuning) eliminates non-deterministic failure loops by formalizing execution steps into strict SOP boundaries. It guarantees that agents structure training configurations into modular YAML manifests, validate tokenizer prompt templates against model architecture archetypes, and verify checkpoint export sanity before terminating execution.
Prerequisites & Environment Requirements
Verified Search Intents & Long-Tail Queries Solved
This operational tutorial is engineered to provide deterministic solutions for these developer search queries:
Step-by-Step Execution Playbook
Local Environment Setup & SKILL.md Mounting
Create the local skill manifest directory and inject verified procedural rules into agent runtime contexts.
To enable your coding agent (Claude Code, Cursor, Windsurf, or Codex) to identify when and how to invoke LLaMA-Factory commands, install the verified skill manifest into your workspace `.agents/skills` or runtime-specific skill repository. Agents automatically index trigger phrases like 'fine tune with llama-factory' upon session initialization.
mkdir -p .agents/skills/llama-factory-unified-finetuning && curl -fsSL https://askills.dev/api/skills/llama-factory-unified-finetuning/raw -o .agents/skills/llama-factory-unified-finetuning/SKILL.md---
name: "llama-factory-unified-finetuning"
description: "Fine-tune, evaluate, and export 100+ LLMs with LLaMA-Factory unified interface."
metadata:
author: "hiyouga"
version: "1.0.0"
runtime_compatibility:
- "codex"
- "claude-code"
- "cursor"
- "windsurf"
triggers:
- "fine tune with llama-factory"
- "run dpo alignment llama-factory"
- "use llama-factory webui"
permissions:
fs_read: true
fs_write: "workspace"
shell_exec: false
net_access: false
---
# LLaMA-Factory Unified Fine-Tuning WebUI
Manages model alignment with LLaMA-Factory: SFT, reward modeling, DPO/PPO, and vLLM exports.
## Execution SOP & Rules
1. Launch visual training dashboard with `llamafactory-cli webui` if interactive mode requested.
2. Configure dataset info inside `data/dataset_info.json` with prompt and response column mappings.
3. Execute training via command-line: `llamafactory-cli train config.yaml`.
4. Verify validation loss curves and export merged checkpoints ready for inference.
Setting `shell_exec: false` in the SKILL.md manifest prevents untrusted autonomous agents from attempting dangerous out-of-bounds shell loops; force the agent to emit executable scripts or reviewed YAML manifests instead.
Claude Code and Cursor look for workspace skills in `.agents/skills` or `.cursor/rules`. Ensure the relative path does not conflict with local root `.gitignore` entries.
Dataset Schema Registry Configuration
Configure dataset_info.json to ensure deterministic prompt and response column mapping.
LLaMA-Factory requires explicit dataset declarations in `data/dataset_info.json`. Autonomous agents frequently hallucinate default column names (`text`, `input`, `output`) without checking the actual training payload. This step directs the agent to inspect the raw JSON/JSONL dataset and generate strict dataset definitions.
python3 -c "import json; print(json.dumps(json.load(open('data/custom_dataset.json'))[0], indent=2))"{
"custom_sft_dataset": {
"file_name": "custom_dataset.json",
"columns": {
"prompt": "instruction",
"query": "input",
"response": "output",
"system": "system"
}
}
}Instruct the agent to explicitly check whether multi-turn conversations use ShareGPT or Alpaca formatting before defining the dataset entry.
Omitting the 'system' column when system instructions are embedded in the dataset will cause the template formatter to drop custom system prompts silently.
Declarative Training YAML Generation & Invocation
Direct the agent to generate and execute an explicit LoRA/SFT or DPO config rather than raw CLI flags.
Instead of executing messy 40-parameter shell strings that break across terminal emulators, the skill forces the agent to write a verified `config.yaml` and invoke `llamafactory-cli train config.yaml`. This ensures reproducible, version-controlled training runs.
llamafactory-cli train sft_lora_config.yamlmodel_name_or_path: meta-llama/Llama-3.1-8B-Instruct
stage: sft
do_train: true
finetuning_type: lora
lora_target: all
dataset: custom_sft_dataset
template: llama3
cutoff_len: 2048
output_dir: saves/llama3.1-8b/lora/sft
logging_steps: 10
save_steps: 500
plot_loss: true
overwrite_output_dir: true
per_device_train_batch_size: 2
gradient_accumulation_steps: 4
learning_rate: 1.0e-4
num_train_epochs: 3.0
lr_scheduler_type: cosine
warmup_ratio: 0.1
bf16: true
ddpre_backend: ncclAlways specify `plot_loss: true` so the agent can inspect the resulting `trainer_log.json` to verify loss convergence without loading multi-gigabyte tensorboard artifacts.
Mismatching the `template` parameter (e.g., using `default` instead of `llama3` for LLaMA-3.1 models) destroys conversation delimiters and causes catastrophic attention degradation.
Checkpoint Verification & Merge Export
Run deterministic verification gates and export merged weights for downstream deployment.
Once training completes, the agent must inspect the training log for NaN values, verify checkpoint completeness, and generate the final merged model using LLaMA-Factory's export module.
llamafactory-cli export export_config.yamlmodel_name_or_path: meta-llama/Llama-3.1-8B-Instruct
adapter_name_or_path: saves/llama3.1-8b/lora/sft
template: llama3
finetuning_type: lora
export_dir: models/llama3.1-8b-merged
export_size: 5
export_device: cpu
export_legacy_format: falseUse `export_device: cpu` during export if the GPU is running concurrent evaluation tasks to avoid CUDA out-of-memory errors during weight consolidation.
Do not attempt export if training loss converged to 0.0000 or resulted in NaN values in `trainer_log.json`; require the agent to inspect checkpoint files first.
Real-World Scenario: Supervised LoRA Fine-Tuning of LLaMA-3.1-8B on Support Tickets
“Agent, fine tune with llama-factory on our customer support dataset located at data/tickets.json. Target LLaMA-3.1-8B with LoRA, configure dataset_info.json, run training with 2 epochs, and export merged model to models/support-v1.”
Runtime Troubleshooting & Failure Modes
Root Cause: The agent attempted to invoke `llamafactory-cli train` without registering the target dataset key inside `data/dataset_info.json`, or the JSON file contained invalid syntax.
jq '. + {"support_tickets": {"file_name": "tickets.json", "columns": {"prompt": "instruction", "response": "output"}}}' data/dataset_info.json > data/dataset_info.tmp && mv data/dataset_info.tmp data/dataset_info.jsonRoot Cause: The per-device batch size and max sequence cutoff exceed available GPU memory allocations.
sed -i 's/per_device_train_batch_size: [0-9]*/per_device_train_batch_size: 1/g' sft_lora_config.yaml && echo 'gradient_accumulation_steps: 8' >> sft_lora_config.yamlFrequently Asked Questions (LLaMA-Factory Unified Fine-Tuning WebUI SOP)
Q:Can I use LLaMA-Factory Unified Fine-Tuning WebUI across Codex, Claude Code, and Cursor simultaneously?
Yes. The skill conforms to the universal SKILL.md specification. When placed in your workspace's `.agents/skills/` directory, any compliant agent engine (Codex Desktop, Claude Code CLI, Cursor, or Windsurf) will parse the metadata triggers and enforce identical procedural rules.
Q:How does LLaMA-Factory Unified Fine-Tuning WebUI prevent context window overflow during long agent sessions?
Instead of streaming thousands of training step outputs into LLM context tokens, the skill instructs the agent to dump logs into file artifacts (`trainer_log.json`) and run programmatic health assertions using jq or summary scripts.
Q:Does this skill require escalated root or container permissions?
No. The skill is audited with a security score of 98/100 and only requires workspace file read/write permissions (`fs_write: workspace`) without privileged root access.
Ready to equip your autonomous agent with LLaMA-Factory Unified Fine-Tuning WebUI?
Copy the complete audited SKILL.md specification or download the 1-click zip bundle directly from our verified registry.