The Complete Production SOP: How to Deploy and Run Axolotl LoRA & QLoRA LLM Fine-Tuning Pipeline in Autonomous AI Agents
Production SOP to install and run Axolotl LoRA & QLoRA LLM Fine-Tuning Pipeline in Codex, Claude Code, and Cursor. Includes verified prompts and configs.
Technical Overview & Architectural Context
Autonomous coding agents like Claude Code, Codex, and Cursor frequently fail when generating distributed fine-tuning configurations. Without structured guidelines, agents routinely hallucinate mismatched Hugging Face tokenizer configurations, incompatible quantization flags (e.g., mixing bitsandbytes 4-bit configurations with incompatible optimizer implementations), or broken DeepSpeed ZeRO-3 parameter definitions that crash distributed runs hours into execution. The axolotl-llm-finetuning-config skill provides a deterministic boundary that forces autonomous agents to emit syntactically sound, verified Axolotl YAML manifests. By constraining the agent's file generation to validated schemas—incorporating FlashAttention-2 flags, exact gradient accumulation intervals, and memory-safe LoRA rank specifications—this skill prevents out-of-memory (OOM) failures and tokenization corruptions before training compute is wasted.
Prerequisites & Environment Requirements
Verified Search Intents & Long-Tail Queries Solved
This operational tutorial is engineered to provide deterministic solutions for these developer search queries:
Step-by-Step Execution Playbook
Local Environment Setup & SKILL.md Mounting
Mount the SKILL.md manifest into your agent's runtime discovery directory to enable deterministic triggers.
Autonomous runtimes discover capabilities through standardized directory scans. Placing SKILL.md in the workspace `.agents/skills` or `.claude/skills` directory guarantees the LLM parses the YAML frontmatter, recognizing trigger phrases like 'configure qlora axolotl yaml' or 'train llama 3 with axolotl' without manual system prompt injection.
mkdir -p .agents/skills/axolotl-llm-finetuning-config && curl -fsSL https://askills.dev/api/skills/axolotl-llm-finetuning-config/raw -o .agents/skills/axolotl-llm-finetuning-config/SKILL.md---
name: "axolotl-llm-finetuning-config"
description: "Fine-tune open-source language models efficiently with Axolotl YAML configurations."
metadata:
author: "OpenAccess AI Collective"
version: "1.0.0"
runtime_compatibility:
- "codex"
- "claude-code"
- "cursor"
- "windsurf"
triggers:
- "fine tune model with axolotl"
- "configure qlora axolotl yaml"
- "train llama 3 with axolotl"
permissions:
fs_read: true
fs_write: "workspace"
shell_exec: false
net_access: false
---
# Axolotl LoRA & QLoRA LLM Fine-Tuning Pipeline
Orchestrates LLM fine-tuning pipelines using Axolotl: LoRA, QLoRA, full parameter tuning, FlashAttention-2, DeepSpeed ZeRO-2/3, multi-GPU FSDP, and Weights & Biases telemetry.
## Execution SOP & Rules
1. Define training hyperparameters in a single declarative YAML configuration file.
2. Use adapter: qlora and load_in_4bit: true to train large models on single GPUs.
3. Enable flash_attention: true to drastically cut memory usage and boost training speed.
4. Export merged 16-bit model weights using python -m axolotl.cli.merge_lora config.yml.Keep the SKILL.md frontmatter minimal and avoid putting full raw training datasets inside the skill directory so the agent's context window stays unburdened.
Ensure permissions allow fs_write to 'workspace' only. If running inside restricted containers, verify the agent does not attempt to write configs outside the current project root.
Telemetry & Checkpoint Directory Configuration
Configure required local directories, dataset paths, and optional Hugging Face / W&B credentials via environment files.
Axolotl runs often require Hugging Face Hub authentication to pull gated weights (such as Meta-Llama-3-8B) and optionally Weights & Biases (W&B) for tracking loss metrics. Inject these through local environment variables rather than hardcoding them into YAML files.
cat << 'EOF' > .env.training
HF_TOKEN=hf_yourSecretTokenHere
WANDB_API_KEY=yourWandbKeyHere
WANDB_PROJECT=axolotl-experiments
ACCELERATE_USE_FSDP=false
EOF
echo ".env.training" >> .gitignoreNever allow the agent to echo raw token variables in chat outputs; instruct the agent to reference $HF_TOKEN or read from environment files directly.
Missing HF_TOKEN when fetching gated repositories like Llama-3 will cause immediate exit codes during tokenizer retrieval.
Intent Triggering & YAML Configuration Generation
Prompt the agent using validated triggers to generate production-ready QLoRA/LoRA configurations.
Invoking the exact trigger phrases registered in the frontmatter causes Claude Code, Cursor, or Codex to load the Axolotl procedural rules. The agent will then generate a strictly structured, fully verified configuration file tailored to the target hardware.
base_model: meta-llama/Meta-Llama-3-8B
model_type: LlamaForCausalLM
tokenizer_type: AutoTokenizer
load_in_8bit: false
load_in_4bit: true
strict: false
datasets:
- path: data/instructions.jsonl
type: alpaca
dataset_prepared_path: ./prepared_data
output_dir: ./qlora-llama3-out
adapter: qlora
lora_model_dir:
lora_r: 16
lora_alpha: 32
lora_dropout: 0.05
lora_target_linear: true
sequence_len: 4096
sample_packing: true
pad_to_sequence_len: true
wandb_project: axolotl-experiments
wandb_run_id:
wandb_watch:
wandb_name: llama3-qlora-4k
gradient_accumulation_steps: 4
micro_batch_size: 2
num_epochs: 3
optimizer: paged_adamw_8bit
lr_scheduler: cosine
learning_rate: 0.0002
train_on_inputs: false
group_by_length: false
bf16: auto
fp16: false
tf32: false
gradient_checkpointing: true
early_stopping_patience:
resume_from_checkpoint:
local_rank:
logging_steps: 1
xformers_attention:
flash_attention: true
warmup_steps: 20
eval_steps: 50
save_steps: 100
debug:
deepspeed:
weight_decay: 0.0
fsdp:
fsdp_config:
special_tokens:
pad_token: "<|end_of_text|>"Setting sample_packing: true along with pad_to_sequence_len: true eliminates wasteful padding tokens across variable-length inputs, cutting training duration by up to 40%.
Config Syntax Pre-Flight Validation
Execute Axolotl's pre-flight dataset and syntax validation script before initiating long compute jobs.
Before launching GPU training runs, use Axolotl's dataset pre-processing utility to validate the generated YAML file. This guarantees that column mappings match the dataset structure, FlashAttention-2 is functional, and token sequences are properly aligned.
python -m axolotl.cli.preprocess configs/llama3-8b-qlora.ymlReview the tokenized preview generated by Axolotl in stdout to ensure your dataset's prompt template does not accidentally inject double BOS/EOS tokens.
Running preprocessing on multi-GPU setups without specifying CUDA_VISIBLE_DEVICES may cause accidental multi-process lock contention on prepared dataset caches.
Real-World Scenario: Configure and Launch a 4-bit QLoRA Fine-Tuning Run for Llama-3-8B on an RTX 4090
“configure qlora axolotl yaml for meta-llama/Meta-Llama-3-8B on an RTX 4090 24GB. Dataset is in ./data/finetune.jsonl with columns 'instruction', 'input', and 'output'. Use sequence length 4096, sample packing, paged_adamw_8bit, and FlashAttention-2.”
Runtime Troubleshooting & Failure Modes
Root Cause: Meta-Llama-3 base models do not ship with a distinct official pad_token in their default tokenizer configuration, causing PyTorch collators to fail when batching.
Add the special_tokens block to the YAML configuration:
special_tokens:
pad_token: "<|end_of_text|>"Root Cause: FlashAttention-2 requires an Ampere or newer GPU (compute capability >= 8.0) and precompiled C++ CUDA extensions matching torch versions.
pip install --no-build-isolation flash-attn --no-cache-dir
# Alternatively, fallback in config YAML:
flash_attention: false
sdp_attention: trueFrequently Asked Questions (Axolotl LoRA & QLoRA LLM Fine-Tuning Pipeline SOP)
Q:Can I use Axolotl LoRA & QLoRA LLM Fine-Tuning Pipeline across Codex, Claude Code, and Cursor simultaneously?
Yes. This skill adheres strictly to the universal SKILL.md specification. All modern agents parse the YAML frontmatter triggers and execution SOP uniformly across CLI and IDE platforms.
Q:How does Axolotl LoRA & QLoRA LLM Fine-Tuning Pipeline prevent context window overflow during long agent sessions?
The skill establishes declarative constraints on configuration keys, preventing the agent from re-prompting or generating pages of redundant Python boilerplate. It delegates training execution to Axolotl's compiled CLI.
Q:Does this skill require escalated root or container permissions?
No. The skill is audited with a 98/100 security score and operates with shell_exec set to false and net_access disabled. It only requires read and write access within the current project workspace.
Ready to equip your autonomous agent with Axolotl LoRA & QLoRA LLM Fine-Tuning Pipeline?
Copy the complete audited SKILL.md specification or download the 1-click zip bundle directly from our verified registry.