trl-training
Post-trains LLMs with TRL (SFT, DPO, GRPO, KTO and reward models) through the Python API or the trl CLI.
- Kind
- Skill
- Works with
- Claude Code
- Languages
- Python
- Source
- Hugging Face Skills @ 25910b3d0ba3
- Path
- skills/trl-training/SKILL.md
- Jobs
- Implementation
- Topics
- Machine learning, LLM apps & RAG
Credit
- Made by
- huggingface
- Original
- github.com/huggingface/skills/blob/25910b3d0ba31a2ba102f901cd…
- License
- Apache-2.0
Listed by Skill Manifest; not made by us.
InstallInstall it by hand, following the source.
hf skills add trl-training
Install factsScanned 2 Nov 2026
- Adds
- .claude/skills/trl-training/ (the skill folder)
- Runs scripts
- None found
- Uses the network
- None found
- Needs secrets
- None found
- Source revision
- 25910b3d0ba3
- Licence
- Apache-2.0
- Last activity
- 30 Oct 2026
- Review
- Reviewed
Found by a static scan of the stored text; nothing is run. “None found” is not a guarantee, so read what you install before you run it.
- Only the main file is read; other files in the folder are not scanned
# TRL
Each method pairs a `*Trainer` class with a `*Config` dataclass. Configs extend `transformers.TrainingArguments`, so all of its arguments work in any trainer config.
| Trainer | Dataset type |
|---|---|
| `SFTTrainer` | language modeling or prompt-completion |
| `DPOTrainer` | preference (chosen/rejected pairs) |
| `GRPOTrainer` | prompt-only + reward function(s) |
| `DistillationTrainer` | prompt-only + a teacher model (on-policy distillation) |
| `KTOTrainer` | unpaired preference (per-sample bool label) |
| `RewardTrainer` | preference (chosen/rejected pairs); trains a scalar reward model, not a policy |
Many more trainers (OnlineDPO, ORPO, CPO, GKD, …) live in `trl.experimental` with unstable APIs: https://huggingface.co/docs/trl/experimental_overview
```python
from datasets import load_dataset
from trl import SFTConfig, SFTTrainer
trainer = SFTTrainer(
model="Qwen/Qwen2.5-0.5B", # model ID or a PreTrainedModel instance
args=SFTConfig(output_dir="Qwen2.5-0.5B-SFT"),
train_dataset=load_dataset("trl-lib/Capybara", split="train"),
)
trainer.train()
```
…