{
  "$type": "site.standard.document",
  "bskyPostRef": {
    "cid": "bafyreigwh4mlydxwi6lagn72rveymkevx4ug2x4s3cs5cr4xxinymyurha",
    "uri": "at://did:plc:5sgu76a53rz3n6unbykmovqy/app.bsky.feed.post/3mlodh6xmv762"
  },
  "description": "Fine-tuning is the process of taking a pretrained model and continuing to train it on a target dataset, so its weights adapt to a specific task, domain, style, or output format. It is the standard way to customise a model when prompting alone is not enough.\n\n\nVariants\n\n * Full fine-tuning. Updates every parameter. Maximum capacity, maximum cost.\n * LoRA (Low-Rank Adaptation). Trains a small low-rank update on top of frozen weights. Fast, cheap, and the result is a small adapter file.\n * QLoRA. L",
  "path": "/engineering-glossary/fine-tuning-language-models/",
  "publishedAt": "2026-05-12T17:39:36.000Z",
  "site": "https://sahilkapoor.com",
  "tags": [
    "RAG",
    "Embeddings"
  ],
  "textContent": "**Fine-tuning** is the process of taking a pretrained model and continuing to train it on a target dataset, so its weights adapt to a specific task, domain, style, or output format. It is the standard way to customise a model when prompting alone is not enough.\n\n## Variants\n\n  * **Full fine-tuning.** Updates every parameter. Maximum capacity, maximum cost.\n  * **LoRA (Low-Rank Adaptation).** Trains a small low-rank update on top of frozen weights. Fast, cheap, and the result is a small adapter file.\n  * **QLoRA.** LoRA on a quantised base model, enabling fine-tuning of large models on a single GPU.\n  * **Instruction tuning.** Fine-tuning on instruction-response pairs to make a base model follow instructions.\n  * **RLHF and DPO.** Aligning model outputs with human preferences, used in modern chat models.\n\n\n\n🔗\n\n**Related Terms**\nRAG, Embeddings.",
  "title": "Fine-tuning",
  "updatedAt": "2026-05-13T19:15:20.259Z"
}