Fine-Tuning From the Inside: What LoRA Actually Trains

We’ve already written about whether you need fine-tuning at all — and the honest answer for most teams, most of the time, is “not yet.” This post is about what happens when the answer is yes: what fine-tuning actually does to a model, why almost nobody updates the real weights anymore, and which of the dozens of knobs genuinely move the outcome.

What fine-tuning actually changes

A language model is a very large set of weight matrices. Full fine-tuning continues training all of them on your data — every parameter free to move. For a 7B model in 16-bit that means holding the weights, the gradients and the optimizer state in memory at once: roughly 8× the weight size, or ~112 GB for the smallest model anyone takes seriously. Multi-GPU territory before you’ve trained a single step, and every checkpoint you save is a full copy of the model.

Full fine-tuning also has a failure mode subtler than cost: with every weight free to move, the model can drift away from abilities you never intended to touch. Push hard on your legal-summarisation data and the model quietly gets worse at arithmetic, at other languages, at refusing bad requests. This is catastrophic forgetting, and the smaller your dataset, the sharper the risk — there’s nothing anchoring the rest of the behaviour.

The modern default avoids both problems by not touching the weights at all.

LoRA: train the difference, not the weights

LoRA starts from an empirical observation: the change that fine-tuning needs to make to a weight matrix is almost always low-rank — it has far less structure than the matrix itself. So instead of updating a large matrix W, LoRA freezes it and learns two thin matrices A and B whose product is the update:

W′ = W + B·A     — W frozen, only A and B train

If W is 4096×4096 and the rank is 16, then A is 16×4096 and B is 4096×16 — about 0.8% of the parameters of the matrix they’re modifying. Across a whole model, a typical LoRA run trains 0.1–1% of the parameters and produces an adapter file of 50–500 MB instead of a multi-gigabyte checkpoint.

Three consequences follow, and they’re the reason LoRA won:

• Memory collapses. Gradients and optimizer state exist only for A and B. The 8× multiplier applies to megabytes, not gigabytes.
• Forgetting is structurally bounded. The base weights are untouched; delete the adapter and the original model is exactly back. The model can’t drift far from what it was, because what it was is still there.
• The adapter is a file, not a model. You can version it, diff-test it against the base, and ship ten of them against one deployed model.

QLoRA: the reason it fits on one GPU

LoRA shrinks the training state, but the frozen base weights still have to sit in memory. QLoRA’s move: since the base is frozen anyway, store it quantized — 4-bit NF4, the normal-float data type we covered in the quantization post — and dequantize on the fly during the forward pass. Add paged optimizer states that spill to CPU RAM when a batch spikes, and the arithmetic changes completely: a 70B-class model fine-tunes on a single 80 GB card, and a 65B model fits on a 48 GB workstation GPU.

The precision caveat from the quantization post applies here in reverse: the gradients flow through the low-rank adapter in 16-bit, so training quality holds up far better than a naive “train on a quantized model” would suggest. The published QLoRA results match 16-bit LoRA on standard benchmarks — the reason it became the default rather than the budget option.

The knobs that actually matter

Fine-tuning frameworks expose dozens of hyperparameters. Four of them decide most outcomes:

Rank deserves the most thought. A style-transfer task (“answer like our support team”) lives happily at r=16. Teaching structured multi-turn behaviour or a coding dialect wants 64. Doubling rank doubles adapter size and nudges the forgetting risk up — it is not a free “more quality” dial.

Everything else — warmup ratios, schedulers, dropout — is worth touching only after a run with defaults has told you what’s actually wrong.

Data beats every hyperparameter

The threshold that matters more than any knob: roughly 500 high-quality, format-consistent examples. Below that, the model learns your formatting quirks and noise faster than your task, and prompting a frontier model will beat your fine-tune. Above it — assuming the examples genuinely demonstrate the behaviour you want — quality climbs quickly.

“High-quality” is doing heavy lifting there:

• Consistency outranks volume. 500 examples that all follow the same schema beat 50,000 scraped ones that don’t. The model learns the distribution you show it, inconsistencies included.
• Your production logs are the best source — real inputs, with outputs corrected by someone who knows what good looks like. This is the same discipline as building an eval set from logs, and the two artifacts overlap: your eval set is the held-out slice of your training source, never a subset of it.
• The eval harness is a prerequisite, not a follow-up. A fine-tune without a baseline measurement is a vibe with a GPU bill. You need the number the base model scores on your task before you train, or “it seems better” is all you’ll ever have.

Beyond imitation: preference tuning

Supervised fine-tuning teaches the model to imitate examples. Some targets — “more helpful”, “less verbose”, “reason more carefully” — are easier to express as comparisons than as demonstrations. That’s preference tuning, and the 2026 landscape has settled into three tiers:

• DPO takes pairs — “this response is better than that one” — and derives the policy update in closed form. No reward model, no RL loop, stable to train. The default when you have preference pairs, which you can often mine from production (“user accepted / user regenerated”).
• GRPO is the one behind the reasoning-model wave: it needs a verifiable reward — the answer is right or it isn’t, the code passes tests or it doesn’t — and optimizes against groups of sampled attempts. When your domain has a checkable answer, this is the strongest tool in the box.
• Full RLHF with a learned reward model still wins on deep subjective alignment, and it’s the one tier that mostly stays in the labs — the operational complexity is rarely worth it for a product team.

The practical ladder: QLoRA SFT first, DPO layered on if you have preference data, GRPO when the reward is verifiable. Each stage keeps the previous one’s evals.

Adapters change how you deploy

The 50–500 MB adapter file has a serving consequence that’s easy to miss: modern inference servers can hold one base model and many adapters, routing each request to the right one. Ten clients, ten adapters, one set of base weights in VRAM — instead of ten fine-tuned model copies. Rolling out a new fine-tune becomes swapping a file, not redeploying a model, and rolling back is instant.

For self-hosted setups this is the difference between fine-tuning being a science project and being a product feature. It pairs naturally with the small-model router pattern: a fine-tuned 8B with your domain adapter routinely beats a generic model several times its size on your task — which is the entire economic argument in one sentence.

When not to fine-tune

The list from the decision post still rules, compressed: don’t fine-tune to inject facts (that’s retrieval — facts baked into weights go stale and can’t be audited), don’t fine-tune before prompting has plateaued on a measurement, and don’t fine-tune without those ~500 good examples. Fine-tuning changes how a model behaves, not what it knows — the teams that internalise that sentence skip the most expensive category of disappointment in applied AI.

The bottom line

Fine-tuning in 2026 is not the heroic, cluster-scale exercise the phrase still evokes. LoRA turned it into training a small difference against a frozen base; QLoRA fit that on one GPU; adapters turned the result into a deployable, reversible file. The hard part moved to where it always belonged: the 500 examples, the eval baseline, and the honesty to check whether the base model plus a better prompt was enough.

The machinery is now the easy half. The data discipline is the product.

Fine-tuning a model on your data is literally the first thing on our services page — if you have the task and the examples, we’ll bring the rest.