RAG vs
Fine-Tuning.

Side by side

Start with neither

Before either, try a careful prompt with a few worked examples. Prompt engineering is free, instant to iterate, and resolves a surprising share of what teams assume requires training. The number of projects that fine-tuned when a better-specified prompt would have worked is considerable.

The reason to be disciplined about this ordering is that both alternatives create ongoing maintenance. A fine-tune is pinned to a base model version; a retrieval index needs re-embedding when the embedding model changes. A prompt has neither problem.

When to reach for RAG

Choose RAG when the gap is factual. Your internal documentation, current pricing, this quarter's policies, a customer's account history — none of it is in the model's weights and none of it should be. It also wins whenever you need to show sources, which covers most legal, clinical, financial, and support applications where an unverifiable answer is worthless.

The honest caveat: RAG systems fail at retrieval far more often than at generation. If the right passage is not in the context, the model will usually produce a confident wrong answer anyway. Budget effort for chunking strategy, hybrid keyword-plus-vector search, and a reranking stage — that is where the quality actually comes from, not the model choice.

When to reach for fine-tuning

Choose fine-tuning when prompting has plateaued on a behaviour you need reliably: a strict output structure, a house voice, a specialised classification judgement, or domain vocabulary the model handles awkwardly. It is also the right answer economically when a small fine-tuned model can replace a large general one on a high-volume narrow task, cutting inference cost substantially.

LoRA changed the calculus here. Training a small adapter against a frozen base is cheap, produces a small artefact, and lets you serve many task-specific adapters against one shared model. Fine-tuning is no longer a heavyweight commitment.

What fine-tuning will not do is make the model factually reliable. Training on question-answer pairs teaches it to produce confident answers in that shape without adding any verification mechanism, so it can hallucinate more assertively. Facts belong in retrieval.

The combination most production systems land on

Mature systems typically use both, and the division of labour is clean. Retrieval supplies the current, private, citable facts. A light fine-tune enforces the output contract and domain register so you are not spending hundreds of prompt tokens per request re-explaining the format.

A pragmatic sequence: prompt until it plateaus, add retrieval for the knowledge gap, measure, and fine-tune last — only for the behavioural residue that prompting could not deliver consistently. Whichever path you take, build the evaluation set first. Without a held-out set of real inputs with known good outputs, you cannot tell whether any of these changes helped.