Fine-tuning vs. prompting vs. RAG

A general-purpose LLM wasn't trained specifically for your use case — it needs to be adapted somehow. Three fundamentally different techniques exist for this, and they solve genuinely different problems, not interchangeable versions of the same fix.

Prompting (also called "prompt engineering") means carefully crafting the instructions and examples sent as part of the input context — no training involved at all, purely inference-time (from the training-vs-inference lesson). This is the cheapest, fastest, and most flexible option to iterate on, and should almost always be tried first.

Prompting's real limit: it can only shape behavior using knowledge the model already has, or facts explicitly included in the prompt itself — it cannot teach the model new facts it was never trained on, and every token spent on instructions eats into the context window's finite budget.

RAG retrieves relevant documents (using the embeddings-based semantic search from two lessons ago) and inserts them directly into the prompt's context before generation, so the model answers based on real, specific, up-to-date source material rather than only what it happened to learn during training.

RAG is the right tool specifically when the need is fresh or proprietary knowledge the model was never trained on (your company's internal docs, today's data, content published after the model's training cutoff) — and it's also the most direct practical mitigation for hallucination covered in the previous lesson, since the model is grounded in a specific, checkable source rather than relying purely on trained-in patterns.

Fine-tuning takes an already-trained model and continues training it further on a smaller, task-specific dataset — genuinely changing its weights (a real, if smaller-scale, training run from the earlier lessons in this domain), not just changing what's sent at inference time.

Fine-tuning is the right tool specifically for teaching a consistent style, format, or specialized skill that's hard to fully capture through instructions alone — matching a very particular tone across thousands of examples, or reliably outputting a specific structured format. It is not a good tool for teaching new factual knowledge (RAG solves that far more directly and cheaply) and it is meaningfully more expensive and slower to iterate on than prompting, since each change requires an actual training run.

These aren't mutually exclusive — most serious production systems combine all three: a fine-tuned or carefully prompted model, with RAG providing grounded, current context, itself framed by a well-engineered system prompt. The mistake worth actively avoiding is reaching for the most expensive option (fine-tuning) first, when prompting or RAG would have solved the actual problem for a fraction of the cost and iteration time.