Custom AI Models: Prompting, RAG or Fine-Tuning? A Guide

“We want our own AI model.” That usually means one of three things with very different costs and timelines. Which one is yours?

Most businesses don’t need to train a model from scratch. The right order is: better prompts and instructions first, then RAG to answer from your own documents, and fine-tuning only if you need a fixed format or behaviour. Choose by measuring accuracy on real examples, not by guessing.

By Saman Ghahreman · Published: · Last updated: · 9 min read

AI agents, chatbots and automation

Custom AI Models: Prompting, RAG or Fine-Tuning? A Guide

“We want an AI model of our own.” We hear this often, but it usually means three different things with very different costs and timelines. This guide helps you work out exactly what you need.

Four ways to “customise” AI

ApproachWhat it solvesCost and timeWhen?
Better prompts and instructionsFormat, tone and answer qualityVery low; daysAlways start here
RAGAnswering from your documents and dataMedium; weeksWhen the model must know your knowledge
Fine-tuningFixed, repeatable behaviour, format or styleHigher; needs quality examplesWhen prompting and RAG weren’t enough
Training from scratch or classical MLForecasting and scoring on structured data; training a language model from scratch is rareHighOnly for special needs with plenty of data
RAG or fine-tuning? A simple rule
  1. The model doesn’t know something

    Like prices or a technical document: RAG.

  2. The model doesn’t behave right

    Like output format or tone: fine-tuning.

  3. Your knowledge changes often

    RAG; you only update the knowledge base.

The recommended order: simple to heavy

OpenAI’s official guidance on improving accuracy suggests the same path: start with a prompt, add fixed examples, then a retrieval step (RAG), and finally, if needed, fine-tuning. These methods can be combined. For example, one assistant can be fine-tuned and also answer from your documents.

RAG or fine-tuning? A simple rule

  • If the problem is “the model doesn’t know something” (prices, policies, technical documents): RAG.
  • If the problem is “the model doesn’t behave correctly” (output format, tone, fixed sorting): fine-tuning.
  • If your knowledge changes often: RAG, because you only update the knowledge base and don’t retrain the model.
  • If you have both problems: combine them.

What data do you need?

For RAG, your existing documents are a good start, as long as they are organised and current. For fine-tuning you need correct input-and-output examples. OpenAI’s guidance suggests starting with at least about 50 high-quality examples. Quality and variety matter more than the raw number.

How do you measure accuracy?

  1. Collect 30 to 100 real questions or tasks from your daily work, including hard cases.
  2. Define the correct answer or pass rule for each.
  3. Test and compare every approach on this same set.
  4. Sort the errors: not knowing, making things up, wrong tone.
  5. Before launch, review the responsible-AI checklist.

What if the model must also take actions?

When a model must work with tools and move through several steps, you enter the world of AI agents, where guardrails and access control matter even more.

Before deciding, check what a chatbot and AI model cost so your choice fits your budget.

Reduce the risk of building a model with a proof of concept before payment.

For a real example of AI running on the device itself, see offline face-recognition attendance: how Hazeri works.

Frequently asked questions

Should we train a language model from scratch?

Almost never for an ordinary business. The cost and data needed are huge, and prompting, RAG or fine-tuning an existing model usually reaches the same result.

Can RAG and fine-tuning be used together?

Yes, and it’s common: fine-tuning fixes behaviour and format, while RAG supplies current, company-specific knowledge.

Can you run the model on our own server?

If data privacy matters, open-source models can run on your server. The choice between an outside model and your own depends on accuracy needs, cost and how sensitive the data is.

How do we know the model works well?

With a test set of your own real tasks. Compare results before and after each change and sort the errors.

Sources

Tags: Custom AI model · RAG · Fine-tuning · Model development · Model evaluation

Get advice on Custom AI Model Development

One short consultation is enough to clarify the path, timeline and rough cost of your project.

My promise
If you have a problem you think AI might solve, write to me. I’ll tell you honestly whether it will; sometimes the answer is no, and hearing that early is cheaper.
Book a consultation