RAG vs. Fine-Tuning vs. Prompting: What Does a Meeting Assistant Actually Need?

Quick answer: Most meeting assistants need good prompting plus retrieval (RAG), not fine-tuning. The facts change with every meeting and differ per user, so they belong in a searchable index, not in model weights. Use prompting for single-meeting tasks, add RAG for questions across meetings, and fine-tune only to fix a consistent behaviour or format problem your evaluation proves prompting cannot.

Teams building meeting AI usually hit the same fork early: "Should we fine-tune a model on our meetings?" The question sounds reasonable, since the model does not know your customers, your codenames or what was decided last Tuesday. But the three techniques solve different problems, and mixing them up leads to expensive training runs that still answer "When is the launch?" with last month's date.

This article sorts meeting questions by which technique they need. Its companion, Build RAG over meeting transcripts and documents, covers the retrieval architecture in detail once you have decided you need it.

What do prompting, RAG and fine-tuning actually change?

• Prompting changes the instructions and the input. You give the model the transcript (or part of it), a task and an output format. Nothing is stored; every call starts fresh.
• RAG changes what the model sees at question time. A retriever finds relevant chunks from many meetings and documents, and the model answers from them. The knowledge lives in an index you can update, filter and delete from. The original formulation is Lewis et al.; the RAG survey by Gao et al. maps the later variants.
• Fine-tuning changes the model's weights by training on examples. Parameter-efficient methods such as LoRA train small adapter matrices instead of the whole model, which makes it practical on a single GPU. Fine-tuning is good at teaching a behaviour: a format, a style, a classification habit.

The short version: prompting tells the model what to do, RAG tells it what is true right now, and fine-tuning changes how it tends to behave.

Why is the choice different for meeting assistants?

• The knowledge changes daily. Every meeting adds facts and some reverse earlier ones. A model fine-tuned on last quarter's calls has last quarter's answers built in, and retraining on every new meeting is not realistic.
• Each user sees different meetings. A board discussion or a hiring panel must not leak to someone who was not invited. With RAG you filter by access before retrieval. Knowledge trained into shared weights cannot be filtered per user.
• Deletion has to work. When a recording is deleted, its content should stop showing up in answers. Removing a row from an index is simple; removing what a model learned from it means retraining.
• Answers need proof. People act on "we agreed to a 48-hour SLA" only if they can check the moment it was said. RAG can cite the meeting and timestamp; a fine-tuned model's memory cannot point to a source.
• Transcripts are noisy. Crosstalk, misheard names and half sentences are normal. Training on raw transcripts teaches the noise as well as the content.
• Long meetings. A single transcript often fits in a modern context window, but a quarter of meetings does not, and models use long inputs unevenly: Lost in the Middle found information in the middle of long contexts is used less reliably than at the start or end.

What does the research say about fine-tuning for knowledge?

Two papers are worth reading before you plan a training run. Fine-Tuning or Retrieval? (Ovadia et al.) compared unsupervised fine-tuning with RAG on knowledge-intensive tasks and found RAG consistently did better, both for facts seen in pre-training and for new ones; they also report that models struggle to learn new facts through unsupervised fine-tuning. Gekhman et al. found that fine-tuning examples with new knowledge are learned slowly, and once learned they increase the model's tendency to hallucinate.

Practitioners report the same thing. In a Hugging Face forum thread on adding domain knowledge via fine-tuning, a user describes the model picking up some domain knowledge while losing general ability, and a reply argues that the model mostly learns the format of the training data rather than the knowledge.

Fine-tuning and RAG are not exclusive, though. RAFT fine-tunes a model to answer from retrieved documents and to ignore the ones that do not help, which is a behaviour, not a fact.

Which meeting questions need which approach?

Here is how common requests map, with invented examples:

• "Summarise this call" / "List the action items from today's stand-up." Prompting. The whole transcript is the context; you need a clear instruction and an output schema.
• "What did we tell Acme about pricing over the last three calls?" RAG. The answer spans meetings, needs date and customer filters, and should cite each call.
• "Is the May launch date still current?" RAG over transcripts plus a structured decision record, sorted by date, so a later reversal wins. See multi-meeting RAG for decisions and commitments.
• "Write a MEDDPICC deal report from this discovery call." Prompting with a template and a schema first. Fine-tuning becomes worth it only if, at high volume, a smaller model must produce the format reliably.
• "What does 'Project Bluebird' mean?" RAG or a glossary in the prompt. Internal codenames change; do not train them into weights.
• "Answer in our team's house style, always with owners and dates." Prompting first; fine-tuning if prompting cannot hold the style consistently.

How should you decide, step by step?

1. Write down the questions. Collect 50-200 real questions from users and tag each: single-meeting, cross-meeting, people ("who owes what"), decision status, report format, unanswerable.
2. Build a prompting baseline. For single-meeting tasks, pass the transcript with speaker names and timestamps, a precise instruction and a JSON output schema. Measure it.
3. Add retrieval where the answer is outside one transcript. Index chunks of speaker turns with hybrid search and permission filters, and require citations. Measure again on the cross-meeting questions.
4. Constrain the output. Many "the model is bad" problems are format problems: missing owners, invented dates, free text where you need fields. Fix them with schemas, examples in the prompt and validation before you consider training.
5. Fine-tune only for a measured behaviour gap. If the error analysis shows a stable pattern prompting cannot fix (format drift in a small model, a classification the base model gets wrong), train a LoRA adapter on examples that include retrieved context, so the model learns to use sources rather than memorise facts.
6. Re-run the same evaluation. Keep RAG as the source of truth for facts, compare all variants on the same test set, and keep the cheapest one that meets your bar.

A routing table makes the decision explicit in code. This is an illustrative configuration, not a library format:

{
  "single_meeting_summary": {"approach": "prompt", "context": "full_transcript", "output": "summary_schema"},
  "action_items":           {"approach": "prompt", "context": "full_transcript", "output": "action_item_schema"},
  "cross_meeting_question": {"approach": "rag", "filters": ["allowed_meetings", "date_range"], "cite": true},
  "decision_status":        {"approach": "rag", "sources": ["decision_records", "transcripts"], "sort": "date_desc", "cite": true},
  "deal_report":            {"approach": "prompt", "template": "meddpicc", "fine_tune_candidate": true},
  "unanswerable":           {"approach": "rag", "fallback": "Not found in the selected meetings."}
}

If you do fine-tune, train behaviour with the context included. A record in the conversational prompt-completion format that the TRL SFTTrainer docs describe might look like this (invented content):

{
  "prompt": [
    {"role": "system", "content": "Answer only from the excerpts. Cite [id] after each claim. If the excerpts do not answer, say so."},
    {"role": "user", "content": "Excerpts:\n[M2 09:12] Priya Shah: Launch moves to June 3.\n[M1 22:40] Tom Becker: We launch May 20.\n\nQuestion: When is the launch?"}
  ],
  "completion": [
    {"role": "assistant", "content": "The launch is on June 3 [M2 09:12]. It was earlier planned for May 20 [M1 22:40] and was moved in the later meeting."}
  ]
}

The adapter itself is a few lines with PEFT; this follows the quickstart in the PEFT README:

from transformers import AutoModelForCausalLM
from peft import LoraConfig, TaskType, get_peft_model

model_id = "Qwen/Qwen2.5-3B-Instruct"
model = AutoModelForCausalLM.from_pretrained(model_id)
peft_config = LoraConfig(r=16, lora_alpha=32, task_type=TaskType.CAUSAL_LM)
model = get_peft_model(model, peft_config)
model.print_trainable_parameters()
# train on records like the one above (e.g. with TRL's SFTTrainer), then:
model.save_pretrained("meeting-answer-lora")

The facts in that training record (June 3, May 20) are throwaway examples; what the adapter should learn is the pattern of citing, preferring the later statement and flagging the change. The PEFT quicktour covers saving and loading adapters, and the earlier guide on fine-tuning an LLM on one GPU walks through a full run.

What are the trade-offs and pitfalls?

• Cost moves around, it does not vanish. RAG adds retrieval and longer prompts to every query. Fine-tuning adds training runs, a dataset to maintain and a model to host. Prompting alone is cheapest to build but can be expensive per call with very long transcripts.
• Stuffing everything into the context is not RAG. Pasting ten transcripts into a long-context model avoids an index but loses precision, citations and permission control, and costs more per question.
• Fine-tuning on transcripts leaks across users. Anything in the training set can surface for anyone using the model. If you must train on meeting data, use only data every user of that model may see.
• Forgetting. Narrow fine-tuning can degrade general ability; evaluate on general tasks as well as your target task.
• Retrieval fixes need retrieval changes. If the right chunk is never retrieved, no amount of prompting or fine-tuning the generator helps. Check retrieval recall first.
• Fine-tune the retriever before the generator. If misses come from domain vocabulary, fine-tuning the embedding model with sentence-transformers keeps knowledge in the index and adjusts only how text is matched.

How do you evaluate the options?

Run every variant (prompt only, prompt plus RAG, RAG plus fine-tuned generator) against the same tagged question set and compare:

• Answer correctness against a reference answer, per question type.
• Faithfulness: are the answer's claims supported by the provided context? Ragas defines this alongside context precision and recall, and is available on GitHub.
• Citation accuracy: does each cited meeting and timestamp support the claim?
• Freshness: on questions whose answer changed in a later meeting, does the system return the latest one?
• Format validity: the share of outputs that parse against your schema.
• Abstention on unanswerable questions, plus latency and cost per query.

Watch freshness and abstention closely for any fine-tuned variant: those are the metrics where knowledge baked into weights tends to show up as confident, outdated answers. The hallucinations article in this series covers more checks.

How does Acta approach it?

Acta's product shows the split described here. Ask Acta answers questions across the meetings and files you choose with retrieval-augmented generation over their transcripts, and answers cite the meeting. The decision log tracks decisions across meetings and keeps the history when a later meeting changes or reverses one, with the quote and the moment it was said. Report templates read a meeting and write a structured report in a role's format, such as MEDDPICC deal reports or incident post-mortems. Transcripts include speaker names from the meeting and the calendar invite, transcription happens after the meeting, and AI assistants connected to Acta only see meetings the user can see. See the meeting decision log for how decision history is presented.

Frequently asked questions

Can I just fine-tune a model on all our meeting transcripts?

You can, but it is a poor fit for facts. Research such as Ovadia et al. found RAG more effective for knowledge, and fine-tuned facts go stale, cannot be cited, cannot be filtered per user and are hard to delete.

When is fine-tuning worth it for a meeting assistant?

When you have a stable, high-volume task with a fixed format (for example a specific report structure), prompting with a schema still fails often enough to matter, and you want a smaller, cheaper model to do it. Train on behaviour, with retrieved context in the examples.

Is a long context window a replacement for RAG?

For one meeting, often yes: pass the transcript and prompt well. Across many meetings, no: you still need filtering by access and date, citations and a way to surface the latest decision.

Should I fine-tune the embedding model or the LLM first?

If retrieval misses domain terms, fine-tune or swap the embedding model first, and add a reranker. It is cheaper and keeps facts in the index.