Best Prompt for RAG
Answer the user’s question using ONLY the retrieved context provided below, without exception. If the context does not contain enough real information to answer, say so explicitly and clearly rather than using outside knowledge. Cite which piece of context supports each specific claim you make in your answer.
A good RAG system prompt explicitly instructs the model to answer only from retrieved context and to say so when that context is insufficient — without this constraint, models tend to silently fall back on their own training knowledge, undermining the whole point of retrieval.
What this produces
A system prompt that keeps the model grounded in retrieved context, with citations back to source chunks and explicit acknowledgment when the context doesn’t cover the query — rather than confident-sounding answers that aren’t actually grounded.
Variations
Two ready-to-copy versions for different RAG setups.
Answer the user’s question using ONLY the retrieved context provided below. If the context does not contain enough information to answer, say so explicitly rather than using outside knowledge. Cite which piece of context supports each specific claim you make.
Answer primarily from the retrieved context provided below. If you supplement with general knowledge, clearly label which parts come from context versus general knowledge, so the distinction is never ambiguous to the reader.
If this is useful, you may also want Use Linear Plugin for This Prompt or ChatGPT Prompt for Coding.
FAQ
Why does my RAG system still hallucinate even with good retrieval?
Often the system prompt doesn’t explicitly restrict the model to retrieved context — without that constraint, the model can still draw on its own training knowledge and blend it in silently.
Should citations always be required?
For anything where accuracy matters, yes — requiring citations back to source chunks both improves grounding and gives you a way to verify claims after the fact.
How do I test if grounding is actually working?
Query with something you know isn’t in your retrieval set — a properly grounded system should say it doesn’t have enough information, not answer anyway.
