RAG keeps showing up in the same breath as context and memory tools I’ve already covered, which makes it easy to blur the three together. They’re solving genuinely different shaped problems, so it’s worth pulling apart.
RAG stands for Retrieval-Augmented Generation. The idea: instead of a model answering purely from what it learned during training, or from whatever you’ve manually pasted into the conversation, a RAG system searches an external knowledge source at the moment you ask, pulls back the specific pieces that look relevant, and hands only those to the model as part of its context before it generates an answer. The model isn’t trying to recall a fact from training — it’s reading a handful of retrieved documents and answering from what’s actually in front of it, closer to an open-book exam than a memory test.
Why this beats just training a bigger model or pasting more in: retraining a model to include new information is slow and expensive, and doesn’t help at all with anything that changes after training finishes — this week’s documentation update, a codebase that changed yesterday, a support ticket filed an hour ago. Pasting everything conceivably relevant directly into the prompt runs straight into the context math I covered separately — a bigger haystack costs more in tokens every single step, whether or not any given piece ends up mattering to this particular question. RAG’s whole bet is doing the search first, narrowing a large knowledge base down to a handful of genuinely relevant pieces, and only paying the context cost for those.
Where this differs from a memory tool like claude-mem: memory is about your own history with an agent — what happened in past sessions, decisions already made, context specific to your ongoing work. RAG is typically about a shared external knowledge base — documentation, a codebase, a set of support tickets — that isn’t really about your session’s history at all and would be exactly the same whether you’re the first person querying it or the thousandth. A tool can combine both, but they’re addressing different gaps: memory is “what did we already establish,” RAG is “what does this shared source of truth actually say.”
The frontier past basic RAG, worth naming since it showed up in this week’s trending projects: reasoning-based retrieval, where a system doesn’t just embed text and search for similarity, but reasons over a structured representation of the knowledge — a graph of how pieces connect to each other — to find an answer that requires connecting several pieces rather than matching one chunk that happens to look similar to the question. Plain RAG is good at “find the paragraph that answers this.” Graph-based approaches are aimed at questions where the answer isn’t sitting in any single paragraph, but has to be assembled from several connected ones.
The practical shorthand: if what you need is “pull the right facts from a large, shared source I don’t want to retrain a model on,” that’s RAG. If what you need is “remember what we already decided in earlier sessions,” that’s a memory layer. If what you need is “give the model everything at once and hope it finds the relevant part,” that’s neither — that’s just a bigger, more expensive prompt, with all the cost and none of the retrieval logic doing the actual filtering for you.
