RAG, explained: how an AI support agent actually knows your store

A language model on its own is a brilliant, confident stranger. It has read most of the internet, but it has never seen *your* returns policy, *your* order #10482, or the fact that you stopped shipping to Switzerland last month. Ask it a customer’s question and it will answer fluently, and sometimes it will be fluently wrong. RAG is how you fix that.

What RAG means

RAG stands for retrieval-augmented generation. The idea is to *retrieve* relevant information from your own data first, then hand it to the model as context so its answer is *generated* from your facts rather than its memory. Instead of "answer this from what you happen to know," the instruction becomes "here is the relevant passage from our policies, answer using only this."

That one change is the difference between an agent that guesses and an agent that cites. For support, where a wrong answer about a refund or a warranty is a real cost, grounding is not a nice-to-have, it is the product.

How it works, step by step

The mechanics are simpler than the acronym suggests. There is an offline step where you prepare your knowledge, and an online step that runs on every customer message.

Preparing your knowledge (once)

1. 01Chunk. Your documents, policies, FAQs, product pages, past tickets, are split into small, self-contained passages a few hundred words each.
2. 02Embed. Each chunk is run through an embedding model that turns its meaning into a vector, a list of numbers where similar meanings land close together in space.
3. 03Index. Those vectors are stored in a vector database so they can be searched by meaning, not just keywords.

Answering a question (every message)

1. 01Retrieve. The customer’s question is embedded the same way, and the database returns the handful of chunks whose meaning is closest, the passages most likely to contain the answer.
2. 02Augment. Those chunks are inserted into the prompt alongside the question and instructions about tone and boundaries.
3. 03Generate. The model writes the reply using the retrieved context, so the answer reflects your actual store.

What to feed an ecommerce support agent

RAG is only as good as its source material. For an ecommerce brand, the knowledge divides into two kinds, static reference and live, per-customer data, and a good system uses both.

• Policies and FAQs. Shipping, returns, warranty, sizing, care instructions. The evergreen answers that drive the majority of tickets.
• Product catalogue. Descriptions, materials, compatibility, specs, so the agent can answer "will this fit my…" questions accurately.
• Historical tickets. Past resolutions teach the agent how your team actually handles the tricky cases, in your voice.
• Live order data via tools. Order status, tracking, and account details are not "retrieved" from a document, they are fetched in real time through a secure tool call, then reasoned over. RAG for the reference knowledge, tools for the live facts.

Where naive RAG goes wrong

RAG is not magic, and the difference between a demo and a production agent is in handling its failure modes. The ones we design around:

• Bad chunking. Split a policy in the wrong place and the retrieved passage is missing the exception that mattered. Chunk boundaries are a design decision, not a default.
• Retrieving the almost-right thing. Semantic search can return a passage that is topically close but factually wrong for this case. Re-ranking and tighter retrieval fix it.
• Stale index. If the source changed but the index did not, the agent grounds confidently on last month’s policy. Freshness is an operational discipline.
• No grounding guardrail. A well-built agent is instructed to say "let me hand you to a human" when the retrieved context does not actually contain the answer, instead of filling the gap with a guess.

“A hallucinated answer about a refund is not a cute AI quirk. It is a chargeback, a bad review, and a lost customer. Grounding is the whole game.”

RAG, the wiki, and the bigger picture

RAG is one half of a good support agent. The other half is a small, always-loaded core of curated knowledge, the approach we cover in our piece on Karpathy’s wiki idea. Put simply: the wiki is what the agent always knows; RAG is what it looks up on demand. Combine them and you get an agent that is both consistent and deeply informed.

This is the core of what we deliver, a grounded AI support agent for Gorgias and Zendesk, trained on your policies, your catalogue, and your history, with real-time order lookups and clean escalation to your team. If your support inbox is a bottleneck, book a call and we will scope it.