I've been reading about RAG for a couple of days now, and honestly, the core idea clicked almost immediately: give an LLM some outside information, pull out the useful parts, and let it answer using that. Simple enough.
Then I actually looked at the pieces involved — embeddings, vector stores, retrievers, chunks, prompts — and it stopped feeling simple. Every article seemed to assume I already knew what half these words meant.
So yesterday I stopped reading and just built something: a small RAG app that answers questions about a YouTube video's transcript. Turns out that was the right call, because a lot of this stuff only made sense once I had to wire it together myself.
Here's roughly what the pipeline looks like:
It looks like a lot when you see it all at once. It's really not, once you walk through it one piece at a time — so that's what I'm going to do here.
Step one: get the actual content
A YouTube video isn't something you can search through directly, obviously. So the first real step is grabbing the transcript. That transcript is the raw material — the thing the whole system is actually built around.
If the video's an hour-long lecture, that's easily a few thousand words. Now say someone asks, "What did the speaker say about embeddings?" You really don't want to be stuffing the entire hour-long transcript into the LLM every time. You want to find the one part that's actually relevant. That's the whole problem the rest of this pipeline is solving.
Cutting the transcript into chunks
You can't just treat the transcript as one giant blob of text. It gets split into smaller pieces — chunks — so that later on, when a question comes in, the system can go find the relevant chunk instead of scanning the whole thing.
This part seems almost too obvious to mention, but it was actually the first moment where I realized RAG isn't really "an LLM thing." A surprising amount of the work — and honestly, a lot of what determines whether the final answer is any good — happens before the LLM is even in the picture.
Where it gets a bit strange: embeddings
Okay, so now you've got a pile of chunks. How do you search through them by meaning instead of just matching keywords?
That's what embeddings are for. An embedding model takes a chunk of text and turns it into a list of numbers — a vector. Something like:
"What is a vector database?" → [0.12, -0.43, 0.81, 0.27, ...]
Don't worry about what any individual number represents — I didn't either, and it's fine. What matters is that pieces of text with similar meaning end up with similar-looking vectors. So if the transcript says something like "a vector database stores numerical representations of information," and I ask "what is used to store these representations," the system can still connect the two even though the wording doesn't match. That's the whole trick — searching by meaning, not by exact words.
Storing all these vectors somewhere
Once every chunk has an embedding, you need somewhere to actually put them and search through them. I used FAISS for this project. At this point the data basically looks like:
- Chunk 1 → embedding
- Chunk 2 → embedding
- Chunk 3 → embedding
- Chunk 4 → embedding
FAISS handles the job of comparing vectors and finding the ones closest to whatever you're searching for.
The retriever — probably the part I learned the most from
Say I ask: "What did the video say about embeddings?"
Before anything else, the system has to figure out which chunks are actually worth looking at. That's the retriever's job — it takes the question, searches the vector store, and comes back with whatever chunks seem most relevant.
Roughly: question → retriever → vector store → relevant chunks.
This is where I finally understood the difference between a vector store and a retriever, which had been genuinely confusing me. The vector store is just where the vectors and documents live — the database, essentially. The retriever is the thing that actually goes and fetches relevant chunks for a specific query. Related, but not the same job.
And it turns out there's a whole world of ways to do retrieval — basic similarity search, MMR, multi-query retrieval, contextual compression, and more. I started poking at these yesterday too, but that's probably its own post.
Now we actually have something useful
At this point I've got two things: the user's question, and a handful of relevant chunks pulled from the transcript. So what happens next is pretty anticlimactic, honestly — we just hand both to the LLM. Something like:
Context: [relevant chunks from the transcript]
Question: What did the speaker say about embeddings?
Answer using the context above.
That's the "augmentation" part — we're basically telling the model, "here's what you need, now go answer the question." The model does that, and that's the "generation" part.
Zoomed all the way out, the whole thing looks like:
YouTube video → transcript → chunks → embeddings → FAISS → retriever → relevant chunks → prompt + question → LLM → answer
That's RAG. At least, the basic version of it.
Where the bigger picture started showing up
Before I built this, my mental model of RAG was basically: "put documents in a vector database, ask the LLM questions." That's it. That's all I thought there was to it.
But there's a lot happening in between those two ideas, and once you see it, you can't unsee it. What if the chunks are bad? What if the actually-relevant chunk never gets retrieved in the first place? What if the top results are all basically saying the same thing? What if the question itself is ambiguous? What if you retrieve something relevant, but 90% of it is noise and only one line actually matters?
None of that has anything to do with which LLM you're using. The quality of the answer depends just as much — maybe more — on what you feed the model in the first place. That was probably the single biggest thing I took away from actually building this instead of just reading about it.
Still a lot to figure out
What I've got right now is a pretty bare-bones pipeline. It works, but there's clearly more depth here, and a few things I want to dig into next:
- MMR — for when the retriever hands back five chunks that all say roughly the same thing
- Multi-query retrieval — for when a question can reasonably be interpreted a few different ways
- Contextual compression — for when a retrieved chunk is relevant, but only a sentence or two of it actually matters
- Hybrid search — combining semantic search with plain keyword matching, for when you need both
- Reranking — pulling in a larger set of candidates first (say, 20) and using a second model to figure out which ones are genuinely the best
I'll probably write about these as I work through them.
For now, though, I'm just glad the pipeline actually makes sense to me now. And the thing that got me there wasn't another article — it was building the thing and watching where it broke.