My First Useful RAG App in About 100 Lines

I had heard the term RAG often enough by this point, but I wanted to build the smallest version possible so I could understand what was actually happening.

The result was a tiny knowledge-base chat.

I paste some text into the app. The text is split into smaller chunks, each chunk is converted into an embedding, and when I ask a question the question is converted into an embedding too.

Then I compare them.

That is basically the whole retrieval part.

document
↓
split into chunks
↓
create embeddings

question
↓
create embedding
↓
find the most similar chunks
↓
send those chunks + question to the model

For the first version I did not even need a vector database. The embeddings could stay in memory and I could calculate cosine similarity myself. For a small learning project, that was enough.

The language model then receives only the most relevant pieces of the original text and answers using that context.

What I liked about building it this way was that RAG stopped looking like some special AI framework and became a very understandable pipeline. LangChain helped with things like text splitting and OpenAI integration, but the core idea was still simple enough to see directly.

I also finally understood why embeddings matter. I am not asking the model to “search” the document. I am first finding the pieces that are semantically close to the question, and only then asking the model to reason over those pieces.

For a prototype, that was enough to make RAG feel much less mysterious.