Blueprint: an AI Q&A app (RAG)

We are building a Q&A app that answers questions from your own documents. A plain large language model (LLM) only knows what it saw during training, so it cannot answer about your private handbook — and it may confidently make things up. RAG (Retrieval-Augmented Generation) fixes this: first retrieve the most relevant pieces of your documents, then let the LLM generate an answer using only those pieces.

RAG has two phases. An offline ingest phase reads your documents, turns them into vectors (lists of numbers that capture meaning) and stores them. An online query phase takes a user question, finds the closest stored chunks, and feeds them to the LLM as context. Here is the whole picture:

Five Cloudflare pieces work together. The Worker is the conductor; Workers AI provides the brains (both embeddings and the LLM); Vectorize is the searchable memory; D1 keeps the original readable text; R2 holds the raw source files.

Ingest runs once per document (or whenever it changes). We split the document into small chunks, embed each chunk with Workers AI, then store the vectors in Vectorize and the original text in D1. The id links the two: the same id lives in both stores.

At query time we embed the question with the SAME model used at ingest, search Vectorize for the top-k closest chunks, pull their original text from D1, and stitch them into a prompt. The LLM reads that context plus the question and writes a grounded answer.

Here is exactly what happens inside the Worker when a user asks one question — every hop between Workers AI, Vectorize and D1:

Below is a complete, runnable Worker with two routes: POST /ingest to index a document, and POST /ask to answer a question. Create the resources, define the table, wire the bindings, then add the code.

1. Create the index & database

   The index dimensions (768) must match the embedding model below. cosine is the recommended distance metric for bge models.

   bashcontent_copyCopy

   # Index sized for the bge-base embedding model (768 dims)
   wrangler vectorize create rag-index --dimensions=768 --metric=cosine
   
   # A D1 database to keep each chunk's original text
   wrangler d1 create rag-db

2. Create the chunks table

   Save this as schema.sql, then run: wrangler d1 execute rag-db --remote --file=./schema.sql

   sqlcontent_copyCopy

   CREATE TABLE IF NOT EXISTS chunks (
     id      TEXT PRIMARY KEY,
     doc_id  TEXT,
     text    TEXT NOT NULL
   );

3. Wire the bindings

   Bindings expose AI, Vectorize and D1 to your code as env.AI, env.VECTORIZE and env.DB — no API keys in your source. Replace database_id with the id printed by 'wrangler d1 create'.

   jsonccontent_copyCopy

   {
     "name": "rag-worker",
     "main": "src/index.js",
     "compatibility_date": "2025-06-01",
     "ai": { "binding": "AI" },
     "vectorize": [
       { "binding": "VECTORIZE", "index_name": "rag-index" }
     ],
     "d1_databases": [
       { "binding": "DB", "database_name": "rag-db", "database_id": "<your-d1-id>" }
     ]
   }

4. Ingest: chunk, embed, store

   One env.AI.run() call embeds every chunk at once. We upsert vectors into Vectorize (idempotent, so re-ingesting a doc is safe) and INSERT OR REPLACE the text into D1 under the same id.

   jscontent_copyCopy

   // POST /ingest  { docId, text }
   async function ingest(env, docId, fullText) {
     // 1) Split the document into small overlapping chunks
     const chunks = chunkText(fullText, 200, 40);
   
     // 2) Embed every chunk in one Workers AI call
     const { data } = await env.AI.run("@cf/baai/bge-base-en-v1.5", {
       text: chunks,
     });
   
     // 3) Save original text in D1, vectors in Vectorize
     const vectors = [];
     for (let i = 0; i < chunks.length; i++) {
       const id = docId + ":" + i;
       await env.DB.prepare(
         "INSERT OR REPLACE INTO chunks (id, doc_id, text) VALUES (?, ?, ?)"
       ).bind(id, docId, chunks[i]).run();
       vectors.push({ id, values: data[i], metadata: { docId } });
     }
     await env.VECTORIZE.upsert(vectors);
     return chunks.length;
   }
   
   // Split text into word chunks with a little overlap for context
   function chunkText(text, size, overlap) {
     const words = text.split(/\s+/);
     const out = [];
     for (let i = 0; i < words.length; i += size - overlap) {
       out.push(words.slice(i, i + size).join(" "));
     }
     return out;
   }

5. Ask: embed, retrieve, generate

   This is the heart of RAG. Embed the question, query() Vectorize for the top-5, load those chunks' text from D1, and pass them to the LLM as context. The system prompt forces the model to answer only from that context.

   jscontent_copyCopy

   // POST /ask  { question }
   async function ask(env, question) {
     // 1) Embed the question with the SAME model used at ingest
     const { data } = await env.AI.run("@cf/baai/bge-base-en-v1.5", {
       text: [question],
     });
     const queryVector = data[0];
   
     // 2) Retrieve the top-k most similar chunks from Vectorize
     const { matches } = await env.VECTORIZE.query(queryVector, {
       topK: 5,
       returnMetadata: true,
     });
   
     // 3) Load the original text of those chunks from D1
     const ids = matches.map((m) => m.id);
     const slots = ids.map(() => "?").join(", ");
     const { results } = await env.DB.prepare(
       "SELECT text FROM chunks WHERE id IN (" + slots + ")"
     ).bind(...ids).all();
     const context = results.map((r) => r.text).join("\n---\n");
   
     // 4) Generate an answer grounded ONLY in that context
     const out = await env.AI.run("@cf/meta/llama-3.1-8b-instruct", {
       messages: [
         {
           role: "system",
           content:
             "Answer using ONLY the context below. If it is not there, say you do not know.",
         },
         {
           role: "user",
           content: "Context:\n" + context + "\n\nQuestion: " + question,
         },
       ],
     });
     return out.response;
   }

6. Route the requests

   The default export dispatches POST /ingest and POST /ask to the two functions above and returns JSON.

   jscontent_copyCopy

   export default {
     async fetch(request, env) {
       const url = new URL(request.url);
   
       if (request.method === "POST" && url.pathname === "/ingest") {
         const { docId, text } = await request.json();
         const count = await ingest(env, docId, text);
         return Response.json({ ingested: count });
       }
   
       if (request.method === "POST" && url.pathname === "/ask") {
         const { question } = await request.json();
         const answer = await ask(env, question);
         return Response.json({ answer });
       }
   
       return new Response("POST JSON to /ingest or /ask", { status: 404 });
     },
   };

7. Deploy & try it

   Deploy, ingest a document, then ask about it. The answer should quote your document, not the model's training data.

   bashcontent_copyCopy

   wrangler deploy
   
   # Index a document
   curl -X POST https://rag-worker.example.workers.dev/ingest \
     -d '{ "docId": "handbook", "text": "Refunds are accepted within 30 days..." }'
   
   # Ask a question about it
   curl -X POST https://rag-worker.example.workers.dev/ask \
     -d '{ "question": "How long do I have to get a refund?" }'

• Dimensions must match: the index size, the embedding model, and the vectors you upsert all have to agree (768 here).
• Always embed questions with the same model used for documents — mismatches silently return irrelevant results.
• Vectorize stores numbers, not text. Keep the readable text in D1 (small chunks) or R2 (whole files) and join by id.
• Tune chunk size and top-k: smaller chunks are more precise, a higher top-k is more thorough but costs more and risks overflowing the context window.
• Put AI Gateway in front of Workers AI to cache, rate-limit and observe both the embedding and LLM calls.
• Re-ingest when documents change; upsert by a stable id so old vectors are replaced, not duplicated.