End-to-end implementation of retrieval-augmented generation using Cloudflare’s serverless vector database, SQL layer, and inference platform.
Architecture Overview
A minimal RAG system on Cloudflare needs three primitives: D1 for structured document metadata and chunk provenance, Vectorize for cosine-similarity search over embeddings, and Workers AI for both embedding generation and final answer synthesis. This stack runs entirely at the edge with no external services.
Store raw chunks and their metadata in D1 so you retain full text and can enforce access control or versioning. Keep only the vector ID, embedding, and a foreign key to D1 in Vectorize. At query time retrieve the top-k vectors, join back to D1 for the original text, then feed the context window to a Workers AI model.
See the production baseline in The Complete Cloudflare Production Setup before adding AI components.
Schema and Index Creation
Create a D1 table that records document identity, chunk index, and the original text. The Vectorize index is created once via the dashboard or wrangler and referenced by name inside the Worker.
CREATE TABLE chunks (
id INTEGER PRIMARY KEY,
doc_id TEXT NOT NULL,
chunk_index INTEGER NOT NULL,
text TEXT NOT NULL,
created_at INTEGER DEFAULT (unixepoch())
);Vectorize index dimensions must match the embedding model; @cf/baai/bge-small-en-v1.5 produces 384-dimensional vectors. Name the index “rag-vectors” and set metric to cosine.
Ingestion Pipeline
Split source documents into 400–600 token chunks with 10–20% overlap. Generate embeddings with Workers AI, write the text and metadata to D1 in a single transaction, then insert the vector with the D1 row ID as metadata. Use a Queue to decouple ingestion from the HTTP request when processing large corpora.
const embedding = await env.AI.run('@cf/baai/bge-small-en-v1.5', {text: chunk});
const {id} = await env.DB.prepare(
'INSERT INTO chunks (doc_id, chunk_index, text) VALUES (?, ?, ?)'
).bind(docId, i, chunk).run();
await env.VECTORIZE.upsert([{
id: `chunk-${id}`,
values: embedding.data[0],
metadata: {d1_id: id}
}]);Retrieval and Answer Generation
At query time embed the user question with the same model, run a top-5 Vectorize query, then SELECT the corresponding rows from D1. Concatenate the retrieved text into a prompt and call a chat model.
const qEmb = await env.AI.run('@cf/baai/bge-small-en-v1.5', {text: question});
const matches = await env.VECTORIZE.query(qEmb.data[0], {topK: 5});
const ids = matches.matches.map(m => m.metadata.d1_id);
const rows = await env.DB.batch(ids.map(id =>
env.DB.prepare('SELECT text FROM chunks WHERE id = ?').bind(id)
));
const context = rows.map(r => r.results[0].text).join('\n\n');
const answer = await env.AI.run('@cf/meta/llama-3-8b-instruct', {
messages: [{role: 'user', content: `Context:\n${context}\n\nQuestion: ${question}`}]
});Production Trade-offs and Limits
Vectorize indexes are eventually consistent; allow a few seconds between upsert and query in the same Worker invocation. D1 has a 1 GB free limit per database; monitor growth with a simple COUNT query and shard by tenant when you exceed a few hundred thousand chunks.
Workers AI has per-model rate limits and token caps. Cache frequent questions with a Durable Object or KV to stay under the free tier. See Building Production AI Agents for patterns that keep token usage predictable.
Cost scales linearly with embedding calls and Vectorize read units. For most internal tools the free allowances are sufficient until you reach thousands of daily queries.
Next Steps
Add reranking with a cross-encoder if recall is insufficient, or move to a larger embedding model once you outgrow bge-small. Instrument both Vectorize result quality and final answer latency with Workers Analytics. Once the basic loop is stable, layer on the event-driven patterns from Event-Driven SaaS Automation to handle document updates and deletions reliably.