Production AI systems and digital engineering for enterprise teams.Explore our work
Retrieval design, chunking strategy, and access control for knowledge bases at scale.
Article
RAG fails in enterprises when retrieval is an afterthought. Security, freshness, and relevance matter as much as the model. This guide covers ingestion through access control, evaluation, and production debugging—from empty index to audited answers.
Audience
ML & platform engineers
Stage
Design → production
Focus
Retrieval & ACLs
Prerequisites
Document sources identified
You connected a vector database to a folder of PDFs and got impressive demo answers. Legal asks who can see what. IT asks how deletes propagate. Support gets a wrong answer with no way to see which chunks were retrieved.
How to use this guide
RAG is a data platform problem first. Models summarize; retrieval decides what truth the user is allowed to see.
Follow the phases below from scoping through steady-state operations. Each section builds on the previous—skip ahead only if your foundations are already in place.
Concepts, tradeoffs, and implementation notes from programs we've run.
01
Inventory systems of record—SharePoint, Confluence, tickets, CRM—and how permissions are expressed. Your retrieval layer must enforce the same ACL model, not a flattened copy of documents.
Define freshness SLAs per source: real-time for tickets, hourly for wikis, daily for legal libraries. Deletions must remove or tombstone embeddings, not leave stale chunks live.
02
Chunk by semantic boundaries where possible—headings, sections, slides. Tables, code, and numbered policies need specialized parsers; naive fixed-size windows break compliance answers.
Store metadata rich enough to filter: tenant, product line, doc type, effective date, and language. Metadata filters often beat bigger embedding models for precision.
03
Build eval sets from support tickets, sales engineer FAQs, and onboarding questions— not synthetic trivia. Measure recall@k, citation accuracy, and refusal correctness when no chunk should match.
Run evals when ingestion rules change, not only when the LLM version changes.
04
Log query, retrieval set, rerank scores, model version, and final answer with a shared trace ID. Support should replay why an answer appeared without reproducing production manually.
Add feedback loops: thumbs down should attach to traces for weekly quality review and dataset growth.
A phased path you can adapt to your team size and risk profile.
How we deliver
RAG is a data platform problem first. Models summarize; retrieval decides what truth the user is allowed to see.
Use before you expand scope or declare general availability.
Source systems and ACL mapping documented
Chunking strategy per document type
Sync jobs for upsert and delete
Tenant-scoped retrieval API
Golden eval set in CI
Trace logging with retrieval replay for support
Key takeaways
Topics & tools
Keep reading
More notes on engineering, AI, and delivery from our team.
Tell us what you're building—we'll share relevant experience from programs like this.
