A governed retrieval system that answers employee questions from internal docs — with permissions, citations, and an eval harness that catches drift.
Every large company has the same problem: the answer exists, but it is buried across a wiki, a dozen Slack channels, a SharePoint nobody trusts, and three people''s heads. A naive "chat with your docs" demo is easy; an enterprise-grade internal RAG system is not, because it has to respect document-level permissions, cite its sources, handle conflicting versions, and not hallucinate policy. The systems that survive contact with auditors share four traits: permission-aware retrieval (a user only retrieves what they can already see), mandatory citations, an explicit "I don''t know" path, and a continuous eval harness. Build the eval harness first — it is what separates a system you can trust from a demo you cannot.
Catalogue every knowledge source, its owner, freshness, and sensitivity. Decide what is in scope — and explicitly exclude what is stale or unowned.
Index documents with their access-control metadata so retrieval filters to what the asking user is already entitled to see. Permissions are a retrieval-time filter, not an afterthought.
Every answer links the exact passages it used. No citation, no answer — the model returns 'not found' rather than guessing.
Curate a golden set of 100-300 real questions with verified answers. Score retrieval recall and answer faithfulness on every change, in CI.
Launch to one team, capture thumbs-down with reasons, fix the underlying docs or retrieval, then expand. Treat bad answers as documentation bugs.
Use these templates as-is or customize for your business.
{"question":"...","expected_answer":"...","must_cite":["doc-id-1"],"acceptable_to_say_unknown":false,"owner":"team"}For each generated answer: extract every claim, verify each is supported by a cited passage. Faithfulness = supported claims / total claims. Block deploys that drop below 0.95 on the golden set.
Get a new AI workflow every week. Prompts, tool stacks, and ROI math included.
Retrieval-augmented generation: the agent answers strictly from a curated corpus of your documents and history. Cheaper, more controllable, and fewer hallucinations than open-ended generation.
Learn the agentic glossary →Where this workflow tends to break in production — and what to put in place before you ship it.
Permission leak — user retrieves a document they should not see
Mitigation: Enforce access control at retrieval time from the source system; never rely on the prompt to withhold content. Audit with red-team queries.
Hallucinated policy when retrieval finds nothing
Mitigation: Hard-require citations; an answer with no supporting passage returns 'not found' and offers to route to a human owner.
Silent quality drift as docs change
Mitigation: Run the golden eval set in CI on every index rebuild and model change; alert on faithfulness regressions.
Skip this if your underlying documentation is wrong or contradictory — RAG will faithfully retrieve the wrong answer. It is a retrieval system, not a fact-checker. Fix document ownership and freshness first.
A phased approach to get this workflow running and delivering ROI.
Days 1–30
Foundation
Days 31–60
Optimization
Days 61–90
Scale
Same workflow, tuned for your niche with tailored copy, examples, and ROI numbers.
None of the three big legal AI platforms publishes a per-seat price. Here is what each one does publish (research content, security commitments, plan terms) and how a 25-lawyer firm should structure the quote.
Only one of the three big enterprise AI assistants publishes a seat price. Here is what Glean, Microsoft 365 Copilot and Writer each charge for, what they include, and how to budget a 1,000-person rollout.
Seven vector database options priced from each vendor's own pricing page and docs, with a worked monthly bill for one million 1,536-dimension vectors and when plain pgvector is the right answer.
One practical AI workflow per week. No fluff.
Get the full guide with step-by-step setup, workflow templates, and copy-paste assets.