RAG can connect a language model to company knowledge, but its quality depends on source governance, access control, retrieval, citations and evaluation—not merely embeddings.

The short answer

Use retrieval-augmented generation when an answer must be grounded in current, citable sources. Do not use it to disguise poor document governance or to bypass the permissions enforced by the original business systems.

The next step is not choosing a tool. It is clarifying the decision, ownership and evidence that the team will accept.

Decision model

01. Sources

Select relevant documents, identify their owners and define the update and deletion cycle.

02. Access control

Preserve each user’s rights and filter retrieval before any content reaches the model.

03. Retrieval quality

Test segmentation, metadata, search and reranking against representative business questions.

04. Evaluation

Measure retrieval recall, answer support, citation accuracy, appropriate refusal and latency.

Applying the model

01. Starting context

Create a source register before indexing. Record the owner, permitted audience, update date, version, sensitivity and retention rule for every document. Retrieval-augmented generation does not resolve contradictions between policies or obsolete files; it can simply retrieve them faster. Without governance, a response may accurately cite a source that is no longer valid or is not authorised for that user, so source quality and access control must precede embeddings and prompt design.

02. Controlled execution

Design retrieval separately from generation. Test chunking, metadata, permission filters, lexical and semantic search, the number of candidates and reranking on representative questions. Display citations that let users inspect the supporting passage and document version. The model must be allowed to say that evidence is insufficient instead of filling gaps with a plausible explanation, and access rules must be enforced during retrieval rather than left to a prompt that can be ignored.

03. Useful evidence

Build the evaluation set from real questions and include direct answers, comparisons, unsupported questions, source conflicts and unauthorised requests. Measure whether the correct source was retrieved, whether every material statement is supported and whether permissions were respected. Evaluate retrieval and answer quality separately; otherwise the team cannot tell whether a failure came from search, context assembly or generation, and the next change becomes guesswork rather than a controlled improvement.

04. Decision threshold

RAG is ready for sustained use when sources have owners, updates and deletions are repeatable, permissions apply at query time and regressions can be detected. If the team frequently repairs documents only after incorrect answers expose them, the organisation first has a knowledge-management problem. A valuable pilot may therefore end with a cleaner source system and explicit governance instead of an immediate public launch, which is a valid outcome rather than a failed AI project.

Scenario and working plan

01. Diagnostic example

A knowledge base may contain current procedures, obsolete versions, management documents and information restricted to one team. If everything is indexed without metadata and filtering, an answer can combine incompatible sources or expose a valid passage to an unauthorised person. Build the source register and permission filters before embeddings. For an unsupported question, the correct behaviour is a refusal that explains what evidence is missing. Citations should let the user verify the statement, document version and right to use that source.

02. Implementation plan

Start the pilot with one bounded domain and questions collected from real users. Evaluate retrieval before generation: if the correct passage is absent, changing the response model will not solve the failure. Test chunking, metadata and permissions, then assess claims and citations. Add repeatable reindexing and deletion, monitor unanswered questions and retain verified feedback. Extend to another department only when source owners accept update responsibility and the access rules can be applied consistently to every query and cited document.

03. Decision log

To make the recommendations in “RAG for company data: sources, permissions and evaluation” traceable, open a simple decision log before the first change. Record the observed problem, baseline, hypothesis, owner, evaluation window and the condition for stopping or continuing. Evidence should come from sources suited to the topic, while technical indicators remain separate from commercial outcomes. The first measure reviewed is relevant evidence retrieved for each question, without treating it in isolation from data quality, total cost and downstream effects. This turns a favourable dashboard into an explainable decision rather than a conclusion based on intuition.

04. Review and next decision

At the end of the cycle, compare the result with the baseline and record what changed, what remains uncertain and which side effects appeared. Check explicitly whether “Sources have named owners and version information” and “Permissions are enforced during retrieval” are true. If the evidence cannot support a conclusion, keep the hypothesis open instead of declaring success. The risk “Indexing every file without selection” stays visible during review so that pressure to show progress does not replace analysis. Choose the next step only when the team can explain what it learned and why the new priority matters more than the alternatives.

Pre-implementation checklist

  • Sources have named owners and version information.
  • Permissions are enforced during retrieval.
  • Answers cite the exact evidence used.
  • The system refuses when reliable support is absent.
  • Evaluation includes unanswerable and adversarial questions.
  • Source updates and deletion are tested end to end.

What to measure

Metrics are defined before launch and separate technical signals from confirmed business outcomes.

  • relevant evidence retrieved for each question;
  • supported answers and accurate citations;
  • appropriate refusal and escalation rates;
  • latency, cost and user feedback.

Mistakes and limits

  • Indexing every file without selection.
  • Using one chunking strategy for every format.
  • Ignoring document-level access rules.
  • Evaluating only a handful of easy questions.
  • Keeping stale content after its source has changed.

Conclusion

A trustworthy RAG system makes both its evidence and its uncertainty visible. Quality comes from information governance and repeatable evaluation, not from the size of the vector index.