1. Choose the source collection.
Start with documents that have an owner and a defined audience: internal policies, product manuals or operating instructions. Identify the authoritative copy of each document and remove superseded versions from the searchable collection. If conflicting guidance is intentional, preserve the context that explains which team, product or location each document covers.
A searchable folder is not automatically a dependable knowledge base. Drafts, attachments and duplicate exports can give an assistant contradictory evidence. Decide who can add material and who approves it. Record a document's title, effective date, owner and access level as metadata, meaning descriptive information stored alongside its content.
2. Prepare readable text.
Optical character recognition, or OCR, converts text in scanned images into machine-readable text. It can misread characters, omit footnotes and lose table structure. Compare extracted content with the original, particularly where a number, qualification or exception changes the meaning. A searchable PDF can still have an incorrect reading order.
Divide long documents into passages that retain enough context to stand alone. This process is commonly called chunking. A passage containing an exception should remain connected to the rule it qualifies. Preserve headings and page references so a reviewer can find the source. There is no universal passage length that suits every collection.
3. Understand retrieval before generation.
Retrieval-augmented generation, or RAG, finds relevant material and supplies it to a language model before an answer is generated. Embeddings are numerical representations used to compare the similarity of text. Similarity search can find related wording; keyword search can be more useful for exact product codes, policy identifiers and unusual terms.
A combined search approach may be appropriate when a collection contains both descriptive prose and precise identifiers. Evaluate retrieval independently: did the system find the passage containing the answer? If it did not, changing the writing instructions will not resolve the underlying problem. Fine-tuning changes model behaviour through additional training; it is not a replacement for controlled retrieval of changing documents.
4. Preserve each user's access.
Apply permissions before restricted passages reach the model. Hiding a citation afterwards does not undo the disclosure of information in an answer. Connect the assistant to the organisation's identity system and establish how access changes propagate to the search index, cached results and stored conversations.
Microsoft SharePoint and Google Drive both organise documents through sharing permissions, but an additional retrieval system must deliberately preserve those boundaries. Do not assume that copying documents into an index also copies every access rule. Check group membership, inherited permissions and documents shared by direct link as part of the integration design.
5. Make answers inspectable.
Require the assistant to distinguish supported answers from questions the collection cannot answer. Provide source links to specific documents or passages, and let people inspect the text used. A citation is useful evidence for review, but it does not prove the answer accurately represents that evidence. The model may omit an exception or attach a relevant-looking source to an unsupported conclusion.
For policies involving employment, legal duties or financial decisions, keep the appropriate human specialist in the process. The assistant can help locate and summarise information; it should not silently become the authority that decides an individual's circumstances. Explain how a user can report an incorrect answer and reach the document owner.
6. Evaluate and maintain the collection.
Build questions covering straightforward answers, ambiguous wording, conflicting documents and missing information. Include questions whose correct outcome is a refusal or a request for clarification. Review whether answers are supported, whether citations point to the right passages and whether restricted content stays restricted.
When a source changes, update the index and check that the previous material is no longer being served. Assign responsibility for document retirement, ingestion failures and permission updates. A document AI engagement should include this maintenance route as well as the assistant itself. Use the evaluation guide to define acceptance criteria before expanding the source collection.