Preparing your knowledge base for an AI agent

To prepare a knowledge base for an AI agent, audit every source you plan to use, remove duplicates and stale or conflicting articles, pick one source of truth per topic, and rewrite the important articles so each one states a single rule with its conditions and exceptions spelled out. Then give every topic an owner and a review date, because an agent repeats whatever the knowledge says, including what went out of date last quarter. This playbook works for any AI support agent: audit, decide, restructure, rewrite, then own.

​

Why this matters more for an AI agent than for a help center

A human reading your help center skims, notices that one article looks old, and asks a colleague. An AI agent does not have that instinct. If two articles disagree, it can pick either one. If the real answer is “it depends on the plan”, and the article never says which plan, the agent has to guess or escalate. If the answer only exists in a screenshot, the agent may never see it. Most of the quality problems teams blame on “the AI” turn out to be knowledge problems: a refund window that changed but was updated in only one of three articles, a regional exception that lives in a support lead’s head, or a “contact us” link where the answer should be. Fixing the knowledge first is cheaper than tuning prompts around it.

​

The process at a glance

​

Step 1: Audit what you have

​

Inventory every source

List every place an answer can come from today, not only the public help center. A typical inventory includes:

• Public help center articles
• Internal wikis and runbooks your agents use
• Macros and saved replies in your helpdesk
• Policy documents (terms, refund policy, fee schedules)
• Product release notes and changelogs
• Files: PDFs, slide decks, spreadsheets

For each source, record the owner, the last real update, the audience (customers, internal, both) and whether you plan to give it to the agent.

​

Find duplicates, stale and conflicting articles

Search the inventory topic by topic. For every high-volume topic (refunds, cancellations, account access, fees, delivery times), collect every article and macro that answers it and read them side by side. You are looking for:

• Duplicates: two or more articles that answer the same question. Keep one, merge anything unique into it, retire the rest.
• Stale content: old product names, retired plans, screenshots of a UI that changed, dates in the past.
• Conflicts: two sources that give different answers for the same situation. These are the most dangerous, because the agent may quote either one.

A useful trick: write the question a customer would ask, then write down every answer your sources give. If you get more than one answer, you have a conflict to resolve.

​

Find gaps from ticket data

Your tickets show what customers actually ask. Export a recent sample of resolved tickets, group them by topic, and check each topic against the knowledge you have. Mark a gap when:

• Agents answer it often, but no article covers it.
• The article exists but agents routinely add information that is not in it.
• The answer is “it depends” and the dependency is not written down.

You do not need to fill every gap before launch, but you do need to know where they are, so you can route those topics to a human until they are covered.

​

Step 2: Decide the source of truth for each topic

Before anyone rewrites an article, decide which source wins for each topic. Without this decision, the cleanup produces a new set of conflicts. When two sources conflict and nobody can say which is right, that is a policy question, not a writing task. Escalate it to the policy owner and keep the topic routed to humans until it is answered.

​

Step 3: Structure articles for retrieval

• One topic per article. An article titled “Billing FAQ” with fifteen questions is hard to retrieve and hard to keep current. Split it into one article per question or per closely related set.
• Make conditions and exceptions explicit. Who does this apply to, under what conditions, and what are the exceptions? Write them as conditions, not hints.
• Add dates and versions where they matter. If a policy changed on a date, say which version applies to orders, accounts or claims before and after that date.
• Scope by region and plan. If the answer differs for the EU and the US, or for a premium plan, say so in the article, or keep separate articles per region or plan and make sure the agent can tell which one applies.
• Use the words customers use. Titles and headings in your internal vocabulary (“Chargeback reversal flow”) miss customers who write “my refund disappeared”. Pull real phrasings from tickets.

​

Step 4: Write for AI and humans at the same time

Good articles for an AI agent are also better for people. The rules are the same: clear, specific and complete. State the rule first, then the exceptions. Compare: The “after” version gives a rule, the exceptions, and a concrete next step. An agent can answer every common variant from it, and knows when to hand over. Avoid “contact us” dead ends. “Contact support for details” tells the agent nothing it can say. If the answer genuinely requires a person, write down when and why (“Account ownership changes need identity verification by the support team”) so the agent escalates with a reason instead of guessing. Do not bury answers in images or scanned PDFs. Text inside screenshots, diagrams and scanned documents may not be readable by the agent. Put every fact the agent needs in plain text: tables as real tables, steps as numbered text, fees as text rather than an image of a fee table. Write numbers and limits once. If a fee or limit appears in five articles, it will drift. Keep it in the source of truth and refer to it.

​

Step 5: Separate internal-only from public knowledge

Some knowledge should guide the agent without being quoted to customers: internal escalation paths, fraud signals, negotiation limits, how to handle an abusive contact. Decide for each topic:

• Public: safe to quote to customers word for word.
• Internal guidance: shapes how the agent behaves, never quoted (“Do not offer a goodwill credit above the example limit without a human”).
• Human-only: stays out of the agent’s knowledge entirely (for example, security investigation procedures).

Keep these clearly separated in your system, with different folders, labels or visibility settings, so a public article never inherits internal-only text by accident. If you operate in a regulated industry, check your regulator’s rules and your own compliance team’s guidance on what may be disclosed and how.

​

Step 6: Own it after launch

A clean knowledge base goes stale within months without owners. Set up:

• A named owner per topic. The person who approves changes to the source of truth, usually the policy owner, not the person who writes the article.
• A change trigger. When a policy, price or product changes, the owner updates the source of truth first, then the derived articles, on the same day.
• A review cadence. High-risk topics (fees, refunds, eligibility, compliance wording) reviewed on a short cycle, such as monthly; everything else on a longer one. Record the review date on the article.
• A feedback loop from conversations. When the agent answers wrongly or escalates for lack of knowledge, someone turns that into an article fix or a new article, and checks for the same mistake elsewhere.

​

Common mistakes

• Connecting the whole help center on day one and fixing conflicts after customers find them.
• Rewriting articles before deciding the source of truth, which produces new conflicts.
• Keeping one long FAQ page per area instead of one topic per article.
• Leaving the real answer in a macro or a support lead’s head.
• Treating the cleanup as a one-off project with no owner afterward.

​

Audit checklist

​

Doing this in Fini

In Fini (usefini.com), the inventory and cleanup steps above map to these features:

• Fini’s knowledge model treats connected content as candidates: the agent answers from approved Articles only, so the source-of-truth decision maps directly to what you publish there. Fini does not chunk documents or use embeddings. It uses LLMs to reason over a knowledge graph built from your approved knowledge.
• Connect your inventory under Sources, and use Refresh and the Diff modal when a source changes so updates flow into Review instead of going stale.
• Use Magic Articles with Detect Duplicates enabled and Suggest for Review as the status to draft structured articles from messy content without publishing duplicates.
• Work conflicts, gaps and duplicates in the Review Queue, filtered by Type (New, Update, Duplicate); background AI drafts always land in In Review for a human.
• Keep internal guidance in Agent Instruction and control customer visibility with Public in helpcenter in Articles.
• Before you publish a large article change, run the affected Test Suite cases with those article drafts selected in the run dialog. Selected drafts replace their published versions for that run only, and nothing is published.
• For the full cleanup workflow, see How Fini handles messy or conflicting docs.

​

Related