Decide Which Business Sources Are Ready for an AI Knowledge Collection
Prepared with AI assistance by Alactic Editorial. Examples identified as hypothetical are illustrative, not customer results.
A team uploads a service guide, a presentation, meeting notes and a price sheet. All four discuss the same service, but only one may be suitable for answering customers’ questions. The presentation might omit exceptions, the notes might record an unresolved suggestion, and the price sheet might still be awaiting approval.
To assess source quality for an AI knowledge base, decide what questions the collection may answer, then require evidence that each source is authoritative, sufficiently complete, approved and suitable for that use. Give every candidate one of three decisions: admit for a defined scope, hold for repair, or exclude. Do not let a strong result in one area compensate for a failure in another.
The practical work is not assigning every file a quality score. It is recording why a particular version can support particular answers—and what it cannot support.
1. Define the answering job before reviewing files
A document cannot be judged fit without a stated use. A high-level presentation may be adequate for an internal overview but inadequate for deciding whether a customer qualifies for a service. A current internal procedure may be authoritative yet unsuitable for customer-facing answers because it includes staff-only instructions.
Write a short admission brief before opening the upload folder. Include:
- Audience: Who will receive answers, and what information may they receive?
- Allowed questions: Which operational topics should the collection support?
- Excluded questions: Which topics must remain outside its remit?
- Required detail: Must answers include eligibility, exceptions, effective dates, prerequisites or escalation instructions?
- Authority: Who can approve source material for these topics?
Hypothetical admission brief: A customer-support collection may explain service eligibility, preparation requirements and standard exclusions. It may not quote prices, approve exceptions or disclose internal account-handling instructions. The service owner approves operational guidance; the commercial owner approves pricing separately.
This boundary makes review concrete. The reviewer is no longer asking whether a document looks useful. They are asking whether it supports an authorised set of answers.
2. Apply four admission gates, not an average score
Use four gates to examine each candidate. A source must pass every applicable gate for the scope in which it will be used.
Provenance: can you trace the source?
Identify the owner, original location, version and relevant date. Distinguish the person who uploaded the file from the person responsible for its contents. An exported PDF may be perfectly usable, but an unexplained copy in a shared folder leaves uncertainty about whether it reflects the maintained original.
A recent upload date is not evidence of recent content. Likewise, a familiar logo or polished layout does not establish ownership.
Approval: is there evidence this version is accepted?
Look for a recorded decision from the responsible owner, tied to the version being reviewed. This might be an approval record, a controlled publication entry or an explicit confirmation identifying the document and its intended use.
Approval of an earlier version does not automatically cover later edits. Nor does permission to circulate a draft mean its claims are approved reference material.
Completeness: does it support an answer without guesswork?
Check whether the document includes the conditions needed to interpret its claims. A service statement may require eligibility rules, exceptions and an effective date. A procedure may rely on an attachment that was not included in the export.
Completeness is relative to the admitted question set. A source need not describe the entire business, but it must not leave essential conditions unstated for the questions it is supposed to answer.
Fitness: is it suitable for this audience and purpose?
Confirm that the material applies to the intended service, audience and operational context. Check that meaningful content survives extraction: headings, qualifications and relationships between text and figures must remain understandable.
Also examine audience restrictions. A source that mixes customer guidance with confidential staff notes may need a separately approved extract rather than admission of the whole file.
3. Use a source admission checklist that records evidence
Copy the following checklist into a review record. For each item, record pass, fail or unknown, followed by the evidence location. Treat unknown as unresolved, not as a quiet pass.
- Intended use: The allowed questions and audience are written down.
- Identity: The source has a title, original location and identifiable version or controlled snapshot.
- Ownership: A named role or person is accountable for the content.
- Currency: The effective date is known, and the owner has confirmed whether the source remains current.
- Approval: Evidence identifies the approved version, approving authority and permitted use.
- Scope: Applicable services, audiences and operating conditions are explicit.
- Completeness: Necessary prerequisites, exceptions, definitions and referenced attachments are present.
- Consistency: Overlapping claims have been checked against other proposed or admitted sources; material conflicts are resolved.
- Usability: Extracted content preserves the meaning of headings, lists, figures and qualifications.
- Audience suitability: The admitted material contains only information appropriate for the collection’s intended audience.
- Boundaries: Unsupported topics and known gaps are recorded.
- Maintenance: An owner and a review trigger are assigned.
Finish the record with the decision, permitted scope, reason, reviewer and review date. For anything held for repair, add a specific action and responsible owner. “Needs checking” is not an actionable repair request; “service owner to confirm whether the exclusions on the final page remain current” is.
This is a recommended operating record, not a claim that any particular software provides these fields or enforces these decisions.
4. Filled fictional worked example: four candidate sources
All documents, roles and circumstances in this worked example are hypothetical. The review uses the customer-support admission brief above: service eligibility, preparation and exclusions are in scope; pricing and discretionary approvals are not.
Current service guide: admit for a defined scope
- Candidate: Service Guide, version 3, held in the service team’s maintained reference folder.
- Provenance and currency: The service operations owner confirms this is the current effective version and identifies the version it replaces.
- Approval evidence: A publication record names version 3 and approves its customer-facing service guidance.
- Completeness and fitness: Eligibility, preparation requirements, exclusions and escalation instructions are present. Referenced material is included, and a text-extraction check preserves the conditions attached to each service description.
- Decision: Admit the approved customer-facing sections for eligibility, preparation and standard exclusions.
- Reason: The source passes all applicable gates for those topics. It does not authorise price quotations or discretionary exceptions.
- Maintenance: The service operations owner initiates re-review when the service rules change or a replacement guide is published.
Undated slide deck: hold for repair
- Candidate: Service Overview presentation, copied from a team folder.
- Provenance and currency: The author is identifiable, but the effective version and publication date are unknown.
- Approval evidence: No record establishes that the slides were approved for operational reference.
- Completeness and fitness: Slides describe broad availability without the exclusions found in the service guide. Several statements depend on speaker explanation that is not included.
- Decision: Hold outside the collection pending repair.
- Reason: The deck cannot independently support the allowed operational answers. Its omissions could turn a qualified service rule into an unconditional claim.
- Repair request: The service owner must verify its claims against the current guide, add necessary qualifications and approve a dated reference version. If it adds no distinct value, retire it from the candidate list instead.
Informal meeting notes: exclude as an answer source
- Candidate: Notes from a service-planning discussion.
- Provenance and currency: The meeting date and note-taker are known.
- Approval evidence: There is no approval of the notes as operational guidance.
- Completeness and fitness: The notes combine questions, proposals and participant comments. A suggestion to broaden eligibility has no recorded final decision or effective date.
- Decision: Exclude the notes from the answering collection.
- Reason: Reliable provenance does not turn discussion into policy. The document’s purpose is to record a conversation, not establish service rules.
- Follow-up: Use the unresolved eligibility question to request a formal decision. Any resulting approved guidance becomes a new candidate; the original notes remain outside.
Draft price sheet: exclude from this collection
- Candidate: Draft price sheet prepared by the commercial team.
- Provenance and currency: The owner and draft revision are identifiable.
- Approval evidence: Commercial approval is pending.
- Completeness and fitness: Some commercial conditions are unresolved, and pricing is outside the collection’s stated answering scope.
- Decision: Exclude from this collection, rather than hold it as an expected addition.
- Reason: Both approval and scope fail. Even a later approved version would not belong here unless the collection’s remit changed.
- Follow-up: If a separate pricing use is proposed, assess the final approved version against that use. Do not treat removal of the draft label as sufficient evidence.
5. Resolve conflicts before admitting overlapping material
Two individually credible sources can still disagree. Before admitting a candidate, compare its operational claims with material already approved for the same topics.
Do not assume the newest file wins. A recently edited presentation may repeat an old rule, while an older guide may still be the maintained authority. Ask the responsible owner which statement applies and obtain evidence of that decision.
Record the resolution at the disputed topic level. If a document is authoritative for preparation requirements but not for eligibility, its admission scope should reflect that distinction. Partial admission is useful only when the approved portion can be separated without losing necessary context.
There is a trade-off: admitting fewer, clearer sources can leave gaps in coverage, while admitting every overlapping document can preserve contradictions. Prefer an explicit gap to unsupported certainty. Create a repair task for the missing guidance rather than admitting a convenient substitute.
6. Keep repair work separate from approved reference material
A hold decision should keep the source out of the answering collection while work continues elsewhere. Otherwise, the team has documented uncertainty without preventing the uncertain material from being used.
Repair should address the failed gate. Formatting an undated deck does not establish currency. Adding a date to meeting notes does not establish approval. Converting a scanned file to readable text fixes usability, not authority.
When repair changes substantive claims, seek approval for the repaired version. Retain a link between the original candidate, the changes and the final decision so that a later reviewer can understand what happened.
If unsuitable material has already been admitted, remove or isolate it from the collection first. Identify the unsupported topics, locate any approved replacement, and re-review the affected material. Admission review cannot establish whether earlier answers were correct; any investigation of those answers is separate work.
7. Make admission decisions maintainable
Approval is not permanent simply because it was once justified. Record triggers that require another review: a replacement source, a changed service rule, a withdrawn approval, a newly discovered missing attachment or an unresolved conflict.
Keep the record tied to the admitted version. A link that silently changes underneath the review record makes it difficult to know what was actually approved. Where a source is maintained continuously, identify how substantive changes will be brought back for review.
Avoid giving every source an arbitrary expiry date as a substitute for ownership. A calendar review can be useful, but it does not replace a named owner who knows when the underlying operation changes. Conversely, an owner’s name without a review trigger can leave stale material unnoticed.
8. Run a bounded admission review tomorrow
Start with one answering job and a small candidate set. Do not begin by cataloguing every shared drive.
- Write the audience, allowed topics and excluded topics in a short admission brief.
- Review four candidate sources using the checklist, recording evidence rather than impressions.
- Assign each source a decision and permitted scope, or an explicit reason it remains outside.
- Give each repair task an owner and a required piece of evidence.
- Check the proposed collection against the admission record before making the material available for answers.
The immediate completion measure is simple: every reviewed candidate has a traceable decision, every admitted source has version-specific evidence, and every unresolved issue has an owner. Also record which allowed topics still lack an approved source. That gap list is the next writing or repair brief—not permission to upload something less reliable.
Put the next conversation to work.
Explore Alactic Communications for WhatsApp conversations, campaigns and AI-assisted workflows. Choose the product and plan that fit your team.
Explore Communications