Check your data and systems before an AI pilot by doing three things for the one workflow you chose: map every system the work touches, test a sample of real records against the pilot's needs, and confirm in writing what any tool can read and write. If the records are missing, stale, or labeled differently in each system, the pilot will measure your data problems instead of the AI. Fixing that first is often the cheapest step in the whole project.
What does it mean to check data and systems before an AI pilot?
It means confirming that the information the workflow starts from, and the places where its results must land, are good enough to test fairly. This article assumes you have already chosen a first workflow and written down a baseline. If you have not, start with our articles on choosing the first workflow for an AI pilot and on measuring an AI pilot before it starts. This one covers the step between choosing and launching.
The AWS Smart Business Hub's AI readiness checklist for SMBs describes readiness as covering information, workflows, people, and governance, not only technology. Its first step is a simple data health check. It is vendor guidance, so use it as a starting framework rather than a standard. The sections below turn it into questions an owner can answer in an afternoon.
What should you map before an AI pilot touches your data?
AWS suggests listing your source systems, who owns each, where the system of record lives, how data is accessed, and how often it is updated. For a pilot, do that only for the systems your chosen workflow touches. A one-page table is enough. Record the following for each system.
- Name and purpose: for example the CRM, the accounting package, the shared inbox, or a spreadsheet someone keeps by hand.
- Named owner: one person who can answer questions about the data and approve access, not a team or a shared login.
- System of record: which system wins when two systems disagree about the same customer, quote, or invoice.
- Identifier: the field that links one customer across systems. AWS suggests defining one ID per customer, such as email, phone, or account ID, so records stitch together.
- Access: who can read, who can write, and how often access is reviewed. AWS suggests reviewing monthly.
- Update pattern: whether changes appear in real time, nightly, or only when someone remembers to enter them.
How do you test whether the data is good enough for this workflow?
Pull a sample of recent real records and check them against what the pilot needs. NIST's AI RMF Playbook, in its Map function, warns that the difficulty of finding the right data can lead teams to choose data for being available rather than suitable. It also lists questions an organization can document, including whether there are known errors, sources of noise, or redundancies in the data, and over what time frame the data was collected. The sample test below is our practical adaptation, not NIST's wording.
- Completeness: how many sampled records are missing a field the workflow depends on, such as a contact method, a quote amount, or a status?
- Consistent labels: do two systems use different words for the same stage, so that a rule written for one breaks on the other?
- Freshness: how many records show a status that is out of date compared with what staff know actually happened?
- Duplicates: is the same customer entered more than once under different spellings or emails?
- Representativeness: does the sample include your normal busy and quiet periods and your unusual cases, not only the clean ones?
- Sensitivity: does any record contain information the tool should not see, and can it be excluded or redacted?
What does a data check look like in a hypothetical example?
This is a hypothetical illustration with invented numbers. It is not an Omni Workforce customer result and not a prediction. Suppose a service business wants to pilot AI-drafted follow-ups on open quotes. An owner and the person who sends quotes pull 50 recent quote records from the CRM. Six are missing both an email and a phone number. Eleven show the stage Quoted although the customer had already accepted or declined by phone. Assume no record has both problems, so 17 of 50 records, or 34 percent, have a problem the pilot would inherit.
There is also a labeling gap. The CRM marks a won job Closed Won, while the accounting spreadsheet marks the same job Paid only after the invoice clears. A follow-up rule that treats Quoted as unanswered would chase customers who have already said yes. The practical result is not to cancel the pilot. It is to set an entry rule before launch: the sender updates the stage on the same day, a missing contact method blocks a draft, and the pilot starts with the cleanest slice of records. The baseline from the earlier article then reflects the real process rather than a data artifact.
How do you confirm a tool can actually connect to your systems?
AWS advises checking whether your current tools can connect to AI add-ons or APIs and favoring systems that offer webhooks or APIs and clear permission models. A feature listed on a vendor page is an idea, not a confirmed capability for your setup. This applies to any provider, including Omni Workforce: whether a specific integration works with your specific tool and permissions should be confirmed for your case before you plan around it. Ask these questions in writing.
- Which exact systems will the tool read from, and which will it write to?
- Is each connection read-only, or can it create and change records? Can write access be limited to one object type?
- Who approves access, and how is it removed at the end of the pilot?
- What happens to the work if the connection fails, and who notices?
- Where does the final outcome get recorded, so reporting is not rebuilt by hand?
- If a step cannot be connected, what is the manual fallback and who does it?
NIST's Map function also suggests documenting dependencies on upstream data and inventorying third-party software and data used by the system. For a small business, that translates to a short list of which outside tools the pilot depends on and what happens if one changes or goes down.
Should you fix the data first, narrow the pilot, or proceed?
Use the sample results to choose one of three paths. The thresholds are yours to set before you look at the numbers. We do not have a universal percentage that is right for every workflow.
- Proceed: the sampled records have what the workflow needs, owners and access are named, and the connection has been confirmed for your systems.
- Narrow the pilot: the data is good in one slice, such as one service line or one lead source. Start there and say in advance what is excluded. NIST's Map function notes that systems with a narrow scope tend to be easier to map, measure, and manage.
- Fix first: a key field is routinely missing or two systems disagree about the same record. Repair the intake form, the entry rule, or the shared identifier first, and treat that fix as its own measurable improvement. NIST also suggests considering non-AI alternatives, and a cleaner process sometimes delivers most of the gain.
What is the data and systems checklist before you launch?
- Have you listed every system the chosen workflow reads from or writes to?
- Does each system have one named owner, not a team or a shared login?
- Is the system of record decided for each type of record?
- Is there one identifier that links a customer across systems?
- Have you checked a sample of recent real records for missing fields, stale statuses, duplicates, and mismatched labels?
- Did the sample include busy periods and unusual cases?
- Did you set a pass threshold before reviewing the sample results?
- Is sensitive information excluded or redacted, and is that written down?
- Is each tool connection confirmed for your systems, with read and write access limited and a named approver?
- Is there a manual fallback if a connection fails, and a named person who watches for it?
- Is the end-of-pilot access removal written down?
Where can a consultation help?
A consultation can pressure-test your system map and sample results before you commit to a pilot. The right answer for a given workflow might be a data cleanup, a process change, an integration, an automation, an AI agent, or no AI at all. An idea discussed in a consultation is not the same as a capability Omni Workforce has verified for your specific tools, so anything that depends on a particular integration should be confirmed for your case before you plan around it.
Sources: AWS Smart Business Hub, AI readiness checklist for SMBs, https://aws.amazon.com/smart-business/resources-for-smb/ai-readiness-checklist/, read November 3, 2026; this is vendor guidance, dated September 1, 2025 on the page. NIST AI Resource Center, AI RMF Playbook, Map function, https://airc.nist.gov/airmf-resources/playbook/map/, read November 3, 2026 (Map 2.2, 2.3, 3.3, 4.1 and 1.1). AWS and NIST do not endorse Omni Workforce. The sample test, the hypothetical example, the three paths, and the checklist are Omni Workforce's own illustration, not AWS or NIST guidance.
Book a consultation with Omni Workforce to map the systems behind one workflow, test a sample of your records, and decide what a measurable pilot would need before it starts.