Ten questions to ask before signing with any AI vendor
If a vendor can't answer these clearly in five minutes, find out why. Most of these matter more than the demo did.
Owners often ask us what to ask other vendors. We're happy to share the list — partly because we use the same list ourselves when we're scoping a new model provider or integration partner, and partly because the answers tell you fast whether the vendor knows their craft.
1. What happens to my data? Specifically. Where it goes, who can see it, what training agreements apply, what retention policies are in force, what happens if I leave. If the answer is vague, that's the answer.
2. Who runs my engagement after the kickoff call? Is it the engineer I've been talking to, or does it become someone else's project? Engagements where ownership transfers tend to drift.
3. What's your eval practice? How will I know in six months if the system is still doing what it should? If they don't have a real answer, the system probably won't be.
4. What's the failure mode? When the AI gets something wrong, what happens? Specifically. If the answer is 'it doesn't get things wrong,' walk away.
5. What's the on-call SLA, and who picks up? Concrete numbers. Specific people. If something breaks at 3am during a peak business period, what happens?
6. What integrations are included, and what's billed extra? Integration scope creep is a leading cause of unhappy AI engagements. Get this in writing.
7. What does the system not do that I might assume it does? A good vendor will volunteer the limitations. A bad vendor will pretend everything works.
8. Show me a real audit log from a real engagement. (Anonymized.) If they can't, they probably don't keep one.
9. What does month nine look like? Forget the launch. What's the engagement at month nine — what's the cadence, what's the cost, what are we discussing? If they haven't thought about month nine, the engagement won't survive to it.
10. What would make you decline this engagement? If the answer is nothing, they're a salesperson. If the answer is specific, they're a practitioner.