Questions worth asking before you hire anyone for AI work

This is a list of questions worth asking anyone pitching you AI consulting, agent work, or “automation,” plus my own answers to each one. It’s for small businesses vetting a consultant, not a sales page — if a question here makes someone squirm, that’s the point.

What should I ask an AI consultant before I hire them?

Fewer questions than you’d think, asked in the right order: what have you actually shipped, who owns my data while you’re working with it, what won’t you automate, and what happens when it breaks. Most of the useful signal in an hour-long pitch comes down to how directly those four get answered. Everything else — methodology slides, framework names, case study logos from companies you’ve never heard of — is decoration around those four answers.

I’d rather someone ask me these than sit through my pitch. So here they are, with my own answers next to them, because “trust me” isn’t an answer worth accepting from anyone, including me.

Can you show me three systems you’ve actually shipped?

Ask for this before anything else. Not a demo, not a slide with a client logo and no link — something running, or a build log you can read. A lot of AI consulting right now is a slide deck wrapped around a ChatGPT subscription, and the fastest way to tell the difference is asking for links, not promises.

My own answer is the /building page: ten real projects, most still running, with honest status labels rather than everything marked “success.” LocumOS is the admin system I run my own locum pharmacist work through. Najcos is a client signage business site shipped to production. Strength Base was a client gym site, built and handed over to the owner in full. If a consultant can’t point you to something with a URL or a repo, that’s the red flag, not a minor gap.

What happens to my data while you’re working with it?

This is the question I take most seriously, because I come from a regulated profession where getting it wrong isn’t a bug report — it’s a compliance breach. As an AHPRA-registered pharmacist doing AI work, my own rule is stricter than most clients ask for: nothing identifiable goes anywhere near a model, and a human — usually me — signs off before anything clinical gets acted on. The full version of that rule, and what it costs, is on /writing/de-identify-first.

You should get a specific answer here, not a reassurance. “We take privacy seriously” is not an answer. “Here’s exactly what touches an AI model, what doesn’t, and who checks the output” is. This is also the one question I’d walk away over. If a consultant dodges it, hedges it, or answers a different question instead, that’s not a style difference — it’s a sign they haven’t actually thought about where your data goes, and everything else they tell you is downstream of that gap.

What won’t you automate?

Ask this one directly, because the answer tells you more than the pitch does. Every AI opportunity audit I run ends with a ranked roadmap of what to build — and a separate list of what I’m telling the client not to automate yet. That second list is usually the more useful page. It covers anything where a wrong output would go unnoticed until real damage is done, anything resting on a judgment call the business hasn’t actually agreed on internally, and anything touching identifiable personal data without a clear handling plan.

If someone’s pitch has no version of a “don’t build this yet” list, either they haven’t looked hard enough, or they’re optimizing for the sale over the outcome. A consultant who never says no to a feature isn’t giving you an audit — they’re giving you a sales pitch with extra steps.

What happens if it breaks?

Ask who gets the call at 9pm when the automation misfires, and what the fallback is while it’s down. Software breaks; the question is whether there’s a plan for that moment or whether you find out the plan doesn’t exist the first time it happens. I build with a human checkpoint on anything where a silent failure would cause real harm — that’s not extra scope, it’s part of building the thing correctly the first time.

A consultant who hasn’t thought about failure modes hasn’t actually built the system yet, even if it demos cleanly.

Are you one person or an agency layer?

Neither is automatically better — ask because it changes what you’re buying. An agency gives you more capacity for a multi-team programme; one accountable person gives you someone who can’t quietly hand your project to a junior halfway through. I work as one named person, which means slower on raw capacity and faster on decisions, because there’s no layer between you and whoever’s actually writing the code.

Know which one you need before you pick based on the pitch deck.

How do you price this, and what am I actually buying?

You should get scope before you get a number: what’s included, what’s explicitly out, and what “done” looks like. I don’t publish a pricing table, and I’d be skeptical of anyone who quotes you before understanding your process — that’s a sign the price is generic, not scoped to what you actually need. What I do offer up front is the shape of the engagement: an AI Opportunity Audit runs about a week and ends in that ranked roadmap plus the don’t-automate list, and you own the roadmap outright whether you build it with me or hand it to someone else.

What’s the short version?

Ask for links, not logos. Ask what won’t be automated, not just what will. Ask who’s accountable when it breaks. If those three questions get straight answers, you’ve already learned more than most pitches will tell you.

If you want to see how I answer them in practice, the details are on /work.