I get some version of the same call every few weeks. An executive has seen a demo — a genuinely impressive demo — and there's a contract sitting in their inbox with a signature line on it. What they want from me is a gut check: is this real, or am I about to buy an expensive science experiment?

The uncomfortable truth is that most AI initiatives that fail don't fail because the model was bad. The models are mostly good now. They fail because the question was wrong — because nobody could say precisely what the system was for, what a mistake would cost, or whose job it would be after the confetti settled.

So before you sign anything, sit with three questions. Not for a quarter of analysis — just long enough that you can answer each one in a single, plain sentence. If you can, buy with confidence. If you can't, the contract can wait. It will still be there next month, probably cheaper.

1. What decision is the AI making?

Not "what is it doing" — what decision. The difference matters more than anything else on this list.

Drafting a reply is not a decision. Summarizing a document is not a decision. Those are conveniences, and conveniences are fine — but they're bought like office chairs, not like strategy. A decision is different: routing a ticket to the right team is a decision. Approving an invoice is a decision. Flagging a transaction as suspicious is a decision. Decisions have owners, error rates, and consequences, and that's exactly why they're where the real value lives.

When a vendor can't tell you which decision their product makes — when the pitch is a fog of "productivity" and "transformation" — what they're selling is a demo. If you can't name the decision, you're buying a productivity demo and calling it strategy. Name it first. One sentence: "This system decides ___."

If you cannot name the decision the AI is making, you are buying a productivity demo and calling it strategy.

2. What does a wrong answer cost?

Every model is wrong sometimes. That's not a flaw to be negotiated away in the contract; it's the nature of the thing. So the second question is about the shape of the downside: when this system is wrong — and it will be — what happens?

The best first deployments share two properties: the answers are auditable (a human can check the work without redoing it) and the downside is bounded (a mistake costs an awkward email, not a regulatory filing). AI is at its most valuable doing the eighty percent of work that is high-volume and low-stakes, which frees your people for the twenty percent that is rare and consequential. That's not a limitation to apologize for. That's the whole business case.

Run the arithmetic out loud: how often will it be wrong, who catches it, and what does each miss cost? If the honest answer to "who catches it" is nobody, you don't have an AI problem — you have a process problem, and the AI will simply automate it at scale.

3. Who owns it after the pilot?

This is the question that kills the most projects, and it's the one nobody asks in the sales cycle.

A pilot with no operator is a science experiment. Someone has to live with the system in production — tune the prompts when the outputs drift, watch the metrics, notice when a vendor update quietly changes behavior, and take the page when it misbehaves at 4 p.m. on a Friday. That's a real job. It doesn't have to be a full-time job on day one, but it has to be a named job, attached to a specific human who agreed to it.

If that person doesn't exist in your organization yet, hire or designate them before the pilot, not after. The alternative is a pattern I've seen too many times: a successful eight-week pilot, glowing internal reviews, and then a slow, unowned decay into a line item that everyone is vaguely embarrassed to cancel.

The one-page test

Here's the whole framework on an index card. Before you buy the AI, write three sentences:

  • This system decides ___.
  • When it's wrong, catches it, and the mistake costs .
  • After the pilot, ___ owns it.

If you can fill in every blank, you're not buying hype — you're buying a tool, with a job, and an owner. That deal tends to work out. And if a blank stays stubbornly empty, that's not a failure of imagination. That's the framework doing exactly what it's for: saving you the money before the invoice, instead of after.