Building AI Products that Win

Mistakes That Kill AI Products—and What to Do Instead

Big waves from the recent MIT report, where the disconcerting bottom line is that 95% of enterprise AI projects don't deliver any value in return.

I think reality is more complex than bundling together a 95% of failures.

What are the successful 5% about?

It sounds like many failures are rooted in poor product thinking and execution rather than bad tech.

Here are some mistakes I did or witnessed first-hand on AI-enabled products. I picked the ones that are shared with other entrepreneurs, engineers and product people so you can make an internal check with your team and steer away from these dangerous rocks.

1) Don’t build tech‑backwards. Build customer‑backwards.

A powerful model is not a product.

A product is a repeatable way to produce outcomes people care about.

What can you do?

1. PR/FAQ first: before you write a line of code, write a one‑page press release announcing the launch as if it goes live tomorrow. Force yourself to articulate the press‑worthy value, the one‑sentence headline, the proof points and the call to action. Then write the FAQ: who it’s for, what it does, why it’s better, how it works, pricing, risks and what happens when it fails.
   If you can’t make it sound compelling on paper, you won’t make it compelling in code.

2. Go to Gemba: Gemba is “the real place” where work happens. Go sit with the people you want to serve. Watch what they actually do.

   1. Document the steps they take today. Build a job map and share it with your team, better you all know it by heart.

   2. Capture pains, workarounds, delays, rework, struggles and unmet desires.

   3. Identify outcomes that matter and where they’re not being met by current alternatives.

   4. Translate insights into job stories: When I’m [situation], I want to [motivation], so I can [outcome]. For example "When I’m on a live sales call and the prospect asks for pricing, I want to auto‑generate a tailored proposal from our CRM and product catalog, so I can send it before the call ends and increase the chance of closing."

Your model is a means. Their outcome is the product.

2) Don’t scale excellence. Scale learning.

Perfection is a stall tactic.

AI products are probabilistic; they only get good by encountering reality.

• Your first prototype should be essential. If it looks polished, you shipped too late.

• Ship a useful version to a narrow audience and measure usage, not opinions.

• Ask the only question that matters early: “Will someone pay even $1 for this?” Revenue, even tiny, becomes your most honest signal and your fastest teacher.

Optimize for learning velocity over build velocity. The team that learns fastest compounds advantage.

3) Don’t ignore edge cases. Design for the messy real world.

AI fails at the edges: ambiguous intent, adversarial inputs, unusual contexts. That’s where trust is won or lost.

• You don’t know what’s broken until customers use it wrong. Invite misuse, like when a QA tester enters a bar and orders "0 beers and 2.3 car tires".

• Role‑play scenarios: rushed users, noisy environments, partial data, misleading prompts, hostile actors.

• Run PR drills for your worst nightmare: “What if the model confidently outputs something harmful or wrong?” Decide in advance on detection, escalation, fallback and apology. Build those paths into the product.

Designing for failure is how you earn the right to scale.

4) Don’t fall in love with the idea. Kill your darlings.

Good founders test their assumptions.

Great founders love truth more than their favorite ideas.

• Set the outcome metric first. What must change for the user or the business? Examples: “Time to file a claim reduced by 60%”, “First‑contact resolution +20%”, “Error rate below 2%”.

• Instrument the journey. Measure the outcome metric ruthlessly.

• When data contradicts your pet idea, cut it quickly. Reallocate energy to what the metric says matters.

Discipline beats romance. Investors notice.

5) Don’t just automate. Reimagine.

If you automate a broken process, you get faster broken outcomes. Use AI to remove old constraints and redesign the journey.

• Focus on the need, not the workflow. Workflows are fossilized compromises based on old constraints.

• Ask “What would this look like if we started from zero?”

  • From “warehousing books” to “printing on demand.”

  • From “drop‑down menus” to “voice commands.”

  • From “form filling” to “conversation that produces a compliant document.”

• Test interfaces beyond standard UX conventions: voice, multimodal, agentic flows, proactive assistants. Your job isn’t to make a UI; it’s to make the result inevitable.

6) Don’t solve toy problems. Think big.

Small problems rarely generate urgency or moats. Big problems attract talent, customers and capital.

• Are you building something with leverage? Does each incremental user/data point make the product smarter for everyone?

• Is there urgency? Who loses money, time, or reputation every day this doesn’t exist?

• Look for:

  • Pain points affecting millions or high‑value users.

  • System‑level inefficiencies (hand‑offs, backlogs, compliance bottlenecks, data silos).

  • Untapped behavioral shifts (voice‑first work, ambient copilots, AI‑native content, agent‑to‑agent workflows).

A big, credible vision is a magnet. For customers, for great people and for long‑term capital.

7) Don’t underestimate the power of communication.

You don’t just ship features. You ship clarity.

• Great products are remembered because they’re easy to explain, not because they’re technically complex.

• How? Nail the line: “It does X, so you get Y, without Z”.

• Lead with a simple demo that shows an undeniable before/after. Save the architecture for the appendix.

• Positioning > features. “The fastest way for finance teams to close the books” is stronger than “AI for spreadsheets”.

Clarity is a growth feature. It lowers acquisition cost, accelerates adoption, and rallies your team.

Build the Compounding Loop

The durable advantage in AI is not a model; it’s a flywheel:

• More users → more data (and richer interaction traces)

• More data → better models and heuristics (fine‑tuning, retrieval, evaluation, guardrails)

• Better product → more users (and higher willingness to pay)

Your job is to remove friction at every turn of that loop:

• Distribution: Nail a wedge (a narrow job, a specific role, a painful step) and a channel (partner integrations, marketplaces, PLG).

• Data advantage: Capture proprietary data ethically; annotate high‑leverage cases; create feedback hooks that improve performance automatically.

• Reliability: Invest in evals, monitoring, fallbacks and human‑in‑the‑loop where quality matters.

• Unit economics: Track cost per successful outcome, not per token. Improve with caching, small models where possible, batching, and product nudges that reduce unnecessary calls.

What Investors Want to See (Because It Signals Real Value)

• Clear, urgent use case with a specific market segment and obvious ROI.

• Evidence of learning velocity: short cycles, shipping cadence, metrics improving weekly.

• Moat trajectory: proprietary data, embedded workflows, switching costs or network effects from agent‑to‑agent or team‑to‑team usage.

• Early monetization signal: paid pilots, expansion within accounts, retention on the outcome (not the novelty or wow! effect).

• Healthy margins today with a path to great margins tomorrow: smart model selection, inference cost control and product design that limits waste.

A Simple Operating Cadence

• Week 0: Write the PR/FAQ. Circle three customers to observe in Gemba (their environment).

• Week 1: Ship the narrowest working slice that proves the core outcome. Put a price on it.

• Week 2: Run edge‑case drills. Add evals, logging and a human fallback.

• Week 3: Kill one darling. Double down on the feature that moves your outcome metric.

• Week 4: Re‑imagine one step of the journey. Replace a form with a conversation or a click path with an automation.

• Repeat: Shorten the loop time. Sharpen the positioning. Raise the bar on reliability.

The Bottom Line

AI doesn’t excuse you from fundamentals; it amplifies them.

Build customer‑backwards. Learn faster than anyone.

Design for failure. Reimagine the journey.

Aim at real, urgent problems. Communicate with piercing clarity.

Do that, and your product will earn adoption. Then the compounding loop will do the rest for retention and ambassadorship.

If you’re working on this right now, write a comment with your PR/FAQ headline and the one outcome metric you’re committing to move this month. Or book a meeting.

I’m happy to learn your way and give feedback.

Ad maiora,
Pan