50 calls before the requirements doc

Before we wrote a line of Helply I ran 50+ calls with support teams.

That sounds like a discipline thing. It wasn't. I did it because I'd just spent 18 months building the wrong thing and I no longer trusted my own read on a category I'd been in for 12 years.

The first 15 were mostly useless

Because I ran them wrong.

I went in with a thesis. I'd describe the agentic idea, watch them nod, and write down that they nodded. Support leaders are professionally agreeable. They will nod at almost anything, and if you're the guy who owns a help desk they'll be nice about it, and you will walk away with 15 confirmations of something nobody said.

Around call 16 I switched to asking people to walk me through yesterday. Not their strategy. Yesterday. What did you do first, then what, what took the longest, what did you complain about in Slack.

That's when it got useful, and it also meant the first 15 calls were data about my own bias more than about the market. Which I suppose is worth something.

Three things came up over and over

The volume isn't the problem. The sameness is. Nobody said "we get too many tickets." They said some version of "it's the same 40 questions forever." Teams weren't overwhelmed by hard work. They were worn down by repetitive work that still required a human because the tools couldn't be trusted with it.

Nobody trusts the bot because nobody can see what it did. Every single team that had tried an AI layer had a story about it telling a customer something confidently wrong. The response was always the same: turn it down until it's basically off, keep paying for it, feel bad. What they wanted wasn't a smarter bot. It was visibility into what the bot decided and why, and an easy way to correct it.

The answer is never in the help desk. To resolve properly you need the billing state, the product event, the open bug, the note from the account manager. Support agents spend an enormous amount of their day being a human API between 5 systems.

What I got wrong that I'd have shipped

Two things I was confident about going in.

I thought teams wanted better reporting. They don't, particularly. Everyone says yes to reporting and nobody opens it. The people who genuinely need numbers export to a spreadsheet and always will, and I'd have spent 2 months building dashboards for a use case that doesn't exist.

I thought a big deal would be made of tone and brand voice in AI replies. Some, but far less than expected. What people actually cared about was whether the reply was correct. A slightly stiff correct answer beats a warm wrong one and it isn't close.

What a good call looked like

The best ones were the people who were annoyed. Not the reference customers, not the ones a mutual friend introduced with a nice note. The ones mid-migration, or who'd just been burned, or who agreed to talk because they wanted to complain to someone in the industry.

I also stopped talking about Helply on the calls entirely after a while. The moment you describe what you're building, the person starts helping you build it, and you get a co-designed answer instead of a real one.

50 is not a magic number. Somewhere around 35 the answers stopped surprising me, and I kept going another 15 mostly to be sure the pattern held rather than to find more.

The part I'd push back on

Customer discovery cannot tell you the architecture. Not one of those 50 people said "your data model is wrong and you need to rebuild." They described symptoms. The diagnosis was ours, and if we'd only built what was literally asked for we'd have shipped a better version of the retrofit and lost.

So the calls were necessary and not sufficient, which is an unsatisfying thing to write and the honest version of it.

We're 8 months post launch. Most of what those calls told us has held. The one thing I'd flag: everyone I spoke to already had an AI layer they didn't trust, which means my whole sample was pre-selected for people who'd been burned. I have no idea what the teams who haven't tried anything yet want.

Probably should have called some of them.