Why I Test My Products Like Someone Who Has Never Used Them

Forgetting what I know

Something I keep relearning from building software is that a feature can make complete sense to the developer and still confuse the person using it. By the time something ships, I know where every button is, what each screen is for and which steps come in which order. Someone opening the product for the first time knows none of that, and nobody is sitting next to them to explain.

So before I put something in front of real users, I try to use it as someone who has never seen it. Sometimes I go further and imagine I’m seven, or thirteen, or an adult who knows nothing about the subject. Users are smart. What I’m looking for is the knowledge I’ve quietly assumed they already have, because that’s the part I can no longer see.

The hardest part of testing your own product is forgetting everything you already know about it. Developers understand the terminology, navigation and business rules, so they can’t easily see where a newcomer will get stuck. Walking through the product as specific people with less context, such as a first-time visitor, someone new to the field or a younger user, exposes those assumptions before real usability testing does.

Developers know too much

Familiarity builds up without anyone noticing. After a few weeks on a feature, the internal name for a screen starts to feel like plain English. I stop reading labels and start recognizing them. The workflow feels obvious because I designed it, and the business rules feel natural because I argued about them in planning. None of that context ships with the product.

Teams make this harder in a perfectly reasonable way. Everyone in the review shares the same vocabulary, so a confusing term never gets challenged. A feature can work perfectly from a technical point of view and still be hard to use, and the people closest to it are often the last ones able to tell.

Who I pretend to be

When I test, I pick a specific person and stay in character for the whole walkthrough. A vague “new user” doesn’t help much, because I end up quietly testing as myself.

For each of them I ask the same three things. Can they tell what the product does? Do they know where to start? When they finish one step, do they know what to do next? If the honest answer depends on someone explaining it, I’ve found work to do.

FedPath, for a first-time contractor

FedPath helps small businesses find federal contracts and decide whether they’re worth bidding on. Government contracting has a lot of terminology and some complicated processes. Someone who has done it for a while reads NAICS codes (the industry classifications agencies use), set-asides (contracts reserved for certain kinds of small business), solicitations and compliance requirements without slowing down. A small business owner exploring federal work for the first time probably doesn’t know what most of those terms mean, or which ones matter for them.

That owner is who I try to be when I test it. Can they find an opportunity that fits their work? Can they tell whether it’s relevant without reading the whole solicitation? Do they know what to do next? Parts of the product are built for exactly that person. The GovCon Readiness check is a five-minute check for businesses deciding whether federal work suits them at all. Fit Analysis labels each finding as known or inferred, so nobody has to guess how far to trust it. And anyone can browse real opportunities before creating an account, which answers a first-time visitor’s real question, whether this is worth their time, before the product asks them for anything. The FedPath case study shows how those pieces fit together.

Berkeley Omnium, for every kind of rider

The Berkeley Omnium website has the same problem with a different vocabulary. Someone who has raced bikes for years understands race categories, licenses, registration and the difference between a road race and a criterium. Someone new to racing may not know where to begin, and may not want to ask.

So I test the site as a junior racer, a parent, a first-time racer, an experienced racer, a woman deciding whether there’s a race for her, and someone coming from outside the Bay Area. Each arrives with a different question. The experienced racer wants the course, the start time and the registration link, quickly. A parent wants to understand what the day involves for their kid. The first-timer wants to know whether they’re allowed to enter and which category they’d race in. The rider traveling in wants to know how to get there. The site has separate pages for juniors, women’s racing, getting into racing and traveling to race, because one page can’t answer all of those questions well.

I do the same with smaller projects. A calculator on Useful Little Tools gets a minute of someone’s attention, so if the first field asks for a number they don’t know how to find, the tool has failed them. With internal tools, I picture whoever uses it after me, at a bad moment, without my notes.

Working and usable are different tests

A feature can pass every automated test, meet its acceptance criteria, work exactly as designed and have no obvious bugs, and people can still struggle to use it. Those checks confirm we built what we agreed to build. They don’t tell us whether someone who wasn’t part of those conversations can figure it out.

I think engineering teams overlook this partly because nothing fails. No test goes red when a label is confusing. The cost shows up later, as support questions, abandoned sign-ups and people who quietly stop coming back.

This is the other half of why writing good user stories is so hard. That piece is about how difficult it is for a team to agree on what to build. This one is about what happens when we build exactly what we agreed and it still doesn’t work for the person using it.

Why I push back on complexity

Testing this way is a big part of why I challenge complexity in planning, reviews and design. When a simpler approach reaches the same goal, everyone gains. Users understand the product faster. Developers have less to build, QA has fewer complicated scenarios to test, and the team can ship earlier and hear from real users sooner. There’s also less to maintain afterward.

Simpler still has to do the job. If a contractor needs a compliance matrix, removing it to make a screen look cleaner helps nobody. What I want to cut is the effort and confusion that don’t need to be there: the extra step that exists because it was easy to build, the setting nobody asked for, the label that only makes sense inside the team. I wrote more about that habit in what engineering leadership actually does, and about the architecture version in the architecture you don’t build.

Before real people, not instead of them

Different people reveal different problems, which is why I don’t test as one generic user. A beginner struggles with terminology. An experienced user gets frustrated by too many steps. A parent can’t find the information that applies to their child. A business owner understands an opportunity and still doesn’t know what action to take. The same product can be a completely different experience depending on who is using it.

Imagined walkthroughs don’t replace usability testing with real users. When I pretend to be a first-time racer, I’m still guessing, and a real first-time racer will hit things I never thought of. The walkthroughs catch the obvious problems early, while they’re cheap to fix, so that time with real people goes to the problems nobody on the team could predict.

Short answers