The One-Question Playtest
Throw away the survey and ask only whether they wanted to play again.
- Difficulty
- Starter
- Time to result
- ~weeks to results
- Steps
- 6
- Confidence
- 95%
Exploding Kittens has stopped issuing surveys in playtests entirely. The input is a set of half-built concepts and at least one other human; the mechanism is a single binary question asked immediately after play, plus a follow-up interrogation of the reason behind any yes. The output is both a done-check — one hundred percent yes — and a development priority, because whatever pulled them back is the thing to make shine.
Origin
Lee developed it across seven years of running the Kitty Test Pilots programme and hundreds of internal tests, arriving at the conclusion that everything else he was measuring was noise around this one signal. He applies the same rule at home: the moment his daughter says can we play again and he genuinely means yes too, the design has worked.
Core principles
- 01There is exactly one question worth asking after someone uses your product: when you finished, did you want to do it again?
- 02Surveys collect opinions; the replay question collects behaviour, which is the only thing that predicts spread.
- 03The done bar is one hundred percent yes, and it never starts anywhere near that — it always starts as a pile of noes.
- 04A no comes with reasons attached, and those reasons are the actual work; a yes comes with a cause you must go and identify.
- 05The reason behind a yes matters more than the yes: they may have had fun, or they may sense a mastery they can reach next time.
- 06Once you know what pulled them back, double down on that one thing and build every surrounding component to emphasise it.
- 07Positive signal from a single tester on a single concept is enough to justify developing it — you do not need statistical comfort at this stage.
How to run it
- 1
Generate many rough concepts fast
Lee says he can rattle off twelve core gameplay loops in an hour with blank cards and Sharpies, and that generation is honestly not the hard part.
Pro tip Blank white poker cards ordered by the pallet plus Sharpies is his entire kit; six cards shuffled become dice.
- 2
Recruit one other person
You need at least one human to hand the concepts to. The test is inherently relational, so a solo evaluation cannot produce the signal.
Watch out Testing alone will tell you whether the mechanism works and nothing about whether it is worth repeating.
- 3
Ask the single question
After each concept, ask only whether they wanted to play again. No surveys, no scales, no feature ratings.
Pro tip Ask immediately at the end, before the experience has been rationalised into an opinion.
Watch out Adding a second question invites politeness, which contaminates the first.
- 4
Interrogate the yes
When someone says ooh let's play that one again, work out why. Lee's two candidate causes are that they had so much fun, or that they sense a mastery they could reach on a second attempt.
Pro tip Watch for a third cause: they want to show someone else. Lee treats the instinct to put a game in your bag and take it to a friend's house as the strongest possible signal.
- 5
Double down on the pull
Identify the compelling thing that kept pulling them back and make it the element that shines most, then build every surrounding component to emphasise it over and over.
Watch out Adding features that dilute the pull will move you backwards even if each one tests fine in isolation.
- 6
Hold the bar at one hundred percent
The product is not finished until everyone says yes. Lee is explicit that it never starts that way — the honest starting state is a whole bunch of noes with explanations attached.
Pro tip Treat the noes as a free specification document.
Watch out Shipping at eighty percent yes means shipping a product that a fifth of buyers will never open twice.
In the wild
Given Tim's hypothetical of a week locked in a room with blank cards and Sharpies, Lee's plan was to generate a dozen loops in the first hour — including an invented one where two players each hold a card, the higher card wins, but first there is a conversation in which each tries to bluff the other into folding. He points out that the crux is the conversation, not the cards. He then needs one person to play them with, and asks only whether they wanted to play again.
→ A method that converts a week of solo ideation into a single, unambiguous ranking of concepts by the end of day one.
Lee hated playing commercial kids' games with his daughter — his line about Candyland is unprintable — so for two years he and his then four-year-old made games every night with paper, stickers and markers. They built twenty or thirty. The bar was brutal and doubled: he could not let her win, he had to be trying his hardest, and she had to beat him anyway. Games that failed it got scrapped. The keeper condition was that at the end she asked can we play again and he genuinely wanted to say yes.
→ A handful of games that satisfied both criteria, co-designed by a five-year-old, heading to retail shelves that year.
Common mistakes
Running surveys instead of asking the question
Lee's team stopped giving out surveys entirely. Multi-question instruments generate a defensible-looking dataset that lets a team argue itself into shipping a product nobody wanted to open twice.
Accepting a yes without a cause
A yes you cannot explain is unrepeatable. Without identifying the specific pull — fun, mastery, sharing — you have no idea which component to amplify and will improve the wrong things.
Being demoralised by the opening noes
Lee is explicit that it never starts at one hundred percent and always starts as a whole bunch of noes with long explanations. Treating that as evidence the concept is dead kills products at exactly the stage where the feedback is most useful.
Is it for you?
Best for
Early-stage designers and founders testing rough prototypes who need a cheap, unambiguous signal.
Not ideal for
Mature products optimising conversion or retention at the margin, where finer-grained instrumentation is genuinely necessary.
From the transcript
“we do all these play tests we don't even give out surveys anymore there's only one question we ask which is when you finished playing…”
“a game is done when 100% of the people say yes and it never starts that way it's always a whole bunch of nose”
“i try to identify what that thing is what that compelling that thing that kept pulling them back in and double down on that”
From the episode
#653: Elan Lee, Co-Creator of Exploding Kittens — How to Raise Millions on Kickstarter, Deconstructing Mega-Successes, Secrets of Game Design, The Power of Positive Constraints, The Delights of Craftsmanship, and The Art of Turning Fans into Superfans