5of 8
Fake the product to learn about the experience
You don't need to build the technology to find out whether people want your product. A human-backed prototype lets you test the experience before committing to the engineering.
Adapted by the EWOR Team from an article on EWOR (opens in a new tab)4 min read
Wizard of Oz: Test the interaction before the automation
Wizard of Oz is a prototyping method in which a person performs some or all of a product's work behind the scenes while users experience what appears to be a functioning system. Use it to test how people interact with a proposed product before investing in the technology to build it.
The method works best when you need to understand the user experience and can simulate the missing functionality consistently. It won't prove that the final technology can deliver the same speed, quality, cost or scale.
Begin with one decision
Before designing the experience, decide what you need to learn. Can users complete a workflow without guidance? Do they provide the right information naturally? Is the response useful enough to keep them engaged, or does the interaction leave them confused?
Turn these questions into observable behaviours. “People like it” tells you very little, whereas completion rates, repeat actions, corrections, escalations and points of abandonment give you something concrete to assess. The behaviour you track should connect directly to the decision you're trying to make.
Choose the simplest test that can answer your question. If you only need to know whether people will click on an offer, a fake-door test may be enough. Wizard of Oz is more useful when people need to experience the interaction itself before you can judge whether the idea works.
Design what the user sees and what the operator does
The frontstage is the smallest interface that makes the target interaction credible. Build only the screens, messages or physical touchpoints needed for that path.
The backstage is the operator system. Define:
- Who acts as the wizard;
- What information they receive;
- Which responses they may choose;
- Expected timing and quality bounds;
- What happens when the request falls outside the script;
- What is logged and by whom.
The wizard should not quietly rescue every confusing interaction.If they rescue users whenever the experience becomes confusing, you may end up testing their ability to improvise rather than the product you're considering building. Give the wizard clear boundaries so that differences between sessions reflect user behaviour, not changes in how the test is run.
Write the protocol before the session
A lightweight protocol helps keep sessions comparable. Before recruiting participants:
- Define the participant profile and recruitment condition.
- Give only the context needed to attempt the task.
- Detail the starting state and target task.
- Specify the wizard’s permitted actions and response library.
- Record behaviour, timing, questions, errors and workarounds.
- Make the stopping condition and the post-session questions clear.
- Document the disclosure, consent and debriefing plan reviewed and approved for the specific context before inviting participants.
Record where the simulation differs from the product
The most valuable insights often come from comparing what users experienced with what your actual product would need to deliver. A human operator can interpret vague requests, fill in missing information and recover from mistakes in ways your future system may not be able to replicate. If you overlook these differences, you risk validating an experience your product cannot reliably deliver.
After each session, document where the wizard exceeded, differed from or fell short of the planned product. Use three columns:
- Expected simulations: Behaviours the proposed product should be able to reproduce.
- Wizard advantage: Judgement, context or error recovery the human operator provided but the product cannot yet replicate.
- Unproven implementation: Capabilities the test did not validate, such as latency, accuracy, unit cost, security or scalability.
The same principle applies when you use an existing AI model to simulate missing functionality. Its capabilities may make your prototype feel more complete than the underlying product really is. Assess its fidelity, data handling and reproducibility separately, and identify what would need to change for your own system to deliver the same experience.
End with a build, revise or stop decision
Once you've run the experiment, look beyond whether people completed the task or said they liked it. What did their behaviour reveal about the problem, the workflow and the value they received? Which parts of the experience worked without intervention, and which depended on the operator?
Use those findings to decide what deserves your attention next. If...
- Users struggle with the core interaction, improve it and test again.
- The experience works but depends on a difficult technical capability, investigate that constraint before investing in the full build.
- The key assumptions hold up, build the smallest real component that lets you test what remains uncertain.
The test should leave you able to decide what deserves to become real. A convincing demo is not enough; the aim is to get a clearer picture on whether to to build, revise, investigate feasibility or stop.





No comments yet.
Be the first to comment on this signal.