An empty glass beaker with a single etched line at mid-height beside a jug not yet pouring.

6of 8

Decide what counts as demand before running the test

A click, a payment and a compliment tell you different things. Set the threshold and disclosure rules in advance, and protect participants’ trust.

Build & Validate
ExperimentationEvidence-led building

Adapted by the EWOR Team from an article on EWOR (opens in a new tab)4 min read

Compliments are not commitments

It's easy to mistake enthusiasm for demand. Someone praises your idea, clicks on your landing page or completes a task. Each action tells you something, but not necessarily what you need to know. The question is: what would genuinely convince you that your hypothesis is right?

The answer depends on your product, price, audience and setting. A payment might seem stronger than a compliment, but it isn't always meaningful. Decide what evidence would change your next decision before running the test, and agree on participant protections at the same time. It's harder to stay objective once you've seen the results you hoped for.

Write down what counts before the test

There are four elements you should note before testing your idea:

  • Target. The specific people and situation being tested.
  • Behaviour. The observable action connected to the hypothesis.
  • Threshold or comparison. The result that would change the next decision.
  • Window. Where and when the behaviour can occur.

This gives you a clear basis for interpreting results. If you wanted to see people complete a workflow, page views and verbal praise don't become validation just because the intended behaviour never happened.

Evidence might range from attention and information-seeking to concrete next steps, repeated use or a costly commitment. But there's no universal ranking. A payment may mean little if it's coerced, refundable, mispriced or made by the wrong customer. In another business model, repeated non-financial use might be the strongest signal.

Work out what else could explain the result

Even a well-designed test can tell you less than you think. A person's response might reflect any combination of:

  • The problem you're solving or the outcome you've promised
  • The price and payment terms
  • The channel you used to reach them
  • Your brand or presentation
  • The time and location of the test
  • The effort required to take part
  • Whether they believe the product will actually be delivered

If nobody buys, that doesn't automatically mean the problem isn't worth solving. The price might be wrong, the offer unclear or the test too demanding. Consider these possibilities before drawing conclusions. Where possible, change one important factor at a time. If your test can't distinguish between explanations, acknowledge that uncertainty.

Agree disclosure and safeguards before launch

Pretotyping helps you learn whether people want something before you've built it. But some tests work because participants believe they're interacting with a real product or offer. That makes trust part of the test itself.

Before launching, get appropriate human review of disclosure, consent, data handling and participant safeguards. Decide:

  • What you can disclose upfront without invalidating the test
  • Whether temporary non-disclosure is justified and permitted
  • When and how participants will be debriefed
  • How money, data, recordings and contact details will be handled
  • How harm or confusion will be prevented or remedied, and who can stop the test

This checklist isn't legal clearance.

If someone reaches a purchase page for a product that doesn't exist yet, tell them it's still in development and don't leave them feeling cheated. A pop-up or disclaimer doesn't automatically make every misleading offer or data practice acceptable.

Don't collect money unless the offer and its fulfilment or refund terms have been reviewed and can be honoured. And never enter someone else's shop, platform, brand environment or audience without permission.

Separate result from interpretation

After the test, resist jumping straight to a conclusion. Review:

  • Behaviour: Who participated, what did they see, and what did they do?
  • Test quality: Where did people drop off? Were there protocol changes or failures?
  • Participant impact: Were there complaints or harms, and how were they addressed?
  • Confounds: What else might have influenced the results?
  • Next decision: What will you do, and what remains uncertain?

Your conclusion should be one of four things:

  1. Hypothesis not supported: The evidence didn't back your expectation.
  2. Inconclusive: There isn't enough evidence to decide.
  3. Test invalid: Problems with the design or execution make the result unreliable.
  4. Next investment justified: The evidence supports taking the next bounded step, not necessarily building the entire product.

“People loved it” isn't a conclusion. You need to understand what participants believed they were doing, what the test showed and what might have influenced their behaviour. Manipulation, confusion or harm undermines both trust and the evidence itself.

Source: EWOR article (opens in a new tab)

No comments yet.

Be the first to comment on this signal.

Get the next signal before everyone else.

Frameworks and contrarian theses for outlier founders, straight to your inbox. No fluff, no recycled advice, weekly.

The EWOR Signals newsletter

Continue learning

Related modules