A small football goal frame pegged firmly to the table with a ball resting just short of it and faint marks around its base.

8of 8

Iterate experiments without rewriting success criteria

A disappointing result does not always mean your idea is wrong. Learn to spot a faulty test without moving the goalposts or changing the offer until something works.

Build & Validate
ExperimentationFounder judgment

By the EWOR Team3 min read

Your experiment falls short. Maybe the wrong people saw it, the message was unclear or the offer simply failed to convince anyone. It is tempting to keep tweaking things until the results improve. But there is a difference between fixing a broken test and finding new ways to avoid an answer you do not like.

The challenge is to understand what went wrong without losing sight of what you set out to learn.

Set a small budget for diagnosed failures

Before launching, agree on what success looks like, how many iterations you are willing to fund and what would justify another attempt. Set aside a small budget for fixing identifiable problems with exposure, comprehension, the opportunity to act or measurement.

The key word is identifiable. If your invitation never reached the intended audience, you have a clear reason to repair the test. If you simply suspect that a prettier landing page might attract more people, you have a hypothesis to investigate, not proof that the original test was faulty.

Even when a repair is justified, preserve the original audience, offer, success threshold and result. Otherwise, it becomes difficult to tell whether you fixed the test or changed the question.

Distinguish a faulty test from a weak result

Start by checking whether the right people encountered the experiment, understood the offer and had a fair opportunity to respond. Make sure your data collection worked as intended.

If the invitation never reached its intended audience, you have learned little about demand. But if relevant people saw a clear offer and declined, that is evidence worth taking seriously, even if it is disappointing.

Do not dismiss refusals because people seemed insufficiently enthusiastic or redefine your audience around the people who said yes. If the sample was too small or the observation period too short, call the result inconclusive rather than treating it as either success or failure.

Record any identified fault alongside the original result, including the evidence that supports your diagnosis.

Keep revisions visible

Once you have identified a plausible fault, fix it or test one explanation within your agreed budget. Maintain the necessary consent, disclosure and site permissions throughout.

Be especially careful when changing the audience, price, delivery promise or acquisition channel. Each change can introduce a different question. If a lower price attracts more customers, you have learned something about the lower-priced offer. You have not proved that the original price or business model works.

Keep your original success criteria visible. If new information genuinely changes what the business needs to achieve, document the change and evaluate the revised hypothesis separately. Do not lower the bar simply because you missed it.

Know when to stop

At the agreed limit, put the original result and any follow-up results side by side. Explain what was faulty, what changed and what the evidence now supports.

Fixing an exposure problem may give you a clearer picture of demand. Changing the price or offer tests something different. Neither automatically turns an inconclusive experiment into a successful one.

Any further iteration should have a clear rationale and a deliberate decision to invest more time or money. The point is not to keep experimenting until you get the answer you want. It is to learn whether the problem was the test, the offer or the underlying idea.

No comments yet.

Be the first to comment on this signal.

Get the next signal before everyone else.

Frameworks and contrarian theses for outlier founders, straight to your inbox. No fluff, no recycled advice, weekly.

The EWOR Signals newsletter

Continue learning

Related modules