You lead product growth at a digital company in Florianópolis. In every meeting, strong opinions clash over what to change on the site and in the product, and the custom is that whoever gets paid the most decides —the HiPPO, the boss's opinion—. But almost every idea, even the experts', fails or moves nothing when tested; the value lies in experimenting cheaply and at high speed so the few winners emerge, often counterintuitive ones. Booking.com runs thousands of experiments at once; Bing found a tiny change in its ad format worth $100 million a year that had sat ignored in the backlog for ages.
The situation
A/B testing and growth experiments replace opinion with evidence: you split traffic, test variants, and let the data —not the hierarchy— decide. The counterintuitive idea is that almost every idea fails or doesn't move the needle, even the best people's; that's why the value isn't in getting it right the first time, but in testing a lot, cheaply, and fast so the few winners emerge. And those winners are often surprising: Bing discovered that a small change in how it displayed ads was worth $100 million a year, something nobody prioritized because it "seemed minor." Booking.com made continuous experimentation its engine, trusting the data over the HiPPO.
Here's the trap, in two directions. Shipping based on the boss's gut feeling —or big redesigns without testing— feels decisive, but since most bets are wrong, you rack up expensive, late failures that a cheap test would have ruled out earlier. And testing everything to exhaustion, demanding certainty on every detail produces analysis paralysis: few experiments, slow cycles, and you never reach the speed that makes winners emerge. The defensible move is high-speed, data-driven experimentation: many cheap tests, killing losers fast and scaling winners, with data ranked above opinion.