Karl Popper
Go looking for the counter-example
One black swan beats a hundred white ones
The Logic of Scientific Discovery
No number of white swans settles the question. This page is about what you should be collecting instead.
What actually happened?
The swan is his illustration of the problem of induction. However many white swans you observe, none of it establishes that all swans are white, and a single black one settles it. Evidence is wildly asymmetric. A confirming instance carries almost no information; a refuting one is decisive. People run the other way by default, gathering supportive cases and feeling steadier at the hundredth than at the first. So his method: propose boldly, then attack your own proposal as hard as you can.
Go and ask the people who left
Where you take the sample decides the answer. Ask current users whether they like it and the reply is close to guaranteed. The information lives with the ones who stopped. This is the most practical thing that falls out of Popper.
To test whether customers love the feature, do not survey the ones still using it. Interview three who cancelled.
Bold conjectures, severe attempts to refute
He is not asking for cautious hypotheses. He wants large ones, paired with serious efforts to destroy them. The two halves go together: conjecture without refutation is daydreaming, refutation without conjecture is timidity.
Put forward the aggressive plan, and name the three observations that would show the route is closed. Then go and look for them.
Confirmation bias is the factory setting
Hunting for support is pleasant and fast, because the motive sits on that side. Good intentions do not correct it. Only procedure does: make finding the counter-example a step on the checklist rather than something you remember to do.
Add a permanent field to the review template for the strongest argument against, and require the proposer to fill it in himself.
How do I use it today?
Where you are: the more you check, the more certain you get.
Ask first: in the last hour, were you collecting support or hunting a counter-example? What is the strongest evidence against, and did you look for it?
Where it goes wrong: making refutation a way of never deciding — the point is a judgement you can rely on, not a paralysed one.
Lines to keep
No number of white swans establishes that all swans are white.
One black swan is enough to settle it.
It is easy to obtain confirmations for nearly every theory, if we look.
Same situation, other people are asking
Everything I find supports me. Should that worry me?I want this to be true, and I'm the one checking it.The numbers came out badly. Do I fix them or fix my view?What would have to happen for me to admit I was wrong?I might be fooling myself · all 6 questions →If this one named what you are going through,
send it to someone who needs it, or keep it somewhere you will find it again.