In 1620 Sir Francis Bacon published Novum Organum and kicked off the scientific revolution by defining its basic method: hypothesis, experiment, and result. By 1687 we had Newton’s Principia, and the rest is history.
Today, public primary schools teach the Scientific Method. It’s well known.
It turns out that following the method is not so simple. People, including scientists, are not perfectly rational. People have biases and even when we try to be good, sometimes, unconsciously, we do wrong. When the outcome of an experiment has career implications, things start to get complicated. And when the outcome has financial implications for a powerful institution, people have been known to actively game the system. For example, starting in 1953 the Tobacco Industry Research Committee waged a war on truth, until it was dissolved as part of the master settlement in 1998.
The stakes are high. Tobacco killed 100 million people in the 20th century. Climate change threatens our very way of life.
In the years since the 17th century, science has developed a much more detailed playbook for the scientific method, in order to defend itself against bias. Double Blind experiments are an essential part of the modern gold standard scientific method.
What is a double blind experiment? Consider this scenario:
A pharmaceutical company develops a new arthritis pill, and hires you to prove its efficacy. The obvious experiment is to give the drug to a group of people, and ask them if it relieved their pain. What’s not so obvious is that you should also have a control group. These subjects get placebos (inactive pills) and this is done without the subject’s knowledge of which kind of pill they get. That’s a single blind experiment, and the idea is that it prevents the subjects’ expectations or desires that this pill will do something that influences their reported results.
There is a remaining problem, however. You, the experimenter, also have expectations and desires, and those could be communicated to the subjects, or influence how the data is recorded. Another layer of blindness can be introduced, so the experimenter does not know which pills are which as they are administered, the subjects are surveyed, and the data is collected. Normally this is done by using a third party to randomly assign the subjects to the groups, and keeping the assignment secret from even the researchers, until after the experiment is complete. The result is a double blind experiment: Both the subjects and experimenters are unaware of who got what.
The first single blind experiment was performed in 1784 by Benjamin Franklin and Antoine Lavoisier. They were commissioned by the French Academy of Sciences to investigate Franz Mesmer’s claims of animal magnetism. The claims were debunked.
The first recorded double blind experiment was done in in 1835 in Nürnberg, in what was then Bavaria. Friedrich Wilhelm von Hoven, a public health official and hospital administrator got into a public dispute with Johann Jacob Reuter, who claimed odds were 10 to 1 that a single grain of salt dissolved in 100 drops of snow-melt, and then diluted 30 times by a factor of 100 each time would produce “extraordinary sensations” in one who drank it. Twenty-five samples of homeopathic salt-water and 25 samples of plain distilled water were randomly assigned to the subjects. The assignment was sealed, and the water was administered. In the end, 8 subjects did report feeling something, but 3 of those had actually had plain water, the placebo. Reuter lost the “bet” by the rules they had agreed on in advance.
That was huge progress, and science and medicine have come a long way as a result.
Invaluable though double blind experiments are, the process is still imperfect. The existence of one double blind study cannot be considered conclusive. If the question is of consequence and you do your research you will like find many competing studies, authors, and institutions. Reputations and careers come into play. Research labs get long-term funding from corporations and governments who have skin in the game, and over time their influence works its magic.
Truth can be lost in the hall of mirrors of the Internet.
There is a constant struggle in science to distinguish signal from noise, to discern a pattern in experimental data, and to come up with a theory that explains it all. That’s hard enough! But there is another, deeper struggle, as well, against bias and influence, both in ourselves and in society.
This struggle is not just against our ignorance, but also against intentional adversaries to the process. Over and over again, blinding has proven itself as a key to fighting biases, and discovering truth more quickly.
It’s ironic that controlled blindness is an engine of insight, and even perhaps a cornerstone of our civilization.