Or: Correlation causation smackdown, the end

I post a link to Richard Feynman’s CalTech address. This essay should be read about every 2-3 months to remind oneself of the ridiculousness that exists in the world (the ridiculousness called social science (not all social science, but certainly most of it)) and to inspire oneself to fight against it.

To be read after Feynman’s essay has been read and digested:

Correlation and causation and the rat

Correlation without causation is dangerous and destructive and the rat in the maze experiment makes it very easy to demonstrate why this is so. You do an experiment and have some reason to believe that the rat has some cue to be able to solve the maze. You want to find out what this cue is because you believe that in doing so you will come to understand the world better. (Never mind that this might itself not be true. Let’s take it as given that if you learn how the rat gets through the maze then that is interesting knowledge.) You paint the doors a certain color. This does not affect the outcome. You paint the correct door a different color. This does seem to affect the outcome: the rat chooses the differently colored door. You have a perfect correlation. You decide to rotate through all of the doors, painting each a different color while leaving all others the same color. Now you discover that the rat still finds its way out of the maze; the results are perfectly anti-correlated, still a correlation but with the opposite sign. This still seems to be something. You still have a magnificent R^2 value. You do the same thing as above (cueing the rat with either perfectly correlating or anti correlating ‘things’) but now with marks on the doors (all of which are now the same color). So now the rat has some pattern recognition or anti-recognition ability. Magnificent! And you do the same thing but now with heat sources behind the doors. Triplly magnificent! The rat can sense heat.

You’ve done a tremendous number of experiments because, as a good “scientist”, you’ve done then all individually, and you’ve also done them all together for all the doors. So, if there are 20 doors in the maze you’ve done: 20 + 20 + 20 (all individually) + 400 + 400 + 400 (coupling two of the experiments in all possible combinations; AB, AC, BC) + 8000. You’ve done over 9,000 experiments and seven years later write a paper and a thesis on “Evidence that rats are pattern-recognizing, color-aware, heat-seeking missiles” and get your PhD.

Of course, correlation or anti-correlation and 9,000 experiments aside you’ve learned nothing of the world. As Mr. Young discovered, the rat listens. You can correlate as many other things as you want with the rat’s behavior. None of those correlations mean anything. None of them tells you anything about the world. Not even the correlation that the rat is listening tells you anything about the world until you know that there is CAUSATION lurking behind the correlation. Or, as my favorite philosophy professor would have said: Do the clock that is not functioning but happens to be pointing at Noon when it actually is Noon tell you anything?

People, science is hard. Correlations don’t help. Don’t join a cargo cult.

-JD Cross

Advertisements