Understanding selection bias is one of the most useful things I have learned as far as mental models goes. It’s one of those models that I’d consider a moderator in the sense that it helps prevent one from jumping to conclusions and addressing the wrong problem or a non-problem.
Put simply, selection bias (in the sense that I use it) is the bias for a certain type of person to select into a group, a product, a service, or a region.
This is a useful bias to keep in mind when thinking about supposed “effects” of different products, services, or policies. Rather than thinking, “oh, this product/service/policy created this specific outcome,” first ask yourself, “did the type of person who select into this create this outcome?”
A few quick examples:
Policy and Voting – Texas
A lot of Californians move to Texas. Understandable. California has an unbearable tax regime and is getting worse by the day. Texas has a good tax regime and is generally prosperous.
Texans worry that Californians will move to Texas and make it like California by voting for the same kinds of politicians and policies that turned California into a prospective third world country.
But in many cases, the kind of Californian who selects into Texas is the kind of person who wants to avoid the kinds of policies that ruined California. For example, transplants to Texas voted more for Ted Cruz than for Beto O’Rourke in the recent senate race there.
Policies and Voting – Immigrants
There’s no reason why this would end at people emigrating between different states. Many of the people who select into moving to the United States are the types who are explicitly trying to escape what their home countries have turned into.
Vietnamese immigrants are a great example of this. They tend to be even more anti-communist than native Americans. The type of person who chooses to leave Communist Vietnam for the United States is more likely to be somebody who opposes communism.
Higher Education and Earnings – Credentials
Higher ed policy is a really useful place to apply this bias.
If you look at earnings data, it looks like people who get higher education earn more money. But that’s not really because they went to university. Often it’s that the type of person who selects to university is more likely to pursue careers that pay better.
(Some good data on this is in Richard Vedder’s book Restoring the Promise.)
Higher Education and Earnings – Geography
The higher education selection bias extends even further. Elite schools appear to have higher median starting salaries. But it’s less that graduates from elite schools earn more because they are elite graduates and more that they earn more because they are more likely to move to high cost of living areas like Manhattan and San Francisco.
(Of course, you could say that’s because they were recruited from those elite universities, but that’s a separate question of where recruiters go.)
So if a school has (relatively) a lot of grads who go to high cost of living areas, of course the median starting salary will be higher than a school where most of the grads go to low cost of living areas.
These are just a few quick examples of selection biases being overlooked. I try to think about this bias any time I see before:after case studies or any claims about causes and effects. It’s been a useful tool for not falling into the same mental traps as the common person getting duped.