Tag: Bias

Science Sunday: A Big Social Science Oops

Wow:

Social science can be so amusing. There is a bit of a contretemps over several recent articles that used datasets supposedly measuring the personality traits of liberals and conservatives which has resulted in several abashed corrections. The researchers used the data in an effort to show that personality traits are not the cause of political attitudes, but instead both are correlated with some other factor, most likely genetic. Interesting enough. This finding is not what is being corrected.

Instead, what is being corrected is the rather casual assumption in the studies by the researchers that a personality factor identified in the datasets they used is supposedly associated with conservative political views. That factor is called Psychoticism. They hasten to explain that Pyschoticism is not the same thing as psychotic. The original article, “Correlation not Causation: The Relationship between Personality Traits and Political Ideologies,” in the American Journal of Political Science explains:

Having a high Psychoticism score is not a diagnosis of being clinically psychotic or psychopathic. Rather, P is positively correlated with tough-mindedness, risk-taking, sensation-seeking, impulsivity, and authoritarianism (Adorno et al. 1950; Altemeyer 1996; Eysenck and Eysenck 1985, McCourt et al. 1999). In social situations, those who score high on P are more uncooperative, hostile, troublesome, and socially withdrawn, but lack feelings of inferiority and have an absence of anxiety. At the extremes, those scoring high on P are manipulative, tough-minded, and practical (Eysenck 1954). By contrast, people low on P are more likely to be more altruistic, well socialized, empathic, and conventional (Eysenck and Eysenck 1985; Howarth 1986). As such, we expect higher P scores to be related to more conservative political attitudes, particularly for militarism and social conservatism.

Well, guess what. It turned out that they’d coded their spreadsheet wrong. Higher “psychoticism” scores actually correlated with liberal beliefs, not conservative ones. So their study, cited by many liberals as proof that Conservatives Be crazy, showed the exact opposite of their conclusions.

Digression time:

The best thing about science is that it has a corrective mechanism: someone else can do the experiment and check the results and see if they’re borne out. This mechanism works well in the physical sciences, where mechanisms are fairly deterministic — no matter how many times you drop a steel ball, it will always follow the same law of gravity. It works reasonably well in the biological sciences. In biology, systems are more complex and a bit more unpredictable. On balance, heavy drinking will kill you. But there are people who drink like fish and live long lives because genes or other factors or just plain luck keep them going. You also have a problem of reproducing experiments — I can mix chemicals over and over again and weed out the bad results. But I only get to do a 40-year study of people’s eating habits once.

In the social sciences, though, all bets are off. Part of it is that you are dealing with complex systems. Economies are complex, humans are complex and we only get to live out history once. Part of it is an “observer effect”. People behave differently or even lie to researchers when they know they are part of an experiment. For example, Sweden claimed the number of men who had ever seen a prostitute dropped massively after they imposed their “Nordic Model” on sex work, which only makes sense if massive numbers of Swedish men were struck dead by the legislation. In reality, fewer men were willing to admit they had because of the social pressure.

But it’s also ideological. Physicists, chemists, engineers and biologists tend to have a mix of political views; social scientists tend to be almost exclusively liberal. Physical and biological research only occasionally has big political implications (e.g., global warming, GMOs, evolution). And even in these cases, the science is not political; the science is politicized by opportunistic politicians.

But in the social sciences, almost everything has some political implication. So results that confirm the ideological bias of the researchers sometimes isn’t questioned too carefully. Massive tomes on income inequality are praised despite serious methodological flaws. Papers supporting Keynesian economics are taken as gospel despite huge flaws. Garbage research claiming massive amount of sex trafficking is used to inform policy.

An example more germane? A lot of people have claimed that Donald Trump’s supporters are authoritarian. This sounds about right to me except … that analysis is based on sociological debris. Here are the questions used to determine if someone is authoritarian:

Please tell me which one you think is more important for a child to have: independence or respect for elders?

Please tell me which one you think is more important for a child to have: obedience or self-reliance?

Please tell me which one you think is more important for a child to have: to be considerate or to be well-behaved?

Please tell me which one you think is more important for a child to have: curiosity or good manners?

Everyone claims this is the “definitive” test of authoritarian tendencies. Is it? Those traits seem to track culture far more than they track politics. My grandparents’ generation would have shown up as very authoritarian even though they voted FDR in four times and huge numbers of them had fled Europe because of the rise of authoritarianism. But because this test shows that conservatives are more authoritarian and all the sociologists believe that conservatives are more authoritarian, everyone accepts it.

But which is more authoritarian? Believing in a government that governs least? Or believing in a government that controls our lives? The problem here is that liberals don’t think of themselves as authoritarian even thought they are. If you believe in government controlling healthcare, education, retirement and half of the country’s wealth, you’re authoritarian, no matter how sincerely you believe that gays should be able to get married or how liberal your parenting methods are.

(This problem of nomenclature comes up a lot. I can’t find the link, but McArdle has written about a study that showed that liberals valued “fairness” more than conservatives. Every liberal scholar and pundit cited it was proof of how unfair conservative ideas were. But conservatives objected, arguing that wealth redistribution was not “fairness”. They saw it as plunder. Conservatives think that allowing people to keep what they’ve earned is “fairness”. In the end, the researchers agreed that people might differ on the definition of “fairness” and changed their word choice.)

In any case, this is yet another demonstration of how bias clouds the social sciences. This was a very basic error, something that even a modicum of checking would have shown. but no one questioned it, no referee gainsaid it, no one reproduced the results because it confirmed what liberals wanted to believe.