Can you measure the politics of social science?

An interview with James Manzi about his new paper

In March 2026, the journal Theory and Society published a sweeping analysis of academic social science research spanning 1960 to 2024. The paper, “The ideological orientation of academic social science research 1960-2024,” ran over 600,000 article abstracts through a large language model to map the ideological orientation of an entire field across six decades.

James Manzi, a DPhil (PhD) student in sociology at the University of Oxford, unleashed a wide-ranging discussion across social media with the publication of his paper, as researchers around the world discussed the methodology, findings, and implications of the research.

In the conversation below, Manzi walks us through the paper’s central findings, responds to questions about methodology, and sketches next steps. The interview has been lightly edited for clarity and brevity.

Justin McBrayer: For the benefit of our readers, I want to start with a high-level overview of your findings. Your paper draws five conclusions:

1. Roughly 90% of politically relevant social science articles leaned left 1960–2024, and the mean political stance of every social science discipline was left-of-center every year during the period under review.

2. All disciplines showed leftward movement between 1990 and 2024.

3. Policy-proximal disciplines generally showed limited rightward moderation between roughly 1970 and 1990, though policy-distal disciplines did not.

4. Disciplines with greater leftward orientation generally displayed greater ideological homogeneity.

5. Sociocultural content was more consistently left-leaning than economic content, and that gap widened over time.

In your view, which of these findings is most important and why?

Figure 6 from Manzi (2026) showing change in political stance of social science over time; Political stance: 0 = Far Right, 10 = Far Left.

James Manzi: The most important finding is the consistency of the left-of-center orientation across all disciplines and across the full 1960–2024 period. That pattern is not driven by a small number of extreme cases, but by the relative scarcity of right-of-center work in most fields. This makes it a structural feature of the published output rather than a transient fluctuation. It also provides a baseline for thinking about how ideology might shape research agendas over time.

McBrayer: You did this analysis using ChatGPT-4. It’s amazing that we can do things like this — it would take a human reader years just to read all of the abstracts, much less code them! However, LLMs have well-established biases of their own. To cite just one example, David Rozado has shown that when LLMs answer questions on standard political orientation tests, they are routinely categorized as left-of-center. Given this, why should we trust the output of a LLM analysis of political ideology?

Manzi: The key point is that the results do not rely on taking the model’s outputs at face value without validation. The paper tests the method against multiple external benchmarks, including blinded think tank texts, where the model places content very accurately on the scale. It also shows extremely high stability across repeated runs and robustness to alternative prompts and model choices. If model bias were driving the results, we would expect those checks to fail or produce unstable estimates, which they do not.

McBrayer: You conclude that 90% of the politically relevant abstracts lean left in the period under review and that the leftward lean becomes more apparent over time. But, of course, ‘left’ is a relative measure, not an absolute one. In general, what counts as conservative or liberal is always measured against a contrast class. For example, there are likely positions that would have been described as liberal in the 1960s (where this analysis picks up) that were mainstream in 2024 (when this analysis ends). So, why think that academic papers have moved left rather than thinking that mainstream culture moved right?

Manzi: The analysis uses a fixed 2025 reference frame precisely to address that issue. By holding the scale constant, it measures how texts from different periods map onto the same contemporary ideological benchmark. That means the results are about how the content of published work would be interpreted today, not how authors understood themselves at the time. Within that frame, the observed shift reflects changes in the composition and positioning of research output rather than changes in the scale itself.

McBrayer: Stephen Colbert famously quipped that “reality has a well-known liberal bias.” The aphorism makes me think that something like that could be going on here. In other words, it’s not clear that the leftward tilt of social science is a distortion. Perhaps it’s not that the research is biased but that good research happens to confirm liberal assumptions over conservative ones. In the article, you mention this possibility by suggesting that it’s possible that “sustained, disinterested inquiry into social phenomena has arrived at conclusions that happen to align more closely with liberal than conservative viewpoints.” Can you explain this interpretive limit on the study? Why doesn’t the paper show that social science is biased, and what would show such a bias?

Manzi: The paper measures ideological orientation, not the truth or falsity of the claims being made. A body of work can lean in one direction and still be correct if the underlying evidence supports those conclusions. Demonstrating bias would require showing systematic distortion — such as selective use of evidence or consistent error in one direction — which this study does not attempt to do. The results therefore describe a pattern in output, not a judgment about its validity.

McBrayer: OK, so pivoting from methodology to significance, what is this research really telling us? I was struck by the explanation that you offer for the leftward shift: “…positional change in stance scores is driven primarily by the kinds of abstracts entering the publication stream — new contributors and subfields whose work is classified farther to the left under the fixed 2025 rubric — while within-author shifts exist but play a materially smaller role under the same evaluative frame,” (p. 25). As I read this passage, it means that social science research is moving left not because the same body of scholars is producing different results over time but because the body of scholars itself is changing. People are driving the leftward shift. Is that right?

Manzi: That is broadly correct. The evidence suggests that most of the movement comes from changes in the mix of contributors and subfields entering the publication stream, rather than large shifts in the positions of the same individuals over time. In other words, newer areas of research and newer cohorts of scholars tend to be positioned further to the left under the fixed scale. Within-author changes do exist, but they appear to play a smaller role in the aggregate trend.

McBrayer: What’s the epistemic upshot? Should your findings undermine our trust in social science research?

Manzi: It should not be read as a reason to dismiss social science research. The findings show a pattern in how research is positioned, but they do not evaluate the quality of individual studies or the validity of their conclusions. At the same time, any persistent asymmetry is worth understanding because it can shape which questions are asked and how results are interpreted. The appropriate response is closer scrutiny and replication, not blanket skepticism.

McBrayer: Given all of this, what should we do? If universities were to commit to truth-seeking and knowledge transmission as their highest goals, how would this change the stream of social science research?

Manzi: Universities already articulate truth-seeking and knowledge transmission as core goals, so this question is best understood as asking how those goals are operationalized in practice. This study does not evaluate institutional performance or prescribe specific reforms, but it does highlight a measurable pattern in research output that may be relevant to ongoing discussions about how inquiry is conducted and evaluated. The appropriate response is to continue applying standard scientific norms — transparency, replication, and open debate — to ensure that findings are as robust as possible.

---

To learn more about HxA’s work on viewpoint diversity in higher education, see our website.

---