Irrelevant Events like College Football Games Do Not Likely Influence Elections

Previous studies claim that politically irrelevant events influence voting behavior. For example, droughts, floods, tornadoes, shark attacks, and even college football games purportedly influence elections—with voters typically punishing incumbent politicians for bad outcomes that were out of their control. Of course, just because an event wasn’t caused by a politician doesn’t mean it shouldn’t affect election outcomes (see here), but it’s hard to imagine why rational, reasonable voters would punish their incumbent U.S. Senator because their favorite football team lost. So are voters idiots? If events as politically irrelevant as college football games affect election outcomes, we should probably worry about the competence of voters, the health of democracy, and the quality of electoral accountability (although this paper shows that while irrational/incompetent voting is bad for electoral selection, the effects on electoral incentives are theoretically ambiguous).

Andrew Healy, Neil Malhotra, and Cecilia Mo published the first study on college football games and elections in 2010. They implemented a compelling design that exploited the quasi-random nature of sports outcomes, and they found that the incumbent party appears to perform better in presidential, gubernatorial, and senatorial elections when the local college football team won versus lost in the 10 days before the election. Although we found this design to be compelling—that is, we don’t have any reason to think that these estimates are systematically biased, we worried that this result was a chance false positive. Ex ante, we had little reason to think that college football games meaningfully influence elections, the statistical power of this test was low, and this is the kind of result that would have not likely been published if the authors had obtained a null result. This is precisely the situation where one should be especially skeptical of published claims (see here). Although the authors had a compelling design and followed the standard practices of their field, their result could nevertheless be attributable to chance rather than a genuine phenomenon among American voters.

Typically, concerns about false positives can be addressed through replication. But pure replication is infeasible in this setting since we can’t re-run the last 50 years of college football games and elections. So instead, we decided to reassess this finding by testing additional hypotheses that should hold if the original finding were genuine. For example, if college football games affect the vote shares of the incumbent party, we would expect that the effects of college football games should be stronger in places where people care more about college football, in the home county of the team rather than elsewhere in the state, and in places where incumbents were seeking reelection. We might also expect to see similar effects for NFL games. We found that none of these predictions held up, and we published the results in 2015, concluding that the previous finding on college football and elections was most likely a chance false positive.

In a recent Journal of Politics paper, Matthew Graham, Gregory Huber, Neil Malhotra, and Cecilia Mo revisit this question and claim to (re-)reassess the effect of college football on elections by collecting new data and preregistering their analyses. They find that after they add new data, the estimated effects of college football are weaker but the sign of most of the estimates is still in the expected direction. They conclude that “the body of evidence presented here is broadly supportive of HMM’s [Healy, Malhotra, and Mo’s] conclusion that college football games affect election outcomes.”

While we applaud the general approach taken by Graham et al., we believe they have misinterpreted the evidence. In our own forthcoming Journal of Politics paper, we use simulations to demonstrate that their results are very much in line with what we would expect if the original result was a chance false positive, and much weaker than what we would expect if the original result were genuine. If the result were genuine, it would have become statistically stronger when more data was added, but it got weaker. Of course, most of their estimates are still in the expected direction because they include data from the previous study that might have been a false positive. If we focus on just the new data—something Graham et al. never do—we see that most of the estimates go in the “wrong” direction. Furthermore, we can statistically reject the possibility that there is a genuine effect of college football games consistent with the magnitudes reported in the original study. If there were such an effect, it would have been very unlikely for Graham et al.’s estimates to be as weak as they were.

Substantively, we conclude that college football games likely do not meaningfully influence elections. Additional analyses by Graham et al. suggest that previously published results on tornadoes, droughts, and floods are also likely false positives or overestimates. And other research suggests that previously published findings on shark attacks were also likely false positives. So we’re left with little compelling evidence that politically irrelevant events affect elections. Perhaps voters are more competent than previously thought, and perhaps electoral selection and accountability work better than these studies suggest.

Methodologically, we hope this debate proves useful to researchers who are trying to adjudicate between genuine findings and false positives when independent replication is not feasible. Collecting new data is a great way to reassess previous findings. But researchers should think about the alternatives between which they are hoping to adjudicate, and they should collect data and design tests that have the potential to distinguish these alternatives. Simulations like those in our paper may be useful to future researchers who want to know whether a previous finding is genuine or not.