Don’t look now, but we’ve got a big data fight on our hands. On Saturday, just three days prior to Election Day, some of the simmering pressures and tensions in the world of poll aggregation exploded into a public feud on social media, with one of the biggest names in polling data pulling no punches.
FiveThirtyEight founder Nate Silver unloaded on Ryan Grim of the Huffington Post over an article accusing him of “unskewing” the 2016 polls and needlessly “panicking the world” about the prospect of a Donald Trump victory.
Silver was so incensed he tweeted that the article was “so fucking idiotic and irresponsible.”
Silver, 38, is no newcomer to this game, having earned plaudits for his successful presidential election forecasts in 2008 and 2012. He’s also absorbed a lot of heated (and often ill-informed) criticism along the way from the likes of partisan poll-truthers like Dean Chambers.
In the run-up to the 2012 election, Chambers devoted a lot of time to “unskewing” the mainstream media’s polling numbers, to make them more favorable to Republican nominee Mitt Romney. Chambers also hurled some thinly-veiled homophobia toward Silver, calling him “of very small stature,” “thin and effeminate,” and comparing him to a castrated character on Rush Limbaugh’s radio show.
Needless to say, Chambers was wrong and Silver and his polling model were right, a fact Chambers himself acknowledged after it was all over. But it’s fair to guess that having the old “unskewing” accusation thrown at him irritated Silver from the get-go. The reasoning in Grim’s piece—that Silver’s trend line adjustments, which account for previous pollster performance and national trends, amounted to “putting his thumb on the scales”—clearly infuriated him. So much so that he put Grim on blast on Twitter.
What followed was a 14-part tweetstorm from the polling maven, excoriating the article, Grim, and FiveThirtyEight’s rival polling models for failing to reflect the levels of uncertainty he sees in the 2016 data. It culminated in a pretty sharp paraphrasing of Michelle Obama.
This article is so fucking idiotic and irresponsible. https://t.co/VNp02CvxlI
— Nate Silver (@NateSilver538) November 5, 2016
The reason we adjust polls for the national trend is because **that's what works best emperically**. It's not a subjective assumption.
— Nate Silver (@NateSilver538) November 5, 2016
It's wrong to show Clinton with a 6-point lead (as per HuffPo) when **almost no national poll shows that**. Doesn't reflect the data.
— Nate Silver (@NateSilver538) November 5, 2016
Every model makes assumptions but we actually test ours based on the evidence. Some of the other models are barley even empirical.
— Nate Silver (@NateSilver538) November 5, 2016
There are also a gajillion ways to make a model overconfident, whereas it's pretty hard to make one overconfident.
— Nate Silver (@NateSilver538) November 5, 2016
If you haven't carefully tested how errors are correlated between states, for example, your model will be way overconfident.
— Nate Silver (@NateSilver538) November 5, 2016
Not just an issue in elections models. Failure to understand how risks are correlated is part of what led to the 2007/8 financial crisis.
— Nate Silver (@NateSilver538) November 5, 2016
There's a reasonable range of disagreement. But a model showing Clinton at 98% or 99% is not defensible based on the empirical evidence.
— Nate Silver (@NateSilver538) November 5, 2016
We constantly write about our assumptions and **provide evidence** for why we think they're the right ones. https://t.co/IhLKXdxGGK
— Nate Silver (@NateSilver538) November 5, 2016
That's what makes a model a useful scientific & journalistic tool. It's a way to understand how elections work. Not just about the results.
— Nate Silver (@NateSilver538) November 5, 2016
The problem is that we're doing this in a world where people—like @ryangrim—don't actually give a shit about evidence and proof.
— Nate Silver (@NateSilver538) November 5, 2016
The philosophy behind 538 is: Prove it. Doesn't mean we can't be wrong (we're wrong all the time). But prove it. Don't be lazy.
— Nate Silver (@NateSilver538) November 5, 2016
And especially don't be lazy when your untested assumptions happen to validate your partisan beliefs.
— Nate Silver (@NateSilver538) November 5, 2016
When you go low, I go high 80% of the time, and knee you in the balls the other 20% of the time.
— Nate Silver (@NateSilver538) November 5, 2016
When Silver acknowledges a “reasonable range of disagreement” before calling models that show Clinton as a 98 or 99 percent favorite as “indefensible,” it’s not hard to guess who he’s talking about. While some polling models like The New York Times‘ The Upshot have pegged Clinton as a much stronger favorite than FiveThirtyEight (she currently has an 84 percent chance of winning in its model, compared to FiveThirtyEight’s 64.2 percent), others have been far more certain.
In particular, the Princeton Election Consortium, run by longtime Silver rival Dr. Sam Wang, currently places Clinton’s odds at a staggering 99 percent. Wang has also pledged to eat a bug if Trump nets more than 240 electoral votes.
In short, it seems like the looming Election Day is bringing mounting pressure on some of the people who make these calls and interpret this data for a living, and in the case of Silver, his outrage at Grim’s article caused a bit of a boil-over. Fortunately, it won’t be much longer until the results are finally in, and all the arguing can be buttressed by some hard, concrete numbers.