Nate Silver Rates The Polls

Regular readers will have heard me talk about Nate Silver (pictured) before. He writes the FiveThirtyEight blog, and I firmly believe he is the best poll analyst in this country. He received a lot of flack a few days before the election for predicting a fairly easy win for the president. But he was vindicated on election day when his predictions turned out to be uncannily accurate.

Now that the election is over, Silver has reviewed the polls and determined which were the most accurate in this election season. He examined their results for the last 21 days before the election, averaged the polls produced by each polling organization, and then compared them to the actual results of the election. (Note -- this examination is different from the previous poll analysis I posted about because Silver examined both state and national polls, while the previous post just looked at national polls.)

In all, Silver looked at the results from 90 polls, but he warned that the averages for those polls that had not conducted many surveys in those last 21 days may be suspect (since there is not enough product to get a truly reliable average). Here are the polls that produced the most surveys in the last 21 days before the election, along with their average error:

Note that two of the most respected and widely used polls (Gallup and Rasmussen) did very poorly -- with Rasmussen having an average error of 4.2 points, and the average error of Gallup was a whopping 7.2 points. These two companies need to examine what they are doing and make changes, because their reputation for accuracy has been seriously damaged.

If you would like to see the examination of all 90 polls, you can go to Silver's FiveThirtyEight blog. I was pleasantly surprised to see that a poll from my own state (University of Texas Poll) was very accurate, with an error of only 0.2 points. But they only did one state poll in the last 21 days before the election (so it can't be known if they are accurate on a long-term basis, or they just got lucky in that one poll).

One small surprise was how well the Google polls did, producing an average error of only 1.6 points -- much better than many older, bigger, and more established polls. It will be interesting to see if they can do that well in the future.