As the 2008 general election approaches, politicians, pundits and citizens alike will all be looking to the polls to gain insight into the outcome of this year’s historic election. Yet one of the most accurate predictors during the recent primary season was not any of the prominent U.S. polling institutions, or any of the major media outlets or television news networks. Instead, it was from an unlikely source: an anonymous blogger writing under the name Poblano (which is a mild chili pepper), who later revealed himself as Nate Silver, a young statistical guru in the world of baseball stats.According to Newsweek, Silver pioneered what is now considered to be one of the most accurate statistical methods for predicting outcomes in baseball. The 30-year-old developed his algorithm as a hobby while working as a consultant out of college (it helped having access to powerful analytical software). After attaining fame for his novel technique, he began leveraging his predictive prowess by starting his own sports statistics company, called Baseball Prospectus. Using the same approaches he used in predicting baseball games--that utilizing the right historic data could help predict outcomes in the future-- he started applying these concepts to politics in his (formerly) anonymous blog, FiveThirtyEight.
Silver’s approach to parsing election data is based in regression analysis, a statistical method commonly employed by economists to tease out single variables from complicated data sets. Rather than taking current polling data at face value, Silver used demographic information along with prior election results to determine his predictions. For instance, to forecast the election results in a particular city, he would use demographic parameters such as age, sex and race to find the results of demographically similar congressional districts that had already held elections. His results were astonishing; according to Newsweek, he came within 20 delegates (out of a total of 1,700) in predicting the results of Super Tuesday, and had a solid record for predicting the other primaries held in March (although he did underestimate Clinton’s performances in Kentucky and South Dakota).
Silver’s results are significant, not just in the world of politics and baseball, but because it demonstrates the power of using demographic data to explain and predict the things going on in the world. For investors this may mean taking new approaches to looking at devastating events such as the foreclosure crisis, and determining what, if any, variables can be pulled out. While it is true that this type of analysis has long been used in market analysis, Silver’s novel approaches are sure to bring some much-needed fresh air into the stodgy world of statistics and demographics.
Labels: statistics
