The Top Line On Polls:
Are poll results really that reliable? There are plenty of flaws in the system

by Mitch Frank

Monday, Oct. 25, 2004

As the presidential race heads for its final days, the polls are being watched more closely than ever. They continue to diverge widely, showing everything from a 3% Kerry lead to an 8% advantage for Bush. How accurate can they be? Notebook asks some common questions.

 Are pollsters sampling the whole country?
Pollsters usually interview about 1,000 registered voters and, thanks to the magic of statistical math, 95% of the time those 1,000 accurately reflect the opinions of the entire country, give or take a margin of error of plus or minus 3%. But some people are not reached. Polls are conducted by phone, which leaves out about 8% of adult Americans, including those in institutions (prisons, hospitals, military bases), some low-income people and the approximately 4% of adults who have only cell phones. Then there are the people who hang up on pollsters, which is happening more often in the age of telemarketers. Most pollsters make 5 to 10 attempts to reach a voter at varying times of day, and usually about half the people they contact answer their questions.

 How do pollsters compensate for such omissions?
Once they've finished interviewing, pollsters weight the data. For example, 80% of adults are high school graduates, according to the last census, so if only 60% of the respondents are high school grads, the pollster gives their answers more weight, making them count as 80% of the sample. Most election polls — including TIME's — weight for age, sex, race, education and region of the country. A few pollsters, like Zogby, also weight for party identification, to make sure there's a representative number of Democrats, Republicans and independents in the sample. More traditional pollsters, including TIME's, believe that party ID is not a stable factor but varies according to which candidate the voter is supporting and how well the candidates are doing.

 How do pollsters decide who is a "likely voter"?
This is tricky. Pollsters assign each surveyed voter a score based on the answers to multiple questions (such as "Did you vote in the last election?") that indicate the likelihood that he or she will vote. The highest-scoring voters are deemed "likely." How high a score produces a "likely"? It depends. Pollsters first estimate what the turnout will be on Election Day and then designate the same percentage of their respondents — again, based on highest scores — as "likely." Assumptions of who will vote thus have an enormous impact on poll results (especially because minority and low-income voters, who tend to vote less consistently, usually get lower scores). Predictions could be thrown off by big get-out-the-vote efforts as well as unforeseen events in the final days of the campaign.

The End