The Top Line On Polls:
Are poll results really that reliable? There are plenty
of flaws in the system
by Mitch Frank
Monday, Oct. 25, 2004
As the presidential race heads for its final days, the polls are being watched more closely than ever. They continue to diverge widely, showing everything from a 3% Kerry lead to an 8% advantage for Bush. How accurate can they be? Notebook asks some common questions.
Are pollsters sampling the whole country?
Pollsters usually interview about 1,000 registered voters
and, thanks to the magic of statistical math, 95% of the time those 1,000
accurately reflect the opinions of the entire country, give or take a margin
of error of plus or minus 3%. But some people are not reached. Polls are
conducted by phone, which leaves out about 8% of adult Americans, including
those in institutions (prisons, hospitals, military bases), some low-income
people and the approximately 4% of adults who have only cell phones. Then
there are the people who hang up on pollsters, which is happening more
often in the age of telemarketers. Most pollsters make 5 to 10 attempts
to reach a voter at varying times of day, and usually about half the people
they contact answer their questions.
How do pollsters compensate for such omissions?
Once they've finished interviewing, pollsters weight
the data. For example, 80% of adults are high school graduates, according
to the last census, so if only 60% of the respondents are high school grads,
the pollster gives their answers more weight, making them count as 80%
of the sample. Most election polls — including TIME's — weight for age,
sex, race, education and region of the country. A few pollsters, like Zogby,
also weight for party identification, to make sure there's a representative
number of Democrats, Republicans and independents in the sample. More traditional
pollsters, including TIME's, believe that party ID is not a stable factor
but varies according to which candidate the voter is supporting and how
well the candidates are doing.
How do pollsters decide who is a "likely voter"?
This is tricky. Pollsters assign each surveyed voter
a score based on the answers to multiple questions (such as "Did you vote
in the last election?") that indicate the likelihood that he or she will
vote. The highest-scoring voters are deemed "likely." How high a score
produces a "likely"? It depends. Pollsters first estimate what the turnout
will be on Election Day and then designate the same percentage of their
respondents — again, based on highest scores — as "likely." Assumptions
of who will vote thus have an enormous impact on poll results (especially
because minority and low-income voters, who tend to vote less consistently,
usually get lower scores). Predictions could be thrown off by big get-out-the-vote
efforts as well as unforeseen events in the final days of the campaign.
The End