Do polls accurately reflect how we feel about the upcoming presidential election? It depends on who's conducting themand who's spinning the results.
IT SEEMS LIKE A NEW presidential election poll is released every three days. Do the numbers mean anything? “In the right hands, a poll is a really useful tool,” explains Missouri City native Paul Begala, a veteran political consultant who is a co-host of CNN’s Crossfire, “but in the wrong hands, somebody is going to get hurt.” Put another way: Assuming the numbers you’re seeing came from a professional pollster—and not your buddy Bob handing out a survey at the mall—they can provide a useful snapshot of how the nation is feeling. Collected over time, they show trends. But watch out for pundits, who don’t always interpret them correctly.
How so? For one thing, commentators sometimes place too much emphasis on single-poll, head-to-head results. In the past few weeks, you’ve probably read or heard something like “A new Gallup poll shows George W. Bush leading John Kerry among registered voters 53 percent to 47 percent,” followed by some analysis of why Bush is pulling away or what those numbers mean for Kerry’s election chances. But to any sensible political strategist, the results of one week’s horse-race poll don’t mean anything.
Why not? First, because head-to-head numbers are meaningless until the defining moments of a campaign—debates, conventions, battleground bus tours—have had a chance to influence the decisions of swing voters. Case in point: Michael Dukakis held a seventeen-point lead over George H. W. Bush following the Democratic convention in 1988. Oops. Second of all, a national head-to-head poll tells you only about the popular vote, not about the state-by-state allocation of electoral votes (ask Al Gore). But perhaps most important, the numbers from a single poll are potentially full of errors.
What kinds of errors? Polling firms typically survey a representative sample of the electorate over the phone—say, a thousand people (the number is posted in most news articles). That strategy is accurate only if the survey is truly random, meaning every citizen has the same statistical chance of being contacted. But even if the pollsters use advanced random-digit dialing techniques, they can’t change the fact that some people just don’t have phones (often poor people who are more likely to vote Democratic) or that others refuse to answer a survey (a high percentage of whom, some studies show, are Hispanic voters, who also lean Democratic). “If you’re not talking to a representative sample of the population,” explains Austin-based GOP pollster Mike Baselice, “it doesn’t matter what question you’re asking.”
Doesn’t a poll’s margin of error take care of that? Not necessarily. It won’t take into account inherent errors in a polling firm’s methodology, such as failing to use random-digit dialing or making phone calls during the daytime, when most people are at work. But even if the methodology is flawless, the margin of error (which is always printed with the poll results) is often overlooked by the media. In the Bush-Kerry example above, for instance, if the margin of error is 3 percent, Bush isn’t really leading Kerry; they’re statistically tied. Each could end up with a 50-50 split of the vote. So follow this rule: A candidate isn’t leading until the percentage-point difference between him and his opponent is more than double the margin of error.
So I shouldn’t even be looking at head-to-head polls? Right—at least not until the last week or so before the election, when most voters have likely made up their minds for good. Even then, says Baselice, look to trends, not single-time results. “One night—I don’t get excited about anything,” he says. “Two nights—I get one eyebrow up. Three nights in a row—we have data that can actually tell us something.” In the meantime, if you’re looking for clues, Begala recommends looking at other poll numbers. “I prefer the right directionwrong direction results,” he says. “For a president—more than a congressman or a governor—if only thirty percent of the country says we’re going in the right direction, as they do now, you’re gonna lose.” Of course, remember what we said about pundits.