Polls, Again

You all know what I think about polls by now. While I'm not a professional statistician, I do work with statistics and have a fairly pronounced level of cynicism about how easily numbers can be manipulated.

My main beefs about political polls are these:

1. First and foremost, POLLS ARE NOT VOTES. At best, they are predictors. Fluctuating predictors at that. I get so annoyed at hearing, six, nine, or even twelve months before Election Day, that "your man is behind in the polls" as though that were significant in some way. It's not. It ain't over 'til it's over. Worrying about polls too far in advance of an election is like worrying about stock prices on a blue chip in your retirement portfolio: if you're not planning on selling, who cares? Unless there are serious indications that something is wrong, take a Zanax and chill. It's just market jitters.

2. And who is being polled anyway? Are they likely voters, registered voters, "certain to vote" voters? Did they even bother to vote in the last few elections? How do we know? Most of the time, they're not even registered voters. Why should I care about the opinions of people who can't be bothered to register?

3. Fake runoffs: were all the candidates included? Please don't leave off Ralph Nader. And spare me the ubiquitous "unnamed opponent" - he or she won't be on my ballot come November 2nd.

4. How was the poll question phrased? "Please tell me you're not stupid enough to vote for that incompetent moron, George Bush when we all know he was selected, not elected in 2000" is not an unbiased poll question.

5. Methodology/Sampling Bias- this is the hardest to assess: Have the numbers been 'massaged' in some way to account for real or perceived defects in the sample? Or worse, was the sample flawed in the first place? We assume the sample will be a unbiased one: in other words, that if the subjects were truly chosen randomly, their overall characteristics will reflect the general population. But this is often not the case due to sloppiness. A recent case in point was the Newsweek poll that touted a Kerry comeback.

Hindrocket at Power Line pointed out (Good Lord, I cannot believe I remembered this, much less found it...) that the before and after samples were skewed by party affiliation, thus biasing the conclusion that Kerry's numbers had surged (unless one assumed an unprecedented number of voters would cross party lines):

Reader Meg Kreikemeier points out that according to RealClearPolitics, Newsweek's most recent poll included 345 Republicans, 364 Democrats and 278 independents. This compares to Newsweek's published data for their most recent prior poll, which showed President Bush with a comfortable lead: 391 Republicans, 300 Democrats and 270 independents. Yes, if you drop 46 Republicans and add 64 Democrats, you will get considerably better results for the Democratic nominee. This is a good reminder of why poll data always need to be taken with a grain of salt, especially until you see the underlying data.

My reservations about polls out of the way, Daly Thoughts has a good analysis that's well worth reading on the latest Harris poll results. We're close enough to the election now that these things are getting more relevant and I thought this was a thoughtful and interesting analysis:

Using one definition of likely voters, those who are registered to vote and are “absolutely certain” to vote, the poll shows President Bush with a modest two-point lead (48% to 46%). Using this definition but excluding all those who were old enough to vote in 2000 but did not do so, President Bush has a commanding eight-point lead (51% to 43%). This second definition has proved more accurate in the past, but there are some indications that in this election many people who did not vote in 2000 will turn out to vote, in which case it would be wrong to exclude them.

Check it out - great post.

- Cassandra