I despise polls.
I despise the fact that people think the mood of a nation of over 300,000,000 can be determined based on the answers given by 1,012 adults who happen to have phones, were by their phones, and were not busy or lonely enough to take the time to respond to some worker ask questions about this and that.
Regardless if it’s asking who would make a good President, whether the Patriot Act is a good thing, or if you prefer paper over plastic, polling is a lazy way to make policy.
In addition to the aforementioned problem of the type of people being asked (have phones, by their phones, not doing else important), polls can be skewed depending on how the question is worded. Carl Bialik of The Wall Street Journal writes a much better piece than I could about this phenomenon. Bialik writes about how a presidential approval rating poll comes out differently depending if the option of “somewhat” is used. He also talks about how results can be different if more options are given to a question.
However, what I despise more than polls is the conclusions drawn from those polls and in this respect, I heap most of my disdain on media outlets for being incredibly lazy (and not the first time I’ve called them to task for being lazy).
My case in point today is a Gallup poll that came out that asked respondents if the health care overhaul legislation should be repealed or kept in place.
You will notice, if you go to the link provided above, that out of 1,012 adults asked, 47 percent favor repealing and 42 percent favor keeping the law in place.
With those numbers in hand, media outlets touted that the American people had spoken and wanted the law undone…
From Rueters: More American than not want health law repeal
From RTT News: Poll Shows More Americans Favor Repealing Healthcare Reform Law
From The Weekly Standard: Americans Want Repeal
From UPI: Americans lean toward healthcare repeal
And even from Gallup itself: Americans Tilt Toward Favoring Repeal of Healthcare Law
There’s just a slight problem. The poll has an error in it.
This error is so well-known that the pollsters even mention it at the bottom of their poll. Here it is…
For results based on the total sample of national adults, one can say with 95% confidence that the maximum margin of sampling error is ±4 percentage points.
The error that I speak of is called “the margin of error“. What this means is that the percentage numbers cited by any poll can actually be plus or minus the margin of error figure. In Gallup’s poll, this means that the 47% that want to repeal the health care law is actually somewhere between 43% and 51%. This also means that the number who want to keep the law as it is between 39% and 47%.
Have you noticed that, after taking the margin of error into account, these percentages now overlap?
There is a scenario, given the margin of error, that 47% of the 1,024 adults asked want to keep the law as is and that 43% favor repealing.
There is another scenario, given the margin of error, that 45 percent of the respondents favor repeal and 45 percent favor keeping the law as is. That’s a tie in my book.
Instead of any thoughtful analysis of what the numbers actually say or the discussing the folly of divining the mood of a population by asking a sliver of its inhabitants, the media is lazy and writes headlines that fit their pre-conceived notions because thinking is hard…and there’s always another story that needs to be written in our 24-hour news spin cycle.
End Note: Proper respect must be given to MyFox (Fox 26) in Houston, Texas, for at least putting some proper perspective on the Gallup poll. Their headline reads Poll Says US Nearly Divided on Health Care Reform and their article even goes on to state that “…it seems more Americans want the law repealed than those who don’t”.
Nice to see that at least one media outlet has its lights on.