Monday, February 06, 2012

Shocker: Social Science Should Be Held To Scientific Standards

From Hot Air: WaPo/ABC ends sample transparency in national polling

The Washington Post and ABC News has a new national poll out today. It purports to show that Barack Obama has a 50% approval rating and that he would beat Mitt Romney in a head-to-head matchup. And heck, that might even be true, except for a couple of problems. First, this is a poll of general population adults rather than registered or likely voters, so it’s not even a proper polling type for the predictive outcome they claim.

More importantly, though, the poll series has dropped its reporting of partisan identification within their samples. It’s the second time that the poll has not included the D/R/I split in its sample report, and now it looks as though this will be policy from this point forward. Since this is a poll series that has handed double-digit partisan advantages to Democrats in the past (for instance, this poll from April 2011 where the sample only had 22% Republicans), it’s not enough to just hear “trust us” on sample integrity from the Washington Post or ABC.


This critique is entirely correct. I teach social science methods for a living, and I tell my charges if an opinion poll does not come clean about its methodology (including the breakdown of party ID) you have to throw it out as being untrustworthy. In fact, the only reason not to give this basic information is you have something to hide.

This Washington Post/ABC poll is neither social science nor journalism. It's advocacy.

No comments: