Defending the UT/Tribune poll’s methodology
The Texas Tribune’s web site has a lengthy (26 pages, many of which involve a lot of funny looking symbols and squiggly lines and tiny letters that aren’t parts of words of just the sort that caused me, after my freshman year at Rice, to change my major from math to history) explanation of the UT/Tribune poll’s methodology. I am going to post the abstract. Just don’t ask me to ’splain it. Read at your own risk.
SAMPLING FOR WEB SURVEYS
DOUGLAS RIVERS
STANFORD UNIVERSITY AND POLIMETRIX, INC.
ABSTRACT. Web surveys are frequently based on samples drawn from panels with large amounts of nonresponse or haphazard selection. The availability of large-scale consumer and voter databases provides large amounts of auxilliary information for both panelists and population members. Sample matching, where a conventional random sample is selected from a population frame and the closest matching respondent from the panel is selected for interviewing, is proposed. It is shown that under suitable assumptions (primarily ignorability of panel membership conditional upon the matching
variables), the resulting survey estimates are consistent with an asymptotic normal distribution. Simulation results show that the matched sample estimators are superior to weighting a random subsample from the panel and have a similar sampling distribution to simple random sampling from the population. In an example involving the 2006 U.S. Congressional elections, estimates using sample matching from an opt-in Web panel outperformed estimates based on phone interviews with RDD samples.
For a link to the complete article, click here.
Tagged: UT/texas tribune poll.
M. Gomez says:
Un, ok. Mah favor-ite statistic: 2/3rds of every sandwitch is made of bread
Reply »
Jamie says:
From what I can tell, this was not some “web-based poll” as people think of polls on the internet where you can just go and vote on it a million times and rig the poll. It was basically the same methodology as a phone poll, but on the web. Am I wrong?
Reply »
Vince Leibowitz Reply:
November 6th, 2009 at 3:02 pm
Jamie,
It is nowhere near the same methodology ore reliability as a telephone poll. Among other things, it requires participants to self identify in every way: where they live, age, race, income, registered voter status, etc. Basically, anyone who wants to win PollingPoint tote bags can participate and say whatever they want about who they are. They can put out as many .pdfs about methodology as they want, but the poll is unreliable.
Consider that when telephone pollsters start out on a poll, they have a list of verifiable registered voters with phone numbers. Not so for this kind of poll. For all we know, they polled a bunch of 12-year-old girls who wanted PollingPoint tote bags.
There is a hilarious video that might help explain why it is not as good as a phone poll:
http://www.youtube.com/watch?v=-G_JoLTd8vk
Vince Leibowitz
Communications/Policy
Hank Gilbert For Texas Governor
Reply »
scooter Reply:
November 6th, 2009 at 3:48 pm
who is hank gilbert?
Reply »
Sam Reply:
November 6th, 2009 at 8:59 pm
Ehh, it’s no different than Survey USA and Rasmussen doing random digit dial polls. All their data is self-id. For a general election poll, or one of ‘adults,’ they can be properly weighted and give pretty good results. They just suck at primary polls, because people lie about whether they will vote in a primary. Primary and municipal elections are decided by the same small subset of the population that vote every time, but that doesn’t stop the rest of the population from pretending that they have an opinion and by god, they are going to do their civic duty and vote.
If you just asked past Dem primary voters, Gilbert would do better than 0.3% and Friedman would do worse than 19%. I don’t think Gilbert would have very strong numbers, but they wouldn’t be an embarrassment.
Reply »
Robert says:
Why worry about it or question it? It cannot possibly be wrong as it is from a not-for-profit group devoid of all outside influence!?
Reply »
scooter Reply:
November 6th, 2009 at 2:07 pm
really? not-for-profits are devoid of “all outside influence”? Have you bothered to read the new Texas Tribune yet? Non-partisan, not-for-profit, and non-biased??? Yeah right! If you believe that you probably also believe that Rick Perry has your best interests in mind as your governor.
Reply »
paulburka Reply:
November 6th, 2009 at 4:02 pm
Hey, Scooter, the Texas Tribune has only been up for three days and you’re already calling it biased? Aren’t you jumping the gun a little? As far as I can tell, they have run mostly straight news.
Reply »
Snarky Dem says:
Vince Leibowitz thinks that video is hilarious?? I laugh more watching coffee brew.
Reply »
Briscoe Democrat says:
Hank Gilbert has ZERO chance of winning statewide in 2010.
Reply »