This report from AAPOR filed by Colleen Carlin (with apologies to Jimmy Cliff).
Finally, a sunshine-filled day here at AAPOR and maybe some good news for survey researchers…
Doug Rivers of Stanford University presented a paper on professional respondents in web surveys; having received the data only 24 hours prior to his presentation, in-depth analyses of the data are pending. A professional respondent is defined as someone with frequent survey participation, multiple panel membership, paid to take surveys, provide inaccurate answers and exhibit fraudulent behavior. A long, uninteresting survey was administered to respondents from four online panels. The study was limited to the state of Delaware in an attempt to maximize panel overlap. Traps were imbedded in the questionnaire as a way to capture inconsistent responses and 'cookies' were used to track the same respondent across panels. Numbers indicate that panel overlap can be as high as 25 percent. Panel membership for professional and non-professional respondents was as follows:
|
Number of Panels |
Non-professional |
Professional |
|
1 |
70.7% |
14.4% |
|
2-5 |
23.4% |
40.8% |
|
6-10 |
5.2% |
41.6% |
|
More than 10 |
0.7% |
3.2% |
Professional respondents tended to be female, lower education, older and come from low income households. As would be expected, professionals complete surveys faster and are less likely to report positive responses to industry screeners. The good news is that despite the presence of professional respondents a comparison with non-professional respondents revealed no difference in straightlining behavior, falling into 'traps', no difference in 'select all' screeners and no difference in a discrete choice experiment embedded within the survey. Doug had a lot of data to cover in his presentation and a lot of the findings went by too quickly to record. I have requested a copy of his presentation and will be writing up a more in-depth summary for use at Market Strategies.
The rest of the news from the day was not as good, but I'm going to leave things on a positive note for now, but look for a pending post on why list-assisted RDD really only covers 60 percent of the population.