A few weeks back I recommended the special POQ issue on Total Survey Error. One particular article caught my eye, "Satisficing Among Reluctant Respondents in a Cross-National Context." The data for the paper is from the European Social Survey, a high quality face-to-face survey in 30 European countries. The article looks at three constructs of respondent behavior:
- Satisficing measured by four primary indicators (straightlining, item nonresponse, frequent selection of middle or extreme responses, and inconsistency among answers). Sound familiar?
- Reluctance defined by having refused to participate at least once and interviewer coding of reluctance to answer or apparent lack of effort in answering during the interview.
- Cognitive ability measured by education and age along with interviewer coding of the respondetn's apparent understanding of the survey questions and frequency of requests for clarification.
The first interesting finding is a significant association between satisificng and reluctance. It's not especially surprising that people who didn't want to do the survey to start with tended toward classic satisficing behaviors. More interesting still, this association falls apart when the authors controlled for cognitive ability. The bottom line seems to be that people who don't enjoy thinking tend to be reluctant survey respondents who, if we do convince them to respond, do a poor job at it. In the current language of online, they are "bad respondents."
What to do? The answer seems to be simpler survey tasks and generally lowering the response burden. As we know, there are people like this in pretty much every survey sample. If you think it's important to hear from them, then design accordingly. The widespread practice in MR of rooting them out, deleting them from individual surveys, and kicking them off panels seems misguided. (I can't help but wonder whether throwing some fancy Flash pages at them is a step in the right direction.)
Most importantly, I think this is one more example of how many elements of good survey practice are indifferent to mode. Some things, it seems, are almost timeless.