Dan Zahs pointed me to this online article http://www.surveysampling.com/frame.jsp?ID=theframe/2005/September3.html from one of our principal sample providers, SSI. The general point is this: the longer you stay in the field with a survey, regardless of mode, the better representation you get among respondents. There are a whole lot of factors that determine when a sampled person is most likely to respond to a survey request. In addition to general receptivity to surveys there are things like attitude toward the survey topic, lifestyle, and basic demographics. A couple of concrete examples:
- On customer sat surveys we sometimes see respondents with very positive attitudes toward the surveyed company responding earlier than those with less positive attitudes. So as the field period moves along, satisfaction scores may decline.
- On phone surveys where we do few callbacks we often see a bias toward people who are likely to be at home and relatively easy to reach (e.g.,the elderly and larger households) and against people less likely to be at home and more difficult to reach (e.g.,younger people and one or two person households).
This is terribly important because a fundamental assumption on every survey we do is that the sampled persons who did not respond are no different attitudinally or behaviorally from those who did. Put another way, for our results to be valid we must be sure that they would not change substantially if we interviewed everyone in the sample instead of just those who cooperated.
A number of factors work against long field periods including cost, client need for fast turnaround, and, in transaction surveys, the need to limit recall periods. With the Web especially we can turn surveys very quickly and clients appreciate that. But fast turnaround has its downside and we need to be alert to any potential bias it might introduce.