There is an interesting article in the current issue of POQ that reports on a meta-analysis of the literature on the effect of advance letters on response rates in telephone surveys. The analysis shows that they continue to have an effect, increasing response rates on average at around eight or 10 points. Because the authors could not lay hands on all of the actual letters used in many of the studies their analysis of what works and does not work is somewhat incomplete , but in general I came away with two messages:
- Any contact with a sample member prior to the first call is a plus, even if it’s just a postcard.
- Follow the rules set out by Dillman and you will be ok (but pay less attention to this guidance on design of Web surveys). Our own white paper on the topic has good guidance as well.
While the research genuflects to the recent focus on non-response bias it really can’t address the key issue: do advance letters draw different kinds of people into the survey who otherwise might not participate and therefore increase representativeness or do they just add more of the same?