The State of Telephone Research

"RDD telephone surveys are still the only scientifically valid alternative to face-to-face interviewing." — Michael Battaglia, Abt Associates

I’ve just come back from the Second International Conference on Telephone Survey Methodology (aka, TSMII).  This was a gathering of the best and the brightest survey methodologists and practitioners to assess the state of telephone interviewing.  The news is not good:

• Response rates on telephone surveys have been declining at the rate of 1.5 to two points a year over the last 10 years.
• While the refusal rate has been more or less constant, the non-contact rate has been increasing and driving the fall off in response rates.
• A well-designed RDD survey using a simple respondent rule such as last birthday, making 10 or more calls to a line of sample and with a serious refusal conversion effort is now generating on average about a 30% response rate.
• The RDD sample frame continues to deteriorate as the number of cell phone only households slowly increases.

So telephone research would seem to be in decline, except there still is nothing better to take its place.  The main candidate, Web, has the serious problem of having no viable sample frame from which to pull a probability sample.  The "best" Web studies are recruited by, you guessed it, RDD!

What’s a poor survey researcher to do?  The best advice would seem to be to pay more attention to nonresponse, learn more about how the people who do not do our surveys may be different from those who do, and develop better ways to adjust for it.  Probably the best news was that the effects of nonresponse are not necessarily large.  Some folks showed models to suggest that the bias might be only 2 or 3 percentage points change in estimates, while some showed larger effects more in the 7 to 8 percentage point range.

But all in all I came away thinking that this is not the most exciting time to be a telephone survey researcher.