Less than a week ago I posted an item on this topic that more or less concluded that we don’t need to worry yet. Then I went to AAPOR and my nervousness on this issue is up a notch, maybe two.
The talk of the conference was a new report from CDC saying that 15.8 percent of US households do not have a landline and most of those use a wireless phone. (Here is the link to the report courtesy of Dan Zahs.) So the cell-only household rate is now estimated at 12.8 Percent. The report also reinforced earlier findings about the differences in reports of health behaviors in cell-only vs. landline households. These findings were discussed in formal sessions for almost two full days along with considerable attention to the practical issues of doing research with this population. Some highlights:
- While you can’t call cell phones with an automated dialer you can nonetheless call them and interview them. In the past there has been a great deal of concern about the charges incurred by the respondent, but with the evolution in pricing plans toward large blocks of minutes cell users seem to be less concerned about it. Nonetheless, one probably should be prepared to compensate them if they ask.
- There are sampling frames that one can buy from both SSI and Marketing Systems Group, but these frames are generally not as clean as what you get with traditional RDD frames. There tend to be lots of non-working numbers and very poor matches between expected geographic location (based on exchange) and actual location.
- For both of the above reasons, doing research with these folks is expensive, as little as three and as much as five times more expensive than RDD and the response rates are no better. And, of course, you get all kinds of cell phone users, most of whom also have landlines.
- There also is some experimentation around using a Web panel to identify cell-only households and then trying to convince those respondents to do a telephone interview. Getting them to the telephone avoids a potential mode effect. Of course, the problem with this strategy is that you are then mixing apples with oranges and weighting is very challenging.
I also had the opportunity to talk at some length with two people who are spending a lot of their time tracking all of this: Stephen Blumberg from CDC and Clyde Tucker from BLS. Clyde’s "night job" is doing electoral forecasting for the TV networks. To both I posed the key question: given what we know so far about the differences between cell-only and other households is the problem serious enough to have a significant impact on the overall estimates? Put another way, if we weight by age, gender and maybe an SES indicator like education, does the problem go away? They both had the same response. First, they said it depends on the questions. Clearly, health behaviors are an issue while the most recent research on political polling suggests that political attitudes and behavior are less affected. Second, they argued that this is a very public issue and at some point not including cell-only households raises issues about the credibility of the work, even though empirically the impact may be small. Clyde believes, for example, that by the time the 2008 election rolls around the cell-only rate will be very close to 20 percent and therefore wonders how one can possibly not include them in political polls.