To tell you the truth . . .

Those of us who are old enough to remember the early days of online may also recall that one troubling finding was the disparity between customer sat ratings across modes, that is, online vs. phone. Online always seemed to be lower. There were multiple hypotheses advanced but once the empirical work got going it seemed to come down to one of two mode effects: good old fashioned social desirability bias or simply differences in the way people use scales in a visual mode like Web as compared to an aural mode like telephone.

I admit to having always been skeptical of social desirability as an explanation when it comes to something like satisfaction with someone’s product or service. The literature on social desirability generally has focused on really heavy issues like drug use and abortion. Then I saw some stuff from Harris Interactive that claimed to find effects on behaviors as mundane as how often one brushes one’s teeth or goes to church. And now an article (with too many authors to cite) in the summer issue of POQ goes even further and seems to show that subjective items such as attitudes and opinions also are affected by social desirability bias. The experiment was not with Web, but rather using a method they call TCASI (for Telephone Computer-Assisted Self-Interviewing). In this approach a person is called and the interview started with conventional telephone. At one point the respondent is switched over to IVR and the interview moves from interviewer administration to self-administration. In the experiment respondents were randomly assigned either to full telephone or to TCASI. Respondents in the TCASI treatment were more supportive of traditional gender roles and corporal punishment, less supportive of integrated neighborhoods and same –gender sex, and more likely to agree that occasional marijuana use is harmless and to describe themselves as attractive. In other words, the telephone interviews were more likely to deliver socially tolerant and politically correct answers than were the self-administered interviews.

The thing I like about this study is that it doesn’t confound the issue with other factors in the way a straight Web-telephone study might. There is no visual dimension; it’s all aural. There also is no issue of comparing panel sample (non probability) with probability.

Of course, the key issue in these discussions always comes down to the simple question: what is the truth and therefore the better method for getting at it?