Back for the second session. We are hearing from Peter Lynne and Olena Kaminska from the University of Essex. They are going to talk about survey error (the old TSE again) in mobile research. This is more focused on interviewer administered surveys rather than Web surveys on smartphones.
Olena has started by showing the potential errors at all stages of response formation—inability to hear the question clearly, distraction, low effort because of multitasking, unwillingness to answer truthfully because of possibly being overheard. I like this already. So they designed an experiment as part of the European Social Survey. They looked at two classic mode effects: social desirability and satisficing:
- They found just 2 of 27 questions were significantly different on sensitive questions. She wants to make something of this but it seems to me that it is at the border of random.
- They found no differences in satisficing.
The lack of real findings she traces back to the fact that there were few differences between landline and mobile on location or amount of multitasking. So those factors may or may not make a difference, but it probably makes sense to try to collect those data and look for differences, just in case.
Now we are hearing from Tanja Pferdekamper and Michael Bosnjak from Globalpark and Free University of Bozen-Balzono . They are going to talk about usability of self-administered surveys on mobiles. They seem to have three parts to the research: mobile survey, mixed mode with user choosing mobile or online, and a follow-up with questions about usability of the mobile survey.
- They offered a range of question types and found that single response seems to work best. Multiple response, text input, matrix questions, and semantic differential all performed slightly worse but not by a lot.
- Any time you force the R to scroll, it’s a real problem.
- They found that people responded very quickly—something like 40% in the first hour.
- They found that iPhone users had the fewest self-reported usability problems.
- They asked about a follow-up survey and only about 10 percent asked for the follow-on survey on their mobile. The vast majority wanted it on their PCs. But almost 70 percent of iPhone users asked to have it on their iPhone again next time.
The takeaway seems to be what we have seen in other research: simple questions and shorter surveys work best, but people really prefer Web. Except those crazy iPhone users! It’s hard to see this as being especially encouraging of using mobile except for pretty simple surveys.
One more Globalpark presenter: Sven Scherrer. His topic is a first look at online research going mobile. He is set on using his own laptop so they’ve had to call for the A/V troops to get him going. Nothing loses an audience faster. We were on a roll and now everyone is talking about the cocktails in about 25 minutes. In a sort of mini-Tweet up I’ve discovered that the guy sitting next to me (Adam Winter) is monitoring me on Twitter and reading my blog. He is tweeting to #mrc2010. Some world!
At last we are off. He is running through a quick overview of panels. And then their panel management software. I sense sales under the guise of presenting (SUPing?). He is arguing that now is the time to build mobile panels—people are on the move, the technology is ready, the need to reach all audiences, the ability to get “in the moment” feedback, and some more reasons that I don’t find especially compelling. Now onto the “first look” part which seems to be a demo of their panel management solution and how it looks to an iPhone user. He seems to be using some sort of iPhone simulator on a Macbook but it’s not working very well. In fact, it’s not working at all. Now the audience is trying to coach him. He’s in. I’m gone.