I’m here at the CASRO Panels Conference and have just heard an interesting paper by Jeff Miller from Burke and Andrew Jeavons from Nebu. They were inspired by the work of Bateson, Nettle, and Roberts (2006) showing that posting sets of eyes in a university lounge increased the generosity of faculty members depositing money in a box after helping themselves to a cup of coffee. Miller and Jeavons designed an online experiment to see if including a pair of eyes in the logo on the upper left of the screen mediated undesirable panelist behaviors (e.g., straightlining, overly fast completion, etc.).
Their results were interesting. They managed to produce effects in the expected/hoped for direction although they tended to be minimal and the absence of significance testing makes it hard to draw convincing conclusions. They also looked at the flip side, that is, an increase in social desirability reporting. They claimed there was none but the graphic was hard to read.
I had two reactions:
- Banner blindness may have muted any effect. It would be interesting to see if the results change when the eyes are somehow displayed in the question space.
- It seems counterintuitive that the eyes would create a social presence effect on the one hand (mediating bad behavior) but not also lead to increased social desirability reporting on the other.
In short, this was kind of neat but there seem to be some obvious flaws. It would be interesting to see it replicated with a stronger design and some good old-fashioned significance testing.