Today’s update from Research-live.com has this headline: Online trackers not
optimised for mobile could 'compromise data quality.' It goes on to
explain:
GMI, which manages more than 1,000 tracking studies, claims
that online trackers that haven’t been optimised for mobile platforms may
exclude this growing audience, which could lead to a drop in data quality,
reduced feasibility and the possibility of missing whole sections of the
required population from research.
Let me be clear. I don’t disagree that online surveys need
to be optimized for mobile and that the numbers of unintentional mobile
respondents (aka UMRs) is large and growing. But a warning from an online panel
company that scaring away UMRs may be leading to a drop in data quality because
of “the possibility of missing whole sections of the required population from
research” just drips with irony.
Let’s start with the fact that online research, at least in
the US, by definition is excluding the roughly 20% of the population that is
not online. Research using an online panel of, say, two million active members
is excluding about 99% of the adult population. As the industry has moved more
and more to dynamic sourcing it’s hard now to know how big the pool of
prospective online respondents is, but it’s a safe bet that that the vast
majority of US adults are missing, and not at random.
Surely, if we have figured out a way to deal with the
massive coverage error inherent in the online panel model, we can handle the
mobile problem.
I suspect that the real issue here is feasibility, not data
quality. Just as the now near-universal use of routers is about inventory
management rather than improved representativeness. I wish that online panel
companies would spend more time trying to deal with real data quality issues
like poor coverage and inadequate sampling methods, but that’s only going to
happen if their customers start demanding it.