Most studies into the effects of social media use and screen time are badly flawed. This is because researchers use surveys to find out people鈥檚 technology use, but how much we think we spend glued to our devices and how much we actually use them are almost completely unrelated.
Researchers have used self-reported surveys to justify claims like increasing smartphone use is causing teenage suicide聽and a drop in sexual activity. But now a study from David Ellis at Lancaster University and his colleagues is casting doubt on these findings.
The team looked at 10 widely used surveys for measuring screen time use, such as the 鈥淢obile Phone Problem Use Scale鈥� and the 鈥淪martphone Addiction Scale鈥�. These surveys ask users how often they use their phones or how problematic they consider their use.
Advertisement
They then compared 280 people鈥檚 responses to those surveys to objective data from an app, Apple Screen Time, which measures how much people use their phones: how many minutes, how often they pick it up, and how many notifications they received. It found that the objective data and the self-reported data were only very tangentially related.
On one statistical measure of how well the self-reported data and the objective data correlated, the surveys scores ranged from weak to modest. The best test scored 0.4 and the worst scored 0.13, where 1 is complete correlation. Overall, this means that knowing how much someone thinks they use their phone tells you very little about how much they actually do.
Not completely useless
This doesn鈥檛 mean that self-reported data 鈥� which almost all smartphone research is based on 鈥� is completely useless, says Ellis. But it does mean that grand, sweeping claims about the effects our phones and social media are having on mental health and behaviour are probably not grounded in good evidence, he says.
鈥淚t鈥檚 good that someone鈥檚 checking this,鈥� says Peter Etchells, at the University of Bath, who was not involved in the research. Self-reporting is also of limited accuracy in other areas, such as measuring diet and exercise, but people tend not be very critical of claims that smartphones are affecting metal health because they tie into our pet theories, says Etchells.
Self-reporting can be useful if you鈥檙e doing basic science, says Andrew Przybylski of Oxford, whose research often involves self-reported data. But 鈥渋f you鈥檙e out there saying stuff like 鈥榙on鈥檛 give teens mobile phones until they鈥檙e 15鈥�, or if you鈥檙e the health secretary wanting to use age verification software to limit kids鈥� social media time on the basis of these dodgy estimates, that would be bad.鈥�
Ellis agrees. 鈥淚f you鈥檙e making claims about how it ruins lives, or if you鈥檙e going to change policy or give advice to parents, you need to be sure your measurements are as good as they can be,鈥� he says.
A further question is what these scales are actually measuring if they don鈥檛 align with actually behaviour, says Brittany Davidson of Bath University, another author of the study. She and Ellis think people鈥檚 anxiety over their phone use could be a better predictor of their self-reported use, and hope to do another study to look at it.
Reference: PsyArXiv,
