èƵ

Plan to feed phone data of NHS mental health patients to AI mothballed

An AI was designed to predict when people are at risk of having a mental health crisis, based on their health records, but plans to extend the project with mobile phone data seem to have been scrapped
Woman looking at phone
A stock image of a woman looking at her phone
Prostock-studio / Alamy

A controversial AI project that used the health records of thousands of NHS England patients to predict the occurrence of mental health crises was able to do so with 58 per cent accuracy, but a mooted follow-up that intended to increase the accuracy using people’s mobile phone data has been scrapped. Campaigners say it is an example of the risks involved when using people’s data to train algorithms.

As part of the project, more than 5 million pieces of information relating to 17,122 patients at Birmingham and Solihull Mental Health NHS Foundation Trust were pseudonymised whereby patient names were replaced with a unique identifier and handed to Alpha, a division of the Spanish telecoms firm Telefónica, which also owns the O2 phone network in the UK. The company used the data to create an AI model that could predict when someone may be close to a mental health crisis.

The results of that trial, which took place in 2020, have now been published in a paper in Nature Medicine. The algorithm used 167 variables to make predictions about the likelihood that a patient would experience a crisis in the next four weeks. During the six-month pilot scheme, the algorithm made 1011 predictions, 846 of which were assessed by NHS staff for how useful they found the AI. Nurses, doctors and managers reported finding the information provided by the AI to be useful 67 per cent of the time.

at Koa Health a company spun-out from the who was one of the authors of the paper, says that the results were “beyond expectations” and that the instances in which staff reported that the information was of no value were down to cases when patients were already on their radar. He says that the AI model was designed to report those patients most at-risk, so it was inevitable that some would already be under close observation by staff.

“When you provide the list to doctors, and they start on the top, kind of the riskiest patients, they inevitably see patients that are already taken care of. It was a conscious decision.”

Despite the AI model showing some promise, Matic says that plans to combine medical data and phone data never took place, and that the research would have been “tricky” because of the need to anonymise information that is unique. “Plans were put on hold due to covid, but there is currently no intention to revisit them right now,” he says. “Nothing moved with that project.”

Phil Booth at says that simply removing people’s names from a data set doesn’t truly anonymise them, because medical data is so personal that it can easily be linked to your real identity. He suspects that negative press coverage at the time of the trial would have stopped the second phase, which was to have included mobile phone data that could have led to even worse privacy implications.

“It’s utterly farcical, utterly, because the phone company knows who you are, they know whose phone number it is. So if they are able to link the data that they hold with the data that they’ve got, how can that be anonymised?” he says.

at the Electronic Frontier Foundation says that so-called anonymised data can often be analysed to reveal real identities. “Anonymisation is very, very difficult,” he says. “The privacy research community has a phrase, the ‘curse of dimensionality’ that is, the more data points are associated with a person, the harder it is to anonymise that data. If the only information you have about someone is their age, it’s easy to anonymise that data set. But if you have 10 different attributes associated with each person, it’s nearly impossible to anonymise the data set without completely destroying the data’s utility.”

The data set used in the project was collected between September 2012 and November 2018, about patients aged between 16 and 102 years, but none of those were specifically informed about the research or given the chance to opt-out. The UK’s Health Research Authority ruled at the time that consent wasn’t needed, but didn’t answer questions this week from èƵ about the decision or whether the same conditions would be allowed today on similar research.

Louise Hudson at Birmingham and Solihull Mental Health NHS Foundation Trust says that the project has been paused due to “team redeployment” and that an assessment before the original trial suggested that the scheme was compliant with data protection legislation and that all NHS patients have the opportunity to opt-out of their data being used for research at all times. She didn’t respond to questions about how patients would have known the research was happening and how they could have opted out.

Rachel Power at the Patients Association, a UK patient advocacy charity, says she supports sharing data to improve medical care but that oversight is needed. “Our long-held position is that patient data must be used anonymously and patients must have the right to opt out from their data being shared for uses beyond the purposes of their own healthcare,” she says. “Many patients are willing to share their health data for research purposes, but they do want to be able to agree to this and feel confident that their data will be used appropriately and kept secure.”

Nature Medicine

Sign up to our free Health Check newsletter that gives you the health, diet and fitness news you can trust, every Saturday

Topics: AI / Mental health / Nhs