żěè¶ĚĘÓƵ

Medical AIs are advancing – when will they be in a clinic near you?

Trained on real electronic health records, medical AIs are making rapid progress. How long before we see these tools widely used in the clinic, wonders Alex Wilkins

Robot doctor analyzing heart work with tablet, 3d rendering; Shutterstock ID 1466384780; purchase_order: -; job: -; client: -; other: -

HOW would you feel if your doctor, rather than consult their own clinical knowledge, turned instead to an AI trained on your medical history to help diagnose your next ailment or write your next prescription?

These sorts of scenarios have been hypothetical for decades – the technology has been subpar and the stakes too high to risk offloading medical advice to a machine. However, the success of large language models like ChatGPT, a popular, artificially intelligent chatbot from the OpenAI research lab, has led to a rethink of what might be possible.

In December, I was reading through a list of machine learning preprints – scientific papers that have yet to undergo peer review – when I came across , a medical machine learning model from researchers at King’s College London (KCL). It uses GPT-3, the model that powers ChatGPT, and real electronic health records (EHRs) to predict “future events, estimate risk, suggest alternative diagnoses or forecast complications” for simulated or real people whose information is fed into it. While intriguing, I would have paid less attention if it wasn’t for a personal connection.

King’s is my alma mater and the model had been trained on real-world, anonymised data from King’s College Hospital from 2010 to 2019. I had visited its emergency departments at least twice during that period while studying. I soon found Foresight wasn’t the only medical AI. At the end of 2022, Google announced , a version of its enormous general purpose AI model PaLM, which is trained on text from the web and books and fine-tuned using medical documents.

Some of Google’s claims were remarkable: its AI could answer common medical questions requiring a long written response, and real doctors said 92.6 per cent of Med-PaLM’s replies “aligned with scientific consensus”, just 0.3 per cent less than answers given by human doctors. While the medics assessing its abilities noted that gaps in some answers and possible safety issues meant the model wasn’t yet suitable for clinical use, it was clear such AIs were making rapid progress.

It looks like medical AI models might achieve a clinical level of competency before regulatory bodies catch up. All this left me wondering how close we are to seeing these tools in clinics and whether my own medical data helped train an AI (and if it was truly anonymised). I spoke with two of Foresight’s creators to get some clarity. Yes, if I had visited a King’s College Hospital A&E during that period, then my data would have been used to train the model, at KCL told me. And it was almost certainly anonymised, he said.

The EHRs had had any potentially identifying information removed, like rare diseases where there were less than 100 samples, and you couldn’t ultimately get patient-level data out of the AI system. “The risk of re-identification within the model is effectively zero,” says Foresight team member James Teo. I was reassured, but still a little spooked at the thought of a digital twin within the model.

Those behind Foresight, as with Med-PaLM, have (for now) opted not to use the AI in a clinical setting and so it won’t be assisting medics yet. But it, too, is producing encouraging : five doctors assessed its predictions for future health issues for 34 simulated patients and found its top forecasted condition was relevant 97 per cent of the time.

Teo couldn’t give me an exact date on when Foresight might be ready for real-world use and said that they needed another year or so to collect data on the model’s accuracy and explainability. But it looks like the models might achieve a clinical level of competency before bodies like the UK’s Medicines and Healthcare Products Regulatory Agency (MHRA) are in a position to properly assess them, he added. “To a certain degree, [Foresight] reaches technical feasibility,” says Teo. “Whether it meets regulatory feasibility still requires the regulators to develop maturity in their frameworks.”

We don’t know what those frameworks might be, but they are likely to look at whether the AIs can reliably produce accurate answers and the transparency of their decision-making. AIs will also need to show they aren’t biased towards certain groups of people, a risk for machine learning models because of the way they are trained. That is especially pertinent in healthcare, where demographics can affect medical outcomes, says at The Alan Turing Institute, UK.

Another pitfall could be so-called AI hallucinations, where the system erroneously generates “fantastical, unfaithful, or nonsensical outputs”, says Leslie.

All these will be questions for the MHRA when the time comes, but, if the preliminary results from these models are improved upon, it will be a matter of when, not if, your next diagnosis is AI assisted.

Alex Wilkins is a żěè¶ĚĘÓƵĚýreporter who coversĚýartificial intelligence, physicsĚýand space. “Artificially intelligent” is a column that cuts through the hype, looks at what AI is really capable of and what it meansĚýfor us. You can follow him @AlexWilkins22

Alex’s week

What I’m reading

Barbarians At the Gate by Bryan Burrough and John Helyar, an exhilarating ride through the world of 1980s greed and excess.

What I’m watching

The Traitors on BBC, which lives up to the hype.

What I’m working on

As a very limited programmer, I’m seeing if ChatGPT can help me get my personal website into tip-top shape.

Ěý

Topics: AI / Health