A small but growing number of people in the US are receiving messages from their doctors drafted with the help of artificial intelligence – and some may not even know it.
It is the first step in a larger plan to use OpenAI’s large language models – the line of technology powering chatbots such as ChatGPT – within one of the largest US electronic health records systems operated by the company Epic. Although Epic and healthcare organisations participating in the pilot deployment describe this as a low-risk use of large language models in healthcare, some AI researchers warn of challenges with the technology’s biases and tendencies to make up false information.
“We see this amazing technical achievement from large language models… but I’m surprised that there aren’t more concerns about some of the specific use cases,” says at the Massachusetts Institute of Technology.
Advertisement
More than two dozen physicians, nurses and other clinicians at three major healthcare organisations are among the first to test whether Epic’s AI-generated message drafts can speed up responses to patients while providing breathing room for healthcare teams. Research has found that clinicians can experience when messages fill their inboxes within an electronic health records system, and many answer such messages outside official work hours.
“Even if a [healthcare] provider can just get 30 seconds or 60 seconds back with a thought process, that adds up throughout the day,” says at UW Health in Wisconsin, one of the organisations that has already begun pilot testing.
Such AI-written drafts can be used as is, edited by the clinician or discarded entirely. But a human is always responsible for manually clicking to send the message – a one-time alert within Epic’s system reminds the doctor to review and update the AI-generated draft reply before sending it out.
The AI-generated message feature is powered by OpenAI’s GPT-3 large language model. Each time it drafts a response, it uses information from the initial patient message and some of the patient’s medical history in the electronic health records system. Microsoft, which hosts GPT-3 in its data centres, said that it complies with US laws protecting and that any prompts or requests are never used to train or improve the core models owned by Microsoft or OpenAI such as GPT-3.
“If [AI] can draft these personalised, data-driven, well-written, empathetic responses to patients, then that should be able to decrease the mental effort and the time that it takes the caregiver to address the messages,” says at Stanford Health Care in California, another participating organisation that plans to begin testing in May.
But not all patients who receive AI-generated messages through Epic’s system get alerted to how they were composed. Stanford Health Care and UW Health say they don’t have a system for notifying patients who receive AI-generated messages during the pilot deployments, although they have discussed that possibility.
The third participating organisation, UC San Diego Health in California, adds a disclaimer stating that part of the message was generated by AI in a secure environment and reviewed by the human sender. “We’re in healthcare and we want to generate trust with our patients,” says at UC San Diego Health. “And so we’re really erring on the side of disclosure.”
Healthcare organisations should make it standard practice to tell patients up front, even if social expectations around such technology may change in the future, says at the University of Michigan. “This is a little bit of a social experiment,” says Singh.
Ghassemi also expressed concern about whether “overworked and understaffed” clinician teams will always carefully check AI-drafted messages for the factual errors that large language models are known to generate. She urged companies to address such issues before any real-world deployments, and pointed to previous AI tools deployed by Epic that in predicting the onset of sepsis in hospitalised patients or in predicting which patients may fail to show up for appointments.
Epic hasn’t publicly detailed how it evaluates AI performance. But the company said it will continue to update the AI-generated message feature with feedback from the participating healthcare organisations.
“Our goal in getting this out in a staged manner is to have organisations be able to study and publish on that accuracy directly within their system rather than through our internal testing and validation, as we think that’ll provide more nuance to the conversation,” says , senior vice president of research and development at Epic.
Epic’s plans for deploying more AI tools may soon face new regulatory requirements. The US government has requiring transparency for AIs deployed in electronic health records systems, says , president-elect of the American Medical Association. “These proposals are under review by the AMA, but appear to be a good first step towards responsible, ethical and transparent deployment of these tools,” he says.
Article amended on 12 May 2023
We clarified that Microsoft hosts GPT-3 in its data centres
