Artificial intelligence is comparable to human experts when it comes to making medical diagnoses based on images, according to a study.
The potential of artificial intelligence in healthcare has sparked enthusiasm, with advocates saying it will ease the strain on resources, free up time for doctor-patient interactions, and even help develop tailor-made treatment. Last month, the government announced £ 250million in funding for a new NHS artificial intelligence lab.
However, experts have warned that the latest findings are based on a small number of studies, as the field is littered with low-quality research.
A growing application is the use of AI in the interpretation of medical images – a field that relies on deep learning, a sophisticated form of machine learning in which a series of labeled images are fed into algorithms that select features within them and learn to classify similar images. images. This approach has shown promise in the diagnosis of diseases ranging from cancers to eye conditions.
However, questions remain as to the extent to which these deep learning systems measure up to human skills. Now the researchers say they’ve conducted the first comprehensive review of studies published on the issue, and have found that humans and machines are on an equal footing.
Professor Alastair Denniston, of the University Hospitals Birmingham NHS Foundation Trust and co-author of the study, said the results were encouraging, but the study was a reality check for some of the hype around the IA.
Dr Xiaoxuan Liu, lead author of the study and the same NHS trust, agreed. “There are a lot of headlines about AI outperforming humans, but our message is that it can be matched at best,” she said.
Write in the Lancet Digital Health, Denniston, Liu and their colleagues discussed how they focused on research papers published since 2012 – a pivotal year for deep learning.
An initial search found more than 20,000 relevant studies. However, only 14 studies – all based on human diseases – reported good quality data, tested the deep learning system with images from a separate data set from the one used to train it, and showed the same images to human experts.
The team grouped together the most promising results from each of the 14 studies to reveal that deep learning systems correctly detected a medical condition 87% of the time, compared to 86% for healthcare professionals, and did correctly. given the green light 93% of the time, compared to 91% for human experts.
However, healthcare professionals in these scenarios were not given additional information about the patients they would have in the real world that could guide their diagnosis.
Professor David Spiegelhalter, chairman of the Winton Center for Risk and Evidence Communication at Cambridge University, said the field was inundated with poor research.
“This excellent review demonstrates that the massive hype about AI in medicine obscures the dismal quality of almost any evaluation study,” he said. “Deep learning can be a powerful and awe-inspiring technique, but clinicians and commissioners should ask the crucial question: what does this actually add to clinical practice? “
However, Denniston remained optimistic about the potential of AI in healthcare, saying such deep learning systems could serve as a diagnostic tool and help tackle the backlog of analyzes and research. images. Plus, Liu said, they might come in handy in places that lack experts to interpret the footage.
Liu said it would be important to use deep learning systems in clinical trials to assess whether patient outcomes have improved compared to current practices.
Dr Raj Jena, an oncologist at Addenbrooke’s Hospital in Cambridge who was not involved in the study, said deep learning systems would be important in the future, but stressed that they needed to robust testing in the real world. He also said it was important to understand why such systems sometimes go badly.
“If you are a deep learning algorithm, when you fail you can often fail in very unpredictable and dramatic ways,” he said.