Acute otitis media (ear infection) is one of the most common infections in children and a top indication for antibiotics, but diagnostic accuracy is relatively low, despite an ongoing search for ways to improve clinical skills—everything from training programs to identifying serum biomarkers.
But now, results from a large study of children suggest that an artificial intelligence (AI)-based tool to help interpret tympanic membrane (TM, or eardrum) videos during the clinical exam may boost accuracy and reduce unnecessary antibiotic prescribing. Researchers based at the University of Pittsburgh Medical center published their findings this week in JAMA Pediatrics.
AI tool examines videos from a smartphone app
A research team based at the University of Pittsburgh Medical Center developed a medical grade smartphone application that uses a smartphone camera to capture video otoscope of the tympanic membrane through an endoscope or otoscope. Using the app, the scientists collected a training library of otoscope assessments of children younger than 36 months old who were seen for sick or well visits at two pediatric clinics near Pittsburgh in 2018 and 2019.
Two validated otoscopists reviewed the video and assigned a final diagnosis. They excluded samples when the TM was almost completely occluded by earwax or if the video was out of focus.
Using 1,151 videos from 365 children and the diagnosis information, the researchers developed an AI algorithm to evaluate TM features on the videos and make a diagnosis.
They found that the AI tool has a sensitivity of 93.8% (95% confidence interval [CI], 92.6% to 95.0%) and a specificity of 93.3% (95% CI, 92.5% to 94.1%). The team also administered a questionnaire to parents, who were favorable about the AI tool: 80% wanted the doctor to use the AI tool during future visits. Comments from parent interviews were mostly positive.
Better accuracy, less improper antibiotic prescribing
AI accuracy was better than that of pediatricians, primary care physicians, and advance-practice clinicians, and the authors wrote that the tool could reasonably be used in those settings to help with decisions about treatment.
They said other advantages include use by trained nonphysicians, documentation for the electronic health record, and discussion with parents.
Improved diagnosis can help reduce inappropriate use of antimicrobials for this frequently diagnosed condition.
"Improved diagnosis can help reduce inappropriate use of antimicrobials for this frequently diagnosed condition," the group added.
Tool could transform care, but hurdles remain
In a related commentary in the same issue, Hojjat Salmasian, MD, PhD, MPH, and Lisa Biggs, MD, both with the Children's Hospital of Philadelphia, said that, of 692 AI-enabled medical devices approved by the Food and Drug Administration, only a few apply to pediatrics, and, of those only 2 are designed for ear, nose, and throat exams.
They wrote that the strengths of the study are the large dataset of video recordings and validation obtained with different instruments. They noted a few drawbacks, however, such as how training and testing data were selected and that expert otoscopists were the gold standard, rather than myringotomy and tympanocentesis. (The study's researchers opted not to use the procedures as the reference standard, because they are invasive and not practical for use in a large cohort of children.)
"Nevertheless, the high accuracy of the algorithm, at least in this retrospective analysis, as well as its implementation as a mobile application that could be used in real time, can lead to the hope that diagnosis of otitis media could be transformed using such technology," the two wrote.
The high accuracy of the algorithm ... lead to the hope that diagnosis of otitis media could be transformed using such technology.
Before the AI model for diagnosing otitis media reaches clinical practice, it needs to be studied proactively and compared with clinician performance, Salmasian and Biggs wrote.
Also, fairness and bias need to be studied, as well, they said. For example, an AI model trained on lighter skin tones might make it less accurate for patients who have darker skin tones.
Tool could promote antibiotic stewardship
Aside from improving efficiency in time-limited clinical settings, a more accurate AI-guided tool could also have other benefits.
"The model could promote antibiotic stewardship as well, since the ability to show a parent the visual findings may have significant impact on the parent’s acceptance of a treatment without antibiotics," Salmasian and Biggs wrote.
