Deep Machine Learning for Identification of Diabetic Retinopathy in Retinal Fundus Photographs

Deep machine learning used to evaluate retinal fundus images for evidence of diabetic retinopathy can successfully identify these conditions with high specificity and high sensitivity, according to research published in JAMA.¹

Lily Peng, MD, PhD, a researcher at Google Inc. in Mountain View, California, and colleagues used applied deep learning techniques to create an algorithm to automatically detect both diabetic retinopathy and diabetic macular edema in retinal fundus photographs.

Automated detection has a number of potential benefits, according to background information in the study,¹ including increasing the efficiency and coverage of screening programs, reducing barriers to access, and ultimately improving patient outcomes through early treatment and detection.

Using a data set of 128,175 images, 54 licensed ophthalmologists and ophthalmology senior residents from across the United States graded each image (mean images graded: 9774; median: 2021) 3 to 7 times for referable diabetic retinopathy, diabetic macular edema, and image gradability; images were graded between May and December 2015. The resulting algorithm, graded by at least 7 US board-certified ophthalmologists, was validated using the EyePACS-1 and Messidor-2 data sets.

The first data set—EyePACS-1—included 9963 images from 4497 patients, 7.8% of which were fully gradable. The second data set—Messidor-2—included 1748 images from 874 patients, 14.6% of which were fully gradable.¹ When detecting referable diabetic retinopathy, the algorithm had an area under the receiver operating curve of 0.991 (95% confidence interval [CI], 0.988-0.993) and 0.99 (95% CI, 0.986-0.995) for EyePACS-1 and Messidor-2, respectively.¹ Researchers found that the algorithm generally achieved both high sensitivity and high specificity: 90.3% sensitivity (95% CI, 87.5%-92.7%) and 98.1% specificity (95% CI, 97.8%-98.5%) for EyePACS-1 and 87% sensitivity (95% CI, 81.1%-90%) and 98.5% specificity (95% CI, 97.7%-99.1%) for Messidor-2.¹

“These results demonstrate that deep neural networks can be trained, using large data sets and without having to specify lesion-based features, to identify diabetic retinopathy or diabetic macular edema in retinal fundus images with high sensitivity and high specificity,” wrote Dr Peng and colleagues. The researchers went on to note the additional advantages of an automated identification system, including “consistency of interpretation … and near instantaneous reporting of results.”

“Further research is necessary to determine the feasibility of applying this algorithm in the clinical setting and to determine whether use of the algorithm could lead to improved care and outcomes compared with current ophthalmologic assessment,” they concluded.

In an accompanying JAMA editorial,² Tien Yin Wong, MD, PhD, of the Singapore Eye Research Institute and Duke-NUS Medical School at the National University of Singapore, and  Neil M. Bressler, MD, of the Wilmer Eye Institute at Johns Hopkins University in Baltimore and the editor of JAMA Ophthalmology, commented on the research conducted by Dr Peng and colleagues They noted several critical challenges, including a lack of focus on sight-threatening diabetic retinopathy, the software’s inability to identify conditions such as age-related macular degeneration or glaucoma, and the need for the algorithm software to be “validated further in larger patient cohorts under different settings and conditions.”

Ultimately, though, Drs Wong and Bressler commended the study.

“The study … truly represents the brave new world in medicine. Rather than simply a device that monitors various physiological characteristics, deep machine learning provides a thoughtful analysis of data. The push of artificial intelligence into the health care arena is timely, welcome, and much needed.”

Study Limitations

• The reference standard was the majority decision of ophthalmologist graders, meaning the algorithm may not perform as well for subtle findings that a majority of ophthalmologists may not identify
• Because the network “learned” the features that were most predictive for the referability, it is possible that features were used that were previously unknown to or ignored by humans.
• The algorithm was trained only to address diabetic retinopathy and diabetic macular edema, and may ignore nondiabetic retinopathy lesions.

Disclosures: Drs Peng, Gulshan, Coram, Stumpe, and Narayanaswamy, Mr Wu, and Mr Nelson all report a patent pending on processing fundus images using machine learning models. Dr Cuadros reports receiving grants from Google Inc and the California Health Care Foundation for the preparation of data analysis. This study was sponsored by Google Inc. Dr Wong reports a patent on automated diabetic retinopathy screening software and has received consulting fees and is an advisory board member at Abbot, Novartis, Pfizer, Allergan, and Bayer. Dr Bressler reports a patent on a system and method for automated detection of age-related macular degeneration and other retinal abnormalities.

References

1. Glushan V, Peng L, Coram M, et al. Development and validation of a deep learning algorithm for detection of diabetic retinopathy in retinal fundus photographs. JAMA. 2016 Nov 29. doi:10.1001/jama.2016.17216 [ePub ahead of print].
2. Wong TY, Bressler NM. Artificial intelligence with deep learning technology looks into diabetic retinopathy screening. JAMA. 2016 Nov 29. doi:10.1001/jama.2016.17563 [ePub ahead of print].