Content uploaded by Dawn A Sim
Author content
All content in this area was uploaded by Dawn A Sim on Feb 25, 2015
Content may be subject to copyright.
MICROVASCULAR COMPLICATIONS—RETINOPATHY (JK SUN, SECTION EDITOR)
Automated Retinal Image Analysis for Diabetic
Retinopathy in Telemedicine
Dawn A. Sim &Pearse A. Keane &Adnan Tufail &
Catherine A. Egan &Lloyd Paul Aiello &Paolo S. Silva
#Springer Science+Business Media New York 2015
Abstract There will be an estimated 552 million persons with
diabetes globally by the year 2030. Over half of these individ-
uals will develop diabetic retinopathy, representing a nearly
insurmountable burden for providing diabetes eye care. Tele-
medicine programmes have the capability to distribute quality
eye care to virtually any location and address the lack of ac-
cess to ophthalmic services. In most programmes, there is
currently a heavy reliance on specially trained retinal image
graders, a resource in short supply worldwide. These factors
necessitate an image grading automation process to increase
the speed of retinal image evaluation while maintaining accu-
racy and cost effectiveness. Several automatic retinal image
analysis systems designed for use in telemedicine have recent-
ly become commercially available. Such systems have the
potential to substantially improve the manner by which diabe-
tes eye care is delivered by providing automated real-time
evaluation to expedite diagnosis and referral if required. Fur-
thermore, integration with electronic medical records may
allow a more accurate prognostication for individual patients
and may provide predictive modelling of medical risk factors
based on broad population data.
Keywords Automated retinal image analysis .Te leme dici ne .
Diabetes mellitus .Diabetic retinopathy
Abbreviations
ARIA Automated retinal image analysis
JVN Joslin Vision Network
CADe Computer-aided detection
CADx Computer-aided diagnosis
ETDRS Early treatment diabetic retinopathy study
ARIS Automated Retinal Imaging System
PCA Principal component analysis
DRS Diabetic retinopathy study
Introduction
Eduard Jaeger [1] in 1856 first described the controversial
observations of “yellowish spots and extravasations”in the
macula of a diabetic patient. It would take over 113 years
before the Airlie House Symposium on the Treatment of Di-
abetic Retinopathy established basis for the current photo-
graphic method of quantifying the presence and severity of
diabetic retinopathy. The characteristic lesions of diabetic ret-
inopathy may develop in anyone with diabetes mellitus, and
these lesions are presently estimated to affect nearly half of
those diagnosed with diabetes at any given time [2–5]. The
advances in the medical management of diabetes that begun
with discovery of insulin by Frederick Banting, Charles Best
and colleagues in 1921 [6] have substantially increased patient
survival and life expectancy. Patients with diabetes are now
living longer; however, in doing so, they are at increased risk
This article is part of the Topical Collection on Microvascular
Complications—Retinopathy
D. A. Sim :L. P. Aiello:P. S . Si l v a ( *)
Department of Ophthalmology, Harvard Medical School and
Beetham Eye Institute, Joslin Diabetes Center, One Joslin Place,
Boston, MA 02215, USA
e-mail: PaoloAntonio.Silva@joslin.havard.edu
D. A. Sim :P. A. Keane:A. Tufail :C. A. Egan
NIHR Biomedical Research Centre for Ophthalmology, Moorfields
Eye Hospital NHS Foundation Trust, London, UK
D. A. Sim :P. A. Keane:A. Tufail
Institute of Ophthalmology, University College London,
London, UK
P. S. Silva
Teleophthalmology and Image Reading Center, Philippine Eye
Research Institute, National Institutes of Health, University of the
Philippines, Manila, Philippines
Curr Diab Rep (2015) 15:14
DOI 10.1007/s11892-015-0577-6
for developing diabetes-related microvascular complications,
the most common of which is diabetic retinopathy [4,5].
The current treatments available for diabetic retinopathy are
highly effective in preventing visual loss [7–10]. Early detec-
tion and accurate evaluation of diabetic retinopathy severity,
coordinated medical care and prompt appropriate treatment
represent an effective paradigm of diabetic eye care. However,
current estimates suggest that access to such care in developed
countries ranges between 60 and 90%, with significantly lower
rates in developing countries [11]. Without access to eye care,
individuals do not benefit from these remarkably effective ther-
apeutic advances achieved over the past century.
Telemedicine programmes addressing diabetic retinopathy
have the potential to distribute quality eye care to virtually
any location and address the lack of access to eye care. In
England and Wales, a systematic population-based diabetic ret-
inopathy screening programme together with documented im-
provements in diabetes medical care may have, for the first time
in at least five decades, resulted in diabetic retinopathy no lon-
ger being the leading cause of blindness among working age
adults [12]. However, with a projected 552 million individuals
with diabetes globally by the year 2030, current telemedicine
programmes will require technical enhancements addressing
image acquisition, automated image analysis algorithms and
predictive biomarkers to successfully manage this epidemic.
This review evaluates the current state-of-the-art in retinal im-
age analysis for diabetic retinopathy telemedicine programmes.
Potential Impact of Automated Retinal Image Analysis
on Diabetic Retinopathy Telemedicine Programmes
With the aid of digital retinal colour photography, telemedi-
cine has allowed for timely and accurate detection of diabetic
retinopathy, especially for populations where eye care deliv-
ery was not previously feasible. Economic analysis of
teleophthalmology programmes for diabetic retinopathy based
on the Joslin Vision Network (JVN) used by three US federal
health care agencies found these telemedicine efforts to be a
less costly and a more effective strategy than conventional
clinic-based ophthalmoscopy for identifying and accurately
determining diabetic retinopathy severity and preventing
cases of severe vision loss [13]. Moreover, the implementation
of the JVN programme in primary care settings over a 5-year
period has resulted in a 50 % increase in diabetic retinopathy
surveillance rate and a proportional 50 % increase in the rate
of laser treatment [14].
Telemedicine programmes for diabetic retinopathy include
the following key clinical components: (1) a remote, reliable,
cost-effective image acquisition system capable of reproduc-
ibly acquiring high quality retinal images; (2) an image read-
ing centre for retinal image analysis to assess diabetic retinop-
athy severity; and (3) a clinical coordinating centre that
communicates the findings to primary care providers and/or
patients and facilitates clinic appointments or therapy as re-
quired. In addition, telemedicine programmes require expen-
sive technical, information technology and administrative
support. Nonetheless, one of the main barriers towards a wider
implementation of teleophthalmology has remained the re-
quirement and costs of trained personnel for image reading
and grading of a large volume of retinal images.
Since its first deployment 10 years ago, the JVN has pro-
vided diabetes care to over 100,000 patients, and the current
database servers contain over 2 million retina images [15].
The Indian Health Service–JVN Teleophthalmology Program
generated retinal images from 1624 patients in 2003 [14]. This
has now grown to 14,804 patients per year in 2013 with an
average grading time of 10 min per patient. A total of at least
2500 man-hours per year will be required to grade diabetic
retinopathy severity from retinal images within this pro-
gramme alone. In the UK (2010 to 2011), 1.9 million people
were evaluated under the National Health Service Diabetic
Eye Screening Program [16]. With a surveillance rate of over
90 %, the UK programme generates retinal images from at
least 1.7 million patients per a year that have to be graded
manually with an estimated equivalent workload of more than
300,000 man-hours per year. Reading retinal images is a high-
ly skilled process, which requires training, continual quality
control, maintenance of a specialised skill set, and heavy reli-
ance on the experience and knowledge of the individual read-
er. Since trained retinal image readers are expensive and in
limited supply worldwide, it has become a necessity to seek
automation processes in ocular telemedicine in order to in-
crease throughput while maintaining cost effectiveness and
accuracy. Importantly, automation would also benefit the pa-
tient by permitting more rapid diagnosis and the potential for
more time devoted to patient education and possibly real-time
communication with the primary care physician.
For the foreseeable future, human readers will be required
for the purpose of quality control, adjudication, and interpreta-
tion of atypical retinal images. However, rather than fully re-
place the human reader, ARIA could improve workflow, dimin-
ish observer fatigue and reduce bias while performing important
initial triage of low risk images. Ideally, the ARIA system
should link a person’s electronic medical record with their ret-
inal images before performing automated grading and diagnosis
at the time of imaging. This will allow relevant individualised
medical information to be considered in the stratification of an
individual’s risk profile and facilitate counselling and treatment
or follow-up planning during the imaging visit.
Various ARIA strategies for assessing diabetic retinopathy
severity have been extensively evaluated [17–20]. In this re-
view, we examine the potential impact of automated image
analysis on ocular telemedicine for diabetic retinopathy, sum-
marise the different ARIA approaches, and evaluate the suit-
ability of the current ARIA programmes for this indication.
14 Page 2 of 9 Curr Diab Rep (2015) 15:14
Approaches to Automated Retinal Image Analysis
Since the first stereoscopic fundus photograph by Jackson and
Weber in 1886, which required a camera fixed onto the pa-
tient’s head and a 2.5-min exposure, the documentation of
anatomy and pathological changes in the human retina has
been a subject of interest to ophthalmologists, artists, and en-
gineers alike. However, it was the introduction of high-
resolution digital retinal imaging systems in the 1990s com-
bined with exceptional growth in computing power that per-
mitted the development of computer algorithms capable of
computer-aided detection (CADe), and computer-aided diag-
nosis (CADx). CADe is the identification of pathologic le-
sions. CADx provides a classification that incorporates addi-
tional lesion or clinical information to stratify risk or estimate
the probability of disease. These recent imaging and comput-
ing advances allowed ARIA, a topic of interest for more than
40 years, to finally realistically address clinical applications
over the past decade.
A key component to the telemedicine approach is the clin-
ical validation of retinal image analyses against the current
gold-standard established by the Early Treatment Diabetic
Retinopathy Study (ETDRS) consisting of 30°,stereoscopic,
seven-standard field, colour 35-mm slides [21]. The American
Telemedicine Associated has published position statements in
order to provide standards and guidelines specifically for dia-
betic retinopathy [22,23] and recommends that automated
algorithms be compared to the accuracy and precision provid-
ed by the ETDRS imaging and evaluation standards. Over that
past decade, there has been a host of algorithms developed for
automated detection of diabetic retinopathy, the details of
which has been reviewed elsewhere [17,18,24]. Broadly,
the approach to ARIA can be categorised into two compo-
nents: (1) image quality assessment and (2) image analysis.
Image Quality Assessment
Only recently have digital cameras approached the resolution
attained by their 35-mm film counterparts. Independent com-
parative validation studies [25,26] and analysis from
multicentre clinical trials [27,28]havereportedgoodtoex-
cellent agreement in comparing diabetic retinopathy severity
on film and digital images, indicating that the use of digital
images at their current resolution do not systematically alter
detection of diabetic retinopathy severity. In general, the
agreement between film and digital images is less precise at
two particular points on the severity scale, first in determining
the presence of mild nonproliferative retinopathy (driven pri-
marily by microaneurysms) and at moderate nonproliferative
diabetic retinopathy (driven primarily by intraretinal micro-
vascular abnormalities) [27]. Although modern electronic sen-
sors have low image noise, they can be susceptible to pattern
artefacts due to the grid arrangement of digital sensors. The
addition of Bayer filters is helpful in reducing noise in the
image.
Currently, retinal digital photography has progressed to a
stage where colour retinal photographs can be obtained using
low levels of illumination through an undilated pupil.[29,30].
However, human factors such as movement and positioning
and ocular factors like cataract and reflections from retinal
tissues can produce artefacts. Without pupillary dilation, arte-
facts are observed in 3–30 % of retinal images to an extent
where they impede human grading [29,31]. Thus, the impor-
tance of good image quality prior to automated image analysis
has been recognised, and much ancillary research has been
conducted in the field of image pre-processing. In general, this
consists at least of comparing the histogram of an image ob-
tained to that of an ideal histogram describing the brightness,
contrast and signal/noise ratio, and/or determination of image
clarity by assessing vessels surrounding the macula [32].
Image acquisition can also be automated, but the rate of
ungradable images needs to be closely evaluated. The Auto-
mated Retinal Imaging System (ARIS
TM
, Visual Pathways,
Inc., Prescott, AZ, USA) [33] and Centervue DRS (Centervue
SpA, Padova, Italy) [34] are imaging systems that may be
programmed to acquire sequential colour digital images of
defined fundus fields using minimally trained technicians.
However, these systems have ungradable rates for individual
images as high as 30 %. The automated image acquisition in
these systems would be helpful in obtaining consistent fields
using minimally trained technician; however, this would need
to be combined with image quality analysis to determine the
need for re-imaging a field. Until much improved automated
acquisition systems are available, image pre-processing will
likely need to be incorporated into automated image acquisi-
tion retinal cameras, allowing real-time assessment of image
quality, providing real-time feedback to the technician and
allowing a patient to be re-imaged at the same visit if the
retinal image does not meet a given standard. Real-time image
quality assessment systems are not yet available commercial-
ly. One such a system is being developed by VisionQuest
Biomedical, LLC, Albuquerque, NM, USA, and has been
validated on 2000 colour retinal images showing 100 % sen-
sitivity and 96 % specificity in identifying “rejected”images
[35].
Image Analysis
An important first step of ARIA for diabetic retinopathy be-
gins with identification and localisation of normal anatomy,
better known as “segmentation”,ofstructuressuchastheoptic
nerve head, fovea and large retinal vessels. The main purposes
of identifying these structures are to exclude them in the anal-
ysis for abnormal lesions and also to determine their locations
for image registration (e.g. alignment with a reference image
or a previously acquired image) and for use as a reference for
Curr Diab Rep (2015) 15:14 Page 3 of 9 14
measuring distances within the image. Localisation of the op-
tic nerve head is a well-established approach. Some examples
of techniques used in this regard include the identification of
high pixel intensity within a retinal image (the optic disc is
usually the brightest “spot”in a retinal image), principal com-
ponent analysis (PCA; a method used for face recognition
software that differentiates the optic disc from other bright
spots such as exudates or reflections in the retina), the Hough
transform (which utilises shape orientations to plot locations)
and geometrical parametric modelling (a technique that ex-
ploits patterns from which retinal vessels emerge from the
optic disc) [19,24].
Automated detection of abnormal diabetic retinopathy le-
sions was initially performed on fluorescein angiograms [36].
Although fluorescein provides a background with high con-
trast between vascular and nonvascular structures and allows
easier identification of vascular pathology, the requirement of
intravenous infusion precluded its wide-scale application. The
development of ARIA for diabetic retinopathy has therefore
largely focussed on the use of digital colour images of the
retina. The main challenge encountered with processing of
colour images is the presence of numerous “distractors”with-
in the retinal image (retinal capillaries, underlying choroidal
vessels, and reflection artefacts),all of which may be confused
with diabetic retinopathy lesions. As a result, much research
has been focussed on the selective identification of diabetic
retinopathy features, including microaneurysms,
heamorrhages, hard or soft exudates, cotton-wool spots and
venous beading. These clinical features have been described
in great detail in the landmark clinical trials, Diabetic Retinop-
athy Study (DRS) and ETDRS. [21,37].
Earlier work in ARIA utilised basic image processing tech-
niques such as thresholding, edge detection, filters and mor-
phological processing [24]. These techniques focussed on
identifying individual diabetic retinopathy features mimicking
what is typically performed with manual human grading of
individual lesions. Techniques on the extraction of
microaneurysms from colour fundus images by the process
of thresholding after excluding normal anatomical structures
have been extensively reported with varying levels of success
[18]. Recent developments in ARIA have moved away from
individual detectors to approaches that include the following:
an ensemble-based approach (integrates components of
microaneurysm detectors) [38], a multiple-lesion approach
(fuses multiple classifiers) [39], intelligent systems (which
require training algorithms with data derived from human op-
erators) [40] and content-based image retrieval techniques
(analyses the contents of large digital image databases and
compares it with an archive of labelled images) [41]. The
details of algorithms and mathematical modelling methods
used and their comparative efficiencies have been published
in several reviews [17–19,24]. These emerging trends of
ARIA may prove most relevant to telemedicine given their
potential to go beyond traditional retinal image analysis, pos-
sibly providing more detailed assessment of retinopathy se-
verity and greater accuracy of progression risk. Addition of
electronicmedical record integration may not only allow more
accurate prognostication for individual patients but also pro-
vide predictive modelling of risk factors based on broad pop-
ulation data.
Overview of ARIA Systems Currently Deployed
in Telemedicine and Screening Programmes
Current commercially available systems that have been used
in telemedicine or screening programmes include the
iGradingM (Medalytix Group Ltd), the TRIAD
TM
Network
(Hubble Telemedical, Inc.), Iowa Detection Program (IDx,
LLC), RetmarkerDR (Retmarker Ltd, Coimbra, Portugal)
and Retinalyze System (Retinalyze A/S, Hørsholm, Denmark)
(Table 1). A comparison of published sensitivity and specific-
ity of each programme as well as a brief description of the
associated published studies are presented in Table 2.Direct
head-to-head comparisons between systems have proven dif-
ficult, mainly because of different photographic protocols, al-
gorithms and patient populations used for validation. A com-
mon thread amongst these automated systems is to identify
referable retinopathy: diabetic retinopathy, which requires the
attention of an ophthalmologist. To date, we have not
progressed to where human intervention can be fully removed
from such programmes. All of the systems described below
are semi-automated at some point in the workflow pathway
and require assistance of a human reader/grader.
iGradingM
iGradingM is a product of Medalytix Group Ltd, which received
its class 1 Conformité Européenne (CE) mark in 2013 and per-
forms “disease/no disease”grading for diabetic retinopathy [42,
43]. It was developed at the University of Aberdeen in Scotland
and uses previously published algorithms to assess both image
quality and disease and has been described in detail elsewhere
[32,44]. A previously trained automated classifier on a set of 35
images containing 198 individually annotated microaneurysm or
dot haemorrhages was used in its development.
It was first deployed on a large-scale population in the
Scottish Diabetic Retinopathy Screening Program in 2010,
after being validated using several large screening populations
in Scotland [39,43,45•,46,47], the largest being 78,601
single-field 45° colour fundus images from 33,535 consecu-
tive patients [45•]. In this retrospective study, 6.6 % of the
cohort had referable retinopathy, and iGradingM attained a
sensitivity of 97.8 % for referable retinopathy. Although the
specificity for referable retinopathy was not reported in the
paper, it has since been calculated at 41.1 % [20]. iGradingM
14 Page 4 of 9 Curr Diab Rep (2015) 15:14
has been further validated on 8271 screening episodes from a
South London population in the UK [48]. Here, there was a
higher percentage (7.1 %) of referable disease, and a sensitiv-
ity of 97.4–99.1 % and specificity of 98.3–99.3 % were
attained depending on the configuration used.
The TRIAD
™
Network
The TRIAD
™
Network (Telemedical Retinal Image Analysis
and Diagnosis) was launched commercially in 2012 by its devel-
opers at Hubble Telemedical Inc., and was developed at the
University of Tennessee Health Science Center and the Oak
Ridge National Laboratory [49]. It consists of a web-based tech-
nical network infrastructure that has been functional as a telemed-
icine network for diabetic retinopathy screening in the mid-south
region of the USA since 2009 [50,51]. ARIA was introduced to
the TRIAD
™
Network over a period of time, beginning with
image quality analysis at the point of image capture [52]and
most recently with the use of their patented content-based image
retrieval techniques for automated diagnosis [53•]. The technique
of content-based image retrieval involves the comparison of im-
ages to large database collections using pictorial content. These
image features include various description models, perceptual
organisation, spatial relationships and may including clinical
metadata [53•].
The authors themselves have reviewed challenges that they
have faced with validation of their ARIA software, due mostly to
the lack of large, varied and publically available datasets [50].
Briefly, they note that TRIAD
TM
Network infrastructure has cre-
ated a useful internal validation mechanism by which new algo-
rithms can be developed and tested on datasets from different
time points [53•,54,55]. Although external validation has been
performed for anomaly detection using publically available
datasets, these only contain a small number of images (<100),
and no large-scale validation has been published to date [50].
IDx-DR
The IDx-DR was the first commercial product of IDx LLC,
Iowa, USA, that has in 2013 received CE approval as a Class
Tabl e 2 Efficiency of current ARIA systems for detecting referable retinopathy
ARIA system iGradingM [45]IDx-DR[65] RetmarkerDR [67]Retinalyze[76]
Sensitivity 0.98 (0.97–0.99) 0.97 (0.94–0.99) 0.96 (0.94–0.98) 0.97 (not provided)
Specificity 0.41 (not provided) 0.59 (55.7–63.0) 0.52 (0.50–0.53) 0.71 (not provided)
Data presented is sensitivity/specificity (95 % confidence interval) when available. Each of diabetic retinopathy grading programmes listed abovehave
been evaluated using different image set and grading methods and generally do not represent comparative performance. The difference in the perfor-
mance measures listed above may be due to difference in the image set, grading methods, algorithm threshold or a combination of various factors.
Referable retinopathy is defined as retinopathy more severe than mild nonproliferative diabetic retinopathy
Tabl e 1 Overview of current ARIA systems developed for diabetic retinopathy in telemedicine
ARIA system Company Developed at Grading details Current deployment
iGradingM Medalytix LLC; Digital
Healthcare
University of Aberdeen,
Scotland, UK
Diabetic retinopathy absent/
diabetic retinopathy present
CE mark 2013 as a class 1
medical device; Scottish
Diabetic Retinopathy
Screening Program
The TRIAD
TM
Network
Hubble Telemedicine Inc. University of Tennessee
Health Science Center
and the Oak Ridge National
Laboratory, USA
CBIR technology not
approved for use; diagnosis
diabetic retinopathy severity
with a supervising retinal
specialist
Primary care services,
mid-south region of
the USA
IDx-DR IDx LLC University of Iowa, USA Diabetic retinopathy index;
referable retinopathy (more
than mild nonproliferative
diabetic retinopathy)
CE mark 2013 as a class IIa
medical device
RetmarkerDR Retmarker Ltd. Coimbra University,
Portugal
Diabetic retinopathy absent/
diabetic retinopathy present;
microaneurysm turnover
CE mark 2011 as a Class IIa
medical device; diabetic
retinopathy screening
programmes in Portugal
RetinaLyze A/S RetinaLyze A/S Denmark Microaneurysm and
haemorrhage detection
CE mark 2011 as a class
IIa medical device;
software re-launched
commercially 2013
Curr Diab Rep (2015) 15:14 Page 5 of 9 14
lla medical device for sale in the European Economic Area
[56]. It utilises the Iowa Detection Program, which consists of
previously published algorithms including features such as
image quality assessment, detection of microaneurysms,
haemorrhage, cotton wool spots, and a fusion algorithm that
combines these analyses to produce the diabetic retinopathy
index [57–61]. This index represents a dimensionless number
from 0 to 1 and represents the likelihood that the image con-
tains referable disease.
The IDx-DR has been validated in several large screening
populations [62–64]. Most recently, the IDx-DR was validat-
ed on 1748 eyes with single-field 45° colour fundus images
acquired in French primary care clinics [65•]. In this study, the
proportion of referable diabetic retinopathy was high at
21.7 % and sensitivity and specificity reported at 96.8 and
59.4 %, respectively.
RetmarkerDR
RetmarkerDR was developed at Coimbra University, Portu-
gal, and has been used in screening programmes in Portugal
since 2011 [66]. It was launched commercially in 2011 by
“Critical Health”, which has since been renamed Retmarker
Ltd and attained CE approval as a Class lla medical device in
April of 2010. The main features of RetmarkerDR are its
image quality assessment algorithm, which has been validated
on publically available datasets [67], a co-registration algo-
rithm, which allows comparisons of the same location in the
retina to be made between visits [68,69•].
The strength of the RetmarkerDR programme lies not in eval-
uating the severity of diabetic retinopathy but rather in predictive
longitudinal analysis using microaneurysm turnover. Two inde-
pendent prospective longitudinal studies using the RetmarkerDR
on single-field images have demonstrated the relationship of in-
creased microaneurysm turnover rates identified by the
RetmarkerDR programme and an increased rate for developing
clinically significant macular oedema [70–72].
Retinalyze
The Retinalyze System A/S (Retinalyze A/S, Hørsholm, Den-
mark) was developed in the late 1990s and combines image
quality assessment, with automated red microaneurysm and
haemorrhage lesion detection [73]. This system has been validat-
ed in two different patient populations in Denmark and Wales,
UK. In 2003, a Danish retrospective study of 260 eyes reported a
sensitivity of 93.1 % and specificity at 71.6 % in detecting reti-
nopathy [74]. A year later, a prospective validation study on 164
eyes from the same population, reported similar efficiencies, with
a sensitivity of 97.0 % and specificity of 75.0 % [75].
In 2003, a study on 200 eyes from the Welsh Community
Diabetic Retinopathy Study reported a sensitivity of 96.7 %
and specificity of 71.4 % [76]. Several years later (2008), a
modified version of the Retinalyze software (version 1.0.6.1),
described to be more sensitive to large “red and bright le-
sions”, was validated on 192 eyes from a similar population,
with a reported sensitivity of 93 % and specificity of 78 %
[77•]. However, by then, Retinalyze was no longer commer-
cially available. More recently (2013), Retinalyze has been
reintroduced to the commercial market with updates to the
platform in keeping with advancements in computing plat-
forms. Retinalyze is currently available via the “Cloud”,there-
by allowing automated analysis via an internet connection
within 20 s of image upload.
Future Development of Retinal Image Analysis Approaches
The development of the ARIA algorithms typically requires a
“training”set of images where the computer programme is
“taught”using human-labelled sample images. The accompa-
nying limitations can be grouped into two categories. First is
the requirement of a large and varied dataset of training im-
ages. Due to regulatory and proprietary barriers, these datasets
are not readily available to researchers developing these algo-
rithms. However, two publicly accessible image data sets are
available, namely, the Methods for Evaluating Segmentation
and Indexing techniques Dedicated to Retinal Ophthalmology
(MESSIDOR) database, which is funded by the French Min-
istry of Research and Defense contains 1200 retinal images
with diagnosed level of diabetic retinopathy and macular oe-
dema [78], while the Retinopathy Online Challenge (ROC) by
the University of Iowa [79] contains 50 unique retinal images
that have been annotated for haemorrhages and/or
microaneurysms by a weighted consensus from four ophthal-
mologists. Second is the need for human-labelled images,
which can be labour intensive and costly. An intriguing ap-
proach to this problem utilises a Mechanical Turk approach
via a crowdsourcing internet market place. Proof-of-concept
studies suggest potential applicability for acquiring large
datasets of labelled images [80,81]. However, current studies
that have evaluated this approach have done so in a research
environment, and it should be emphasised that there are sig-
nificant limitations to this approach particularly when it is
used in an active clinical programme. A background of an
anonymous crowdsourced workforce is largely unknown,
and varying levels of potential bias may be introduced as
methods of standardisation would be limited. Furthermore,
ethical and privacy concerns remain a significant hurdle that
would prevent the release and online access of anonymised
clinical data. The data demonstrating effectiveness of using a
crowdsourced workforce is early and limited, but it provides
further evidence that the presence or absence of retinopathy or
referrable retinopathy can accurately be identified by nonphy-
sician graders provided that a standardised protocol is closely
followed. Should this approach be pursued in the future, rig-
orous quality control and quality assurance metrics need to be
14 Page 6 of 9 Curr Diab Rep (2015) 15:14
implemented but the actual grading of images may be pursued
in regions of the world wherein the cost efficiency of image
evaluation may be substantially reduced.
Conclusions
The application of ARIA to the infrastructure of telemedicine has
for the first time provided a strategy by which we may eventually
be able to address the enormous requirements of retinal exami-
nations needed in order to eliminate visual loss as a result of
diabetes. Like the extraordinary achievement of the WHO Small-
pox Eradication Program (1966–1980), this will require the col-
laboration of national health authorities, commercial companies
and the medical profession as a whole. The combination of
ARIA with telemedicine has the potential to substantially im-
prove the manner by which diabetes eye care is delivered, in
theory providing automated real-time patient evaluation, predic-
tive patient and population analyses and possible identification of
previously unrecognised novel markers of disease risk.
A major challenge to fully realising the potential of this tech-
nology remains the lack of a uniform validation for these sys-
tems. Current ARIA systems identify the presence of retinopathy
or the presence of referable retinopathy but do not identify the
specific levels of ETDRS severity from retinal images. A recent
collaborative effort between several international research groups
have published a list of recommendations for the validation of
ARIA algorithms and highlighted several key points, including
the need to focus on creating large repositories of real-life
datasets from international consortia, a multicentre approach to
create expert-annotated image datasets and an agreed-on perfor-
mance criteria to which different ARIA systems can be assessed
[82••]. Furthermore, critical to large-scale implementation of
ARIA systems for diabetic retinopathy is the involvement of
primary care providers on the frontlines of treating diabetes. Such
involvement will be important to promote management of blood
pressure, serum glucose, and lipid levels, all of which impact
diabetic retinopathy progression, while providing primary care
practitioners a simple action-oriented process to allow easy de-
tection of referable ocular disease and prompt access to appro-
priate eye care services.
Acknowledgments The research was supported by a grant from Fight
for Sight, UK (Grant number 1987), The Special Trustees of Moorfields
Eye Hospital and the National Institute for Health Research (NIHR) Bio-
medical Research Centre based at Moorfields Eye Hospital NHS Foun-
dation Trust and UCL Institute of Ophthalmology.
Compliance with Ethics Guidelines
Conflict of Interest Dawn A. Sim, Pearse A. Keane, Adnan Tufail,
Lloyd Paul Aiello and Paolo S. Silva declare that they have no conflict
of interest.
Catherine A. Egan reports grants and personal fees from Novartis. Her
husband is a retinal specialist and receives honoraria, grants and speakers
fees from academic and pharmaceutical industry relevant to this work.
Human and Animal Rights and Informed Consent This article does
not contain any studies with human or animal subjects performed by any
of the authors.
References
Papers of particular interest, published recently, have been
highlighted as:
•Of importance
•• Of major importance
1. Jaeger E. Beitr zur Pathol des Auges. Wien p. 1856;33 Fig 12.
2. Kempen JH, O'Colmain BJ, Leske MC,Haffner SM, Klein R, Moss
SE, et al. The prevalence of diabetic retinopathy among adults in the
United States. Arch Ophthalmol. 2004;122:552–63.
3. Roy MS, Klein R, O'Colmain BJ, Klein BE, Moss SE, Kempen JH.
The prevalence of diabetic retinopathy among adult type 1 diabetic
persons in the United States. Arch Ophthalmol. 2004;122:546–51.
4. Klein R, Klein BE, Moss SE, Davis MD, DeMets DL. The
Wisconsin epidemiologic study of diabetic retinopathy. III.
Prevalence and risk of diabetic retinopathy when age at diagnosis
is 30 or more years. Arch Ophthalmol. 1984;102:527–32.
5. Klein R, Klein BE, Moss SE, Davis MD, DeMets DL. The
Wisconsin epidemiologic study of diabetic retinopathy. II.
Prevalence and risk of diabetic retinopathy when age at diagnosis
is less than 30 years. Arch Ophthalmol. 1984;102:520–6.
6. Banting FG, Best CH, Collip JB, Campbell WR, Fletcher AA.
Pancreatic extracts in the treatment of diabetes mellitus. Can Med
Assoc J. 1922;12:141–6.
7. Diabetic Retinopathy Clinical Research N, Writing C, Aiello LP,
Beck RW, Bressler NM, Browning DJ, et al. Rationale for the
diabetic retinopathy clinical research network treatment protocol
for center-involved diabetic macular edema. Ophthalmology.
2011;118:e5–14.
8. Brown DM, Nguyen QD, Marcus DM, Boyer DS, Patel S, Feiner
L, et al. Long-term outcomes of ranibizumab therapy for diabetic
macular edema: the 36-month results from two phase III trials:
RISE and RIDE. Ophthalmology. 2013;120:2013–22.
9. Korobelnik JF, Do DV, Schmidt-Erfurth U, Boyer DS, Holz FG,
Heier JS, Midena E, Kaiser PK, Terasaki H, Marcus DM, et al.
Intravitreal aflibercept for diabetic macular edema.
Ophthalmology 2014;121:2247–54.
10. Early photocoagulation for diabetic retinopathy. ETDRS report
number 9. Early Treatment Diabetic Retinopathy Study Research
Group. Ophthalmology. 1991;98:766–785.
11. Hartnett ME, Key IJ, Loyacano NM, Horswell RL, Desalvo KB.
Perceived barriers to diabetic eye care: qualitative study of patients
and physicians. Arch Ophthalmol. 2005;123:387–91.
12. Liew G, Michaelides M, Bunce C. A comparison of the causes of
blindness certifications in England and Wales in working age adults
(16–64 years), 1999–2000 with 2009–2010. BMJ Open. 2014;4:
e004015.
13. Whited JD, Datta SK, Aiello LM, Aiello LP, Cavallerano JD,
Conlin PR, et al. A modeled economic analysis of a digital tele-
ophthalmology system as used by three federal health care agencies
for detecting proliferative diabetic retinopathy. Telemed J E Health.
2005;11:641–51.
Curr Diab Rep (2015) 15:14 Page 7 of 9 14
14. Wilson C, Horton M, Cavallerano J, Aiello LM. Addition of pri-
mary care-based retinal imaging technology to an existing eye care
professional referral program increased the rate of surveillance and
treatment of diabetic retinopathy. Diabetes Care. 2005;28:318–22.
15. Silva PS, Cavallerano JD, Aiello LM, Aiello LP. Telemedicine and
diabetic retinopathy: moving beyond retinal screening. Arch
Ophthalmol. 2011;129:236–42.
16. Program, N.D.E.S. 2010–2011 Annual report. http://diabeticeye.
screening.nhs.uk/reports. Accessed 16 July 2014.
17. Li B, Li HK. Automated analysis of diabetic retinopathy images:
principles, recent developments, and emerging trends. Curr Diab
Rep. 2013;13:453–9.
18. Mookiah MR, Acharya UR, Chua CK, Lim CM, Ng EY, Laude A.
Computer-aided diagnosis of diabetic retinopathy: a review.
Comput Biol Med. 2013;43:2136–55.
19. Teng T, Lefley M, Claremont D. Progress towards automated dia-
betic ocular screening: a review of image analysis and intelligent
systems for diabetic retinopathy. Med Biol Eng Comput. 2002;40:
2–13.
20. Fleming AD, Philip S, Goatman KA, Prescott GJ, Sharp PF, Olson
JA. The evidence for automated grading in diabetic retinopathy
screening. Curr Diabetes Rev. 2011;7:246–52.
21. Grading diabetic retinopathy from stereoscopic color fundus pho-
tographs—an extension of the modified Airlie House classification.
ETDRS report number 10. Early Treatment Diabetic Retinopathy
Study Research Group. Ophthalmology 1991;98:786–806.
22. Silva PS, Cavallerano JD, Aiello LM. Ocular telehealth initiatives
in diabetic retinopathy. Curr Diab Rep. 2009;9:265–71.
23. Li HK, Horton M, Bursell SE, Cavallerano J, Zimmer-Galler I,
Tennant M, et al. Telehealth practice recommendations for diabetic
retinopathy, second edition. Telemed J E Health. 2011;17:814–37.
24. Patton N, Aslam TM, MacGillivray T, Deary IJ, Dhillon B,
Eikelboom RH, et al. Retinal image analysis: concepts, applications
and potential. Prog Retin Eye Res. 2006;25:99–127.
25. Li HK, Esquivel A, Hubbard LD, Florez-Arango JF, Danis RP,
Krupinski EA. Mosaics versus early treatment diabetic retinopathy
seven standard fields for evaluation of diabetic retinopathy severity.
Retina. 2011;31:1553–63.
26. Silva PS, Cavallerano JD, Sun JK, Soliman AZ, Aiello LM, Aiello
LP. Peripheral lesions identified by mydriatic ultrawide field imag-
ing: distribution and potential impact on diabetic retinopathy sever-
ity. Ophthalmology. 2013;120:2587–95.
27. Hubbard LD, Sun W, Cleary PA, Danis RP, Hainsworth DP, Peng
Q, et al. Comparison of digital and film grading of diabetic retinop-
athy severity in the diabetes control and complications trial/
epidemiology of diabetes interventions and complications study.
Arch Ophthalmol. 2011;129:718–26.
28. Gangaputra S, Almukhtar T, Glassman AR, Aiello LP, Bressler N,
Bressler SB, et al. Comparison of film and digital fundus photo-
graphs in eyes of individuals with diabetes mellitus. Invest
Ophthalmol Vis Sci. 2011;52:6168–73.
29. Silva PS, Cavallerano JD, Tolls D, Omar A, Thakore K, Patel B,
et al. Potential efficiency benefits of nonmydriatic ultrawide field
retinal imaging in an ocular telehealth diabetic retinopathy program.
Diabetes Care. 2014;37:50–5.
30. Ahmed J, Ward TP, Bursell SE, Aiello LM, Cavallerano JD,
Vigersky RA. The sensitivity and specificity of nonmydriatic dig-
ital stereoscopic retinal imaging in detecting diabetic retinopathy.
Diabetes Care. 2006;29:2205–9.
31. Scanlon PH, Foy C, Malhotra R, Aldington SJ. The influence of
age, duration of diabetes, cataract, and pupil size on image quality
in digital photographic retinal screening. Diabetes Care. 2005;28:
2448–53.
32. Fleming AD, Philip S, Goatman KA, Olson JA, Sharp PF.
Automated assessment of diabetic retinal image quality based on
clarity and field definition. Invest Ophthalmol Vis Sci. 2006;47:
1120–5.
33. Maker MP, Noble J, Silva PS, Cavallerano JD, Murtha TJ, Sun JK,
et al. Automated Retinal Imaging System (ARIS) compared with
ETDRS protocol color stereoscopic retinal photography to assess
level of diabetic retinopathy. Diabetes Technol Ther. 2012;14:515–
22.
34. Smolek MKN N, Jaramillo A, Diamond JG, Batlle OR. Image
quality and disease screening performance comparison of clinic-
based and telemedicine-based retinal cameras. ARVO/ISIE
Imaging Conference 2012. 2012 Poster Board Number: P55.
35. Davis H, Russell S, Barriga E, Abramoff M, Soliz P. Vision-based,
real-time retinal image quality assessment. In Computer-based
medical systems, 2009. CBMS 2009. 22nd IEEE International
Symposium on 2009. 1–6.
36. Spencer T, Phillips RP, Sharp PF, Forrester JV. Automated detection
and quantification of microaneurysms in fluorescein angiograms.
Graefes Arch Clin Exp Ophthalmol. 1992;230:36–41.
37. Diabetic retinopathy study. Report Number 6. Design, methods,
and baseline results. Report Number 7. A modification of the
Airlie House classification of diabetic retinopathy. Prepared by
the Diabetic Retinopathy. Invest Ophthalmol Vis Sci. 1981;21:1–
226.
38. Antal B, Hajdu A. An ensemble-based system for microaneurysm
detection and diabetic retinopathy grading. IEEE Trans Biomed
Eng. 2012;59:1720–6.
39. Fleming AD, Goatman KA, Philip S, Williams GJ, Prescott GJ,
Scotland GS, et al. The role of haemorrhage and exudate detection
in automated grading of diabetic retinopathy. Br J Ophthalmol.
2010;94:706–11.
40. Quellec G, Lamard M, Abramoff MD, Decenciere E, Lay B,
Erginay A, et al. A multiple-instance learning framework for dia-
betic retinopathy screening. Med Image Anal. 2012;16:1228–40.
41. Quellec G, Lamard M, Cazuguel G,Bekri L, Daccache W, Roux C,
et al. Automated assessment of diabetic retinopathy severity using
content-based image retrieval in multimodal fundus photographs.
Invest Ophthalmol Vis Sci. 2011;52:8342–8.
42. http://www.medalytix.com Accessed 15th July 2014.
43. Philip S, Fleming AD, Goatman KA, Fonseca S, McNamee P,
Scotland GS, et al. The efficacy of automated “disease/no disease”
grading for diabetic retinopathy in a systematic screening pro-
gramme. Br J Ophthalmol. 2007;91:1512–7.
44. Fleming AD, Philip S, Goatman KA, Olson JA, Sharp PF.
Automated microaneurysm detection using local contrast normali-
zation and local vessel detection. IEEE Trans Med Imaging.
2006;25:1223–32.
45.•Fleming AD, Goatman KA, Philip S, Prescott GJ, Sharp PF, Olson
JA. Automated grading for diabetic retinopathy: a large-scale audit
using arbitration by clinical experts. Br J Ophthalmol. 2010;94:
1606–10. Key paper for the iGrading Program.
46. Scotland GS, McNamee P, Philip S, Fleming AD, Goatman KA,
Prescott GJ, et al. Cost-effectiveness of implementing automated
grading within the national screening programme for diabetic reti-
nopathy in Scotland. Br J Ophthalmol. 2007;91:1518–23.
47. Scotland GS, McNamee P, Fleming AD, Goatman KA, Philip S,
Prescott GJ, et al. Costs and consequences of automated algorithms
versus manual grading for the detection of referable diabetic reti-
nopathy. Br J Ophthalmol. 2010;94:712–9.
48. Goatman K, Charnley A, Webster L, Nussey S. Assessment of
automated disease detection in diabetic retinopathy screening using
two-field photography. PLoS One. 2011;6:e27524.
49. http://www.hubbletelemedical.com Accessed 15th July 2014.
50. Karnowski TP, Giancardo L, Li Y, Tobin KW, Chaum E. Retina
image analysis and ocular telehealth: The Oak Ridge National
Laboratory–Hamilton Eye Institute case study. Conf Proc IEEE
Eng Med Biol Soc. 2013;2013:7140–3.
14 Page 8 of 9 Curr Diab Rep (2015) 15:14
51. Garg S, Jani PD, Kshirsagar AV, King B, Chaum E. Telemedicine
and retinal imaging for improving diabetic retinopathy evaluation.
Arch Intern Med. 2012;172:1677–8.
52. Tobin KW, Chaum E, Govindasamy VP, Karnowski TP. Detection
of anatomic structures in human retinal imagery. IEEE Trans Med
Imaging. 2007;26:1729–39.
53.•Chaum E, Karnowski TP, Govindasamy VP, Abdelrahman M, Tobin
KW. Automated diagnosis of retinopathy by content-based image re-
trieval. Retina. 2008;28:1463–77. Key paper for the TRIAD program.
54. Tobin KW, Abramoff MD, Chaum E, Giancardo L, Govindasamy V,
Karnowski TP, et al. Using a patient image archive to diagnose retinop-
athy. Conf Proc IEEE Eng Med Biol Soc. 2008;2008:5441–4.
55. Tobin KW, Abdelrahman M, Chaum E, Govindasamy V,
Karnowski TP. A probabilistic framework for content-based diag-
nosis of retinal disease. Conf Proc IEEE Eng Med Biol Soc.
2007;2007:6744–7.
56. http://www.eyediagnosis.net Accessed 15th July 2014.
57. Niemeijer M, Abramoff MD, van Ginneken B. Image structure
clustering for image quality verification of color retina images in
diabetic retinopathy screening. Med Image Anal. 2006;10:888–98.
58. Niemeijer M, van Ginneken B, Staal J, Suttorp-Schulten MS,
Abramoff MD. Automatic detection of red lesions in digital color
fundus photographs. IEEE Trans Med Imaging. 2005;24:584–92.
59. Quellec G, Lamard M, Josselin PM, Cazuguel G, Cochener B, Roux C.
Optimal wavelet transform for the detection of microaneurysms in
retina photographs. IEEE Trans Med Imaging. 2008;27:1230–41.
60. Niemeijer M, van Ginneken B, Russell SR, Suttorp-Schulten MS,
Abramoff MD. Automated detection and differentiation of drusen,
exudates,and cotton-wool spots indigital color fundus photographs
for diabetic retinopathy diagnosis. Invest Ophthalmol Vis Sci.
2007;48:2260–7.
61. Tang L, Niemeijer M, Reinhardt JM, Garvin MK, Abramoff MD. Splat
feature classification with application to retinal hemorrhage detection in
fundus images. IEEE Trans Med Imaging. 2013;32:364–75.
62. Abramoff MD, Niemeijer M, Suttorp-Schulten MS, Viergever MA,
Russell SR, van Ginneken B. Evaluation of a system for automatic
detection of diabetic retinopathy from color fundus photographs in
a large population of patients with diabetes. Diabetes Care.
2008;31:193–8.
63. Abramoff MD, Reinhardt JM, Russell SR, Folk JC, Mahajan VB,
Niemeijer M, et al. Automated early detection of diabetic retinopa-
thy. Ophthalmology. 2010;117:1147–54.
64. Niemeijer M, Abramoff MD, van Ginneken B. Information fusion
for diabetic retinopathy CAD in digital color fundus photographs.
IEEE Trans Med Imaging. 2009;28:775–85.
65.•Abramoff MD, Folk JC, Han DP, Walker JD, Williams DF, Russell
SR, et al. Automated analysis of retinal images for detection of
referable diabetic retinopathy. JAMA Ophthalmol. 2013;131:351–
7. Key paper for the IDx Program.
66. In http://www.retmarker.com/ Accessed 15th July 2014.
67. Pires Dias JM, Oliveira CM, da Silva Cruz LA. Retinal image
quality assessment using generic image quality indicators.
Information Fusion. 2014;19:73–90.
68. Nunes S, Pires I, Rosa A, Duarte L, Bernardes R, Cunha-Vaz J.
Microaneurysm turnover is a biomarker for diabetic retinopathy
progression to clinically significant macular edema: findings for
type 2 diabetics with nonproliferative retinopathy.
Ophthalmologica. 2009;223:292–7.
69.•Bernardes R, Nunes S, Pereira I, Torrent T, Rosa A, Coelho D, et al.
Computer-assisted microaneurysm turnover in the early stages of
diabetic retinopathy. Ophthalmologica. 2009;223:284–91. Key pa-
per for the Retmarker program.
70. Nunes S, Ribeiro L, Lobo C, Cunha-Vaz J. Three different pheno-
types of mild nonproliferative diabetic retinopathy with different
risks for development of clinically significant macular edema.
Invest Ophthalmol Vis Sci. 2013;54:4595–604.
71. Haritoglou C, Kernt M, Neubauer A, Gerss J,Oliveira CM, Kampik
A, et al. Microaneurysm formation rate as a predictive marker for
progression to clinically significant macular edema in
nonproliferative diabetic retinopathy. Retina. 2014;34:157–64.
72. Ribeiro ML, Nunes SG, Cunha-Vaz JG. Microaneurysm turnover at
the macula predicts risk of development of clinically significant
macular edema in persons with mild nonproliferative diabetic reti-
nopathy. Diabetes Care. 2013;36:1254–9.
73. http://retinalyze.net. Accessed 10th August 2014.
74. Larsen N, Godt J, Grunkin M, Lund-Andersen H, Larsen M.
Automated detection of diabetic retinopathy in a fundus photo-
graphic screening population. Invest Ophthalmol Vis Sci.
2003;44:767–71.
75. Hansen AB, Hartvig NV, Jensen MS, Borch-Johnsen K, Lund-
Andersen H, Larsen M. Diabetic retinopathy screening using digital
non-mydriatic fundus photography and automated image analysis.
Acta Ophthalmol Scand. 2004;82:666–72.
76. Larsen M, Godt J, Larsen N, Lund-Andersen H, Sjolie AK, Agardh
E, et al. Automated detection of fundus photographic red lesions in
diabetic retinopathy. Invest Ophthalmol Vis Sci. 2003;44:761–6.
77.•Bouhaimed M, Gibbins R, Owens D. Automated detection of dia-
betic retinopathy: results of a screening study. Diabetes Technol
Ther. 2008;10:142–8. Key paper for the retinalyze program.
78. MESSIDOR: Digital Retinal Images. http://messidor.crihan.fr/
download-en.php.
79. The ROC website—University of Iowa. http://webeye.ophth.
uiowa.edu/ROC/var.1/www/.
80. Mitry D, Peto T, Hayat S, Morgan JE, Khaw KT, Foster PJ.
Crowdsourcing as a novel techniquefor retinal fundus photography
classification: analysis of images in the EPIC Norfolk cohort on
behalf of the UK Biobank Eye and Vision Consortium. PLoS
One. 2013;8:e71154.
81. Brady, C.J. ARVO 2014. Abstract 4826 Available at http://www.
arvo.org/webs/am2014/abstract/sessions/444.pdf.
82.•• Trucco E, Ruggeri A, Karnowski T, Giancardo L, Chaum E,
Hubschman JP, et al. Validating retinal fundus image analysis algo-
rithms: issues and a proposal. Invest Ophthalmol Vis Sci. 2013;54:
3546–59. Provides a framework and reference standard for the
evaluation of automated retinal image anaylsis systems.
Curr Diab Rep (2015) 15:14 Page 9 of 9 14