No full-text available
To read the full-text of this research,
you can request a copy directly from the authors.
Classification of fundus autofluorescence images based on macular function in retinitis pigmentosa using convolutional neural networks
Abstract
To determine whether convolutional neural networks (CNN) can classify the severity of central vision loss using fundus autofluorescence (FAF) images and color fundus images of retinitis pigmentosa (RP), and to evaluate the utility of those images for severity classification.
Retrospective observational study.
Medical charts of patients with RP who visited Nagoya University Hospital were reviewed. Eyes with atypical RP or previous surgery were excluded. The mild group was comprised of patients with a mean deviation value of > − 10 decibels, and the severe group of < − 20 decibels, in the Humphrey field analyzer 10-2 program. CNN models were created by transfer learning of VGG16 pretrained with ImageNet to classify patients as either mild or severe, using FAF images or color fundus images.
Overall, 165 patients were included in this study; 80 patients were classified into the severe and 85 into the mild group. The test data comprised 40 patients in each group, and the images of the remaining patients were used as training data, with data augmentation by rotation and flipping. The highest accuracies of the CNN models when using color fundus and FAF images were 63.75% and 87.50%, respectively.
Using FAF images may enable the accurate assessment of central vision function in RP. FAF images may include more parameters than color fundus images that can evaluate central visual function.
ResearchGate has not been able to resolve any citations for this publication.
Importance:
There is no widespread effective treatment to halt the progression of retinitis pigmentosa. Consequently, adequate assessment and estimation of residual visual function are important clinically.
Objective:
To examine whether deep learning can accurately estimate the visual function of patients with retinitis pigmentosa by using ultra-widefield fundus images obtained on concurrent visits.
Design, setting, and participants:
Data for this multicenter, retrospective, cross-sectional study were collected between January 1, 2012, and December 31, 2018. This study included 695 consecutive patients with retinitis pigmentosa who were examined at 5 institutions. Each of the 3 types of input images-ultra-widefield pseudocolor images, ultra-widefield fundus autofluorescence images, and both ultra-widefield pseudocolor and fundus autofluorescence images-was paired with 1 of the 31 types of ensemble models constructed from 5 deep learning models (Visual Geometry Group-16, Residual Network-50, InceptionV3, DenseNet121, and EfficientNetB0). We used 848, 212, and 214 images for the training, validation, and testing data, respectively. All data from 1 institution were used for the independent testing data. Data analysis was performed from June 7, 2021, to December 5, 2022.
Main outcomes and measures:
The mean deviation on the Humphrey field analyzer, central retinal sensitivity, and best-corrected visual acuity were estimated. The image type-ensemble model combination that yielded the smallest mean absolute error was defined as the model with the best estimation accuracy. After removal of the bias of including both eyes with the generalized linear mixed model, correlations between the actual values of the testing data and the estimated values by the best accuracy model were examined by calculating standardized regression coefficients and P values.
Results:
The study included 1274 eyes of 695 patients. A total of 385 patients were female (55.4%), and the mean (SD) age was 53.9 (17.2) years. Among the 3 types of images, the model using ultra-widefield fundus autofluorescence images alone provided the best estimation accuracy for mean deviation, central sensitivity, and visual acuity. Standardized regression coefficients were 0.684 (95% CI, 0.567-0.802) for the mean deviation estimation, 0.697 (95% CI, 0.590-0.804) for the central sensitivity estimation, and 0.309 (95% CI, 0.187-0.430) for the visual acuity estimation (all P < .001).
Conclusions and relevance:
Results of this study suggest that the visual function estimation in patients with retinitis pigmentosa from ultra-widefield fundus autofluorescence images using deep learning might help assess disease progression objectively. Findings also suggest that deep learning models might monitor the progression of retinitis pigmentosa efficiently during follow-up.
Purpose:
To assess the macular function by focal macular electroretinography (FMERG) and static perimetry in eyes with retinitis pigmentosa (RP).
Methods:
Eighty-eight eyes of 88 RP patients were analyzed. The relationships between the FMERG components and the mean deviations (MDs) of the Humphrey Field Analyzer (HFA) 10-2 were determined. Spectral domain optical coherence tomography was used to determine the integrity of the ellipsoid zone (EZ) and the interdigitation zone (IZ).
Results:
Forward-backward stepwise regression analyses showed that the amplitudes (r=0.45, P<0.01) and implicit times (r=-0.29, P<0.01) of the b-waves were significantly correlated with the MDs. Some of the eyes had reduced b-wave amplitudes (<1.0 µV) and disrupted IZ despite having a better MD (≥-10.0 dB) and intact EZ. Subgroup analyses of eyes with better MD (≥-10.0 dB) showed that the EZ width was correlated with the MDs but not with the b-wave amplitude. The thickness of the EZ-retinal pigment epithelium as an alternative indicator of IZ was correlated with the b-wave amplitude (r=0.32, P=0.04) but not with the MDs (r=-0.10, P=0.53).
Conclusions:
The fact that the FMERG amplitudes are reduced before the shortening of the EZ in the early stage of RP indicates that the FMERG amplitudes are an earlier indicator of macular dysfunction than the HFA10-2 findings.
Background
The efficiency of clinical trials for retinitis pigmentosa (RP) treatment is limited by the screening burden and lack of reliable surrogate markers for functional end points. Automated methods to determine visual acuity (VA) may help address these challenges. We aimed to determine if VA could be estimated using confocal scanning laser ophthalmoscopy (cSLO) imaging and deep learning (DL).
Methods
Snellen corrected VA and cSLO imaging were obtained retrospectively. The Johns Hopkins University (JHU) dataset was used for 10-fold cross-validations and internal testing. The Amsterdam University Medical Centers (AUMC) dataset was used for external independent testing. Both datasets had the same exclusion criteria: visually significant media opacities and images not centred on the central macula. The JHU dataset included patients with RP with and without molecular confirmation. The AUMC dataset only included molecularly confirmed patients with RP. Using transfer learning, three versions of the ResNet-152 neural network were trained: infrared (IR), optical coherence tomography (OCT) and combined image (CI).
Results
In internal testing (JHU dataset, 2569 images, 462 eyes, 231 patients), the area under the curve (AUC) for the binary classification task of distinguishing between Snellen VA 20/40 or better and worse than Snellen VA 20/40 was 0.83, 0.87 and 0.85 for IR, OCT and CI, respectively. In external testing (AUMC dataset, 349 images, 166 eyes, 83 patients), the AUC was 0.78, 0.87 and 0.85 for IR, OCT and CI, respectively.
Conclusions
Our algorithm showed robust performance in predicting visual impairment in patients with RP, thus providing proof-of-concept for predicting structure-function correlation based solely on cSLO imaging in patients with RP.
Background:
Little is known about genotype-phenotype correlations of RP1-associated retinal dystrophies in the Japanese population. We aimed to investigate the genetic spectrum of RP1 variants and provide a detailed description of the clinical findings in Japanese patients.
Methods:
In total, 607 patients with inherited retinal diseases were examined using whole-exome/whole-genome sequencing (WES/WGS). PCR-based screening for an Alu element insertion (c.4052_4053ins328/p.Tyr1352AlafsTer9) was performed in 18 patients with autosomal-recessive (AR)-retinitis pigmentosa (RP) or AR-cone dystrophy (COD)/cone-rod dystrophy (CORD), including seven patients with heterozygous RP1 variants identified by WES/WGS analysis, and 11 early onset AR-RP patients, in whom no pathogenic variant was identified. We clinically examined 25 patients (23 families) with pathogenic RP1 variants, including five patients (five families) with autosomal-dominant (AD)-RP, 13 patients (11 families) with AR-RP, and seven patients (seven families) with AR-COD/CORD.
Results:
We identified 18 pathogenic RP1 variants, including seven novel variants. Interestingly, the Alu element insertion was the most frequent variant (32.0%, 16/50 alleles). The clinical findings revealed that the age at onset and disease progression occurred significantly earlier and faster in AR-RP patients compared to AD-RP or AR-COD/CORD patients.
Conclusions:
Our results suggest a genotype-phenotype correlation between variant types/locations and phenotypes (AD-RP, AR-RP, and AR-COD/CORD), and the Alu element insertion was the most major variant in Japanese patients with RP1-associated retinal dystrophies.
Background/Aims
To investigate the utility of a data-driven deep learning approach in patients with inherited retinal disorder (IRD) and to predict the causative genes based on fundus photography and fundus autofluorescence (FAF) imaging.
Methods
Clinical and genetic data from 1302 subjects from 729 genetically confirmed families with IRD registered with the Japan Eye Genetics Consortium were reviewed. Three categories of genetic diagnosis were selected, based on the high prevalence of their causative genes: Stargardt disease ( ABCA4 ), retinitis pigmentosa ( EYS ) and occult macular dystrophy ( RP1L1 ). Fundus photographs and FAF images were cropped in a standardised manner with a macro algorithm. Images for training/testing were selected using a randomised, fourfold cross-validation method. The application program interface was established to reach the learning accuracy of concordance (target: >80%) between the genetic diagnosis and the machine diagnosis ( ABCA4 , EYS , RP1L1 and normal).
Results
A total of 417 images from 156 Japanese subjects were examined, including 115 genetically confirmed patients caused by the three prevalent causative genes and 41 normal subjects. The mean overall test accuracy for fundus photographs and FAF images was 88.2% and 81.3%, respectively. The mean overall sensitivity/specificity values for fundus photographs and FAF images were 88.3%/97.4% and 81.8%/95.5%, respectively.
Conclusion
A novel application of deep neural networks in the prediction of the causative IRD genes from fundus photographs and FAF, with a high prediction accuracy of over 80%, was highlighted. These achievements will extensively promote the quality of medical care by facilitating early diagnosis, especially by non-specialists, access to care, reducing the cost of referrals, and preventing unnecessary clinical and genetic testing.
Various advances have been made to treat retinal diseases, including new treatment strategies and innovations in surgical devices. However, treatment of degenerative retinal diseases, such as retinitis pigmentosa (RP) and age‐related macular degeneration (AMD), continues to pose a significant challenge. In this review, we focus on the use of embryonic stem cells (ESCs) and induced pluripotent stem cells (iPSCs) to treat retinal diseases by harnessing the ability of stem cells to differentiate into different body tissues. The retina is a tissue specialized for light sensing, and its degradation leads to vision loss. As part of the central nervous system, the retina has very low regenerative capability, and therefore treatment options are limited once it degenerates. Nevertheless, innovations in methods to induce the generation of retinal cells and tissues from ESCs/iPSCs enable the development of novel approaches for these irreversible diseases. Here we review some historical background and current clinical trials involving the use of stem cell‐derived retinal pigment epithelial cells for AMD treatment and stem cell‐derived retinal cells/tissues for RP therapy. Finally, we discuss our future vision of regenerative treatment for retinal diseases with a partial focus on our studies and introduce other interesting approaches for restoring vision.
Background. In recent years, deep learning has been increasingly applied to a vast array of ophthalmological diseases. Inherited retinal diseases (IRD) are rare genetic conditions with a distinctive phenotype on fundus autofluorescence imaging (FAF). Our purpose was to automatically classify different IRDs by means of FAF images using a deep learning algorithm. Methods. In this study, FAF images of patients with retinitis pigmentosa (RP), Best disease (BD), Stargardt disease (STGD), as well as a healthy comparable group were used to train a multilayer deep convolutional neural network (CNN) to differentiate FAF images between each type of IRD and normal FAF. The CNN was trained and validated with 389 FAF images. Established augmentation techniques were used. An Adam optimizer was used for training. For subsequent testing, the built classifiers were then tested with 94 untrained FAF images. Results. For the inherited retinal disease classifiers, global accuracy was 0.95. The precision-recall area under the curve (PRC-AUC) averaged 0.988 for BD, 0.999 for RP, 0.996 for STGD, and 0.989 for healthy controls. Conclusions. This study describes the use of a deep learning-based algorithm to automatically detect and classify inherited retinal disease in FAF. Hereby, the created classifiers showed excellent results. With further developments, this model may be a diagnostic tool and may give relevant information for future therapeutic approaches.
Ophthalmological analysis plays a vital role in the diagnosis of various eye diseases, such as glaucoma, retinitis pigmentosa (RP), and diabetic and hypertensive retinopathy. RP is a genetic retinal disorder that leads to progressive vision degeneration and initially causes night blindness. Currently, the most commonly applied method for diagnosing retinal diseases is optical coherence tomography (OCT)-based disease analysis. In contrast, fundus imaging-based disease diagnosis is considered a low-cost diagnostic solution for retinal diseases. This study focuses on the detection of RP from the fundus image, which is a crucial task because of the low quality of fundus images and non-cooperative image acquisition conditions. Automatic detection of pigment signs in fundus images can help ophthalmologists and medical practitioners in diagnosing and analyzing RP disorders. To accurately segment pigment signs for diagnostic purposes, we present an automatic RP segmentation network (RPS-Net), which is a specifically designed deep learning-based semantic segmentation network to accurately detect and segment the pigment signs with fewer trainable parameters. Compared with the conventional deep learning methods, the proposed method applies a feature enhancement policy through multiple dense connections between the convolutional layers, which enables the network to discriminate between normal and diseased eyes, and accurately segment the diseased area from the background. Because pigment spots can be very small and consist of very few pixels, the RPS-Net provides fine segmentation, even in the case of degraded images, by importing high-frequency information from the preceding layers through concatenation inside and outside the encoder-decoder. To evaluate the proposed RPS-Net, experiments were performed based on 4-fold cross-validation using the publicly available Retinal Images for Pigment Signs (RIPS) dataset for detection and segmentation of retinal pigments. Experimental results show that RPS-Net achieved superior segmentation performance for RP diagnosis, compared with the state-of-the-art methods.
We propose a technique for producing ‘visual explanations’ for decisions from a large class of Convolutional Neural Network (CNN)-based models, making them more transparent and explainable. Our approach—Gradient-weighted Class Activation Mapping (Grad-CAM), uses the gradients of any target concept (say ‘dog’ in a classification network or a sequence of words in captioning network) flowing into the final convolutional layer to produce a coarse localization map highlighting the important regions in the image for predicting the concept. Unlike previous approaches, Grad-CAM is applicable to a wide variety of CNN model-families: (1) CNNs with fully-connected layers (e.g.VGG), (2) CNNs used for structured outputs (e.g.captioning), (3) CNNs used in tasks with multi-modal inputs (e.g.visual question answering) or reinforcement learning, all without architectural changes or re-training. We combine Grad-CAM with existing fine-grained visualizations to create a high-resolution class-discriminative visualization, Guided Grad-CAM, and apply it to image classification, image captioning, and visual question answering (VQA) models, including ResNet-based architectures. In the context of image classification models, our visualizations (a) lend insights into failure modes of these models (showing that seemingly unreasonable predictions have reasonable explanations), (b) outperform previous methods on the ILSVRC-15 weakly-supervised localization task, (c) are robust to adversarial perturbations, (d) are more faithful to the underlying model, and (e) help achieve model generalization by identifying dataset bias. For image captioning and VQA, our visualizations show that even non-attention based models learn to localize discriminative regions of input image. We devise a way to identify important neurons through Grad-CAM and combine it with neuron names (Bau et al. in Computer vision and pattern recognition, 2017) to provide textual explanations for model decisions. Finally, we design and conduct human studies to measure if Grad-CAM explanations help users establish appropriate trust in predictions from deep networks and show that Grad-CAM helps untrained users successfully discern a ‘stronger’ deep network from a ‘weaker’ one even when both make identical predictions. Our code is available at https://github.com/ramprs/grad-cam/, along with a demo on CloudCV (Agrawal et al., in: Mobile cloud visual media computing, pp 265–290. Springer, 2015) (http://gradcam.cloudcv.org) and a video at http://youtu.be/COjUB9Izk6E.
Evaluating the discrimination ability of a deep convolution neural network for ultrawide-field pseudocolor imaging and ultrawide-field autofluorescence of retinitis pigmentosa. In total, the 373 ultrawide-field pseudocolor and ultrawide-field autofluorescence images (150, retinitis pigmentosa; 223, normal) obtained from the patients who visited the Department of Ophthalmology, Tsukazaki Hospital were used. Training with a convolutional neural network on these learning data objects was conducted. We examined the K-fold cross validation (K = 5). The mean area under the curve of the ultrawide-field pseudocolor group was 0.998 (95% confidence interval (CI) [0.9953-1.0]) and that of the ultrawide-field autofluorescence group was 1.0 (95% CI [0.9994-1.0]). The sensitivity and specificity of the ultrawide-field pseudocolor group were 99.3% (95% CI [96.3%-100.0%]) and 99.1% (95% CI [96.1%-99.7%]), and those of the ultrawide-field autofluorescence group were 100% (95% CI [97.6%-100%]) and 99.5% (95% CI [96.8%-99.9%]), respectively. Heatmaps were in accordance with the clinician's observations. Using the proposed deep neural network model, retinitis pigmentosa can be distinguished from healthy eyes with high sensitivity and specificity on ultrawide-field pseudocolor and ultrawide-field autofluorescence images.
Given the importance of visual information to many daily activities, retinal degenerative diseases—which include both inherited conditions (such as retinitis pigmentosa) and acquired conditions (such as age-related macular degeneration)—can have a dramatic impact on human lives. The therapeutic options for these diseases remain limited. Since the discovery of the first causal gene for retinitis pigmentosa almost three decades ago, more than 250 genes have been identified, and gene therapies have been rapidly developed. Simultaneously, stem cell technologies such as induced pluripotent stem cell–based transplantation have advanced and have been applied to the treatment of retinal degenerative diseases. Here, we review recent progress in these expanding fields and discuss the potential for precision medicine in ophthalmic care.
Expected final online publication date for the Annual Review of Genomics and Human Genetics Volume 22 is August 30, 2019. Please see http://www.annualreviews.org/page/journal/pubdates for revised estimates.
Retinitis pigmentosa (RP) encompasses a group of inherited retinal dystrophies characterized by the primary degeneration of rod and cone photoreceptors. RP is a leading cause of visual disability, with a worldwide prevalence of 1:4000. Although the majority of RP cases are non-syndromic, 20-30% of patients with RP also have an associated non-ocular condition. RP typically manifests with night blindness in adolescence, followed by concentric visual field loss, reflecting the principal dysfunction of rod photoreceptors; central vision loss occurs later in life due to cone dysfunction. Photoreceptor function measured with an electroretinogram is markedly reduced or even absent. Optical coherence tomography (OCT) and fundus autofluorescence (FAF) imaging show a progressive loss of outer retinal layers and altered lipofuscin distribution in a characteristic pattern. Over the past three decades, a vast number of disease-causing variants in more than 80 genes have been associated with non-syndromic RP. The wide heterogeneity of RP makes it challenging to describe the clinical findings and pathogenesis. In this review, we provide a comprehensive overview of the clinical characteristics of RP specific to genetically defined patient subsets. We supply a unique atlas with color fundus photographs of most RP subtypes, and we discuss the relevant considerations with respect to differential diagnoses. In addition, we discuss the genes involved in the pathogenesis of RP, as well as the retinal processes that are affected by pathogenic mutations in these genes. Finally, we review management strategies for patients with RP, including counseling, visual rehabilitation, and current and emerging therapeutic options.
The aim of this study is to determine the progress of the visual field defects obtained by the Humphrey Field Analyzer 10-2 program (HFA 10-2) in patients with retinitis pigmentosa (RP). The medical records of 45 eyes of 45 RP patients who had at least 3 visual field tests were reviewed. Linear mixed models were used to follow the changes of the mean deviation and the average sensitivity of 4, 12, and 20 points in three concentric squares, designated as S4, S12, and S20. The median follow-up time was 3.86 years [range: 1.93 to 9.86, IQR (Interquartile range): 3.01 to 4.93]. The median number of the visual field tests was 3 (range: 3 to 15, IQR: 3 to 4). The mean change of the MD was −0.46 dB/year (−5.80%/year). When the patients were grouped by the average initial MD, the less advanced group had slower progressions than the more advanced group in S4, S12, and S20. These results should be useful in understanding the pathological changes of RP in the central visual field.
Fundus autofluorescence (FAF) is a non-invasive retinal imaging modality used in clinical practice to non-invasively map changes at the level of the retinal pigment epithelium (RPE)/photoreceptor complex and alterations of macular pigment distribution. This imaging method is based on the visualization of intrinsic fluorophores and may be easily and rapidly used in routine patient care. Excessive accumulation of lipofuscin granules in the lysosomal compartment of RPE cells represents a common downstream pathogenic pathway in various hereditary and complex retinal diseases. The clinical applications of FAF continue to expand. It is now an essential tool for evaluating macular dystrophies and various hereditary retinal disorders. Fundus autofluorescence (FAF) may detect abnormalities beyond those detected on funduscopic examination, fluorescein angiography (FA) or optical coherence tomography (OCT). Fundus autofluorescence (FAF) imaging is particularly helpful for differential diagnosis, detection and extent delineation of involved retinal areas, genotype-phenotype correlations and monitoring of changes overtime. Given its ease of use, non-invasive nature and value in characterizing retinal disease, FAF enjoys increasing clinical relevance. This review summarizes basic principles and FAF findings in various hereditary retinal diseases.
We present a class of efficient models called MobileNets for mobile and embedded vision applications. MobileNets are based on a streamlined architecture that uses depth-wise separable convolutions to build light weight deep neural networks. We introduce two simple global hyper-parameters that efficiently trade off between latency and accuracy. These hyper-parameters allow the model builder to choose the right sized model for their application based on the constraints of the problem. We present extensive experiments on resource and accuracy tradeoffs and show strong performance compared to other popular models on ImageNet classification. We then demonstrate the effectiveness of MobileNets across a wide range of applications and use cases including object detection, finegrain classification, face attributes and large scale geo-localization.
Neuroprotection may prevent or forestall the progression of incurable eye diseases, such as retinitis pigmentosa, one of the major causes of adult blindness. Decreased cellular ATP levels may contribute to the pathology of this eye disease and other neurodegenerative diseases. Here we describe small compounds (Kyoto University Substances, KUSs) that were developed to inhibit the ATPase activity of VCP (valosin-containing protein), the most abundant soluble ATPase in the cell. Surprisingly, KUSs did not significantly impair reported cellular functions of VCP but nonetheless suppressed the VCP-dependent decrease of cellular ATP levels. Moreover, KUSs, as well as exogenous ATP or ATP-producing compounds, e.g. methylpyruvate, suppressed endoplasmic reticulum stress, and demonstrably protected various types of cultured cells from death, including several types of retinal neuronal cells. We then examined their in vivo efficacies in rd10, a mouse model of retinitis pigmentosa. KUSs prevented photoreceptor cell death and preserved visual function. These results reveal an unexpected, crucial role of ATP consumption by VCP in determining cell fate in this pathological context, and point to a promising new neuroprotective strategy for currently incurable retinitis pigmentosa.
To examine the presence and functional significance of annular fundus autofluorescence abnormalities in patients with different retinal dystrophies.
Eighty one patients were ascertained who had a parafoveal ring of high density on fundus autofluorescence imaging. Sixty two had had a clinical diagnosis of retinitis pigmentosa (RP) or Usher syndrome with normal visual acuity. Others included a case of Leber congenital amaurosis and genetically confirmed cases of cone or cone-rod dystrophy (GUCA1A, RPGR, RIMS1), "cone dystrophy with supernormal rod ERG" (KCNV2) and X-linked retinoschisis (RS1). International-standard full-field and pattern electroretinography (ERG; PERG) were performed. Some patients with rod-cone or cone-rod dystrophy underwent multifocal ERG (mfERG) testing and photopic and scotopic fine matrix mapping (FMM).
In patients with RP, the radius of the parafoveal ring of high density correlated with PERG P50 (R = 0.83, P < 0.0005, N = 62) and encircled areas of preserved photopic function. In the other patients, AF rings either resembled those seen in RP or encircled an area of central atrophy. Ring radius was inversely related to the PERG P50 component in 4 of 18 cases with a detectable response. FMM showed that arcs of high density were associated with a gradient of sensitivity change.
Parafoveal rings of high density autofluorescence are a non-specific manifestation of retinal dysfunction that can occur in different retinal dystrophies. Electrophysiology remains essential for accurate diagnosis. The high correlation of autofluorescence with PERG, mfERG and FMM demonstrates that AF abnormalities have functional significance and may help identify suitable patients and retinal areas amenable to future therapeutic intervention.
Recent work has shown that convolutional networks can be substantially deeper, more accurate, and efficient to train if they contain shorter connections between layers close to the input and those close to the output. In this paper, we embrace this observation and introduce the Dense Convolutional Network (DenseNet), which connects each layer to every other layer in a feed-forward fashion. Whereas traditional convolutional networks with L layers have L connections - one between each layer and its subsequent layer - our network has L(L+1)/2 direct connections. For each layer, the feature-maps of all preceding layers are used as inputs, and its own feature-maps are used as inputs into all subsequent layers. DenseNets have several compelling advantages: they alleviate the vanishing-gradient problem, strengthen feature propagation, encourage feature reuse, and substantially reduce the number of parameters. We evaluate our proposed architecture on four highly competitive object recognition benchmark tasks (CIFAR-10, CIFAR-100, SVHN, and ImageNet). DenseNets obtain significant improvements over the state-of-the-art on most of them, whilst requiring less computation to achieve high performance. Code and pre-trained models are available at https://github.com/liuzhuang13/DenseNet .
Photoreception requires amplification by mammalian rhodopsin through G protein activation, which requires a visual cycle. To achieve this in retinal gene therapy, we incorporated human rhodopsin cytoplasmic loops into Gloeobacter rhodopsin, thereby generating Gloeobacter and human chimeric rhodopsin (GHCR). In a murine model of inherited retinal degeneration, we induced retinal GHCR expression by intravitreal injection of a recombinant adeno-associated virus vector. Retinal explant and visual thalamus electrophysiological recordings, behavioral tests, and histological analysis showed that GHCR restored dim-environment vision and prevented the progression of retinal degeneration. Thus, GHCR may be a potent clinical tool for the treatment of retinal disorders.
Timely treatment is essential in the management of glaucoma. However, subjective assessment of visual field (VF) progression is not recommended, because it can be unreliable. There are two types of artificial intelligence (AI) strong and weak (machine learning). Weak AIs can perform specific tasks. Linear regression is a method of weak AI. Using linear regression in the real-world clinic has enabled analyzing and predicting VF progression. However, caution is still required when interpreting the results, because whenever the number of VF data sets investigated is small, the predictions can be inaccurate. Several other non-ordinal, or modern AI methods have been constructed to improve prediction accuracy, such as clustering and more modern AI methods of Analysis with Non-Stationary Weibull Error Regression and Spatial Enhancement (ANSWERS), Variational Bayes Linear Regression (VBLR), Kalman Filter and sparse modeling (The least absolute shrinkage and selection operator regression: Lasso). It is also possible to improve the prediction performance using retinal thickness measured with optical coherence tomography by using machine learning methods, such as multitask learning.
PurposeTo report the clinical course and high resolution images of autosomal recessive retinitis pigmentosa (RP) associated with a variant of the RP1 gene (c.4052_4053ins328/p.Tyr1352Alafs*9; m1), a high frequency founder variant in Japanese RP patients.Study designRetrospective case series.Methods
Nine patients from 5 unrelated Japanese families were studied. Five patients had the m1 variant homozygously, and 4 patients had the m1 variant compound heterozygously with another frameshift variant (c.4196delG/p.Cys1399Leufs*5). Ophthalmic examinations including adaptive optics (AO) fundus imaging were performed periodically.ResultsThe fundus photographs, fundus autofluorescence (FAF) images, and optical coherence tomographic (OCT) images indicated severe retinal degeneration in all the patients involving the macula even at a young age (20 s). The areas of surviving photoreceptors in the central macula were seen as hyper-autofluorescent regions in the FAF images and preserved outer retinal structure in the OCT images; they were identifiable in the AO fundus images in 8 eyes. The borders of the surviving photoreceptor areas were surrounded by hyporeflective clumps, presumably containing melanin, and the size of these areas decreased progressively during the 4-year follow-up period. The disappearance of the surviving photoreceptor areas was associated with complete blindness.Conclusion
Patients with RP associated with the m1 variant have a progressive and severe retinal degeneration that begins at an early age. Monitoring the surviving photoreceptor areas by AO fundus imaging can provide a more precise pathological record of retinal degeneration.
Mutations in the gene encoding the phosphodiesterase 6 alpha subunit (PDE6A) account for 3-4% of autosomal recessive retinitis pigmentosa (RP), and currently no treatment is available. There are four animal models for PDE6A-RP: a dog with a frameshift truncating mutation (p.Asn616ThrfsTer39) and three mouse models with missense mutations (Val685Met, Asp562Trp, and Asp670Gly) showing a range of phenotype severities. Initial proof-of-concept gene augmentation studies in the Asp670Gly mouse model and dog model used a subretinally delivered adeno-associated virus serotype 8 with a 733 tyrosine capsid mutation delivering species-specific Pde6a cDNAs. These restored some rod-mediated function and preserved retinal structure. Subsequently, a translatable vector (AAV8 with a human rhodopsin promoter and human PDE6A cDNA) was tested in the dog and the Asp670Gly mouse model. In the dog, there was restoration of rod function, a robust rod-mediated ERG, and introduction of dim-light vision. Treatment improved morphology of the photoreceptor layer, and the retina was preserved in the treated region. In the Asp670Gly mouse, therapy also preserved photoreceptors with cone survival being reflected by maintenance of cone-mediated ERG responses. These studies are an important step toward a translatable therapy for PDE6A-RP.
This brief review summarizes the major proof-of-concept gene therapy studies for autosomal dominant retinitis pigmentosa (RP) caused by mutations in the rhodopsin gene (RHO-adRP) that have been conducted over the past 20 years in various animal models. We have listed in tabular form the various approaches, gene silencing reagents, gene delivery strategies, and salient results from these studies.
Purpose
To assess the performance of machine learning classifiers for prediction of progression of normal-tension glaucoma (NTG) in young myopic patients.
Study design
Cross-sectional study.
Methods
One hundred and fifty-five eyes of 155 myopic NTG patients (axial length [AL] ≥ 24.00 mm and refractive error ≤ − 3.0 D) between the ages of 20 and 40 were enrolled and divided into training (110) and test (45) sets. Sixty-five eyes showed glaucoma progression as defined by standard automated perimetry, while 91 eyes (nonprogressors) had been stable over the course of a follow-up period of at least 3 years. Two machine learning classifiers were built using the random forest and extremely randomized trees (extra-trees) models. Baseline clinical measurements obtained only at the initial visit were used as input features. The area under the receiver operating characteristic curve (AUC) was calculated to evaluate the accuracy of prediction.
Results
Mean age and AL did not significantly differ between the 2 groups on either the training or the test set. The extra-trees model achieved an AUC of 0.881 [95% CI 0.814–0.945], higher than that of the random forest model (0.811 [0.731–0.888]; P = 0.010). The extra-trees model also outperformed all the clinical measurements for prediction of NTG progression, including average macular ganglion cell-inner plexiform layer thickness (0.735 [0.639–0.831]) and average circumpapillary retinal nerve fiber layer thickness (0.691 [0.590–0.792]; both P < 0.001).
Conclusions
In young myopic patients, the machine learning classifier with the extra-trees model can predict glaucomatous progression more effectively than clinical diagnostic parameters.
Purpose:
To determine the relationship between the sensitivity of the retina in the central 10° and the thickness of the retinal layers in patients with retinitis pigmentosa (RP).
Methods:
Fifty-two RP patients were studied. All of the patients had been examined by the Humphrey Field Analyzer 10-2 program (HFA10-2) and spectral-domain optical coherence tomography (SD-OCT). The thicknesses of the photoreceptor outer segment (OS), outer nuclear layer (ONL), inner nuclear layer (INL), and the retinal nerve fiber layer (RNFL) were measured at 1°, 3°, 5°, 7°, and 9° from the fovea. The same measurements were made on the SD-OCT images of 40 healthy subjects and used as controls. The relationships between the retinal sensitivities and retinal layer thicknesses were determined.
Results:
The thicknesses of the OS and ONL and their product were significantly and positively correlated with the retinal sensitivities. The thickness of the INL was significantly and negatively correlated with the sensitivity. The strongest correlation with the sensitivity was with the OS thickness (marginal R2 [mR2] = 0.525, P < 0.001), followed by the product of the OS and ONL thicknesses (mR2 = 0.420, P < 0.001), ONL thickness (mR2 = 0.416, P < 0.001), and the INL thickness (mR2 = 0.014, P = 0.044). The thickness of the RNFL was not correlated with the sensitivity (mR2 = 0.005, P = 0.331).
Conclusions:
In contrast to previous reports, the thickness of the OS reflected the retinal sensitivity better than the product of OS and ONL.
In this work we investigate the effect of the convolutional network depth on its accuracy in the large-scale image recognition setting. Our main contribution is a thorough evaluation of networks of increasing depth, which shows that a significant improvement on the prior-art configurations can be achieved by pushing the depth to 16-19 weight layers. These findings were the basis of our ImageNet Challenge 2014 submission, where our team secured the first and the second places in the localisation and classification tracks respectively.
Background
Phase 1 studies have shown potential benefit of gene replacement in RPE65-mediated inherited retinal dystrophy. This phase 3 study assessed the efficacy and safety of voretigene neparvovec in participants whose inherited retinal dystrophy would otherwise progress to complete blindness.
Methods
In this open-label, randomised, controlled phase 3 trial done at two sites in the USA, individuals aged 3 years or older with, in each eye, best corrected visual acuity of 20/60 or worse, or visual field less than 20 degrees in any meridian, or both, with confirmed genetic diagnosis of biallelic RPE65 mutations, sufficient viable retina, and ability to perform standardised multi-luminance mobility testing (MLMT) within the luminance range evaluated, were eligible. Participants were randomly assigned (2:1) to intervention or control using a permuted block design, stratified by age (<10 years and ≥10 years) and baseline mobility testing passing level (pass at ≥125 lux vs <125 lux). Graders assessing primary outcome were masked to treatment group. Intervention was bilateral, subretinal injection of 1·5 × 10¹¹ vector genomes of voretigene neparvovec in 0·3 mL total volume. The primary efficacy endpoint was 1-year change in MLMT performance, measuring functional vision at specified light levels. The intention-to-treat (ITT) and modified ITT populations were included in primary and safety analyses. This trial is registered with ClinicalTrials.gov, number NCT00999609, and enrolment is complete.
Findings
Between Nov 15, 2012, and Nov 21, 2013, 31 individuals were enrolled and randomly assigned to intervention (n=21) or control (n=10). One participant from each group withdrew after consent, before intervention, leaving an mITT population of 20 intervention and nine control participants. At 1 year, mean bilateral MLMT change score was 1·8 (SD 1·1) light levels in the intervention group versus 0·2 (1·0) in the control group (difference of 1·6, 95% CI 0·72–2·41, p=0·0013). 13 (65%) of 20 intervention participants, but no control participants, passed MLMT at the lowest luminance level tested (1 lux), demonstrating maximum possible improvement. No product-related serious adverse events or deleterious immune responses occurred. Two intervention participants, one with a pre-existing complex seizure disorder and another who experienced oral surgery complications, had serious adverse events unrelated to study participation. Most ocular events were mild in severity.
Interpretation
Voretigene neparvovec gene replacement improved functional vision in RPE65-mediated inherited retinal dystrophy previously medically untreatable.
Funding
Spark Therapeutics.
In this work we investigate the effect of the convolutional network depth on
its accuracy in the large-scale image recognition setting. Our main
contribution is a thorough evaluation of networks of increasing depth, which
shows that a significant improvement on the prior-art configurations can be
achieved by pushing the depth to 16-19 weight layers. These findings were the
basis of our ImageNet Challenge 2014 submission, where our team secured the
first and the second places in the localisation and classification tracks
respectively.
Retinitis pigmentosa is a clinically and genetically heterogeneous group of hereditary disorders in which there is progressive loss of photoreceptor and pigment epithelial function. The prevalence of retinitis pigmentosa is between 1/3000 and 1/5000 making it one of the most common causes of visual impairment in all age groups. The natural history, differential diagnosis, diagnostic clinical and electrophysiologic findings are reviewed. Generalization about the different genetic subtypes of retinitis pigmentosa are reviewed along with the uses of DNA probes for linkage studies. Syndromes in which retinitis pigmentosa is a manifestation are summarized.
Hereditary degenerations of the human retina are genetically heterogeneous, with well over 100 genes implicated so far. This Seminar focuses on the subset of diseases called retinitis pigmentosa, in which patients typically lose night vision in adolescence, side vision in young adulthood, and central vision in later life because of progressive loss of rod and cone photoreceptor cells. Measures of retinal function, such as the electroretinogram, show that photoreceptor function is diminished generally many years before symptomic night blindness, visual-field scotomas, or decreased visual acuity arise. More than 45 genes for retinitis pigmentosa have been identified. These genes account for only about 60% of all patients; the remainder have defects in as yet unidentified genes. Findings of controlled trials indicate that nutritional interventions, including vitamin A palmitate and omega-3-rich fish, slow progression of disease in many patients. Imminent treatments for retinitis pigmentosa are greatly anticipated, especially for genetically defined subsets of patients, because of newly identified genes, growing knowledge of affected biochemical pathways, and development of animal models.
To determine the efficient parameters to evoke electrical phosphenes is essential for the development of a retinal prosthesis. We studied the efficient parameters in normal subjects and investigated if suprachoroidal-transretinal stimulation (STS) is effective in patients with advanced retinitis pigmentosa (RP) using these efficient parameters.
The amplitude of pupillary reflex (PR) evoked by transcorneal electrical stimulation (TcES) was determined at different frequencies in eight normal subjects. The relationship between localized phosphenes elicited by transscleral electrical stimulation (TsES) and the pulse parameters was also examined in six normal subjects. The phosphenes evoked by STS were examined in two patients with RP with bare light perception. Biphasic pulses (cathodic first, duration: 0.5 or 1.0 ms, frequency: 20 Hz) were applied through selected channel(s). The size and shape of the phosphenes perceived by the patients were recorded.
The maximum PR was evoked by TcES with a frequency of 20 Hz. The brightest phosphene was elicited by TsES with a pulse train of more than 10 pulses, duration of 0.5-1.0 ms and a frequency of 20 to 50 Hz. In RP patients, localized phosphenes were elicited with a current of 0.3-0.5 mA (0.5 ms) in patient 1 and 0.4 mA (1.0 ms) in patient 2. Two isolated or dumbbell-shaped phosphenes were perceived when the stimulus was delivered through two adjacent channels.
Biphasic pulse trains (> or =10 pulses) with a duration of 0.5-1.0 ms and a frequency of 20-50 Hz were efficient for evoking phosphenes by localized extraocular stimulation in normal subjects. With these parameters, STS is a feasible method to use with a retinal prosthesis even in advanced stages of RPs.
A retinal prosthesis was permanently implanted in the eye of a completely blind test subject. This report details the results from the first 10 weeks of testing with the implant subject. The implanted device included an extraocular case to hold electronics, an intraocular electrode array (platinum disks, 4 x 4 arrangement) designed to interface with the retina, and a cable to connect the electronics case to the electrode array. The subject was able to see perceptions of light (spots) on all 16 electrodes of the array. In addition, the subject was able to use a camera to detect the presence or absence of ambient light, to detect motion, and to recognize simple shapes.
Gene therapy for retinitis pigmentosa
- J B Ducloyer
- Le Meur
- G Cronin
- T Adjali
- O Weber
Ducloyer JB, Le Meur G, Cronin T, Adjali O, Weber M. Gene therapy for retinitis pigmentosa. Med Sci (Paris). 2020;36:607-15.
Deep learning with depthwise separable convolutions
- F Chollet
- Xception
Chollet F, Xception. Deep learning with depthwise separable
convolutions. IEEE conference on computer vision and pattern
recognition (CVPR), Honolulu, Hawaii, USA, July, 2017. pp
1800-7.
ultra-widefield fundus images of eyes with retinitis pigmentosa
ultra-widefield fundus images of eyes with retinitis pigmentosa.
JAMA Ophthalmol. 2023;141:305-13.
Prediction of visual impairment in retinitis pigmentosa using deep learning and multimodal fundus images
- Tya Liu
- C Ling
- L Hahn
- C K Jones
- C J Boon
- M S Singh
Liu TYA, Ling C, Hahn L, Jones CK, Boon CJ, Singh MS. Prediction of visual impairment in retinitis pigmentosa using deep
learning and multimodal fundus images. Br J Ophthalmol.
2023;107:1484-9.
Estimation of visual function using deep learning from
- D Nagasato
- T Sogawa
- M Tanabe
- H Tabuchi
- S Numa
- A Oishi
Nagasato D, Sogawa T, Tanabe M, Tabuchi H, Numa S, Oishi
A, et al. Estimation of visual function using deep learning from