The computer based method to diabetic retinopathy assessment in retinal images : a review

Diabetic retinopathy accounts for a considerable amount of the blindness especially among the patients who are between 20 to 60 years. The early detection of this disease plays an important role in preventing vision damages and appropriate follow-up care of diabetic eye. Manual investigation of color fundus images to check morphological changes in dark and bright lesions is tedious work and very time-consuming that can be made easily with the help of computer-aided diagnosis system. Many techniques were proposed for early detection of the abnormalities in the retinal images help the ophthalmologists recognize the retinopathy earlier. This paper presents a review of various automated algorithms that have been used for the detection of diabetic retinopathy.


INTRODUCTION
Nowadays, Diabetes Mellitus (DM) is considered to be one of the highly prevalent diseases all over the world.Almost 1 person out of 10 persons suffers from that.This disease is of two types and has direct and deadly effect on the eyes of the patients.In type 1 diabetes, the cells that release insulin destroyed by the body's immune system, and in type 2 diabetes, the body isn't able to use insulin the right way.Both types of diabetes incredibly increase a person's risk for a variety of severe complications (1,2).Their effects on retina may lead to the blindness of the patients.The effect of diabetes on the eye is called Diabetic Retinopathy.The prevalence of DR is 50% after 10 years and rising to 90% after 30 years of acquired diabetes.Up to 21% of patients with Type-2 diabetes have DR (3).Retinopathy is hard to be detected early on its formation and it can grow to be very serious overtime.By early detection, the follow-up effects of the illness including the blindness of the patient can be avoided.Diabetic retinopathy may progress through four stages: In the earliest stage of the DR tiny balloon-like swelling in the wall of small blood vessels, called microaneurysms, occur.These microaneurysms may break and allow blood to leak into nearby tissue.This stage is called Mild nonproliferative retinopathy (4).As the disease progresses, the nourishing blood vessels of the retina may swell and distort and may also lose their capability to carry blood.Both conditions may change the appearance of the retina and may contribute to DME.This stage is called Moderate nonproliferative retinopathy (5).In the Severe nonproliferative retinopathy stage, many more blood vessels are blocked, abdacting blood supply to areas of the retina.These areas spatter growth factors that promote the retina to grow new blood vessels (5).At the Proliferative diabetic retinopathy (PDR) stage, the proliferation of new blood vessels inside the surface of the retina is done, and grow along to the vitreous gel, the fluid that fills the eye (6).The new blood vessels are fragile, which makes them more likely to leak and bleed.Accompanying scar tissue pull away the retina from underlying tissue and cause retinal detachment.Retinal detachment can lead to constant vision loss.Typical normal fundus images, and different stages of DR is shown in Figure 1.

Signs and Symptoms
Clinical features of diabetic retinopathy include the following: Microaneurysms (MAs): The earliest clinical sign of DR that occur in the capillary wall outpouching due to pericyte loss; they appear as small(less than 125 μm in size) red dots with sharp margins, in the superficial retinal layers.
Hemorrhages (HEM): Can be located under, within, and over the retina and occur due to bleeding of weak capillaries, their size is greater than 125 μm, and they are defined as red/dark lesions and have irregular boundary.
Cotton-wool spots: They are caused by the ischemia of the nerve fibers from occlusion of precapillary arterioles and are a result of accumulations of axoplasmic debris within the nerve fiber layer.They emerge as fluffy white patches on the retina.
Exudates: are made up of extracellular lipid which has leaked from abnormal retinal capillaries and appears as waxy, yellow/bright deposits.
Neovascularization (NV): Is the natural formation of new blood vessels that form to serve as parallel circulation due to local poor perfusion or ischemia.These new vessels are fragile and frequently bleed into vitreous cavity, obscuring the vision.
Venous loops and venous beading: Frequently occur adjacent to areas of nonperfusion; they reflect increasing retinal ischemia, and their incidence is the most considerable prognosticator of progression to proliferative diabetic retinopathy (PDR).
The features by which the bright and dark lesions are categorized are shown in Figure 2.
Several authors have stated several segmentation methods to extract the anatomical regions and abnormal lesions from color retinal fundus image.They are considered briefly in the following sections.

REVIEW OF THE RELATED LITERATURE Optic Disk Localization and Segmentation
Optic nerve head or Optic disc (OD) is the point of exit for ganglion cell axons leaving the eye.Because there are no rods or cones overlying the optic disc, it corresponds to a small physiological blind spot in each eye.It is a bright circular area in the images taken from fundus of the eye.It is the origin and a source to extract the other properties such as fovea, a small depression in the retina of the eye where visual acuity is highest.In retinal images optic disk is applied to diagnose eye disease.In addition the OD is used to referring the other anatomical structures in retina such as fovea and macula as a reference.So the accurate localization and boundary extraction of OD is significant.Foracchia et al. introduced a new geometrical model to identify blood vessels orientation in order to localization of OD (7).Youssif et al. used the 2D Gaussian matched filter and some of morphological operations to investigate the structure of the retinal vessels to detect the OD (8).Mendel et al. were able to achieve a structure in order to segmentation of OD.They used morphological operations and active contours model (9).Walter et al. used area thresholding and watershed transform method to segment OD region (10).
Niemeijer et al. in a research investigated a method for detecting the fovea and the optic disk in the digital photographs.As they statement, there is a problem about the position of the structures in the image.They investigated 1100 digital colors fundus photographs in order to execute the screening program for the diabetic retinopathy.They investigated all the features of each image in order to detect the amount of fovea.They concluded that their method of finding the optic disk and fovea was effective and successful (11).Ahmed et al. proposed a marker controlled watershed transform based technique for OD segmentation.In an approximate location of the OD is estimated where the location of the OD is hypothesized by searching for regions of high intensity, diversity of gradient directions, and convergence of blood vessels (12).Sinthanayothin located the position of the OD by finding the region with the highest local variation in the intensity (13).Hoover utilized the geometric relationship between the OD and main blood vessels to identify the disc location.He described a method based on a fuzzy convergence voting mechanism to find the OD location (14).S.Sedai et al. introduce an automatic regression based method in order to segmentation of optic cup and optic disc in retinal color fundus images (15).Handayani et al. detected OD by applying Hough Transform and Active Contour Models.In this study the color fundus images from DRIVE database were used for assessment of their proposed methods (16).
Huiqi Li et al. introduce new method to detect main structures in color retinal images such as fovea and OD automatically.In this study an active shape model by principal component analysis, is used to obtain the disc boundary (17).
Esmaeili et al. take digital curve let transform of the enhanced retinal image and modified its coefficients based on the sparsity of curve let coefficients to get probable location of OD.If there were not yellowish objects in retinal images or their size are negligible, their proposed method directly detected OD location by performing Canny edge detector to reconstructed image with modified coefficients.Otherwise, if the size of these objects were eminent, the method extract circular regions in edge map as candidate regions for OD.In that case, they used some morphological operations to fill these circular regions and eroded them to get final locations for candidate regions and removed undesired pixels in edge map.As OD is surrounded by vessels, they choose the candidate region that has maximum summation of pixels in strongest edge map, which obtained by performing an appropriate threshold on the curvelet based enhanced image, as final location of OD.Finally, the boundary of the OD is extracted by using level set deformable model (18).Samanta et al. presented a Mathematical Morphology based method to find the fovea region.This algorithm were simple and fast by detection the structure of the blood vessels and extracted information is used to detect optic disc region (19).
Paintamilselvi et al. used Mathematical Morphology based method to detect the fovea in color retinal images from DRIVE database.They first detected and isolated retinal blood vessels from images and then they detected fovea region.Also the origin of OD is detected by adaptive mathematical morphology method (20).Amin Dehghani et al. inroduced a method to locate the center of OD in color fundus image from DRIVE, STARE and local datasets.In each color component they computed average of histogram to locate the OD (21).
Defeng Zhang et al. used deep learning method to detection the OD.In first stage they used Faster R-CNN+ZF algorithm to locate the OD by a bounding box (B-box) and then the main blood vessels in box by Hessian matrix are removed and at last stage a shape constrained level set algorithm is applied for segmentation the boundary of the OD.The algorithm was trained on kaggle database (4000 image) and test on MESSIDOR (120 image) (22).
Xianjing Meng et al. used Convolutional Neural Networks (CNNs) method to localize the OD.In data preparation, the blue channel in the color fundus image, is replaced by segmented vascular maps to reduce the noisy and inappropriate data.In the training stage, to solve the class imbalance problem, a large number of positive samples from the OD center were selected as models for next stage.After that a probability guided search algorithm is applied to enhance the efficiency of OD localization (23).
A summary of the contents above is presented in Table 1.

Segmentation of Exudates and CWS
Exudates are accumulations of lipid and protein in the retina and are one of the main clinical signs of the presence of diabetic retinopathy.It appears as white/yellow soft structures in the color of retina images (24).However, when they appear close to the center of macula, they are considered as a sight threatening lesions.The size of exudate is in a variety of levels.According to the size of these exudates, it can be state that their size is different from small to a large one.There are three types of exudates i.n patients' retina who suffer from retinopathy: hard exudates, soft white exudates and soft gray exudates.
Sopharak et al. in a research argued about the primary sign of diabetic retinopathy which is called "exudates".They examined an automatic method in order to recognize exudate from the low contrast digital images among these patients.They implemented a Fuzzy C-Mean (FCM) clustering based method for extraction of exudates.They developed preprocessing algorithms in order to contrast enhancement and used following four parameters as intensity, standard deviation on intensity, hue and a number of edge pixels (25).Clara et al. in a research argued about automatic image processing algorithm in order to recognize the hard exudates.According to their statements, this method is based on the Fisher's linear discriminant analysis and focuses on the color information.This method is also implemented the lesion-based performance evaluation (26).Usman Akram et al. in a study investigated about the methods which are able to detect exudates.The model which was presented in their study called: "Gaussian Mixture Model".The above model can combine with the m-Mediods modeling in order to describe the spatial relationship between the lesions and exudates in two retinal images.They also implemented the fuzzy neural network in order to detect dark and bright lesions (27).
Partovi et al. in a research argued that diagnosing the exudate regions in the retinal images in patients who suffer from the diabetic retinopathy is possible through especial methods.They investigated the variety of methods in order to comprehend the process of reacting the retina's faced with fundus images.According to the statements of these physicians, the regions of exudations are recognizable by using a camera which is able to take a picture with TIEF format.Also the function with a binary feature was implemented in order to enhance this procedure.The findings illustrated that this type of procedure is effective because of its high performance.In other words the recognition of exudate regions is achievable by means of this procedure (28).
Sopharak et al. investigated Mathematical morphology based method for exudate detection in color fundus images.In first stage after transforming the RGB space to HSI contrast limited adaptive histogram equalization (CLAHE) is used.Then a morphological operator is applied to clear the vessels and exudates were detected (29).
In the proposed method by esmaeili et al. the bright candidate lesions in the image are extracted by employing DCUT and modification of curvelet coefficients of enhanced retinal image.The authors applied a new bright lesions enhancement on green plane of retinal image to obtain adequate illumination normalization in the regions near the OD, and to increase brightness of lesions in dark areas such as fovea.Following this step, the authors introduced a new OD detection and boundary extraction method based on DCUT and level set method.Finally, bright lesions map (BLM) image was generated and to distinguish between exudates and OD, the extracted candidate pixels in BLM that are not in OD regions were considered as actual bright lesions (30).Fraz M et al. presented an ensemble classifier of boot strapped decision trees for localization and segmentation of exudates in color fundus images.After removing the artifact by using contextual information the candidate exudate regions was defined by using gabor filter, morphological reconstruction, top hat transformation.Finally the exudate regions were extracted by ensemble classifier of bootstrapped decision trees (31).
Gardner et al. introduced an automatic detection algorithm for DR they and used an artificial neural network as a screening tools in order to detect the excaudate in color fundus images (32).Niemeijer et al. in a research argued about automated detection of lesion in patient who suffer from DR.They are using Pixel classification which could Detects and differentiates exudates, CWS and drusen (33).Sanchez et al. in a research investigated an analysis method to detect hard exudate in retinal images.They used histogram modelling and dynamic thresholding which uses at least interval discriminant criteria, to detect hard exudates (34).Zhang et al. used random forest classifier and trained the manually detected exudate area, in color fundus images in order to classify bright and non-bright regions.In this study the normalization, image de-noising and mathematical morphology method was used (35).
Surya et al. applied novel method to detect exudates in color fundus image.They used orientation scores of retinal images in order to map the location and orientation angle of any pixel to form an orientation enhanced image (36).
Qing Liu et al. presented a procedure for automatic segmentation of exudate in color fundus images.This procedure consist of 3 stage: first, anatomic structures are eliminated, second, exudates are localized and then in the third stage exudates are segmented.Exudate regions are classified based on random forest classifier and then the local variance of exudate patches is used for localization of exudates (37).Kemal et al. introduced an automatic system for detection of hard exudates in color fundus images.After extracting the hard exudate location by DAISY algorithm, different methods and classifier were used to categorize the hard exudate on raw dataset.Eventually, after analyzing the data, random forest classifier is detected as the best classifier (38).Naqvi SA et al. presented an automated system for extraction of soft exudates as cotton wool spots (CWS).Their proposed method consist of support vector machine, visual dictionaries and scale invariant feature transform in order to discriminate CWS (39).Pereira et al. (40) in research present a new method based on ant colony optimization algorithm to detection of exudate in retinal color fundus images.The result of Comparison of this algorithm with Kirsch filter showed that the algorithm was applied in the study was better.
A summary of the contents above is presented in Table 2.

Segmentation of Microaneurysms and Hemorrhages
Proposed methods for automatic detection of microaneurysms (MAs) and hemorrhages (HMs) are based on angiography and color based photography.These techniques have their own drawbacks and deficiencies.The manual extraction of these lesions (especially in the case of angiography) is not an easy task; they are very complicated and can be performed only by highly skillful staffs and specialists.So, there is a remarkable necessity to utilize the computerbased system for automatic extraction of these red lesions.Before processing the image, the areas of the image that are of low quality should be put aside from the test, because the regions of poor quality may negatively interfere in diagnosing the disease.
Spencer et al. in a research studied about segmentation and quantification of microaneurysms in fluorescein angiograms.They used morphological methods and matched filters and found out that this strategy is valuable to provide a manner in track and monitoring of DR (41) (44).Streeter and Cree used Top-hat, matched filter and region growing algorithms in order to extract these lesions and their proposed method can recognize microaneurysms with bigger than ten pixels (45).Vijay M Mane et al. presented threshold based method for detection of microaneurysms and hemorrhages in color fundus images.But this study had a major limitation for detection of lesion in vessels segmentation stage (46).Amiri et al. presents Circular Hough Transform method for detecting MA in the retinal angiographic images.After preprocessing and identifying the center of MA, the pixels related with lesion are specified by region growing technique (47).Syed Ayaz et al. presented an automated method based on curvelet transform to detect the MA.After removing the blood vessels for identify the MA, a positional entropy thresholding manner was used (48).Ivo soares et al. introduce scale space based method to detect the MA by analyzing the neighborhood pixels.In this study after vessels segmentation, coarser and finer scales are used for analyzing the data.For reduce number of MA location a collection of Gaussian matched filters is applied (49).Álvarez Cervera et al. introduce an automatic system to detect MA in color fundus image.The methods used in this study were contrast limited adaptive histogram equalization (CLAHE) and Hessian matrix.After Appling the methods, the results were compared with the angiographic image (50).Lama Seoud et al. present a novel method based on shape features to detection of hemorrhages and microaneurysms in retinal color fundus images.The method used in this study called Dynamic Shape Features that validated by using different database (51).
Esmaeili et al. proposed a new curvelet based algorithm to separate red lesions from the rest of the color retinal image.In order to prevent fovea to be considered as red lesion, a new illumination equalization algorithm was proposed and applied to green plane of retinal image.Curvelet coefficients of retinal image was modified in order to lead dark objects to zero.These lesions were separated as candidate region by applying an appropriate threshold.Finally, the total structure of blood vessel is extracted based on curvelet transform and the false positives are eliminated by subtracting the vessel structure from the candidate images (52).Priyakshi et al. presented an automated computer aided method in order to detection of hemorrhages in retinal color fundus images.In this study, various methods such as Contrast Limited Adaptive Histogram Equalization (CLAHE), Median and Average Filtering, Morphological Operations, Region Growing and Thresholding have been used (53).Mumtaz et al. present a computer-aided diagnosis system to detect hemorrhages in color retinal fundus images.The research was carried out in three stages first, the contrast enhancement and normalization method was done to improve image quality then the blood vessels were extracted by using scale-based method and finally hemorrhages are segmented by gamma correction and thresholding method (54).
A summary of the contents above is presented in Table 3.

Segmentation of Vascular and Neovascularization
Retinopathy can be detected by many ways.So many image processing techniques have been developed in the last decades to provide information required by Ophthalmologist in early detection of the disease.Providing a manual pattern of the retinal blood vessels can take a lot of time.Computerized segmentation of blood vessels reduces the amount of analyzing time that the large volume of images precludes manual analysis.(55).Zhou et al. presented matched filtering based algorithm for detection of retinal vessel borders, and specified the midline of the vessels and proposed indices which may be beneficial to the eye clinics (56).Nayak et al. proposed a morphological opening and OTSU's approach.The ridge detection was used to form line elements and partition the image into patches belonging to each line element.Then pixel features were generated based on this representation to detect blood vessels (57).The scrutinizing and diagnosing the blood vessels can reveal the changes caused by ocular disease.In the areas that vessels are not thick enough, the vessel patterns are ambiguous and hardly visible.In order to solve this problem, we have to improve the pattern of the vessels by using an algorithm that can probe the retinal vessels by matched filter response (MFR) image Hoover et al. (58).Usman Akram et al. used a multi-layered threshold method in order to determine the optimal threshold value for precisely segmenting the vessels.In this method various threshold rates are applied and then their effect on the vessels in the layers is examined.At early stages of segmenting, the threshold value T is chosen, then the histogram of the wavelet image is applied, it just maintains the pixels at the initially segmented image that wavelet coefficients for them is greater than T value.Then the segmented image is depicted by applying thinning morphological operator.Consequently, all of the vessels will have the single pixel width.The edges of the vessels are measured and the edges that are not considered to be true edges are omitted.The threshold was decreased by repeating the process.The process was continued by decreasing the threshold till no considerable change is observed in two consecutive repetition of the process (59).
Esmaeili et al. proposed curvele transform based method for detection of low contrast and narrow retinal vasculature in color retinal images.Their proposed method consist of four steps: curvelet-based contrast enhancement, match filtering, curvelet-based edge extraction, and length filtering (60).Kulwinder et al. presented a novel method based on neural networks to segmentation of blood vessel in color fundus images.The method has been tested on several wellknown databases and the results were compared with Human Observer that were very close to each other (61).
Jadhav et al. presented a method to trace the blood vessels in retinal color fundus images.In this study after using enhancement operation in order to remove the noise from images, the morphological operation was applied to extract blood vessels (62).Chengzhang et al. presented a supervised method based on Extreme Learning Machine (ELM) in order to segmentation of blood vessels.In this research, various methods such as, a set of 39-D discriminative feature vectors, They represented the combination of such method in order to achieve to an optimum method to detect neovascularization.According to their statements their methods were tested on the images from different database sources (64).Gupta et al. presented a novel procedure to detecting the neovascularization in color fundus images.First the patches on image is defined then trained model of patch neovascularity is applied.In the next stage some filter and algorithm such as Gabor filter, and Hessian matrix were used to extract retinal vasculature (65).Sudeshna et al. presented an automated for detection of NVE in retinal color fundus images.They used curvelet transform to enhance the tiny blood vessels then they applied maximization of mutual information (MI) on the maximum matched filter response for finding of optimal thresholding to classification of the vessels into the thick and thin class.Vessel density and tortuosity are computed from the thin vessel class followed by MI maximization and post-processing for neovascularization detection (66).Huang et al. presented an automated system for detection NVE by using extreme learning machine.Also they use series of filter bank to obtain the property NVE in retinal color fundus images (67).Akram et al. presented d a new method for detection of neovascularization using multivariate m-Mediods based classifier in retinal color fundus images.Also this method can detect the PDR and classify it in NVE and NVD (68).A summary of the contents above is presented in Table 4.

CONCLUSION AND DISCUSSION
Image processing techniques for detection of retinopathy have been developed in the last decades to provide information required by Ophthalmologist for early detection of the disease.Providing a manual pattern of the retinal abnormalities can take a lot of time.Computerized segmentation reduces the amount of time that corresponding authorities spent on this process.The retinopathy in diabetic patients may leads to the blindness but it can be prevented by the earlier diagnosis.This paper succinctly presents recent progress in automated detection of abnormal signs of diabetic retinopathy and can help researchers interested in this field in order to develop more efficient algorithms.

Figure 2 :
Figure 2: Classification Properties for different abnormal lesions

Table 1 :
Summary of available methods for segmentation and localization of OD http://www.ejgm.co.uk 5 / 13

Table 2 :
(43)ame et al. in a research studied about classification methods in Summary of available methods for Segmentation of exudates and CWS to detect microaneurysms in fluorescein angiograms.According to their statements MAs can be detected by using matched filters, region growing, and Rule based method in a grayscale image(42).Rasta et al. presented a different method for detection of ischemic retina, non-perfused area, in fundus fluorescein angiogram images.They used homomorphic filtering technique to correct light illumination and detect the ischemic area surrounded in healthy capillaries(43).Fleming et al. used Multi-scale and morphological technique to detect blot hemorrhages (BH), after segmentation of BH is done based on SVM classification http://www.ejgm.co.uk 7 / 13 order

Table 3 :
Summary of available methods for Segmentation of microaneurysms and hemorrhages (63)://www.ejgm.co.uk 9 / 13 morphological features, phase congruency, Hessian and divergence of vector fields and ELM classifier were used(63).Syafinah et al. in a research argued about different types of method which were able to detect the diabetic retinopathy.

Table 4 :
Summary of available methods for Segmentation of Vascular and Neovascularization