CN109472781B - Diabetic retinopathy detection system based on serial structure segmentation - Google Patents

Abstract

The invention discloses a diabetic retinopathy detection system based on serial structure segmentation, which comprises: the collection obtains the fundus image acquisition device of retina fundus image and the data processing device of the fundus image of analysis processing collection, and data processing device includes: the system comprises a preprocessing function module, a blood vessel segmentation function module, a optic disc segmentation function module, a fovea centralis determination function module, an exudation segmentation function module, a statistical calculation function module and a doctor diagnosis function module, wherein the exudation region is counted, the probability of diabetic macular edema lesion existing in an input eye fundus image is calculated, and finally, a final diagnosis and treatment scheme is given by combining the statistical calculation result and the eye fundus doctor by referring to the segmented exudation region and the diseased probability and combining self specialty. The invention systematically considers various related physiological structures of the eyeground, divides the pathological change area and gives a diagnosis report by an eyeground doctor, has high detection efficiency and more accurate pathological change detection, can greatly reduce the workload of the doctor and improve the working efficiency.

Description

Diabetic retinopathy detection system based on serial structure segmentation

Technical Field

The invention belongs to the field of machine vision and medical image computer processing, and particularly relates to a diabetic retinopathy detection system based on serial structure segmentation.

Background

Diabetic retinopathy is one of the four major blinding diseases worldwide. The International Diabetes Federation (IDF) in 2017 published an eighth version of the global diabetes map, which, based on IDF data, shows that adult patients with global diabetes have reached 4.25 billion, and are predicted to reach 6.29 billion by 2045. China is the country with the most diabetes patients worldwide, and the number of patients and the incidence of diseases are obviously increased in recent years. Of the population, 35% of the diabetic population suffer from diabetic retinopathy. According to the different clinical pathological features, the diabetic retinopathy (sugar network) is mainly divided into two categories, namely non-proliferative retinopathy (NPDR) and proliferative retinopathy (PDR). In the non-proliferative stage of retinopathy, Diabetic Macular Edema (DME), which is the first cause of vision loss, may occur.

DME can be classified into three classes depending on the presence or absence of hard bleed and the location of presence. A first stage: there was no macular edema lesion, hard exudation did not occur, there was no thickening of the retina, and there were no obvious symptoms in the patient, so the lesion was hard to be found. And a second stage: there was no significant clinical macular edema pathology, when there was a hard exudation or thickening of the retina, but the exudation was more than 500um from the fovea. And a third stage: the clinical macular edema lesion is remarkable, the retina is thickened at the moment, the hard exudation is close to the fovea region, the distance is less than 500 mu m, the vision of a patient is seriously affected, and the treatment should be immediately adopted. The eye examination by shooting the eye fundus retina picture can discover the pathological changes as early as possible, start the treatment in time and effectively avoid the deterioration of the disease. Therefore, early detection, diagnosis and treatment of diabetic macular edema lesion are of great significance to prevent blindness. However, at present, early detection of diabetic macular edema disease still relies on manual analysis, requiring a large number of highly skilled professional fundus physicians to process and view the captured retinal fundus pictures. These professionals rely on human vision to interpret fundus images for lesion diagnosis, often with some problems. For example: misdiagnosis may occur due to the subjective understanding and lack of experience of the doctor, and early lesions are difficult to detect visually; the large number of fundus photographs requires a large expenditure of human resources and inexperienced ophthalmologists in urban and rural areas, so an automated computer processing system is needed to assist ophthalmologists in performing the analysis of these photographs. The retina image is shot by the fundus camera, the related structures of the fundus and the reference probability of pathological changes are divided by the technology of machine vision and the like, the fundus is referred and further observed by a fundus doctor, and the pathological change grade is finally given by the fundus doctor, so that the workload of the doctor is reduced, the misdiagnosis rate of the doctor is reduced, and the working efficiency of the doctor is improved.

However, the use of image processing techniques to detect DME lesions is still in a stage of development. Philips et al achieve hard bleed segmentation by global and local thresholds; walter and Sopharak dissected the exudation using mathematical morphology. Osareh et al achieve accurate segmentation of exudation through fuzzy C-means clustering and neural networks. In addition, multi-layer perceptrons, radial basis functions, support vector machines, and the like are also used to implement exudation detection. Kumar et al propose a bit plane-based method to realize the detection of diabetic macular edema; sekhar et al first segment the lesion area by contrast enhancement and threshold segmentation, then extract and classify the features of the lesion area by a classifier based on a Gaussian mixture model, and give the prevalence probability of diabetic macular edema lesion in the image processing level. Meanwhile, other methods such as KNN, naive Bayes, deep learning and the like are also used in the detection system of the diabetic macular edema lesion. However, the prior art is directed to single target to perform individual detection, such as exudative lesion segmentation, optic disc segmentation, macular segmentation, etc., and the relationship between the two is rarely considered; the system of the invention fully considers the positions, characteristics and the like of the structures and the pathological changes in the fundus image, adopts a serial structure to sequentially segment, and finally obtains the structures and the pathological change areas which can assist doctors to observe better.

Disclosure of Invention

The invention aims to: the problem that the relation between the current detection system and other structures such as optic discs, yellow spots, blood vessels and the like, such as positions, characteristics and the like in fundus images, and accurate and efficient detection of the data of the diabetic retinopathy region cannot be easily realized due to the fact that the current detection system for detecting the diabetic macular edema lesion only detects hard exudation singly is solved, and the diabetic retinopathy detection system based on serial structure segmentation is provided to realize systematic and comprehensive diabetic retinopathy detection.

The technical scheme adopted by the invention is as follows:

a serial structure segmentation based diabetic retinopathy detection system, the system comprising:

fundus image acquisition device: the device is used for acquiring and acquiring a fundus image of the retina;

a data processing device: the data processing device is used for analyzing and processing the acquired fundus images to obtain lesion region data and lesion probability, and comprises:

a preprocessing function module: preprocessing the acquired fundus image;

a blood vessel segmentation functional module: performing blood vessel segmentation based on morphology on the preprocessed fundus image;

the optic disc segmentation function module: performing optic disc segmentation on the preprocessed fundus image by combining the processing result of the blood vessel segmentation functional module;

foveal determination function: determining the fovea of the preprocessed image by combining the processing result of the optic disc segmentation functional module;

and a seepage segmentation functional module: combining the processing result of the optic disc segmentation functional module, performing exudation segmentation on the preprocessed image to obtain an exudation area;

a statistical calculation functional module: and combining the processing results of the fovea centralis determination function module and the exudation segmentation function module, counting exudation areas and calculating the probability of the diabetic macular edema lesion existing in the input fundus image.

Further, the data processing device further comprises a doctor diagnosis function module: and (4) combining the processing result of the statistical calculation functional module, and giving a final diagnosis and treatment scheme by referring to the segmented exudation area and the disease probability by the fundus doctor and combining the specialty of the fundus doctor.

Further, the preprocessing operation steps of the preprocessing function module are specifically:

step 1.1, RGB three-channel extraction is carried out on the input fundus image G, median filtering processing is carried out on each channel, and then three channels are synthesized into a single color image G_new；

Step 1.2, for G_newContrast-limited adaptive histogram equalization is performed to enhance image contrast, and fundus enhanced image G is obtained_clahe。

Further, the blood vessel segmentation operation steps of the blood vessel segmentation functional module are specifically as follows:

step 2.1, firstly extracting a fundus enhanced image G obtained by preprocessing the fundus image by the preprocessing module_claheFiltering the image by using median filtering to obtain a background estimation image G_backgroundAnd estimating an image G of the background by using a disc structural element with a certain radius_backgroundRespectively carrying out top cap and bottom cap operations to obtain G_topAnd G_black；

Step 2.2, G in the step 2.1_topAnd G_blackAdding the weighted image to the original image G and subtracting the background estimation image G_backgroundAnd performing threshold segmentation to obtain a blood vessel binary estimation map G_BL；

Step 2.3, extracting the blood vessel binary estimation map G in the step 2.2_BLThe connected region with the middle connected region larger than a certain area is obtained to obtain a fundus blood vessel binary image G corresponding to the fundus image_BV。

Further, the optic disc segmentation operation steps of the optic disc segmentation functional module are specifically as follows:

step 3.1, fundus enhancement image G after the pretreatment of fundus image_claheA two-channel threshold value division is carried out,obtaining a binary image G_RG；

Step 3.2, comparing the binary image G in the step 3.1_RGAnalyzing the connected domains, extracting the minimum rectangular frame of each connected domain, and expanding the minimum rectangular frame in the left and right directions to obtain an image G after the connected domains are expanded_expand；

Step 3.3, counting the image G after the connected domain expansion in the step 3.2_expandThe number of pixels on two sides of the vertical center line of the image in (1);

step 3.4, selecting corresponding Toeplitz matrix templates based on the number of pixels on two sides of the vertical center line of the fundus image in the step 3.3, and selecting a right Toeplitz matrix template Mask when more than half of the counted pixels are positioned on the right side of the image, as shown in the formula (1):

otherwise, selecting a left Toeplitz matrix template Mask as shown in the formula (2):

and 3.5, performing AND operation on the fundus blood vessel image segmented in the blood vessel segmentation functional module and the expanded connected region in the step 3.2 to obtain a blood vessel image G of the candidate region_candidate：

G_candidate＝G_BV∩G_{exp and} (3)

Step 3.6, blood vessel image G of candidate region_candidateFiltering to obtain filtered image G_filterThe formula is as follows:

step 3.7, determine filtered image G_filterThe pixel with the highest middle gray scale value is the coordinate of the fundus optic disc positioning position;

step 3.8, with the optic disc positioning position coordinate as the center, framing a rectangular area with a certain size in the input fundus image G as the optic disc candidate area G_OD；

Step 3.9, extracting R channel of the video candidate area in step 3.8, and recording as image G'_R；

Step 3.10, carrying out alternate expansion corrosion operation by using the structure elements with the increasing radiuses to remove blood vessels in the candidate region of the video disc, firstly selecting a disc structure element B with a certain radius, and carrying out image G'_RCarrying out alternate expansion corrosion operation:

σ^(B)＝δ^(B)(ε^(B)(G′_R)) (5)

wherein, delta^(B)The expansion operation is carried out by taking B as a structural element; epsilon^(B)Representing that B is used as a structural element for corrosion operation;

step 3.11, increasing the radius of the disc structure element B to obtain a new disc structure element B', and performing alternate expansion corrosion operation on the image processed in the step 3.10 again:

σ^(B)＝δ^(B′)(ε^(B′)(σ^(B))) (6)

and 3.12, continuously increasing the radius of the disc structural element B 'to obtain a new disc structural element B', and performing the alternate dilation corrosion operation on the image processed in the step 3.11 again to remove blood vessels in the candidate region of the optic disc:

σ^(B″)＝δ^(B″)(ε^(B″)(σ^(B′))) (7)

step 3.13, performing threshold segmentation on the optic disc candidate region with blood vessels removed by adopting the maximum inter-class variance method to obtain a binary image G_OtsuAnd extracting the binary image G by using a Canny operator_OtsuTo obtain an edge image G_edge；

Step 3.14, extracting edge image G_edgePerforming least square ellipse fitting on the contour coordinates;

and 3.15, drawing the elliptic equation curve in the step 3.14 in the fundus image G, namely the optic disc elliptic ROI area.

Further, the least squares ellipse fitting in step 3.14 specifically comprises the following steps:

assuming the ellipse equation is: ax²+bxy+cy²+ dx + ey 1, the optimization problem of least squares ellipse fitting can be expressed as:

min||Dα||²

s.t.α^TCα＝1 (8)

wherein α ═ a, b, c, d, e ]; d represents a contour coordinate information set, the dimension is n multiplied by 6, and n is the number of contour pixels; matrix C is as follows:

further, the foveal determination operation step of the foveal determination function module specifically includes:

step 4.1.1, obtaining the macula lutea ROI area G by using the optic disc position and the optic disc size obtained by the optic disc segmentation functional module_m-ROI；

Step 4.1.2: extraction of the macular ROI region G in step 4.1.1_m-ROIGreen channel G of_m-GAnd is subjected to adaptive contrast enhancement to obtain G'_m-G(ii) enhanced image G'_m-GAnd G_m-ROIPerforming subtraction to obtain a decorrelated image G_m-decor；

Step 4.1.3: for the decorrelated image G obtained in step 4.1.2_m-decorPerforming adaptive threshold segmentation to obtain G_m-otsuThen performing a morphological etching operation to obtain G_m-erodeThen take G_m-erodeThe region of the largest connected domain area in (A) is the central concave ROIG_fovea-ROI；

Step 4.1.4: calculating the foveal ROIG_fovea-ROICenter of mass (x)_fovea，y_fovea) It is approximated as a foveal position.

Further, the exudation segmentation operation steps of the exudation segmentation function module are specifically as follows:

step 4.2.1: function module to be preprocessedFundus enhanced image G obtained by preprocessing fundus image_claheCarrying out mean value filtering to obtain a background estimation image G_clahe-meanAnd performing morphological reconstruction to obtain G_morp；

Step 4.2.2: the morphological reconstruction graph G obtained in the step 4.2.1_morpAnd background estimation map G_clahe-meanDifferencing to obtain a normalized image G_ex-normAnd obtaining a bleeding candidate region I by using an adaptive threshold: g_ex-cand1；

Step 4.2.3: the fundus enhanced image G obtained by preprocessing the fundus image by the preprocessing function module_claheCalculating local variance to obtain boundary graph G of bright area_clahe-stdFor the boundary graph G_clahe-stdPerforming an opening operation and performing threshold segmentation to obtain G_std-otsu；

Step 4.2.4: for G in step 4.2.3_std-otsuSequentially performing expansion operation, closing operation and hole filling to obtain a seepage candidate area II: g_ex-cand2；

Step 4.2.5: g in step 4.2.2_ex-cand1And the oozing candidate region G in step 4.2.4_ex-cand2Get a logical OR operation to get G_cand-orG is_cand-orAnd fundus enhanced image G_claheMorphological reconstruction to obtain G_clahe-or；

Step 4.2.6: using the disc area G obtained in step 3.8_ODDelete G_clahe-orIn the candidate region in the region, the final exudation candidate region G is obtained_ex-cand。

Further, the operation steps of the statistical computation function module are specifically:

step 5.1: to obtain a foveated pixel (x)_fovea，y_fovea) As the circle center, different radiuses are given by the eyeground doctor according to the corresponding standard self-definition, and concentric circles are divided, wherein 3 radiuses are taken as examples and are marked as R from inside to outside in sequence₁、R₂、R₃；

Step 5.2: note R₁And R₂The ring in between is C₁，R₂And R₃The ring in between is C₂Counting the oozing region G obtained by oozing the segmentation function module_ex-candAt R₁、C₁、C₂The number of pixels in the input retinal fundus image, thereby calculating the probability of diabetic macular edema lesion of the input retinal fundus image.

In summary, due to the adoption of the technical scheme, the invention has the beneficial effects that:

in the invention, a retina fundus image is collected by a fundus image collecting device and is input into a data processing module; performing blood vessel segmentation and exudation segmentation on an input image; carrying out template filtering on the blood vessel segmentation graph to obtain the optic disc position, and segmenting the optic disc by ellipse fitting; determining a macular region of interest (ROI) according to prior physiological information between a optic disc and macula lutea, calculating the coordinate of a fovea centralis of the macula lutea, dividing concentric circles with different radiuses by taking the fovea centralis as a center, counting the number of exudations in each region and giving out the probability of diabetic macular edema Disease (DME), fully considering the structures of retinal exudation lesion, optic disc, macula lutea and the like and the position, characteristics and the like of lesions in fundus images, accurately dividing the fundus related structure and lesion region by combining the structures in multiple aspects, simultaneously counting the lesion region area and lesion probability, giving quantitative value reference to a certain extent, detecting high efficiency and detecting lesions more accurately, thereby providing an effective and reliable auxiliary means for doctors in practical clinical application, and reducing misdiagnosis rate by the doctor by referring to the value and then deciding the final lesion level through the system, and greatly reduces the workload of doctors and improves the working efficiency.

Drawings

In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings needed to be used in the embodiments will be briefly described below, it should be understood that the following drawings only illustrate some embodiments of the present invention and therefore should not be considered as limiting the scope, and for those skilled in the art, other related drawings can be obtained according to the drawings without inventive efforts.

FIG. 1 is a schematic flow chart of a serial segmentation-based diabetic retinopathy detection system according to the present invention;

FIG. 2 is a diagram of an input image and a vessel segmentation result according to the present invention;

FIG. 3 is a disc segmentation result chart and a macular segmentation result chart according to the present invention;

FIG. 4 is a graph of the oozing segmentation result and a concentric circle segment according to the present invention.

Detailed Description

In order to make the objects, technical solutions and advantages of the present invention more apparent, the present invention is described in further detail below with reference to the accompanying drawings and embodiments. It should be understood that the detailed description and specific examples, while indicating the preferred embodiment of the invention, are intended for purposes of illustration only and are not intended to limit the scope of the invention. The components of embodiments of the present invention generally described and illustrated in the figures herein may be arranged and designed in a wide variety of different configurations.

Thus, the following detailed description of the embodiments of the present invention, presented in the figures, is not intended to limit the scope of the invention, as claimed, but is merely representative of selected embodiments of the invention. All other embodiments, which can be derived by a person skilled in the art from the embodiments of the present invention without making any creative effort, shall fall within the protection scope of the present invention.

It is noted that relational terms such as "first" and "second," and the like, may be used solely to distinguish one entity or action from another entity or action without necessarily requiring or implying any actual such relationship or order between such entities or actions. Also, the terms "comprises," "comprising," or any other variation thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements does not include only those elements but may include other elements not expressly listed or inherent to such process, method, article, or apparatus. Without further limitation, an element defined by the phrase "comprising an … …" does not exclude the presence of other identical elements in a process, method, article, or apparatus that comprises the element.

A serial structure segmentation based diabetic retinopathy detection system, the system comprising:

fundus image acquisition device: the device is used for acquiring and acquiring a fundus image of the retina;

a data processing device: the data processing device is used for analyzing and processing the acquired fundus images to obtain lesion region data and lesion probability, and comprises:

a preprocessing function module: preprocessing the acquired fundus image;

a blood vessel segmentation functional module: performing blood vessel segmentation based on morphology on the preprocessed fundus image;

the optic disc segmentation function module: and (3) performing optic disc segmentation on the preprocessed fundus image by combining the processing result of the blood vessel segmentation functional module:

foveal determination function: determining the fovea of the preprocessed image by combining the processing result of the optic disc segmentation functional module;

and a seepage segmentation functional module: combining the processing result of the optic disc segmentation functional module, performing exudation segmentation on the preprocessed image to obtain an exudation area;

a statistical calculation functional module: and combining the processing results of the fovea centralis determination function module and the exudation segmentation function module, counting exudation areas and calculating the probability of the diabetic macular edema lesion existing in the input fundus image.

Further, the data processing device further comprises a doctor diagnosis function module: and (4) combining the processing result of the statistical calculation functional module, and giving a final diagnosis and treatment scheme by referring to the segmented exudation area and the disease probability by the fundus doctor and combining the specialty of the fundus doctor.

The fundus image acquisition device can adopt various fundus cameras or other photographing devices capable of acquiring fundus images to acquire the fundus images, and the data processing device is realized by adopting a computer or a cloud server.

Further, the preprocessing operation steps of the preprocessing function module are specifically:

step 1.1, RGB three-channel extraction is carried out on the input fundus image G, median filtering processing is carried out on each channel, and then three channels are synthesized into a single color image G_new；

Step 1.2, for G_newContrast-limited adaptive histogram equalization (CLAHE) is performed to enhance image contrast, and a fundus enhanced image G is obtained_clahe。

Further, the blood vessel segmentation operation steps of the blood vessel segmentation functional module are specifically as follows:

step 2.1, extract the fundus enhanced image G in step 1.2 first_claheFiltering the image by using median filtering to obtain a background estimation image G_backgroundAnd estimating an image G of the background by using a disc structural element with a certain radius_backgroundRespectively carrying out top cap and bottom cap operations to obtain G_topAnd G_black；

Step 2.2, G in the step 2.1_topAnd G_blackAdding the weighted image to the original image G and subtracting the background estimation image G_backgroundAnd performing threshold segmentation to obtain a blood vessel binary estimation map G_BL；

Step 2.3, extracting the blood vessel binary estimation map G in the step 2.2_BLThe connected region with the middle connected region larger than a certain area is obtained to obtain a fundus blood vessel binary image G corresponding to the fundus image_BV。

Further, the optic disc segmentation operation steps of the optic disc segmentation functional module are specifically as follows:

step 3.1, enhancing the image G of the fundus oculi in step 1.2_clahePerforming dual-channel (R channel and G channel) threshold segmentation to obtain binary image G_RG；

Step 3.2, comparing the binary image G in the step 3.1_RGAnalyzing the connected domains, extracting the minimum rectangular frame of each connected domain, and expanding the minimum rectangular frame in the left and right directions to obtain an image G after the connected domains are expanded_expand；

Step 3.3, counting the image G after the connected domain expansion in the step 3.2_expandIn (1)The number of pixels on both sides of the vertical center line of the image;

step 3.4, selecting corresponding Toeplitz matrix templates based on the number of pixels on two sides of the vertical center line of the fundus image in the step 3.3, and selecting a right Toeplitz matrix template Mask when more than half of the counted pixels are positioned on the right side of the image, as shown in the formula (1):

otherwise, selecting a left Toeplitz matrix template Mask as shown in the formula (2):

and 3.5, performing AND operation on the fundus blood vessel image segmented in the step 2.3 and the expanded connected region in the step 3.2 to obtain a blood vessel image G of the candidate region_candidate：

G_candidate＝G_BV∩G_{exp and} (3)

Step 3.6, blood vessel image G of candidate region_candidateFiltering to obtain filtered image G_filterThe formula is as follows:

step 3.7, determine filtered image G_filterThe pixel with the highest middle gray scale value is the coordinate of the fundus optic disc positioning position;

step 3.8, with the optic disc positioning position coordinate as the center, framing a rectangular area with a certain size in the input fundus image G as the optic disc candidate area G_OD；

Step 3.9, extracting R channel of the video candidate area in step 3.8, and recording as image G'_R；

Step 3.10, removing by alternate expansion corrosion operation with structure elements with increasing radiusAccording to the blood vessel in the candidate area of the disk, a disk structure element B with a certain radius is selected first, and the image G 'is subjected to'_RCarrying out alternate expansion corrosion operation:

σ^(B)＝δ^(B)(ε^(B)(G′_R)) (5)

wherein, delta^(B)The expansion operation is carried out by taking B as a structural element; epsilon^(B)Representing that B is used as a structural element for corrosion operation;

step 3.11, increasing the radius of the disc structure element B to obtain a new disc structure element B', and performing alternate expansion corrosion operation on the image processed in the step 3.10 again:

σ^(B′)＝δ^(B′)(ε^(B′)(σ^(B))) (6)

and 3.12, continuously increasing the radius of the disc structural element B 'to obtain a new disc structural element B', and performing the alternate dilation corrosion operation on the image processed in the step 3.11 again to remove blood vessels in the candidate region of the optic disc:

σ^(B″)＝δ^(B″)(ε^(B″)(σ^(B))) (7)

step 3.13, performing threshold segmentation on the optic disc candidate region with blood vessels removed by adopting the maximum inter-class variance method to obtain a binary image G_OtsuAnd extracting the binary image G by using a Canny operator_OtsuTo obtain an edge image G_edge；

Step 3.14, extracting edge image G_edgePerforming least square ellipse fitting on the contour coordinates;

and 3.15, drawing the elliptic equation curve in the step 3.14 in the fundus image G, namely the optic disc elliptic ROI area.

Further, the least squares ellipse fitting in step 3.14 specifically comprises the following steps:

assuming the ellipse equation is: ax²+bxy+cy²+ dx + ey 1, the optimization problem of least squares ellipse fitting can be expressed as:

min||Dα||²

s.t.α^TCα＝1 (8)

wherein α ═ a, b, c, d, e ]; d represents a contour coordinate information set, the dimension is n multiplied by 6, and n is the number of contour pixels; matrix C is as follows:

further, the foveal determination operation step of the foveal determination function module specifically includes:

step 4.1.1, obtaining the macula lutea ROI area G by using the optic disc position and the optic disc size obtained in the step 3_m-ROI；

Step 4.1.2: extraction of the macular ROI region G in step 4.1.1_m-ROIGreen channel G of_m-GAnd is subjected to adaptive contrast enhancement to obtain G'_m-G(ii) enhanced image G'_m-GAnd G_m-ROIPerforming subtraction to obtain a decorrelated image G_m-decor；

Step 4.1.3: for the decorrelated image G obtained in step 4.1.2_m-decorPerforming adaptive threshold segmentation to obtain G_m-otsuThen performing a morphological etching operation to obtain G_m-erodeThen take G_m-erodeThe region of the largest connected domain area in (A) is the central concave ROIG_fovea-ROI；

Step 4.1.4: calculating the foveal ROIG_fovea-ROICenter of mass (x)_fovea，y_fovea) It is approximated as a foveal position.

Further, the exudation segmentation operation steps of the exudation segmentation function module are specifically as follows:

step 4.2.1: for the fundus enhancement image G in step 1.2_claheCarrying out mean value filtering to obtain a background estimation image G_clahe-meanAnd performing morphological reconstruction to obtain G_morp；

Step 4.2.2: the morphological reconstruction graph G obtained in the step 4.2.1_morpAnd background estimation map G_clahe-meanDifferencing to obtain a normalized image G_ex-normAnd obtaining a bleeding candidate region I by using an adaptive threshold: g_ex-cand1；

Step 4.2.3: for the fundus enhancement image G in step 1.2_claheCalculating local variance to obtain boundary graph G of bright area_clahe-stdFor the boundary graph G_clahe-stdPerforming an opening operation and performing threshold segmentation to obtain G_std-otsu；

Step 4.2.4: for G in step 4.2.3_std-otsuSequentially performing expansion operation, closing operation and hole filling to obtain a seepage candidate area II: g_ex-cand2；

Step 4.2.5: g in step 4.2.2_ex-cand1And the oozing candidate region G in step 4.2.4_ex-cand2Get a logical OR operation to get G_cand-orG is_cand-orAnd image G in step 1_claheMorphological reconstruction to obtain G_clahe-or；

Step 4.2.6: using the disc area G obtained in step 3.8_ODDelete G_clahe-orIn the candidate region in the region, the final exudation candidate region G is obtained_ex-cand。

Further, the operation steps of the statistical computation function module are specifically:

step 5.1: with the foveated pixel (x) in step 4.1.4_fovea，y_fovea) As the circle center, different radiuses are given by the eyeground doctor according to the corresponding standard self-definition, and concentric circles are divided, wherein 3 radiuses are taken as examples and are marked as R from inside to outside in sequence₁、R₂、R₃；

Step 5.2: note R₁And R₂The ring in between is C₁，R₂And R₃The ring in between is C₂The oozing region G obtained in step 4.2.6 is counted_ex-candAt R₁、C₁、C₂The number of pixels in the input retinal fundus image, thereby calculating the probability of diabetic macular edema lesion of the input retinal fundus image.

The features and properties of the present invention are described in further detail below with reference to examples.

Example 1

A preferred embodiment of the present invention provides a serial structure segmentation based diabetic retinopathy detection system, a flowchart of which is shown in fig. 1, and the system includes:

an eye fundus camera: the device is used for acquiring and acquiring fundus images of the retina and inputting the fundus images into the data processing device;

a data processing device: the data processing device is used for analyzing and processing the acquired fundus images to obtain lesion region data and lesion probability, and comprises:

a preprocessing function module: preprocessing the acquired fundus image;

a blood vessel segmentation functional module: performing blood vessel segmentation based on morphology on the preprocessed fundus image;

the optic disc segmentation function module: performing optic disc segmentation on the preprocessed fundus image by combining the processing result of the blood vessel segmentation module;

foveal determination function: determining the fovea of the preprocessed image by combining the processing result of the optic disc segmentation module;

and a seepage segmentation functional module: combining the processing result of the optic disc segmentation module, performing exudation segmentation on the preprocessed image to obtain an exudation area;

a statistical calculation functional module: and combining the processing results of the fovea centralis determination module and the exudation segmentation module, counting exudation areas and calculating the probability of the diabetic macular edema lesion existing in the input fundus image.

The doctor diagnosis function module: and (4) combining the processing result of the statistical calculation module, and giving a final diagnosis and treatment scheme by referring to the segmented exudation area and the disease probability by the fundus doctor and combining the specialty of the fundus doctor.

The data processing device adopts a computer to process and analyze data, and each functional module and the operation steps of the data processing device can be completed by relevant hardware instructed by a program, and the program can be stored in a computer-readable storage medium.

Further, the preprocessing operation steps of the preprocessing function module are specifically:

step 1.1, RGB three-channel extraction is carried out on the input fundus image G, andcarrying out median filtering processing on each channel, and synthesizing the three channels into a single color image G_new. In this embodiment, the input image size is 1552 × 1928, and the median filter size is 3 × 3.

Step 1.2, for G_newContrast-limited adaptive histogram equalization (CLAHE) is performed to enhance image contrast, and a fundus enhanced image G is obtained_clahe. In this example, the CLAHE parameter contrast enhancement is limited to 0.01 and the histogram is 256 bins.

Further, the blood vessel segmentation operation steps of the blood vessel segmentation functional module are specifically as follows:

step 2.1, extract the fundus enhanced image G in step 1.2 first_claheFiltering the image by using median filtering to obtain a background estimation image G_backgroundAnd estimating an image G of the background by using a disc structural element with a certain radius_backgroundRespectively carrying out top cap and bottom cap operations to obtain G_topAnd G_black. In this embodiment, the median filter size is 40 × 40, the radius of the top-hat disk structure element is 20, and the radius of the bottom-hat disk structure element is 6.

Step 2.2, G in the step 2.1_topAnd G_blackAdding the weighted image to the original image G and subtracting the background estimation image G_backgroundAnd performing threshold segmentation to obtain a blood vessel binary estimation map G_BL。

In this example, G_background＝0.8*G_top-0.8*G_black+G。

Step 2.3, extracting the blood vessel binary estimation map G in the step 2.2_BLThe connected region with the middle connected region larger than a certain area is obtained to obtain a fundus blood vessel binary image G corresponding to the fundus image_BV. As shown in fig. 2, (a) is an input image, and (b) is a blood vessel segmentation result, in the present embodiment, the connected component threshold is 350.

Further, the optic disc segmentation operation steps of the optic disc segmentation functional module are specifically as follows:

step 3.1, enhancing the image G of the fundus oculi in step 1.2_clahePerforming dual-channel (R channel and G channel) threshold segmentation to obtain binary image G_RG. The range of the pixel values of the RG channel is 0 to 255, in this embodiment, the threshold value of the R channel is 250, and the threshold value of the G channel is 160.

Step 3.2, comparing the binary image G in the step 3.1_RGAnalyzing the connected domains, extracting the minimum rectangular frame of each connected domain, and expanding the minimum rectangular frame in the left and right directions to obtain an image G after the connected domains are expanded_expand. In this embodiment, the size of the expansion is half of the longest side length of the rectangular frame.

Step 3.3, counting the image G after the connected domain expansion in the step 3.2_expandThe number of pixels on both sides of the image perpendicular to the center line.

Step 3.4, selecting corresponding Toeplitz matrix templates based on the number of pixels on two sides of the vertical center line of the fundus image in the step 3.3, and selecting a right Toeplitz matrix template Mask when more than half of the counted pixels are positioned on the right side of the image, as shown in the formula (1):

otherwise, selecting a left Toeplitz matrix template Mask as shown in the formula (2):

and 3.5, performing AND operation on the fundus blood vessel image segmented in the step 2.3 and the expanded connected region in the step 3.2 to obtain a blood vessel image G of the candidate region_candidate：

G_candidate＝G_BV∩G_expand (3)。

Step 3.6, blood vessel image G of candidate region_candidateFiltering to obtain filtered image G_filterThe formula is as follows:

step 3.7, determine filtered image G_filterAnd the coordinate of the pixel with the highest intermediate gray value is the coordinate of the positioning position of the fundus optic disk.

Step 3.8, with the optic disc positioning position coordinate as the center, framing a rectangular area with a certain size in the input fundus image G as the optic disc candidate area G_OD. The rectangular frame size in this embodiment is 400 × 400.

Step 3.9, extracting R channel of the video candidate area in step 3.8, and recording as image G'_R。

Step 3.10, carrying out alternate expansion corrosion operation by using the structure elements with the increasing radiuses to remove blood vessels in the candidate region of the video disc, firstly selecting a disc structure element B with a certain radius, and carrying out image G'_RCarrying out alternate expansion corrosion operation:

σ^(B)＝δ^(B)(ε^(B)(G′_R) (5) wherein, δ^(B)The expansion operation is carried out by taking B as a structural element; epsilon^(B)The etching operation is performed with B as a structural element.

Step 3.11, increasing the radius of the disc structure element B to obtain a new disc structure element B', and performing alternate expansion corrosion operation on the image processed in the step 3.10 again:

σ^(B′)＝δ^(B′)(ε^(B′)(σ^(B))) (6)。

and 3.12, continuously increasing the radius of the disc structural element B 'to obtain a new disc structural element B', and performing the alternate dilation corrosion operation on the image processed in the step 3.11 again to remove blood vessels in the candidate region of the optic disc:

σ^(B″)＝δ^(B″)(ε^(B″)(σ^(B′))) (7)。

step 3.13, performing threshold segmentation on the optic disc candidate region with blood vessels removed by adopting the maximum inter-class variance method to obtain a binary image G_OtsuAnd extracting the binary image G by using a Canny operator_OtsuTo obtain an edge image G_edge。

Step 3.14, extracting edge image G_edgeOutline coordinates of (1), to which the most is performedFitting by a small two times ellipse.

And 3.15, drawing the elliptic equation curve in the step 3.14 in the fundus image G, namely the optic disc elliptic ROI area.

FIG. 3- (a) is a diagram showing the result of optic disc segmentation according to the present invention.

Further, the least squares ellipse fitting in step 3.14 specifically comprises the following steps:

assuming the ellipse equation is: ax²+bxy+cy²+ dx + ey 1, the optimization problem of least squares ellipse fitting can be expressed as:

min||Dα||²

s.t.α^TCα＝1 (8)

wherein α ═ a, b, c, d, e ]; d represents a contour coordinate information set, the dimension is n multiplied by 6, and n is the number of contour pixels; matrix C is as follows:

further, the foveal determination operation step of the foveal determination function module specifically includes:

step 4.1.1, obtaining the macula lutea ROI area G by using the optic disc position and the optic disc size obtained in the step 3_m-ROI. Fig. 3- (b) is a diagram of the segmentation result of the macula lutea according to the present invention, in this embodiment, a rectangular frame with 300 × 400 is taken by taking the center of the optic disc as the starting point, the size of 2.5 times the major axis of the ellipse to the left/right, and 80 pixels down to the end point.

Step 4.1.2: extraction of the macular ROI region G in step 4.1.1_m-ROIGreen channel G of_m-GAnd is subjected to adaptive contrast enhancement to obtain G'_m-G(ii) enhanced image G'_m-GAnd G_mROIPerforming subtraction to obtain a decorrelated image G_m-decor. In this example, the CLAHE parameter contrast enhancement is limited to 0.01 and the histogram is 256 bins.

Step 4.1.3: for the decorrelated image G obtained in step 4.1.2_m-decorPerforming adaptive threshold segmentation to obtain G_m-otsuThen performing morphological rottingEtching to obtain G_m-erodeThen take G_m-erodeThe region of the largest connected domain area in (A) is the central concave ROIG_fovea-ROI. In this example, the etching operation selects a disk structure element with a radius of 3.

Step 4.1.4: calculating the foveal ROIG_fovea-ROICenter of mass (x)_fovea，y_fovea) It is approximated as a foveal position.

Further, the exudation segmentation operation steps of the exudation segmentation module are specifically as follows:

step 4.2.1: for the fundus enhancement image G in step 1.2_claheCarrying out mean value filtering to obtain a background estimation image G_clahe-meanAnd performing morphological reconstruction to obtain G_morp. In this embodiment, the average filtering size is 50 × 50.

Step 4.2.2: the morphological reconstruction graph G obtained in the step 4.2.1_morpAnd background estimation map G_clahe-meanDifferencing to obtain a normalized image G_ex-normAnd obtaining a bleeding candidate region I by using an adaptive threshold: g_ex-cand1。

Step 4.2.3: for the fundus enhancement image G in step 1.2_claheCalculating local variance to obtain boundary graph G of bright area_clahe-stdFor the boundary graph G_clahe-stdPerforming an opening operation and performing threshold segmentation to obtain G_std-otsu. In this embodiment, the opening operation selects a disc structure element with a radius of 3.

Step 4.2.4: for G in step 4.2.3_std-otsuSequentially performing expansion operation, closing operation and hole filling to obtain a seepage candidate area II: g_ex-cand2. In this embodiment, the expansion operation selects a disk structure element with a radius of 5, and the closing operation selects a disk structure element with a radius of 10.

Step 4.2.5: g in step 4.2.2_ex-cand1And the oozing candidate region G in step 4.2.4_ex-cand2Get a logical OR operation to get G_cand-orG is_{cand-or and}image G in step 1_claheMorphological reconstruction to obtain G_clahe-or。

Step 4.2.6: using step 3.8The obtained optic disk region G_ODDelete G_clahe-orIn the candidate region in the region, the final exudation candidate region G is obtained_ex-cand。

Further, the operation steps of the statistical computation function module are specifically:

step 5.1: with the foveated pixel (x) in step 4.1.4_fovea，y_fovea) As the circle center, different radiuses are given by the eyeground doctor according to the corresponding standard self-definition, and concentric circles are divided, wherein 3 radiuses are taken as examples and are marked as R from inside to outside in sequence₁、R₂、R₃. In this example, R₁One third of the length of the major axis of the optic disc ellipse, R₂The size is the length of the long axis of the optic disc ellipse, R₃Twice the length of the major axis of the optic disc ellipse.

Step 5.2: note R₁And R₂The ring in between is C₁，R₂And R₃The ring in between is C₂The oozing region G obtained in step 4.2.6 is counted_ex-candAt R₁、C₁、C₂The number of pixels in the input retinal fundus image, thereby calculating the probability of diabetic macular edema lesion of the input retinal fundus image.

FIG. 4- (a) is a graph showing the result of the exudation segmentation of the present invention, and FIG. 4- (b) is a graph showing the division of concentric circles of the present invention.

The above description is only for the purpose of illustrating the preferred embodiments of the present invention and is not to be construed as limiting the invention, and any modifications, equivalents and improvements made within the spirit and principle of the present invention are intended to be included within the scope of the present invention.

Claims (8)

1. A diabetic retinopathy detection system based on serial structure segmentation is characterized in that: the system comprises:

fundus image acquisition device: the device is used for acquiring and acquiring a fundus image of the retina;

a data processing device: the data processing device is used for analyzing and processing the acquired fundus images to obtain lesion region data and lesion probability, and comprises:

a preprocessing function module: preprocessing the acquired fundus image;

a blood vessel segmentation functional module: performing blood vessel segmentation based on morphology on the preprocessed fundus image;

the optic disc segmentation function module: performing optic disc segmentation on the preprocessed fundus image by combining the processing result of the blood vessel segmentation functional module;

foveal determination function: determining the fovea of the preprocessed image by combining the processing result of the optic disc segmentation functional module;

and a seepage segmentation functional module: combining the processing result of the optic disc segmentation functional module, performing exudation segmentation on the preprocessed image to obtain an exudation area;

a statistical calculation functional module: combining the processing results of the fovea centralis determination function module and the exudation segmentation function module, counting exudation areas and calculating the probability of diabetic macular edema lesion existing in the input fundus image; the optic disc segmentation operation steps of the optic disc segmentation functional module are specifically as follows:

step 3.1, fundus enhancement image G after the pretreatment of fundus image_clahePerforming dual-channel threshold segmentation to obtain a binary image G_RG；

Step 3.2, comparing the binary image G in the step 3.1_RGAnalyzing the connected domains, extracting the minimum rectangular frame of each connected domain, and expanding the minimum rectangular frame in the left and right directions to obtain an image G after the connected domains are expanded_expand；

Step 3.3, counting the image G after the connected domain expansion in the step 3.2_expandThe number of pixels on two sides of the vertical center line of the image in (1);

step 3.4, selecting corresponding Toeplitz matrix templates based on the number of pixels on two sides of the vertical center line of the fundus image in the step 3.3, and selecting a right Toeplitz matrix template Mask when more than half of the counted pixels are positioned on the right side of the image, as shown in the formula (1):

otherwise, selecting a left Toeplitz matrix template Mask as shown in the formula (2):

and 3.5, performing AND operation on the fundus blood vessel image segmented in the blood vessel segmentation functional module and the expanded connected region in the step 3.2 to obtain a blood vessel image G of the candidate region_candidate：

G_candidate＝G_BV∩G_expand (3)

Step 3.6, blood vessel image G of candidate region_candidateFiltering to obtain filtered image G_filterThe formula is as follows:

step 3.7, determine filtered image G_filterThe pixel with the highest middle gray scale value is the coordinate of the fundus optic disc positioning position;

step 3.8, with the optic disc positioning position coordinate as the center, framing a rectangular area with a certain size in the input fundus image G as the optic disc candidate area G_OD；

Step 3.9, extracting R channel of the video candidate area in step 3.8, and recording as image G'_R；

Step 3.10, carrying out alternate expansion corrosion operation by using the structure elements with the increasing radiuses to remove blood vessels in the candidate region of the video disc, firstly selecting a disc structure element B with a certain radius, and carrying out image G'_RCarrying out alternate expansion corrosion operation:

σ^(B)＝δ^(B)(ε^(B)(G′_R)) (5)

wherein, delta^(B)The expansion operation is carried out by taking B as a structural element; epsilon^(B)Representing that B is used as a structural element for corrosion operation;

step 3.11, increasing the radius of the disc structure element B to obtain a new disc structure element B', and performing alternate expansion corrosion operation on the image processed in the step 3.10 again:

σ^(B′)＝δ^(B′)(ε^(B′)(σ^(B))) (6)

and 3.12, continuously increasing the radius of the disc structural element B 'to obtain a new disc structural element B', and performing the alternate dilation corrosion operation on the image processed in the step 3.11 again to remove blood vessels in the candidate region of the optic disc:

σ^(B″)＝δ^(B″)(ε^(B″)(σ^(B′))) (7)

step 3.13, performing threshold segmentation on the optic disc candidate region with blood vessels removed by adopting the maximum inter-class variance method to obtain a binary image G_OtsuAnd extracting the binary image G by using a Canny operator_OtsuTo obtain an edge image G_edge；

Step 3.14, extracting edge image G_edgePerforming least square ellipse fitting on the contour coordinates;

and 3.15, drawing the elliptic equation curve in the step 3.14 in the fundus image G, namely the optic disc elliptic ROI area.

2. The serial structure segmentation based diabetic retinopathy detection system of claim 1 wherein: the data processing device further comprises a doctor diagnosis function module: and (4) combining the processing result of the statistical calculation functional module, and giving a final diagnosis and treatment scheme by referring to the segmented exudation area and the disease probability by the fundus doctor and combining the specialty of the fundus doctor.

3. The serial structure segmentation based diabetic retinopathy detection system of claim 1 wherein: the preprocessing operation steps of the preprocessing function module are specifically as follows:

step 1.1, RGB three-channel extraction is carried out on the input fundus image G, median filtering processing is carried out on each channel, and then three channels are synthesized into a single-frame color imageImage G_new；

Step 1.2, for G_newContrast-limited adaptive histogram equalization is performed to enhance image contrast, and fundus enhanced image G is obtained_clahe。

4. The serial structure segmentation based diabetic retinopathy detection system according to claim 1 or 3, characterized in that: the blood vessel segmentation operation steps of the blood vessel segmentation functional module are specifically as follows:

step 2.1, firstly extracting a fundus enhanced image G obtained by preprocessing the fundus image by the preprocessing function module_claheFiltering the image by using median filtering to obtain a background estimation image G_backgroundAnd estimating an image G of the background by using a disc structural element with a certain radius_backgroundRespectively carrying out top cap and bottom cap operations to obtain G_topAnd G_black；

Step 2.2, G in the step 2.1_topAnd G_blackAdding the weighted image to the original image G and subtracting the background estimation image G_backgroundAnd performing threshold segmentation to obtain a blood vessel binary estimation map G_BL；

Step 2.3, extracting the blood vessel binary estimation map G in the step 2.2_BLThe connected region with the middle connected region larger than a certain area is obtained to obtain a fundus blood vessel binary image G corresponding to the fundus image_BV。

5. The serial structure segmentation based diabetic retinopathy detection system of claim 1 wherein: the specific steps of least squares ellipse fitting in step 3.14 are as follows:

assuming the ellipse equation is: ax²+bxy+cy²+ dx + ey 1, the optimization problem of least squares ellipse fitting can be expressed as:

min||Dα||²

s.t.α^TCα＝1 (8)

wherein α ═ a, b, c, d, e ]; d represents a contour coordinate information set, the dimension is n multiplied by 6, and n is the number of contour pixels; matrix C is as follows:

6. the serial structure segmentation based diabetic retinopathy detection system of claim 1 wherein: the foveal determination operation steps of the foveal determination function module are specifically as follows:

step 4.1.1, obtaining the macula lutea ROI area G by using the optic disc position and the optic disc size obtained by the optic disc segmentation functional module_m-ROI；

Step 4.1.2: extraction of the macular ROI region G in step 4.1.1_m-ROIGreen channel G of_m-GAnd is subjected to adaptive contrast enhancement to obtain G'_m-G(ii) enhanced image G'_m-GAnd G_m-ROIPerforming subtraction to obtain a decorrelated image G_m-decor；

Step 4.1.3: for the decorrelated image G obtained in step 4.1.2_m-decorPerforming adaptive threshold segmentation to obtain G_m-otsuThen performing a morphological etching operation to obtain G_m-erodeThen take G_m-erodeThe region of the largest connected component area in (1) is the foveal ROI G_fovea-ROI；

Step 4.1.4: calculating foveal ROI G_fovea-ROICenter of mass (x)_fovea,y_fovea) It is approximated as a foveal position.

7. The serial structure segmentation based diabetic retinopathy detection system of claim 1 wherein: the exudation segmentation operation steps of the exudation segmentation function module are specifically as follows:

step 4.2.1: the fundus enhanced image G obtained by preprocessing the fundus image by the preprocessing function module_claheCarrying out mean value filtering to obtain a background estimation image G_clahe-meanAnd performing morphological reconstruction to obtain G_morp；

Step 4.2.2: will be provided withStep 4.2.1 morphological reconstruction G_morpAnd background estimation map G_clahe-meanDifferencing to obtain a normalized image G_ex-normAnd obtaining a bleeding candidate region I by using an adaptive threshold: g_ex-cand1；

Step 4.2.3: the fundus enhanced image G obtained by preprocessing the fundus image by the preprocessing function module_claheCalculating local variance to obtain boundary graph G of bright area_clahe-stdFor the boundary graph G_clahe-stdPerforming an opening operation and performing threshold segmentation to obtain G_std-otsu；

Step 4.2.4: for G in step 4.2.3_std-otsuSequentially performing expansion operation, closing operation and hole filling to obtain a seepage candidate area II: g_ex-cand2；

Step 4.2.5: g in step 4.2.2_ex-cand1And the oozing candidate region G in step 4.2.4_ex-cand2Get a logical OR operation to get G_cand-orG is_cand-orAnd fundus enhanced image G_claheMorphological reconstruction to obtain G_clahe-or；

Step 4.2.6: using the disc region G obtained in the disc dividing function block_ODDelete G_clahe-orIn the candidate region in the region, the final exudation candidate region G is obtained_ex-cand。

8. The serial structure segmentation based diabetic retinopathy detection system of claim 1 wherein: the operation steps of the statistical calculation function module are specifically as follows:

step 5.1: to obtain a foveated pixel (x)_fovea,y_fovea) As the circle center, different radiuses are given by the eyeground doctor according to the corresponding standard self-definition, and concentric circles are divided, wherein 3 radiuses are taken as examples and are marked as R from inside to outside in sequence₁、R₂、R₃；

Step 5.2: note R₁And R₂The ring in between is C₁，R₂And R₃The ring in between is C₂Statistical derived from the exuded segmentation functionExudation region G_ex-candAt R₁、C₁、C₂The number of pixels in the input retinal fundus image, thereby calculating the probability of diabetic macular edema lesion of the input retinal fundus image.

Images