CN111489328A

CN111489328A - Fundus image quality evaluation method based on blood vessel segmentation and background separation

Info

Publication number: CN111489328A
Application number: CN202010149988.3A
Authority: CN
Inventors: 刘义鹏; 吕雅俊; 王海霞; 蒋莉; 陈朋; 梁荣华
Original assignee: Zhejiang University of Technology ZJUT
Current assignee: Zhejiang University of Technology ZJUT
Priority date: 2020-03-06
Filing date: 2020-03-06
Publication date: 2020-08-04
Anticipated expiration: 2040-03-06
Also published as: CN111489328B

Abstract

A fundus image quality evaluation method based on blood vessel segmentation and background separation comprises the following steps: 1) firstly, performing vessel segmentation on an input image through a pre-trained U-Net model on a DRIVE public fundus image dataset; 2) multiplying the blood vessel characteristic diagram obtained in the step 1) with the original image element by element to obtain an image only containing blood vessels and background information; 3) respectively inputting the extracted characteristic images into the convolutional neural network branches for training to obtain model parameters; 4) and carrying out quality evaluation on the test picture by using the trained convolutional neural network model. The invention realizes higher evaluation accuracy, reduces the doctor rechecking rate and avoids the treatment opportunity delay possibly caused by the repeated examination. The model provided by the invention has universality, can be embedded into various advanced convolutional neural network structures and improves the network performance, and meanwhile, provides a method for fusing the vascular prior knowledge and the neural network end-to-end feature extraction.

Description

Fundus image quality evaluation method based on blood vessel segmentation and background separation

Technical Field

The invention relates to the field of medical image processing and computer vision, in particular to an image quality evaluation method based on a convolutional neural network.

Background

The fundus images are captured by a special fundus camera, wherein the fundus images comprise important physiological structures such as optic discs, macula lutea and blood vessels in retinas, and are important images in medical images. The optic disc is represented as an approximately circular bright color area in a normal fundus image, has the strongest contrast with a background area, and is the initial area of optic nerves and blood vessels; the macula, because it is rich in lutein, appears as a dark region in a normal fundus image, and the dark region has no vascular structure, and there is an inwardly depressed region called fovea in the midpoint of the macula; the blood vessels start from the optic disc region and extend to the whole eyeball, are distributed in a tree shape in the whole fundus image, are thickest and most dense in the optic disc region, and extend along the vertical direction basically.

Although the fundus image quality evaluation method is more mature, the quality evaluation of most fundus images depends on the subjective judgment of medical researchers, and the subjective judgment mode is mainly influenced by the characteristics of large-area exposure and visual field blurring in a wide-range global scale. The fundus images play an important role in tasks of extracting blood vessel information, positioning an optic disc, positioning a macular region and the like, and the applications provide huge information for diagnosis and treatment of ophthalmic diseases. A study involving extensive databases of retinal images has shown that more than 25% of the image display is of insufficient quality for proper medical diagnosis. In addition to the financial investment required to retrieve a picture of poor quality, it is inconvenient for the patient to return to the medical center for repeated fundus photographic examinations. More seriously, misdiagnosis caused by poor image shooting quality is not easy to be found, and the treatment time is delayed. Therefore, it is important to develop an automatic evaluation system for fundus image quality. Through quality evaluation, the above-mentioned disadvantages of ophthalmic examinations can be alleviated by taking a second photograph immediately after the poor quality photograph is taken. In order to ensure the quality of fundus images while reducing the time and effort of manual screening, it is necessary and urgent to automatically and objectively evaluate the quality of fundus images during the acquisition process.

Disclosure of Invention

The technical problem to be solved by the invention is to provide a fundus image quality evaluation method, which divides a fundus input image into a blood vessel part and a background part, enables the fundus input image to focus on the characteristics of the blood vessel part and the background part respectively, designs a double-branch attention structure, can extract global characteristics influencing the image quality, and inhibits the interference of blood vessels and local textures on the extraction of key characteristics of image quality evaluation. The model provided by the invention has universality, can be embedded into various advanced network structures, and improves the network performance. Meanwhile, the invention provides a path for fusing the vascular prior knowledge and the end-to-end feature extraction of the neural network.

In order to solve the technical problems, the invention provides the following technical scheme:

a background separation fundus image quality evaluation method based on blood vessel segmentation guidance comprises the following steps:

1) firstly, performing vessel segmentation on an input image through a pre-trained U-Net model on a DRIVE public fundus image dataset;

2) multiplying the blood vessel characteristic diagram obtained in the step 1) with the original image element by element to obtain an image only containing blood vessels and background information;

3) inputting the extracted characteristic images into network branches respectively for training to obtain model parameters;

4) and carrying out quality evaluation on the test picture by using the trained convolutional neural network model.

Further, the network structure implemented in step 3) includes two parts: the dual-branch feature extraction path and the channel feature strengthening structure are characterized in that the dual-branch feature extraction path is composed of two feature extraction paths which respectively comprise two convolution blocks, the internal structures of the convolution blocks are the same, the output of the last convolution block of the path branch is fused, the learned different regional features are integrated, the shallow layer features are extracted and learned in the convolution blocks, and finally, a global pooling layer and a full connection layer are added on the original convolution blocks to enhance feature representation in the channels.

The invention has the following beneficial effects: the sensitivity of the conventional convolutional neural network to fundus blood vessels with severe local gradient changes is weakened, the relevant characteristics of larger scales influencing the image quality in the background, such as overexposure, blurring and the like of the area, are enhanced, and the representation capability of the network on the image quality characteristics is improved.

Drawings

FIG. 1 is an overall flow chart of the method employed in the present invention;

fig. 2 is a schematic diagram of a network structure adopted in the present invention.

Detailed Description

The invention is further described below with reference to the schematic drawings.

Referring to fig. 1 and 2, a background separation fundus image quality evaluation method based on blood vessel segmentation guidance includes the following steps:

An embodiment of this example includes the steps of:

first step, image preprocessing

The quality of the image directly influences the precision of the effect, and the purpose of image preprocessing is mainly to eliminate irrelevant information in the image, simplify data to the maximum extent and overcome image interference. First, each color retina image is cut, redundant black edges of the fundus image are removed, only the fundus part is reserved, and the interested area is extracted.

Second step, data expansion

Because the number of pictures in the data set is small, and the network needs a large amount of data to drive the training of the model, the effective data expansion can avoid overfitting on one hand, and can bring the improvement of the model performance on the other hand. The expansion method proposed by the invention is to randomly select in the ranges of left-right, up-down turning and rotation angles of 90, 180 and 270 so as to create different transformation images. According to the image distribution of the training data, the images with good quality are increased by 4 times, the images with poor quality are increased by 8 times, and the training data can be effectively expanded.

Third step, model training

The processed picture block is used as input and sent to a neural network for training, and the flow of the fundus image quality evaluation method based on blood vessel segmentation and background separation designed by the invention is shown in figure 1. Firstly, performing blood vessel segmentation on an input image through a pre-trained U-Net model on a DRIVE public fundus image data set, setting the number of channels of the last layer of convolution layer of the pre-trained U-Net model to be 2, wherein a judgment pixel point in one channel is not a blood vessel, and a pixel point in the other channel is judged to be a blood vessel, separating the obtained two-channel characteristic diagram, making a mask and performing element-by-element multiplication on the mask and the input image, respectively obtaining an image only containing blood vessel information and background information, and inputting the image into a double-branch structure for characteristic extraction.

The invention relates to a method for extracting characteristics of a neural network, which comprises the steps of constructing a path containing a blood vessel, merging the output of the last volume block of the path branch, integrating learned characteristics of different regions, performing scale correction on shallow and deep layers of the neural network by convolution of three 1 × 1, performing scale correction on convolution of each 1 ×, and increasing the total channel size of all the original volume blocks by using the same channel number and enhancing the total channel size of all the original volume blocks.

And connecting the feature graph output after the last layer of convolution with a Softmax classification layer, converting the feature graph into a probability vector representing the high and low quality of each image, and randomly initializing the internal parameters of the convolution kernel in a plurality of modes at the beginning stage of model training by using a cross entropy loss function and a random gradient descent optimization function. And finally obtaining a model which can be finally used for fundus image quality evaluation through iterative training.

Fourth step, model test

And (3) carrying out the same pretreatment on the image data set used in the test and the image data set used in the training, preserving the proportion of the original data category, inputting the image data set into the model for judgment and prediction, and extracting the maximum probability of the judgment category as a prediction result.

Claims

1. A fundus image quality evaluation method based on blood vessel segmentation and background separation is characterized by comprising the following steps:

3) respectively inputting the extracted characteristic images into the convolutional neural network branches for training to obtain model parameters;

2. A fundus image quality evaluation method based on blood vessel segmentation and background separation according to claim 1, characterized in that the network structure implemented in the step 3) comprises two parts: the dual-branch feature extraction path and the channel feature strengthening structure are characterized in that the dual-branch feature extraction path is composed of two feature extraction paths which respectively comprise two convolution blocks, the internal structures of the convolution blocks are the same, the output of the last convolution block of the path branch is fused, the learned different regional features are integrated, the shallow layer features are extracted and learned in the convolution blocks, and finally, a global pooling layer and a full connection layer are added on the original convolution blocks, so that the feature representation capability in the channel is enhanced.