CN113947807A

CN113947807A - Method and system for identifying fundus image abnormity based on unsupervised

Info

Publication number: CN113947807A
Application number: CN202111565273.7A
Authority: CN
Inventors: 姜泓羊; 帅平; 刘玉萍; 张冬冬; 代黎明
Original assignee: Beijing Zhizhen Internet Technology Co ltd
Current assignee: Beijing Zhizhen Internet Technology Co ltd
Priority date: 2021-12-21
Filing date: 2021-12-21
Publication date: 2022-01-18
Anticipated expiration: 2041-12-21
Also published as: CN113947807B

Abstract

The invention relates to an unsupervised fundus image abnormity identification method and system, wherein the method comprises the following steps: acquiring a fundus image to be detected; extracting the fundus image to be detected by using a target detection network to obtain an image of a macular area to be detected; labeling the to-be-detected macular area image by using a similarity measurement network to obtain label data; and classifying by using a deep learning network model according to the label data and the to-be-detected macular region image to obtain an eyeground abnormal value. The invention can solve the problem of coarse granularity of classified identification of the fundus images.

Description

Method and system for identifying fundus image abnormity based on unsupervised

Technical Field

The invention relates to the field of fundus image identification, in particular to a method and a system for identifying fundus image abnormity based on unsupervised.

Background

With the development of artificial intelligence technology, the application of the artificial intelligence technology in medical image analysis is increasing. The color fundus image is combined with the artificial intelligence technology to produce obvious application effect due to the characteristics of easy acquisition, wide application range and the like. The intelligent analysis of the fundus image by the artificial intelligence technology mainly comprises the following steps: image recognition and classification, region of interest (ROI) extraction, quantitative analysis and the like. The image identification and classification are most widely applied, and can be applied to fundus image quality judgment and fundus abnormal condition identification, such as cataract, glaucoma, age-related macular degeneration (AMD) and the like. The computer technology is used for assisting the fundus image analysis, and the research focus in the field of medical artificial intelligence is achieved at present. The existing methods are mainly divided into two categories, namely a traditional machine learning method and a deep learning method. The learning method based on the traditional machine mainly comprises two parts, namely image feature extraction and a classifier algorithm, wherein the effect of the image feature extraction directly influences the effect of the classifier, the detection capability difference of different fundus abnormal features (such as drusen, hemorrhage, atrophy and the like contained in AMD) is large, and the overall generalization capability of the algorithm is weak. Furthermore, this type of method is often used when the data size is not large. The deep learning-based method generally takes an end-to-end algorithm model as a main part, model training is mainly carried out in a supervised mode, and the performance of the model is very dependent on the volume of fine calibration data. In general, the larger the amount of fine-scale data, the better the performance of the model. The main technical bottleneck of fundus image identification and analysis is to improve the overall performance of the algorithm, enhance the generalization capability of the algorithm and break through the limitation of precise standard data.

The color fundus image is a two-dimensional image containing basic biological information of the human body (such as optic disc, macula lutea, blood vessel, etc.) and other abnormal information (such as bleeding, exudation, atrophy, reflection, stain, etc.). Based on the fundus biological level information and other abnormal information, fundus images can be classified and graded according to different criteria. In general, intelligent analysis of fundus images requires a large amount of specific fine-marking data depending on the specific task, which creates difficulties and challenges for the annotating personnel. Due to the problems of high requirements on the professional level of the annotating personnel, continuous high-intensity attention and the like, the economic cost and the time cost for annotating a large number of fundus images with high quality are high. How to weaken the dependence on supervised learning is a problem to be further considered in the current fundus image recognition algorithm design. In addition, in the image classification task, the classification granularity of the mainstream supervised deep learning model is coarse (such as AMD secondary classification, tertiary classification, quinary classification and the like), the contrast and explanation among images in a certain class are lacked, and the method is difficult to be applied to fundus image analysis with a time dimension (such as AMD image follow-up analysis).

Disclosure of Invention

The invention aims to provide an unsupervised fundus image abnormity identification method and system to solve the problem of coarse granularity of fundus image classification and identification.

In order to achieve the purpose, the invention provides the following scheme:

an unsupervised fundus image abnormality identification method comprises the following steps:

acquiring a fundus image to be detected;

extracting the fundus image to be detected by using a target detection network to obtain an image of a macular area to be detected;

labeling the to-be-detected macular area image by using a similarity measurement network to obtain label data;

and classifying by using a deep learning network model according to the label data and the to-be-detected macular region image to obtain an eyeground abnormal value.

Optionally, the extracting the fundus image to be detected by using the target detection network to obtain a macular area image specifically includes:

inputting the fundus image to be detected into a target detection network to obtain a optic disc area and a yellow spot area;

and determining the image of the macular area to be detected according to the optic disc area and the macular area.

Optionally, the training process of the similarity metric network specifically includes:

training an encoding network and a decoding network in an unsupervised mode by taking a macular region training image as input, label data as output and image reconstruction loss and similarity contrast loss as loss functions to obtain a first-stage similarity measurement network; the macular region training image comprises an macular region reference image, an macular region positive example image and an macular region negative example image;

and optimizing the parameters of the first-phase similarity measurement network by using the fine-label macular area training image to obtain the similarity measurement network.

Optionally, the labeling the to-be-detected macular region image by using a similarity measurement network to obtain tag data specifically includes:

inputting the image of the macular area to be detected into the similarity measurement network as a reference image of the macular area to obtain a plurality of support data;

tag data is determined from a plurality of said supporting data.

Optionally, the training process of the deep learning network model specifically includes:

training a basic network by taking the fundus training image with the label data as input and taking the fundus abnormal value as output to obtain a first-stage basic network;

and optimizing the first-stage basic network by using the precisely marked fundus training image to obtain a deep learning network model.

An unsupervised fundus image abnormality recognition system, comprising:

the acquisition module is used for acquiring an eyeground image to be detected;

the extraction module is used for extracting the fundus image to be detected by using a target detection network to obtain a macular area image to be detected;

the labeling module is used for labeling the to-be-detected macular area image by using a similarity measurement network to obtain label data;

and the classification module is used for classifying by using a deep learning network model according to the label data and the to-be-detected macular region image to obtain an eyeground abnormal value.

Optionally, the extraction module specifically includes:

the optical disc area and yellow spot area determining unit is used for inputting the fundus image to be detected into a target detection network to obtain an optical disc area and a yellow spot area;

and the to-be-detected macular area image determining unit is used for determining the to-be-detected macular area image according to the optic disc area and the macular area.

Optionally, the labeling module specifically includes:

the supporting data determining unit is used for inputting the image of the macular area to be detected into the similarity measurement network as a reference image of the macular area to obtain a plurality of supporting data;

a tag data determination unit configured to determine tag data from a plurality of the support data.

According to the specific embodiment provided by the invention, the invention discloses the following technical effects:

the invention provides an unsupervised fundus image abnormity identification method and system, which utilizes a target detection network to extract a fundus image to be detected to obtain a macular area image to be detected; labeling the to-be-detected macular area image by using a similarity measurement network to obtain label data; and classifying by using a deep learning network model according to the label data and the to-be-detected macular region image to obtain an eyeground abnormal value. The invention can solve the problem of coarse granularity of classified identification of the fundus images.

Drawings

In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings needed to be used in the embodiments will be briefly described below, and it is obvious that the drawings in the following description are only some embodiments of the present invention, and it is obvious for those skilled in the art to obtain other drawings without inventive exercise.

FIG. 1 is a flow chart of an unsupervised fundus image anomaly identification method according to the present invention;

FIG. 2 is a frame diagram of the method for identifying abnormality of fundus images based on unsupervised vision;

FIG. 3 is a schematic diagram of a process of extracting an image of a macular region according to the present invention;

FIG. 4 is a diagram illustrating the effect of extracting the macular region image according to the present invention;

FIG. 5 is a schematic diagram of a similarity metric network according to the present invention;

FIG. 6 is a schematic diagram of a labeling process using a similarity measurement network according to the present invention;

FIG. 7 is a schematic diagram illustrating a classification process of a deep learning network model according to the present invention;

fig. 8 is a schematic view of an unsupervised fundus image abnormality recognition system according to the present invention.

Detailed Description

The technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the drawings in the embodiments of the present invention, and it is obvious that the described embodiments are only a part of the embodiments of the present invention, and not all of the embodiments. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present invention.

In order to make the aforementioned objects, features and advantages of the present invention comprehensible, embodiments accompanied with figures are described in further detail below.

As shown in fig. 1, the method for identifying fundus image abnormality based on unsupervised provided by the invention comprises the following steps:

step 101: acquiring an eyeground image to be detected.

Step 102: and extracting the fundus image to be detected by using a target detection network to obtain an image of the macular area to be detected. Step 102 specifically includes: inputting the fundus image to be detected into a target detection network to obtain a optic disc area and a yellow spot area; and determining the image of the macular area to be detected according to the optic disc area and the macular area.

Step 103: and labeling the to-be-detected macular area image by using a similarity measurement network to obtain label data. Step 103 specifically includes: inputting the image of the macular area to be detected into the similarity measurement network as a reference image of the macular area to obtain a plurality of support data; tag data is determined from a plurality of said supporting data. Wherein, the label data is a false label.

Step 104: and classifying by using a deep learning network model according to the label data and the to-be-detected macular region image to obtain an eyeground abnormal value.

The training process of the similarity metric network specifically includes:

training an encoding network and a decoding network in an unsupervised mode by taking a macular region training image as input, label data as output and image reconstruction loss and similarity contrast loss as loss functions to obtain a first-stage similarity measurement network; the macular region training image comprises an macular region reference image, an macular region positive example image and an macular region negative example image; and optimizing the parameters of the first-phase similarity measurement network by using the fine-label macular area training image to obtain the similarity measurement network.

The training process of the deep learning network model specifically comprises the following steps:

training a basic network by taking the fundus training image with the label data as input and taking the fundus abnormal value as output to obtain a first-stage basic network; and optimizing the first-stage basic network by using the precisely marked fundus training image to obtain a deep learning network model.

The method for identifying the fundus image abnormity based on unsupervised provided by the invention can solve the problems of small precise standard data quantity, thick classification and identification granularity of the fundus image and the like. As shown in fig. 2, the method provided by the present invention mainly includes four parts, namely, ROI region (e.g., macular region, optic disc region) extraction, image similarity measurement, image label assignment and image recognition classification.

First, ROI region extraction. In order to realize the accurate analysis of the fundus image content by the artificial intelligence technology, the invention firstly extracts the ROI area (such as a macular area and an optic disc area) from the fundus image, reduces the influence of the image content outside the ROI area and reduces the requirement of a large amount of data caused by noise interference.

The invention uses a target detection network (Faster Rcnn) to extract the ROI area and cut it out for further analysis. In the present invention, the macular region and the optic disc region are extracted simultaneously, and the optic disc region is used to guide the range of the macular region, as shown in fig. 4, the present invention uses the central fovea of the macula as the center and the range of the diameter of the optic disc twice the center of the circle as the macular region of the fundus. As shown in fig. 3, the flow of extracting the macular region is to input a fundus image including a disc and a macular quine into a target detection network, obtain a disc region and a macular fovea, and determine the macular region according to the disc region and the macular fovea.

Second, an image similarity measure. Due to the uneven distribution of the required fine-scale fundus images, the scarcity of ophthalmologists and the high cost of fine-scale image, high-quality data satisfying the data volume, data quality and labeling quality are very deficient.

As shown in fig. 5, the similarity metric network proposed by the present invention is based on an unsupervised metric learning model, and is used to assign labels to a large amount of unlabeled data, wherein A, P, N represents a reference image, a positive example image, and a negative example image, respectively, and a group (a, P, N) represents that the similarity between image P and image a is greater than that between image N and image a, thereby achieving the effect of image similarity comparison.

The network compresses a group of macular region images (A, P, N) into vectors with the dimension of 1 xK through an image self-coding network, wherein the vectors comprise reference image compression vectors

Positive case image compression vector

And negative example image compression vector

. The whole network guides the network to learn through two losses, wherein the image reconstruction loss adopts a traditional binary cross entropy loss function, and the similarity comparison loss adopts triplet loss (triplet loss), as shown in formula (1). Wherein D represents a distance function, and the Euclidean distance is adopted by the invention.

(1)

(2)

(3)

Wherein, L is a loss function, D is an euclidean distance between two vectors, m is a boundary distance, which is a preset positive value, and the boundary distance may be set to 0.5.

In order to reduce the use amount of fine-scale data, the method adopts a two-stage training method when training the similarity measurement network. In the first stage, the invention constructs a macular image (A, At, N) in an unsupervised mode, wherein the image A is an arbitrarily selected macular image in the macular image data, At is based on non-image content transformation (affine transformation, color illumination transformation, etc.) of the image A, and N is other images except the image A, so that a pair of positive example images (A, At) and a pair of negative example images (A, N) are formed. Through the data construction mode, the training of the first stage is completed. In the second stage, the invention fixes the basic coding network parameters, and carries out network fine adjustment through a small amount of precise standard data, and finally realizes the training of the similarity measurement network.

Third, image tag matching. In the invention, a small amount of fine-scale data is used as support data for label matching, and the macular images of unknown labels are labeled through a learned similarity measurement network, wherein the labeling process is shown in fig. 6. The image P and the image N are selected from the support data (fine labeling data), the image A is selected from the non-labeling data, the labeled image (support data) which is more similar to the image A can be found out through judgment of a similarity measurement network, and the label of the image A can be estimated after multiple rounds of similarity comparison and statistical averaging. In this way, label-free data can be labeled after image label matching, and labeled labels have two expression forms, namely classification labels and regression labels, and can serve for training of a classification network and a regression network respectively. The supporting data is used to help label the unlabeled data. The non-label data can obtain a pseudo label after image label matching. Pseudo labels have two manifestations, namely class labels and regression labels.

Fourth, image recognition and classification. Typically, a fine-scaled fundus image (e.g., AMD) is labeled with a classification label. After the unlabeled data is subjected to image label matching as above, a regression label is obtained. The method carries out regression training on labeled data based on a deep learning network model (ResNet 50), and the training is divided into two stages. The first stage, training by using fundus image data with a pseudo label; and in the second stage, parameters of the basic network are fixed, fine adjustment is carried out by adopting fundus image data with fine marks, and training of the regression network is realized. Finally, the module realizes the identification and prediction of the eye fundus abnormal value. The network design of the image recognition classification is shown in fig. 7.

As shown in fig. 8, the present invention provides an unsupervised fundus image abnormality recognition system, including:

an acquiring module 801, configured to acquire a fundus image to be detected.

An extracting module 802, configured to extract the fundus image to be detected by using a target detection network, so as to obtain an image of a macular region to be detected. The extracting module 802 specifically includes: the optical disc area and yellow spot area determining unit is used for inputting the fundus image to be detected into a target detection network to obtain an optical disc area and a yellow spot area; and the to-be-detected macular area image determining unit is used for determining the to-be-detected macular area image according to the optic disc area and the macular area.

And the labeling module 803 is configured to label the to-be-detected macular region image by using a similarity measurement network, so as to obtain label data. The labeling module 803 specifically includes: the supporting data determining unit is used for inputting the image of the macular area to be detected into the similarity measurement network as a reference image of the macular area to obtain a plurality of supporting data; a tag data determination unit configured to determine tag data from a plurality of the support data.

And the classification module 804 is used for classifying by using a deep learning network model according to the label data and the to-be-detected macular region image to obtain an eyeground abnormal value.

The training process of the similarity metric network specifically includes:

The invention provides an unsupervised fundus image abnormity identification method and system, which are used for extracting a macular region in a color fundus image and reducing the interference of regions except the macula. In the aspect of algorithm model design, a plurality of groups of training data are constructed, a similarity measurement network is trained in an unsupervised learning mode, and unmarked data are marked by combining a small amount of precise marking data. By the mode, a large amount of precise marking data are not needed, marking of a large number of fundus image macular regions is achieved, classification labels and regression labels are provided for images without labels, and marking cost is greatly reduced. In addition, the invention trains the fundus abnormal value regression network based on a large number of images with pseudo labels and a small number of images with precise labels, thereby realizing the analysis and prediction of the abnormal value of the medical image. The technology provided by the invention can realize classification and identification of the fundus images under the condition that the precise mark fundus images are fewer or even deficient, and can be applied to fundus image analysis with time dimension (such as follow-up condition).

The embodiments in the present description are described in a progressive manner, each embodiment focuses on differences from other embodiments, and the same and similar parts among the embodiments are referred to each other. For the system disclosed by the embodiment, the description is relatively simple because the system corresponds to the method disclosed by the embodiment, and the relevant points can be referred to the method part for description.

The principles and embodiments of the present invention have been described herein using specific examples, which are provided only to help understand the method and the core concept of the present invention; meanwhile, for a person skilled in the art, according to the idea of the present invention, the specific embodiments and the application range may be changed. In view of the above, the present disclosure should not be construed as limiting the invention.

Claims

1. An unsupervised fundus image abnormality identification method is characterized by comprising the following steps:

acquiring a fundus image to be detected;

2. The unsupervised fundus image abnormality identification method according to claim 1, wherein said extracting the fundus image to be detected by using the target detection network to obtain the macular region image specifically comprises:

3. The unsupervised fundus image abnormality recognition method according to claim 1, wherein the training process of the similarity metric network specifically comprises:

4. The unsupervised fundus image abnormality identification method according to claim 1, wherein the labeling of the to-be-detected macular region image by using a similarity measurement network to obtain label data specifically comprises:

tag data is determined from a plurality of said supporting data.

5. The unsupervised fundus image abnormality recognition method according to claim 1, wherein the training process of the deep learning network model specifically comprises:

6. An unsupervised fundus image abnormality recognition system, comprising:

the acquisition module is used for acquiring an eyeground image to be detected;

7. The unsupervised fundus image abnormality recognition system according to claim 6, wherein said extraction module specifically comprises:

8. The unsupervised fundus image abnormality recognition system according to claim 6, wherein the training process of the similarity metric network specifically comprises:

9. An unsupervised fundus image abnormality recognition system according to claim 6, wherein said labeling module specifically comprises:

10. The unsupervised fundus image abnormality recognition system according to claim 6, wherein the training process of the deep learning network model specifically comprises: