CN113421250A - Intelligent fundus disease diagnosis method based on lesion-free image training - Google Patents

Abstract

The invention relates to an intelligent diagnosis method for fundus diseases based on non-pathological image training, belonging to the technical field of image classification and disease diagnosis. Including: 1. Construct training set and test set and complete data set preprocessing; 2. Build encoder, decoder, discriminator, and restoration decoder models for non-lesion image training; 3. Build surrogate tasks based on image transformation; 4. Construct a weighted loss function based on reconstruction loss, discriminant loss, and restoration loss; 5. Model training; 6. Use the trained encoding-decoding model to test the image to be detected. The method gets rid of the conditional dependence on the coexistence of different types of data in the training set through the training method of image reconstruction; the introduction of the proxy task reduces the data demand of the model; the constraints of the image and the feature space strengthen the model for the image organization structure. The above features together improve the model's ability to recognize disease images.

Description

An intelligent diagnosis method for fundus diseases based on non-lesion image training

technical field

The invention relates to an intelligent diagnosis method for fundus diseases based on non-pathological image training, belonging to the technical field of image classification and disease diagnosis.

Background technique

Fundus images are of great significance to medical disease diagnosis and are often used by ophthalmologists to diagnose various diseases. Various diseases of the eye, as well as diseases affecting blood circulation and the brain, can be visualized in fundus images, including macular degeneration, which causes blindness, glaucoma, and complications of systemic diseases such as diabetic retinopathy, high blood pressure, and more. Compared with other medical images, the equipment requirements for obtaining ophthalmic images are lower, and it is suitable for large-scale census at the grassroots level and provides efficient diagnostic services for patients at the grassroots level. It has broad application prospects and practical social value. Artificial intelligence has the advantages of high speed and high accuracy in medical image-assisted diagnosis, and plays an important role in assisting doctors in analyzing and identifying lesions and improving the efficiency of diagnosis.

The current medical imaging diagnosis algorithms are mainly based on deep neural networks, which use health and disease samples for training, and require a large amount of labeled data as the training basis. In clinical practice, labeled lesion data is scarce. For some new diseases, it is very difficult to obtain a large number of lesion labels in a short period of time. When using a small number of samples for training, it will lead to problems such as model performance degradation. On the other hand, in the field of medical image analysis, a large number of healthy samples cannot be effectively utilized due to the constraints of the sample size of lesions. Although some studies have initially explored the classification algorithm under a single category of samples, the algorithm cannot be used in clinical practice due to the problems of time-consuming and low accuracy in the testing process. Combining the above two points, how to complete the development of a high-performance diagnostic system under the condition of no lesion data, that is, only using the data of healthy people, is a research problem that needs to be solved in the field of medical image analysis.

The purpose of the present invention is to solve the clinical disease diagnosis in the non-lesion image scene by using the unsupervised diagnosis algorithm and combining the fundus image distribution characteristics in the feature space, and propose a deep learning fundus disease intelligent diagnosis method based on non-lesion image training to assist doctors. Complete disease diagnosis with high accuracy.

SUMMARY OF THE INVENTION

The object of the present invention is to address the following two defects in the existing fundus image classification and diagnosis algorithm: 1) the existing algorithm relies on a large amount of labeled data: under the condition of lack of data, the algorithm performance is not good; 2) over-reliance on balanced data labels: Under the premise of only one type of labels, the algorithm performance is lacking. An intelligent diagnosis method of fundus diseases based on non-lesion image training is proposed.

In order to achieve the above object, the present invention adopts the following scheme.

The intelligent diagnosis method for fundus diseases based on non-lesion image training is realized by the following steps:

Step 1: Preprocess the collected images to construct a data set, specifically: screen the collected images, remove the images with poor image quality, and then unify the resolution of the filtered images into the same dimension W*W*c , and normalize the pixel value to [-1,1];

Among them, c is greater than or equal to 1;

The data set is divided into a training set and a test set, both of which are ophthalmological images collected clinically; the images in the training set are composed of ophthalmological images of healthy individuals, and the images in the test set are a collection of images of healthy individuals and diseased individuals; images of poor quality , specifically: the picture is too dark, the shooting angle of view has a large deviation, and the shooting image is shaken;

Step 2: Design a network model including an encoder, a decoder, a discriminator 1, a discriminator 2 and a restoration decoder, which specifically includes the following sub-steps:

Step 2.1 Build an encoder with multiple convolutional layers;

Among them, the encoder includes N groups of downsampling layers, regularization layers and activation functions in series; the encoder input is an image with dimension W*W*c after step 1, and the output is an expression of 1*1*2 ^N+1 The feature vector z of the image essence;

The value of N is less than or equal to log ₂ W;

The downsampling layer includes a convolutional layer with a convolution kernel size of n*n and a stride of 2, and the number of channels increases from 2 ² to 2 ^N+1 in multiples of 2;

The activation function is the LeakyReLU activation function with negative slope L;

The value of n is [3,5]; the value of L is [0,1];

Step 2.2 Build a decoder of multiple deconvolution layers;

Wherein, the decoder includes N groups of upsampling layers, regularization layers and activation functions connected in series; the input of the decoder is the feature vector z output by the encoder in step 2, and the output is an image with dimension W*W*c;

The upsampling layer includes a convolutional layer with a convolution kernel size of n*n and a stride of 2, and the number of channels is sequentially decreased from ^2N+1 to 2 2 in multiples of ² ;

The activation function is the ReLU activation function with negative slope L, and the outermost activation function is the Tanh function;

Step 2.3 uses a multilayer perceptron structure to construct the discriminator 1 of the feature space;

Wherein, the input of the discriminator 1 is the feature vector z that expresses the essence of the image output in step 2.1; and the discriminator 1 includes K series fully connected layers, dropout layers and activation functions;

The number of neurons in each layer of the fully connected layer decreases from ^2K to 20 in multiples of ² ;

The random drop parameter of the Dropout layer is p;

In addition to the activation function of the last layer, the activation function is sigmoid, and the remaining K-1 activation functions are LeakyReLU with a negative slope of L;

Among them, the value range of K is [5, log ₂ W]; the value range of p is [0, 1];

Step 2.4 uses the PatchGAN structure to construct the discriminator 2 of the image space;

Among them, the discriminator 2 includes P convolutional layers; the first P-1 convolutional layer includes a convolutional layer, a regularization layer and an activation function;

The convolution layer includes a convolution layer with a convolution kernel size of n*n and a stride of 2, and the number of channels increases from 2 ² to 2 ^P in a multiple of 2;

The last layer is directly output after convolution, and the remaining P-1 activation functions are LeakyReLU functions with negative slope L;

Among them, the value range of P is [5, log ₂ W];

Step 2.5 uses multiple deconvolution layers to construct a restoration decoder;

Wherein, the restoration decoder includes N groups of up-sampling layers, regularization layers and activation functions connected in series; the input of the decoder is the output vector of the encoder in step 2, and the output is an image of dimension W*W*c;

The upsampling unit includes a convolution layer with a convolution kernel size of n*n and a stride of 2, and the number of channels is sequentially decreased from 2 ^N+1 to 2 2 in multiples of ² ;

The first N-1 groups of "upsampling layer, regularization layer and activation function in activation function" are ReLU activation functions with negative slope L, and the activation function in the last group of "upsampling layer, regularization layer and activation function" is Tanh function;

Step 3: Constructing a proxy task, specifically: using the images in the training set to construct a reconstruction-based proxy task, that is, using local pixel transformation, image brightness nonlinear transformation, and local area patch methods to construct a proxy task, which specifically includes the following sub-steps :

Step 3.1 Local pixel conversion, that is to randomly exchange pixel values at different positions in the local area of the image, and output a randomly transformed image, specifically: randomly select M ₁ integers with a size between [1, 7] in the image. , image block, and randomly swap pixels inside the image block;

Step 3.2 The image brightness is nonlinearly transformed, and the nonlinearly transformed image is output, specifically: constructing a Bezier mapping curve B(t) based on formula (1) through three randomly given control points, and then completing the calculation based on the curve. A map of image pixel values:

B(t)=(1-t) ² P ₀ +2t(1-t)P ₁ +t ² P ₂ ,t∈[0,1] (1)

Among them, t represents the brightness of the pixel, and P ₀ , P ₁ , and P ₂ are three control points obtained randomly;

Step 3.3 Local area patching, that is, filling the randomly selected area in the image with random pixel values to obtain the image after the local area patching;

间整数的图像块，将图像块中包含的像素以从均匀分布中获取的随机噪声值作为填充；Randomly select M ₂ in the image of size as

An image block between integers, and the pixels contained in the image block are filled with random noise values obtained from a uniform distribution;

Step 4: Construct a total loss function, specifically the weighted sum of image reconstruction loss, feature reconstruction loss, discrimination loss between image space and feature space, and restoration loss based on proxy task;

采用L1损失函数，约束真实图像与重构图像的差异，计算公式如(2)：Among them, the image reconstruction loss

The L1 loss function is used to constrain the difference between the real image and the reconstructed image. The calculation formula is as follows (2):

分别代表前述的解码器与编码器；

表示对输入图像x进行编码得到的编码输出，

表示对输入图像x的编码输出，再进行解码，得到解码输出；‖·‖₁表示1范数；where x represents the input image,

respectively represent the aforementioned decoder and encoder;

represents the encoded output obtained by encoding the input image x,

Represents the encoded output of the input image x, and then decodes it to obtain the decoded output; ‖·‖ ₁ represents the 1 norm;

同样利用L1损失函数，在特征空间中的图像的特征表达与重构图像特征表达的差异性，计算公式如(3)：Feature reconstruction loss

Also using the L1 loss function, the difference between the feature expression of the image in the feature space and the feature expression of the reconstructed image is calculated as (3):

表示对输入特征向量z进行解码获得的解码输出；

表示对输入特征向量z先进行解码再进行编码的输出；Among them, z is the feature vector of the image essence output by the encoder in step 2.1;

represents the decoded output obtained by decoding the input feature vector z;

Represents the output of decoding the input feature vector z first and then encoding it;

约束由编码器与解码器的输出与真实空间中图像与特征的差异，计算公式如(4)：Discriminator loss in image space and feature space

The constraint is determined by the difference between the output of the encoder and the decoder and the image and features in the real space. The calculation formula is as follows (4):

为输入图像x经编码器、判别器2输出的结果；

为特征向量z经解码器、判别器1输出的结果；D_I、D_F通过下述损失公式

(5)和(6)进行迭代优化：Among them, D _I and _DF are the discriminator 1 of the image space and the discriminator 2 of the feature space, respectively;

is the result output by the encoder and the discriminator 2 for the input image x;

is the result of the feature vector z output by the decoder and the discriminator 1; D _I , D _F through the following loss formula

(5) and (6) are iteratively optimized:

Wherein, D _I (x) is the output result of the input image x through the discriminator 1, and D _F (z) is the output result of the feature vector z through the discriminator 2;

利用复原的代理任务，强化编码器对于图像特征的提取能力，计算公式如(7)：Proxy task-based restoration loss

Using the restored proxy task to strengthen the encoder's ability to extract image features, the calculation formula is as follows (7):

代表复原解码器；x_a代表由代理任务获得的变换图像输入；

代表变换图像x_a经编码器、复原解码器后的输出；in,

represents the restoration decoder; x _a represents the transformed image input obtained by the proxy task;

represents the output of the transformed image x _a after the encoder and the restoration decoder;

The total loss function is calculated as formula (8):

Among them, a is the image reconstruction loss, b is the feature reconstruction loss, c is the discrimination loss between the image space and the feature space, and d is the weight coefficient of the partial loss;

Step 5: Model training to obtain the trained encoder and decoder, including the following sub-steps:

前向传播获得图像特征向量，输入解码器

中，前向获得重构图像；再将重构图像输入编码器中，前向传播获得重构的特征向量；以步骤3.1-步骤3.3构建的三种图像为编码-复原解码器的输入，原始图像为训练标签，完成对于代理任务的学习，具体为：对输入的健康眼底图像按照代理任务随机变换，输入编码器与复原解码器中，前向获得复原图像；Step 5.1 Input the healthy fundus images of normal people into the encoder

The image feature vector is obtained by forward propagation, which is input to the decoder

, the reconstructed image is obtained in the forward direction; then the reconstructed image is input into the encoder, and the reconstructed feature vector is obtained by forward propagation; the three images constructed in step 3.1-step 3.3 are used as the input of the encoding-restoring decoder, the original The image is a training label to complete the learning of the proxy task, specifically: randomly transform the input healthy fundus image according to the proxy task, input it into the encoder and the restoration decoder, and obtain the restored image forward;

Step 5.2 Calculate image reconstruction loss and feature reconstruction loss;

Step 5.3 Input the reconstructed image and the reconstructed feature vector into the discriminators 1 and 2 of the image space and the feature space respectively, and calculate the discrimination loss of the image space and the feature space;

Step 5.4 Calculate the recovery loss of the agent task;

In step 5.5, backpropagation and parameter optimization are performed, and the discriminator and the encoder-decoder are optimized alternately, and the discriminator is optimized first, and then the encoder-decoder is optimized;

Step 5.6 Repeat steps 5.1-5.5. After traversing all the images in the training set, record the total loss function value in this process and draw a curve. When the loss curve converges smoothly, adjust the learning rate so that the model can continue to learn;

Step 5.7 Save the trained encoder and decoder;

Step 6: Use the trained encoder and decoder to test the images of the test set, select a threshold for judging large batches of images, and output the conclusion of whether the images are normal or not, specifically;

Step 6.1 The input image obtains the image feature vector through the encoder, then obtains the reconstructed image through the decoder, and re-inputs the reconstructed image into the encoder to obtain the reconstructed feature vector;

Step 6.2 Calculate the difference d1 between the input image and the reconstructed image, and calculate the difference d2 between the image feature vector and the reconstructed feature vector;

Step 6.3 Average the difference d1 and d2 of the two parts as the score value of the input image. The larger the score value, the higher the probability that the image is a diseased image, and vice versa; the optimal threshold is selected according to the label of the test set, such as If it is greater than the threshold, the input image is determined to be a diseased image, otherwise, it is a normal image;

So far, from steps 1 to 6, the disease diagnosis method without lesion image training has been completed.

beneficial effect

The present invention is an intelligent diagnosis method for fundus diseases based on non-pathological image training. Compared with the existing disease diagnosis algorithm, the invention has the following beneficial effects:

1. The method directly uses the images of healthy people for training, and does not require a data balance training set with lesions, which effectively solves the problem that existing algorithms rely on disease image data, and caters to the current application status of scarce clinical disease data;

2. The method constrains image reconstruction through two dimensions of image space and feature space, and improves the feature perception ability of the model for normal images;

3. On the basis of reconstruction, the method proposes a discriminative loss between images and feature spaces, which further improves the structural parts of the model: the encoder and decoder feature learning capabilities for the trained healthy images; further improves the model response Recognition ability of disease images;

4. The method proposed for the proxy task, under the condition of a certain amount of data, through different forms of restoration tasks, the model can learn the boundary and structural information of the deep image, which provides help in the task of reconstructing healthy images. , which improves the performance of the model in disease recognition.

Description of drawings

1 is a schematic diagram of the network structure on which an intelligent diagnosis method for fundus diseases based on disease-free image training of the present invention is based;

2 is a schematic flowchart of an example of an intelligent diagnosis method for fundus diseases based on disease-free image training according to the present invention;

Fig. 3 is a kind of test flow of the present invention's method for intelligent diagnosis of fundus diseases based on non-pathological image training;

FIG. 4 is a ROC curve diagram of an intelligent diagnosis method for fundus diseases based on non-pathological image training of the present invention compared with the existing method.

Detailed ways

A method for intelligent diagnosis of fundus diseases based on disease-free image training of the present invention will be further described and described in detail below with reference to the accompanying drawings and embodiments.

Example 1

This embodiment describes the specific implementation of the method for intelligent diagnosis of fundus diseases based on non-lesion image training according to the present invention.

The present invention can be applied to the screening of diseases in hospitals and medical institutions of different scales. By collecting medical images, classifying and recognizing models of the patients, whether the patients are healthy or sick individuals can be determined according to the difference between the images in step 6. . In medical institutions of different scales, the uniform image pixel size W can be adjusted according to the computing power of devices in different locations.

为编码器，

为解码器，

为复原解码器，D_F为特征空间的判别器，D_I为图像空间的判别器，x为输入图像，z为图像的特征向量，

为重构图像，x′_a为代理任务的复原图像；通过图像空间与特征空间的重构损失、图像空间与特征空间的判别损失与代理任务的复原损失对模型进行参数更新；FIG. 1 is a schematic diagram of the network structure on which an intelligent diagnosis method for fundus diseases based on disease-free image training of the present invention is based. In Figure 1

for the encoder,

for the decoder,

In order to restore the decoder, D _F is the discriminator of the feature space, D _I is the discriminator of the image space, x is the input image, z is the feature vector of the image,

In order to reconstruct the image, x′ _a is the restored image of the proxy task; the parameters of the model are updated through the reconstruction loss of the image space and the feature space, the discriminative loss of the image space and the feature space and the restoration loss of the proxy task;

Fig. 2 is a flowchart of the disease identification algorithm in the specific embodiment of the present invention, taking the fundus OCT image as a specific example, including the following steps:

Step A: Construct training set and test set;

10,000 images obtained by optical coherence tomography (OCT) were used as the implementation images, and only the healthy data without lesions was taken as the training set; at the same time, the same amount of 200 healthy data and 200 lesion data were taken as the test set;

After the image quality is filtered and the unqualified data with too high blur and too dark is removed, the image is scaled to the pixel size of 256*256*1, and normalized to the pixel value range of [-1,1];

After step A, the construction of the training set and the test set is completed;

Step B: Construction of the network model;

Build the encoder, decoder, restoration decoder, discriminator model 1, and discriminator model 2; update the model parameters according to the loss function, and save the trained models, such as encoder and decoder, for use model testing;

Step B.1 The encoder model includes 8 groups of downsampling layers, regularization layers and activation functions in series; the input of the encoder model is the image processed in step A with a dimension of 256*256*1, and the output is 1*1*512 The downsampling layer is a convolutional layer with a convolution kernel size of 4*4 and a stride of 2. The number of channels increases from 4 to 512 in multiples of 2; the activation function is LeakyReLU activation with a negative slope of 0.2 function;

Step B.2 The decoder includes 8 groups of upsampling layers, regularization layers and activation functions connected in series; the input of the decoder model is the output vector of the encoder, and the output is an image with a dimension of 256*256*1; the upsampling layer includes It has a convolutional layer with a convolution kernel size of 4*4 and a stride of 2. The number of channels decreases from 512 to 4 in multiples of 2; the activation function is the ReLU activation function with a negative slope of 0.2, and the outermost activation function is Tanh function;

Step B.3 The restoration decoder includes 8 groups of upsampling layers, regularization layers and activation functions connected in series; the input of the decoder model is the output vector of the encoder, and the output is an image with a dimension of 256*256*1; in the upsampling unit Including a convolutional layer with a convolution kernel size of 4*4 and a stride of 2, the number of channels 512 is decreased to 4 in multiples of 2; the activation function is the ReLU activation function with a negative slope of 0.2, and the outermost activation function is Tanh function;

Step B.4 The input of the discriminator model 1 is the feature vector z output by the encoder, including 7 serially connected layers, dropout layers and activation functions; step D.4 of each layer of the fully connected layer uses the discriminator model, using The adversarial discriminant loss improves the encoder’s ability to encode images and the decoder’s ability to reconstruct images from feature vectors; it makes the model more responsive to non-training images (disease images) and improves the model’s ability to recognize diseases ability.

The number of neurons is decreased from 128 to 1 in multiples of 2; the random drop parameter of the Dropout layer is 0.5; the activation function is LeakyReLU with a negative slope of 0.2 except that the activation function of the last layer is sigmoid;

Step B.5 The discriminator model 2 includes 5 convolutional layers; the first 4 convolutional layers include convolutional layers, regularization layers and activation functions; For the convolutional layer of 2, the number of channels increases from 4 to 32 in multiples of 2; the activation function is LeakyReLU with a negative slope of 0.2, and the last layer is directly output after convolution;

Step C: the construction of agency task;

In the training image of step A, randomly perform local pixel conversion, luminance nonlinear conversion and a certain conversion in the local area patch for each image;

Local pixel conversion uses randomly selected 6 image blocks whose size is an integer between [1, 6], and randomly exchanges pixels in each selected image block;

The non-linear brightness transformation uses the Bezier mapping curve constructed in formula (1) to control the pixel value of the image and complete the transformation of the pixel value;

The local area patch randomly selects 6 image blocks whose size is an integer between [42, 51] in the image, and fills the pixels contained in the image block with random noise values obtained from a uniform distribution;

Step D: Loss function and model training;

其中

代表健康的训练图像；x_a为经过代理任务处理后的图像；本方法通过优化加权的损失函数

进行参数更新，具体内容如下：Step D.1 Build the model loss function, representing the data as

in

represents a healthy training image; x _a is the image processed by the proxy task; this method optimizes the weighted loss function

Update the parameters, the details are as follows:

Step D.2 Image Reconstruction Loss

Step D.3 Reconstruction Loss of Features

Step D.4 Discriminator Loss in Image Space and Feature Space

Step D.5 Recovery Loss for Proxy Tasks

Step D.6 Overall Loss Function

where a=1, b=10, c=10, d=10;

与特征重构损失

将重构的特征向量以及重构图像分别初入特征判别器模型1与特征判别器模型2中以计算判别器损失

通过步骤A.2的图像依次通过编码器-复原解码器中，依次获得复原特征向量与重构复原图像，计算图像的复原损失

按照步骤D.1的损失权重对最终的损失

进行反向传播与参数优化，优化方式选用Adam，批大小为32，每一批图像对判别器进行一次优化，再对生成器进行一次优化，学习率设为2e^-4；最后将训练好的模型保存；Step E: Model training, update and save the parameters; the training is shown in Figure 2. First, the processed normal images are input into the encoder-decoder-encoder in turn, and the feature vectors, reconstructed images and reconstructed features are obtained in turn. vector, computes the image reconstruction loss

with feature reconstruction loss

Enter the reconstructed feature vector and reconstructed image into feature discriminator model 1 and feature discriminator model 2 respectively to calculate the discriminator loss

The image in step A.2 passes through the encoder-recovery decoder in turn, obtains the restoration feature vector and reconstructs the restored image in turn, and calculates the restoration loss of the image

Follow the loss weights in step D.1 to the final loss

Backpropagation and parameter optimization are performed. The optimization method is Adam. The batch size is 32. The discriminator is optimized once for each batch of images, and the generator is optimized again. The learning rate is set to 2e ^-4 . Finally, the trained model save;

Wherein, the generator includes an encoder, a decoder and a recovery decoder;

Step F: Model testing and image classification;

As shown in Figure 3, the input image is sent into the model, the difference between the input image and the reconstructed image, the difference between the image feature and the reconstructed image feature are calculated, and the average of the two is calculated as the probability of the image to be judged as the lesion image, and the threshold is filtered. , to identify the lesion image.

So far, the whole process of the disease diagnosis method based on non-lesion images has been realized. The ROC curve of the experimental results is shown in Figure 4. The curve of this method is much higher than that of other methods, indicating that this method can complete the recognition of healthy images and lesion images well under the training condition of lack of lesion images, which makes up for the current situation. There is a problem that the algorithm cannot perform clinical classification tasks under the condition of missing categories.

The method directly uses the images of healthy people for training, and does not require a data balance training set with lesions, effectively solves the problem that existing algorithms rely on disease image data, and caters to the application status quo of clinical disease data is scarce, and is embodied in the following steps: Step A The requirement of the algorithm for the amount of data is reduced by the transformation of the image; the training process of step C solves the dependence of the existing algorithm on multi-category training images;

The method constrains image reconstruction through two dimensions of image space and feature space, and improves the feature perception ability of the model for normal images. Steps D.2 and D.3 use the L1 loss to compare the original input and the reconstructed output of the two spaces, and improve the model's ability to perceive image characteristics by constraining the difference;

On the basis of reconstruction, the method proposes a discriminative loss between images and feature spaces, which further improves each structural part of the model: the encoder and decoder's ability to learn the features of the trained healthy images; further improves the model's ability to deal with diseases The image recognition ability is reflected as follows: Step D.4 uses the discriminator model and adopts the adversarial discriminant loss to improve the encoder's ability to encode images, and at the same time improve the decoder's ability to reconstruct images from feature vectors; The response to non-training images (disease images) is large, which improves the model's ability to identify diseases.

The proposed method for the proxy task, under the condition of a certain amount of data, through different forms of restoration tasks, enables the model to learn the boundary, structure and other information of the deep image, which provides help for the task of healthy image reconstruction. Improve the performance of the model in disease recognition. In step C, the existing healthy images are transformed by proxy tasks, combined with the restoration loss of step D.6, the learning of the boundary and structural information of the model is improved according to different tasks, and the accuracy of the model for the reconstruction task is strengthened; The ability of the model to recognize disease images.

The above descriptions are only the preferred embodiments of the present invention, and the present invention should not be limited to the contents disclosed in the embodiments and the accompanying drawings. All equivalents or modifications accomplished without departing from the disclosed spirit of the present invention fall into the protection scope of the present invention.

Claims (10)

1. An intelligent diagnosis method for fundus diseases based on non-pathological image training is characterized in that: the method is realized by the following steps:

the method comprises the following steps: preprocessing the collected images to construct a data set; wherein, the data set is divided into a training set and a testing set;

step two: designing a network model comprising an encoder, a decoder, a discriminator 1, a discriminator 2 and a restoration decoder, and specifically comprising the following substeps:

step 2.1, constructing an encoder of the multilayer convolution layer;

the encoder comprises N groups of down-sampling layers, a regularization layer and an activation function which are connected in series;

step 2.2, constructing a decoder of the multilayer deconvolution layer;

the decoder comprises N groups of up-sampling layers, a regularization layer and an activation function which are connected in series;

step 2.3, constructing a discriminator 1 of a feature space by adopting a multi-layer perceptron structure;

step 2.4, constructing a discriminator 2 of an image space by adopting a PatchGAN structure;

wherein, the discriminator 2 comprises P convolution layers; the front P-1 convolutional layers comprise convolutional layers, regularization layers and activation functions;

step 2.5, a restoration decoder is constructed by adopting a plurality of layers of deconvolution layers;

the recovery decoder comprises N groups of up-sampling layers, a regularization layer and an activation function which are connected in series;

step three: constructing an agent task, specifically: the method comprises the following steps of constructing an agent task based on reconstruction by using images in a training set, namely constructing the agent task by adopting a local pixel conversion mode, an image brightness nonlinear transformation mode and a local region patch mode, and specifically comprising the following substeps:

step 3.1, local pixel conversion, namely, randomly exchanging pixel values of different positions of a local area in an image and outputting the image after random conversion;

step 3.2, carrying out nonlinear transformation on the image brightness, and outputting the image subjected to the nonlinear transformation;

step 3.3, local area patching, namely, carrying out random pixel value filling on the randomly selected area in the image to obtain the patched image of the local area;

step four: constructing a total loss function, specifically a weighted sum of image reconstruction loss, feature reconstruction loss, discrimination loss of an image space and a feature space and restoration loss based on an agent task;

step five: model training to obtain a trained encoder and decoder, comprising the following substeps:

step 5.1 inputting the healthy eyeground image of the normal person into the encoder

Forward propagation to obtain image feature vector, input to decoder

In the middle, a reconstructed image is obtained in the forward direction; inputting the reconstructed image into an encoder, and carrying out forward propagation to obtain a reconstructed feature vector; taking the three images constructed in the steps 3.1 to 3.3 as the input of a coding-restoring decoder, and the original image as a training label, the learning of the agent task is completed, and the method specifically comprises the following steps: randomly transforming the input healthy eye fundus images according to the proxy task, inputting the transformed healthy eye fundus images into an encoder and a recovery decoder, and obtaining a recovery image in the forward direction;

step 5.2, calculating image reconstruction loss and characteristic reconstruction loss;

step 5.3, inputting the reconstructed image and the reconstructed feature vector into discriminators 1 and 2 of an image space and a feature space respectively, and calculating discrimination loss of the image space and the feature space;

step 5.4, calculating the recovery loss of the agent task;

step 5.5, performing back propagation and parameter optimization, and optimizing the discriminator and the encoder-decoder by adopting a mode of alternately optimizing the discriminator and the encoder-decoder;

step 5.6, repeating the steps 5.1-5.5, traversing all images in the training set once, recording the value of the total loss function in the process, drawing a curve, and adjusting the learning rate after the loss curve is converged stably so as to facilitate the model to continue learning;

step 5.7, storing the trained encoder and decoder;

step six: testing the images of the test set by using a trained encoder and a trained decoder, selecting a threshold value for judging the large-batch images, and outputting a conclusion whether the images are normal or not, specifically;

step 6.1, the input image is processed by an encoder to obtain an image characteristic vector, then processed by a decoder to obtain a reconstructed image, and the reconstructed image is input into the encoder again to obtain the characteristic vector of the reconstructed image;

step 6.2, calculating the difference d1 between the input image and the reconstructed image, and calculating the difference d2 between the image feature vector and the reconstructed feature vector;

step 6.3, averaging the differences d1 and d2 of the two parts to be used as a score value of the input image, wherein the larger the score value is, the higher the probability that the image is a lesion image is, and the lower the probability is otherwise; and selecting an optimal threshold according to the label of the test set, and if the optimal threshold is larger than the optimal threshold, judging that the input image is a lesion image, otherwise, judging that the input image is a normal image.

2. An intelligent diagnosis method for fundus diseases based on non-pathological image training as claimed in claim 1, wherein: the method comprises the following steps: screening the collected images, eliminating images with poor image quality, unifying the resolution of the screened images into the same dimension W C, and normalizing the pixel values to the range of [ -1,1 ].

3. An intelligent diagnosis method for fundus diseases based on non-pathological image training as claimed in claim 2, wherein: c is 1 or more.

4. A method as claimed in claim 3, wherein the method comprises the following steps: the training set and the testing set both adopt clinically collected ophthalmic images; the images in the training set are formed by the ophthalmic images of healthy individuals, and the images in the testing set are a set of the images of the healthy individuals and the diseased individuals; the image with poor quality specifically includes: too dark picture, large deviation of shooting angle of view, and shooting a jittery image.

5. An intelligent diagnosis method for fundus diseases based on non-pathological image training as claimed in claim 4, wherein: in step 2.1 the encoder input is the image with dimension W x c after step one and the output is 1 x 2^N+1A feature vector z expressing the essence of the image; the value of N is less than or equal to log₂W; the down-sampling layer comprises convolution layer with convolution kernel size of n × n and step length of 2, and the number of channels is from 2²Sequentially increasing to 2 by multiple increment of 2^N+1(ii) a The activation function is a LeakyReLU activation function with a negative slope L; n is [3,5]](ii) a L is [0,1]]。

6. An intelligent diagnosis method for fundus diseases based on non-pathological image training as claimed in claim 5, wherein: in step 2.2, the decoder inputs the eigenvector z output by the encoder in step two, and the output is an image with the dimension of W x c; the up-sampling layer comprises convolution layer with convolution kernel size n × n and step length 2, and the number of channels is from 2^N+1Sequentially decreasing to 2 by multiple of 2²(ii) a The activation function is a ReLU activation function with a negative slope L, and the outermost activation function is a Tanh function.

7. An intelligent diagnosis method for fundus diseases based on non-pathological image training as claimed in claim 6, wherein: in step 2.3, the input of the discriminator 1 is the feature vector z expressing the essence of the image output in step 2.1; the discriminator 1 comprises K series-connected full-connection layers, a Dropout layer and an activation function; the number of neurons in each layer of the full-connection layer is from 2^KDecreasing by a factor of 2 to 2⁰(ii) a The random discard parameter of the Dropout layer is p; the activation functions are all K-1 activation functions except the activation function of the last layer which is sigmoid, and the other activation functions are LeakyReLU with the negative slope L; wherein, the value range of K is [5, log₂W](ii) a p has a value range of [0,1]]。

8. An intelligent diagnosis method for fundus diseases based on non-pathological image training as claimed in claim 7, wherein: in step 2.4, the convolution layer includes a convolution kernel having a convolution kernel size of n x n,Convolution layer with step length of 2 and channel number of 2²Sequentially increasing to 2 by multiple increment of 2^P(ii) a Directly outputting the convolution of the last layer, wherein the rest P-1 activation functions are LeakyReLU functions with negative slope L; wherein, the value range of P is [5, log₂W]；

In step 2.5, the decoder inputs the output vector of the encoder in step two and outputs the image with dimension W x c; the up-sampling unit comprises convolution layers with convolution kernel size of n x n and step length of 2, and the number of channels is from 2^N+1Sequentially decreasing to 2 by multiple of 2²(ii) a The activation function in the first N-1 groups of the up-sampling layer, the regularization layer and the activation function is a ReLU activation function with a negative slope L, and the activation function in the last group of the up-sampling layer, the regularization layer and the activation function is a Tanh function.

9. An intelligent diagnosis method for fundus diseases based on non-pathological image training as claimed in claim 8, wherein: step 3.1, specifically: randomly selecting M in an image₁Size of [1,7 ]]The pixels of the image blocks are randomly exchanged inside the image blocks;

step 3.2, specifically: constructing a Bezier mapping curve B (t) based on formula (1) through three control points which are given randomly, and completing mapping of image pixel values based on the curve:

B(t)＝(1-t)²P₀+2t(1-t)P₁+t²P₂，t∈[0，1] (1)

where t denotes the luminance of the pixel, P₀，P₁，P₂Three control points are randomly acquired;

step 3.3, randomly selecting M in the image₂Each size is

Inter-integer image blocks, the pixels contained in the image block are filled with random noise values obtained from a uniform distribution.

10. According to claim9 the intelligent diagnosis method for fundus diseases based on the training of the non-pathological image is characterized in that: in step four, image reconstruction is lost

And (3) constraining the difference between the real image and the reconstructed image by adopting an L1 loss function, and calculating the formula as (2):

wherein, x represents the input image,

respectively representing the decoder and the encoder;

representing the coded output resulting from coding the input image x,

representing the encoded output of the input image x, and then decoding to obtain a decoded output; II-₁Represents a norm of 1;

loss of feature reconstruction

The difference between the feature expression of the image in the feature space and the feature expression of the reconstructed image is calculated by using the L1 loss function, and the formula is shown as (3):

wherein z is a feature vector z of the essence of the image output by the encoder in the step 2.1;

represents a decoded output obtained by decoding the input feature vector z;

represents the output of decoding and then encoding the input eigenvector z;

discriminator loss of image space and feature space

The difference between the output of the encoder and the decoder and the image and the feature in the real space is constrained, and the calculation formula is as follows (4):

wherein D is_I、D_FA discriminator 1 of an image space and a discriminator 2 of a feature space, respectively;

is the output result of the input image x through the encoder and the discriminator 2;

is the result of the output of the characteristic vector z by the decoder and the discriminator 1; d_I、D_FBy the following loss equation

(5) And (6) performing iterative optimization:

wherein D is_I(x) For the input image x, the result, D, is output via a discriminator 1_F(z) the result of the feature vector z output by the discriminator 2;

agent task based recovery loss

And enhancing the extraction capability of the encoder on the image characteristics by utilizing the restored proxy task, wherein the calculation formula is as follows:

wherein,

represents a restoration decoder; x is the number of_aRepresenting transformed image input obtained by the agent task;

representing a transformed image x_aThe output after the encoder and the recovery decoder;

the final loss function is calculated as follows in equation (8):

where a is an image reconstruction loss, b is a feature reconstruction loss, c is a discrimination loss between an image space and a feature space, and d is a weight coefficient of a partial loss.

Images