WO2020211530A1

WO2020211530A1 - Model training method and apparatus for detection on fundus image, method and apparatus for detection on fundus image, computer device, and medium

Info

Publication number: WO2020211530A1
Application number: PCT/CN2020/076501
Authority: WO
Inventors: 张梦蕾
Original assignee: 京东方科技集团股份有限公司
Priority date: 2019-04-19
Filing date: 2020-02-25
Publication date: 2020-10-22
Also published as: CN110070531B; CN110070531A

Abstract

The present application relates to a model training method and apparatus for detection on a fundus image, a method and apparatus for detection on a fundus image, a computer device, and a medium. The model training method for detection on a fundus image comprises: dividing each of N fundus images in a fundus image training set into M superpixels, wherein N and M are both positive integers; performing training to obtain a first network model according to the M×N superpixels, the first network model being used for identifying each inputted superpixel as a key pixel or a background pixel during outputting; and performing training to obtain a second network model according to superpixels which are key pixels among the M×N superpixels, the second network model being used for identifying each inputted superpixel as pathological or non-pathological during outputting.

Description

Model training method and device for detecting fundus pictures, method and device for detecting fundus pictures, computer equipment and medium

Technical field

The present disclosure relates to the field of computer vision information, and in particular to a model training method for detecting fundus pictures, a method and device for detecting fundus pictures.

Background technique

The fundus is the tissue in the back of the eyeball, and the picture of the fundus is the fundus picture. Fundus pictures can be used to diagnose fundus diseases such as glaucoma and fundus macular degeneration, and can also provide reference for the diagnosis of diabetes, hypertension and other diseases.

At present, doctors have a long process of identifying and diagnosing fundus lesions, and it is easy to misdiagnose or miss diagnoses for small initial lesions. The analysis of fundus pictures through computer vision technology can not only assist doctors in rapid diagnosis, but also reduce the probability of misdiagnosis and missed diagnosis.

Summary of the invention

The embodiments of the present disclosure provide a model training method for detecting fundus pictures, a method and device for detecting fundus pictures.

On the one hand, an embodiment of the present disclosure provides a model training method for detecting fundus pictures, including: dividing each of the N fundus pictures in the fundus image training set into M superpixels; N and M are both positive integers; according to the M×N superpixels, a first network model is obtained through training; the first network model is used to identify each input superpixel as a key pixel during output Or background pixels; training to obtain a second network model according to the superpixels belonging to the key pixels in the M×N superpixels; the second network model is used to input each superpixel in the output When marked as diseased or non-pathological.

Optionally, training to obtain the first network model based on the M×N superpixels includes: constructing a deep neural network; each time at least one of the M×N superpixels is selected and input In the deep neural network; wherein, each of the M×N superpixels has been previously marked as a key pixel or a background pixel; and the output result of the deep neural network is compared with the superpixel in advance Compare the marking results of the deep neural network and train the network parameters of the deep neural network until the deep neural network identifies the superpixel as a key pixel or the correct rate of the background pixel is greater than or equal to the first Threshold to obtain the first network model.

Optionally, the deep neural network is a deep belief network.

Optionally, training to obtain a second network model according to the superpixels belonging to the key pixels in the M×N superpixels includes: constructing a convolutional neural network; each time selecting the M×N superpixels , At least one of the superpixels among all the superpixels belonging to the key pixel is input into the convolutional neural network; wherein each of the superpixels belonging to the key pixel has been previously marked as diseased or non-pathological; Compare the output result of the convolutional neural network with the pre-marked results of the superpixels belonging to key pixels, and train the network parameters of the convolutional neural network until the loss value of the convolutional neural network is less than or equal to The second threshold is used to obtain the second network model; the output result of the convolutional neural network includes the identification of the superpixel as diseased or non-pathological.

Optionally, the convolutional neural network is a combination of a residual network and an Inception network.

Optionally, before dividing each of the N fundus pictures in the fundus picture training set into M superpixels, the model training method for detecting fundus pictures further includes: Perform pre-processing; the pre-processing includes at least one of rotation, shearing, distortion, scaling, adjusting color difference, and reducing resolution.

In another aspect, the embodiments of the present disclosure also provide a method for detecting fundus pictures, including: dividing the fundus picture to be detected into P superpixels, and obtaining the addresses corresponding to the P superpixels one-to-one; Input superpixels into the first network model to obtain the P superpixels identified as key pixels or background pixels; input the superpixels identified as key pixels into the second network model to obtain The non-lesion pixel is identified as the super pixel of the key pixel; according to the address corresponding to the super pixel identified as the key pixel and the lesion pixel, the position of the super pixel is found in the fundus image to be detected, and The position is marked on the fundus picture to be detected.

Optionally, before dividing the fundus image to be detected into P superpixels and obtaining the addresses corresponding to the P superpixels one-to-one, the method for detecting the fundus image further includes: pre-processing the fundus image to be detected Processing; The preprocessing includes: at least one of cropping and scaling.

Optionally, the first network model is obtained through the following training process: constructing a deep neural network; each time at least one of the M×N superpixels is selected and input into the deep neural network; where M ×N superpixels are obtained by dividing each of the N fundus pictures in the fundus picture training set into M superpixels, each of the M×N superpixels The super pixel has been pre-marked as a key pixel or background pixel; the output result of the deep neural network is compared with the pre-marked result of the super pixel, and the network parameters of the deep neural network are trained until the deep neural network When the network outputs the super pixel, the accuracy of the super pixel is identified as a key pixel or the background pixel is greater than or equal to a first threshold to obtain the first network model.

Optionally, the deep neural network is a deep belief network.

Optionally, the second network model is obtained through the following training process: constructing a convolutional neural network; each time selecting M×N superpixels, at least one of the superpixels belonging to the key pixels , Input into the convolutional neural network; wherein, the M×N superpixels are obtained by dividing each of the N fundus pictures in the fundus picture training set into M superpixels, Wherein, each of the superpixels belonging to the key pixel has been previously marked as a diseased pixel or a non-pathological pixel; the output result of the convolutional neural network is compared with the pre-marked result of the superpixel belonging to the key pixel, Train the network parameters of the convolutional neural network until the loss value of the convolutional neural network is less than or equal to the second threshold to obtain the second network model; the output result of the convolutional neural network includes The pixels are identified as lesion pixels or non-lesion pixels.

On the other hand, the embodiments of the present disclosure also provide a computer device, including a memory and a processor; the memory stores a computer program that can be run on the processor; when the processor executes the computer program, The aforementioned model training method for detecting fundus pictures or the aforementioned method for detecting fundus pictures.

In another aspect, embodiments of the present disclosure also provide a computer device, including a processor, which implements the aforementioned model training method for detecting fundus pictures or the aforementioned method for detecting fundus pictures when the processor executes a computer program. In another aspect, the embodiments of the present disclosure also provide a computer-readable medium storing a computer program that, when executed by a processor, implements the aforementioned model training method for detecting fundus pictures or the aforementioned fundus pictures The detection method.

In another aspect, an embodiment of the present invention also provides a model training device for detecting fundus pictures, including: a segmentation module configured to divide each of the N fundus pictures in the fundus picture training set, Are M superpixels; N and M are both positive integers; the training module is configured to train according to the M×N superpixels to obtain a first network model; the first network model is used to use each input The super pixels are identified as key pixels or background pixels when output; the training module is also configured to train to obtain a second network model according to the super pixels marked as key pixels; the second network model is used to input Each of the superpixels is marked as diseased or non-pathological when output.

In another aspect, an embodiment of the present invention also provides a detection device for a fundus picture, including: a segmentation module configured to divide the fundus picture to be detected into M superpixels; and an acquisition module configured to acquire the P superpixels One-to-one correspondence address; the obtaining module is also configured to input the P superpixels into the first network model, thereby obtaining the P superpixels identified as key pixels or background pixels; the obtaining module is also configured to The superpixels identified as key pixels are input into the second network model, so as to obtain the superpixels identified as key pixels, which are identified as diseased pixels or non-lesion pixels; the identification module is configured to be configured according to the identified key pixels and identified as The address corresponding to the super pixel of the diseased pixel, the position of the super pixel is found in the fundus picture to be detected, and the position is marked on the fundus picture to be detected.

Description of the drawings

In order to explain the embodiments of the present disclosure or the technical solutions in the prior art more clearly, the following will briefly introduce the drawings that need to be used in the description of the embodiments or the prior art. Obviously, the drawings in the following description are only These are some embodiments of the present disclosure. For those of ordinary skill in the art, other drawings can be obtained based on these drawings without creative work.

FIG. 1 is a flowchart of a model training method for detecting fundus pictures according to an embodiment of the disclosure;

2 is a flowchart of yet another model training method for detecting fundus pictures according to an embodiment of the disclosure;

FIG. 3 is a flowchart of another model training method for detecting fundus pictures according to an embodiment of the disclosure;

4 is a flowchart of yet another model training method for detecting fundus pictures provided by an embodiment of the disclosure;

FIG. 5 is a flowchart of another model training method for detecting fundus pictures according to an embodiment of the disclosure;

FIG. 6 is a schematic structural diagram of a model training device for detecting fundus pictures according to an embodiment of the disclosure;

FIG. 7 is a flowchart of a method for detecting fundus pictures according to an embodiment of the disclosure;

FIG. 8 is a flowchart of yet another method for detecting fundus pictures according to an embodiment of the present disclosure;

9 is a schematic structural diagram of a detection device for fundus pictures provided by an embodiment of the disclosure;

FIG. 10 is a schematic structural diagram of a computer device provided by an embodiment of the disclosure.

detailed description

The technical solutions in the embodiments of the present disclosure will be clearly and completely described below in conjunction with the accompanying drawings in the embodiments of the present disclosure. Obviously, the described embodiments are only a part of the embodiments of the present disclosure, not all of the embodiments. Based on the embodiments in the present disclosure, all other embodiments obtained by those of ordinary skill in the art without creative work shall fall within the protection scope of the present disclosure.

The embodiment of the present disclosure provides a model training method for detecting fundus pictures, as shown in FIG. 1, including:

S10. Divide each of the N fundus pictures in the fundus picture training set into M superpixels; N and M are both positive integers.

Among them, the fundus image training set refers to a collection of multiple fundus images used to train the model. The number of fundus pictures in the fundus picture training set can be set as needed.

Super pixels refer to irregular pixel blocks with a certain visual significance composed of adjacent pixels with similar texture, color, brightness and other characteristics. A small number of super pixels can replace a large number of pixels to express the characteristics of the fundus picture, which reduces the complexity of subsequent processing of the fundus picture.

As an example, a fundus picture is divided into M superpixels, that is, a large number of pixels in a fundus picture are replaced with M superpixels, which reduces the complexity of the fundus picture. On this basis, each fundus picture is divided into M superpixels, so that the complexity of all fundus pictures is reduced and reduced to the same degree. When training the model, it is easier to train and the error is smaller.

The method of dividing each fundus picture into M superpixels is called superpixel division. The principle of the superpixel segmentation method is based on a clustering algorithm, that is, the clustering algorithm is used for the segmentation of fundus pictures.

Based on the above description, the following takes a fundus picture as an example to provide a method of dividing the fundus picture into M superpixels to clearly describe the process:

First, set the number M of super pixels. In the fundus image, M pixels are uniformly selected as the initial cluster centers. For the remaining pixels, according to the distance between the remaining pixels and these cluster centers, according to the principle of nearest neighbor, respectively Assign to the closest cluster.

Second, recalculate the cluster center (the mean value of all pixels in the cluster) of each new cluster obtained, and repeat this process until the cluster center changes very little, then the superpixel segmentation is completed.

S20. According to the M×N superpixels, a first network model is obtained through training; the first network model is used to identify each input superpixel as a key pixel or a background pixel during output.

Among them, when training the first network model, M×N superpixels need to be input in batches, and the number of superpixels input each time can be set as required.

In fundus pictures, the super pixels that best reflect the content of the fundus pictures are called key pixels, and the remaining super pixels are called background pixels. Distinguishing between key pixels and background pixels for all superpixels can eliminate the interference of background pixels in the fundus picture, which is close to the user's detection intention, which is beneficial to the improvement of detection performance.

S30. According to the superpixels belonging to the key pixels in the M×N superpixels, a second network model is obtained by training; the second network model is used to identify each input superpixel as a diseased or non-pathological when outputting.

Among them, when training the second network model, key pixels also need to be input in batches, and the number of key pixels input each time can be set according to needs.

Among the key pixels, the key pixel that best reflects the information of the fundus lesion is marked as a lesion, and the remaining key pixels are marked as non-lesion. Distinguishing the key pixels between lesions and non-lesions can eliminate the interference of non-lesion super-pixels in fundus pictures and realize the user's detection intention.

The embodiment of the present application provides a model training method for detecting fundus pictures. The first network model is trained by using the superpixels to train the first network model by dividing the fundus pictures in the fundus image training set into multiple superpixels. In subsequent applications, the super pixels can be identified as key pixels or background pixels. On this basis, use key pixels to train the second network model so that the second network model can identify the key pixels as lesions or non-lesions in subsequent applications, so that the trained model can recognize lesions through the above simple training method Fast speed and high accuracy rate.

Optionally, before dividing each of the N fundus pictures in the fundus image training set into M superpixels in the above S10, as shown in FIG. 2, the above-mentioned model training method for detecting fundus pictures further includes :

S40. Perform first preprocessing on the fundus picture.

The first preprocessing includes at least one of rotation, shearing, distortion, scaling, adjusting color difference, and reducing resolution.

Rotation is to randomly rotate the fundus picture by a certain angle with the center or a certain vertex as the origin; cutting is to randomly select a part of the image; distortion is to apply a random four-point perspective transformation to the image; scaling is to unify the size of the fundus picture; To adjust the color difference is to randomly process the hue and saturation of the fundus picture.

Before training the model, perform the first preprocessing on the fundus picture to correct the content of the fundus picture, which can expand the fundus picture training set, so that the trained model can process images taken under various shooting conditions and improve model recognition Accuracy.

In addition, when the first preprocessing includes reducing the resolution, by using a low-resolution fundus picture to train the model, the trained model will have a more accurate recognition effect during the actual detection of fundus lesions.

Optionally, after the first network model is obtained by training based on M×N superpixels in the above S20, before S30 is trained based on the superpixels belonging to the key pixels in the M×N superpixels, before the second network model is obtained, as shown in Figure 3 As shown, the model training method for detecting fundus pictures also includes:

S50. Delete the background pixels output by the first network model.

The background pixels output by the first network model are deleted, and only key pixels are retained for subsequent processing, which reduces the amount of calculation and can increase the calculation speed.

Optionally, the first network model is obtained by training according to M×N superpixels in S20, as shown in FIG. 4, which includes:

S201. Construct a deep neural network.

Optionally, the deep neural network in S201 is a deep belief network (Deep Belief Network, DBN).

The deep belief network includes multiple stacked Restricted Boltzmann Machines (RBM). The structural principle of the restricted Boltzmann machine comes from the Boltzmann distribution in physics. Among them, each restricted Boltzmann machine has two layers of neurons. One layer is called the Visible Layer, which is composed of Visible Units and is used for input; the other layer is called Hidden Layer. , Composed of Hidden Units, used for detection. Both the explicit element and the hidden element are binary variables, that is, their state takes the value 0 or 1. In each layer of neurons, there is no connection within the layer, and the layers are fully connected.

When multiple stacked restricted Boltzmann machines form a deep belief network, the hidden layer of the lower restricted Boltzmann machine serves as the visible layer of the higher restricted Boltzmann machine, which is the upper layer of restricted glass. Erzmann machine input data.

The number of restricted Boltzmann machines stacked into a deep belief network can be set as required, which is not limited in the present disclosure.

S202. Each time at least one superpixel among the M×N superpixels is selected and input into the deep neural network; wherein, each of the M×N superpixels has been previously marked as a key pixel or a background pixel .

For example, each superpixel may be pre-marked by manual marking.

S203. Compare the output result of the deep neural network with the pre-marked result of the superpixel, and train the network parameters of the deep neural network, until the deep neural network outputs the superpixel, the correct rate of identifying the superpixel as a key pixel or background pixel If it is greater than or equal to the first threshold, the first network model is obtained.

Based on the above description, when the deep neural network is a deep belief network, the following provides a method of training the first network model based on M×N superpixels to clearly describe its implementation process.

First, construct a deep belief network, and set the deep belief network to be formed by stacking Q restricted Boltzmann machines. Q is a positive integer.

Secondly, each time at least one superpixel is input into the visible layer of the first restricted Boltzmann machine in the deep belief network, unsupervised training is performed, the features of the superpixel are extracted, and the weights are updated through the contrast divergence method. The hidden layer of the first restricted Boltzmann machine is used as the visible layer of the second restricted Boltzmann machine, features are extracted, and the weights are updated. And so on. Use the hidden layer of the Q-1 restricted Boltzmann machine as the visible layer of the Q restricted Boltzmann machine, and set up label neurons representing key pixels and background pixels, continue to extract features, and update Weights. Connect the hidden layer of the Qth restricted Boltzmann machine to the output layer to output.

It should be noted that unsupervised training refers to each restricted Boltzmann machine. In the training phase, the data input to the explicit layer does not need to be manually labeled.

The main steps of the Contrastive Divergence (CD) method include setting the explicit state of the restricted Boltzmann machine according to the superpixel, and calculating the hidden state by using the conditional probability of the hidden layer under the explicit condition; After the state of the hidden element is determined, the state of the next layer is calculated according to the conditional probability of the explicit layer under the condition of the hidden layer, the explicit layer is reconstructed, and sampling is repeated until the model parameters converge.

Then, the output result of the deep belief network is compared with the artificial labeling result, and the correct rate of all superpixels identified as key pixels or background pixels through the deep belief network is calculated. When the accuracy rate is very low, you can use the Error Back Propagation (BP) algorithm to calculate the mean square error of the deep belief network, and continuously adjust the network parameters to make the mean square error of the deep belief network less than or equal to the set value The third threshold, thus get the deep belief network.

Optionally, the second network model is obtained by training according to superpixels belonging to key pixels among the M×N superpixels in S30, as shown in FIG. 5, which includes:

S301. Construct a convolutional neural network.

The convolutional neural network model is a multi-layer structure learning algorithm that uses the relative spatial positions and weights in the picture to reduce the number of network weights to improve the performance of complex network training.

When convolutional neural network is trained, it is a machine learning model that learns under supervision.

Optionally, the convolutional neural network is a combination of the residual network and the Inception network.

The residual network, constructed by jump connection technology, breaks the convention that the output of the S-1 layer of the traditional neural network can only be input to the S layer, so that the output of a certain layer can directly cross several layers as a later layer input of. The stacking of multiple residual networks can reduce the number of network parameters, reduce the amount of calculation, and increase the calculation speed.

The Inception network is a network with a parallel structure. Through an asymmetric convolution kernel structure, it can reduce the amount of calculation and increase the speed of calculation while ensuring that the information loss is small enough.

S302. At least one superpixel among all the superpixels belonging to the key pixel is selected each time among the M×N superpixels, and input into the convolutional neural network; wherein, each superpixel belonging to the key pixel has been previously marked as a lesion Or non-pathological.

For example, each superpixel belonging to a key pixel may be pre-marked by manual marking.

S303. Compare the output result of the convolutional neural network with the pre-marked result of superpixels belonging to the key pixel, and train the network parameters of the convolutional neural network until the loss value of the convolutional neural network is less than or equal to the second threshold, and obtain the first 2. Network model: The output result of the convolutional neural network includes identifying superpixels as diseased or non-pathological.

Based on the above description, when the convolutional neural network is the combination of the residual network and the Inception network, the following provides a method for training the second network model based on the superpixels belonging to the key pixels in the M×N superpixels to make it clear Describe its realization process.

First, build a convolutional neural network.

The number of residual networks included in the convolutional neural network and the number of Inception networks can be set as required, and the present disclosure does not limit this.

Secondly, each time at least one superpixel belonging to the key pixel is input to the input layer of the convolutional neural network, and the residual network and the Inception network are used to convolve it, and the convolution result is input into the fully connected layer, which is marked as diseased or non-pathological After output.

Then, the output result of the convolutional neural network is compared with the artificial labeling result, and the loss value of all superpixels belonging to the key pixel is calculated. When the loss value is large, back propagation can be used to adjust the network parameters until the loss value is less than or equal to the second threshold, thereby obtaining a convolutional neural network.

Among them, the main function of the convolutional neural network is to divide the superpixels belonging to the key pixels into lesions or non-lesions, which is used as a classification model. At this time, the loss function for calculating the loss value uses the cross entropy (Cross Entroy Loss) loss function.

The relational expression of the cross entropy loss function is:

y _i represents the probability distribution of artificial labeling results, and y _i 'represents the probability distribution of the output results of the convolutional neural network.

Cross entropy describes the distance between two probability distributions. The larger the cross entropy, the greater the difference between the two. The smaller the cross entropy, the closer the two are.

It should be noted that before using the cross entropy loss function to calculate the loss value, since the output result of the convolutional neural network is not a probability distribution, it is also necessary to use Softmax regression to normalize the output result of the convolutional neural network to In the (0,1) interval, it becomes a probability distribution.

For example, input M×N superpixels into a convolutional neural network in batches, and output after convolution. The output result of a certain time is data containing K superpixels, 1≤K≤M×N, K is A positive integer, i represents the i-th superpixel. According to Softmax regression, the probability distribution of the output result of the i-th superpixel's convolutional neural network is:

The embodiment of the present disclosure also provides a computer device, as shown in FIG. 10, including a memory 100 and a processor 200; the memory 100 stores a computer program that can run on the processor 200; the processor 200 executes the computer program to realize the above Model training method for detecting fundus pictures. Memory may include, but is not limited to, disk drives, optical storage devices, solid-state storage devices, floppy disks, flexible disks, hard disks, tapes or any other magnetic media, compact disks or any other optical media, ROM (read only memory), RAM (random Access memory), cache memory and/or any other memory chip or cartridge, and/or any other medium from which the processor can read data, instructions and/or code. The processor may be any type of processor, and may include, but is not limited to, one or more general-purpose processors and/or one or more special-purpose processors (such as special-purpose processing chips). In some embodiments of the present disclosure, the computer device may not include the memory 100. Computer equipment can retrieve computer programs by accessing external or remote storage.

The embodiment of the present disclosure also provides a computer-readable medium storing a computer program, and the computer program is executed by a processor to implement the above-mentioned model training method for detecting fundus pictures. The embodiment of the present invention also provides a model training device for detecting fundus pictures, as shown in FIG. 6, including:

The dividing module 10 is configured to divide each fundus picture in the N fundus pictures in the fundus picture training set into M superpixels; N and M are both positive integers.

The training module 20 is configured to train a first network model according to M×N superpixels; the first network model is used to identify each input superpixel as a key pixel or a background pixel when outputting.

The training module 20 is also configured to train to obtain a second network model according to the superpixels that have been marked as key pixels; the second network model is used to identify each superpixel input as a diseased or non-pathological when outputting.

The embodiment of the present application provides a model training device for detecting fundus pictures. The fundus pictures in the fundus image training set are divided into multiple superpixels through the segmentation module, and then the training module is used to train the first network using the superpixels. The model enables the first network model to recognize superpixels as key pixels or background pixels, and continues to use the training module to train the second network model using key pixels, so that the second network model can recognize that the key pixels are lesions or non-lesions, thereby Through the above simple training device, a model that can quickly recognize lesions in fundus pictures with good recognition effect and high accuracy can be trained.

The embodiment of the present disclosure also provides a method for detecting fundus pictures, as shown in FIG. 7, including:

S100. Divide the fundus image to be detected into P superpixels, and obtain addresses corresponding to the P superpixels one-to-one.

S200: Input the P superpixels into the first network model obtained by the above-mentioned model training method for detecting fundus pictures, so as to obtain superpixels identified as key pixels.

S300: Input superpixels identified as key pixels into the second network model obtained by the above-mentioned model training method for detecting fundus pictures, so as to obtain superpixels identified as key pixels and with lesions.

S400: Find the position of the superpixel in the fundus picture to be detected according to the address corresponding to the superpixel identified as the key pixel and the lesion, and mark the position on the fundus picture to be detected.

Based on this, it is possible to sequentially traverse the addresses of superpixels identified as key pixels and lesions. The adjacent addresses are identified as key pixels and superpixels of the lesion are combined, and then the combined position is identified on the fundus image to be detected.

For example, an address L corresponding to a superpixel identified as a key pixel and a diseased superpixel is used as a seed pixel. According to the address L, it is retrieved whether the superpixels at adjacent addresses L-1 and L+1 are also key pixels and Lesions.

If not, the location is identified separately.

If at least one of them is yes, then use the super pixel at address L-1 or L+1 as the seed pixel to find whether the super pixel at the neighboring address of the seed pixel is a key pixel and diseased, and so on, until the super pixel at the neighboring address If none of the key pixels is a key pixel and the lesion is a lesion, then one search ends, and all the adjacent key pixels and superpixels of the lesion found before are merged and identified as positions. Then continue to traverse the next unidentified super pixel that belongs to the key pixel and the lesion.

In addition, when marking the location of the lesion on the fundus picture to be detected, the mark can be a circle, a dot, a check mark, etc., as long as the human eye can distinguish it from the fundus picture, and its shape and color are not limited in the present disclosure. .

The embodiments provided in the present disclosure provide a method for detecting fundus pictures. By dividing the fundus picture to be detected into multiple superpixels, the first network model obtained by training is used to identify the superpixels, and the key pixels are obtained. , The key pixels are input into the second network model obtained by training, the key pixels are identified, the superpixels of the lesions are obtained, and then according to the address of the superpixel, the position of the superpixel is found and marked in the fundus image. Therefore, the above method can quickly and accurately detect the pathological changes in the fundus picture, and when applied, it can assist the doctor in the rapid diagnosis and reduce the probability of misdiagnosis and missed diagnosis.

Optionally, before S100 divides the fundus image to be detected into P superpixels and obtains the one-to-one addresses of the P superpixels, as shown in FIG. 8, the model training method for detecting the fundus image further includes:

S500. Perform a second preprocessing on the fundus image to be detected; the second preprocessing includes at least one of cropping and zooming.

Before detecting the fundus picture, a second preprocessing is performed on the fundus picture to unify the size of the fundus picture, reduce adverse effects, and improve the accuracy of detection.

The embodiment of the present disclosure also provides a computer device, as shown in FIG. 10, including a memory 100 and a processor 200; the memory 100 stores a computer program that can run on the processor 200; the processor 200 executes the computer program to realize the above Detection method of fundus pictures.

The embodiment of the present disclosure also provides a computer-readable medium storing a computer program, and the computer program is executed by a processor to implement the above-mentioned method for detecting fundus pictures.

An embodiment of the present invention also provides a device for detecting fundus pictures, as shown in FIG. 9, including:

The segmentation module 10 is configured to segment the fundus image to be detected into M superpixels.

The obtaining module 30 is configured to obtain the addresses corresponding to the P superpixels one to one.

The obtaining module 30 is further configured to input the P superpixels into the first network model obtained by the above-mentioned model training method for detecting fundus pictures, so as to obtain superpixels identified as key pixels.

The acquiring module 30 is further configured to input the superpixels identified as key pixels into the second network model obtained by the above-mentioned model training method for detecting fundus pictures, so as to acquire the superpixels identified as key pixels and having lesions. Pixels.

The identification module 40 is configured to find the position of the super pixel in the fundus image to be detected according to the address corresponding to the super pixel identified as a key pixel and the lesion, and to identify the position on the fundus image to be detected.

The above are only specific implementations of the present disclosure, but the protection scope of the present disclosure is not limited thereto. Any person skilled in the art can easily think of changes or substitutions within the technical scope disclosed in the present disclosure. It should be covered within the protection scope of the present disclosure. Therefore, the protection scope of the present disclosure should be subject to the protection scope of the claims.

Claims

A model training method for detecting fundus pictures includes:

Divide each of the N fundus pictures in the fundus picture training set into M superpixels; N and M are both positive integers;

Training to obtain a first network model according to the M×N superpixels; the first network model is used to identify each input superpixel as a key pixel or a background pixel during output;

According to the superpixels belonging to the key pixels in the M×N superpixels, a second network model is obtained through training; the second network model is used to identify each of the input superpixels as a lesion when outputting Or non-pathological.
The model training method for detecting fundus pictures according to claim 1, wherein training the first network model according to the M×N superpixels comprises:

Build a deep neural network;

Each time at least one of the M×N superpixels is selected and input into the deep neural network; wherein, each of the M×N superpixels has been previously marked Are key pixels or background pixels;

The output result of the deep neural network is compared with the pre-labeled result of the super pixel, and the network parameters of the deep neural network are trained until the deep neural network outputs the super pixel, the super pixel The correct rate of the identified key pixel or background pixel is greater than or equal to the first threshold, and the first network model is obtained.
The model training method for detecting fundus pictures according to claim 2, wherein the deep neural network is a deep belief network.
The model training method for detecting fundus pictures according to any one of claims 1 to 3, wherein the second network model is obtained by training according to the superpixels belonging to the key pixels among the M×N superpixels, include:

Build a convolutional neural network;

Each time the M×N superpixels are selected, at least one of the superpixels belonging to the key pixel is input into the convolutional neural network; wherein, each of the key pixels is Super pixels have been pre-marked as lesions or non-lesions;

Compare the output result of the convolutional neural network with the pre-marked results of the superpixels belonging to key pixels, and train the network parameters of the convolutional neural network until the loss value of the convolutional neural network is less than or equal to The second threshold is used to obtain the second network model; the output result of the convolutional neural network includes the identification of the superpixel as diseased or non-pathological.
The model training method for detecting fundus pictures according to claim 4, wherein the convolutional neural network is a combination of a residual network and an Inception network.
The model training method for detecting fundus pictures according to claim 1, wherein before dividing each of the N fundus pictures in the fundus picture training set into M superpixels, the The model training method for detecting fundus pictures also includes:

Preprocessing the fundus picture;

The preprocessing includes at least one of rotation, shearing, distortion, scaling, adjusting color difference, and reducing resolution.
A method for detecting fundus pictures includes:

Dividing the fundus image to be detected into P superpixels, and obtaining addresses corresponding to the P superpixels one-to-one;

Input the P superpixels into the first network model to obtain the P superpixels identified as key pixels or background pixels;

Input the superpixels identified as key pixels into the second network model, thereby obtaining the superpixels identified as diseased pixels or non-pathological pixels as key pixels;

According to the address corresponding to the super pixel identified as a key pixel and identified as a diseased pixel, the position of the super pixel is found in the fundus picture to be detected, and the position is marked on the fundus picture to be detected.
The method for detecting a fundus picture according to claim 7, wherein before dividing the fundus picture to be detected into P superpixels, and obtaining the addresses corresponding to the P superpixels one-to-one, the detection method for the fundus picture further include:

Preprocessing the fundus image to be detected;

The preprocessing includes at least one of cropping and scaling.
The method for detecting fundus pictures according to claim 7, wherein the first network model is obtained through the following training process:

Build a deep neural network;

Each time at least one of the M×N superpixels is selected and input into the deep neural network; wherein, the M×N superpixels are selected from the N fundus pictures in the fundus picture training set. Each of the fundus images is obtained by dividing each of the fundus pictures into M superpixels, each of the M×N superpixels has been previously marked as a key pixel or a background pixel;

The output result of the deep neural network is compared with the pre-labeled result of the super pixel, and the network parameters of the deep neural network are trained until the deep neural network outputs the super pixel, the super pixel The correct rate of the identified key pixel or background pixel is greater than or equal to the first threshold, and the first network model is obtained.
The method for detecting fundus pictures according to claim 9, wherein the deep neural network is a deep belief network.
8. The method for detecting fundus pictures according to claim 7, wherein the second network model is obtained through the following training processing:

Build a convolutional neural network;

Each time M×N superpixels are selected, at least one of the superpixels belonging to the key pixel is input into the convolutional neural network; wherein, the M×N superpixels are passed through Obtained by dividing each of the N fundus pictures in the fundus picture training set into M superpixels, wherein each of the superpixels belonging to the key pixel has been previously marked as a lesion pixel or a non-lesion pixel Pixel

Compare the output result of the convolutional neural network with the pre-marked results of the superpixels belonging to key pixels, and train the network parameters of the convolutional neural network until the loss value of the convolutional neural network is less than or equal to The second threshold is used to obtain the second network model; the output result of the convolutional neural network includes identifying the superpixel as a diseased pixel or a non-pathological pixel.
The method for detecting fundus pictures according to claim 11, wherein the convolutional neural network is a combination of a residual network and an Inception network.
A computer device, comprising a memory and a processor; the memory stores a computer program that can be run on the processor; when the processor executes the computer program, the implementation of any one of claims 1-6 The model training method for detecting fundus pictures or the method for detecting fundus pictures according to any one of claims 7-8.
A computer device, comprising a processor, which implements the model training method for detecting fundus pictures according to any one of claims 1-6 or the method according to any one of claims 7-8 when executing a computer program The detection method of fundus pictures described.
A computer-readable medium, which stores a computer program that, when executed by a processor, implements the model training method for detecting fundus pictures according to any one of claims 1-6 or according to claim 7- 8. The detection method of any of the fundus pictures.
A model training device for detecting fundus pictures includes:

A segmentation module, configured to divide each of the N fundus pictures in the fundus picture training set into M superpixels; N and M are both positive integers;

The training module is configured to train to obtain a first network model according to the M×N superpixels; the first network model is used to identify each input superpixel as a key pixel or a background pixel when outputting ；

The training module is further configured to train to obtain a second network model according to the superpixels that have been marked as key pixels; the second network model is used to identify each of the input superpixels as lesions or Non-pathological.
A detection device for fundus pictures includes:

A segmentation module, configured to segment the fundus image to be detected into M superpixels;

An obtaining module configured to obtain the addresses corresponding to the P superpixels one-to-one;

The obtaining module is further configured to input the P superpixels into the first network model, so as to obtain the P superpixels identified as key pixels or background pixels;

The acquiring module is further configured to input the superpixels identified as key pixels into the second network model, so as to acquire the superpixels identified as diseased pixels or non-pathological pixels as key pixels;

The identification module is configured to find the position of the super pixel in the fundus picture to be detected according to the address corresponding to the super pixel that is identified as a key pixel and a diseased pixel, and to mark it on the fundus picture to be detected The location.