WO2018201647A1 - Method for detecting retinopathy degree level, device and storage medium - Google Patents

Abstract

A method for detecting retinopathy degree level, comprising: after a retinopathy image to be identified is received, using a predetermined identification model to identify the received retinopathy image, and outputting an identification result (S10), wherein the predetermined identification model is a convolutional neural network model obtained by training, in advance, a preset number of sample images marked by different retinopathy degree levels; and determining the retinopathy degree level corresponding to the output identification result according to a predetermined mapping relationship between the identification result and the retinopathy degree level (S20). With the described method, it is not necessary to perform complex feature extraction operations on eye images, the process is simplified, corresponding different retinopathy degree levels may be determined according to the identification result, and refined identification of the retinopathy degree levels of a patient may be achieved. Further provided are an electronic device and a computer readable storage medium.

Description

Method, device and storage medium for detecting degree of retinopathy

This application is based on the priority of the Chinese Patent Application entitled "Retinopathy Degree Level Detection System and Method" filed on May 5, 2017, with the application number of CN201710312327.6, which is filed on May 5, 2017. The manner of reference is incorporated in the present application.

Technical field

The present invention relates to the field of computer technologies, and in particular, to a method for detecting a degree of retinopathy degree, an electronic device, and a computer readable storage medium.

Background technique

According to a survey of more than 93 million laborers in developed countries, diabetic retinopathy is a leading cause of blindness in the eye. Currently, the identification of diabetic retinopathy usually requires feature extraction of the ocular image (for example, ocular vascular structure). The extraction of features such as optic disc and retinal center groove, the feature extraction algorithm has poor performance and poor performance. At the same time, it is difficult to refine the degree of retinopathy of patients, and the recognition accuracy is difficult to meet the requirements.

Summary of the invention

The main object of the present invention is to provide a method for detecting the degree of retinopathy degree, an electronic device and a computer readable storage medium, which are intended to accurately and accurately identify the degree of retinopathy of a patient.

In order to achieve the above object, an electronic device according to the present invention includes: a memory, a processor, and a memory of a retinopathy degree level detecting system operable on the processor. The program, when the program of the retinopathy degree level detection system is executed by the processor, implements the following steps:

After receiving the picture of the retinopathy to be identified, the received retinopathy picture is identified by using a predetermined recognition model, and the recognition result is output; wherein the predetermined recognition model is pre-marked by different retinopathy a convolutional neural network model obtained by training a predetermined number of sample pictures of a degree level;

According to the mapping relationship between the predetermined recognition result and the degree of retinopathy degree, the degree of retinopathy corresponding to the output recognition result is determined.

Preferably, the training process of the predetermined recognition model is as follows:

A. setting a corresponding preset number of sample pictures for each preset retinopathy degree level, and marking a corresponding retinopathy degree level for each sample picture;

B. Perform image preprocessing on each sample image to obtain a training picture to be trained by the model;

C. Divide all training pictures into a training set of a first ratio and a verification set of a second ratio;

D. training the predetermined recognition model by using the training set;

E. verifying the accuracy of the trained recognition model by using the verification set, if the accuracy is greater than or equal At the preset accuracy rate, the training ends, or if the accuracy rate is less than the preset accuracy rate, the number of sample pictures corresponding to each retinopathy degree level is increased and the above steps B, C, D, E are re-executed.

In addition, in order to achieve the above object, the present invention also provides a method for detecting a degree of retinopathy degree, the method comprising the following steps:

After receiving the picture of the retinopathy to be identified, the received retinopathy picture is identified by using a predetermined recognition model, and the recognition result is output; wherein the predetermined recognition model is pre-marked by different retinopathy a convolutional neural network model obtained by training a predetermined number of sample pictures of a degree level;

According to the mapping relationship between the predetermined recognition result and the degree of retinopathy degree, the degree of retinopathy corresponding to the output recognition result is determined.

Preferably, the training process of the predetermined recognition model is as follows:

A. setting a corresponding preset number of sample pictures for each preset retinopathy degree level, and marking a corresponding retinopathy degree level for each sample picture;

B. Perform image preprocessing on each sample image to obtain a training picture to be trained by the model;

C. Divide all training pictures into a training set of a first ratio and a verification set of a second ratio;

D. training the predetermined recognition model by using the training set;

E. verifying the accuracy of the training recognition model by using the verification set. If the accuracy rate is greater than or equal to the preset accuracy rate, the training ends, or if the accuracy rate is less than the preset accuracy rate, increasing the level of each retinopathy level The number of sample pictures and re-execute steps B, C, D, E above.

In addition, in order to achieve the above object, the present invention also provides a computer readable storage medium storing a program of a detection system for a degree of retinopathy degree, a program of a detection system for a degree of retinopathy Any step of the method of detecting the level of retinopathy as described above when executed by the processor.

The retinopathy degree level detecting method, the electronic device and the computer readable storage medium proposed by the present invention are received by a deep convolutional neural network model based on a preset number of sample pictures marked with different retinopathy degree levels. The retinopathy picture is identified, and the corresponding retinopathy degree level is determined according to the recognition result. Since only the pre-trained deep convolutional neural network model is needed to recognize the received retinal lesion image, it is not necessary to perform complex feature extraction operation on the eye image, which is simpler and can determine corresponding different retinopathy according to the recognition result. Degree level, can effectively identify the degree of retinopathy of patients.

DRAWINGS

1 is a schematic flow chart of a preferred embodiment of a method for detecting a degree of retinopathy of the present invention;

2 is a schematic view of a preferred embodiment of an electronic device of the present invention;

3 is a functional block diagram of a preferred embodiment of the retinopathy lesion level detecting system of FIG. 2.

The implementation, functional features, and advantages of the present invention will be further described in conjunction with the embodiments.

detailed description

The present invention will be further described in detail below with reference to the accompanying drawings and embodiments, in order to make the present invention. It is understood that the specific embodiments described herein are merely illustrative of the invention and are not intended to limit the invention.

The invention provides a method for detecting the degree of retinopathy.

Referring to FIG. 1, FIG. 1 is a schematic flow chart of a preferred embodiment of a method for detecting a degree of retinopathy of the present invention.

In this embodiment, the method for detecting the degree of retinopathy is as follows:

Step S10: After receiving the retinopathy picture to be identified, the received retinopathy picture is identified by using a predetermined recognition model, and the recognition result is output; wherein the predetermined recognition model is pre-marked by A deep convolutional neural network model obtained by training a preset number of sample images of different degrees of retinopathy.

In this embodiment, the retinopathy degree level detecting system receives a retinopathy degree level detection request sent by the user and includes a retinopathy degree image to be recognized, for example, receiving a degree of retinopathy transmitted by the user through a terminal such as a mobile phone, a tablet computer, or a self-service terminal device. A level detection request, such as receiving a retinopathy degree level detection request sent by a user on a pre-installed client in a terminal such as a mobile phone, a tablet computer, a self-service terminal device, or receiving a user in a terminal such as a mobile phone, a tablet computer, or a self-service terminal device. Retinopathy level level detection request sent on the browser system.

The retinopathy degree level detection system recognizes the received retinopathy image to be recognized by using the pre-trained recognition model after receiving the request for the retinopathy degree level detection issued by the user, and recognizes the retinopathy to be recognized that is to be recognized. The recognition result of the picture in the recognition model. The recognition model can be continuously trained, learned, verified, optimized, etc. by identifying a plurality of preset number of sample pictures marked with different degrees of retinopathy degree, so as to train them to accurately identify different levels of retinopathy. The model of the annotation. For example, the recognition model may employ a Convolutional Neural Network (CNN) model or the like.

Step S20: Determine a retinopathy degree level corresponding to the output recognition result according to a mapping relationship between the predetermined recognition result and the retinopathy degree level.

After obtaining the recognition result by using the pre-trained deep convolutional neural network model to obtain the recognition result, the mapping result of the predetermined recognition result and the retinopathy degree level may be determined according to the output recognition result. The degree of retinopathy is determined by the degree of retinopathy level of the retinopathy image received. For example, in one embodiment, the recognition result includes a first recognition result (eg, labeled "0"), a second recognition result (eg, labeled "1"), and a third recognition result (eg, an annotation a "2"), a fourth recognition result (for example, labeled "3"), and a fifth recognition result (for example, labeled "4"), the retinopathy degree level includes a first level, a second level, and a Three levels, fourth level and fifth level. The mapping relationship between the different recognition results and the retinopathy degree level may be determined in advance, if the first recognition result corresponds to the first level, the second recognition result corresponds to the second level, and the third recognition result corresponds to the third level, and the fourth recognition result corresponds to The fourth level, the fifth recognition result corresponds to the fifth level. For example, specifically, the first level may correspond to normal and mild non-proliferative diabetic retinopathy, and the first level corresponding retinopathy picture appears as only individual hemangiomas, hard exudation, retinal hemorrhage, and the like. The second grade can correspond to non-proliferative diabetic retinopathy without clinically significant macular edema. The second grade corresponding retinopathy picture shows microangioma, hard exudation, retinal hemorrhage, sickle or beaded vein. The third grade can correspond to non-proliferative diabetic retinopathy with clinically significant macular edema (CSME). The third grade corresponding retinopathy picture shows retinal thickening in the macular area and its vicinity, and microangioma, soft infiltration Out, retinal hemorrhage. The fourth grade can correspond to non-high-risk proliferative retinopathy. The fourth-level retinal lesion picture shows neovascularization in the outer area of the optic papilla and proliferative changes in retinal microvessel formation in other areas. The fifth grade can correspond to high-risk proliferative retinopathy. The fifth grade corresponding retinopathy picture shows neovascularization, vitreous or preretinal hemorrhage in the optic papilla area.

In this way, after the received retinopathy image is recognized by the pre-trained recognition model to obtain the recognition result, the corresponding different retinopathy degree levels can be determined according to the obtained different recognition results, thereby realizing various refinements. Accurate identification of the degree of retinopathy.

In this embodiment, the received retinal lesion image is identified by a deep convolutional neural network model based on a preset number of sample images labeled with different degrees of retinopathy degree, and the corresponding degree of retinopathy is determined according to the recognition result. . Since only the pre-trained deep convolutional neural network model is needed to recognize the received retinal lesion image, it is not necessary to perform complex feature extraction operation on the eye image, which is simpler and can determine corresponding different retinopathy according to the recognition result. Degree level, can effectively identify the degree of retinopathy of patients.

Further, in other embodiments, the training process of the predetermined recognition model is as follows:

A. Prepare corresponding presets for each preset level of retinopathy (such as first level, second level, third level, fourth level, and fifth level, or slight, mild, moderate, severe, etc.) a number of sample images, each of which is labeled with a corresponding degree of retinopathy;

B. Perform image preprocessing on each sample image to obtain a training picture to be trained by the model. The model training is carried out by performing image preprocessing such as scaling, cropping, flipping and/or twisting on each sample image to effectively improve the authenticity and accuracy of the model training. For example, in an embodiment, performing image preprocessing on each sample picture may include:

Scaling a shorter side length of each sample picture to a first preset size (eg, 640 pixels) to obtain a corresponding first picture, and randomly cropping a second preset size on each first picture (eg, a second picture of 256*256 pixels);

The standard parameter value corresponding to each predetermined preset type parameter (for example, color, brightness, and/or contrast, etc.) (for example, the standard parameter value corresponding to the color is a1, and the standard parameter value corresponding to the brightness is a2, and the contrast corresponds The standard parameter value is a3), and each predetermined preset type parameter value of each second picture is adjusted to a corresponding standard parameter value, and a corresponding third picture is obtained to eliminate the sample picture as a medical picture at the time of shooting. The picture caused by the condition is not clear, improve the model training The effectiveness of the practice; for example, adjusting the brightness value of each second picture to the standard parameter value a2, adjusting the contrast value of each second picture to the standard parameter value a3;

Performing a preset direction (for example, horizontal and vertical directions) on each of the third pictures, and performing a twist operation on each of the third pictures according to a preset twist angle (for example, 30 degrees) to obtain each third picture. The corresponding fourth picture, each fourth picture is a training picture of the corresponding sample picture. Among them, the function of the flip and twist operation is to simulate various forms of pictures in the actual business scene. Through these flip and twist operations, the size of the data set can be increased, thereby improving the authenticity and practicability of the model training.

C. Divide all training pictures into a training set of a first ratio (for example, 50%) and a verification set of a second ratio (for example, 25%);

D. training the predetermined recognition model by using the training set;

E. verifying the accuracy of the training recognition model by using the verification set. If the accuracy rate is greater than or equal to the preset accuracy rate, the training ends, or if the accuracy rate is less than the preset accuracy rate, increasing the level of each retinopathy level The number of sample pictures and the above steps B, C, D, E are re-executed until the accuracy of the trained recognition model is greater than or equal to the preset accuracy.

Further, in other embodiments, the predetermined recognition model, that is, the deep convolutional neural network model, includes an input layer and a plurality of network layers, and the network layer includes a convolution layer, a pooling layer, a fully connected layer, and a classification. The layer, optionally, the deep convolutional neural network model may also include a network layer (ie, a Dropout layer) having a random drop of some connection weighting mechanism, the role of which is to improve the recognition accuracy of the model.

In a specific embodiment, the deep convolutional neural network model consists of one input layer, eleven convolutional layers, five pooling layers, and one network layer having random connection discarding certain connection weighting mechanisms (ie, Dropout layer), 1 fully connected layer, 1 classifier layer. The detailed structure of the deep convolutional neural network model is shown in Table 1 below:

Layer Name	Batch Size	Kernel Size	Stride Size	Output Size
Input	64
Conv1	64	7	2	112
MaxPool1	64	3	2	56
Conv2	192	3	1	56
Maxpool2	192	2	2	28
Convolution3	256	3	2	28
Convolution4	480	3	2	28
Maxpool3	480	2	2	28
Convolution5	512	3	2	14
Convolution6	512	3	2	14
Convolution7	512	3	2	14
Convolution8	512	3	2	14
Convolution9	512	3	2	14
Maxpool4	832	2	2	7

Convolution10	832	3	2	7
Convolution11	1024	3	2	7
Avgpool5	1024	7	1	1
Dropout	1024			1
Fc1	5			1
Softmax	5			1

Table 1

Among them: Layer Name indicates the name of the network layer, Input indicates the data input layer of the network, Conv indicates the convolution layer of the model, Conv1 indicates the first convolution layer of the model, MaxPool indicates the maximum pooling layer of the model, and MaxPool1 indicates the first One based on the maximum pooling layer, Dropout means a network layer with random discarding of some connection weighting mechanism, Avgpool5 means the fifth pooling layer but pooled by means of averaging, Fc represents the fully connected layer in the model, Fc1 Represents the first fully connected layer, Softmax represents the Softmax classifier layer; Batch Size represents the number of input images of the current layer; Kernel Size represents the scale of the current layer convolution kernel (for example, Kernel Size can be equal to 3, indicating the scale of the convolution kernel) 3x3); Stride Size represents the moving step size of the convolution kernel, that is, the distance moved to the next convolution position after one convolution; Output Size represents the size of the network layer output feature map. It should be noted that the pooling mode of the pooling layer in this embodiment includes, but is not limited to, Mean pooling, Max pooling, Overlapping, L2 pooling, Local. Contrast Normalization, Stochasticpooling, Def-pooling, and more.

Further, in other embodiments, in order to improve the recognition accuracy of the model, each of the network layers (for example, a convolution layer, a pooling layer, a network layer with a random drop of some connection weight mechanism, a fully connected layer, and a classifier) The corresponding activation function f(x) of the layer, etc. is:

f(x)=max(α*x,0)

Where α is the leak rate and x is a numerical input of the neurons in the deep convolutional neural network model. In a preferred embodiment of the embodiment, α is set to 0.5. Through the comparison test of the same test data set, the recognition accuracy of the deep convolutional neural network model is improved by about 3% by the activation function f(x) of the present embodiment compared to other existing activation functions.

Further, in other embodiments, in order to improve the recognition accuracy of the model, each of the network layers (for example, a convolution layer, a pooling layer, a network layer with a random drop of some connection weight mechanism, a fully connected layer, and a classifier) The corresponding cross entropy H(P, Q) of the layer, etc. is:

H(P,Q)=H(P)+D _KL (P||Q)

Where P and Q are two probability distributions, H(P) is the expectation of probability distribution P, H(P)=-∑ _x∈X P(x)log P(x), and x is the sample space of probability distribution P For any sample in X, P(x) represents the probability that sample x is selected; the expression of D _KL (P||Q) is

x is the probability distribution P and Q any sample in the common sample space X, P(x) represents the probability that the sample x is selected on the probability distribution P, and Q(x) represents the probability that the sample x is selected on the probability distribution Q.

Further, in order to ensure the efficiency and accuracy of the model training, the cross entropy loss function corresponding to each of the network layers

for:

Where x is the input to the model,

Indicates the corresponding label of the input, W represents the preset model parameters, X represents the model input space, f(x:W) represents the transformed output of the model to the input x, ζ denotes the statistic factor, ||W|| ² means summing matrix elements:

W _i+1 =W _i +ΔW _i+1

Where ΔW _i+1 represents the update increment of the weight matrix at time i+1, α is the potential energy term, β is the weight attenuation coefficient, γ is the learning rate of the model, and W _i is the state value of the weight matrix at time i , D _i represents the i-th batch input,

Indicates the average gradient corresponding to the i-th batch input.

In this embodiment, cross entropy can be used as a loss function in neural networks (machine learning). For example, P represents the distribution of real markers, Q is the predicted marker distribution of the trained model, and the cross entropy loss function can measure P and Q. The similarity to ensure the accuracy of the model training. Moreover, the cross entropy as a loss function can avoid the problem of the learning rate reduction of the mean square error loss function when the gradient is lowered, and therefore, the efficiency of the model training can be ensured.

Further, in other embodiments, the deep convolutional neural network model includes at least one fully connected layer, and an initial value of each weight in the predetermined recognition model is from a preset weight range (eg, (0, 1) Weight range) Performing random sampling determination, the probability that the connection weight of the fully connected layer is discarded (Dropout) is set to a first preset value (for example, 0.5), and the weight attenuation coefficient in the cross entropy loss function Set to a second preset value (eg, 0.0005), the potential energy term in the cross entropy loss function is set to a third preset value (eg, 0.9).

Further, in other embodiments, the predetermined scoring function of the recognition model

for:

among them,

O _i,j represents the first prediction as i and the second prediction is the number of pictures actually appearing in j, O represents a matrix of N*N, O _i,j represents the matrix element in matrix O, and N represents the prediction of participation. The number of pictures, the prediction result i, j ∈ {0 1 2 3 4}, E _{i, j} represents the number of images in which the first prediction is i and the second prediction is j, and E is the N* of the desired prediction result. The N matrix, E _i,j represents the matrix elements in the matrix E.

Scoring function in this embodiment

The recognition accuracy of the predetermined recognition model is detected to ensure that the recognition accuracy of the trained recognition model is maintained at a high level to ensure accurate recognition of the degree of retinopathy of the patient.

The invention further provides an electronic device. Please refer to FIG. 2, which is a schematic diagram of a preferred embodiment of the electronic device of the present invention.

In the present embodiment, the retinopathy degree level detecting method is applied to an electronic device 1. The electronic device 1 may include, but is not limited to, a memory 11, a processor 12, and a display 13. Figure 2 shows only the electronic device 1 with components 11-13, but it should be understood that not all illustrated components may be implemented, and more or fewer components may be implemented instead.

The memory 11 may in some embodiments be an internal storage unit of the electronic device 1, such as a hard disk or memory of the electronic device 1. The memory 11 may also be an external storage device of the electronic device 1 in other embodiments, such as a plug-in hard disk equipped on the electronic device 1, a smart memory card (SMC), and a secure digital device. (Secure Digital, SD) card, flash card, etc. Further, the memory 11 may also include both an internal storage unit of the electronic device 1 and an external storage device. The memory 11 is configured to store application software and various types of data installed in the electronic device 1, such as program codes of the retinopathy degree level detecting system 10, and the like. The memory 11 can also be used to temporarily store data that has been output or is about to be output.

The processor 12, in some embodiments, may be a central processing unit (CPU), a microprocessor or other data processing chip for running program code or processing data stored in the memory 11, for example The retinopathy degree level detecting system 10 and the like are executed.

The display 13 in some embodiments may be an LED display, a liquid crystal display, a touch-sensitive liquid crystal display, an OLED (Organic Light-Emitting Diode) touch sensor, or the like. The display 13 is used to display information processed in the electronic device 1 and a user interface for displaying visualization, such as an application menu interface, an application icon interface, and the like. The components 11-13 of the electronic device 1 communicate with one another via a system bus.

In the apparatus embodiment shown in FIG. 2, the program of the retinopathy degree level detecting system 10 is stored in the memory 11; when the processor 12 executes the program of the retinopathy degree level detecting system 10 stored in the memory 11, the following steps are implemented:

After receiving the picture of the retinopathy to be identified, the received retinopathy picture is identified by using a predetermined recognition model, and the recognition result is output; wherein the predetermined recognition model is pre-marked by different retinopathy a deep convolutional neural network model obtained by training a predetermined number of sample pictures of a degree level; and

According to the mapping relationship between the predetermined recognition result and the degree of retinopathy degree, the degree of retinopathy corresponding to the output recognition result is determined.

In this embodiment, the retinopathy degree level detecting system receives a retinopathy degree level detection request sent by the user and includes a retinopathy degree image to be recognized, for example, receiving a degree of retinopathy transmitted by the user through a terminal such as a mobile phone, a tablet computer, or a self-service terminal device. A level detection request, such as receiving a retinopathy degree level detection request sent by a user on a pre-installed client in a terminal such as a mobile phone, a tablet computer, a self-service terminal device, or receiving a user in a terminal such as a mobile phone, a tablet computer, or a self-service terminal device. Retinopathy level level detection request sent on the browser system.

The retinopathy degree level detection system recognizes the received retinopathy image to be recognized by using the pre-trained recognition model after receiving the request for the retinopathy degree level detection issued by the user, and recognizes the retinopathy to be recognized that is to be recognized. The recognition result of the picture in the recognition model. The recognition model can be continuously trained, learned, verified, optimized, etc. by identifying a plurality of preset number of sample pictures marked with different degrees of retinopathy degree, so as to train them to accurately identify different levels of retinopathy. The model of the annotation. For example, the recognition model may employ a Convolutional Neural Network (CNN) model or the like.

After obtaining the recognition result by using the pre-trained deep convolutional neural network model to obtain the recognition result, the mapping result of the predetermined recognition result and the retinopathy degree level may be determined according to the output recognition result. The degree of retinopathy is determined by the degree of retinopathy level of the retinopathy image received. For example, in one embodiment, the recognition result includes a first recognition result (eg, labeled "0"), a second recognition result (eg, labeled "1"), and a third recognition result (eg, an annotation a "2"), a fourth recognition result (for example, labeled "3"), and a fifth recognition result (for example, labeled "4"), the retinopathy degree level includes a first level, a second level, The third level, the fourth level, and the fifth level. The mapping relationship between the different recognition results and the retinopathy degree level may be determined in advance, if the first recognition result corresponds to the first level, the second recognition result corresponds to the second level, and the third recognition result corresponds to the third level, and the fourth recognition result corresponds to The fourth level, the fifth recognition result corresponds to the fifth level. For example, specifically, the first level may correspond to normal and mild non-proliferative diabetic retinopathy, and the first level corresponding retinopathy picture appears as only individual hemangiomas, hard exudation, retinal hemorrhage, and the like. The second grade can correspond to non-proliferative diabetic retinopathy without clinically significant macular edema. The second grade corresponding retinopathy picture shows microangioma, hard exudation, retinal hemorrhage, sickle or beaded vein. The third grade can correspond to non-proliferative diabetic retinopathy with clinically significant macular edema (CSME). The third grade corresponding retinopathy picture shows retinal thickening in the macular area and its vicinity, and microangioma, soft infiltration Out, retinal hemorrhage. The fourth grade can correspond to non-high-risk proliferative retinopathy. The fourth-level retinal lesion picture shows neovascularization in the outer area of the optic papilla and proliferative changes in retinal microvessel formation in other areas. The fifth grade can correspond to high-risk proliferative retinopathy. The fifth grade corresponding retinopathy picture shows neovascularization, vitreous or preretinal hemorrhage in the optic papilla area.

In this way, after the received retinopathy image is recognized by using the pre-trained recognition model to obtain the recognition result, the corresponding different retinas can be determined according to the obtained different recognition results. The degree of lesion severity, thus enabling accurate identification of a variety of refinement levels of retinopathy.

In this embodiment, the received retinal lesion image is identified by a deep convolutional neural network model based on a preset number of sample images labeled with different degrees of retinopathy degree, and the corresponding degree of retinopathy is determined according to the recognition result. . Since only the pre-trained deep convolutional neural network model is needed to recognize the received retinal lesion image, it is not necessary to perform complex feature extraction operation on the eye image, which is simpler and can determine corresponding different retinopathy according to the recognition result. Degree level, can effectively identify the degree of retinopathy of patients.

Further, in other embodiments, the training process of the predetermined recognition model is as follows:

A. Prepare corresponding presets for each preset level of retinopathy (such as first level, second level, third level, fourth level, and fifth level, or slight, mild, moderate, severe, etc.) a number of sample images, each of which is labeled with a corresponding degree of retinopathy;

B. Perform image preprocessing on each sample image to obtain a training picture to be trained by the model. The model training is carried out by performing image preprocessing such as scaling, cropping, flipping and/or twisting on each sample image to effectively improve the authenticity and accuracy of the model training. For example, in an embodiment, performing image preprocessing on each sample picture may include:

Scaling a shorter side length of each sample picture to a first preset size (eg, 640 pixels) to obtain a corresponding first picture, and randomly cropping a second preset size on each first picture (eg, a second picture of 256*256 pixels);

The standard parameter value corresponding to each predetermined preset type parameter (for example, color, brightness, and/or contrast, etc.) (for example, the standard parameter value corresponding to the color is a1, and the standard parameter value corresponding to the brightness is a2, and the contrast corresponds The standard parameter value is a3), and each predetermined preset type parameter value of each second picture is adjusted to a corresponding standard parameter value, and a corresponding third picture is obtained to eliminate the sample picture as a medical picture at the time of shooting. The picture caused by the condition is not clear, and the effectiveness of the model training is improved; for example, the brightness value of each second picture is adjusted to the standard parameter value a2, and the contrast value of each second picture is adjusted to the standard parameter value a3;

Performing a preset direction (for example, horizontal and vertical directions) on each of the third pictures, and performing a twist operation on each of the third pictures according to a preset twist angle (for example, 30 degrees) to obtain each third picture. The corresponding fourth picture, each fourth picture is a training picture of the corresponding sample picture. Among them, the function of the flip and twist operation is to simulate various forms of pictures in the actual business scene. Through these flip and twist operations, the size of the data set can be increased, thereby improving the authenticity and practicability of the model training.

C. Divide all training pictures into a training set of a first ratio (for example, 50%) and a verification set of a second ratio (for example, 25%);

D. training the predetermined recognition model by using the training set;

E. Using the verification set to verify the accuracy of the training recognition model, if the accuracy rate is greater than or equal to the preset accuracy rate, the training ends, or if the accuracy rate is less than the preset accuracy rate, increase each The number of sample pictures corresponding to the level of retinopathy is re-executed in steps B, C, D, and E until the accuracy of the trained recognition model is greater than or equal to the preset accuracy.

Further, in other embodiments, the predetermined recognition model, that is, the deep convolutional neural network model, includes an input layer and a plurality of network layers, and the network layer includes a convolution layer, a pooling layer, a fully connected layer, and a classification. The layer, optionally, the deep convolutional neural network model may also include a network layer (ie, a Dropout layer) having a random drop of some connection weighting mechanism, the role of which is to improve the recognition accuracy of the model.

In a specific embodiment, the deep convolutional neural network model consists of one input layer, eleven convolutional layers, five pooling layers, and one network layer having random connection discarding certain connection weighting mechanisms (ie, Dropout layer), 1 fully connected layer, 1 classifier layer. The detailed structure of the deep convolutional neural network model is shown in Table 1 below:

Layer Name	Batch Size	Kernel Size	Stride Size	Output Size
Input	64
Conv1	64	7	2	112
MaxPool1	64	3	2	56
Conv2	192	3	1	56
Maxpool2	192	2	2	28
Convolution3	256	3	2	28
Convolution4	480	3	2	28
Maxpool3	480	2	2	28
Convolution5	512	3	2	14
Convolution6	512	3	2	14
Convolution7	512	3	2	14
Convolution8	512	3	2	14
Convolution9	512	3	2	14
Maxpool4	832	2	2	7
Convolution10	832	3	2	7
Convolution11	1024	3	2	7
Avgpool5	1024	7	1	1
Dropout	1024			1
Fc1	5			1
Softmax	5			1

Table 1

Among them: Layer Name indicates the name of the network layer, Input indicates the data input layer of the network, Conv indicates the convolution layer of the model, Conv1 indicates the first convolution layer of the model, MaxPool indicates the maximum pooling layer of the model, and MaxPool1 indicates the first One based on the maximum pooling layer, Dropout means a network layer with random discarding of some connection weighting mechanism, Avgpool5 means the fifth pooling layer but pooled by means of averaging, Fc represents the fully connected layer in the model, Fc1 Represents the first fully connected layer, Softmax represents the Softmax classifier layer; Batch Size represents the number of input images of the current layer; Kernel Size represents the scale of the current layer convolution kernel (for example, Kernel Size can be equal to 3, indicating the scale of the convolution kernel) 3x3); Stride Size represents the moving step size of the convolution kernel, that is, the distance moved to the next convolution position after one convolution; Output Size represents the size of the network layer output feature map. It should be noted that the pooling mode of the pooling layer in this embodiment includes, but is not limited to, Mean pooling, Max pooling, Overlapping, L2pooling, Local Contrast. Normalization, Stochasticpooling, Def-pooling, and more.

Further, in other embodiments, in order to improve the recognition accuracy of the model, each of the network layers (for example, a convolution layer, a pooling layer, a network layer with a random drop of some connection weight mechanism, a fully connected layer, and a classifier) The corresponding activation function f(x) of the layer, etc. is:

f(x)=max(α*x,0)

Where α is the leak rate and x is a numerical input of the neurons in the deep convolutional neural network model. In a preferred embodiment of the embodiment, α is set to 0.5. Through the comparison test of the same test data set, the recognition accuracy of the deep convolutional neural network model is improved by about 3% by the activation function f(x) of the present embodiment compared to other existing activation functions.

Further, in other embodiments, in order to improve the recognition accuracy of the model, each of the network layers (for example, a convolution layer, a pooling layer, a network layer with a random drop of some connection weight mechanism, a fully connected layer, and a classifier) The corresponding cross entropy H(P, Q) of the layer, etc. is:

H(P,Q)=H(P)+D _KL (P||Q)

Where P and Q are two probability distributions, H(P) is the expectation of probability distribution P, H(P)=-∑ _x∈X P(x)log P(x), and x is the sample space of probability distribution P For any sample in X, P(x) represents the probability that sample x is selected; the expression of D _KL (P||Q) is

x is any one of the probability distribution P and the Q common sample space X, P(x) represents the probability that the sample x is selected on the probability distribution P, and Q(x) represents the probability that the sample x is selected on the probability distribution Q.

Further, in order to ensure the efficiency and accuracy of the model training, the cross entropy loss function corresponding to each of the network layers

for:

Where x is the input to the model,

Indicates the corresponding label of the input, W represents the preset model parameters, X represents the model input space, f(x:W) represents the transformed output of the model to the input x, ζ denotes the statistic factor, ||W|| ² means summing matrix elements:

W _i+1 =W _i +ΔW _i+1

Where ΔW _i+1 represents the update increment of the weight matrix at time i+1, α is the potential energy term, β is the weight attenuation coefficient, γ is the learning rate of the model, and W _i is the state value of the weight matrix at time i , D _i represents the i-th batch input,

Indicates the average gradient corresponding to the i-th batch input.

In this embodiment, cross entropy can be used as a loss function in neural networks (machine learning). For example, P represents the distribution of real markers, Q is the predicted marker distribution of the trained model, and the cross entropy loss function can measure P and Q. The similarity to ensure the accuracy of the model training. Moreover, the cross entropy as a loss function can avoid the problem of the learning rate reduction of the mean square error loss function when the gradient is lowered, and therefore, the efficiency of the model training can be ensured.

Further, in other embodiments, the deep convolutional neural network model includes at least one fully connected layer, and an initial value of each weight in the predetermined recognition model is from a preset weight range (eg, (0, 1) Weight range) Performing random sampling determination, the probability that the connection weight of the fully connected layer is discarded (Dropout) is set to a first preset value (for example, 0.5), and the weight attenuation coefficient in the cross entropy loss function Set to a second preset value (eg, 0.0005), the potential energy term in the cross entropy loss function is set to a third preset value (eg, 0.9).

Further, in other embodiments, the predetermined scoring function of the recognition model

for:

among them,

O _i,j represents the first prediction as i and the second prediction is the number of pictures actually appearing in j, O represents a matrix of N*N, O _i,j represents the matrix element in matrix O, and N represents the prediction of participation. The number of pictures, the prediction result i, j ∈ {0 1 2 3 4}, E _{i, j} represents the number of images in which the first prediction is i and the second prediction is j, and E is the N* of the desired prediction result. The N matrix, Ei,j represents the matrix elements in the matrix E.

Scoring function in this embodiment

The recognition accuracy of the predetermined recognition model is detected to ensure that the recognition accuracy of the trained recognition model is maintained at a high level to ensure accurate recognition of the degree of retinopathy of the patient.

In other embodiments, the retinopathy degree level detection system 10 can be segmented into one or more modules, the one or more modules being stored in the memory 11 and being processed by one or more processors (This embodiment is performed by the processor 12) to complete the present invention. The term "module" as used in the present invention refers to a series of computer program instructions that are capable of performing a particular function, and are more suitable than the program for describing the execution of the retinopathy level detection system 10 in the electronic device 1.

Please refer to FIG. 3, which is a functional block diagram of a preferred embodiment of the retinopathy lesion level detecting system 10 of FIG. In FIG. 3, the retinopathy degree level detecting system 10 can be divided into The module 01 is identified and the module 02 is determined. The functions or operational steps implemented by the module 01-02 are similar to the above, and are not described in detail herein, by way of example, for example:

The identification module 01 is configured to: after receiving the retinopathy picture to be identified, identify the received retinopathy picture by using a predetermined recognition model, and output a recognition result; wherein the predetermined recognition model is pre-determined a convolutional neural network model trained to pre-set a predetermined number of sample images with different levels of retinopathy; and

The determining module 02 is configured to determine a retinopathy degree level corresponding to the output recognition result according to the mapping relationship between the predetermined recognition result and the retinopathy degree level.

The present invention also provides a computer readable storage medium having stored thereon a program of a detection system for a level of retinopathy, the program of the detection system of the degree of retinopathy level being implemented by a processor Any step of the method of detecting the degree of retinopathy.

The specific implementation manner of the computer readable storage medium of the present invention is substantially the same as the specific implementation method of the above-mentioned method for detecting the degree of retinopathy degree, and details are not described herein again.

It is to be understood that the term "comprises", "comprising", or any other variants thereof, is intended to encompass a non-exclusive inclusion, such that a process, method, article, or device comprising a series of elements includes those elements. It also includes other elements that are not explicitly listed, or elements that are inherent to such a process, method, article, or device. An element that is defined by the phrase "comprising a ..." does not exclude the presence of additional equivalent elements in the process, method, item, or device that comprises the element.

Through the description of the above embodiments, those skilled in the art can clearly understand that the foregoing embodiment method can be implemented by means of software plus a necessary general hardware platform, and can also be implemented by hardware, but in many cases, the former is A better implementation. Based on such understanding, the technical solution of the present invention, which is essential or contributes to the prior art, may be embodied in the form of a software product stored in a storage medium (such as ROM/RAM, disk, The optical disc includes a number of instructions for causing a terminal device (which may be a cell phone, a computer, a server, an air conditioner, or a network device, etc.) to perform the methods described in various embodiments of the present invention.

The preferred embodiments of the present invention have been described above with reference to the drawings, and are not intended to limit the scope of the invention. The serial numbers of the embodiments of the present invention are merely for the description, and do not represent the advantages and disadvantages of the embodiments. Additionally, although logical sequences are shown in the flowcharts, in some cases the steps shown or described may be performed in a different order than the ones described herein.

A person skilled in the art can implement the invention in various variants without departing from the scope and spirit of the invention. For example, the features of one embodiment can be used in another embodiment to obtain a further embodiment. Any modifications, equivalent substitutions and improvements made within the technical concept of the invention are intended to be included within the scope of the invention.

Claims (20)

1. An electronic device, comprising: a memory, a processor, wherein the memory stores a program of a retinopathy degree level detecting system operable on the processor, the retinopathy degree level detecting When the program of the system is executed by the processor, the following steps are implemented:

   After receiving the picture of the retinopathy to be identified, the received retinopathy picture is identified by using a predetermined recognition model, and the recognition result is output; wherein the predetermined recognition model is pre-marked by different retinopathy a convolutional neural network model obtained by training a predetermined number of sample pictures of a degree level;

   According to the mapping relationship between the predetermined recognition result and the degree of retinopathy degree, the degree of retinopathy corresponding to the output recognition result is determined.
2. The electronic device according to claim 1, wherein the training process of the predetermined recognition model is as follows:

   A. setting a corresponding preset number of sample pictures for each preset retinopathy degree level, and marking a corresponding retinopathy degree level for each sample picture;

   B. Perform image preprocessing on each sample image to obtain a training picture to be trained by the model;

   C. Divide all training pictures into a training set of a first ratio and a verification set of a second ratio;

   D. training the predetermined recognition model by using the training set;

   E. verifying the accuracy of the training recognition model by using the verification set. If the accuracy rate is greater than or equal to the preset accuracy rate, the training ends, or if the accuracy rate is less than the preset accuracy rate, increasing the level of each retinopathy level The number of sample pictures and re-execute steps B, C, D, E above.
3. The electronic device of claim 2, wherein the step B comprises:

   Scaling a shorter side length of each sample picture to a first preset size to obtain a corresponding first picture, and randomly cutting a second picture of a second preset size on each first picture;

   Adjusting each predetermined preset type parameter value of each second picture to a corresponding standard parameter value according to a standard parameter value corresponding to each predetermined preset type parameter, to obtain a corresponding third picture;

   Performing a preset direction flip operation on each third picture, and performing a twist operation on each third picture according to a preset twist angle to obtain a fourth picture corresponding to each third picture, and each fourth picture As a training picture to be trained by the model.
4. The electronic device according to claim 1 or 2, wherein the deep convolutional neural network model comprises an input layer and a plurality of network layers, the network layer comprising a convolution layer, a pooling layer, a fully connected layer, and Classifier layer.
5. The electronic device according to claim 4, wherein each of said network layers corresponds to The activation function f(x) is:

   f(x)=max(α*x,0)

   Where α is the preset leak rate and x is a numerical input of the neurons in the deep convolutional neural network model.
6. The electronic device according to claim 4, wherein the cross entropy H(P, Q) corresponding to each of the network layers is:

   H(P,Q)=H(P)+D _KL (P||Q)

   Where P and Q are two probability distributions, H(P) is the expectation of the probability distribution P, H(P)=-∑ _x∈X P(x)logP(x), and x is the sample space X of the probability distribution P In any of the samples, P(x) represents the probability that the sample x is selected; the expression of D _KL (P||Q) is

   

   x is any one of the probability distribution P and the Q common sample space X, P(x) represents the probability that the sample x is selected on the probability distribution P, and Q(x) represents the probability that the sample x is selected on the probability distribution Q.
7. The electronic device according to claim 1 or 2, wherein the predetermined scoring function of the recognition model

   

   for:

   

   among them,

   

   O _i,j represents the first prediction as i and the second prediction is the number of pictures actually appearing in j, O represents a matrix of N*N, O _i,j represents the matrix element in matrix O, and N represents the prediction of participation. The number of pictures, the prediction result i, j ∈ {01234}, E _{i, j} represents the number of images in which the first prediction is i and the second prediction is j, and E is the N*N matrix of the desired prediction result, E _{i, j} represents the matrix element in the matrix E.
8. The electronic device according to claim 4, wherein a cross entropy loss function corresponding to each of said network layers

   

   for:

   

   Where x is the input to the model,

   

   Indicates the corresponding label of the input, W represents the preset model parameters, X represents the model input space, f(x:W) represents the transformed output of the model to the input x, ζ denotes the statistic factor, ||W|| ² means summing matrix elements:

   

   W _i+1 =W _i +ΔW _i+1

   Where ΔW _i+1 represents the update increment of the weight matrix at time i+1, α is the potential energy term, β is the weight attenuation coefficient, γ is the learning rate of the model, and W _i is the state value of the weight matrix at time i , D _i represents the i-th batch input,

   

   Indicates the average gradient corresponding to the i-th batch input.
9. The electronic device according to claim 8, wherein said predetermined recognition model comprises at least one fully connected layer, and initial values of respective weights in said predetermined recognition model are randomly selected from a preset weight range Sampling determines that a probability that the connection weight of the fully connected layer is discarded is set to a first preset value, and a weight attenuation coefficient in the cross entropy loss function is set to a second preset value, where the cross entropy loss function is The potential energy item is set to a third preset value.
10. A method for detecting a degree of retinopathy severity, characterized in that the method comprises the following steps:

    After receiving the picture of the retinopathy to be identified, the received retinopathy picture is identified by using a predetermined recognition model, and the recognition result is output; wherein the predetermined recognition model is pre-marked by different retinopathy a convolutional neural network model obtained by training a predetermined number of sample pictures of a degree level;

    According to the mapping relationship between the predetermined recognition result and the degree of retinopathy degree, the degree of retinopathy corresponding to the output recognition result is determined.
11. The method for detecting a degree of retinopathy according to claim 10, wherein the training process of the predetermined recognition model is as follows:

    A. setting a corresponding preset number of sample pictures for each preset retinopathy degree level, and marking a corresponding retinopathy degree level for each sample picture;

    B. Perform image preprocessing on each sample image to obtain a training picture to be trained by the model;

    C. Divide all training pictures into a training set of a first ratio and a verification set of a second ratio;

    D. training the predetermined recognition model by using the training set;

    E. verifying the accuracy of the training recognition model by using the verification set. If the accuracy rate is greater than or equal to the preset accuracy rate, the training ends, or if the accuracy rate is less than the preset accuracy rate, increasing the level of each retinopathy level The number of sample pictures and re-execute steps B, C, D, E above.
12. The method for detecting a degree of retinopathy according to claim 11, wherein the step B comprises:

    Scaling a shorter side length of each sample picture to a first preset size to obtain a corresponding first picture, and randomly cutting a second picture of a second preset size on each first picture;

    Adjusting each predetermined preset type parameter value of each second picture to a corresponding standard parameter value according to a standard parameter value corresponding to each predetermined preset type parameter, to obtain a corresponding third picture;

    Performing a preset direction flip operation on each third picture, and performing a twist operation on each third picture according to a preset twist angle to obtain a fourth picture corresponding to each third picture, and each fourth picture As a training picture to be trained by the model.
13. The method of detecting a degree of retinopathy according to claim 10 or 11, wherein the deep convolutional neural network model comprises an input layer and a plurality of network layers.
14. The method for detecting a degree of retinopathy according to claim 13, wherein the network layer comprises a convolution layer, a pooling layer, a fully connected layer, and a classifier layer.
15. The method for detecting a degree of retinopathy according to claim 14, wherein the activation function f(x) corresponding to each of said network layers is:

    f(x)=max(α*x,0)

    Where α is the preset leak rate and x is a numerical input of the neurons in the deep convolutional neural network model.
16. The method for detecting a degree of retinopathy according to claim 14, wherein the cross entropy H(P, Q) corresponding to each of the network layers is:

    H(P,Q)=H(P)+D _KL (P||Q)

    Where P and Q are two probability distributions, H(P) is the expectation of the probability distribution P, H(P)=-∑ _x∈X P(x)logP(x), and x is the sample space X of the probability distribution P In any of the samples, P(x) represents the probability that the sample x is selected; the expression of D _KL (P||Q) is

    

    x is any one of the probability distribution P and the Q common sample space X, P(x) represents the probability that the sample x is selected on the probability distribution P, and Q(x) represents the probability that the sample x is selected on the probability distribution Q.
17. The method for detecting a degree of retinopathy of claim 10 or 11, wherein the predetermined scoring function of the recognition model

    

    for:

    

    among them,

    

    O _i,j represents the first prediction as i and the second prediction is the number of pictures actually appearing in j, O represents a matrix of N*N, O _i,j represents the matrix element in matrix O, and N represents the prediction of participation. The number of pictures, the prediction result i, j ∈ {01234}, E _{i, j} represents the number of images in which the first prediction is i and the second prediction is j, and E is the N*N matrix of the desired prediction result, E _{i, j} represents the matrix element in the matrix E.
18. A method for detecting a degree of retinopathy according to claim 14, wherein a cross entropy loss function corresponding to each of said network layers is provided.

    

    for:

    

    Where x is the input to the model,

    

    Indicates the corresponding label of the input, W represents the preset model parameters, X represents the model input space, f(x:W) represents the transformed output of the model to the input x, ζ denotes the statistic factor, ||W|| ² means summing matrix elements:

    

    W _i+1 =W _i +ΔW _i+1

    Where ΔW _i+1 represents the update increment of the weight matrix at time i+1, α is the potential energy term, β is the weight attenuation coefficient, γ is the learning rate of the model, and W _i is the state value of the weight matrix at time i , D _i represents the i-th batch input,

    

    Indicates the average gradient corresponding to the i-th batch input.
19. The method for detecting a degree of retinopathy degree according to claim 14, wherein said predetermined recognition model comprises at least one fully connected layer, and initial values of respective weights in said predetermined recognition model are preset The weight range is determined by random sampling, the probability that the connection weight of the fully connected layer is discarded is set to a first preset value, and the weight attenuation coefficient in the cross entropy loss function is set to a second preset value, The potential energy term in the cross entropy loss function is set to a third preset value.
20. A computer readable storage medium, wherein the computer readable storage medium stores a program of a detection system of a degree of retinopathy level, and the program of the detection system of the level of retinopathy is implemented by a processor, for example The method of the method for detecting a degree of retinopathy according to any one of claims 10 to 19.

Images