Disclosure of Invention
In view of the above, the present invention provides a method for constructing a diabetes classification model, comprising:
acquiring sample data comprising fundus images, various diabetes related information and diabetes type information; training a machine learning model by using a large amount of sample data to output an evaluation result, wherein the evaluation result at least comprises a diabetes classification result corresponding to the diabetes type information, and the machine learning model comprises a feature extraction network and at least one output network, wherein the feature extraction network is used for extracting feature information from the fundus image, and the at least one output network is used for outputting the evaluation result according to the feature information; and the machine learning model adjusts parameters of the machine learning model according to the output evaluation result and various diabetes related information and diabetes type information in the sample data.
Optionally, the machine learning model has only one output network for outputting the diabetes classification result according to the feature information and the plurality of diabetes related information.
Optionally, there are a plurality of output networks, where one output network is configured to output the diabetes classification result, and the other output networks are configured to output identification results corresponding to the plurality of diabetes-related information, respectively;
the machine learning model adjusts parameters of the machine learning model according to the output evaluation result and various diabetes related information and diabetes type information in the sample data, and the method comprises the following steps:
determining a second loss value according to the difference between the diabetes classification result and the diabetes type information in the sample data;
determining a third loss value according to the difference between the identification result and the diabetes related information in the sample data;
determining a first loss value according to the second loss value and the third loss value;
and the machine learning model adjusts self parameters according to the first loss value.
Optionally, the other output networks comprise classification networks and/or regression networks;
the identification result output by the classification network is a classification result, and the difference between the classification result and the corresponding diabetes related information in the sample data is represented by a cross entropy function;
and the identification result output by the regression network is a numerical value, and the difference between the regression result and the corresponding diabetes related information in the sample data is represented by an error function.
Optionally, the machine learning model adjusts a parameter of the feature extraction network at least according to the first loss value.
Optionally, the machine learning model further adjusts parameters of the corresponding output network according to the second loss value, and adjusts parameters of the corresponding output network according to the third loss value, respectively.
Optionally, the diabetes-related information includes some or all of age, gender, systolic blood pressure, diastolic blood pressure, history of family diabetes, high density lipoprotein, triolein, BMI, waist circumference, fasting glucose, two hours post-prandial glucose, glucose tolerance test data, blood leukocyte count.
The invention also provides a diabetes classification method, which is characterized by comprising the following steps: acquiring a fundus image of a user; and identifying the fundus images by using the machine learning model trained by the construction method, and at least outputting a diabetes classification result.
Optionally, the machine learning model further outputs a plurality of diabetes related information from the fundus image.
Optionally, the diabetes-related information includes some or all of age, gender, systolic blood pressure, diastolic blood pressure, history of family diabetes, high density lipoprotein, triolein, BMI, waist circumference, fasting glucose, two hours post-prandial glucose, glucose tolerance test data, blood leukocyte count.
Correspondingly, the invention also provides diabetes classification model construction equipment, which comprises: at least one processor; and a memory communicatively coupled to the at least one processor; wherein the memory stores instructions executable by the processor to cause the processor to perform the method of diabetes classification model construction.
Accordingly, the present invention also provides a diabetes classification apparatus comprising: at least one processor; and a memory communicatively coupled to the at least one processor; wherein the memory stores instructions executable by the one processor to cause the at least one processor to perform the diabetes classification method described above.
According to the diabetes classification model construction method provided by the invention, the model with learning capability consisting of the neural network is trained by acquiring the sample data consisting of the fundus image, the various diabetes related information and the diabetes type information, the evaluation result output by the model is compared with the label of the sample data in the training process, and the parameter of the model is optimized according to the difference, so that the model can learn the relation between the diabetes related information, the diabetes type information and the content presented by the fundus image, and the constructed model can obtain the classification result of diabetes through the fundus image.
According to the diabetes classification method provided by the invention, the eyeground image of the user is acquired, the characteristics of the eyeground image are extracted by using the machine learning model, and the classification result for representing the diabetes type of the user can be output according to the characteristics. The scheme does not need to collect blood samples or other body indexes of a user, only needs to collect fundus images, realizes a noninvasive classification process, and optimizes classification performance due to the introduction of various diabetes related information into the model in a training process, so that the output classification result is more accurate. The user can determine the type of the diabetes mellitus in a short time by using electronic equipment such as a computer, a smart phone, a server and the like, and the method has strong convenience and stability; in addition, professional medical equipment such as blood sample collecting equipment does not need to be introduced, and doctors or professional researchers do not need to participate, so that the cost of diabetes classification can be reduced, and reliable reference information is provided for doctors.
Detailed Description
The technical solutions of the present invention will be described clearly and completely with reference to the accompanying drawings, and it should be understood that the described embodiments are some, but not all embodiments of the present invention. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present invention.
In addition, the technical features involved in the different embodiments of the present invention described below may be combined with each other as long as they do not conflict with each other.
One embodiment of the invention provides a diabetes classification model construction method, which relates to a machine learning model, and the model comprises a feature extraction network 11 and a plurality of output networks 12 as shown in figure 1. The network described in the present application refers to a neural network, in particular, a convolutional neural network. Both the classification network and the regression network described below require a feature extraction structure consisting of a convolutional layer, a pooling layer, and an activation layer. One or more feature extraction structures may form a feature extraction network, and the network extracted features may be input into a subsequent output network. The network has initialized parameters that are trained in this embodiment using sample data to optimize the parameters so that the network can output the diabetes classification results.
First, sample data including fundus images, a plurality of diabetes-related information, and diabetes type information is acquired. By way of example, diabetes related information is, for example, age information, weight information, blood pressure information, and some related medical history information, and the like. The diabetes type information may be at least two types of information, i.e., information with diabetes or information without diabetes, and may further include a specific type of diabetes, e.g., type 1 diabetes and type 2 diabetes.
Sample data should be taken from the real subject, such as information about diabetes of a person can be obtained by reading physical examination data or case data, or the information can be collected by means of a questionnaire. The fundus image is an image taken by a fundus camera, and includes images of organs such as the macula lutea, optic disc, and blood vessels. The retinal blood vessels of the fundus are blood vessels visible in the human body and are considered as windows for understanding the blood vessels of other organs. Therefore, the characteristics of the fundus blood vessels reflect the state of some organs to some extent, and fundus images are very closely linked to diabetes.
In practical applications, some preprocessing may be performed on the fundus image, for example, edges may be cropped, the size may be adjusted, the contrast may be enhanced, and the like, to normalize the data content and improve the image quality.
The fundus image, the plurality of diabetes-related information, and the diabetes type information in one sample data should be from the same subject. To train a machine learning model, a large amount of sample data should be acquired and divided into training data and test data.
And then, training the machine learning model by using a large amount of sample data, and enabling the machine learning model to output an evaluation result. In this embodiment, the evaluation result output by the model includes the diabetes classification result corresponding to the diabetes type information and the identification result corresponding to each piece of diabetes-related information, that is, the diabetes-related information and the diabetes type information in the sample data are used as tags, and the output content of the model corresponds to the tags. For example, if one sample data includes a fundus image P and diabetes-related information a, diabetes-related information B, diabetes-related information C, and diabetes type information X, the fundus image P has A, B, C, X four labels, and the model recognizes the fundus image P and outputs a recognition result a 'corresponding to a, a recognition result B' corresponding to B, a recognition result C 'corresponding to C, and a classification result X' corresponding to X.
As shown in FIG. 1The illustrated feature extraction network 11 is configured to extract feature information from the fundus image, and the plurality of output networks 12 are configured to output evaluation results corresponding to the diabetes-related information and the diabetes type information, respectively, based on the feature information. The output networks share the same characteristic extracted by the extraction network, and output the result corresponding to the label. For example, the feature extraction network 11 extracts the fundus image P to obtain feature information featurepAnd the first output network is according to featurepOutputting the evaluation result A 'corresponding to the tag A, and outputting the evaluation result A' by the second output network according to featurepOutputting evaluation result B' corresponding to label B, and outputting third output network according to featurepOutputting the evaluation result C 'corresponding to the label C, and outputting the result C' according to feature by the fourth output networkpThe evaluation result X' corresponding to the label X is output.
And the machine learning model adjusts the parameters of the machine learning model according to the output evaluation result and the difference between the diabetes related information and the diabetes type information in the sample data. According to the characteristics of the neural network, the model needs to determine a loss value (loss) according to the difference, and optimizes the parameters of the model by reverse transmission so as to reduce the difference.
According to the method for constructing the diabetes classification model provided by the embodiment of the invention, the model with learning capability consisting of the neural network is trained by acquiring the sample data consisting of the fundus image, the various diabetes related information and the diabetes type information, the evaluation result output by the model is compared with the label of the sample data in the training process, and the model can learn the relationship among the diabetes related information, the diabetes type information and the content presented by the fundus image according to the parameters of the difference optimization model, so that the constructed model can obtain the classification result of the diabetes through the fundus image.
The expression of the differences may be different or the same for different kinds of assessment results and labels. In one embodiment, the evaluation results all belong to classification results, including two or more classifications, i.e. the plurality of output networks may all be classification networks. The output network (or output structure) of a classification network typically contains an output layer of a Softmax or Sigmoid function, with the output typically being a confidence or probability of 0-1 to describe the probability or confidence that the input belongs to a certain class or classes. For example, if one piece of diabetes-related information is "family diabetes history", the label 0 indicates no family history, and the label 1 indicates family history, the output network will perform two classifications based on the feature information, and output a value of 0-1 to indicate the classification result, and the multi-classification problem is similar to this.
Preferably, the difference between the classification result and the label can be represented using a cross-entropy function, such as
Thus, the loss value of each classification result and the corresponding label can be obtained. In other possible embodiments, the difference between the classification result and the label may also be represented using a log-likelihood function, an exponential loss function, or a quadratic loss function. In one embodiment, the evaluation results all belong to regression prediction results, that is, the output networks may all be regression prediction networks, and the result output by the network is a numerical value. The output of the regression network is a numerical prediction value for a specific index, such as: age, sex, systolic blood pressure, etc. The output network of the network at least comprises a fully-connected layer for weighting and outputting the input, and an activation layer can be added to change the response value of the input, such as outputting a value less than 0 to 0 and outputting a value greater than 0 to be unchanged by using Relu as an activation function. For example, if a piece of diabetes-related information is age, i.e., a label is an age value, the output network performs regression prediction based on the feature information and outputs a value, i.e., a predicted age.
The difference between the regression prediction result and the label can be expressed by using an Error function, such as Mean square Error function (MSE), Mean Absolute Error function (MAE), Mean Absolute Percentage Error function (MAPE), and the like. Thus, the loss value of each regression prediction result and the corresponding label can be obtained.
In a preferred embodiment, the evaluation result includes both the classification result and the regression prediction result, and the plurality of output networks includes both the classification network and the regression prediction network, and these two results respectively express the difference between the classification network and the regression prediction network in the above manner, so as to determine the respective loss values. According to the preferred scheme, the proper types of networks are respectively adopted for various types of labels and output, and the accuracy of the training result is improved by combining the advantages of a classification network and a regression network.
The plurality of diabetes-related information in the sample data used may include part or all of age, sex, systolic blood pressure, diastolic blood pressure, history of family diabetes, high density lipoprotein, triolein, BMI (Body Mass Index), waist circumference, fasting blood glucose, two hours postprandial blood glucose, glucose tolerance test data, blood leukocyte count.
The evaluation results output by the model are the respective results corresponding to the labels, and these recognition targets can be converted into classification problems or regression prediction problems, for example, the age can be the regression prediction result, or the age can be segmented and converted into the classification result.
According to the model structure in this embodiment, a loss value, such as a loss value L of age, can be calculated for each evaluation result and the difference between the evaluation result and the corresponding labelageLoss value L of systolic pressureSBPDiastolic blood pressure loss value LDBPSex loss value LgenderEtc., these loss values corresponding to the diabetes-related information may be referred to as a third loss value, which is plural; classifying the loss value L of diabetesDBReferred to as the second loss value. When the model adjusts the parameters according to these loss values, a total loss value (first loss value) L can be calculated according to these loss valuestotI.e. Ltot=f(Lage,LSBP,LDBP,Lgender,LDB…), and the influence of these third and second loss values on the total loss value may be different, for example, the influence weight of each loss value may be set, and the total loss may be calculated by linear or non-linear weightingThe value is obtained. The final model adjusts the parameters according to the calculated total loss value.
Further, the machine learning model adjusts parameters of the feature extraction network 11 at least according to the first loss value, so that feature extraction performance is optimized, feature information obtained by the feature extraction performance is more accurate, and accuracy of an output network is improved. The parameters of the respective output network 12 may also be adjusted according to the first loss value, or the parameters of the respective output network may be adjusted according to a plurality of third loss values and second loss values, respectively. For example, for an age output network, the loss value L may be based on ageageAdjusting parameters of the network, for an output network of diabetes classification results, a loss value L according to the diabetes classification resultsDBParameters of the network are adjusted.
The above preferred embodiment optimizes the parameters of the feature extraction network by the total loss value, and optimizes the parameters of the corresponding output network by the loss values corresponding to the various evaluation results, thereby providing the efficiency and performance of the model training.
In another embodiment of the present invention, there is provided a machine learning model, as shown in fig. 2, which includes a feature extraction network 11 and a unique output network 12 for outputting a diabetes classification result based on feature information extracted by the feature extraction network 11 and a plurality of diabetes-related information such as age, sex, blood pressure, etc., and based on a loss value LDBAdjusting the parameters of the device.
The machine learning model of the embodiment has a unique output network, takes various diabetes related information as auxiliary information, and combines the auxiliary information to learn the relation between the diabetes classification information and the fundus image, thereby improving the classification efficiency and accuracy.
One embodiment of the present invention provides a diabetes classification method, which relates to a machine learning model, and the model can be obtained by training using the method described in the above embodiment. Referring to fig. 3, the method of the present embodiment includes: the fundus image 20 of the user is acquired, the fundus image 20 is recognized by the machine learning model 21, and a plurality of diabetes-related information and diabetes classification results are output. The machine learning model 21 includes a feature extraction network 211 for extracting feature information from the fundus image, and a plurality of output networks 212, one of which 212 is for outputting diabetes classification information, and the other of which is for outputting diabetes-related information based on the feature information, respectively. The fundus image 20 is an image taken by a fundus camera, and includes images of organs such as the macula lutea, optic disc, and blood vessels. The output network includes a classification network and/or a regression network.
The diabetes related information may be part or all of age, gender, systolic blood pressure, diastolic blood pressure, familial diabetes history, high density lipoprotein, triolein, BMI, waist circumference, fasting glucose, two hours postprandial glucose, glucose tolerance test data, blood leukocyte count.
The diabetes classification information may be at least two types of information, i.e., with diabetes or without diabetes, and may further include specific types of diabetes, such as type 1 diabetes, type 2 diabetes, or complications of diabetes, such as diabetic retinopathy, and the like.
According to the diabetes classification method provided by the embodiment of the invention, the fundus images of the user are acquired, the characteristics of the fundus images are extracted by using the machine learning model, and the classification result for representing the diabetes type of the user can be output according to the characteristics, and meanwhile, other information related to diabetes can be obtained. The scheme does not need to collect blood samples or other body indexes of a user, only needs to collect fundus images, realizes a noninvasive classification process, and optimizes classification performance due to the introduction of various diabetes related information into the model in a training process, so that the output classification result is more accurate. The diabetes type of the subject can be determined in a short time by using electronic equipment such as a computer, a smart phone, a server and the like, and the method has strong convenience and stability; in addition, professional medical equipment such as blood sample collecting equipment does not need to be introduced, and doctors or professional researchers do not need to participate, so that the cost of diabetes classification can be reduced, and reliable reference information is provided for doctors.
One embodiment of the present invention provides a diabetes classification method, which involves a machine learning model, and the model can be obtained by training using the method described in the above embodiment. The method of the embodiment comprises the following steps: a fundus image of the user is acquired, and as shown in fig. 4, the fundus image 20 is an image taken by a fundus camera, and includes images of organs such as the macula lutea, the optic disc, and blood vessels.
And identifying the fundus images by using a machine learning model, and outputting a diabetes classification result. The machine learning model in this embodiment includes a feature extraction network 211 and a unique output network 212. Wherein the feature extraction network 211 is used for extracting feature information from the fundus image, and the output network 212 is used for outputting the diabetes classification result according to the feature information. Alternatively, the user may provide one or more types of diabetes-related information that may be easily collected, such as some non-invasively collected information, and the output network 212 may output the diabetes classification results in conjunction with the characteristic information and the plurality of types of diabetes-related information.
Because the model does not have other output networks, other related information is not presented during classification, but the performance is optimized according to various diabetes related information during model training, so that the output classification result is more accurate, the result is concise and intuitive, and the method is suitable for the application scene of rapid evaluation.
As will be appreciated by one skilled in the art, embodiments of the present invention may be provided as a method, system, or computer program product. Accordingly, the present invention may take the form of an entirely hardware embodiment, an entirely software embodiment or an embodiment combining software and hardware aspects. Furthermore, the present invention may take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, and the like) having computer-usable program code embodied therein.
The present invention is described with reference to flowchart illustrations and/or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the invention. It will be understood that each flow and/or block of the flow diagrams and/or block diagrams, and combinations of flows and/or blocks in the flow diagrams and/or block diagrams, can be implemented by computer program instructions. These computer program instructions may be provided to a processor of a general purpose computer, special purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, create means for implementing the functions specified in the flowchart flow or flows and/or block diagram block or blocks.
These computer program instructions may also be stored in a computer-readable memory that can direct a computer or other programmable data processing apparatus to function in a particular manner, such that the instructions stored in the computer-readable memory produce an article of manufacture including instruction means which implement the function specified in the flowchart flow or flows and/or block diagram block or blocks.
These computer program instructions may also be loaded onto a computer or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer implemented process such that the instructions which execute on the computer or other programmable apparatus provide steps for implementing the functions specified in the flowchart flow or flows and/or block diagram block or blocks.
It should be understood that the above examples are only for clarity of illustration and are not intended to limit the embodiments. Other variations and modifications will be apparent to persons skilled in the art in light of the above description. And are neither required nor exhaustive of all embodiments. And obvious variations or modifications therefrom are within the scope of the invention.