CN117523332A - Model training method, model training system, method, device and storage medium - Google Patents
Model training method, model training system, method, device and storage medium Download PDFInfo
- Publication number
- CN117523332A CN117523332A CN202311462316.8A CN202311462316A CN117523332A CN 117523332 A CN117523332 A CN 117523332A CN 202311462316 A CN202311462316 A CN 202311462316A CN 117523332 A CN117523332 A CN 117523332A
- Authority
- CN
- China
- Prior art keywords
- model
- local
- training
- local model
- global
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000012549 training Methods 0.000 title claims abstract description 219
- 238000000034 method Methods 0.000 title claims abstract description 98
- 230000002792 vascular Effects 0.000 claims abstract description 38
- 238000001514 detection method Methods 0.000 claims description 19
- 238000004891 communication Methods 0.000 claims description 11
- 230000006870 function Effects 0.000 claims description 11
- 230000004044 response Effects 0.000 claims description 10
- 238000007781 pre-processing Methods 0.000 claims description 8
- 238000012986 modification Methods 0.000 claims description 5
- 230000004048 modification Effects 0.000 claims description 5
- 238000002372 labelling Methods 0.000 claims description 4
- 230000005856 abnormality Effects 0.000 claims description 3
- 210000001715 carotid artery Anatomy 0.000 claims 1
- 208000014387 congenital coronary artery anomaly Diseases 0.000 claims 1
- 201000011639 coronary artery anomaly Diseases 0.000 claims 1
- 210000004204 blood vessel Anatomy 0.000 abstract description 28
- 230000002159 abnormal effect Effects 0.000 description 17
- 238000012360 testing method Methods 0.000 description 16
- 210000001168 carotid artery common Anatomy 0.000 description 10
- 238000012795 verification Methods 0.000 description 10
- 230000007246 mechanism Effects 0.000 description 9
- 230000008569 process Effects 0.000 description 8
- 208000024172 Cardiovascular disease Diseases 0.000 description 7
- 238000010586 diagram Methods 0.000 description 6
- 238000002595 magnetic resonance imaging Methods 0.000 description 6
- 238000010968 computed tomography angiography Methods 0.000 description 5
- 230000000694 effects Effects 0.000 description 5
- 238000009826 distribution Methods 0.000 description 4
- 238000005516 engineering process Methods 0.000 description 4
- 230000002093 peripheral effect Effects 0.000 description 4
- 201000001320 Atherosclerosis Diseases 0.000 description 3
- 230000009286 beneficial effect Effects 0.000 description 3
- 230000005540 biological transmission Effects 0.000 description 3
- 230000002490 cerebral effect Effects 0.000 description 3
- 210000004351 coronary vessel Anatomy 0.000 description 3
- 238000012545 processing Methods 0.000 description 3
- 208000006011 Stroke Diseases 0.000 description 2
- 238000002583 angiography Methods 0.000 description 2
- 238000013528 artificial neural network Methods 0.000 description 2
- 230000002526 effect on cardiovascular system Effects 0.000 description 2
- 230000003993 interaction Effects 0.000 description 2
- 210000001525 retina Anatomy 0.000 description 2
- 230000035945 sensitivity Effects 0.000 description 2
- 230000003068 static effect Effects 0.000 description 2
- 238000012546 transfer Methods 0.000 description 2
- 230000000007 visual effect Effects 0.000 description 2
- 206010020751 Hypersensitivity Diseases 0.000 description 1
- 208000005764 Peripheral Arterial Disease Diseases 0.000 description 1
- 208000001647 Renal Insufficiency Diseases 0.000 description 1
- 206010057469 Vascular stenosis Diseases 0.000 description 1
- 230000007815 allergy Effects 0.000 description 1
- 230000000747 cardiac effect Effects 0.000 description 1
- 230000001413 cellular effect Effects 0.000 description 1
- 208000026106 cerebrovascular disease Diseases 0.000 description 1
- 230000008859 change Effects 0.000 description 1
- 238000013145 classification model Methods 0.000 description 1
- 238000003759 clinical diagnosis Methods 0.000 description 1
- 238000004590 computer program Methods 0.000 description 1
- 239000002872 contrast media Substances 0.000 description 1
- 208000029078 coronary artery disease Diseases 0.000 description 1
- 238000012937 correction Methods 0.000 description 1
- 238000013500 data storage Methods 0.000 description 1
- 238000013135 deep learning Methods 0.000 description 1
- 238000002408 directed self-assembly Methods 0.000 description 1
- 230000010365 information processing Effects 0.000 description 1
- 230000010354 integration Effects 0.000 description 1
- 201000006370 kidney failure Diseases 0.000 description 1
- 239000000203 mixture Substances 0.000 description 1
- 230000003287 optical effect Effects 0.000 description 1
- 230000001575 pathological effect Effects 0.000 description 1
- 230000002028 premature Effects 0.000 description 1
- 230000002265 prevention Effects 0.000 description 1
- 238000012216 screening Methods 0.000 description 1
- 238000006467 substitution reaction Methods 0.000 description 1
- 208000019553 vascular disease Diseases 0.000 description 1
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
- G06V10/77—Processing image or video features in feature spaces; using data integration or data reduction, e.g. principal component analysis [PCA] or independent component analysis [ICA] or self-organising maps [SOM]; Blind source separation
- G06V10/774—Generating sets of training patterns; Bootstrap methods, e.g. bagging or boosting
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
- G06V10/82—Arrangements for image or video recognition or understanding using pattern recognition or machine learning using neural networks
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V20/00—Scenes; Scene-specific elements
- G06V20/70—Labelling scene content, e.g. deriving syntactic or semantic representations
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V40/00—Recognition of biometric, human-related or animal-related patterns in image or video data
- G06V40/10—Human or animal bodies, e.g. vehicle occupants or pedestrians; Body parts, e.g. hands
- G06V40/18—Eye characteristics, e.g. of the iris
- G06V40/197—Matching; Classification
-
- G—PHYSICS
- G16—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR SPECIFIC APPLICATION FIELDS
- G16H—HEALTHCARE INFORMATICS, i.e. INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR THE HANDLING OR PROCESSING OF MEDICAL OR HEALTHCARE DATA
- G16H50/00—ICT specially adapted for medical diagnosis, medical simulation or medical data mining; ICT specially adapted for detecting, monitoring or modelling epidemics or pandemics
- G16H50/30—ICT specially adapted for medical diagnosis, medical simulation or medical data mining; ICT specially adapted for detecting, monitoring or modelling epidemics or pandemics for calculating health indices; for individual health risk assessment
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Health & Medical Sciences (AREA)
- Medical Informatics (AREA)
- Multimedia (AREA)
- General Physics & Mathematics (AREA)
- Physics & Mathematics (AREA)
- General Health & Medical Sciences (AREA)
- Evolutionary Computation (AREA)
- Databases & Information Systems (AREA)
- Software Systems (AREA)
- Computing Systems (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Artificial Intelligence (AREA)
- Public Health (AREA)
- Human Computer Interaction (AREA)
- Computational Linguistics (AREA)
- Biomedical Technology (AREA)
- Data Mining & Analysis (AREA)
- Pathology (AREA)
- Ophthalmology & Optometry (AREA)
- Epidemiology (AREA)
- Primary Health Care (AREA)
- Eye Examination Apparatus (AREA)
Abstract
The application discloses a model training method, a model training system, a model training method, model training equipment and a storage medium. The model training method may include: training a local model using a local dataset, wherein the local dataset comprises a fundus image with a label comprising a vascular state of a patient to whom the fundus image belongs; determining global model parameters of a global model based on the local model parameters of the local model obtained through training; and updating local model parameters of the local model based on the global model parameters. The embodiment of the application trains the local model by using the data set of the fundus image comprising the blood vessel state label, so that the local model can learn the characteristics related to the blood vessel state on the fundus image; by setting the global model and updating the local model parameters of the local model based on the global model parameters, the data privacy and the safety of the local model end can be guaranteed.
Description
Technical Field
The present application relates generally to the field of image processing technology. More particularly, the present application relates to a model training method, a model training system, a method, an apparatus, and a storage medium.
Background
In recent years, cardiovascular disease (Cardio vascular disease, CVD) has become a leading cause of global disability and premature death. Atherosclerosis is an important pathological basis of cardiovascular diseases, and directly causes common cardiovascular diseases such as coronary heart disease, stroke, peripheral arterial diseases and the like. Therefore, whether abnormal conditions such as atherosclerosis and the like can be timely and conveniently detected is very important for early prevention and intervention of cardiovascular diseases.
Since early findings of atherosclerosis are mainly focused on the common carotid artery, it is currently common to determine whether there is a risk of cardiovascular disease by detecting whether the common carotid artery is abnormal. For the abnormality detection of the common carotid artery, the following methods are mainly used at present: digital subtraction angiography (Digital subtraction angiography, DSA), computed tomography angiography (Computed tomography angiography, CTA) and magnetic resonance imaging (Magnetic resonance imaging, MRI). However, DSA requires invasive procedures and increases the risk of stroke in the patient. CTA can show the intima thickness, the degree of vascular stenosis, etc. of the common carotid artery by non-invasive methods, but is not applicable to patients with contraindications such as contrast agent allergies or renal insufficiency. MRI equipment is costly and MRI is not available for some patients, such as those with cardiac pacemakers installed.
In view of the foregoing, it is desirable to provide a more convenient and faster solution for detecting whether a blood vessel is abnormal or not, so as to predict the risk of vascular diseases.
Disclosure of Invention
To address at least one or more of the technical problems mentioned above, the present application proposes, in various aspects, a model training method, a model training system, a method, an apparatus, and a computer-readable storage medium for detecting a vascular state based on fundus images.
In a first aspect, the present application provides a model training method for detecting a vascular state based on fundus images, comprising: training a local model using a local dataset, wherein the local dataset comprises a fundus image with a label comprising a vascular state of a patient to whom the fundus image belongs; determining global model parameters of a global model based on the local model parameters of the local model obtained through training; and updating local model parameters of the local model based on the global model parameters.
In some embodiments, the model training method further comprises: and determining updated global model parameters by using the updated local model parameters until the global model converges.
In other embodiments, determining the global model parameters includes: determining the weight of the local model parameter according to the generalized gap between the global model and the local model; and determining global model parameters according to the weights and the local model parameters.
In still other embodiments, determining the weights of the local model parameters comprises: determining the generalization gap in the current round training according to local model parameters obtained in the previous round training of the local model and global model parameters used in the current round training; and determining the weight of the global model obtained in the current round training according to the generalized gap in the current round training.
In some embodiments, determining the generalized gap in the current round training comprises: determining a first experience risk according to global model parameters used by the local model in current round training; determining a second experience risk according to local model parameters obtained after the previous round of training of the local model; and determining the generalized gap in current round training based on a difference between the first empirical risk and the second empirical risk.
In other embodiments, the generalized gap is calculated byTo: wherein (1)>Representing global model parameters used when the local model is trained for the r-th round, +.>Representing the local model parameters trained during the r-1 th round of training of the local model,/->Representing a generalized gap between the global model and the local model,/>Representing a first risk of experience->Representing a second empirical risk, N i Representing the number of fundus images in the local dataset used by the ith local model, +.>Representation->And->The difference between (I) and (II)>Labeling of the jth Zhang Yande image representing use of the ith local model training, +.>Representing the i-th local model based on the j Zhang Yande-th image data->And global model parameter beta r Output prediction result,/->Representation->And->The difference between (I) and (II)>Representing the i-th local model based on the j Zhang Yande-th image data->And local model parameters->And outputting a prediction result.
In still other embodiments, the weights are calculated by: wherein (1)>Weights representing local model parameters of the ith local model, M representing the number of local models involved in training, +.>Representing global models and local modelsGeneralized gap between models, k represents a hyper-parameter for controlling the amplitude of modification in each round of training, R represents the total number of communication rounds at the global model end and each local model end, and >Indicating all->Maximum value of (2); the global model parameters are determined by: />Wherein (1)>Representing global model parameters used when the local model is trained on round r+1,/o>Weights representing local model parameters of the ith local model, M representing the number of local models involved in training, +.>And the local model parameters obtained by training when the local model performs the r-th round of training are represented.
In some embodiments, the local model is integrated from a plurality of base models, wherein integrating the plurality of base models includes integrating initial outputs of the plurality of base models to obtain an output result of the local model.
In other embodiments, the plurality of base models includes at least two of VGG-16, acceptance-v 3, densenet121, and Resnet 50.
In still other embodiments, the loss function used in training the local model includes:wherein (1)>Representing a loss function, p t Representing the probability of the local model predicting the vascular state, alpha t The parameter for balancing the number of positive and negative samples is represented, and the parameter for balancing the difficult-to-separate and easy-to-separate samples is represented by gamma.
In some embodiments, the vascular condition includes at least one of whether the common carotid artery is abnormal, whether the coronary artery is abnormal, and whether the cerebral vessel is abnormal.
In other embodiments, prior to training the local model using the local data set, the model training method further comprises: preprocessing the fundus image in the local data set, wherein the preprocessing comprises at least one of black edge removal, brightness adjustment and contrast adjustment; and/or initializing the global model parameters.
In a second aspect, the present application provides a model training system for detecting a vascular state based on fundus images, comprising: one or more clients, each client configured to train a local model using a local dataset, wherein the local dataset comprises a fundus image with annotations comprising a vascular state of a patient to whom the fundus image belongs; and in response to receiving global model parameters, updating local model parameters of the local model based on the global model parameters; and the server side is configured to determine global model parameters of the global model based on the local model parameters of the one or more local models of the one or more clients.
In a third aspect, the present application provides a method of detecting a vascular state based on fundus images, comprising: inputting a fundus image to be detected into a local model trained by the model training method according to any one of the first aspect of the application; and performing blood vessel state detection operation on the fundus image to be detected by using the local model, and outputting a detection result.
In a fourth aspect, the present application provides an apparatus for detecting a vascular state based on fundus images, comprising: a processor for executing program instructions; and a memory storing the program instructions, which when loaded and executed by the processor, cause the processor to perform the model training method according to any one of the first aspects of the present application or to perform the method according to the third aspect of the present application.
In a fifth aspect, the present application provides a computer readable storage medium, characterized in that it has stored thereon computer readable instructions which, when executed by one or more processors, implement the model training method as claimed in any one of the first aspects of the present application or the method as claimed in the third aspect of the present application.
By the model training method for detecting the blood vessel state based on the fundus image provided above, the embodiment of the application trains the local model by using the data set of the fundus image including the blood vessel state label, so that the local model can learn the characteristics related to the blood vessel state on the fundus image; by setting the global model and updating the local model parameters of the local model based on the global model parameters, the data privacy and the safety of the local model end can be guaranteed. In some embodiments, by providing the local model as being integrated from multiple base models, it may be advantageous to reduce the variance of the local model, as well as to improve the robustness and prediction accuracy of the local model.
Drawings
The above, as well as additional purposes, features, and advantages of exemplary embodiments of the present application will become readily apparent from the following detailed description when read in conjunction with the accompanying drawings. Several embodiments of the present application are illustrated by way of example, and not by way of limitation, in the figures of the accompanying drawings and in which like reference numerals refer to similar or corresponding parts and in which:
FIG. 1 illustrates an exemplary flow chart of a model training method for detecting vascular conditions based on fundus images in accordance with some embodiments of the present application;
FIG. 2 illustrates an exemplary flow chart of a model training method for detecting vascular conditions based on fundus images in accordance with further embodiments of the present application;
FIG. 3 illustrates an exemplary flow chart of a model training method for detecting vascular conditions based on fundus images in accordance with further embodiments of the present application;
FIG. 4 shows a schematic diagram of training a local model according to an embodiment of the present application;
FIG. 5 shows a schematic block diagram of a model training system for detecting vascular conditions based on fundus images in accordance with an embodiment of the present application;
FIG. 6 illustrates a flow chart of a method of detecting a vascular state based on fundus images in accordance with an embodiment of the present application;
fig. 7 shows a schematic block diagram of detecting a blood vessel state based on fundus images according to an embodiment of the present application.
Detailed Description
The following description of the embodiments of the present application will be made clearly and fully with reference to the accompanying drawings, in which it is evident that the embodiments described are some, but not all, of the embodiments of the present application. All other embodiments, which can be made by those skilled in the art based on the embodiments herein without making any inventive effort, are intended to be within the scope of the present application.
It should be understood that the terms "comprises" and "comprising," when used in this specification and in the claims, specify the presence of stated features, integers, steps, operations, elements, and/or components, but do not preclude the presence or addition of one or more other features, integers, steps, operations, elements, components, and/or groups thereof.
It is also to be understood that the terminology used in the description of the present application is for the purpose of describing particular embodiments only, and is not intended to be limiting of the application. As used in the specification and claims of this application, the singular forms "a", "an" and "the" are intended to include the plural forms as well, unless the context clearly indicates otherwise. It should be further understood that the term "and/or" as used in the present specification and claims refers to any and all possible combinations of one or more of the associated listed items, and includes such combinations.
As used in this specification and the claims, the term "if" may be interpreted as "when..once" or "in response to a determination" or "in response to detection" depending on the context. Similarly, the phrase "if a determination" or "if a [ described condition or event ] is detected" may be interpreted in the context of meaning "upon determination" or "in response to determination" or "upon detection of a [ described condition or event ]" or "in response to detection of a [ described condition or event ]".
Specific embodiments of the present application are described in detail below with reference to the accompanying drawings.
The inventor finds that the retina is the only structure in the human body, the vascular network can be directly observed, so that the detection of the vascular state of the human body by a non-invasive method of detecting fundus images becomes possible, and the early screening and prediction of cardiovascular and cerebrovascular diseases are facilitated.
The inventor also discovers that the fundus images in actual use are often from different medical institutions, the conditions of equipment, acquisition environment and the like when the medical institutions acquire the fundus images are different, and the data sharing among the medical institutions involves the problems of privacy, safety and the like, so that the problems that the data distribution of the institutions is different and the data cannot be shared can not be solved by the traditional model based on deep learning.
Based on the findings, the application provides a model training scheme and a detection scheme for realizing blood vessel state detection based on the federal domain generalization idea, so as to realize the invention target of detecting the blood vessel state by utilizing fundus images, and simultaneously protect data privacy and safety in the training process of the model. The following detailed description will proceed with reference to the several figures.
Fig. 1 illustrates an exemplary flowchart of a model training method for detecting a vascular state based on fundus images in accordance with some embodiments of the present application. As shown in fig. 1, the model training method 100 may include: in step 101, a local model may be trained using a local data set, wherein the local data set may comprise a fundus image with a callout, which may comprise the vascular status of the patient to whom the fundus image belongs.
In some embodiments, the fundus image may be derived from clinical medical data, and all fundus images in the local dataset for training are provided with annotations that may be given by a specialist based on clinical diagnosis of the patient to whom the fundus image pertains. In other embodiments, the vascular state may include at least one of a cardiovascular state, a cerebrovascular state, and the like. In still other embodiments, the vascular state may include at least one of whether the common carotid artery is abnormal, whether the coronary artery is abnormal, whether the cerebral blood vessel is abnormal, and the like.
In some embodiments, the local model may include a classification model, the classification categories of which output may include, for example, normal and abnormal, and the like. In some application scenarios, the local model may be deployed in a device, such as a medical facility, using a local data set derived from fundus image data of the medical facility in which the local model is deployed. In other embodiments, the model training methods of embodiments of the present application may include training a plurality of local models, each of the plurality of local models being trained using a respective local data set. In other application scenarios, multiple local models may be deployed separately into the devices of multiple medical institutions, each medical institution using its own local data set to train the local model deployed to its own institution. In some embodiments, after the local model is trained, the local model parameters obtained after training may be sent to the global model for global training without externally transmitting the local data set. According to the arrangement, each medical institution can realize the training and the use of the local model without providing the local data set of the medical institution to an external institution, thereby being beneficial to protecting the safety and the privacy of the local data.
Next, in step 102, global model parameters for the global model may be determined based on the trained local model parameters for the local model. In some embodiments, the global model may be used to update the weights of the local model parameters. In other embodiments, the global model may include logic to update the weights of the local model parameters. For example, in still other embodiments, the logical operations included by the global model may be average operations, i.e., the weights of the local model parameters of each local model are set on average, based on the number of local models involved in the training.
In some embodiments, the weights of the local model parameters may be determined from a generalized gap between the global model and the local model. The generalization gap here may be the gap between the prediction of the local model using global model parameters and the prediction of the local model using local model parameters. In other embodiments, the global model parameters may be determined based on weights and local model parameters. For example, in some embodiments, global model parameters may be determined from the product of weights and local model parameters. In other embodiments, the global model parameters may be determined from the sum of the products of the local model parameters of each local model and the corresponding weights.
Further, in step 103, local model parameters of the local model may be updated based on the global model parameters. In some embodiments, global model parameters may be used to replace current local model parameters of the local model, and the local model may be trained using the local data set, and after training a K-wheel (K is the number of iterations of the local model on the local data set, K is a positive integer), new local model parameters may be obtained.
While the model training method for detecting the blood vessel state based on the fundus image according to the embodiment of the present application is exemplarily described above with reference to fig. 1, it may be understood that by training the local model using the fundus image with the label, the local model may learn the correlation characteristic between the fundus image and the blood vessel state, which are difficult for the human eye to observe, so as to achieve the purpose and effect of predicting the blood vessel state based on the fundus image. Furthermore, the global model parameters are updated by utilizing the global model, so that the global model can be trained without using a local data set, the safety and the privacy of the local data set are guaranteed, and when the global model is utilized to update the local model parameters of a plurality of local models, the data distribution and the characteristics of different local data sets are different, so that the trained local model has better generalization capability by learning the model parameters after different data training through the global processing of the global model.
It is also understood that the foregoing description is by way of example and not limitation, and that in other embodiments, model training method 100 may further comprise, prior to step 101: the fundus image in the local dataset is pre-processed, wherein the pre-processing includes at least one of blacking out, adjusting brightness, adjusting contrast, and the like. The black edge removing process may be to remove a black edge area other than a retina area in the original fundus image collected by the apparatus, so that the preprocessed fundus image may have the same size, and subsequent processing is facilitated. The brightness and contrast are adjusted so that the fundus image is clearer, the image quality of the fundus image is improved, and further the model training effect is improved.
Fig. 2 illustrates an exemplary flowchart of a model training method for detecting a vascular state based on fundus images according to further embodiments of the present application. As will be appreciated from the following description, the model training method 200 shown in FIG. 2 may be one implementation of the model training method 100 described in FIG. 1, and thus the foregoing description of the model training method 100 in conjunction with FIG. 1 may also be applicable to the following description of the model training method 200.
As shown in fig. 2, the model training method 200 may include: in step 201, global model parameters may be initialized. Initializing the global model parameters may be setting initial values of the global model parameters. Next, in step 202, the local model may be trained using the local data set. In some embodiments, in step 202, local model parameters may be initialized based on the initialized global model parameters; after initializing the local model parameters, the local model may be trained using the local data set to obtain trained local model parameters. In other embodiments, the local model parameters may be initialized to the same initial values as the global model parameters.
Flow may then proceed to step 203 where global model parameters for the global model may be determined based on the trained local model parameters for the local model. Step 203 may be the same as or similar to step 102 described above in connection with fig. 1, and will not be repeated here.
Further, after performing step 203, step 204 may be performed to determine whether the global model converges. In some embodiments, the global model convergence may be that global model parameters no longer change. In response to the global model not converging (e.g., the global model parameters obtained from the present round of training are different from the global model parameters obtained from the previous round), step 205 may be performed to train the local model based on the determined global model parameters to update the local model parameters of the local model. After executing step 205, the flow returns to step 203, and updated global model parameters may be determined using the updated local model parameters until it is determined in step 204 that the global model converges, and training is completed; otherwise, continuing to execute in an iterative manner: and training the local model by using the updated global model parameters to obtain updated local model parameters.
While the foregoing description of the manner in which the local model and the global model are trained in an iterative manner according to embodiments of the present application has been described above in connection with fig. 2, it is to be understood that the foregoing description is by way of example and not by way of limitation, e.g., step 204 may not be limited to being performed between step 203 and step 205 in the illustration, but may also be performed after, e.g., step 205, as desired, i.e., whether the global model converges may be determined after step 205, and training is completed in response to the global model converging; in response to the global model not converging, updated global model parameters may be determined in an iterative manner using the updated local model parameters to train the local model using the updated global model parameters until the global model converges.
Fig. 3 shows an exemplary flowchart of a model training method for detecting a blood vessel state based on fundus images according to further embodiments of the present application. As will be appreciated from the following description, the model training method 300 illustrated in FIG. 3 may be a representation of the model training method 100 described in FIG. 1, and thus the foregoing description of the model training method 100 in conjunction with FIG. 1 may also be applicable to the following description of the model training method 300.
As shown in fig. 3, the model training method 300 may include: in step 301, a local model may be trained using a local data set. Step 301 may be the same as or similar to step 101 described above in connection with fig. 1, and will not be described again here.
Next, in step 302, a generalized gap between the global model and the local model may be determined. In some embodiments, the generalization gap required in the current training is determined during each round of training. For example, in other embodiments, step 302 may include: and determining the generalization gap in the current round training according to the local model parameters obtained in the previous round training of the local model and the global model parameters used in the current round training. The global model parameters used in the current round of training are global model parameters obtained after the previous round of training. In the current round training, global model parameters obtained after the previous round training are trained on a local data set through a local model to obtain the local model parameters after the current round training; and then, performing global training on the global model according to the local model parameters obtained after the current round training to obtain global model parameters after the current round training.
For example, a gap between a prediction result of a local model parameter obtained by using the previous round of training and a prediction result of a global model parameter for updating the local model parameter in the current round of training of the local model may be determined as a generalization gap.
In still other embodiments, step 302 may include: determining a first tested risk according to global model parameters used by the local model in current round training; determining a second experience risk according to local model parameters obtained after the previous round of training of the local model; and determining a generalized gap in the current round of training based on a difference between the first empirical risk and the second empirical risk. For example, the generalization gap may be calculated by:
wherein,representing global model parameters used when the local model is trained for the r-th round, +.>Representing the local model parameters trained during the r-1 th round of training of the local model,/->Representing a generalized gap between the global model and the local model,/>Representing a first risk of experience->Representing a second empirical risk, N i Representing the number of fundus images in the local dataset used in the ith local model, i being a positive integer, +. >Representation->And->The difference between (I) and (II)>Labeling (i.e., true value) representing the jth Zhang Yande image used for the ith local model training, and>representing the i-th local model based on the j Zhang Yande-th image data->And global model parameter beta r The output prediction result is used for generating a prediction result,representation->And->The difference between (I) and (II)>Representing the i-th local model based on the j Zhang Yande-th image data->And local model parameters->And outputting a prediction result.
Further, the flow may proceed to step 303, where the weights of the local model parameters may be determined according to the generalized gap between the global model and the local model. In still other embodiments, step 303 may include: and determining the weight obtained by the global model in the current round training according to the generalized gap in the current round training. In some embodiments, step 303 may include: and determining the weight of the global model obtained in the current round training according to the generalization gap in the current round training and the weight determined in the previous round training.
In still other embodiments, the weights may be calculated by the following equation:
wherein,weights representing local model parameters of the ith local model obtained in the training of the r-th round,/->Weights representing local model parameters of an ith local model obtained in the (r-1) th round of training, M represents the number of local models involved in training, i is a positive integer between 1 and M, +. >Representing the generalized gap between the global model and the local model, k representing the hyper-parameters for controlling the amplitude of the modification in each round of training, R representing the total number of communication cycles at the global model end and at each local model end, R<R，/>Indicating all->Is the maximum value of (a).
As further shown in fig. 3, after the weights are determined, step 304 may be performed. In step 304, global model parameters may be determined based on the weights and the local model parameters. In some embodiments, the global model parameters may be determined by:
wherein,representing global model parameters used when the local model is trained on round r+1,/o>The weight of the local model parameter of the ith local model in the r-th round of training is represented, M represents the number of the local models participating in training, i is a positive integer between 1 and M, and +.>And the local model parameters obtained by training when the local model performs the r-th round of training are represented.
It is to be appreciated that steps 302-304 can be one embodiment of step 102 described above in connection with fig. 1. Further, after determining the global model parameters, the model training method 300 may continue to step 305. In step 305, local model parameters may be updated based on the determined global model parameters. For example, the local model may train global model parameters on a local data set, with the trained model parameters being used as updated local model parameters.
While a model training method according to a further embodiment of the present application has been described above in connection with fig. 3, it is to be understood that the above description is exemplary and not limiting, and that a local model may be, for example, not limited to a single model, but may be integrated from multiple base models. An exemplary description will be given below with reference to fig. 4.
Fig. 4 shows a schematic diagram of training a local model according to an embodiment of the present application. In some embodiments, the local model may be integrated from a plurality of base models, wherein integrating the plurality of base models may include integrating initial outputs of the plurality of base models to obtain an output result of the local model. In some embodiments, the base model may include a computer vision model. In still other embodiments, the plurality of base models may include at least two of VGG-16, acceptance-v 3, densenet121, resnet50, and the like.
As shown in fig. 4, the fundus image 400 after preprocessing may be input into the basic model 01, the basic model 02, the basic model 03, and the basic model 04, respectively, to be trained, and an initial output 401, an initial output 402, an initial output 403, and an initial output 404 may be obtained, respectively. An output result 405 can be obtained by integrating the initial output 401, the initial output 402, the initial output 403, and the initial output 404. In some embodiments, base model 01, base model 02, base model 03, and base model 04 may be VGG-16, acceptance-v 3, densenet121, and Resnet50, respectively.
In some embodiments, integrating the plurality of initial outputs may be accomplished by combining the plurality of initial outputs to obtain the output result 405. In other embodiments, integrating the plurality of initial outputs may be accomplished by assigning output weights to the plurality of initial outputs and combining the plurality of initial outputs according to the output weights. In other embodiments, integrating the plurality of initial outputs may be accomplished by computing a loss function of the output result 405, and back-propagating the computation based on the loss function, updating parameters of each initial output.
Compared with training by using a single model as a local model, the integrated multiple basic models according to the embodiment of the application are used as the local model for training, so that the variance of the whole model can be reduced, and the robustness of the local model can be improved. Meanwhile, when one basic model is wrong or abnormal, other basic models can play a role in correction and compensation, and the stability of the whole integrated model is improved.
Further, in still other embodiments, the loss function used in training the local model may include:
wherein, Can represent a loss function, p t Can represent the probability that the local model predicts the vascular state, alpha t The parameter for balancing the number of positive and negative samples is represented, and the parameter for balancing the difficult-to-separate and easy-to-separate samples is represented by gamma.
By using such a loss function, the weights of the negative image sample and the positive image sample can be adjusted, and for the case that the number of the negative image sample and the positive image sample is greatly different, the problem of unbalanced sample types can be effectively solved, and the local model can pay more attention to the characteristics of a few types of samples which are difficult to classify. The negative image sample may be a fundus image labeled as normal in the blood vessel state, and the positive image sample may be a fundus image labeled as abnormal in the blood vessel state.
The foregoing describes, with reference to fig. 4, an implementation manner in which the local model according to the embodiment of the present application is an integrated model, and it may be understood that a blood vessel area in a fundus image is relatively complex, and blood vessel conditions of different individuals are different, and in the embodiment of the present application, a local model integrated by multiple base models is adopted, so that generalization of the local model can be enhanced, and meanwhile, misjudgment risk of the local model can be reduced, which is also beneficial to improving overall prediction accuracy of the local model, and the most relevant features can be obtained, so that redundant information is reduced.
In a second aspect, the present application also provides a model training system for detecting a blood vessel state based on fundus images, which will be exemplarily described below with reference to fig. 5.
Fig. 5 shows a schematic block diagram of a model training system for detecting a vascular state based on fundus images according to an embodiment of the present application. As shown in fig. 5, model training system 500 may include a server side 501 and one or more clients 502. In some embodiments, each client 502 may be configured to train a local model using a local dataset, wherein the local dataset includes fundus images with annotations including vascular status of a patient to whom the fundus images pertain; and in response to receiving the global model parameters, updating local model parameters of the local model based on the global model parameters; the server side may be configured to determine global model parameters of the global model based on local model parameters of one or more local models of one or more clients 502. In other embodiments, each client 502 may send the local model parameters obtained after the local model training to the server 501, and the server 501 may distribute the global model parameters to each client 502. In still other embodiments, client 502 may be a medical facility. The number of clients 502 participating in the training may not be limited to three in the illustration, and may participate more or less as desired.
In some embodiments, the server side 501 may be further configured to: and determining updated global model parameters by using the updated local model parameters until the global model converges.
In other embodiments, the server side 501 may be further configured to: determining the weight of the local model parameters according to the generalized gap between the global model and the local model; and determining global model parameters according to the weights and the local model parameters.
In still other embodiments, each client 502 may be further configured to: determining a generalization gap in the current round training according to local model parameters obtained in the previous round training of the local model and global model parameters used in the current round training; and server side 501 may be further configured to: determining the current situation of the global model according to the generalized gap in the current round trainingWeights obtained by previous training. In other embodiments, each client 502 may compute locally the local model parameters and the generalized gap set (e.g.To the server side 501 for the server side 501 to determine the weights. The generalized gap set may include the generalized gap in the current round training and the generalized gap composition in the historical round training.
In some embodiments, each client 502 may be further configured to: determining a first tested risk according to global model parameters used by the local model in current round training; determining a second experience risk according to local model parameters obtained after the previous round of training of the local model; and determining the generalized gap in the current round of training based on a difference between the first empirical risk and the second empirical risk.
In other embodiments, each client 502 may be further configured to: the generalization gap is calculated by the following formula: wherein (1)>Representing global model parameters used when the local model is trained for the r-th round, +.>Representing the local model parameters trained during the r-1 th round of training of the local model,/->Representing a generalized gap between the global model and the local model,/>Representing a first risk of experience->Representing a second empirical risk, N i Representing the number of fundus images in the local dataset used by the ith local model, +.>Representation->And->The difference between (I) and (II)>Labeling of the jth Zhang Yande image representing use of the ith local model training, +.>Representing the i-th local model based on the j Zhang Yande-th image data- >And global model parameter beta r The output prediction result is used for generating a prediction result,representation->And->The difference between (I) and (II)>Representing the i-th local model based on the j Zhang Yande-th image data->And local model parameters->And outputting a prediction result.
In still other embodiments, the server side 501 may be further configured to: the weights are calculated by the following formula:wherein (1)>Weights representing local model parameters of the ith local model, M representing the number of local models involved in training, +.>Representing the generalized gap between the global model and the local model, k representing the hyper-parameters for controlling the amplitude of the modification in each round of training, R representing the total number of communication rounds at the global model side and at each local model side, +.>Indicating all->Maximum value of (2); determining global model parameters by: />Wherein (1)>Representing global model parameters used when the local model is trained on round r+1,/o>Weights representing local model parameters of the ith local model, M representing the local models involved in trainingQuantity of->And the local model parameters obtained by training when the local model performs the r-th round of training are represented.
In some embodiments, the local model is integrated from a plurality of base models, wherein integrating the plurality of base models may include integrating initial outputs of the plurality of base models to obtain output results of the local model.
In other embodiments, the plurality of base models may include at least two of VGG-16, acceptance-v 3, densenet121, and Resnet 50.
In still other embodiments, the loss function used when client 502 trains the local model may include:wherein (1)>Representing a loss function, p t Representing the probability of the local model predicting the vascular state, alpha t The parameter for balancing the number of positive and negative samples is represented, and the parameter for balancing the difficult-to-separate and easy-to-separate samples is represented by gamma.
In some embodiments, the vascular state may include at least one of whether the common carotid artery is abnormal, whether the coronary artery is abnormal, and whether the cerebral blood vessel is abnormal.
In other embodiments, client 502 may be further configured to, prior to training the local model using the local data set: preprocessing fundus images in a local data set, wherein the preprocessing comprises at least one of black edge removal, brightness adjustment and contrast adjustment; and/or server side 501 may be further configured to: the global model parameters are initialized and sent to each client 502.
According to the model training system disclosed by the embodiment of the application, the model training process can be dispersed to each mechanism (namely the client), each mechanism only needs to train the data of the mechanism independently, then the trained local model parameters are delivered to the server, and the server can update the weights of each mechanism later, so that the requirement of the training process on hardware is greatly simplified, the local model can be better deployed into medical mechanisms at all levels, and patient information is not required to be directly communicated with each mechanism, so that the model training system is extremely important for improving privacy protection of patients. Further, due to the distributivity of the training mode, the subsequent expandability is also strong, and the method can be well adapted to the increasing number of participants (namely the number of clients).
In a third aspect of the present application, there is provided a method of detecting a blood vessel state based on a fundus image, as exemplarily described below with reference to fig. 6.
Fig. 6 shows a flowchart of a method of detecting a blood vessel state based on fundus images according to an embodiment of the present application. As shown in fig. 6, method 600 may include: in step 601, the fundus image to be detected may be input into the local model trained by the model training method described above in connection with any of fig. 1 to 4, or into the local model of the client in the model training system described above in connection with fig. 5. Next, in step 602, a blood vessel state detection operation may be performed on the fundus image to be detected using the local model, and a detection result may be output.
In some embodiments, the detection results may include a category score (i.e., classification result) for each category, which may be a probability value. For example, the detection results may include probability values for common carotid artery normal categories and/or common carotid artery abnormal categories. In other embodiments, when the local model adopts the integrated model, the fundus image to be detected may be input into a plurality of basic models in the integrated model respectively, so as to obtain an output result after integration.
The foregoing aspects of the embodiments of the present application may be implemented by means of program instructions. Thus, the present application also provides an apparatus for detecting a blood vessel state based on a fundus image, comprising: a processor for executing program instructions; and a memory storing the program instructions that, when loaded and executed by the processor, cause the processor to perform the model training method described in accordance with the present application in any one of the preceding paragraphs with reference to fig. 1-4 or to perform the method described in accordance with the present application in any one of the preceding paragraphs with reference to fig. 6.
Fig. 7 shows a schematic block diagram of detecting a blood vessel state based on fundus images according to an embodiment of the present application. The system 700 may include the device 701 according to the embodiment of the present application, and peripheral devices and external networks thereof, where the device 701 is used for training a model for detecting a blood vessel state based on a fundus image or for performing operations such as detecting a fundus image to be detected, so as to implement the technical solutions of the embodiments of the present application described in any of the foregoing embodiments with reference to fig. 1 to 6. The system 700 may be deployed at a client to perform operations related to a local model; or may be deployed on the server side to perform operations related to the global model.
As shown in fig. 7, the device 701 may include a CPU 7011, which may be a general-purpose CPU, a special-purpose CPU, or other execution unit for information processing and program execution. Further, the device 701 may further include a mass memory 7012 and a read only memory ROM 7013, wherein the mass memory 7012 may be configured to store various kinds of data including fundus images, weight parameters, classification results, and the like, and various programs required to run a neural network, and the ROM 7013 may be configured to store a drive program for power-on self-test of the device 701, initialization of each functional module in the system, basic input/output of the system, and data required to boot an operating system.
Further, device 701 also includes other hardware platforms or components, such as TPU 7014, GPU 7015, FPGA 7016, and MLU 7017 as shown. It will be appreciated that while various hardware platforms or components are shown in device 701, this is by way of example only and not limitation, and that one of skill in the art may add or remove corresponding hardware as desired. For example, device 701 may include only a CPU as a well-known hardware platform and another hardware platform as a test hardware platform of the present application.
The device 701 of the present application further comprises a communication interface 7018 whereby it may be connected to a local area network/wireless local area network (LAN/WLAN) 705 via the communication interface 7018, and further to a local server 706 or to the Internet ("Internet") 707 via the LAN/WLAN. Alternatively or additionally, the device 701 of the present application may also be directly connected to the internet or cellular network via the communication interface 7018 based on wireless communication technology, such as third generation ("3G"), fourth generation ("4G"), or 5 th generation ("5G") wireless communication technology. In some application scenarios, the device 801 of the present application may also access the server 708 and possibly the database 709 of the external network as needed in order to obtain various known training data, etc., and may store various parameters or intermediate data remotely.
Peripheral devices of the device 701 may include a display device 702, an input device 703, and a data transmission interface 704. In one embodiment, the display 702 may include, for example, one or more speakers and/or one or more visual displays configured to provide voice prompts and/or visual displays of the computational process or classification results of the device of the present application. The input device 703 may include, for example, a keyboard, mouse, microphone, gesture-capturing camera, or other input buttons or controls configured to receive input of training data or user instructions. The data transfer interface 704 may include, for example, a serial interface, a parallel interface, or a universal serial bus interface ("USB"), a small computer system interface ("SCSI"), serial ATA, fireWire ("FireWire"), PCI Express, and high definition multimedia interface ("HDMI"), etc., configured for data transfer and interaction with other devices or systems. According to aspects of the present application, the data transmission interface 704 may receive data for fundus images and the like, and transmit various types of data and results to the device 701.
The above-described CPU 7011, mass memory 7012, read-only memory ROM 7013, TPU 7014, GPU 7015, FPGA 7016, MLU 7017, and communication interface 7018 of the device 701 of the present application may be connected to each other through a bus 7019, and data interaction with peripheral devices is achieved through the bus. In one embodiment, the cpu 7011 may control other hardware components in the device 701 and its peripherals through the bus 7019.
In operation, the processor CPU 7011 of the apparatus 701 of the present application may receive fundus images for training or fundus images to be detected through the input device 703 or the data transmission interface 704, and retrieve computer program instructions or code (e.g., code relating to a neural network) stored in the memory 7012 to train the received data or detect and classify the fundus images to be detected to obtain local model parameters or detection results of the trained local model. After the CPU 7011 determines the classification result by executing the program instruction, the classification result may be displayed on the display device 702 or output by means of voice prompt. In addition, the device 701 may also upload the classification results to a network, such as a remote database 709, via the communication interface 7018.
It should also be appreciated that any module, unit, component, server, computer, terminal, or device executing instructions illustrated herein may include or otherwise access a computer-readable medium, such as a storage medium, computer storage medium, or data storage device (removable) and/or non-removable) such as, for example, magnetic disks, optical disks, or tape. Computer storage media may include volatile and nonvolatile, removable and non-removable media implemented in any method or technology for storage of information such as computer readable instructions, data structures, program modules or other data.
Based on the foregoing, the present application also provides a computer-readable storage medium having stored thereon computer-readable instructions that, when executed by one or more processors, implement a model training method as described above in connection with any of the embodiments of fig. 1-4 or a method of detecting a vascular state based on fundus images as described above in connection with fig. 6.
The computer readable storage medium may be any suitable magnetic or magneto-optical storage medium, such as, for example, resistive Random Access Memory RRAM (Resistive Random Access Memory), dynamic Random Access Memory DRAM (Dynamic Random Access Memory), static Random Access Memory SRAM (Static Random-Access Memory), enhanced dynamic Random Access Memory EDRAM (Enhanced Dynamic Random Access Memory), high-Bandwidth Memory HBM (High-Bandwidth Memory), hybrid storage cube HMC (Hybrid Memory Cube), etc., or any other medium that may be used to store the desired information and that may be accessed by an application, a module, or both. Any such computer storage media may be part of, or accessible by, or connectable to, the device. Any of the applications or modules described herein may be implemented using computer-readable/executable instructions that may be stored or otherwise maintained by such computer-readable media.
In order to better understand the technical effects of the model training scheme for detecting the blood vessel state based on the fundus image according to the embodiment of the present application, a plurality of test cases will be further described below.
Test example 1:
in test example 1, the local models were trained using VGG-16, acceptance-v 3, densenet121, resnet50, and the integrated models (referred to as integrated models) respectively, and the performance index AUC of the local models was tested to obtain the results in Table one below. AUC (Area Under Curve) is defined as the area under the ROC curve enclosed with the coordinate axis for evaluating the classification performance of the model. The AUC generally ranges between 0.5 and 1, with an AUC closer to 1.0 indicating higher model prediction accuracy.
Table one:
| local model | AUC |
| ResNet-50 | 0.725 |
| Inception-v3 | 0.721 |
| VGG-16 | 0.700 |
| DenseNet-121 | 0.722 |
| Integrated model | 0.760 |
As can be seen from table one, the AUC of the local model using the integrated model is higher than that of the local model using the single model, which indicates that the prediction accuracy of the local model using the integrated model is higher.
Test example 2:
in test example 2, a case where the local model employs a plurality of integrated models was tested, and the test results are shown in the following table two.
And (II) table:
as can be seen from the second table, compared with the local model in which VGG-16, acceptance-v 3, densenet121 and Resnet50 are integrated, the AUC value of the local model is higher, and the prediction accuracy of the local model in which VGG-16, acceptance-v 3, densenet121 and Resnet50 are integrated is better than that of the local model in which three or five basic models are integrated in the test example.
In addition, by comparing the test results of the table two and the table one, it can be seen that the AUC values of the local model using the integrated model are higher than those of the local model using the single model, which further indicates that the prediction accuracy of the local model using the integrated model is higher.
Test example 3:
in test example 3, the local model was trained by using the conventional training method and the model training method according to the embodiment of the present application, and a plurality of performance indexes of the trained local model were tested. In the conventional training method, the training process of the global model is not included, and the local model is trained by using training data of each institution only. The training method comprises a local model local training process and a global model training process in each institution. In order to fully evaluate the training effect of the two training methods on the model, two test modes of internal verification and external verification are also used in the test case, and the test results are shown in the following table three and the following table four respectively.
Table three (internal verification):
table four (external verification):
the internal verification refers to verification by separating a part of fundus images of each institution participating in training and reserving the part as a verification set, and the external verification refers to verification by adopting fundus images of new institutions not participating in training. Sen represents Sensitivity (Sensitivity), which means the proportion of samples that are actually positive and for which the model predicts positive correctly. SpeSpecity refers to the Specificity (Specificity) that is the proportion of samples that are actually negative and for which the model predicts correctly negative. Acc represents the Accuracy (Accuracy) and refers to the ratio of the number of correctly classified samples to the total number of samples. F1 represents the harmonic mean of the precision and recall. PPV represents a positive predictive value, i.e., the proportion of samples that are actually positive in the test positive. NPV represents a negative predictive value, i.e., the proportion of samples that are actually negative in the samples that are detected as negative.
From the results of tables three and four, it can be seen that, when internal verification is performed, most of the model performance indexes of the training method in the embodiment of the present application are better than those of the conventional training method. Compared with the traditional training method, the training method of the embodiment of the application has better overall performance indexes, so that the model trained by the training method of the embodiment of the application has better recognition performance and generalization capability, can be better generalized to a new domain, has better performance on the unseen domain, and is beneficial to ensuring the prediction effect of the model reasoning stage.
In summary, according to the model training method of the embodiment of the application, model training is performed based on fundus images, so that the trained local model can be used for detecting the vascular state of a human body, and compared with detection means such as DSA, CTA, MRI and the like used at present, the model trained by the embodiment of the application is used for detecting more conveniently, more rapidly and higher in detection efficiency. By adopting the local model training phase and the global model training phase, the patient information can be communicated without each mechanism, which is extremely important for improving the information privacy protection of the patient, and simultaneously, model parameters trained by data with different distributions and different characteristics can be integrated through the training of the global model, so that the parameters of each local model are further optimized, and the generalization performance of the local model can be improved.
Further, the model training method according to the embodiment of the application can enable the local model to be applied to detection of fundus images of any mechanism source after model training is completed. By inputting the fundus image into the trained local model, the local model can output a classification probability of whether the common carotid artery is abnormal. The model training method can integrate local model parameters of mechanisms with different data distribution and characteristics to train in a global training stage, so that the trained local model can have stronger generalization capability.
While various embodiments of the present application have been shown and described herein, it will be obvious to those skilled in the art that such embodiments are provided by way of example only. Numerous modifications, changes, and substitutions will occur to those skilled in the art without departing from the spirit and scope of the present application. It should be understood that various alternatives to the embodiments of the present application described herein may be employed in practicing the application. The appended claims are intended to define the scope of the application and are therefore to cover all equivalents and alternatives falling within the scope of these claims.
Claims (16)
1. A model training method for detecting a vascular state based on fundus images, comprising:
training a local model using a local dataset, wherein the local dataset comprises a fundus image with a label comprising a vascular state of a patient to whom the fundus image belongs;
determining global model parameters of a global model based on the local model parameters of the local model obtained through training; and
and updating local model parameters of the local model based on the global model parameters.
2. The model training method of claim 1, further comprising:
and determining updated global model parameters by using the updated local model parameters until the global model converges.
3. Model training method according to claim 1 or 2, characterized in that determining global model parameters comprises:
determining the weight of the local model parameter according to the generalized gap between the global model and the local model; and
and determining global model parameters according to the weights and the local model parameters.
4. A model training method as in claim 3 wherein determining the weights of the local model parameters comprises:
Determining the generalization gap in the current round training according to local model parameters obtained in the previous round training of the local model and global model parameters used in the current round training; and
and determining the weight obtained by the global model in the current round training according to the generalized gap in the current round training.
5. The model training method of claim 4, wherein determining the generalized gap in current round training comprises:
determining a first experience risk according to global model parameters used by the local model in current round training;
determining a second experience risk according to local model parameters obtained after the previous round of training of the local model; and
the generalized gap in the current round of training is determined based on a difference between the first empirical risk and the second empirical risk.
6. The model training method of claim 5, wherein the generalized gap is calculated by:
wherein,representing global model parameters used when the local model is trained for the r-th round, +.>Representing the local model parameters trained during the r-1 th round of training of the local model,/- >Representing a generalized gap between the global model and the local model,/>Representing a first risk of experience->Representing a second empirical risk, N i Representing the number of fundus images in the local dataset used by the ith local model, +.>Representation->And->The difference between (I) and (II)>Labeling of the jth Zhang Yande image representing use of the ith local model training, +.>Representing the i-th local model based on the j Zhang Yande-th image data->And global model parameter beta r Output prediction result,/->Representation->And->The difference between (I) and (II)>Representing the i-th local model based on the j Zhang Yande-th image data->And local model parametersAnd outputting a prediction result.
7. The model training method of any one of claims 3-6, wherein the weights are calculated by:
wherein,weights representing local model parameters of the ith local model, M representing the number of local models involved in training, +.>Representing the generalized gap between the global model and the local model, k representing the hyper-parameters for controlling the amplitude of the modification in each round of training, R representing the total number of communication rounds at the global model side and at each local model side, +.>Representing allMaximum value of (2); and
the global model parameters are determined by:
Wherein,representing global model parameters used when the local model is trained on round r+1,/o>Weights representing local model parameters of the ith local model, M representing the number of local models involved in training, +.>And the local model parameters obtained by training when the local model performs the r-th round of training are represented.
8. A model training method according to claim 1 or 2, characterized in that,
the local model is integrated by a plurality of basic models, wherein integrating the plurality of basic models comprises integrating initial outputs of the plurality of basic models to obtain an output result of the local model.
9. The model training method of claim 8, wherein the plurality of base models comprises at least two of VGG-16, acceptance-v 3, densenet121, and Resnet 50.
10. The model training method of claim 1, wherein the loss function used in training the local model comprises:
wherein,representing a loss function, p t Representing the probability of the local model predicting the vascular state, alpha t The parameter for balancing the number of positive and negative samples is represented, and the parameter for balancing the difficult-to-separate and easy-to-separate samples is represented by gamma.
11. The model training method of claim 1, wherein the vascular state comprises at least one of carotid artery abnormality, coronary artery abnormality, and cerebrovascular abnormality.
12. The model training method of claim 1, wherein prior to training the local model using the local data set, the model training method further comprises:
preprocessing the fundus image in the local data set, wherein the preprocessing comprises at least one of black edge removal, brightness adjustment and contrast adjustment; and/or
Initializing the global model parameters.
13. A model training system for detecting a vascular state based on fundus images, comprising:
one or more clients, each configured to
Training a local model using a local dataset, wherein the local dataset comprises a fundus image with a label comprising a vascular state of a patient to whom the fundus image belongs; and
in response to receiving global model parameters, updating local model parameters of the local model based on the global model parameters;
a server side configured to
Global model parameters of a global model are determined based on local model parameters of one or more local models of the one or more clients.
14. A method of detecting a vascular condition based on fundus images, comprising:
Inputting a fundus image to be detected into a local model trained by the model training method according to any one of claims 1 to 12; and
and performing vascular state detection operation on the fundus image to be detected by using the local model, and outputting a detection result.
15. An apparatus for detecting a vascular state based on a fundus image, comprising:
a processor for executing program instructions; and
memory storing the program instructions that, when loaded and executed by the processor, cause the processor to perform the model training method according to any one of claims 1-12 or the method according to claim 14.
16. A computer-readable storage medium having stored thereon computer-readable instructions which, when executed by one or more processors, implement the model training method of any of claims 1-12 or the method of claim 14.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202311462316.8A CN117523332A (en) | 2023-11-03 | 2023-11-03 | Model training method, model training system, method, device and storage medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202311462316.8A CN117523332A (en) | 2023-11-03 | 2023-11-03 | Model training method, model training system, method, device and storage medium |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN117523332A true CN117523332A (en) | 2024-02-06 |
Family
ID=89750493
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202311462316.8A Pending CN117523332A (en) | 2023-11-03 | 2023-11-03 | Model training method, model training system, method, device and storage medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN117523332A (en) |
-
2023
- 2023-11-03 CN CN202311462316.8A patent/CN117523332A/en active Pending
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| Tekkeşin | Artificial intelligence in healthcare: past, present and future | |
| US20210145404A1 (en) | Systems and methods for a deep neural network to enhance prediction of patient endpoints using videos of the heart | |
| Karatzia et al. | Artificial intelligence in cardiology: Hope for the future and power for the present | |
| US8554580B2 (en) | Automated management of medical data using expert knowledge and applied complexity science for risk assessment and diagnoses | |
| Francia et al. | Chaining a U-net with a residual U-net for retinal blood vessels segmentation | |
| Omar et al. | Quantification of cardiac bull's-eye map based on principal strain analysis for myocardial wall motion assessment in stress echocardiography | |
| JP6975253B2 (en) | Learning and applying contextual similarity between entities | |
| CN112868068B (en) | Processing fundus camera images using machine learning models trained with other modes | |
| US20100217094A1 (en) | Point-of-care enactive medical system and method | |
| Seetharam et al. | Cardiovascular imaging and intervention through the lens of artificial intelligence | |
| Wehbe et al. | Deep learning for cardiovascular imaging: A review | |
| CN111340794B (en) | Quantification method and device for coronary artery stenosis | |
| Murugan et al. | An abnormality detection of retinal fundus images by deep convolutional neural networks | |
| EP3896702A1 (en) | Cardiovascular assessment of patients suspected of having covid-19 | |
| Xie et al. | [Retracted] Artificial Intelligence‐Based Feature Analysis of Ultrasound Images of Liver Fibrosis | |
| Ghorashi et al. | Leveraging regression analysis to predict overlapping symptoms of cardiovascular diseases | |
| EP3905257A1 (en) | Risk prediction for covid-19 patient management | |
| Gearhart et al. | An automated view classification model for pediatric echocardiography using artificial intelligence | |
| CN115590481B (en) | Apparatus and computer-readable storage medium for predicting cognitive impairment | |
| Coronado et al. | Towards stroke biomarkers on fundus retinal imaging: a comparison between vasculature embeddings and general purpose convolutional neural networks | |
| CN117523332A (en) | Model training method, model training system, method, device and storage medium | |
| US20210335457A1 (en) | Mapping a patient to clinical trials for patient specific clinical decision support | |
| JP7485512B2 (en) | Medical information processing device, medical information processing method, and medical information processing program | |
| JP7346419B2 (en) | Learning and applying contextual similarities between entities | |
| Fu et al. | Agc-unet: a global context feature fusion method based on u-net for retinal vessel segmentation |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |