CN110245723B - Safe and reliable image classification semi-supervised machine learning method and device - Google Patents
Safe and reliable image classification semi-supervised machine learning method and device Download PDFInfo
- Publication number
- CN110245723B CN110245723B CN201910565453.1A CN201910565453A CN110245723B CN 110245723 B CN110245723 B CN 110245723B CN 201910565453 A CN201910565453 A CN 201910565453A CN 110245723 B CN110245723 B CN 110245723B
- Authority
- CN
- China
- Prior art keywords
- model
- data
- verification
- image
- machine learning
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
- 238000013106 supervised machine learning method Methods 0.000 title claims abstract description 12
- 238000012795 verification Methods 0.000 claims abstract description 71
- 238000000034 method Methods 0.000 claims abstract description 68
- 238000010801 machine learning Methods 0.000 claims abstract description 54
- 238000012549 training Methods 0.000 claims abstract description 49
- 238000004422 calculation algorithm Methods 0.000 claims abstract description 39
- 238000005457 optimization Methods 0.000 claims description 93
- 239000010410 layer Substances 0.000 claims description 57
- 230000006870 function Effects 0.000 claims description 24
- 238000010200 validation analysis Methods 0.000 claims description 18
- 238000005070 sampling Methods 0.000 claims description 11
- 239000011159 matrix material Substances 0.000 claims description 8
- 239000002356 single layer Substances 0.000 claims description 6
- 239000003550 marker Substances 0.000 claims description 4
- 238000010276 construction Methods 0.000 claims description 3
- 230000009977 dual effect Effects 0.000 claims description 3
- 238000011156 evaluation Methods 0.000 claims description 3
- 238000011478 gradient descent method Methods 0.000 claims description 3
- 238000007405 data analysis Methods 0.000 abstract description 3
- 238000013473 artificial intelligence Methods 0.000 description 3
- 238000010586 diagram Methods 0.000 description 3
- 238000005516 engineering process Methods 0.000 description 3
- 238000002372 labelling Methods 0.000 description 3
- 238000004458 analytical method Methods 0.000 description 2
- 238000004364 calculation method Methods 0.000 description 2
- 238000013524 data verification Methods 0.000 description 2
- 230000003902 lesion Effects 0.000 description 2
- 238000013459 approach Methods 0.000 description 1
- 230000006399 behavior Effects 0.000 description 1
- 238000005094 computer simulation Methods 0.000 description 1
- 230000001419 dependent effect Effects 0.000 description 1
- 238000001514 detection method Methods 0.000 description 1
- 238000011161 development Methods 0.000 description 1
- 230000018109 developmental process Effects 0.000 description 1
- 230000000694 effects Effects 0.000 description 1
- 238000003709 image segmentation Methods 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/21—Design or setup of recognition systems or techniques; Extraction of features in feature space; Blind source separation
- G06F18/214—Generating training patterns; Bootstrap methods, e.g. bagging or boosting
- G06F18/2155—Generating training patterns; Bootstrap methods, e.g. bagging or boosting characterised by the incorporation of unlabelled data, e.g. multiple instance learning [MIL], semi-supervised techniques using expectation-maximisation [EM] or naïve labelling
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/24—Classification techniques
-
- Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
- Y02T—CLIMATE CHANGE MITIGATION TECHNOLOGIES RELATED TO TRANSPORTATION
- Y02T10/00—Road transport of goods or passengers
- Y02T10/10—Internal combustion engine [ICE] based vehicles
- Y02T10/40—Engine management systems
Abstract
The invention discloses a safe and reliable image classification semi-supervised machine learning method and device, wherein the method comprises the following steps: acquiring a target image dataset; constructing a small number of verification image datasets; assigning values to unlabeled data in the target data set and training the unlabeled data on the target data set according to a machine learning algorithm to obtain a machine learning model; calculating the predicted performance and safety of the model on the verification data set; updating the assignment strategy of the unlabeled data to ensure that the prediction performance and the safety of the model obtained by training on the verification data set are continuously optimized until convergence. The resulting model is finally trained to determine a machine learning model of the target image dataset. The method is suitable for the data analysis condition of more data and less marks which are common in the image classification task.
Description
Technical Field
The invention relates to a safe and reliable image classification semi-supervised machine learning method and device, and belongs to the technical field of machine learning of image classification.
Background
With the development of big data and high performance computing, artificial intelligence techniques have received a great deal of attention, with machine learning being the core technology of artificial intelligence. By means of calculation, a large amount of data is analyzed to improve the performance of the system. In computer systems, "experience" typically exists in the form of "data". The "model" can be derived from empirical data by a machine learning algorithm. The model gives corresponding predictions when faced with new data.
Image classification is one of the most common tasks of data analysis. Images provide vivid and easily understood information, and are an important source for people to forward and exchange information. The image classification is to divide images of different categories according to semantic information of the images, is an important basic problem in data analysis, and is also the basis of important tasks such as image detection, image segmentation, behavior analysis and the like. Image classification has wide application in many fields, including face recognition in security and protection fields, traffic scene recognition in traffic fields, content-based image retrieval in the internet field, image picture recognition in medical fields, and the like.
Currently, the existing image classification machine learning technology mainly comprises three types of supervised machine learning, unsupervised machine learning and semi-supervised machine learning. Wherein supervised machine learning requires model training with labeled data, which can be understood as data of known inputs and outputs. The non-supervision machine learning is used for finding hidden data relations in unlabeled sample data to train a model, and the semi-supervision machine learning is used for training the model according to a large amount of unlabeled data in the sample data and with the assistance of a small amount of labeled data. Because in many cases, the sample data has a difficult way of obtaining marked data, the number of marked data is far less than that of unmarked data. For example, a large number of medical images can be acquired, but it is difficult for a doctor to label lesions of a large number of images, so that the medical images are classified with a lot of data, but with few labels. In this case, training of a model for subsequent classification and recognition by semi-supervised machine learning based on the characteristics of the semi-supervised machine learning becomes a major approach in the field.
However, unlabeled data is not helpful in any case, as algorithms often require assumptions about the structure of the data labels. For example, a low density hypothesis considers that the true classification boundary should pass through a region of low density; manifold hypotheses consider adjacent data to have similar labels, etc. If in practical application the true distribution of the data markers and the assumptions of the model are not the same, unlabeled data can have a negative effect on model training, and semi-supervised machine learning can be unsafe and unreliable.
Disclosure of Invention
The invention aims to: aiming at the problems and the shortcomings in the prior art, the invention provides a semi-supervised machine learning method for realizing safe and reliable image classification, and the main purpose is to realize the function of safe and reliable semi-supervised image classification, thereby solving the problem that the existing semi-supervised learning can not be safe and reliable in solving the task of image classification.
The technical scheme is as follows: a safe and reliable image classification semi-supervised machine learning method specifically comprises the following steps:
1) Acquiring a target image dataset, wherein part of the image data in the target image dataset has marks;
2) Constructing a verification image data set, wherein the prediction performance and the safety on the verification image data set are used as indexes for evaluating the marking quality of the target image data set;
3) And performing label assignment on unlabeled image data in the target image data set according to a machine learning algorithm, performing model training on the target image data set, and performing prediction on the verification image data set to obtain a prediction result on the verification image data set.
4) Updating the unlabeled data assignment strategy according to the predicted performance and the safety on the verification image data set, and retraining the model until the performance reaches the optimal value.
5) The final model is acted upon a machine learning model on the target image dataset.
Optionally, the method for acquiring the verification image dataset includes, but is not limited to:
1) The domain expert gives a small amount of marked image data for verification
2) Constructing verification image data set by self-help sampling method in probability statistics
The self-service sampling method is a uniform sampling with a replacement, that is to say, each time a sample is selected, it is replaced in the sampling pool, possibly again.
The performance of the verification image dataset comprises 2 parts. Predicted performance: refers to the gap between the predicted mark value and the true mark value on the verification image data. Safety: refers to the gap between the predictive performance of the model and the predictive performance of the underlying model (i.e., supervised learning practices without unlabeled data) on the validated image data.
Optionally, the assignment policy of the unlabeled image data is a matrix Q, where Q ij The probability that the training sample image i has a label j is represented, and the final label of the image is the label j corresponding to the maximum value of the probability.
Optionally, the adjusting the assignment policy of the unlabeled image record data according to the prediction performance and the security on the verification image data set is a double-layer optimization problem. The inner layer is optimized to train a machine learning model according to training image data and a label assignment strategy and a corresponding machine learning algorithm, the outer layer is optimized to minimize the prediction performance of the model on verification image data and maximize the safety of the model, and the unmarked data assignment strategy which enables the best prediction performance and the highest safety on the verification image data can be obtained by solving the optimization problem.
On the other hand, the invention provides a semi-supervised machine learning device for realizing safe and reliable image classification, which specifically comprises the following components:
1) An acquisition unit configured to acquire a target image dataset, wherein part of data in the target image dataset has a marker;
2) A first determining unit for constructing a verification image dataset;
3) And the assignment unit is used for carrying out label assignment on the unlabeled images in the target image data set.
4) A training unit for training the machine learning model on the target image dataset.
5) And the second determining unit is used for calculating the prediction performance and the security of the model on the verification image data and optimizing the mark assignment of the unlabeled data by adopting a corresponding method.
Optionally, the first determining unit includes:
1) An acquisition module for acquiring labeled image data given for verification by a domain expert as a verification image dataset;
2) The construction module is used for constructing a plurality of verification image data sets based on a self-help sampling method.
Optionally, the assigning unit includes constructing a matrix Q, where Q ij The probability that the training sample image i has a label j is represented, and the label of the final image is the label j corresponding to the maximum value of the probability.
Optionally, the second determining unit includes:
1) The evaluation module is used for calculating the performance and the safety of the model obtained by training on the verification image data;
2) The determining module is used for determining updating of the assignment strategy matrix Q, which is a double-layer optimization problem, and when the inner layer optimization is a convex optimization problem, replacing the inner layer optimization by adopting a corresponding KKT condition, and converting the inner layer optimization into a single-layer optimization problem for solving; when the inner layer optimization is a non-convex optimization problem, a multi-step gradient descent method is adopted to solve the inner layer optimization, the multi-step gradient descent process is regarded as a plurality of constraint conditions, and a Lagrange multiplier method is adopted to solve.
By means of the technical scheme, the safe and reliable image classification semi-supervised machine learning method and device can acquire the target image data set, construct the verification data set which is distributed with the target data set, label and assign the unlabeled image, train the unlabeled image by adopting a machine learning algorithm to obtain a model, use the obtained model on the image data verification set to obtain a prediction result on the image data verification set, calculate the prediction performance and safety of the model on the verification set, and continuously update label and assign values of the unlabeled image to perform performance tuning until convergence.
Drawings
FIG. 1 is a flow chart of a method for implementing safe and reliable semi-supervised machine learning for image classification according to an embodiment of the present invention;
FIG. 2 is a process diagram of updating an unlabeled image label assignment strategy in accordance with an embodiment of the present invention;
FIG. 3 is a process diagram of another exemplary updating an unlabeled image label assignment strategy in accordance with an embodiment of the present invention;
fig. 4 is a block diagram of a semi-supervised machine learning apparatus for image classification with safety and reliability according to an embodiment of the present invention.
Detailed Description
The present invention is further illustrated below in conjunction with specific embodiments, it being understood that these embodiments are meant to be illustrative of the invention only and not limiting the scope of the invention, and that modifications of the invention, which are equivalent to those skilled in the art to which the invention pertains, will fall within the scope of the invention as defined in the claims appended hereto.
With the advent of massive data, artificial intelligence technology has evolved rapidly, and in order to mine bid values from massive data, a great deal of accurate data annotation information is required. However, in practical applications, the sample data has a difficulty in acquiring marked data, so that the number of marked data is far less than that of unmarked data. For example, hospitals can collect a large number of medical images, but it is impractical for doctors to label lesions of these images, so there is much data in medical image classification, but there is little labeling. Analysis of such data, semi-supervised learning techniques are one of the most effective methods.
However, existing semi-supervised machine learning models often require assumptions about the structure of the data markers. If in practical applications the actual distribution of the data markers and the assumptions of the model are not the same, unsafe and reliable situations can occur with unlabeled data, resulting in reduced or very limited performance improvement.
Therefore, the invention hopes to solve the problems by technical means, ensure the safety of unmarked data utilization and promote the performance of semi-supervised learning. In the process, a plurality of technical problems such as insufficient marked samples and missing training data are involved, and the problem of operation efficiency of massive data is also required to be solved.
The embodiment of the invention provides a safe and reliable image classification semi-supervised machine learning method. The specific steps of the method are shown in fig. 1, and mainly comprise:
101. a target image dataset and a verification image dataset are acquired.
Wherein a portion of the sample data in the target data image set has a marker. The target data set may be any image data set such as a natural scene picture data set or a medical image data set. The plurality of sample data included in the data set includes marked data and unmarked data, the marked data may be understood as data of a known classification result, and the unmarked data may be understood as data of an unknown classification result.
The process of acquiring the target data set can be performed according to any existing acquisition mode, for example, an interface dedicated to inputting the target data set can be provided, and the target data set can be acquired through the interface.
102. A verification image dataset is acquired.
Here, the verification image data set refers to data distributed identically to the target image data set. Although in practical application, the cost of labeling a large number of images one by one is relatively high, manual labeling by selecting a small number of verification images can be implemented, so that a small number of verification image data can be obtained. On the basis, in order to reduce the paranoid of the verification image, M verification data sets can be constructed by adopting a self-help sampling method in probability statistics
Further improving reliability.
103. And carrying out mark assignment on the unlabeled images in the target image data set.
Assigning a matrix Q by a label, wherein Q ij The probability that the training sample i has a label j is represented, and the final label is the label j corresponding to the maximum value of the probability.
104. Training on the target image dataset by adopting a machine learning algorithm to obtain a model.
The unlabeled image can be trained to obtain a model according to a corresponding machine learning algorithm after being assigned, and the model is as follows:
wherein θ represents a model obtained by training, θ represents a value space of θ, β represents any value in θ, n represents the number of training data, k represents the total number of classes to which the image belongs, i represents an index of training samples, j represents a value of enumeration class, and Q ij Representing the probability that training sample i has a label j, l represents a loss function characterizing performance, l (x i J, β) represents the loss function value when the i-th image marker in the training set is j and the model parameter is β.
105. The computational model verifies the predictive performance and security of the data set.
Validating the data as
wherein xj Represented as the j-th image, y in the verification data j Representing image x j Class labels, n v Representing the total number of images in the verification data, the loss function value of the model on the verification data is as follows:
representing the error value between the predicted value of the model and the actual signature.
Wherein the model verifies the security of the data set as follows:
wherein ,θ0 Representing a floor model (supervised learning model without unlabeled data), M represents the total number of validation sets,
representing the ith verification set. The model represents a semi-supervised learning model theta and a ground line model theta 0 Gap in predicted performance. The larger the gap, the safer the model obtained by semi-supervised learning.
106. And continuously optimizing the marking assignment strategy of the unlabeled image in the target data set, and repeating the steps until convergence.
The adjustment of the label assignment strategy for unlabeled images in the target dataset based on performance on the validation set is a two-layer optimization problem, as follows:
the inner layer is optimized to train to obtain a model theta according to training data and a mark assignment strategy, the meaning of corresponding parameters is the same as that of the formula (1), and the outer layer is optimized to minimize the loss of the model on verification data and maximize the safety of the model by adjusting a mark assignment matrix Q. For ease of description, we will represent the loss on the validation data as L (θ) and the loss on the training data as E (θ, Q).
When the inner layer optimization is a convex optimization problem, the convex optimization problem can be converted into a corresponding KKT condition, so that the double-layer optimization problem is converted into a single-layer optimization problem, wherein the KKT condition is a condition when the inner layer optimization obtains an optimal solution, and the conditions are as follows:
solving the formula to obtain a function theta (Q) of theta and Q, and carrying theta (Q) into the outer layer optimization to obtain the gradient of the final objective function and Q
Furthermore, the assignment strategy Q of the unlabeled data can be updated according to the gradient optimization algorithm, so that the prediction performance and the safety reliability of the model are improved. Fig. 2 shows the optimization algorithm flow for this problem: setting optimized number of rounds T and initial value Q of label assignment strategy 0 In the k-th round of optimization, the derivative of the loss function L (θ) on the validation set, Q, is first calculated>
Then updating Q by adopting a gradient optimization algorithm, < ->
η is the step size of the gradient optimization algorithm.
When the inner layer optimization is a non-convex optimization problem, a gradient optimization method is adopted to solve the target-type inner layer optimization problem, and the target-type inner layer optimization problem is expressed as a T-step dynamic process, and the method is specifically as follows:
wherein θt Representing the model at step t, θ t-1 Represents the model at step t-1, eta represents the step size of the gradient optimization algorithm, E (theta, Q) represents the loss on training data when the model assigns a strategy Q to the theta mark,
representing the derivative of E (θ, Q) with θ.
Representing the dynamic process as θ t ＝Φ(θ t-1,Q), wherein 
The target can be written as:
min Q L(θ T )(7)
s.t.θ t ＝Φ(θ t-1 ,Q),t＝1,…T
the problem is an optimization problem with constraint, and the gradient of the objective function to Q can be obtained by adopting a Lagrange multiplier method, so that the solution is carried out by adopting a gradient optimization method.
The Lagrange multiplier method can obtain a Lagrange dual formula:
wherein λt As a lagrangian multiplier, it can be finally obtained by biasing it, and the derivative of the variable Q of equation (8) is:
wherein ,
so that a gradient-dependent optimization method can be used to solve Q. Fig. 3 shows the optimization algorithm flow for this problem: first initialize a tag assignment policy Q 0 The number of inner optimized rounds T and the number of outer optimized rounds outT are determined. On the outer layer optimized kth round, θ is first calculated from t=1, …, T t ＝Φ(θ t-1 Q) and then calculate the derivative of the validation set loss with θ:
107. The final model is taken as a machine learning model on the target image dataset.
After the label assignment strategy is determined, training of the machine learning model can be performed according to label assignment of the unlabeled data, so that a final machine learning model is obtained.
As an implementation of the above-mentioned method for implementing safe and reliable image classification semi-supervised machine learning, the embodiment of the invention provides a device for implementing safe and reliable image classification semi-supervised machine learning, which is mainly used for implementing a function of implementing safe semi-supervised machine learning on a target data set, and solves the problem of unsafe and unreliable situations in the semi-supervised machine learning process. For convenience of reading, the details of the foregoing method embodiment are not repeated one by one, but it should be clear that the apparatus in this embodiment can correspondingly implement all the details of the foregoing method embodiment. The device is shown in fig. 4, and specifically comprises:
an acquisition unit 31 for acquiring a target image dataset, part of sample data in the target dataset having markers;
a first determining unit 32 for constructing image verification data;
and an assignment unit 33, configured to perform label assignment on the unlabeled image in the target data set.
The training unit 34 trains the target data set according to a machine learning algorithm to obtain a machine learning model.
The second determining unit 35 calculates the predicted performance and security of the model on the verification data, and adjusts the label assignment policy of the unlabeled image in the target dataset according to the result.
The first determination unit 32 includes:
an acquisition module 321 for acquiring verification image data to a domain expert.
A construction module 322 is configured to construct a plurality of verification data sets according to a self-sampling method.
The second determination unit 35 includes:
the evaluation module 351 computes the predicted performance and security of the model on the verification data.
And the updating module 352 optimizes the assignment strategy of the unlabeled images in the target data set by adopting an optimization method according to the calculation result.
In summary, the method and the device for realizing safe and reliable image classification semi-supervised machine learning provided by the embodiments of the present invention can acquire a target image dataset, construct verification image data, perform label assignment on unlabeled images in the target dataset, apply a machine learning algorithm to the target dataset to obtain a machine learning model, and then continuously optimize label assignment strategies of unlabeled data according to performance of the machine learning model on the verification data. The method can realize the function of semi-supervised machine learning of safe and reliable image classification. Compared with the condition that the performance is reduced or the performance improvement is limited in the existing method, the method provided by the invention explicitly optimizes the prediction performance and the safety of the model according to the verification set, and effectively avoids the problem that semi-supervised machine learning is unsafe and unreliable.
It will be clear to those skilled in the art that, for convenience and brevity of description, specific working procedures of the above-described systems, apparatuses and units may refer to corresponding procedures in the foregoing method embodiments, which are not repeated herein.
Claims (8)
1. A safe and reliable image classification semi-supervised machine learning method is characterized by comprising the following steps:
1) Acquiring a target image dataset, wherein part of image data in the target image dataset is provided with marks;
2) Constructing a verification image data set, wherein the prediction performance and the safety on the verification image data set are used as indexes for evaluating the marking quality of the target image data set;
3) Performing label assignment on unlabeled images of the target data set;
4) Model training is carried out on the target image data set and prediction is carried out on the verification image data set according to the machine learning algorithm and an unlabeled data assignment strategy corresponding to the machine learning algorithm, so that a prediction result on the verification image data set is obtained;
5) Updating the unlabeled data assignment strategy according to the predicted performance and the safety on the verification image data set, and retraining the model until the performance reaches the optimum;
and after the unlabeled image is assigned, training according to a corresponding machine learning algorithm to obtain a model, wherein the model is as follows:
wherein θ represents a model obtained by training, θ represents a value space of θ, β represents any value in θ, n represents the number of training data, k represents the total number of classes to which the image belongs, i represents an index of training samples, j represents a value of enumeration class, and Q ij Representing the probability that training sample i has a label j, l represents a loss function characterizing performance, l (x i J, beta) represents the loss function when the ith image mark in the training set is j and the model parameter is betaNumerical value;
validating the data as
wherein xj Represented as the j-th image, y in the verification data j Representing image x j Class labels, n v Representing the total number of images in the verification data, the loss function value of the model on the verification data is as follows:
representing an error value between the predicted value of the model and the actual signature;
wherein the model verifies the security of the data set as follows:
wherein ,θ0 Representing a floor model, i.e., a supervised learning model that does not utilize unlabeled data, M represents the total number of validation sets,
representing an ith verification set; the semi-supervised learning model theta and the ground line model theta are represented by the formula (3) 0 A gap in predicted performance; the larger the gap is, the safer the model obtained by semi-supervised learning is;
the adjustment of the label assignment strategy for unlabeled images in the target dataset based on performance on the validation set is a two-layer optimization problem, as follows:
the method comprises the steps that an inner layer is optimized to train to obtain a model theta according to training data and a mark assignment strategy, the meaning of corresponding parameters is the same as that of a formula (1), an outer layer is optimized to minimize loss of the model on verification data and maximize safety of the model by adjusting a mark assignment matrix Q, the loss on the verification data is expressed as L (theta), and the loss on the training data is expressed as E (theta, Q);
when the inner layer optimization is a convex optimization problem, the convex optimization problem is converted into a corresponding KKT condition, so that the double-layer optimization problem is converted into a single-layer optimization problem, wherein the KKT condition is a condition when the inner layer optimization obtains an optimal solution, and the conditions are as follows:
solving the equation to obtain a function theta (Q) of theta and Q, and carrying the theta (Q) into the outer layer to optimize to obtain the gradient of the final objective function on Q
Furthermore, the assignment strategy Q of the unlabeled data is updated according to the gradient optimization algorithm, so that the prediction performance and the safety reliability of the model are improved; optimizing the algorithm flow: setting optimized number of rounds T and initial value Q of label assignment strategy 0 In the k-th round of optimization, the derivative of the loss function L (θ) on the validation set, Q, is first calculated>
Then updating Q by adopting a gradient optimization algorithm, < ->
η is the step length of the gradient optimization algorithm;
when the inner layer optimization is a non-convex optimization problem, a gradient optimization method is adopted to solve the inner layer optimization problem, and the inner layer optimization problem is expressed as a T-step dynamic process, and the method is specifically as follows:
wherein θt Representing the model at step t, θ t-1 Represents the model at step t-1, eta represents the step size of the gradient optimization algorithm, E (theta, Q) represents the loss on training data when the model assigns a strategy Q to the theta mark,
representing the derivative of E (θ, Q) with respect to θ;
representing the dynamic process as θ t ＝Φ(θ t-1,Q), wherein 
The target write is:
min Q L(θ T )(7)
obtaining the gradient of the objective function to Q by adopting a Lagrangian multiplier method, and solving by adopting a gradient optimization method;
wherein, the Lagrange dual equation is obtained by Lagrange multiplier method:
wherein λt The derivative of equation (8) with respect to variable Q, being the lagrange multiplier, is:
wherein ,
therefore, a gradient-related optimization method is adopted to solve Q; optimizing the algorithm flow: first initialize a tag assignment policy Q 0 Determining the number of inner optimized rounds T and outer optimizedIs the number of rounds of outT; on the outer layer optimized kth round, θ is first calculated from t=1, …, T t ＝Φ(θ t-1 Q) and then calculate the derivative of the validation set loss with θ: />
The derivative g of the validation set loss to the tag assignment policy Q is then initialized Q =0, then g is calculated from t=t, …,1 Q ＝g Q +α t+1 B t+1 ，α t ＝α t+1 A t+1 Calculate g Q Thereafter, Q is updated using a gradient algorithm k ＝Q k-1 -ηg Q η is the step length of the gradient optimization algorithm;
6) The final model is acted upon a machine learning model on the target image dataset.
2. A secure and reliable image classification semi-supervised machine learning method as recited in claim 1, wherein constructing a validation image dataset includes:
1) Marked image data for verification given by a domain expert;
2) Constructing a verification image data set by a self-help sampling method in probability statistics;
the self-help sampling method refers to a uniform sampling method with replacement.
3. A secure and reliable image classification semi-supervised machine learning method as recited in claim 1, wherein the performance of the validation image dataset includes 2 parts; predicted performance: refers to the gap between the predicted mark value and the true mark value on the verification image data; safety: refers to the gap between the predicted performance of the model and the predicted performance of the underlying model in validating the image data.
4. The method of claim 1, wherein the adjusting of assignment strategies for unlabeled image registration data based on predictive performance and security on the validated image dataset is a two-layer optimization model; the inner layer is optimized to train a machine learning model according to training image data and a label assignment strategy and a corresponding machine learning algorithm, and the outer layer is optimized to minimize the predictive performance of the model on verification image data and maximize the safety of the model.
5. The safe and reliable image classification semi-supervised machine learning method of claim 4, wherein when the inner layer is optimized as a convex optimization problem, the corresponding KKT condition is adopted to replace the inner layer optimization problem, and the inner layer optimization problem is converted into a single-layer optimization problem for solving; when the inner layer optimization is a non-convex problem, a multi-step gradient descent method is adopted to solve the inner layer optimization problem, the multi-step gradient descent process is regarded as a plurality of constraint conditions, and a Lagrange multiplier method is adopted to solve the inner layer optimization problem.
6. A safe and reliable image classification semi-supervised machine learning apparatus, the apparatus comprising:
1) An acquisition unit configured to acquire a target image dataset, part of data in the target image dataset having a marker;
2) A first determining unit for constructing a verification image dataset;
3) The assignment unit is used for carrying out label assignment on the unlabeled images in the target image data set;
4) A training unit for training a machine learning model on the target image dataset;
5) The second determining unit is used for calculating the prediction performance and the safety of the model on the verification image data and optimizing the mark assignment of the unlabeled data by adopting a corresponding method;
and after the unlabeled image is assigned, training according to a corresponding machine learning algorithm to obtain a model, wherein the model is as follows:
wherein θ represents the training resultModel, wherein Θ is the value space of θ, β is any value in Θ, n is the number of training data, k is the total number of classes to which the image belongs, i is the index of the training sample, j is the value of enumeration class, Q ij Representing the probability that training sample i has a label j, l represents a loss function characterizing performance, l (x i J, beta) represents the loss function value when the i-th image mark in the training set is j and the model parameter is beta;
validating the data as
wherein xj Represented as the j-th image, y in the verification data j Representing image x j Class labels, n v Representing the total number of images in the verification data, the loss function value of the model on the verification data is as follows:
representing an error value between the predicted value of the model and the actual signature;
wherein the model verifies the security of the data set as follows:
wherein ,θ0 Representing a floor model, i.e., a supervised learning model that does not utilize unlabeled data, M represents the total number of validation sets,
representing an ith verification set; the semi-supervised learning model theta and the ground line model theta are represented by the formula (3) 0 A gap in predicted performance; the larger the gap is, the safer the model obtained by semi-supervised learning is;
the adjustment of the label assignment strategy for unlabeled images in the target dataset based on performance on the validation set is a two-layer optimization problem, as follows:
the method comprises the steps that an inner layer is optimized to train to obtain a model theta according to training data and a mark assignment strategy, the meaning of corresponding parameters is the same as that of a formula (1), an outer layer is optimized to minimize loss of the model on verification data and maximize safety of the model by adjusting a mark assignment matrix Q, the loss on the verification data is expressed as L (theta), and the loss on the training data is expressed as E (theta, Q);
when the inner layer optimization is a convex optimization problem, the convex optimization problem is converted into a corresponding KKT condition, so that the double-layer optimization problem is converted into a single-layer optimization problem, wherein the KKT condition is a condition when the inner layer optimization obtains an optimal solution, and the conditions are as follows:
solving the equation to obtain a function theta (Q) of theta and Q, and carrying the theta (Q) into the outer layer to optimize to obtain the gradient of the final objective function on Q
Furthermore, the assignment strategy Q of the unlabeled data is updated according to the gradient optimization algorithm, so that the prediction performance and the safety reliability of the model are improved; optimizing the algorithm flow: setting optimized number of rounds T and initial value Q of label assignment strategy 0 In the k-th round of optimization, the derivative of the loss function L (θ) on the validation set, Q, is first calculated>
Then updating Q by adopting a gradient optimization algorithm, < ->
η is the step length of the gradient optimization algorithm;
when the inner layer optimization is a non-convex optimization problem, a gradient optimization method is adopted to solve the inner layer optimization problem, and the inner layer optimization problem is expressed as a T-step dynamic process, and the method is specifically as follows:
wherein θt Representing the model at step t, θ t-1 Represents the model at step t-1, eta represents the step size of the gradient optimization algorithm, E (theta, Q) represents the loss on training data when the model assigns a strategy Q to the theta mark,
representing the derivative of E (θ, Q) with respect to θ;
representing the dynamic process as θ t ＝Φ(θ t-1,Q), wherein 
The target write is:
min Q L(θ T ) (7)
s.t.θ t ＝Φ(θ t-1 ，Q)，t＝1，…T
obtaining the gradient of the objective function to Q by adopting a Lagrangian multiplier method, and solving by adopting a gradient optimization method;
wherein, the Lagrange dual equation is obtained by Lagrange multiplier method:
wherein λt The derivative of equation (8) with respect to variable Q, being the lagrange multiplier, is:
wherein ,
therefore, a gradient-related optimization method is adopted to solve Q; optimizing the algorithm flow: first initialize a tag assignment policy Q 0 Determining the number T of the inner optimized wheels and the number outT of the outer optimized wheels; on the outer layer optimized kth round, θ is first calculated from t=1, …, T t ＝Φ(θ t-1 Q) and then calculate the derivative of the validation set loss with θ: />
The derivative g of the validation set loss to the tag assignment policy Q is then initialized Q =0, then g is calculated from t=t, …,1 Q ＝g Q +α t+1 B t+1 ，α t ＝α t+1 A t+1 Calculate g Q Thereafter, Q is updated using a gradient algorithm k ＝Q k-1 -ηg Q η is the step size of the gradient optimization algorithm.
7. The safe and reliable image classification semi-supervised machine learning apparatus of claim 6, wherein the first determination unit includes:
1) An acquisition module for acquiring labeled image data given for verification by a domain expert as verification image data;
2) The construction module is used for constructing a plurality of verification image data sets based on a self-help sampling method.
8. The safe and reliable image classification semi-supervised machine learning apparatus of claim 7, wherein the second determination unit includes:
1) The evaluation module is used for calculating the performance and the safety of the model obtained by training on the verification image data;
2) The determining module is used for determining updating of the assignment strategy matrix Q, which is a double-layer optimization problem, and when the inner layer optimization is a convex optimization problem, replacing the inner layer optimization by adopting a corresponding KKT condition, and converting the inner layer optimization into a single-layer optimization problem for solving; when the inner layer optimization is a non-convex optimization problem, a multi-step gradient descent method is adopted to solve the inner layer optimization, the multi-step gradient descent process is regarded as a plurality of constraint conditions, and a Lagrange multiplier method is adopted to solve.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910565453.1A CN110245723B (en) | 2019-06-27 | 2019-06-27 | Safe and reliable image classification semi-supervised machine learning method and device |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910565453.1A CN110245723B (en) | 2019-06-27 | 2019-06-27 | Safe and reliable image classification semi-supervised machine learning method and device |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN110245723A CN110245723A (en) | 2019-09-17 |
| CN110245723B true CN110245723B (en) | 2023-06-09 |
Family
ID=67889836
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201910565453.1A Active CN110245723B (en) | 2019-06-27 | 2019-06-27 | Safe and reliable image classification semi-supervised machine learning method and device |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN110245723B (en) |
Families Citing this family (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN110599492B (en) * | 2019-09-19 | 2024-02-06 | 腾讯科技（深圳）有限公司 | Training method and device for image segmentation model, electronic equipment and storage medium |
| CN110874638B (en) * | 2020-01-19 | 2020-06-02 | 同盾控股有限公司 | Behavior analysis-oriented meta-knowledge federation method, device, electronic equipment and system |
| TWI748344B (en) * | 2020-02-14 | 2021-12-01 | 聚積科技股份有限公司 | Establishing Method of Standard Judgment Model for LED Screen Adjustment |
| CN113642671B (en) * | 2021-08-27 | 2024-03-05 | 京东科技信息技术有限公司 | Semi-supervised meta learning method and device based on task distribution change |
Family Cites Families (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US8548428B2 (en) * | 2009-01-28 | 2013-10-01 | Headwater Partners I Llc | Device group partitions and settlement platform |
| CN104392251B (en) * | 2014-11-28 | 2017-05-24 | 西安电子科技大学 | Hyperspectral image classification method based on semi-supervised dictionary learning |
| CN109034205B (en) * | 2018-06-29 | 2021-02-02 | 西安交通大学 | Image classification method based on direct-push type semi-supervised deep learning |
-
2019
- 2019-06-27 CN CN201910565453.1A patent/CN110245723B/en active Active
Also Published As
| Publication number | Publication date |
|---|---|
| CN110245723A (en) | 2019-09-17 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN110245723B (en) | Safe and reliable image classification semi-supervised machine learning method and device | |
| JP6844301B2 (en) | Methods and data processors to generate time series data sets for predictive analytics | |
| Paul et al. | Robust visual tracking by segmentation | |
| CN110163236B (en) | Model training method and device, storage medium and electronic device | |
| Wu et al. | Research on image text recognition based on canny edge detection algorithm and k-means algorithm | |
| US20080071708A1 (en) | Method and System for Data Classification Using a Self-Organizing Map | |
| CN109492750B (en) | Zero sample image classification method based on convolutional neural network and factor space | |
| CN112149717A (en) | Confidence weighting-based graph neural network training method and device | |
| CN111127364A (en) | Image data enhancement strategy selection method and face recognition image data enhancement method | |
| CN112950569B (en) | Melanoma image recognition method, device, computer equipment and storage medium | |
| CN111126155B (en) | Pedestrian re-identification method for generating countermeasure network based on semantic constraint | |
| CN113283282A (en) | Weak supervision time sequence action detection method based on time domain semantic features | |
| Wu et al. | Uncertainty-aware label rectification for domain adaptive mitochondria segmentation | |
| Oza et al. | Utilizing patch-level category activation patterns for multiple class novelty detection | |
| CN111460883A (en) | Video behavior automatic description method based on deep reinforcement learning | |
| Kang et al. | An active learning framework featured Monte Carlo dropout strategy for deep learning-based semantic segmentation of concrete cracks from images | |
| CN115909336A (en) | Text recognition method and device, computer equipment and computer-readable storage medium | |
| Hu et al. | A computer‐aided melanoma detection using deep learning and an improved African vulture optimization algorithm | |
| CN111898528B (en) | Data processing method, device, computer readable medium and electronic equipment | |
| Liu et al. | Evidence fusion theory in healthcare | |
| Zhang et al. | Lancet: labeling complex data at scale | |
| CN116883768A (en) | Lung nodule intelligent grading method and system based on multi-modal feature fusion | |
| CN116108363A (en) | Incomplete multi-view multi-label classification method and system based on label guidance | |
| CN115797642A (en) | Self-adaptive image semantic segmentation algorithm based on consistency regularization and semi-supervision field | |
| CN114255381B (en) | Training method of image recognition model, image recognition method, device and medium |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| CB03 | Change of inventor or designer information |
Inventor after: Li Yufeng Inventor after: Guo Lanzhe Inventor after: Zhou Zhihua Inventor before: Li Yufeng Inventor before: Guo Lanzhe |
|
| CB03 | Change of inventor or designer information | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |