CN114743043B - Image classification method, electronic device, storage medium and program product - Google Patents
Image classification method, electronic device, storage medium and program product Download PDFInfo
- Publication number
- CN114743043B CN114743043B CN202210253478.XA CN202210253478A CN114743043B CN 114743043 B CN114743043 B CN 114743043B CN 202210253478 A CN202210253478 A CN 202210253478A CN 114743043 B CN114743043 B CN 114743043B
- Authority
- CN
- China
- Prior art keywords
- model
- sample
- target class
- class
- classification
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
- 238000000034 method Methods 0.000 title claims abstract description 55
- 238000003860 storage Methods 0.000 title claims abstract description 13
- 238000013145 classification model Methods 0.000 claims abstract description 71
- 230000006870 function Effects 0.000 claims description 71
- 238000009826 distribution Methods 0.000 claims description 26
- 238000004590 computer program Methods 0.000 claims description 22
- 238000012545 processing Methods 0.000 abstract description 9
- 238000012549 training Methods 0.000 description 17
- 238000010586 diagram Methods 0.000 description 13
- 230000008569 process Effects 0.000 description 8
- 238000005516 engineering process Methods 0.000 description 7
- 238000013473 artificial intelligence Methods 0.000 description 6
- 241000282376 Panthera tigris Species 0.000 description 4
- 238000001514 detection method Methods 0.000 description 4
- 230000009471 action Effects 0.000 description 3
- 230000000694 effects Effects 0.000 description 3
- 238000010801 machine learning Methods 0.000 description 3
- 241000282326 Felis catus Species 0.000 description 2
- 230000003796 beauty Effects 0.000 description 2
- 238000013135 deep learning Methods 0.000 description 2
- 238000002474 experimental method Methods 0.000 description 2
- 238000002372 labelling Methods 0.000 description 2
- 238000012986 modification Methods 0.000 description 2
- 230000004048 modification Effects 0.000 description 2
- 238000011160 research Methods 0.000 description 2
- 238000012795 verification Methods 0.000 description 2
- 230000004075 alteration Effects 0.000 description 1
- 238000013528 artificial neural network Methods 0.000 description 1
- 230000003190 augmentative effect Effects 0.000 description 1
- 230000006399 behavior Effects 0.000 description 1
- 238000009529 body temperature measurement Methods 0.000 description 1
- 238000004891 communication Methods 0.000 description 1
- 239000002537 cosmetic Substances 0.000 description 1
- 230000008878 coupling Effects 0.000 description 1
- 238000010168 coupling process Methods 0.000 description 1
- 238000005859 coupling reaction Methods 0.000 description 1
- 238000011161 development Methods 0.000 description 1
- 238000004519 manufacturing process Methods 0.000 description 1
- 230000003287 optical effect Effects 0.000 description 1
- 230000002265 prevention Effects 0.000 description 1
- 230000000750 progressive effect Effects 0.000 description 1
- 230000001360 synchronised effect Effects 0.000 description 1
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/24—Classification techniques
- G06F18/241—Classification techniques relating to the classification model, e.g. parametric or non-parametric approaches
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/21—Design or setup of recognition systems or techniques; Extraction of features in feature space; Blind source separation
- G06F18/214—Generating training patterns; Bootstrap methods, e.g. bagging or boosting
Landscapes
- Engineering & Computer Science (AREA)
- Data Mining & Analysis (AREA)
- Theoretical Computer Science (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Bioinformatics & Cheminformatics (AREA)
- Bioinformatics & Computational Biology (AREA)
- Artificial Intelligence (AREA)
- Evolutionary Biology (AREA)
- Evolutionary Computation (AREA)
- Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Life Sciences & Earth Sciences (AREA)
- Image Analysis (AREA)
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
Abstract
The application provides an image classification method, electronic equipment, a storage medium and a program product, relates to the technical field of image processing, and aims to accurately classify images. The method comprises the following steps: acquiring an image to be classified; inputting the image to be classified into an image classification model to obtain a classification prediction result of the image to be classified; the image classification model learns whether each sample in a sample set is a target class classification prediction result from a target class dimension to a first model trained in advance, and learns a non-target class probability prediction result of each sample in the sample set to each non-target class from a non-target class dimension to the first model.
Description
Technical Field
The present application relates to the field of image processing technologies, and in particular, to an image classification method, an electronic device, a storage medium, and a program product.
Background
With the rapid development of machine learning, the image classification model obtained through training can classify images, and the accuracy of image classification depends on the training degree of the image classification model. In the related art, in order to train to obtain an image classification model capable of identifying whether an image is a class a image, a basic model is usually trained by using a sample set including the class a image, so that the basic model can learn the features of the class a image, and then the trained image classification model is obtained.
However, only the features of the class a images are learned based on the image classification model trained on the sample set containing the class a images, while features for other images not belonging to class a are learned poorly. Therefore, the accuracy of the image classification model obtained by training in the related art is still to be improved, and the accuracy of image classification by using the image classification model is also to be improved.
Disclosure of Invention
In view of the above, embodiments of the present invention provide an image classification method, an electronic device, a storage medium, and a program product to overcome or at least partially solve the above-described problems.
In a first aspect of an embodiment of the present invention, there is provided an image classification method, including:
Acquiring an image to be classified;
inputting the image to be classified into an image classification model to obtain a classification prediction result of the image to be classified;
The image classification model learns whether each sample in a sample set is a target class classification prediction result from a target class dimension to a first model trained in advance, and learns a non-target class probability prediction result of each sample in the sample set to each non-target class from a non-target class dimension to the first model.
Optionally, the learning of the image classification model from the target class dimension and the non-target class dimension to the first model is achieved by:
Inputting the sample set into the first model and a second model to be trained;
Acquiring a classification prediction result of whether each sample in the sample set predicted by the first model and the second model is a target class or not, and acquiring a probability prediction result of each sample in the sample set predicted by the first model and the second model as each non-target class;
Updating model parameters of the second model by taking a learning of a two-class prediction result corresponding to each sample in the sample set predicted by the first model and a learning of a probability prediction result of each sample in the sample set predicted by the first model as each non-target class as a target;
and determining the second model with the updated multiple parameters as the image classification model.
Optionally, obtaining a classification prediction result of whether each sample in the sample set predicted by each of the first model and the second model is a target class includes:
Acquiring a class probability distribution result of each sample in the sample set predicted by each of the first model and the second model;
Respectively extracting the probability of each sample in the sample set predicted by the first model and the second model as a target class from the class probability distribution result;
and determining whether each sample in the sample set predicted by the first model and the second model is a classification prediction result of the target class according to the probability that each sample is the target class.
Optionally, obtaining a probability prediction result of each sample in the sample set predicted by the first model and the second model for each non-target class includes:
Acquiring a class probability distribution result of each sample in the sample set predicted by each of the first model and the second model;
And respectively extracting the probability prediction result of each sample in the sample set predicted by the first model and the second model as each non-target class from the class probability distribution result.
Optionally, the updating the model parameters of the second model with the objective of learning the two classification prediction results corresponding to each sample in the sample set predicted by the first model and learning the probability prediction result of each sample in the sample set predicted by the first model as each non-target class includes:
establishing a two-classification loss function according to the difference between the two-classification prediction results respectively predicted by the first model and the second model for the same sample;
Establishing a non-target class loss function according to the difference between probability prediction results of the samples, which are predicted by the first model and the second model for the same sample, for each non-target class;
Obtaining a total loss function of the second model according to the two classification loss functions and the non-target class loss function;
And updating the model parameters of the second model according to the total loss function.
Optionally, obtaining a total loss function of the second model according to the two-class loss function and the non-target class loss function, including:
acquiring respective weight parameters of the two classification loss functions and the non-target class loss function;
And carrying out weighted summation on the two classification loss functions and the non-target class loss function according to respective weight parameters to obtain a total loss function of the second model.
Optionally, the target class is a class of a sample carrying a label in the sample set, and the sample set further includes a non-labeled sample; the image classification model performs supervised learning from the object class dimension with a sample carrying a label.
In a second aspect of the embodiment of the present application, an electronic device is provided, including a memory, a processor, and a computer program stored on the memory, where the processor executes the computer program to implement the image classification method disclosed in the embodiment of the present application.
In a third aspect of embodiments of the present application, there is provided a computer readable storage medium having stored thereon a computer program/instruction which, when executed by a processor, implements the image classification method as disclosed in the embodiments of the present application.
In a fourth aspect of embodiments of the present application, there is provided a computer program product comprising computer programs/instructions which when executed by a processor implement the image classification method as disclosed in embodiments of the present application.
The embodiment of the invention has the following advantages:
In this embodiment, the image classification model may be used to classify the image to be classified, where the image classification model learns from the target class dimension and the non-target class dimension, respectively, to the first model trained in advance. Because the first model is already trained, the first model can accurately distinguish between target class features and non-target class features. Thus, the image classification model of whether each sample in the sample set is a classification prediction result of the target class is learned from the dimension of the target class to the first model, and the characteristics of the target class can be accurately resolved; the image classification model of the probability prediction result of each sample in the sample set for each non-target class is learned from the non-target class dimension to the first model, so that the characteristics of each non-target class can be accurately distinguished. Therefore, the learned image classification model can accurately distinguish the target class characteristics and the non-target class characteristics, and has higher accuracy when classifying the images to be classified.
Drawings
In order to more clearly illustrate the technical solutions of the embodiments of the present application, the drawings that are needed in the description of the embodiments of the present application will be briefly described below, it being obvious that the drawings in the following description are only some embodiments of the present application, and that other drawings may be obtained according to these drawings without inventive effort for a person skilled in the art.
FIG. 1 is a flow chart of steps of a method for classifying images according to an embodiment of the present invention;
FIG. 2 is a schematic diagram of the result of a class probability distribution in an embodiment of the invention;
FIG. 3 is a schematic diagram of learning an image classification model in an embodiment of the invention;
FIG. 4 is a schematic diagram of an image classification apparatus according to an embodiment of the present invention;
fig. 5 is a schematic diagram of an electronic device according to an embodiment of the present invention.
Detailed Description
In order that the above-recited objects, features and advantages of the present application will become more readily apparent, a more particular description of the application will be rendered by reference to the appended drawings and appended detailed description.
In recent years, technology research such as computer vision, deep learning, machine learning, image processing, image recognition and the like based on artificial intelligence has been advanced significantly. Artificial intelligence (ARTIFICIAL INTELLIGENCE, AI) is an emerging scientific technology for studying and developing theories, methods, techniques and application systems for simulating and extending human intelligence. The artificial intelligence discipline is a comprehensive discipline and relates to various technical categories such as chips, big data, cloud computing, internet of things, distributed storage, deep learning, machine learning, neural networks and the like. Computer vision is an important branch of artificial intelligence, and particularly, machine recognition is a world, and computer vision technologies generally include technologies such as face recognition, living body detection, fingerprint recognition and anti-counterfeit verification, biometric feature recognition, face detection, pedestrian detection, object detection, pedestrian recognition, image processing, image recognition, image semantic understanding, image retrieval, word recognition, video processing, video content recognition, behavior recognition, three-dimensional reconstruction, virtual reality, augmented reality, synchronous positioning and map building (SLAM), computational photography, robot navigation and positioning, and the like. With research and progress of artificial intelligence technology, the technology expands application in various fields, such as fields of security prevention and control, city management, traffic management, building management, park management, face passing, face attendance, logistics management, warehouse management, robots, intelligent marketing, computed photography, mobile phone images, cloud services, intelligent home, wearable equipment, unmanned driving, automatic driving, intelligent medical treatment, face payment, face unlocking, fingerprint unlocking, personnel verification, intelligent screen, intelligent television, camera, mobile internet, network living broadcast, beauty, cosmetic, medical beauty, intelligent temperature measurement and the like.
The inventor considers that the second model is trained by using the marked sample, and the second model can learn marked information, but the information carried by the unmarked sample is not learned enough. Therefore, the second model can be trained by using the output of the first model which is trained well, has excellent performance and has high information learning degree on the unlabeled sample, so that the second model can learn the information carried by the unlabeled sample.
In the related art, in the method for guiding the second model to train by using the first model, the respective outputs of the second model and the first model are always made to be as similar as possible, so that the output result of the second model integrally learns the output result of the first model. However, this method is too strong in coupling, resulting in poor learning effect of the second model. Therefore, the inventors propose to decouple the learning process so that the second model learns the first model from the target class dimension and the non-target class dimension, respectively. Experiments prove that compared with the whole learning method, the decoupled learning method can improve the learning effect of the second model and shorten the learning time.
Referring to fig. 1, a step flowchart of an image classification method according to an embodiment of the present application is shown, and as shown in fig. 1, the image classification method may be used in electronic devices such as a computer, a mobile phone, a tablet computer, a server, etc., and the image classification method includes the following steps:
Step S11: acquiring an image to be classified;
Step S12: inputting the image to be classified into an image classification model to obtain a classification prediction result of the image to be classified; the image classification model learns whether each sample in a sample set is a target class classification prediction result from a target class dimension to a first model trained in advance, and learns a non-target class probability prediction result of each sample in the sample set to each non-target class from a non-target class dimension to the first model.
The image to be classified may be any image, such as a video frame, a photograph, a produced image, etc. And inputting the images to be classified into a trained image classification model, wherein the image classification model can predict the classification prediction result of the images to be classified. The classification prediction result may be whether the image to be classified belongs to the target image or not, or may be which type of image the image to be classified belongs to.
In the training process, the image classification model learns from the target class dimension and the non-target class dimension to the first model which is trained in advance. The target class is a class of a sample carrying a label in a sample set used when training an image classification model.
The first model is a trained and high-accuracy model, and can accurately distinguish the target class characteristics and the non-target class characteristics. The training of the first model may be training using a sample set comprising target class sample images. A sample set for training an image classification model to be trained (i.e., a second model to be described later) is predicted using the trained first model. Wherein the sample set used for training the image classification model to be trained may or may not be the sample set used for training the first model.
After the first model predicts the sample set of the image classification model to be trained, the image classification model to be trained predicts the sample set, and the image classification model to be trained learns the prediction result of the sample set to the first model, so that the classification prediction result of the image classification model can be similar to the first model, and accurate classification of the image is realized.
Specifically, the image classification model may be trained from the target class dimension and the non-target class dimension, respectively, to the first model. The method comprises the steps of obtaining a two-class prediction result of whether each sample in a sample set predicted by a first model is a target class or not, and then enabling an image classification model to learn from the dimension of the target class to the first model whether each sample in the sample set is the two-class prediction result of the target class or not. The probability prediction result of each sample in the sample set predicted by the first model for each non-target class can be obtained, and then the image classification model learns the probability prediction result of each sample in the sample set for each non-target class from the non-target class dimension to the first model.
Alternatively, each sample in the sample set used for training the image classification model may be an unlabeled sample, and the classification prediction result of each sample according to the first model may be used as a virtual label of each sample. The target class may be determined based on the characteristics of the first model that were primarily learned during the training process. For example, in the training process, the first model is trained by using a sample carrying a B-class label, and learns B-class image features, and then the B-class can be taken as the target class. The present disclosure is not limited to the training method of the first model. Therefore, when the image classification model is trained, the labeling of a sample set can be omitted, and meanwhile, the target class characteristics are learned through the learning of the first model.
However, it will be appreciated that having the image classification model learn only with respect to the first model in the target class dimension is less effective than having the image classification model learn with respect to both the first model and the labeled sample in the target class dimension. Thus, optionally, the image classification model may also be subjected to supervised learning from the target class dimension using a sample set containing labeled samples, based on learning from the first model.
The sample set comprises samples carrying target class labels and unlabeled samples, and whether each sample in the sample set is a target class sample is predicted by using an image classification model to be trained. According to the prediction result of each sample and the difference between whether the sample is actually the target class sample or not and whether the sample is carried with a label or not, a loss function is established, and an image classification model to be trained is trained based on the loss function, so that the trained image classification model can learn the target class characteristics. Thus, after the image classification model is trained, not only the labeling information in the sample set is learned in the dimension of the target class, but also the first model is learned, so that the image classification model has higher accuracy in classifying the images.
Optionally, on the basis of the above technical solution, the image classification model may be a model with a smaller parameter, so that the image classification model with a smaller parameter learns the target class features according to the sample set, which generally requires a longer learning time, and only learns the target class features, and even if the learning effect of the target class features is still to be improved, the accuracy of the image classification model after learning the sample set is still not high. The first model may be a model with a large parameter, which can accurately distinguish between target class features and non-target class features after training. Therefore, the image classification model learns from the target dimension and the non-target dimension to the image classification model, and the accuracy of the image classification model is improved.
By adopting the technical scheme of the embodiment of the application, the image to be classified can be classified by utilizing the image classification model, and the image classification model learns from the target class dimension and the non-target class dimension to the first model which is trained in advance respectively. Because the first model is already trained, the first model can accurately distinguish between target class features and non-target class features. Thus, the image classification model of whether each sample in the sample set is a classification prediction result of the target class is learned from the dimension of the target class to the first model, and the characteristics of the target class can be accurately resolved; the image classification model of the probability prediction result of each sample in the sample set for each non-target class is learned from the non-target class dimension to the first model, so that the characteristics of each non-target class can be accurately distinguished. Therefore, the learned image classification model can accurately distinguish the target class characteristics and the non-target class characteristics, and has higher accuracy when classifying the images to be classified.
Optionally, on the basis of the above technical solution, the learning of the image classification model from the target class dimension and the non-target class dimension to the first model may be achieved by: inputting the same sample set into a trained first model and a second model to be trained; acquiring a classification prediction result of whether each sample in a sample set predicted by the first model and the second model is a target class or not, and acquiring a probability prediction result of each sample in the sample set predicted by the first model and the second model as each non-target class; allowing the second model to learn the two classification prediction results corresponding to each sample in the sample set predicted by the first model, and learning the probability prediction result of each sample in the sample set predicted by the first model as each non-target class, continuously learning, and updating the model parameters of the second model; and determining the second model with the updated multiple parameters as an image classification model.
The two-class prediction result for predicting whether the sample is the target class refers to a result of predicting whether the sample or not belongs to the target class, for example, when the target class is tiger, the two-class prediction result of an image may be "the image belongs to tiger" or "the image does not belong to tiger". Alternatively, the classification prediction result may refer to a probability size that the prediction sample belongs to the target class and a probability size that the prediction sample does not belong to the target class. It will be appreciated that the sum of the probability that an image belongs to a target class and the probability that it does not belong to the target class is 1.
The probability prediction result of the prediction sample for each non-target class refers to the respective probability that the prediction sample belongs to each non-target class. For example, when the target class is tiger, the non-target class may be cat, dog, background, etc., and the probability prediction result of one image for each non-target class may be: the probability of cat was 0.07, the probability of dog was 0.06, the probability of background was 0.02.
The first model is a model with high accuracy, so after the two classification prediction results corresponding to each sample predicted by the first model and the second model respectively and the probability prediction result of each sample being each non-target class are obtained, the second model is enabled to learn the two classification prediction results of the first model and the probability prediction result of each non-target class, so that the second model can more accurately realize classification of the samples, including whether the samples are classified as target classes or not and the probability of each non-target class.
Optionally, based on the above technical solution, the class probability distribution result of each sample in the sample set predicted by each of the first model and the second model may be obtained first, where the class probability distribution result of one sample includes the probability that the sample is the target class and the probability that the sample is each non-target class. Fig. 2 shows a schematic diagram of the result of a class probability distribution, wherein differently shaped columns represent different classes, the height of each column representing the probability magnitude. In the probability distribution results of the categories, the categories are distributed in the same order.
After the class probability distribution result of each sample in the sample set predicted by the first model and the second model is obtained, the probability that the sample is the target class can be extracted from each class probability distribution result. Because the sum of the probability that a sample is a target class and the probability that the sample is a non-target class is 1, the probability that the sample is a non-target class can be obtained according to the probability that the sample is a target class; the probability that the sample is a non-target class can also be obtained according to the sum of the probabilities that the sample is each non-target class. Further, the probability that the sample is the target class and the probability that the sample is the non-target class can be compared, so that a two-class prediction result of the sample can be obtained. For example, the first model predicts the class probability distribution result of one sample as: class a (target class) is 0.7, class b is 0.2, class c is 0.1, then the probability of the sample being a non-target class is 0.3, based on 1-0.7=0.3 or based on 0.2+0.1=0.3; by comparing the sizes of 0.7 and 0.3, the first model can obtain the classification prediction result of predicting the sample as the target class.
Alternatively, from each class probability distribution result, the probability that the sample is each non-target class may also be extracted. For example, the first model predicts the class probability distribution result of one sample as: class a (target class) is 0.7, class B is 0.2, class C is 0.1, then the sample can be extracted with a class B probability of 0.2, and a class C probability of 0.1.
Alternatively, on the basis of the above technical solution, after the two-class prediction results of the first model and the second model for each sample are obtained, a two-class loss function may be established according to the difference between the two-class prediction results of the first model and the second model for the same sample, respectively. The second model can be guided to learn towards the first model by the aid of the two classification loss functions, and the two classification prediction results of the learned second model on the images can be more accurate.
After obtaining the probability prediction result of each sample predicted by the first model and the second model as each non-target class, a non-target class loss function is established according to the difference between the probability prediction results of the sample predicted by the first model and the second model as each non-target class for the same sample. The non-target class loss function can guide the second model to learn towards the first model, and the learned second model is more accurate when judging whether the image belongs to each non-target class.
After the two kinds of the loss functions and the non-target loss function are obtained, a total loss function of the second model can be obtained according to the two kinds of the loss functions, and model parameters of the second model are updated according to the total loss function so as to obtain the image classification model.
Optionally, on the basis of the above technical solution, respective weight parameters may be applied to the two-class loss function and the non-target class loss function, and the two-class loss function and the non-target class loss function to which the respective weight parameters are applied may be summed to obtain the total loss function of the second model.
The weight parameters of the two classification loss functions and the non-target class loss function can be super parameters obtained according to experiments. The second model may be better trained than the non-target class loss function and the class-two class loss function to which the weight parameter is applied.
Optionally, fig. 3 shows a learning schematic of the image classification model. Predicting the first model after training and the second model to be trained aiming at the same sample to obtain two kinds of probability distribution results of the sample; respectively extracting target class probability from the two class probability distribution results, obtaining non-target class probability, and obtaining a two-class prediction result according to the target class probability and the non-target class probability; establishing a two-class loss function according to respective two-class prediction results of the two models; respectively extracting the probability of the sample as each non-target class from the probability distribution results of the two classes, and respectively predicting the probability of the sample as each non-target class according to the two models to establish a non-target class loss function; the total loss function for training the second model is derived from the two-class loss function and the non-target class loss function. The trained second model is the image classification model.
Compared with the method that the second model directly learns the class probability distribution result output by the first model, the second model provided by the disclosure learns from the target class dimension and the non-target class dimension to the first model respectively, and the learned second model has higher accuracy.
It should be noted that, for simplicity of description, the method embodiments are shown as a series of acts, but it should be understood by those skilled in the art that the embodiments are not limited by the order of acts, as some steps may occur in other orders or concurrently in accordance with the embodiments. Further, those skilled in the art will appreciate that the embodiments described in the specification are presently preferred embodiments, and that the acts are not necessarily required by the embodiments of the invention.
Fig. 4 is a schematic structural diagram of an image classification apparatus according to an embodiment of the present invention, as shown in fig. 4, the image classification apparatus includes an image acquisition module and a result prediction module, wherein:
The image acquisition module is used for acquiring images to be classified;
The result prediction module is used for inputting the image to be classified into an image classification model to obtain a classification prediction result of the image to be classified; the image classification model learns whether each sample in a sample set is a target class classification prediction result from a target class dimension to a first model trained in advance, and learns a non-target class probability prediction result of each sample in the sample set to each non-target class from a non-target class dimension to the first model.
Optionally, the learning of the image classification model from the target class dimension and the non-target class dimension to the first model is achieved by:
Inputting the sample set into the first model and a second model to be trained;
Acquiring a classification prediction result of whether each sample in the sample set predicted by the first model and the second model is a target class or not, and acquiring a probability prediction result of each sample in the sample set predicted by the first model and the second model as each non-target class;
Updating model parameters of the second model by taking a learning of a two-class prediction result corresponding to each sample in the sample set predicted by the first model and a learning of a probability prediction result of each sample in the sample set predicted by the first model as each non-target class as a target;
and determining the second model with the updated multiple parameters as the image classification model.
Optionally, obtaining a classification prediction result of whether each sample in the sample set predicted by each of the first model and the second model is a target class includes:
Acquiring a class probability distribution result of each sample in the sample set predicted by each of the first model and the second model;
Respectively extracting the probability of each sample in the sample set predicted by the first model and the second model as a target class from the class probability distribution result;
and determining whether each sample in the sample set predicted by the first model and the second model is a classification prediction result of the target class according to the probability that each sample is the target class.
Optionally, obtaining a probability prediction result of each sample in the sample set predicted by the first model and the second model for each non-target class includes:
Acquiring a class probability distribution result of each sample in the sample set predicted by each of the first model and the second model;
And respectively extracting the probability prediction result of each sample in the sample set predicted by the first model and the second model as each non-target class from the class probability distribution result.
Optionally, the updating the model parameters of the second model with the objective of learning the two classification prediction results corresponding to each sample in the sample set predicted by the first model and learning the probability prediction result of each sample in the sample set predicted by the first model as each non-target class includes:
establishing a two-classification loss function according to the difference between the two-classification prediction results respectively predicted by the first model and the second model for the same sample;
Establishing a non-target class loss function according to the difference between probability prediction results of the samples, which are predicted by the first model and the second model for the same sample, for each non-target class;
Obtaining a total loss function of the second model according to the two classification loss functions and the non-target class loss function;
And updating the model parameters of the second model according to the total loss function.
Optionally, obtaining a total loss function of the second model according to the two-class loss function and the non-target class loss function, including:
acquiring respective weight parameters of the two classification loss functions and the non-target class loss function;
And carrying out weighted summation on the two classification loss functions and the non-target class loss function according to respective weight parameters to obtain a total loss function of the second model.
Optionally, the target class is a class of a sample carrying a label in the sample set, and the sample set further includes a non-labeled sample; the image classification model performs supervised learning from the object class dimension with a sample carrying a label.
It should be noted that, the device embodiment is similar to the method embodiment, so the description is simpler, and the relevant places refer to the method embodiment.
The embodiment of the application also provides an electronic device, and referring to fig. 5, fig. 5 is a schematic diagram of the electronic device according to the embodiment of the application. As shown in fig. 5, the electronic device 100 includes: the image classification method comprises a memory 110 and a processor 120, wherein the memory 110 is in communication connection with the processor 120 through a bus, and a computer program is stored in the memory 110 and can run on the processor 120, so that the steps in the image classification method disclosed by the embodiment of the application are realized.
The embodiment of the application also provides a computer readable storage medium, on which a computer program/instruction is stored, which when executed by a processor, implements the image classification method as disclosed in the embodiment of the application.
The embodiment of the application also provides a computer program product, which comprises a computer program/instruction, wherein the computer program/instruction realizes the image classification method disclosed by the embodiment of the application when being executed by a processor.
In this specification, each embodiment is described in a progressive manner, and each embodiment is mainly described by differences from other embodiments, and identical and similar parts between the embodiments are all enough to be referred to each other.
It will be apparent to those skilled in the art that embodiments of the present invention may be provided as a method, apparatus, or computer program product. Accordingly, embodiments of the present invention may take the form of an entirely hardware embodiment, an entirely software embodiment or an embodiment combining software and hardware aspects. Furthermore, embodiments of the invention may take the form of a computer program product on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, etc.) having computer-usable program code embodied therein.
Embodiments of the present invention are described with reference to flowchart illustrations and/or block diagrams of methods, apparatus, electronic devices, and computer program products according to embodiments of the invention. It will be understood that each flow and/or block of the flowchart illustrations and/or block diagrams, and combinations of flows and/or blocks in the flowchart illustrations and/or block diagrams, can be implemented by computer program instructions. These computer program instructions may be provided to a processor of a general purpose computer, special purpose computer, embedded processor, or other programmable data processing terminal device to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing terminal device, create means for implementing the functions specified in the flowchart flow or flows and/or block diagram block or blocks.
These computer program instructions may also be stored in a computer-readable memory that can direct a computer or other programmable data processing apparatus to function in a particular manner, such that the instructions stored in the computer-readable memory produce an article of manufacture including instruction means which implement the function specified in the flowchart flow or flows and/or block diagram block or blocks.
These computer program instructions may also be loaded onto a computer or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer implemented process such that the instructions which execute on the computer or other programmable apparatus provide steps for implementing the functions specified in the flowchart flow or flows and/or block diagram block or blocks.
While preferred embodiments of the present invention have been described, additional variations and modifications in those embodiments may occur to those skilled in the art once they learn of the basic inventive concepts. It is therefore intended that the following claims be interpreted as including the preferred embodiment and all such alterations and modifications as fall within the scope of the embodiments of the invention.
Finally, it is further noted that relational terms such as first and second, and the like are used solely to distinguish one entity or action from another entity or action without necessarily requiring or implying any actual such relationship or order between such entities or actions. Moreover, the terms "comprises," "comprising," or any other variation thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or terminal that comprises a list of elements does not include only those elements but may include other elements not expressly listed or inherent to such process, method, article, or terminal. Without further limitation, an element defined by the phrase "comprising one … …" does not exclude the presence of other like elements in a process, method, article, or terminal device that comprises the element.
The above description of the image classification method, the electronic device, the storage medium and the program product provided by the present application applies specific examples to illustrate the principles and the implementation of the present application, and the above examples are only used to help understand the method and the core idea of the present application; meanwhile, as those skilled in the art will have variations in the specific embodiments and application scope in accordance with the ideas of the present application, the present description should not be construed as limiting the present application in view of the above.
Claims (10)
1. An image classification method, comprising:
Acquiring an image to be classified;
inputting the image to be classified into an image classification model to obtain a classification prediction result of the image to be classified;
The image classification model learns whether each sample in a sample set is a target class classification prediction result from a target class dimension to a first model trained in advance, and learns a non-target class probability prediction result of each sample in the sample set to each non-target class from a non-target class dimension to the first model.
2. The method of claim 1, wherein the image classification model learns from the target class dimension and the non-target class dimension to the first model by:
Inputting the sample set into the first model and a second model to be trained;
Acquiring a classification prediction result of whether each sample in the sample set predicted by the first model and the second model is a target class or not, and acquiring a probability prediction result of each sample in the sample set predicted by the first model and the second model as each non-target class;
Updating model parameters of the second model by taking a learning of a two-class prediction result corresponding to each sample in the sample set predicted by the first model and a learning of a probability prediction result of each sample in the sample set predicted by the first model as each non-target class as a target;
and determining the second model with the updated multiple parameters as the image classification model.
3. The method of claim 2, wherein obtaining a binary class prediction result for each sample in the set of samples predicted by each of the first model and the second model, comprises:
Acquiring a class probability distribution result of each sample in the sample set predicted by each of the first model and the second model;
Respectively extracting the probability of each sample in the sample set predicted by the first model and the second model as a target class from the class probability distribution result;
and determining whether each sample in the sample set predicted by the first model and the second model is a classification prediction result of the target class according to the probability that each sample is the target class.
4. The method of claim 2, wherein obtaining a probability prediction result for each sample in the set of samples predicted by each of the first model and the second model for each non-target class comprises:
Acquiring a class probability distribution result of each sample in the sample set predicted by each of the first model and the second model;
And respectively extracting the probability prediction result of each sample in the sample set predicted by the first model and the second model as each non-target class from the class probability distribution result.
5. The method according to any one of claims 2-4, wherein updating model parameters of the second model with the objective of learning a binary class prediction result corresponding to each sample in the sample set predicted by the first model, and learning a probability prediction result for each sample in the sample set predicted by the first model as each non-target class, comprises:
establishing a two-classification loss function according to the difference between the two-classification prediction results respectively predicted by the first model and the second model for the same sample;
Establishing a non-target class loss function according to the difference between probability prediction results of the samples, which are predicted by the first model and the second model for the same sample, for each non-target class;
Obtaining a total loss function of the second model according to the two classification loss functions and the non-target class loss function;
And updating the model parameters of the second model according to the total loss function.
6. The method of claim 5, wherein deriving the total loss function of the second model from the two classification loss functions and the non-target class loss function comprises:
acquiring respective weight parameters of the two classification loss functions and the non-target class loss function;
And carrying out weighted summation on the two classification loss functions and the non-target class loss function according to respective weight parameters to obtain a total loss function of the second model.
7. The method of any of claims 1-6, wherein the target class is a class of samples in the sample set that carry labels, the sample set further comprising unlabeled samples; the image classification model performs supervised learning from the object class dimension with a sample carrying a label.
8. An electronic device comprising a memory, a processor and a computer program stored on the memory, characterized in that the processor executes the computer program to implement the image classification method of any of claims 1 to 7.
9. A computer-readable storage medium, on which a computer program/instruction is stored which, when executed by a processor, implements the image classification method according to any one of claims 1 to 7.
10. A computer program product comprising computer programs/instructions which, when executed by a processor, implement the image classification method according to any one of claims 1 to 7.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202210253478.XA CN114743043B (en) | 2022-03-15 | 2022-03-15 | Image classification method, electronic device, storage medium and program product |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202210253478.XA CN114743043B (en) | 2022-03-15 | 2022-03-15 | Image classification method, electronic device, storage medium and program product |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN114743043A CN114743043A (en) | 2022-07-12 |
| CN114743043B true CN114743043B (en) | 2024-04-26 |
Family
ID=82276176
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202210253478.XA Active CN114743043B (en) | 2022-03-15 | 2022-03-15 | Image classification method, electronic device, storage medium and program product |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN114743043B (en) |
Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN110378366A (en) * | 2019-06-04 | 2019-10-25 | 广东工业大学 | A kind of cross-domain image classification method based on coupling knowledge migration |
| WO2020073951A1 (en) * | 2018-10-10 | 2020-04-16 | 腾讯科技（深圳）有限公司 | Method and apparatus for training image recognition model, network device, and storage medium |
| CN111753863A (en) * | 2019-04-12 | 2020-10-09 | 北京京东尚科信息技术有限公司 | Image classification method and device, electronic equipment and storage medium |
| CN112529188A (en) * | 2021-02-18 | 2021-03-19 | 中国科学院自动化研究所 | Knowledge distillation-based industrial process optimization decision model migration optimization method |
| WO2022016556A1 (en) * | 2020-07-24 | 2022-01-27 | 华为技术有限公司 | Neural network distillation method and apparatus |
| WO2022042043A1 (en) * | 2020-08-27 | 2022-03-03 | 京东方科技集团股份有限公司 | Machine learning model training method and apparatus, and electronic device |
-
2022
- 2022-03-15 CN CN202210253478.XA patent/CN114743043B/en active Active
Patent Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| WO2020073951A1 (en) * | 2018-10-10 | 2020-04-16 | 腾讯科技（深圳）有限公司 | Method and apparatus for training image recognition model, network device, and storage medium |
| CN111753863A (en) * | 2019-04-12 | 2020-10-09 | 北京京东尚科信息技术有限公司 | Image classification method and device, electronic equipment and storage medium |
| CN110378366A (en) * | 2019-06-04 | 2019-10-25 | 广东工业大学 | A kind of cross-domain image classification method based on coupling knowledge migration |
| WO2022016556A1 (en) * | 2020-07-24 | 2022-01-27 | 华为技术有限公司 | Neural network distillation method and apparatus |
| WO2022042043A1 (en) * | 2020-08-27 | 2022-03-03 | 京东方科技集团股份有限公司 | Machine learning model training method and apparatus, and electronic device |
| CN112529188A (en) * | 2021-02-18 | 2021-03-19 | 中国科学院自动化研究所 | Knowledge distillation-based industrial process optimization decision model migration optimization method |
Non-Patent Citations (1)
| Title |
|---|
| 基于卷积核剪枝的遥感目标检测模型压缩方法;韩要昌 等;《火力与指挥控制》;20210228;全文 * |
Also Published As
| Publication number | Publication date |
|---|---|
| CN114743043A (en) | 2022-07-12 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN110414432B (en) | Training method of object recognition model, object recognition method and corresponding device | |
| CN111222500B (en) | Label extraction method and device | |
| CN113139628B (en) | Sample image identification method, device and equipment and readable storage medium | |
| CN111428448B (en) | Text generation method, device, computer equipment and readable storage medium | |
| CN112070071B (en) | Method and device for labeling objects in video, computer equipment and storage medium | |
| CN112165639B (en) | Content distribution method, device, electronic equipment and storage medium | |
| CN113569627A (en) | Human body posture prediction model training method, human body posture prediction method and device | |
| CN116385850A (en) | Multi-target detection method, device, electronic equipment and storage medium | |
| CN114372580A (en) | Model training method, storage medium, electronic device, and computer program product | |
| CN111898528B (en) | Data processing method, device, computer readable medium and electronic equipment | |
| CN113705293A (en) | Image scene recognition method, device, equipment and readable storage medium | |
| CN117765432A (en) | Motion boundary prediction-based middle school physical and chemical life experiment motion detection method | |
| CN114743043B (en) | Image classification method, electronic device, storage medium and program product | |
| CN114385846A (en) | Image classification method, electronic device, storage medium and program product | |
| CN112529116B (en) | Scene element fusion processing method, device and equipment and computer storage medium | |
| CN114005017A (en) | Target detection method and device, electronic equipment and storage medium | |
| CN114627085A (en) | Target image identification method and device, storage medium and electronic equipment | |
| CN111582404A (en) | Content classification method and device and readable storage medium | |
| CN116758332A (en) | Training method of scene classification model, electronic equipment and storage medium | |
| CN114821150A (en) | Image classification method, electronic device, storage medium and program product | |
| CN115147455A (en) | Optical flow prediction method, electronic device, storage medium, and program product | |
| CN114387547A (en) | Method, device, medium and program product for determining behavior video clip | |
| CN117011662A (en) | Training method, training device, training equipment, training medium and training program product for face recognition network | |
| CN114821026A (en) | Object retrieval method, electronic device, and computer-readable medium | |
| CN116933070A (en) | Content identification method, apparatus, device, storage medium, and program product |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |