CN110321952B

CN110321952B - Training method of image classification model and related equipment

Info

Publication number: CN110321952B
Application number: CN201910592095.3A
Authority: CN
Inventors: 王晓宁; 孙钟前; 付星辉; 郑瀚; 杨巍
Original assignee: Tencent Healthcare Shenzhen Co Ltd
Current assignee: Tencent Healthcare Shenzhen Co Ltd
Priority date: 2019-07-02
Filing date: 2019-07-02
Publication date: 2024-02-09
Anticipated expiration: 2039-07-02
Also published as: CN110321952A

Abstract

The embodiment of the invention discloses a training method of an image classification model and related equipment, wherein the method comprises the following steps: acquiring a training image set; inputting the related training images into an image classification learning model to obtain the prediction classification probability of the related training images for each target category, and inputting the unrelated training images into the image classification learning model to obtain the prediction classification probability of the unrelated training images for each target category; determining the probability distribution difference degree between the real probability distribution and the predicted probability distribution of the target category under the related training image; determining the uncertainty of the prediction classification of the related training image, and determining the uncertainty of the prediction classification of the unrelated training image; and adjusting the current model parameters of the image classification learning model according to the probability distribution difference degree, the prediction classification uncertainty of the related training images and the prediction classification uncertainty of the unrelated training images. The generalization capability of the image classification model can be improved through the embodiment of the invention.

Description

Training method of image classification model and related equipment

Technical Field

The present disclosure relates to the field of machine learning, and in particular, to a training method for an image classification model and related devices.

Background

With the development of the artificial intelligence field, more and more work can be realized through a computer, wherein the image classification is that the computer classifies the image into a certain type of image classification according to the characteristics of the image by extracting the characteristics of the image, and replaces the technology of human interpretation classification through vision. Currently, there are many algorithms for image classification, such as KNN (K-nearest neighbor classifier), SVM (support vector machine), CNN (convolutional neural network), etc., and CNN is a mainstream method in the field of image classification because it does not need to perform operations such as preprocessing and additional feature extraction manually.

In the image classification process through the CNN-based image classification model, the image classification model extracts image features of an input image, and further calculates the prediction probability of the input image for each preset category according to the image features, and further determines the category corresponding to the maximum prediction probability as the classification of the input object. For example, there are two preset categories of classifying the image classification model, namely, a cat and a dog, respectively, that is, after any image is input into the image classification model, the image classification model calculates the prediction probability that the image is a cat and a dog, and then outputs a category (one of a cat and a dog) corresponding to the relatively high prediction probability. In some cases, inputting the images of the cat or the dog into the image classification model can output accurate classification, in other cases, after inputting the images of the cat or the dog outside into the image classification model, one of the cat or the dog is still output, and the prediction probability corresponding to the inside of the model is higher, for example, 0.8 or more than 0.8, that is, the image classification model misjudges the images of the cat or the dog outside into the cat or the dog more certainly, the current training method of the image classification model is described, so that the image classification model cannot reasonably predict the input images which do not belong to the preset category, and further the generalization capability of the image classification model is weaker.

Disclosure of Invention

The invention provides a training method and related equipment for an image classification model, and the generalization capability of the image classification model can be improved through the training method and the related equipment.

In one aspect, the embodiment of the invention provides a training method for an image classification model, which comprises the following steps:

acquiring a training image set, wherein the training image set comprises related training images under different target categories and irrelevant training images which do not belong to the target categories, and the related training images carry category labels corresponding to the respective target categories;

inputting the related training images into an image classification learning model to obtain the prediction classification probability of the related training images for each target category, and inputting the unrelated training images into the image classification learning model to obtain the prediction classification probability of the unrelated training images for each target category;

determining the probability distribution difference degree between the real probability distribution and the predicted probability distribution of the target category under the relevant training image according to the category label corresponding to the target category of the relevant training image and the predicted classification probability of the relevant training image for each target category;

Determining the prediction classification uncertainty of the related training image according to the prediction classification probability of the related training image for each target category, and determining the prediction classification uncertainty of the unrelated training image according to the prediction classification probability of the unrelated training image for each target category;

and adjusting current model parameters of the image classification learning model according to the probability distribution difference degree, the prediction classification uncertainty of the related training image and the prediction classification uncertainty of the unrelated training image, so that the probability distribution difference degree determined by the adjusted image classification learning model, the determined prediction classification uncertainty of the related training image and the determined prediction classification uncertainty of the unrelated training image meet preset adjustment result conditions.

Wherein, the determining the probability distribution difference degree between the real probability distribution and the predicted probability distribution of the target category under the related training image according to the category label corresponding to the target category of the related training image and the predicted classification probability of the related training image for each target category comprises:

And determining the classification cross entropy of the related training image according to the class label corresponding to the target class of the related training image and the prediction classification probability of the related training image for each target class, and determining the classification cross entropy of the related training image as the probability distribution difference degree.

Wherein the determining the prediction classification uncertainty of the related training image according to the prediction classification probability of the related training image for each target category, and determining the prediction classification uncertainty of the unrelated training image according to the prediction classification probability of the unrelated training image for each target category includes:

determining the classification information entropy of the related training image according to the prediction classification probability of the related training image for each target class, and determining the classification information entropy of the related training image as the prediction classification uncertainty of the related training image;

and determining the classification information entropy of the irrelevant training images according to the prediction classification probability of the irrelevant training images for each target class, and determining the classification information entropy of the irrelevant training images as the prediction classification uncertainty of the irrelevant training images.

Wherein the method further comprises:

respectively carrying out normalization processing on the prediction classification uncertainty of the related training image and the prediction classification uncertainty of the irrelevant training image to obtain the normalization uncertainty of the related training image and the normalization uncertainty of the irrelevant training image;

the adjusting the current model parameters of the image classification learning model according to the probability distribution difference degree, the prediction classification uncertainty of the related training image and the prediction classification uncertainty of the unrelated training image comprises:

and adjusting current model parameters of the image classification learning model according to the probability distribution difference degree, the normalization uncertainty of the related training images and the normalization uncertainty of the unrelated training images.

The normalizing the uncertainty of the prediction classification of the related training image and the uncertainty of the prediction classification of the unrelated training image respectively to obtain the normalized uncertainty of the related training image and the normalized uncertainty of the unrelated training image comprises:

determining the maximum classification uncertainty of the target class under the condition of uniform probability distribution;

And determining the ratio of the predicted classification uncertainty of the related training image to the maximum classification uncertainty as the normalized uncertainty of the related training image, and determining the ratio of the predicted classification uncertainty of the unrelated training image to the maximum classification uncertainty as the normalized uncertainty of the unrelated training object.

Wherein adjusting the current model parameters of the image classification learning model according to the probability distribution variability, the prediction classification uncertainty of the related training image, and the prediction classification uncertainty of the unrelated training image comprises:

constructing a classification loss function corresponding to the current model parameters of the image classification learning model according to the probability distribution difference degree, the prediction classification uncertainty of the related training image and the prediction classification uncertainty of the unrelated training image;

performing bias guide on the classification loss function, and determining a loss gradient corresponding to the current model parameter of the image classification learning model;

and adjusting the current model parameters of the image classification learning model according to the loss gradient corresponding to the current model parameters of the image classification learning model and a preset parameter learning rate.

The constructing a classification loss function corresponding to the current model parameter of the image classification learning model according to the probability distribution difference degree, the prediction classification uncertainty of the related training image and the prediction classification uncertainty of the unrelated training image comprises:

the classification loss function L corresponding to the current model parameters of the image classification learning model is as follows:

L＝L _c +αL _r -βL _ur ，

wherein L is _c For the probability distribution difference degree, alpha is a preset weight corresponding to the prediction classification uncertainty of the related training image, L _r For the uncertainty of the prediction classification of the related training image, beta is the preset weight corresponding to the uncertainty of the prediction classification of the unrelated training image, L _ur And classifying uncertainty for prediction of the unrelated training image.

Wherein, the determining the classification cross entropy of each relevant training image according to the class label corresponding to the target class of the relevant training image and the prediction classification probability of the relevant training image for each target class comprises:

classification cross entropy L of the related training images _ce-ur The method comprises the following steps:

wherein i is the index of the related training image, N is the number of the related training images, i and N are positive integers, i is not more than N, j is the index of the target category, K is the category number of the target category, j and K are positive integers, j is not more than K, y _j For the class label corresponding to the target class j, p _ij The probability of the predicted classification for the target class j for the relevant training image i.

Wherein, the determining the classification information entropy of the relevant training image according to the prediction classification probability of the relevant training image for each target class includes:

classification information entropy L of the related training images _e-r The method comprises the following steps:

wherein i is the index of the related training image, N is the number of the related training images, i and N are positive integers, i is not more than N, j is the index of the target category, K is the category number of the target category, j and K are positive integers, j is not more than K, and p _ij The prediction classification probability of the related training image i for the target class j is calculated;

the determining the classification information entropy of the irrelevant training images according to the prediction classification probability of the irrelevant training images for each target class comprises:

classification information entropy L of the irrelevant training images _e-ur The method comprises the following steps:

wherein l is the index of the irrelevant training images, M is the number of the irrelevant training images, l and M are positive integers, l is less than or equal to M, j is the index of the target category, K is the category number of the target category, j and K are positive integers, j is less than or equal to K, g _lj The probability of predictive classification for the target class j for the unrelated training image l.

Another aspect of the embodiment of the present invention provides a training device for an image classification model, including:

the image set acquisition module is used for acquiring a training image set, wherein the training image set comprises related training images under different target categories and irrelevant training images which do not belong to the target categories, and the related training images carry category labels corresponding to the respective target categories;

the confidence degree determining module is used for inputting the related training images into an image classification learning model to obtain the prediction classification probability of the related training images for each target category, and inputting the irrelevant training images into the image classification learning model to obtain the prediction classification probability of the irrelevant training images for each target category;

the difference degree determining module is used for determining the probability distribution difference degree between the real probability distribution and the predicted probability distribution of the target category under the related training image according to the category label corresponding to the target category of the related training image and the predicted classification probability of the related training image for each target category;

The uncertainty determining module is used for determining the uncertainty of the prediction classification of the related training image according to the prediction classification probability of the related training image for each target category and determining the uncertainty of the prediction classification of the unrelated training image according to the prediction classification probability of the unrelated training image for each target category;

and the parameter adjusting module is used for adjusting the current model parameters of the image classification learning model according to the probability distribution difference degree, the prediction classification uncertainty of the related training image and the prediction classification uncertainty of the unrelated training image, so that the probability distribution difference degree determined by the adjusted image classification learning model, the determined prediction classification uncertainty of the related training image and the determined prediction classification uncertainty of the unrelated training image meet preset adjusting result conditions.

The difference degree determining module is specifically configured to determine a classification cross entropy of the related training image according to a class label corresponding to a target class of the related training image and a prediction classification probability of the related training image for each target class, and determine the classification cross entropy of the related training image as the probability distribution difference degree.

The uncertainty determining module is specifically configured to determine, according to the prediction classification probabilities of the relevant training images for the target categories, classification information entropy of the relevant training images, and determine the classification information entropy of the relevant training images as the prediction classification uncertainty of the relevant training images;

The device further comprises a normalization module, wherein the normalization module is used for respectively normalizing the prediction classification uncertainty of the related training image and the prediction classification uncertainty of the irrelevant training image to obtain the normalization uncertainty of the related training image and the normalization uncertainty of the irrelevant training image;

the parameter adjusting module is specifically configured to adjust a current model parameter of the image classification learning model according to the probability distribution difference degree, the normalization uncertainty of the related training image, and the normalization uncertainty of the unrelated training image.

The normalization module comprises a maximum uncertainty determination unit and an uncertainty normalization unit:

the maximum uncertainty determining unit is used for determining the maximum classification uncertainty of the target category under the condition that probabilities are uniformly distributed;

the uncertainty normalization unit is configured to determine a ratio of the predicted classification uncertainty of the related training image to the maximum classification uncertainty as a normalized uncertainty of the related training image, and determine a ratio of the predicted classification uncertainty of the unrelated training image to the maximum classification uncertainty as a normalized uncertainty of the unrelated training object.

The parameter adjusting module comprises a loss function constructing unit, a gradient determining unit and a parameter adjusting unit:

the loss function construction unit is used for constructing a classification loss function corresponding to the current model parameters of the image classification learning model according to the probability distribution difference degree, the prediction classification uncertainty of the related training image and the prediction classification uncertainty of the unrelated training image;

the gradient determining unit is used for performing bias guide on the classification loss function and determining a loss gradient corresponding to the current model parameter of the image classification learning model;

The parameter adjusting unit is used for adjusting the current model parameters of the image classification learning model according to the loss gradient corresponding to the current model parameters of the image classification learning model and a preset parameter learning rate.

The loss function construction unit is specifically configured to construct a classification loss function L corresponding to a current model parameter of the image classification learning model, where the classification loss function L is:

L＝L _c +αL _r -βL _ur ，

Wherein the difference determining module is specifically configured to determine a classification cross entropy L of the related training image _ce-ur The method comprises the following steps:

wherein i is the index of the related training image, N is the number of the related training images, i and N are positive integers, i is not more than N, j is the index of the target category, K is the category number of the target category, j and K are positive integers, j is not more than K, y _j For target class j Class labels, p _ij The probability of the predicted classification for the target class j for the relevant training image i.

Wherein the uncertainty determination module is specifically configured to determine a classification information entropy L of the relevant training image _e-r The method comprises the following steps:

determining the classification information entropy L of the irrelevant training images _e-ur The method comprises the following steps:

Another aspect of the embodiment of the present invention provides a training device for an image classification model, including: a processor and a memory;

the processor is connected to a memory, wherein the memory is configured to store program code, and the processor is configured to invoke the program code to perform a method according to an embodiment of the present invention.

Embodiments of the present invention also provide a computer readable storage medium storing a computer program comprising program instructions which, when executed by a processor, perform a method as in embodiments of the present invention.

In the embodiment of the invention, a training image set comprising a relevant training image and an irrelevant training image is obtained, the relevant training image is input into an image classification learning model to obtain the prediction classification probability of the relevant training image for each target class, the irrelevant training image is input into an image classification learning model to obtain the prediction classification probability of the irrelevant training image for each target class, then the probability distribution difference degree between the true probability distribution and the prediction probability distribution of the target class under the relevant training image is determined according to the class label corresponding to the target class of the relevant training image and the prediction classification probability of each target class of the relevant training image, the prediction classification uncertainty of the relevant training image is determined according to the prediction classification probability of the relevant training image for each target class, and then the prediction classification uncertainty of the irrelevant training image is determined according to the probability distribution difference degree, the prediction classification uncertainty of the relevant training image and the prediction classification uncertainty of the irrelevant training image, and the current model parameters of the image classification learning model are adjusted. In the training process of the image classification learning model, the classification precision of the image classification learning model is guided and optimized through the probability distribution difference, and the confidence level of the prediction probability of the image classification learning model is guided and optimized through the prediction classification uncertainty of the related training image and the prediction classification uncertainty of the unrelated training image, so that the generalization capability of the image classification model obtained through training is improved.

Drawings

In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings that are needed in the embodiments will be briefly described below, and it is obvious that the drawings in the following description are only some embodiments of the present invention, and other drawings may be obtained according to these drawings without inventive effort for a person skilled in the art.

Fig. 1a is a schematic diagram of a network structure of a DenseNet according to an embodiment of the present invention;

fig. 1b is a schematic diagram of a classification mechanism of a CNN-based image classification model according to an embodiment of the present invention;

FIG. 1c is a schematic diagram of a training image set according to an embodiment of the present invention;

FIG. 2 is a schematic flow chart of a training method of an image classification model according to an embodiment of the present invention;

FIG. 3 is a schematic diagram of a true probability distribution and a predicted probability distribution according to an embodiment of the present invention;

FIG. 4 is a flowchart of another training method of an image classification model according to an embodiment of the present invention;

fig. 5 is a schematic structural diagram of a training device for an image classification model according to an embodiment of the present invention;

fig. 6 is a schematic structural diagram of another training device for image classification model according to an embodiment of the present invention.

Detailed Description

The following description of the embodiments of the present invention will be made clearly and completely with reference to the accompanying drawings, in which it is apparent that the embodiments described are only some embodiments of the present invention, but not all embodiments. All other embodiments, which can be made by those skilled in the art based on the embodiments of the invention without making any inventive effort, are intended to be within the scope of the invention.

Before introducing the training method of the image classification model in the embodiment of the invention, the image classification model based on the CNN is briefly introduced by taking DenseNet (dense convolutional network) as an example, and referring to FIG. 1a, FIG. 1a is a schematic diagram of a network structure of DenseNet provided in the embodiment of the invention, after an image to be classified is input into DenseNET, the classification result of the image to be classified is obtained after the DenseNET processing. As shown, the DenseNet comprises a convolution layer, a pooling layer, a Dense Block (Dense Block) and a Softmax layer, wherein the convolution layer is used for extracting image features or compressing the image features extracted by the Dense Block; the pooling layer is used for converting a feature map obtained by extracting image features into a fixed size; each dense block can comprise 5 convolution layers for extracting image features of different dimensions of an input image, each layer of convolution layer takes the output of all convolution layers before the convolution layer in the same dense block as input, the problem of network gradient disappearance can be relieved through network connection, multiplexing of the image features in the dense block is enhanced, and the calculated amount is reduced; the Softmax layer is used to calculate the predictive probability of the input image for each class.

Referring next to fig. 1b, fig. 1b is a schematic diagram of a classification mechanism of a CNN-based image classification model according to an embodiment of the present invention, where the image classification model shown in fig. 1b is a DenseNET, and the DenseNET is a classification model for cats and dogs, that is, any image is input into the DenseNET, and there are two possible classification results to be output: cats or dogs. As shown in fig. 1b, the four images to be classified are respectively a first image, a second image, a third image and a fourth image, and after the four images are input into the DenseNET, classification results for the four images can be respectively obtained, and the classification results are as follows: cats, dogs, and cats.

The application scenario of the training method of the image classification model provided by the embodiment of the invention can be that in the field of medical image identification, when disease screening is carried out according to medical images (namely, target classification is diseased or not diseased), an unqualified image caused by reflection, shake, foreign matters and other reasons of a picture is trained, and an image classification model with low confidence is output as a disease screening result.

Next, a specific implementation manner of the training method of the image classification model provided by the embodiment of the present invention is described, referring to fig. 2, fig. 2 is a schematic flow chart of the training method of the image classification model provided by the embodiment of the present invention, and as shown in the figure, the method may include the following steps:

S201, acquiring a training image set.

Here, the training images in the training image set include related training images under different target categories and unrelated training images not belonging to the target categories, and the related training images carry category labels corresponding to the respective target categories. The target category is a category which can be classified by aiming at the input image through an image classification model obtained after the preset training. For example, before training, the DenseNET in FIG. 1b includes dogs and cats for the preset target class, the image classification model obtained after training is a classification model, that is, the input image is classified as a cat or a dog, for example, the first image in FIG. 1b is an image of a cat, and the classification result is a cat; the second image is an image of a dog, and the classification result is the dog; the third image is an image of an airplane, and the classification result is a dog; the fourth image is an image of a lion, and the classification result is a cat. For another example, if the preset target class includes a bicycle, an airplane and a flower, the trained image classification model is a three-classification model, similar to classifying any input image as one of a cat and a dog in fig. 1b, in this example, the input image may be classified as one of a bicycle, an airplane or a flower, and specific examples are not listed.

The target class comprises at least two kinds of related training images, the related training images are training images under the at least two kinds of target classes, and the unrelated training images are unrelated training images which do not belong to the at least two kinds of target classes. For example, if the target class includes two classes of cats and dogs, referring to fig. 1c, fig. 1c is a schematic diagram of a training image set provided in an embodiment of the present invention, as shown in fig. 1c, the training image set includes a relevant training image and an irrelevant training image, where the relevant training image may be a plurality of images including cats and a plurality of images including dogs as shown in the figure, and the irrelevant training image may be a plurality of images including airplanes but not including cats and dogs, an image including bicycles but not including cats and dogs, an image including lions but not including cats and dogs, an image including mice but not including cats and dogs, an image including keyboards but not including cats and dogs, and an image including houses but not including cats and dogs as shown in the figure.

Here, the relevant training images and the irrelevant training images in the training image set may be selected by user identification, or may be selected by other machine model identification. In order to facilitate the computer to distinguish the categories of the training images in the training image set and calculate the subsequent relevant training parameters, different category labels can be allocated to the training images in the image training set, and category labeling is carried out on the training images, so that the relevant training images carry category labels corresponding to respective target categories, and the irrelevant training images carry a uniform category label corresponding to the categories other than the target categories. For example, if the target class includes only cats and dogs, the relevant training image including cats in the image training set may be labeled with class labels 1, the relevant training image including dogs in the image training set may be labeled with class labels 2, and the irrelevant training image including dogs and cats in the image training set may be labeled with class labels 3.

S202, inputting the relevant training images into an image classification learning model to obtain the prediction classification probability of the relevant training images for each target class, and inputting the irrelevant training images into the image classification learning model to obtain the prediction classification probability of the irrelevant training images for each target class.

The image classification learning model may be an original model after initial model parameter initialization is trained, the model parameter initialization includes pre-configuration of a weight matrix in the image classification learning model, and the initialized image classification learning model has a certain image classification capability (usually has a low classification capability at this time), or may be an intermediate model after model parameter adjustment is performed on the image classification learning model for several times. The predictive classification probability herein may indicate the relevant training image or irrelevant training image being classified, which is classified as the confidence level of each of the target categories. For the DenseNet network shown in FIG. 1a, the Softmax layer can output the predicted classification probabilities for the respective target classes for the input images. For example, the target class only contains cats and dogs, after the relevant training image or the irrelevant training image is input into the image classification learning model, the probability of containing cats and the probability of containing dogs for the input relevant training image or the irrelevant training image are obtained, and the sum of the probabilities of the two is 1 for the same input image. If the predictive classification probability for cat for image a is 0.9, the predictive classification probability for dog is 0.1, the predictive classification probability for cat for image B is 0.6, the predictive classification probability for dog is 0.4, and although both would be classified as cats, the predictive classification probability for cat for image a is higher than image B, i.e., the confidence that image a is classified as cat is higher than image B.

S203, determining the probability distribution difference degree between the real probability distribution and the predicted probability distribution of the target category under the relevant training image according to the category labels corresponding to the target category of the relevant training image and the predicted classification probability of the relevant training image for each target category.

Probability distribution for expressing the probability law of the random variable value. Here, the target category may be used as a random variable, the value range of the random variable is a category label corresponding to each target category, the real probability distribution of the training image may be obtained according to the respective label category of the relevant training image, the probability corresponding to the label category of the relevant training image is 1, and the probability corresponding to the label category of other target categories is 0; the prediction probability distribution of the relevant training image can be obtained according to the prediction classification probabilities for the target classes obtained in the step S202. Referring to fig. 3, fig. 3 is a schematic diagram of a true probability distribution and a predicted probability distribution provided in an embodiment of the present invention, assuming that in two categories of cats and dogs, the category label of the cat is 0, the category label of the dog is 1, the category labels corresponding to other categories are 2, and a relevant training image including the cat is obtained, the predicted classification probability for the cat is 0.8, and the predicted classification probability for the dog is 0.2, the true probability distribution and the predicted probability distribution are shown in fig. 3. It is easy to understand that the closer the predicted probability distribution of the related training image obtained by the image classification learning model is to the real probability distribution of the related training image, the better the classification ability of the image classification learning model is, so that the probability distribution difference degree between the real probability distribution and the predicted probability distribution of the target class under the related training image can be used to reversely guide the adjustment of model parameters in the image classification learning model so as to optimize the classification ability of the image classification learning model.

Wherein, the target category is between the true probability distribution and the prediction probability distribution under the related training imageThe probability distribution difference of (2) can be expressed in various forms, such as Euclidean distance, manhattan distance, chebyshev distance, minkowski distance, mahalanobis distance, cosine angle, cross entropy, relative entropy, etc., and the corresponding Euclidean distance isThe corresponding cross entropy is h= - (0×log0.8+1×log0.2) ≡0.70, and the specific form of the probability distribution difference degree is not limited here.

S204, determining the prediction classification uncertainty of the related training image according to the prediction classification probability of the related training image for each target class, and determining the prediction classification uncertainty of the unrelated training image according to the prediction classification probability of the unrelated training image for each target class.

Here, the prediction classification probability of each relevant training image or irrelevant training image for each target classification characterizes the confidence level of each classification in the target classification, and then the classification confidence level of the relevant training image can be characterized according to the prediction classification uncertainty of the relevant training image determined by the prediction classification probability of the relevant training image, and the classification confidence level of the irrelevant training image can be characterized according to the prediction classification uncertainty of the irrelevant training image determined by the prediction classification probability of the irrelevant training image. The image classification learning model with high generalization capability has higher classification confidence degree on the related training images and lower classification confidence degree on the unrelated training images, so that the prediction classification uncertainty of the related training images and the prediction classification uncertainty of the unrelated training images can be used for reversely guiding the adjustment of model parameters in the image classification learning model so as to optimize the generalization capability of the image classification learning model.

In one implementation, the prediction classification uncertainty of the related training image may be represented by a classification information entropy of the related training image, and the prediction classification uncertainty of the unrelated training image may be represented by a classification information entropy of the unrelated training image.

S205, adjusting current model parameters of the image classification learning model according to the probability distribution difference degree, the prediction classification uncertainty of the related training images and the prediction classification uncertainty of the unrelated training images.

Specifically, a classification loss function corresponding to the current model parameters of the image classification learning model can be constructed according to the probability distribution difference degree, the prediction classification uncertainty of the related training images and the prediction classification uncertainty of the unrelated training images, for example, L is used _c Represents the degree of difference of probability distribution, L _r Representing predictive classification uncertainty, L, for a relevant training image _ur Representing the predictive classification uncertainty of the unrelated training image, the classification loss function may be constructed as: l=l _c +L _r -L _ur Prediction classification uncertainty L of related training images _r Prediction classification uncertainty L for irrelevant training images _ur As a regularization term for constraining confidence of the prediction probability of the image classification learning model in the classification loss function; and then, performing bias guide on the classification loss function, determining a loss gradient corresponding to the current model parameter of the image classification learning model, and further adjusting the current model parameter of the image classification learning model according to the loss gradient corresponding to the current model parameter of the image classification learning model and a preset parameter learning rate, so that the probability distribution difference degree determined by the adjusted image classification learning model, the prediction classification uncertainty of the determined related training image and the prediction classification uncertainty of the determined unrelated training image meet preset adjustment result conditions.

In an alternative manner, the preset adjustment result conditions may be preset for the probability distribution difference degree, the prediction classification uncertainty of the related training image, and the prediction classification uncertainty of the unrelated training image, respectively. For example, the above-described adjustment result conditions may include: the probability distribution difference degree redetermined by the adjusted image classification model is within a first ratio threshold range compared with the probability distribution difference degree reduction rate obtained before adjustment; the uncertainty of the prediction classification of the related training image is determined again, and the reduction rate of the uncertainty of the prediction classification of the related training image obtained before adjustment is within a second ratio threshold value range; the predicted classification uncertainty of the re-determined irrelevant training images is within a third scale threshold compared with the predicted classification uncertainty of the irrelevant training images obtained before adjustment.

In another alternative, the preset adjustment result condition may be one that is preset uniformly by aiming at the probability distribution difference degree, the prediction classification uncertainty of the related training image, and the prediction classification uncertainty of the unrelated training image. For example, the classification loss function may be constructed according to the probability distribution difference degree, the prediction classification uncertainty of the related training image, and the prediction classification uncertainty of the unrelated training image, and the adjustment result condition may be a constraint condition for the classification loss function: and comparing the value reduction rate of the classification loss function corresponding to the image classification learning model obtained before adjustment with the value reduction rate of the classification loss function corresponding to the image classification learning model obtained before adjustment within a preset loss reduction proportion threshold value range.

In the training process of the actual image classification learning model, the training method of the image classification model provided by the embodiment of the invention can be iteratively executed in a computer until the probability distribution difference degree of the related training image of the image classification learning model is smaller than a preset difference degree threshold value, the prediction classification uncertainty of the related training image is smaller than a preset first uncertainty, and the iteration is stopped under the condition that the prediction classification uncertainty of the unrelated training image is larger than a preset second uncertainty, or until the value of the classification loss function is smaller than a preset classification loss threshold value.

In the neural network model, the normalization processing of the data can limit the numerical value within a certain range, solve the problem of numerical value overflow caused by overlarge or overlarge numerical value in the calculation process of a computer, ensure that the training of the neural network model can be completed by setting a uniform parameter learning rate in the process of training the neural network model by a gradient descent method, and reduce the calculated data quantity.

Therefore, optionally, after step S204, the training method for an image classification model provided in this embodiment may further include the following steps: and respectively carrying out normalization processing on the prediction classification uncertainty of the related training image and the prediction classification uncertainty of the irrelevant training image to obtain the normalization uncertainty of the related training image and the normalization uncertainty of the irrelevant training image. In step S205, the current model parameters of the image classification learning model may be specifically adjusted according to the probability distribution difference, the normalization uncertainty of the related training image, and the normalization uncertainty of the unrelated training image.

Further optionally, the specific way of normalizing the prediction classification uncertainty of the related training image and the prediction classification uncertainty of the unrelated training image may be: determining the maximum classification uncertainty of the target class under the condition of uniform probability distribution; the ratio of the predicted classification uncertainty to the maximum classification uncertainty of the relevant training image is determined as the normalized uncertainty of the relevant training image, and the ratio of the predicted classification uncertainty to the maximum classification uncertainty of the irrelevant training image is determined as the normalized uncertainty of the irrelevant training object.

Referring to fig. 4, fig. 4 is a flowchart of another training method for an image classification model according to an embodiment of the present invention, as shown in the drawing, the method may include the following steps:

s401, acquiring a training image set.

The training image set comprises related training images under different target categories and irrelevant training images which do not belong to the target categories, and the related training images carry category labels corresponding to the respective target categories.

S402, inputting the relevant training images into an image classification learning model to obtain the prediction classification probability of the relevant training images for each target class, and inputting the irrelevant training images into the image classification learning model to obtain the prediction classification probability of the irrelevant training images for each target class.

The specific implementation manner of step S401 and step S402 may refer to the specific implementation manner of step S201 and step S202 in the corresponding embodiment of fig. 2, and will not be described herein again.

S403, determining the classification cross entropy of the relevant training image according to the class label corresponding to the target class of the relevant training image and the prediction classification probability of the relevant training image for each target class.

In an alternative embodiment, the cross-class entropy L of the associated training image may be determined according to the following formula _ce-ur ：

In another alternative embodiment, the classification cross entropy L of the associated training image may be determined according to the following formula _ce-ur ：

The meaning of each parameter in the formula (2) is consistent with that in the formula (1), the total classification cross entropy of the related training image is determined in the formula (1), the average classification cross entropy of the related training image is determined in the formula (2), and the two can represent the difference degree between the true probability distribution and the prediction probability distribution of the target category under the related training image, which is not limited herein.

S404, determining the classification information entropy of the related training image according to the prediction classification probability of the related training image for each target category, and determining the classification information entropy of the unrelated training image according to the prediction classification probability of the unrelated training image for each target category.

In particular, the classification information entropy L of the relevant training image can be determined by the following formula _e-r ：

Wherein i is the index of the related training image, N is the number of the related training images, i and N are positive integers, i is not more than N, j is the index of the target category, K is the category number of the target category, j and K are positive integers, j is not more than K, and p _ij For a related training diagramThe prediction classification probability of the image i for the target class j;

the classification information entropy L of the irrelevant training images can be determined by the following formula _e-ur ：

S405, determining the maximum classification information entropy of the target class under the condition that the probabilities are uniformly distributed.

Here, according to the principle of maximum entropy, without more information about random variables, it is assumed that random variables are uniformly distributed, i.e., probabilities of classifying them into various target categories are equal for one training image, and the information entropy of probability distribution is the maximum at this time, which is the maximum classification information entropy. Specifically, the maximum classification information entropy H is determined by the following formula _max-entropy ：

Wherein K is the number of categories of the target category.

S406, determining the ratio of the classification information entropy of the related training image to the maximum classification information entropy as the normalized classification information entropy of the related training image, and determining the ratio of the classification information entropy of the unrelated training image to the maximum classification information entropy as the normalized classification information entropy of the unrelated training image.

Specifically, normalized classification information entropy L of the related training image is obtained according to the formula (3) and the formula (5) _N-e-r The method comprises the following steps:

wherein the meaning of each parameter in the formula (6) is consistent with that in the formula (3). Normalized classification information entropy L of irrelevant training images obtained according to formula (4) and formula (5) _N-e-ur The method comprises the following steps:

wherein the meaning of each parameter in the formula (7) is identical to that in the formula (4).

S407, adjusting current model parameters of the image classification learning model according to the classification cross entropy, the normalized classification information entropy of the related training image and the normalized classification information entropy of the unrelated training image.

Here, a classification loss function is constructed according to the classification cross entropy, the normalized classification information entropy of the related training image and the normalized classification information entropy of the unrelated training image, and then the current model parameters of the image classification learning model are adjusted according to the error direction propagation algorithm.

Specifically, the sorting loss function may be l=l _ce-ur +L _N-e-r -L _N-e-ur May also be L=L _ce-ur +αL _N-e-r -βL _N-e-ur Wherein alpha is the weight corresponding to the normalized classification information entropy of the preset related training image, and beta is the weight corresponding to the normalized classification information entropy of the unrelated training image. And parameters such as class labels, prediction classification probability and the like corresponding to each relevant training image and irrelevant training image are substituted into the classification loss function to obtain a loss function form which takes model parameters such as weight W, offset b and the like in the image classification learning model as independent variables, the loss function can be used for deviant the weight W, and the weight W in the current model parameters is determined ₀ Is further calculated by the formula W _t ＝W ₀ -axdL/dW, determining weights W in current model parameters ₀ Is to update the target W _t Wherein a is a preset parameter learning rate. Similarly, the offset b is calculated and led toEquation b _t ＝b ₀ -a x dL/db determining the offset b in the current model parameters ₀ Update target b of (2) _t 。

In the training process of the actual image classification learning model, the training method of the image classification model provided by the embodiment of the invention can be iteratively executed in a computer until the classification cross entropy of the related training image of the image classification learning model is smaller than a preset cross entropy threshold, the normalized classification information entropy of the related training image is smaller than a preset first information entropy threshold, and the normalized classification information entropy of the unrelated training image is larger than a preset second information entropy threshold, or until the value of the classification loss function of the image classification learning model is smaller than a preset classification loss threshold.

In the embodiment of the invention, a training image set comprising a relevant training image and an irrelevant training image is obtained, the relevant training image is input into an image classification learning model to obtain the prediction classification probability of the relevant training image for each target class, the irrelevant training image is input into an image classification learning model to obtain the prediction classification probability of the irrelevant training image for each target class, then the classification cross entropy of the relevant training image is determined according to the class label corresponding to the target class of the relevant training image and the prediction classification probability of the relevant training image for each target respectively, the classification information entropy of the relevant training image is determined according to the prediction classification probability of the relevant training image for each target class, the classification information entropy of the irrelevant training image is determined according to the prediction classification probability of the irrelevant training image for each target class, then the normalization of the maximum information entropy is carried out on the classification cross entropy of the relevant training image and the classification information entropy of the irrelevant training image, and the normalization of the relevant training image, and the current model parameters of the image classification learning model are adjusted according to the normalization of the classification information entropy of the relevant training image. In the training process of the image classification learning model, the classification precision of the image classification learning model is guided and optimized through the classification cross entropy of the related training image, and the confidence level of the prediction probability of the optimized image classification learning model is guided and optimized through the normalized classification information entropy of the related training image and the normalized classification information entropy of the unrelated training image, so that the generalization capability of the image classification model obtained through training is improved.

Referring to fig. 5, fig. 5 is a schematic structural diagram of a training device for an image classification model according to an embodiment of the present invention, as shown in the fig. 5, the training device 50 for an image classification model may at least include an image set acquisition module 501, a confidence determining module 502, a difference determining module 503, an uncertainty determining module 504, and a parameter adjusting module 505, where:

an image set obtaining module 501, configured to obtain a training image set, where the training image set includes related training images under different target categories, and unrelated training images that do not belong to the target categories, and the related training images carry category labels corresponding to respective target categories;

the confidence determining module 502 is configured to input the relevant training image into an image classification learning model to obtain a prediction classification probability of the relevant training image for each target class, and input the irrelevant training image into the image classification learning model to obtain a prediction classification probability of the irrelevant training image for each target class;

a difference determining module 503, configured to determine a probability distribution difference between a true probability distribution and a predicted probability distribution of the target category under the relevant training image according to a category label corresponding to the target category of the relevant training image and the predicted classification probability of the relevant training image for each target category;

An uncertainty determination module 504, configured to determine a prediction classification uncertainty of the relevant training image according to a prediction classification probability of the relevant training image for each of the target categories, and determine a prediction classification uncertainty of the irrelevant training image according to a prediction classification probability of the irrelevant training image for each of the target categories;

and the parameter adjusting module 505 is configured to adjust current model parameters of the image classification learning model according to the probability distribution difference degree, the prediction classification uncertainty of the related training image, and the prediction classification uncertainty of the unrelated training image, so that the probability distribution difference degree determined by the adjusted image classification learning model, the determined prediction classification uncertainty of the related training image, and the determined prediction classification uncertainty of the unrelated training image satisfy preset adjustment result conditions.

Optionally, the difference determining module 503 is specifically configured to determine a classification cross entropy of the relevant training image according to a class label corresponding to a target class of the relevant training image and a prediction classification probability of the relevant training image for each target class, and determine the classification cross entropy of the relevant training image as the probability distribution difference.

Optionally, the uncertainty determination module 504 is specifically configured to: determining the classification information entropy of the related training image according to the prediction classification probability of the related training image for each target class, and determining the classification information entropy of the related training image as the prediction classification uncertainty of the related training image;

Optionally, the apparatus further includes a normalization module 506, configured to normalize the prediction classification uncertainty of the related training image and the prediction classification uncertainty of the unrelated training image, to obtain a normalized uncertainty of the related training image and a normalized uncertainty of the unrelated training image;

the parameter adjustment module 505 is specifically configured to adjust a current model parameter of the image classification learning model according to the probability distribution difference, the normalized uncertainty of the related training image, and the normalized uncertainty of the unrelated training image.

Optionally, the normalization module 506 includes a maximum uncertainty determination unit 5061 and an uncertainty normalization unit 5062:

the maximum uncertainty determining unit 5061 is configured to determine a maximum classification uncertainty of the target class in a case where probabilities are uniformly distributed;

the uncertainty normalization unit 5062 is configured to determine a ratio of the predicted classification uncertainty of the related training image to the maximum classification uncertainty as a normalized uncertainty of the related training image, and determine a ratio of the predicted classification uncertainty of the unrelated training image to the maximum classification uncertainty as a normalized uncertainty of the unrelated training object.

Optionally, the parameter adjustment module 505 includes a loss function construction unit 5051, a gradient determination unit 5052, and a parameter adjustment unit 5053:

the loss function construction unit 5051 is configured to construct a classification loss function corresponding to a current model parameter of the image classification learning model according to the probability distribution difference degree, the prediction classification uncertainty of the related training image, and the prediction classification uncertainty of the unrelated training image;

the gradient determining unit 5052 is configured to bias the classification loss function, and determine a loss gradient corresponding to a current model parameter of the image classification learning model;

The parameter adjusting unit 5053 is configured to adjust the current model parameter of the image classification learning model according to a loss gradient corresponding to the current model parameter of the image classification learning model and a preset parameter learning rate.

Optionally, the loss function construction unit 5051 is specifically configured to construct a classification loss function L corresponding to a current model parameter of the image classification learning model, where the classification loss function L is:

L＝L _c +αL _r -βL _ur ，

wherein L is _c For the probability distribution difference degree, alpha is presetWeight, L, corresponding to the prediction classification uncertainty of the related training image _r For the uncertainty of the prediction classification of the related training image, beta is the preset weight corresponding to the uncertainty of the prediction classification of the unrelated training image, L _ur And classifying uncertainty for prediction of the unrelated training image.

Optionally, the difference determining module 503 is specifically configured to determine a classification cross entropy L of the related training image _ce-ur The method comprises the following steps:

Optionally, the uncertainty determination module 504 is specifically configured to determine the classification information entropy L of the relevant training image _e-r The method comprises the following steps:

In a specific implementation, the training device of the image classification model may execute each step in the training method of the image classification model as shown in fig. 2 or fig. 4 through each built-in functional module, and specific implementation details may refer to implementation details of each step in the embodiment corresponding to fig. 2 and fig. 4, which are not described herein again.

In the embodiment of the invention, an image set acquisition module acquires a training image set comprising a relevant training image and an irrelevant training image, a confidence level determination module inputs the relevant training image into an image classification learning model to obtain the prediction classification probability of the relevant training image for each target class, inputs the irrelevant training image into the image classification learning model to obtain the prediction classification probability of the irrelevant training image for each target class, then a difference level determination module determines the probability distribution difference level between the true probability distribution and the prediction probability distribution of the target class under the relevant training image according to the class label corresponding to the target class of the relevant training image and the prediction classification probability of the relevant training image for each target class, and an uncertainty level determination module determines the prediction classification uncertainty of the relevant training image according to the prediction classification probability of the relevant training image for each target class, and further a parameter adjustment module adjusts the current model parameters of the image classification learning model according to the probability distribution difference level, the prediction classification uncertainty of the relevant training image and the prediction classification uncertainty of the irrelevant training image. In the training process of the image classification learning model, the parameter adjusting module guides and optimizes the classification precision of the image classification learning model through the probability distribution difference degree, and guides and optimizes the confidence level of the prediction probability of the image classification learning model through the prediction classification uncertainty of the related training image and the prediction classification uncertainty of the unrelated training image, so that the generalization capability of the image classification model obtained through training is improved.

Referring to fig. 6, fig. 6 is a schematic structural diagram of another training apparatus for an image classification model according to an embodiment of the present invention, as shown in the drawing, the training apparatus 60 for an image classification model includes: at least one processor 601, such as a CPU, at least one network interface 604, a user interface 603, a memory 605, at least one communication bus 602. Wherein the communication bus 602 is used to enable connected communications between these components. The user interface 603 may include a Display screen (Display), a Camera (Camera), and the optional user interface 603 may further include a standard wired interface, a wireless interface. The network interface 604 may optionally include a standard wired interface, a wireless interface (e.g., WI-FI interface). The memory 605 may be a high-speed RAM memory or a non-volatile memory (non-volatile memory), such as at least one disk memory. The memory 605 may also optionally be at least one storage device located remotely from the processor 601. As shown in FIG. 6, memory 605, which is a computer storage medium, may include an operating system, a network communication module, a user interface module, and a training application for the image classification model.

In the terminal 6 shown in fig. 6, the user interface 603 is mainly an interface for receiving category labels of training images of a training image set for a user; and processor 601 may be used to invoke a training application of the image classification model stored in memory 605 and specifically:

It should be understood that the training device 60 for an image classification model according to the embodiment of the present invention may perform the description of the training method for an image classification model according to the embodiment of fig. 2 or fig. 4, and may also perform the description of the training device 50 for an image classification model according to the embodiment of fig. 5, which is not repeated herein. In addition, the description of the beneficial effects of the same method is omitted.

Embodiments of the present invention also provide a computer storage medium storing a computer program comprising program instructions which, when executed by a computer, cause the computer to perform a method as described in the previous embodiments, which may be part of a training apparatus for an image classification model as mentioned above.

Those skilled in the art will appreciate that implementing all or part of the above-described methods in accordance with the embodiments may be accomplished by way of a computer program stored on a computer readable storage medium, which when executed may comprise the steps of the embodiments of the methods described above. The storage medium may be a magnetic disk, an optical disk, a Read-Only Memory (ROM), a random access Memory (Random Access Memory, RAM), or the like.

The foregoing disclosure is illustrative of the present invention and is not to be construed as limiting the scope of the invention, which is defined by the appended claims.

Claims

1. A method for training an image classification model, comprising:

Determining the classification information entropy of the irrelevant training images according to the prediction classification probability of the irrelevant training images for each target category, and determining the classification information entropy of the irrelevant training images as the prediction classification uncertainty of the irrelevant training images;

2. The method according to claim 1, wherein determining the probability distribution difference between the true probability distribution and the predicted probability distribution of the target class under the relevant training image according to the class label corresponding to the target class of the relevant training image and the predicted classification probability of the relevant training image for each target class comprises:

3. The method according to claim 1, wherein the method further comprises:

4. A method according to claim 3, wherein normalizing the predicted classification uncertainty of the associated training image and the predicted classification uncertainty of the unrelated training image, respectively, to obtain a normalized uncertainty of the associated training image and a normalized uncertainty of the unrelated training image comprises:

and determining the ratio of the predicted classification uncertainty of the related training image to the maximum classification uncertainty as the normalized uncertainty of the related training image, and determining the ratio of the predicted classification uncertainty of the unrelated training image to the maximum classification uncertainty as the normalized uncertainty of the unrelated training image.

5. The method according to claim 1 or 2, wherein said adjusting current model parameters of the image classification learning model according to the probability distribution variability, the prediction classification uncertainty of the related training image, and the prediction classification uncertainty of the unrelated training image comprises:

6. The method of claim 5, wherein constructing the classification loss function corresponding to the current model parameters of the image classification learning model based on the probability distribution variability, the predictive classification uncertainty of the associated training image, and the predictive classification uncertainty of the unrelated training image comprises:

，

wherein L is _c For the degree of difference of the probability distribution,for the preset weight corresponding to the uncertainty of the prediction classification of the related training image, L _r Uncertainty of prediction classification for said associated training image,/for said associated training image>For the preset weight corresponding to the uncertainty of the prediction classification of the irrelevant training images, L _ur And classifying uncertainty for prediction of the unrelated training image.

7. The method of claim 2, wherein determining the cross-class entropy of each of the associated training images based on the class labels corresponding to the target classes of the associated training images and the predicted classification probabilities of the associated training images for each of the target classes comprises:

，

wherein i is the index of the related training images, N is the number of the related training images, i and N are positive integers, andj is the index of the target category, K is the number of categories of the target category, j and K are positive integers, and +.>，y _j For the class label corresponding to the target class j, p _ij The probability of the predicted classification for the target class j for the relevant training image i.

8. The method of claim 1, wherein determining the class information entropy of the associated training image based on the predicted class probabilities of the associated training image for each of the target classes comprises:

，

wherein i is the index of the related training images, N is the number of the related training images, i and N are positive integers, andj is the index of the target category, K is the number of categories of the target category, j and K are positive integers, and +.>，p _ij The prediction classification probability of the related training image i for the target class j is calculated;

，

wherein l is the index of the irrelevant training images, M is the number of the irrelevant training images, l and M are positive integers, andj is the index of the target category, K is the number of categories of the target category, j and K are positive integers, and +.>，g _lj The probability of predictive classification for the target class j for the unrelated training image l.

9. A training device for an image classification model, comprising:

The uncertainty determining module is used for determining the classification information entropy of the related training image according to the prediction classification probability of the related training image for each target class, and determining the classification information entropy of the related training image as the prediction classification uncertainty of the related training image; determining the classification information entropy of the irrelevant training images according to the prediction classification probability of the irrelevant training images for each target class, and determining the classification information entropy of the irrelevant training images as the prediction classification uncertainty of the irrelevant training images;

10. A training device for an image classification model, comprising: a processor and a memory;

The processor being connected to a memory, wherein the memory is adapted to store program code, the processor being adapted to invoke the program code to perform the method of any of claims 1 to 8.

11. A computer readable storage medium, characterized in that the computer readable storage medium stores a computer program comprising program instructions which, when executed by a processor, perform the method of any of claims 1 to 8.