CN115631381A - Classification model training method, image classification device and electronic equipment - Google Patents
Classification model training method, image classification device and electronic equipment Download PDFInfo
- Publication number
- CN115631381A CN115631381A CN202211350323.4A CN202211350323A CN115631381A CN 115631381 A CN115631381 A CN 115631381A CN 202211350323 A CN202211350323 A CN 202211350323A CN 115631381 A CN115631381 A CN 115631381A
- Authority
- CN
- China
- Prior art keywords
- image
- network
- feature extraction
- parameter
- loss value
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
- G06V10/764—Arrangements for image or video recognition or understanding using pattern recognition or machine learning using classification, e.g. of video objects
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/08—Learning methods
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
- G06V10/77—Processing image or video features in feature spaces; using data integration or data reduction, e.g. principal component analysis [PCA] or independent component analysis [ICA] or self-organising maps [SOM]; Blind source separation
- G06V10/7715—Feature extraction, e.g. by transforming the feature space, e.g. multi-dimensional scaling [MDS]; Mappings, e.g. subspace methods
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
- G06V10/82—Arrangements for image or video recognition or understanding using pattern recognition or machine learning using neural networks
Abstract
The invention provides a classification model training method, an image classification device and electronic equipment, relates to the technical field of artificial intelligence, particularly relates to the technical field of deep learning, image processing and computer vision, and can be applied to scenes such as face recognition. The specific implementation scheme is as follows: acquiring a first image characteristic of a sample image through a first characteristic extraction network in a target neural network model; predicting first classification information of the sample image by using the first image characteristics, and calculating a first loss value corresponding to the first classification information and a preset label of the sample image; acquiring a second image characteristic of the sample image through a second characteristic extraction network; predicting second classification information of the sample image by using the second image characteristic, and calculating a second loss value corresponding to the first classification information and the second classification information; parameters in the first feature extraction network are updated based on the set of loss values. The present disclosure may improve the robustness of a target neural network model.
Description
Technical Field
The present disclosure relates to the field of artificial intelligence technologies, and in particular, to the field of deep learning, image processing, and computer vision technologies, which can be applied to scenes such as face recognition, and in particular, to a classification model training method, an image classification device, and an electronic apparatus.
Background
The image classification is a very common application scene of electronic equipment at present, in addition, a neural network model is often adopted for image classification at present, and the neural network model for image classification at present is mainly used for parameter updating based on a loss value calculated by a loss function.
Disclosure of Invention
The disclosure provides a classification model training method, an image classification device and electronic equipment.
According to an aspect of the present disclosure, there is provided a classification model training method, including:
acquiring first image characteristics of a sample image through a first characteristic extraction network in a target neural network model, wherein the target neural network model is used for classifying the image;
predicting first classification information of the sample image by using the first image features through a first prediction network of the target neural network model, and calculating a first loss value corresponding to the first classification information and a preset label of the sample image through a first loss function;
acquiring a second image characteristic of the sample image through a second characteristic extraction network;
predicting second classification information of the sample image by using the second image feature through a second prediction network, and calculating a second loss value corresponding to the first classification information and the second classification information through a second loss function;
updating a parameter in the first feature extraction network based on a set of penalty values, the set of penalty values including the first penalty value and the second penalty value.
According to an aspect of the present disclosure, there is provided an image classification method including:
acquiring a first image characteristic of an image to be classified through a first characteristic extraction network in a target neural network model;
predicting classification information of the image to be classified by using the first image features through a first prediction network of the target neural network model;
the parameter of the first feature extraction network is a parameter updated based on a loss value set, where the loss value set includes a first loss value and a second loss value, the first loss value is a loss value corresponding to a preset label of a sample image and first classification information of the sample image predicted by the first prediction network, the second loss value is a loss value corresponding to the first classification information and second classification information, and the second classification information is classification information of the sample image predicted by the second prediction network.
According to another aspect of the present disclosure, there is provided a classification model training apparatus including:
the first acquisition module is used for acquiring first image characteristics of the sample image through a first characteristic extraction network in a target neural network model, and the target neural network model is used for classifying the image;
the first prediction module is used for predicting first classification information of the sample image by using the first image characteristics through a first prediction network of the target neural network model, and calculating a first loss value corresponding to a preset label of the sample image through a first loss function;
the second acquisition module is used for acquiring a second image characteristic of the sample image through a second characteristic extraction network;
a second prediction module, configured to predict, through a second prediction network, second classification information of the sample image by using the second image feature, and calculate, through a second loss function, a second loss value corresponding to the first classification information and the second classification information;
a first update module to update a parameter in the first feature extraction network based on a set of penalty values, the set of penalty values including the first penalty value and the second penalty value.
According to another aspect of the present disclosure, there is provided an image classification apparatus including:
the extraction module is used for acquiring first image characteristics of the image to be classified through a first characteristic extraction network in the target neural network model;
the classification module is used for predicting classification information of the image to be classified by utilizing the first image characteristics through a first prediction network of the target neural network model;
the parameter of the first feature extraction network is a parameter updated based on a loss value set, the loss value set includes a first loss value and a second loss value, the first loss value is a loss value corresponding to a preset label of a sample image predicted by the first prediction network, and the second loss value is a loss value corresponding to the first classification information and the second classification information, and the second classification information is classification information of the sample image predicted by the second prediction network.
According to another aspect of the present disclosure, there is provided an electronic device including:
at least one processor; and
a memory communicatively coupled to the at least one processor; wherein the content of the first and second substances,
the memory stores instructions executable by the at least one processor to enable the at least one processor to perform a classification model training method or an image classification method provided by the present disclosure.
According to another aspect of the present disclosure, there is provided a non-transitory computer-readable storage medium storing computer instructions for causing a computer to perform a classification model training method or an image classification method provided by the present disclosure.
According to another aspect of the present disclosure, a computer program product is provided, comprising a computer program which, when executed by a processor, implements the classification model training method or the image classification method provided by the present disclosure.
It should be understood that the statements in this section do not necessarily identify key or critical features of the embodiments of the present disclosure, nor do they limit the scope of the present disclosure. Other features of the present disclosure will become apparent from the following description.
Drawings
The drawings are included to provide a better understanding of the present solution and are not to be construed as limiting the present disclosure. Wherein:
FIG. 1 is a flow chart of a classification model training method provided by the present disclosure;
FIG. 2 is a schematic diagram of a model training provided by the present disclosure;
FIG. 3 is a flow chart of an image classification method provided by the present disclosure;
FIGS. 4a to 4d are structural diagrams of a classification model training apparatus provided by the present disclosure;
fig. 5 is a structural diagram of an image classification apparatus provided by the present disclosure;
FIG. 6 is a block diagram of an electronic device used to implement embodiments of the present disclosure.
Detailed Description
Exemplary embodiments of the present disclosure are described below with reference to the accompanying drawings, in which various details of embodiments of the present disclosure are included to assist understanding, and which are to be considered as merely exemplary. Accordingly, those of ordinary skill in the art will recognize that various changes and modifications of the embodiments described herein can be made without departing from the scope and spirit of the present disclosure. Also, descriptions of well-known functions and constructions are omitted in the following description for clarity and conciseness.
Referring to fig. 1, fig. 1 is a flowchart of a classification model training method provided by the present disclosure, as shown in fig. 1, including the following steps:
s101, obtaining first image characteristics of a sample image through a first characteristic extraction network in a target neural network model, wherein the target neural network model is used for classifying the image.
The target neural network model is a neural network model including a feature extraction network and a prediction network, for example: convolutional Neural Networks (CNNs), artificial Neural Networks (ANNs), and the like, the type of target Neural Network model is not limited by this disclosure.
The first feature extraction network may be a network for performing image feature extraction, for example: deep convolutional networks, such as Residual networks (ResNet), mobile networks (MobileNet).
In the present disclosure, the sample image set includes a positive sample image and an attack sample image, and the proportion of the positive sample image and the attack sample image may be one-to-one or approximately one-to-one. The sample image in step S101 is any sample image in the sample image set, that is, a corresponding operation is performed for each sample image.
The image classification may be to perform a living body detection on the image, and in some embodiments, may also be to perform a vehicle information classification, a road information classification, or the like on the image.
Step S102, predicting first classification information of the sample image by using the first image feature through a first prediction network of the target neural network model, and calculating a first loss value of the first classification information corresponding to a preset label of the sample image through a first loss function.
The first prediction network may be a network for classifying the sample image, and the network may include at least one of a pooling layer, a fully-connected layer, and a normalized exponential function (Softmax) layer, etc.
The first loss function may be a loss function for calculating difference information between the first classification information and the preset label. In some embodiments, the first Loss function may be a cross entropy Loss function (CE Loss).
The first classification information may be a classification probability or an explicit classification.
The preset label of the sample image may be label information preset for the sample image.
And step S103, acquiring a second image characteristic of the sample image through a second characteristic extraction network.
The second feature extraction network may be a network for performing image feature extraction, for example: deep convolutional networks such as ResNet, mobileNet.
The first image feature and the second image feature are different image features extracted for the same sample image.
Step S104, predicting second classification information of the sample image by using the second image feature through a second prediction network, and calculating a second loss value corresponding to the first classification information and the second classification information through a second loss function.
The second prediction network may be a network for classifying the sample image, and the network may include at least one of a pooling layer, a full connection layer, a Softmax function layer, and the like.
The second loss function may be a loss function that calculates difference information between the first classification information and the second classification information. In some embodiments, the second loss function may be a cross entropy loss function (Distill CE loss) of soft tags.
It should be noted that the execution sequence between steps S101 and S102 and S103 and S104 may not be limited, for example: s101 and S103 are executed simultaneously or sequentially.
Step S105, updating the parameter in the first feature extraction network based on a loss value set, where the loss value set includes the first loss value and the second loss value.
In step S105, parameters in the first feature extraction network may be updated uniformly based on the first loss value and the second loss value, that is, parameters in the first feature extraction network may be updated based on the first loss value, and parameters in the first feature extraction network may be updated based on the second loss value. That is, the execution sequence of the steps is not limited in the present disclosure, for example: it may be executed sequentially in the order shown in fig. 1, or may be executed at step S105 simultaneously with steps S103 and S104.
In addition, each update in step S105 may update the parameters in the first feature extraction network based on the loss values of part or all of the sample images.
In the disclosure, since the first loss value is a loss value corresponding to the first classification information and the preset label of the sample image, and the second loss value is a loss value corresponding to the first classification information and the second classification information, parameters in the first feature extraction network are updated based on the loss value set, so that the first feature extraction network can be supervised through the preset label of the sample image and the second classification information, and thus the robustness of the first feature extraction network can be improved, and further the robustness of the target neural network model can be improved.
In the present disclosure, the method may be applied to an electronic device, that is, the method includes all steps performed by the electronic device, and the electronic device may be a server, a computer, a mobile phone, a tablet computer, or other electronic devices.
In one embodiment, the method further comprises:
carrying out N times of image augmentation operations on the sample image to obtain N augmented images of the sample image, wherein N is an integer greater than 1;
the first image feature is an image feature obtained by feature extraction of a first augmented image in the N augmented images by the first feature extraction network;
the second image feature is an image feature obtained by performing feature extraction on the first augmented image by the second feature extraction network.
The N image augmentation operations performed on the sample image may be performed outside or inside the target neural network.
The N augmented images may be N different homologous images, that is, each augmented image performs different image augmentation operations, such as performing augmentation operations through different data. In some embodiments, the performing N image augmentation operations on the sample image may be performing N random perturbations on the sample image.
The first feature extraction network and the second feature extraction network respectively extract features of the first augmented image to obtain the first image feature and the second image feature.
Therefore, the first image characteristic and the second image characteristic are image characteristics extracted from the same augmented image, so that the second classification information can better supervise the first characteristic extraction network, and the accuracy of the first characteristic extraction network is further improved.
In one embodiment, the method further comprises:
performing feature extraction on a second augmented image in the N augmented images through the first feature extraction network to obtain a third image feature;
calculating a third loss value corresponding to the first image feature and the third image feature through a contrast loss function;
the set of penalty values further includes the third penalty value.
The third loss function may be a loss function for calculating difference information between the first image feature and the third image feature.
In some embodiments, the third Loss function is a contrast Loss function (contrast Loss).
In some embodiments, the gradient update may be performed on the parameter in the first feature extraction network based on the first loss value, the second loss value, and the third loss value.
In this embodiment, since the loss value further includes a third loss value, it can be realized by the third loss value that the parameter of the first feature extraction network is updated in an unsupervised manner, so that the first feature extraction network extracts image features more accurately, for example, the first feature extraction network extracts consistent image features from different homologous images.
It should be noted that, in some embodiments, the first feature extraction network and the second feature extraction network may also directly perform feature extraction on the sample image.
In one embodiment, the method further comprises:
updating the parameters in the second feature extraction network based on the parameters in the first feature extraction network.
In the above, updating the parameter in the second feature extraction network based on the parameter in the first feature extraction network may be updating the parameter in the second feature extraction network in a momentum update manner.
In this embodiment, the parameters in the second feature extraction network may be updated based on the parameters in the first feature extraction network after the parameters in the first feature extraction network are updated, or the parameters in the second feature extraction network may be updated based on the parameters in the first feature extraction network in the parameter updating process of the first feature extraction network.
In this embodiment, the parameters in the second feature extraction network are updated based on the parameters in the first feature extraction network, so that the accuracy of the second feature extraction network can be improved, and the first feature extraction network is updated based on the second feature extraction network, so that the accuracy of the first feature extraction network can be improved.
It should be noted that, in the present disclosure, the parameters in the first feature extraction network and the second feature extraction network may be updated once or multiple times.
In one embodiment, the parameter in the first feature extraction network comprises a first parameter, and the parameter in the second feature extraction network comprises a second parameter, wherein the updating the parameter in the second feature extraction network based on the parameter in the first feature extraction network comprises:
updating the second parameter according to a preset mapping relation between the first parameter and the second parameter;
the second parameter in the preset mapping relationship is equal to the sum of a first weight product and a second weight product, the first weight product is the product of the second parameter and a first preset weight, the second weight product is equal to the product of the first parameter and a second preset weight, and the second preset weight is equal to 1 minus the difference of the first preset weight.
The first parameter and the second parameter may be parameters of the same attribute and the same position in the first feature extraction network and the second feature extraction network.
And the first parameter may represent a learnable parameter in the first feature extraction network, and the second parameter may represent a learnable parameter in the second feature extraction network.
The preset mapping relationship can be expressed by the following formula:
θ 2 ＝θ 2 .λ+θ 1 (1-λ)
theta as defined above 2 Denotes a second parameter, θ 1 Represents a first parameter, and λ represents the first predetermined weight, and in some embodiments, the first predetermined weight may be a hyperparameter, and its value may range from 0 to 0.99.
In this embodiment, since the parameters in the second feature extraction network are updated according to the preset mapping relationship, the accuracy of the second feature extraction network can be improved. In addition, since the parameters of the first feature extraction network are updated based on the second loss value, the fact that the first feature extraction network is supervised by using the parameter weighted average of the first feature extraction network and the second feature extraction network can be realized, so that the accuracy of the first feature extraction network is improved
In one embodiment, the first feature extraction network is a feature extraction network of the target neural network model at a first time, the first prediction network is a prediction network of the target neural network model at the first time, the second feature extraction network is a feature extraction network of the target neural network model at a second time, and the second prediction network is a prediction network of the target neural network model at the second time, wherein the first time is later than the second time.
The first feature extraction network and the second feature extraction network may be feature extraction networks having the same structure. The second prediction network and the second prediction network may be prediction networks having the same structure.
In this embodiment, since the first time is later than the second time, the second feature extraction network and the second prediction network may be understood as a feature extraction and prediction network at a previous time or a previous time, and the first feature extraction network and the first prediction network may be understood as a feature extraction and prediction network at a current time. Therefore, supervision on the network at the current moment based on the network at the previous moment or the network at the previous moment and the network at the current moment can be realized, and the robustness of the first feature extraction network is further improved.
In one embodiment, the first prediction network comprises a Global Average Pooling (GAP) layer and a normalized exponential function (Softmax) layer, the second prediction network comprises a GAP layer and a normalized exponential function layer, and the first prediction network and the second prediction network comprise the same GAP layer and normalized exponential function layer.
In this embodiment, the first prediction network and the second prediction network may include the same GAP layer and Softmax layer, so that the second loss value more accurately represents the difference between the first feature extraction network and the second feature extraction network, and thus, when the first feature extraction network is updated based on the second loss value, more reliable supervision may be achieved, so as to further improve the accuracy of the first feature extraction network.
In one embodiment, the classifying the image includes live-detecting the image, and the target neural network model is used for live-detecting the image through the first feature extraction network and the first prediction network.
The image living body detection may be face living body detection, in the scene, the sample image set may include a face living body sample and an attack sample image set, and the number of the face living body sample images and the number of the attack sample images may be one to one or approximately one to one.
In some embodiments, the image biopsy may be an animal biopsy.
In the embodiment, the second feature extraction network and the second prediction network can be omitted in the process of carrying out the living body detection on the image, and only the first feature extraction network and the first prediction network need to be used, so that the complexity of carrying out the living body detection on the image is reduced.
In the embodiment, the image living body detection can be applied to a plurality of scenes such as security, attendance, finance, entrance guard passage and the like, and the living body detection performance can be improved through the target neural network model.
In the present disclosure, the image classification is not limited to the image biopsy, and for example: it is also possible to classify the image into vehicle information, road information, and the like.
In one embodiment, the training process of the target neural network may be as shown in fig. 2:
acquiring a training sample image 201;
aiming at any input sample image chi, the images respectively pass through the image amplifier 202 twice to obtain two inconsistent homologous images chi 1 ，χ 2 。
Among them, in FIG. 1
And
representing a first feature extraction network and a second feature extraction network,
and
extracting networks for two features with the same structure, wherein learnable parameters of the networks are theta respectively 1 And theta 2 Homologous image χ through two augmentation operations 1 ，χ 2 Respectively through a feature extraction network
Obtaining the characteristic f 1 And f 2 . Computing the feature f by introducing a contrast Loss function (contrast Loss) 205 1 And f 2 The loss value of (2) is used for realizing stability supervision provided for the model in an unsupervised mode, and prompting the model to extract consistent characteristics from the homologous images with different data augmentation.
In addition, characteristic f 1 Obtaining a classification probability q through a GAP206 and a normalized exponential function (Softmax function) 207, and performing supervised classification learning through a preset real label c (Hard label) 208, wherein the classification probability q and a Loss value of the real label c are calculated through a cross entropy Loss function (CE Loss) 209 as follows:
wherein L is ce Represents the loss value, N is the total number of samples, c i Label representing the ith sample image, q i Representing the classification probability q of the ith sample image.
Further improving the robustness of the model, and learning through a momentum self-distillation mechanism, specifically as follows:
image x 1 While passing through a feature extraction network
And the same GAP210 and Softmax function 211, the corresponding classification probability vector p is obtained, and the partial classification probability is used as a soft label to supervise the feature f simultaneously 1 The loss function is the cross entropy loss function (Distill CE loss) 212 of the soft label as follows:
wherein L is dce Represents the loss value, N is the total number of samples, p is i Representing the classification probability p, q of the ith sample image i Representing the classification probability q of the ith sample image.
By the above, the network in which the feature is extracted can be realized
Parameter theta of 1 The gradient update is performed by three part loss values.
In addition, a feature extraction network
Parameter theta of 2 Then, through the momentum update, the update rule is shown as 213 in fig. 2, which may specifically refer to the corresponding description of the above embodiment, and is not described herein again.
In addition, in this embodiment, only the feature extraction network may be needed in the model testing phase and the application phase
That is, the image passes through a feature extraction network
And obtaining characteristics, and then obtaining corresponding classification probability through GAP and Softmax functions, namely obtaining an output result.
In the disclosure, since the first loss value is a loss value corresponding to the first classification information and the preset label of the sample image, and the second loss value is a loss value corresponding to the first classification information and the second classification information, parameters in the first feature extraction network are updated based on the loss value set, so that the first feature extraction network can be supervised through the preset label of the sample image and the second classification information, and thus the robustness of the first feature extraction network can be improved, and further the robustness of the target neural network model can be improved.
Referring to fig. 3, fig. 3 is a flowchart of an image classification method provided by the present disclosure, as shown in fig. 3, including the following steps:
s301, acquiring first image characteristics of an image to be classified through a first characteristic extraction network in a target neural network model;
step S302, through a first prediction network of the target neural network model, predicting classification information of the image to be classified by using the first image characteristics;
the parameter of the first feature extraction network is a parameter updated based on a loss value set, where the loss value set includes a first loss value and a second loss value, the first loss value is a loss value corresponding to a preset label of a sample image and first classification information of the sample image predicted by the first prediction network, the second loss value is a loss value corresponding to the first classification information and second classification information, and the second classification information is classification information of the sample image predicted by the second prediction network.
The target neural network model may be the target neural network model in the above embodiment. All embodiments of the target neural network model in the above embodiments can be applied to this embodiment, and are not described herein again.
The image classification may be a live body detection of the image, and in some embodiments, may be a vehicle information classification, a road information classification, or the like of the image.
In this embodiment, since the first loss value is a loss value corresponding to the first classification information and the preset label of the sample image, and the second loss value is a loss value corresponding to the first classification information and the second classification information, parameters in the first feature extraction network are updated based on the loss value set, and the first feature extraction network can be supervised through the preset label of the sample image and the second classification information, so that the robustness of the first feature extraction network is improved, and the accuracy of image classification is further improved.
Referring to fig. 4a, the present disclosure provides a schematic diagram of a classification model training apparatus, as shown in fig. 4a, the classification model training apparatus 400 includes:
a first obtaining module 401, configured to obtain a first image feature of the sample image through a first feature extraction network in a target neural network model, where the target neural network model is used to classify the image;
a first prediction module 402, configured to predict, through a first prediction network of the target neural network model, first classification information of the sample image by using the first image feature, and calculate, through a first loss function, a first loss value of the first classification information corresponding to a preset label of the sample image;
a second obtaining module 403, configured to obtain a second image feature of the sample image through a second feature extraction network;
a second prediction module 404, configured to predict, through a second prediction network, second classification information of the sample image by using the second image feature, and calculate, through a second loss function, a second loss value corresponding to the first classification information and the second classification information;
a first updating module 405, configured to update a parameter in the first feature extraction network based on a set of loss values, where the set of loss values includes the first loss value and the second loss value.
In one embodiment, as shown in fig. 4b, the apparatus further comprises:
an augmentation module 406, configured to perform image augmentation operations on the sample image for N times to obtain N augmented images of the sample image, where N is an integer greater than 1;
the first image feature is an image feature obtained by feature extraction of a first augmented image in the N augmented images by the first feature extraction network;
the second image feature is an image feature obtained by performing feature extraction on the first augmented image by the second feature extraction network.
In one embodiment, as shown in fig. 4c, the apparatus further comprises:
an extracting module 407, configured to perform feature extraction on a second augmented image in the N augmented images through the first feature extraction network to obtain a third image feature;
a calculating module 408, configured to calculate a third loss value corresponding to the first image feature and the third image feature through a contrast loss function;
the set of penalty values further includes the third penalty value.
In one embodiment, as shown in fig. 4d, the apparatus further comprises:
a second updating module 409, configured to update the parameters in the second feature extraction network based on the parameters in the first feature extraction network.
In one embodiment, the parameter in the first feature extraction network comprises a first parameter, and the parameter in the second feature extraction network comprises a second parameter, wherein the second updating module 409 is configured to:
updating the second parameter according to a preset mapping relation between the first parameter and the second parameter;
the second parameter in the preset mapping relationship is equal to the sum of a first weight product and a second weight product, the first weight product is the product of the second parameter and a first preset weight, the second weight product is equal to the product of the first parameter and a second preset weight, and the second preset weight is equal to the difference between 1 and the first preset weight.
In one embodiment, the first feature extraction network is a feature extraction network of the target neural network model at a first time, the first prediction network is a prediction network of the target neural network model at the first time, the second feature extraction network is a feature extraction network of the target neural network model at a second time, and the second prediction network is a prediction network of the target neural network model at the second time, the first time being later than the second time.
In one embodiment, the first prediction network comprises a global average pooled GAP layer and a normalized index function layer, the second prediction network comprises a GAP layer and a normalized index function layer, and the first prediction network and the second prediction network comprise the same GAP layer and normalized index function layer.
In one embodiment, the classifying the image includes live-detecting the image, and the target neural network model is used for live-detecting the image through the first feature extraction network and the first prediction network.
The classification model training device provided by the disclosure can realize each process realized by the classification model training method provided by the disclosure, and achieve the same technical effect, and in order to avoid repetition, the details are not repeated here.
Referring to fig. 5, the present disclosure provides a schematic diagram of an image classification apparatus, as shown in fig. 5, an image classification apparatus 500 includes:
an extraction module 501, configured to obtain a first image feature of an image to be classified through a first feature extraction network in a target neural network model;
a classification module 502, configured to predict, through a first prediction network of the target neural network model, classification information of the image to be classified by using the first image feature;
the parameter of the first feature extraction network is a parameter updated based on a loss value set, where the loss value set includes a first loss value and a second loss value, the first loss value is a loss value corresponding to a preset label of a sample image and first classification information of the sample image predicted by the first prediction network, the second loss value is a loss value corresponding to the first classification information and second classification information, and the second classification information is classification information of the sample image predicted by the second prediction network.
The image classification device provided by the present disclosure can implement each process implemented by the image classification method provided by the present disclosure, and achieve the same technical effect, and for avoiding repetition, the details are not repeated here.
The present disclosure also provides an electronic device, a readable storage medium, and a computer program product according to embodiments of the present disclosure.
Wherein, above-mentioned electronic equipment includes: at least one processor; and a memory communicatively coupled to the at least one processor; wherein the memory stores instructions executable by the at least one processor to enable the at least one processor to perform a classification model training method or an image classification method provided by the present disclosure.
The readable storage medium stores computer instructions for causing the computer to execute the classification model training method or the image classification method provided by the present disclosure.
The above computer program product comprises a computer program, which when executed by a processor implements the classification model training method or the image classification method provided by the present disclosure.
FIG. 6 illustrates a schematic block diagram of an example electronic device 600 that can be used to implement embodiments of the present disclosure. Electronic devices are intended to represent various forms of digital computers, such as laptops, desktops, workstations, personal digital assistants, servers, blade servers, mainframes, and other appropriate computers. Electronic devices may also represent various forms of mobile devices, such as personal digital processors, cellular telephones, smart phones, wearable devices, and other similar computing devices. The components shown herein, their connections and relationships, and their functions, are meant to be examples only, and are not intended to limit implementations of the disclosure described and/or claimed herein.
As shown in fig. 6, the apparatus 600 includes a computing unit 601, which can perform various appropriate actions and processes according to a computer program stored in a Read Only Memory (ROM) 602 or a computer program loaded from a storage unit 608 into a Random Access Memory (RAM) 603. In the RAM 603, various programs and data required for the operation of the device 600 can also be stored. The calculation unit 601, the ROM 602, and the RAM 603 are connected to each other via a bus 604. An input/output (I/O) interface 605 is also connected to bus 604.
A number of components in the device 600 are connected to the I/O interface 605, including: an input unit 606 such as a keyboard, a mouse, or the like; an output unit 607 such as various types of displays, speakers, and the like; a storage unit 608, such as a magnetic disk, optical disk, or the like; and a communication unit 609 such as a network card, modem, wireless communication transceiver, etc. The communication unit 609 allows the device 600 to exchange information/data with other devices via a computer network such as the internet and/or various telecommunication networks.
The computing unit 601 may be a variety of general and/or special purpose processing components having processing and computing capabilities. Some examples of the computing unit 601 include, but are not limited to, a Central Processing Unit (CPU), a Graphics Processing Unit (GPU), various dedicated Artificial Intelligence (AI) computing chips, various computing units running machine learning model algorithms, a Digital Signal Processor (DSP), and any suitable processor, controller, microcontroller, and so forth. The calculation unit 601 performs the respective methods and processes described above, such as a classification model training method or an image classification method. For example, in some embodiments, the classification model training method or the image classification method may be implemented as a computer software program tangibly embodied in a machine-readable medium, such as storage unit 608. In some embodiments, part or all of the computer program may be loaded and/or installed onto the device 600 via the ROM 602 and/or the communication unit 609. When the computer program is loaded into RAM 603 and executed by the computing unit 601, one or more steps of the classification model training method or the image classification method described above may be performed. Alternatively, in other embodiments, the calculation unit 601 may be configured by any other suitable means (e.g. by means of firmware) to perform a classification model training method or an image classification method.
Various implementations of the systems and techniques described here above may be implemented in digital electronic circuitry, integrated circuitry, field Programmable Gate Arrays (FPGAs), application Specific Integrated Circuits (ASICs), application Specific Standard Products (ASSPs), system on a chip (SOCs), load programmable logic devices (CPLDs), computer hardware, firmware, software, and/or combinations thereof. These various embodiments may include: implemented in one or more computer programs that are executable and/or interpretable on a programmable system including at least one programmable processor, which may be special or general purpose, receiving data and instructions from, and transmitting data and instructions to, a storage system, at least one input device, and at least one output device.
Program code for implementing the methods of the present disclosure may be written in any combination of one or more programming languages. These program codes may be provided to a processor or controller of a general purpose computer, special purpose computer, or other programmable data processing apparatus, such that the program codes, when executed by the processor or controller, cause the functions/operations specified in the flowchart and/or block diagram to be performed. The program code may execute entirely on the machine, partly on the machine, as a stand-alone software package partly on the machine and partly on a remote machine or entirely on the remote machine or server.
In the context of this disclosure, a machine-readable medium may be a tangible medium that can contain, or store a program for use by or in connection with an instruction execution system, apparatus, or device. The machine-readable medium may be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium may include, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any suitable combination of the foregoing. More specific examples of a machine-readable storage medium would include an electrical connection based on one or more wires, a portable computer diskette, a hard disk, a Random Access Memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.
To provide for interaction with a user, the systems and techniques described here can be implemented on a computer having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to a user; and a keyboard and a pointing device (e.g., a mouse or a trackball) by which a user can provide input to the computer. Other kinds of devices may also be used to provide for interaction with a user; for example, feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user may be received in any form, including acoustic, speech, or tactile input.
The systems and techniques described here can be implemented in a computing system that includes a back-end component (e.g., as a data server), or that includes a middleware component (e.g., an application server), or that includes a front-end component (e.g., a user computer having a graphical user interface or a web browser through which a user can interact with an implementation of the systems and techniques described here), or any combination of such back-end, middleware, or front-end components. The components of the system can be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include: local Area Networks (LANs), wide Area Networks (WANs), and the Internet.
The computer system may include clients and servers. A client and server are generally remote from each other and typically interact through a communication network. The relationship of client and server arises by virtue of computer programs running on the respective computers and having a client-server relationship to each other. The server may be a cloud server, a server of a distributed system, or a server with a combined blockchain.
It should be understood that various forms of the flows shown above may be used, with steps reordered, added, or deleted. For example, the steps described in the present disclosure may be executed in parallel, sequentially or in different orders, and are not limited herein as long as the desired results of the technical solutions disclosed in the present disclosure can be achieved.
The above detailed description should not be construed as limiting the scope of the disclosure. It should be understood by those skilled in the art that various modifications, combinations, sub-combinations and substitutions may be made in accordance with design requirements and other factors. Any modification, equivalent replacement, and improvement made within the spirit and principle of the present disclosure should be included in the protection scope of the present disclosure.
Claims (21)
1. A classification model training method, comprising:
acquiring first image characteristics of a sample image through a first characteristic extraction network in a target neural network model, wherein the target neural network model is used for classifying the image;
predicting first classification information of the sample image by using the first image features through a first prediction network of the target neural network model, and calculating a first loss value of the first classification information corresponding to a preset label of the sample image through a first loss function;
acquiring a second image characteristic of the sample image through a second characteristic extraction network;
predicting second classification information of the sample image by using the second image feature through a second prediction network, and calculating a second loss value corresponding to the first classification information and the second classification information through a second loss function;
updating a parameter in the first feature extraction network based on a set of penalty values, the set of penalty values including the first penalty value and the second penalty value.
2. The method of claim 1, further comprising:
carrying out N times of image augmentation operations on the sample image to obtain N augmented images of the sample image, wherein N is an integer greater than 1;
the first image feature is an image feature obtained by feature extraction of a first augmented image in the N augmented images by the first feature extraction network;
the second image feature is an image feature obtained by performing feature extraction on the first augmented image by the second feature extraction network.
3. The method of claim 2, further comprising:
performing feature extraction on a second augmented image in the N augmented images through the first feature extraction network to obtain a third image feature;
calculating a third loss value corresponding to the first image feature and the third image feature through a contrast loss function;
the set of penalty values further includes the third penalty value.
4. The method of any of claims 1-3, further comprising:
updating the parameters in the second feature extraction network based on the parameters in the first feature extraction network.
5. The method of claim 4, wherein the parameter in the first feature extraction network comprises a first parameter, wherein the parameter in the second feature extraction network comprises a second parameter, and wherein updating the parameter in the second feature extraction network based on the parameter in the first feature extraction network comprises:
updating the second parameter according to a preset mapping relation between the first parameter and the second parameter;
the second parameter in the preset mapping relationship is equal to the sum of a first weight product and a second weight product, the first weight product is the product of the second parameter and a first preset weight, the second weight product is equal to the product of the first parameter and a second preset weight, and the second preset weight is equal to 1 minus the difference of the first preset weight.
6. The method of any one of claims 1 to 3, the first feature extraction network being a feature extraction network of the target neural network model at a first time, the first prediction network being a prediction network of the target neural network model at the first time, the second feature extraction network being a feature extraction network of the target neural network model at a second time, the second prediction network being a prediction network of the target neural network model at the second time, the first time being later than the second time.
7. The method of claim 6, the first prediction network comprising a global average pooled GAP layer and a normalized index function layer, the second prediction network comprising a GAP layer and a normalized index function layer, and the first prediction network and the second prediction network comprising the same GAP layer and normalized index function layer.
8. The method of any one of claims 1 to 3, the classifying the image comprising lively detecting the image, and the target neural network model being used to lively detect the image through the first feature extraction network and the first prediction network.
9. An image classification method, comprising:
acquiring a first image characteristic of an image to be classified through a first characteristic extraction network in a target neural network model;
predicting classification information of the image to be classified by using the first image features through a first prediction network of the target neural network model;
the parameter of the first feature extraction network is a parameter updated based on a loss value set, where the loss value set includes a first loss value and a second loss value, the first loss value is a loss value corresponding to a preset label of a sample image and first classification information of the sample image predicted by the first prediction network, the second loss value is a loss value corresponding to the first classification information and second classification information, and the second classification information is classification information of the sample image predicted by the second prediction network.
10. A classification model training apparatus comprising:
the first acquisition module is used for acquiring first image characteristics of the sample image through a first characteristic extraction network in a target neural network model, and the target neural network model is used for classifying the image;
the first prediction module is used for predicting first classification information of the sample image by using the first image characteristics through a first prediction network of the target neural network model, and calculating a first loss value corresponding to a preset label of the sample image through a first loss function;
the second acquisition module is used for acquiring second image characteristics of the sample image through a second characteristic extraction network;
a second prediction module, configured to predict, through a second prediction network, second classification information of the sample image by using the second image feature, and calculate, through a second loss function, a second loss value corresponding to the first classification information and the second classification information;
a first updating module configured to update a parameter in the first feature extraction network based on a set of penalty values, where the set of penalty values includes the first penalty value and the second penalty value.
11. The apparatus of claim 10, further comprising:
the augmentation module is used for carrying out N times of image augmentation operations on the sample image to obtain N augmented images of the sample image, wherein N is an integer greater than 1;
the first image feature is an image feature obtained by feature extraction of a first augmented image in the N augmented images by the first feature extraction network;
the second image feature is an image feature obtained by performing feature extraction on the first augmented image by the second feature extraction network.
12. The apparatus of claim 11, further comprising:
the extraction module is used for extracting the features of a second augmented image in the N augmented images through the first feature extraction network to obtain third image features;
the calculation module is used for calculating a third loss value corresponding to the first image feature and the third image feature through a contrast loss function;
the set of penalty values further includes the third penalty value.
13. The apparatus of any of claims 10 to 12, further comprising:
and the second updating module is used for updating the parameters in the second characteristic extraction network based on the parameters in the first characteristic extraction network.
14. The apparatus of claim 13, wherein the parameter in the first feature extraction network comprises a first parameter, wherein the parameter in the second feature extraction network comprises a second parameter, and wherein the second update module is configured to:
updating the second parameter according to a preset mapping relation between the first parameter and the second parameter;
the second parameter in the preset mapping relationship is equal to the sum of a first weight product and a second weight product, the first weight product is the product of the second parameter and a first preset weight, the second weight product is equal to the product of the first parameter and a second preset weight, and the second preset weight is equal to 1 minus the difference of the first preset weight.
15. The apparatus of any one of claims 10 to 12, the first feature extraction network being a feature extraction network of the target neural network model at a first time, the first prediction network being a prediction network of the target neural network model at a first time, the second feature extraction network being a feature extraction network of the target neural network model at a second time, the second prediction network being a prediction network of the target neural network model at a second time, the first time being later than the second time.
16. The device of claim 15, the first prediction network comprising a global average pooled GAP layer and a normalized index function layer, the second prediction network comprising a GAP layer and a normalized index function layer, and the first prediction network and the second prediction network comprising the same GAP layer and normalized index function layer.
17. The apparatus of any one of claims 10 to 12, the classifying the image comprising live-detecting the image, and the target neural network model being configured to live-detect the image through the first feature extraction network and the first prediction network.
18. An image classification apparatus comprising:
the extraction module is used for acquiring first image characteristics of the image to be classified through a first characteristic extraction network in the target neural network model;
the classification module is used for predicting classification information of the image to be classified by utilizing the first image characteristics through a first prediction network of the target neural network model;
the parameter of the first feature extraction network is a parameter updated based on a loss value set, the loss value set includes a first loss value and a second loss value, the first loss value is a loss value corresponding to a preset label of a sample image predicted by the first prediction network, and the second loss value is a loss value corresponding to the first classification information and the second classification information, and the second classification information is classification information of the sample image predicted by the second prediction network.
19. An electronic device, comprising:
at least one processor; and
a memory communicatively coupled to the at least one processor; wherein, the first and the second end of the pipe are connected with each other,
the memory stores instructions executable by the at least one processor to enable the at least one processor to perform the method of any one of claims 1-8 or to enable the at least one processor to perform the method of claim 9.
20. A non-transitory computer readable storage medium having stored thereon computer instructions for causing the computer to perform the method of any one of claims 1-8 or causing the computer to perform the method of claim 9.
21. A computer program product comprising a computer program which, when executed by a processor, implements the method according to any one of claims 1-8, or which, when executed by a processor, implements the method according to claim 9.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202211350323.4A CN115631381A (en) | 2022-10-31 | 2022-10-31 | Classification model training method, image classification device and electronic equipment |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202211350323.4A CN115631381A (en) | 2022-10-31 | 2022-10-31 | Classification model training method, image classification device and electronic equipment |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN115631381A true CN115631381A (en) | 2023-01-20 |
Family
ID=84908712
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202211350323.4A Pending CN115631381A (en) | 2022-10-31 | 2022-10-31 | Classification model training method, image classification device and electronic equipment |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN115631381A (en) |
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN116663650A (en) * | 2023-06-06 | 2023-08-29 | 北京百度网讯科技有限公司 | Training method of deep learning model, target object detection method and device |
| CN117173493A (en) * | 2023-11-02 | 2023-12-05 | 腾讯科技（深圳）有限公司 | Classification model training method, device, equipment, program product and storage medium |
-
2022
- 2022-10-31 CN CN202211350323.4A patent/CN115631381A/en active Pending
Cited By (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN116663650A (en) * | 2023-06-06 | 2023-08-29 | 北京百度网讯科技有限公司 | Training method of deep learning model, target object detection method and device |
| CN116663650B (en) * | 2023-06-06 | 2023-12-19 | 北京百度网讯科技有限公司 | Training method of deep learning model, target object detection method and device |
| CN117173493A (en) * | 2023-11-02 | 2023-12-05 | 腾讯科技（深圳）有限公司 | Classification model training method, device, equipment, program product and storage medium |
| CN117173493B (en) * | 2023-11-02 | 2024-02-27 | 腾讯科技（深圳）有限公司 | Classification model training method, device, equipment, program product and storage medium |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN113553864B (en) | Translation model training method and device, electronic equipment and storage medium | |
| CN115631381A (en) | Classification model training method, image classification device and electronic equipment | |
| CN112560996B (en) | User portrait identification model training method, device, readable storage medium and product | |
| CN113971751A (en) | Training feature extraction model, and method and device for detecting similar images | |
| CN114549840B (en) | Training method of semantic segmentation model and semantic segmentation method and device | |
| CN112580733B (en) | Classification model training method, device, equipment and storage medium | |
| CN113657289B (en) | Training method and device of threshold estimation model and electronic equipment | |
| CN113705628B (en) | Determination method and device of pre-training model, electronic equipment and storage medium | |
| CN112784778A (en) | Method, apparatus, device and medium for generating model and identifying age and gender | |
| CN112907552A (en) | Robustness detection method, device and program product for image processing model | |
| CN112966744A (en) | Model training method, image processing method, device and electronic equipment | |
| CN115147680B (en) | Pre-training method, device and equipment for target detection model | |
| CN113627536A (en) | Model training method, video classification method, device, equipment and storage medium | |
| CN114881129A (en) | Model training method and device, electronic equipment and storage medium | |
| CN113947188A (en) | Training method of target detection network and vehicle detection method | |
| CN114120454A (en) | Training method and device of living body detection model, electronic equipment and storage medium | |
| CN115273148B (en) | Pedestrian re-recognition model training method and device, electronic equipment and storage medium | |
| CN113361621B (en) | Method and device for training model | |
| CN114445668A (en) | Image recognition method and device, electronic equipment and storage medium | |
| CN113935482A (en) | Training method and device for pedestrian re-recognition network | |
| CN113989568A (en) | Target detection method, training method, device, electronic device and storage medium | |
| CN114707638A (en) | Model training method, model training device, object recognition method, object recognition device, object recognition medium and product | |
| CN114330576A (en) | Model processing method and device, and image recognition method and device | |
| CN113936158A (en) | Label matching method and device | |
| CN113657596A (en) | Method and device for training model and image recognition |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |