CN115953819B - Training method, device, equipment and storage medium of face recognition model - Google Patents
Training method, device, equipment and storage medium of face recognition model Download PDFInfo
- Publication number
- CN115953819B CN115953819B CN202211702258.7A CN202211702258A CN115953819B CN 115953819 B CN115953819 B CN 115953819B CN 202211702258 A CN202211702258 A CN 202211702258A CN 115953819 B CN115953819 B CN 115953819B
- Authority
- CN
- China
- Prior art keywords
- face
- recognition model
- sample image
- face recognition
- image
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
- 238000012549 training Methods 0.000 title claims abstract description 137
- 238000000034 method Methods 0.000 title claims abstract description 84
- 238000003860 storage Methods 0.000 title claims abstract description 15
- 238000004590 computer program Methods 0.000 claims description 13
- 238000000605 extraction Methods 0.000 claims description 3
- 238000012545 processing Methods 0.000 abstract description 2
- 238000010586 diagram Methods 0.000 description 8
- 238000004891 communication Methods 0.000 description 7
- 238000001514 detection method Methods 0.000 description 7
- 230000000694 effects Effects 0.000 description 5
- 230000006870 function Effects 0.000 description 5
- 238000005516 engineering process Methods 0.000 description 4
- 238000007781 pre-processing Methods 0.000 description 4
- 230000008569 process Effects 0.000 description 4
- 238000009826 distribution Methods 0.000 description 3
- 238000013527 convolutional neural network Methods 0.000 description 2
- 230000007547 defect Effects 0.000 description 2
- 239000004973 liquid crystal related substance Substances 0.000 description 2
- 230000003287 optical effect Effects 0.000 description 2
- 230000002411 adverse Effects 0.000 description 1
- 238000004364 calculation method Methods 0.000 description 1
- 238000004140 cleaning Methods 0.000 description 1
- 238000005520 cutting process Methods 0.000 description 1
- 238000007418 data mining Methods 0.000 description 1
- 238000011161 development Methods 0.000 description 1
- 238000001914 filtration Methods 0.000 description 1
- 230000006872 improvement Effects 0.000 description 1
- 238000013508 migration Methods 0.000 description 1
- 230000005012 migration Effects 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 238000012544 monitoring process Methods 0.000 description 1
- 238000013441 quality evaluation Methods 0.000 description 1
- 238000013139 quantization Methods 0.000 description 1
- 238000010187 selection method Methods 0.000 description 1
- 238000006467 substitution reaction Methods 0.000 description 1
- 230000009466 transformation Effects 0.000 description 1
- 238000013519 translation Methods 0.000 description 1
Classifications
-
- Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
- Y02T—CLIMATE CHANGE MITIGATION TECHNOLOGIES RELATED TO TRANSPORTATION
- Y02T10/00—Road transport of goods or passengers
- Y02T10/10—Internal combustion engine [ICE] based vehicles
- Y02T10/40—Engine management systems
Landscapes
- Image Analysis (AREA)
Abstract
The invention provides a training method, a training device, training equipment and a training storage medium of a face recognition model, and relates to the technical field of image processing, wherein the training method comprises the following steps: acquiring a plurality of first face sample images; extracting image characteristics of each first face sample image, and determining the quality score of each first face sample image based on the image characteristics of each first face sample image; and training an initial face recognition model based on the label information of each first face sample image and the quality score to obtain a face recognition model, wherein the label information is used for representing user information corresponding to the first face sample image. The training method, the device, the equipment and the storage medium of the face recognition model can improve the accuracy of the face recognition model training, and obtain a more accurate face recognition model, thereby improving the accuracy of face recognition.
Description
Technical Field
The present invention relates to the field of image processing technologies, and in particular, to a training method, apparatus, device, and storage medium for a face recognition model.
Background
Along with the rapid development of science and technology, the face recognition technology has been widely applied to various fields such as authentication of financial systems, attendance check-in systems, security monitoring and the like, and the importance of accurately recognizing faces is self-evident. The correct selection of the training set in the face recognition model is one of important preconditions for accurately recognizing the face.
However, in the face data set disclosed at present, the face data set is basically some celebrity images, and the number of the celebrity images is numerous, so that the type of the face image selected by the training set is too single, the face recognition performance of the training face recognition model in an actual scene is poor, and the service requirement is difficult to meet.
In order to improve the accuracy of model training, in the process of actual model training, training staff can collect more other types of face images in a network picture collecting mode so as to make up for the defects of the training set. However, the pictures in the actual service are difficult to collect, and most of the pictures are poor in quality, so that the picture data and the face recognition model which is collected and trained by the public face data set are easy to be in the condition of overfitting. In addition, the domain difference of the two image data is large, the generalization of the model is also seriously influenced, so that the accuracy of the face recognition model obtained by training is poor, and the accuracy of face recognition is reduced.
Disclosure of Invention
The invention provides a training method, device, equipment and storage medium of a face recognition model, which are used for solving the defect that the trained face recognition model is inaccurate due to the influence of low-quality images in the prior art, and achieving the purpose of improving the accuracy of the face recognition model so as to more accurately carry out face recognition.
The invention provides a training method of a face recognition model, which comprises the following steps:
acquiring a plurality of first face sample images;
extracting image characteristics of each first face sample image, and determining the quality score of each first face sample image based on the image characteristics of each first face sample image;
and training an initial face recognition model based on the label information of each first face sample image and the quality score to obtain a face recognition model, wherein the label information is used for representing user information corresponding to the first face sample image.
According to the training method of the face recognition model provided by the invention, the quality score of each first face sample image is determined based on the image characteristics of each first face sample image, and the training method comprises the following steps:
for each first face sample image, determining a positive sample set and a negative sample set of the first face sample image, wherein a user corresponding to the positive sample image in the positive sample set is the same as a user corresponding to the first face sample image, and a user corresponding to the negative sample image in the negative sample set is different from the user corresponding to the first face sample image;
Based on the image characteristics of each first face sample image, respectively determining a first distance between the first face sample image and each positive sample image in the positive sample set and a second distance between the first face sample image and each negative sample image in the negative sample set;
a quality score of the first face sample image is determined based on the first distance and the second distance.
According to the training method of the face recognition model provided by the invention, the initial face recognition model is trained based on the label information and the quality score of each first face sample image to obtain the face recognition model, and the training method comprises the following steps:
inputting each first face sample image into the initial face recognition model, and determining a recognition probability value corresponding to each first face sample image based on the tag information and the feature information extracted by the initial face recognition model;
determining a new recognition probability value corresponding to each first face sample image based on the recognition probability value and the quality score;
and training an initial face recognition model based on the new recognition probability value to obtain the face recognition model.
According to the training method of the face recognition model provided by the invention, the initial face recognition model is trained based on the label information and the quality score of each first face sample image to obtain the face recognition model, and the training method comprises the following steps:
removing the first face sample image with the quality score smaller than a preset value based on the quality score of each first face sample image to obtain a removed first face sample image;
and training the initial face recognition model based on the label information and the quality score of the removed first face sample image to obtain the face recognition model.
According to the training method of the face recognition model provided by the invention, the method further comprises the following steps:
acquiring a plurality of second face sample images, wherein the image quality of the second face sample images is greater than that of the first face sample images;
pre-training the first face recognition model based on the second face sample image to obtain a second face recognition model;
determining a first face sample image belonging to the same user;
and fine-tuning model parameters of the second face recognition model based on the first face sample images, belonging to the same user, of which the number is larger than a preset value, so as to obtain the initial face recognition model.
The invention also provides a face recognition method, which comprises the following steps:
acquiring a face image to be identified;
and inputting the face image to be recognized into a face recognition model to obtain a face recognition result of the face image to be recognized, wherein the face recognition model is obtained by training according to the training method of any face recognition model.
The invention also provides a training device of the face recognition model, which comprises:
the acquisition module is used for acquiring a plurality of first face sample images;
the extraction module is used for extracting the image characteristics of each first face sample image;
the determining module is used for determining the quality score of each first face sample image based on the image characteristics of each first face sample image;
the training module is used for training the initial face recognition model based on the label information and the quality score of each first face sample image to obtain a face recognition model, and the label information is used for representing the user information corresponding to the first face sample image.
The invention also provides a face recognition device, which comprises:
the acquisition module is used for acquiring the face image to be identified;
the recognition module is used for inputting the face image to be recognized into a face recognition model to obtain a face recognition result of the face image to be recognized, wherein the face recognition model is obtained by training the training method of any one of the face recognition models.
The invention also provides an electronic device comprising a memory, a processor and a computer program stored on the memory and executable on the processor, wherein the processor realizes the training method of the face recognition model according to any one of the above and realizes the face recognition method according to the above when executing the program.
The invention also provides a computer readable storage medium having stored thereon a computer program which, when executed by a processor, implements a training method of a face recognition model as described in any of the above, and implements a face recognition method as described above.
The invention also provides a computer program product comprising a computer program which when executed by a processor implements a training method of a face recognition model as described in any one of the above, and implements a face recognition method as described above.
According to the training method, the training device, the training equipment and the training storage medium for the face recognition model, the image characteristics of each first face sample image are extracted by extracting the image characteristics of the plurality of acquired first face sample images, and the quality score of each first face sample image is determined based on the image characteristics of the first face sample images. On the basis, training an initial face recognition model through label information and quality scores corresponding to each first face sample image to obtain the face recognition model. Because the quality score can be used as a pseudo tag when the face recognition model is trained, and the network parameters of the face recognition model can be adjusted together with the tag information, the problem of poor model generalization caused by training a sample image with good image quality and a sample image with poor image quality in the same mode can be avoided because the image quality of each first face sample image is considered, the accuracy and the robustness of the face recognition model training are improved, a more accurate face recognition model is obtained, and the accuracy of face recognition is improved.
Drawings
In order to more clearly illustrate the invention or the technical solutions of the prior art, the following description will briefly explain the drawings used in the embodiments or the description of the prior art, and it is obvious that the drawings in the following description are some embodiments of the invention, and other drawings can be obtained according to the drawings without inventive effort for a person skilled in the art.
Fig. 1 is a schematic flow chart of a training method of a face recognition model according to an embodiment of the present invention;
FIG. 2 is a schematic flow chart of image preprocessing according to an embodiment of the present invention;
fig. 3 is a schematic diagram illustrating determination of quality scores in a training method of a face recognition model according to an embodiment of the present invention;
fig. 4 is a second flowchart of a training method of a face recognition model according to an embodiment of the present invention;
fig. 5 is a schematic flow chart of a face recognition method according to an embodiment of the present invention;
fig. 6 is a schematic structural diagram of a training device for a face recognition model according to an embodiment of the present invention;
fig. 7 is a schematic structural diagram of a face recognition device according to an embodiment of the present invention;
fig. 8 is a schematic structural diagram of an electronic device provided by the present invention.
Detailed Description
For the purpose of making the objects, technical solutions and advantages of the present invention more apparent, the technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings, and it is apparent that the described embodiments are some embodiments of the present invention, not all embodiments. All other embodiments, which can be made by those skilled in the art based on the embodiments of the invention without making any inventive effort, are intended to be within the scope of the invention.
As a new identification technology, face recognition is widely used in various industries, but in some special situations, it still cannot achieve satisfactory effects.
In order to improve the accuracy of face recognition, in the prior art, a public face data set is generally combined with face images acquired in practical application to increase the number of face images of different classes, so that a face recognition model obtained through training is more accurate, and the accuracy of face recognition is improved. However, the selection method inevitably introduces some face images with poor quality, such as face images with insufficient definition or detail noise, that is, noise may exist in the label sample, so that the recognition performance of the face recognition model is generalized in some scenes, and the accuracy of face recognition is reduced.
Aiming at the problems that the face image quality in the training set is poor and the recognition capability of the face recognition model is poor under certain scenes due to the fact that the label sample is noisy, the embodiment of the invention provides a training method of the face recognition model, and the method trains the face recognition model by utilizing a data mining method, so that the performance of the face recognition model under various scenes is obviously improved. Specifically, the quality score of each face sample image is calculated on the basis of extracting the image characteristics corresponding to each face sample image, the quality score can be used as a pseudo tag, and the network parameters of the face recognition model are adjusted together with tag information, so that the problem of poor model generalization caused by training a sample image with good image quality and a sample image with poor image quality in the same mode is avoided because the image quality of each first face sample image is considered, and the negative influence of a low-quality image on the face recognition model is avoided, so that the accuracy and robustness of the face recognition model are improved, and the problem of face recognition performance generalization under various scenes is solved.
The following describes a training method of a face recognition model according to an embodiment of the present invention with reference to fig. 1 to fig. 4. The method can be applied to face recognition scenes, and the main body for executing the method can be any terminal equipment which is in communication connection with a camera, such as a mobile phone, a computer or any other face recognition device.
Fig. 1 is a schematic flow chart of a training method of a face recognition model according to an embodiment of the present invention, as shown in fig. 1, the method includes:
step 101: a plurality of first face sample images are acquired.
The plurality of first face sample images may include face sample images of a plurality of users, where each user may correspond to a plurality of face sample images in order to improve accuracy of model training. It should be appreciated that the plurality of first face sample images are herein face sample images of different image quality.
Specifically, a plurality of sample images can be collected on a public website through a web crawler method, and in order to facilitate subsequent training of a face recognition model and application of an actual scene, after the plurality of sample images are obtained, the sample images are usually required to be preprocessed to obtain a plurality of first face sample images.
Fig. 2 is a schematic flow chart of image preprocessing according to an embodiment of the present invention, and as shown in fig. 2, the preprocessing process includes:
step 201: a sample image is input.
Step 202: and carrying out face detection on the sample image.
Specifically, after the sample image is acquired, face detection can be performed on the input sample image by means of network model detection. If no face exists in the sample image, discarding the sample image, and re-executing step 201 to perform face detection on other sample images; if there is a face in the sample image, step 203 is performed.
Step 203: judging whether the face meets the requirements.
Specifically, after a face is detected to be included in the sample image, the confidence coefficient or the size of the face is calculated so as to be compared with a preset threshold value. If the comparison result is smaller than the preset threshold, for example, 60×60, filtering the face image, that is, discarding the sample image, and re-executing step 201; otherwise, the following step 204 is performed.
Step 204: and detecting the key points of the human face in the human face image.
The face key points at least comprise 25 face key points including a left eye, a right eye, a nose tip, a left mouth corner and a right mouth corner, and the detection of the face key points can be performed in a network model mode, so that the detection is not particularly limited.
Step 205: and carrying out radial transformation according to the coordinates of 5 key points of the left eye, the right eye, the nose tip, the left mouth corner and the right mouth corner in the face image to align the face.
Specifically, the face images can be aligned to the front face through operations such as translation, rotation, scaling and the like according to the 5 key point coordinates, so that the execution of the face recognition task is facilitated.
Step 206: and cutting the aligned face image into a face sample image with a fixed size.
Specifically, after the aligned face image is obtained, the aligned face image is cut to obtain a face image with an aligned face and a fixed size, for example, a face image with a size of 112×112, that is, a first face sample image, may be obtained.
Further, after a plurality of first face sample images are obtained in the above manner, the following step 102 will be performed.
Step 102: and extracting the image characteristics of each first face sample image, and determining the quality score of each first face sample image based on the image characteristics of each first face sample image.
The quality score is a quantization score for evaluating the quality of each face sample image.
Specifically, after a plurality of first face sample images are obtained in step 101, image feature extraction may be performed on the first face sample images based on the initial face recognition model, so as to obtain image features of each first face sample image.
The pictures of the celebrities can be collected on the public website in a crawler mode, and in order to improve the accuracy of the initial face recognition model, the pictures of each celebrity can be collected by more than a preset number, wherein the preset number can be 5. After gathering pictures of celebrities, an initial face recognition model may be trained using convolutional neural networks. In order to enable the initial face recognition model to have certain recognition capability in various scenes, training can be performed through other classification loss functions such as cosface or arcface, of course, in practical application, training can also be performed through other loss functions, and specific modes of the loss functions are not limited in this embodiment.
Further, after the image features of each first face sample image are obtained, the quality score of each first face sample image may be obtained by using the image feature similarity between the plurality of first face sample images.
Step 103: based on the label information and the quality fraction of each first face sample image, training the initial face recognition model to obtain a face recognition model, wherein the label information is used for representing the user information corresponding to the first face sample image.
The label information of each first face sample image can be used for representing user information, and can be marked manually or through a pre-trained network model.
For example, an initial face recognition model may be trained by using the disclosed data set, and then, the quality score of each first face sample image calculated in step 102 is used to clean the label information of each first face sample image, the cleaned label information is used as label information for supervised learning, and the quality score is used as a pseudo label, so that the initial face recognition model is subjected to supervised training, and the face recognition model is obtained.
Further, the image features and the quality scores of each first face sample image in step 102 may be updated in an iterative manner, so that the model has good robustness to both noise of tag information in the training set and the quality of the first face sample image, thereby obtaining a more accurate face recognition model.
According to the training method of the face recognition model, image characteristics of each first face sample image are extracted by extracting the image characteristics of the plurality of acquired first face sample images, and the quality score of each first face sample image is determined based on the image characteristics of the first face sample image. On the basis, training an initial face recognition model through label information and quality scores corresponding to each first face sample image to obtain the face recognition model. Because the quality score can be used as a pseudo tag when the face recognition model is trained, and the network parameters of the face recognition model can be adjusted together with the tag information, the problem of poor model generalization caused by training a sample image with good image quality and a sample image with poor image quality in the same mode can be avoided because the image quality of each first face sample image is considered, the accuracy and the robustness of the face recognition model training are improved, a more accurate face recognition model is obtained, and the accuracy of face recognition is improved.
In one possible implementation, when determining the quality score of each first face sample image based on the image features of each first face sample image in step 102, the determination may be performed by: for each first face sample image, determining a positive sample set and a negative sample set of the first face sample image, wherein a user corresponding to the positive sample image in the positive sample set is the same as a user corresponding to the first face sample image, and a user corresponding to the negative sample image in the negative sample set is different from a user corresponding to the first face sample image; based on the image characteristics of each first face sample image, respectively determining a first distance between the first face sample image and each positive sample image in the positive sample set and a second distance between the first face sample image and each negative sample image in the negative sample set; a quality score of the first face sample image is determined based on the first distance and the second distance.
In order to improve the reliability of the quality scores of the first face sample images, after a plurality of first face sample images are acquired, each first face sample image may be traversed first to pick out face sample images corresponding to the same user to form a positive sample set, where the number of face sample images corresponding to the same user is greater than a first preset value, for example, the number is at least 5, and in addition, face sample images corresponding to different users need to be picked out to form a negative sample set, and the number of face sample images corresponding to different users is greater than a second preset value, for example, the number is at least 10. For example, each face sample image may be classified and marked or stored in advance, so as to increase the determination speed of the positive sample set and the negative sample set corresponding to each first face sample image.
Fig. 3 is a schematic diagram illustrating determination of quality scores in the training method of the face recognition model according to the embodiment of the present invention, as shown in fig. 3, after determining a positive sample set and a negative sample set of each first face sample image, image features of all face sample images may be extracted through an initial face recognition model to obtain image features x of each face sample image i ∈{x 1 ,x 2 ,…,x m1+m2+1 Image features of each positive sample image in the positive sample setImage features of each negative image in a negative set
Then, using the obtained image features to calculate the cosine distance between the image features of each face sample image and the image features of each positive sample image in the positive sample set, namely the first distanceCalculating cosine distance between image features of each face sample image and image features of each negative sample image in the negative sample set, namely second distance +.>Substituting the first distance and the second distance into the following formula (1) to calculate the mass fraction of each face sample image>
Wherein, the liquid crystal display device comprises a liquid crystal display device,represents Wasserstein distance, < >>Representing the edge distribution as +.>Is a set of all joint distributions gamma, +. >Representation->And->European distance between->Indicating that under the joint distribution gamma +_>For->The desired value of the distance inf indicates the desired +.>Taking the lower bound.
In this embodiment, for each first face sample image, a positive sample set corresponding to the same user and a negative sample set corresponding to different users are determined, and based on image features of each first face sample image, a first distance between each positive sample image in the first face sample image and the positive sample set and a second distance between each negative sample image in the first face sample image and the negative sample set are obtained; and then, the quality score of each first face sample image is obtained through the first distance and the second distance, and reference information is provided for quality evaluation of subsequent face sample images and cleaning of label information, so that the trained face recognition model is more accurate, and the accuracy of face recognition is improved.
On the basis of any one of the above embodiments, when training the initial face recognition model based on the label information and the quality score of each first face sample image to obtain the face recognition model, the following manner may be further performed: removing the first face sample image with the quality score smaller than a preset value based on the quality score of each first face sample image to obtain a removed first face sample image; and training the initial face recognition model based on the label information and the quality score of the removed first face sample image to obtain the face recognition model.
In order to prevent too many face sample images from being removed, which indirectly results in the model training set not being used or the number of sample images being small, the preset value may be set to a small value, such as 0.1.
Specifically, after the quality score of each first face sample image is obtained, the quality score may be normalized, and the first face sample image with too low quality in the same face sample image is cleaned according to the quality score, and the label information of the cleaned first face sample image is used as final label information. In practical application, sample images of the same user can be placed in one folder, so that the number of the folders is the number of the users, and similar pictures in each folder are cleaned based on the mass fraction, wherein if the mass fraction is smaller than a preset value, the first face sample image is considered to be poor in quality and unsuitable for being used as a training sample of a face recognition model, and therefore the first face sample image after being removed is removed.
And then training the initial face recognition model by using the label information and the quality fraction of the removed and cleaned first face sample image to obtain a more accurate face recognition model.
In this embodiment, the quality score of each first face sample image is compared with a preset value to reject the first face sample image with poor quality, and the label information and the quality score of the rejected first face sample image are used to train the initial face recognition model, so as to avoid the negative influence of the low-quality image on the face recognition model, thereby improving the accuracy of the face recognition model.
Fig. 4 is a second flow chart of the training method of the face recognition model according to the embodiment of the present invention, as shown in fig. 4, based on the label information and the quality score of each first face sample image in the step 103, the initial face recognition model is trained to obtain the face recognition model, which may be implemented in the following manner: inputting each first face sample image into an initial face recognition model, and determining a recognition probability value corresponding to each first face sample image based on tag information and characteristic information extracted through the initial face recognition model; determining a new recognition probability value corresponding to each first face sample image based on the recognition probability value and the quality score; based on the new recognition probability value, training the initial face recognition model to obtain the face recognition model.
Specifically, after the feature information of each first face sample image is extracted after each first face sample image is input into the initial face recognition model, the initial face recognition model may be trained based on the feature information and the tag information of each first face sample image, so as to obtain, at the classification layer (Fully Connected Layers, FC), a recognition probability value corresponding to each first face sample image.
In order to reduce the negative influence of the low-quality face sample image on model training, after the recognition probability value corresponding to each first face sample image is obtained, the recognition probability value and the quality score can be combined, for example, the recognition probability value and the quality score are multiplied to obtain a new recognition probability value, and then the new recognition probability value is sent to a loss function layer (Loss Function Layers, LF) to train the initial face recognition model so as to obtain a more accurate face recognition model.
Further, a new face recognition model may be applied in step 102 to update the quality score and the label information of each face sample image, and the trained face recognition model is repeatedly updated in an iterative manner, so as to train a face recognition model that has good recognition performance in a specific scene and is more robust to the image quality.
In this embodiment, the recognition probability value corresponding to each first face sample image is obtained through the tag information and the feature information extracted by the initial face recognition model, and a new recognition probability value is obtained by calculation based on the recognition probability value and the quality score, that is, adverse effects of image quality on the model are also considered, so that the model is trained based on the new recognition probability value, thereby reducing negative effects of low-quality face sample images on the face recognition model, and improving accuracy of the face recognition model.
On the basis of any one of the above embodiments, in order to improve the model training efficiency, the initial face recognition model may be trained by using the collected images in advance, so that in a specific application scenario, the model parameters of the initial face recognition model that is trained in advance may be fine-tuned by using the face images collected in the scenario. The initial face recognition model can be obtained by the following steps: acquiring a plurality of second face sample images, wherein the image quality of the second face sample images is greater than that of the first face sample images; pre-training the first face recognition model based on the second face sample image to obtain a second face recognition model; determining a first face sample image belonging to the same user; and fine-tuning model parameters of the second face recognition model based on the first face sample images, belonging to the same user, of which the number is larger than a preset value, so as to obtain an initial face recognition model.
The second face sample image is a face image collected on the public website by a crawler method, and in order to improve accuracy of model training, at least more than 5 face images of each user can be collected. Wherein the image quality of the second face sample image is greater than the first face sample image.
Specifically, after a plurality of second face sample images are obtained, the first face recognition model may be pre-trained through a convolutional neural network to obtain a second face recognition model. In order to obtain a more accurate initial face recognition model, the first face sample images can be classified according to the number of face sample images corresponding to the same user, and the number of the face sample images is more than or equal to a preset value, namely, a normal training set of the model is regarded as a normal class, and conversely, the face sample images are regarded as a weak correlation class; the preset value may be 5 or 10.
On the basis, under the condition that a normal training set is obtained through classification, the model parameters of the second face recognition model are finely adjusted, so that an initial face recognition model which is higher in performance and higher in recognition accuracy than preset accuracy is obtained.
In this embodiment, the first face recognition model is pre-trained by acquiring a plurality of second face sample images with better image quality, so as to obtain a second face recognition model. On the basis, the model parameters of the second face recognition model are further finely adjusted through the first face sample images of the same user, the number of which is larger than the preset value, so that the acquired initial face recognition model is more accurate, a foundation is laid for subsequently acquiring the more accurate face recognition model, and therefore, not only can the domain migration model with obvious performance improvement in a specific scene be obtained, but also the training efficiency of the face recognition model can be improved.
Further, the embodiment of the invention further provides a face recognition method, and fig. 5 is a schematic flow chart of the face recognition method provided by the embodiment of the invention, as shown in fig. 5, the method includes:
step 501: and acquiring a face image to be identified.
After the initial face image to be identified is acquired, the face key point detection and clipping can be performed on the initial face image to be identified by adopting the preprocessing mode as shown in fig. 2, so that the face image to be identified can be obtained.
Step 502: and inputting the face image to be recognized into a face recognition model to obtain a face recognition result of the face image to be recognized, wherein the face recognition model is obtained by training the face recognition model training method in any mode.
In the embodiment, the face image to be recognized is input to a more accurate face recognition model, so that the accuracy of face recognition is improved. The above face recognition model is obtained by training the face recognition model training method according to any one of the above embodiments, and specific details may be referred to the description in the foregoing embodiments, which is not repeated here.
The following describes the training device of the face recognition model provided by the invention, and the training device of the face recognition model described below and the training method of the face recognition model described above can be referred to correspondingly.
Fig. 6 is a schematic structural diagram of a training device for a face recognition model according to an embodiment of the present invention, as shown in fig. 6, where the device includes:
an acquiring module 610, configured to acquire a plurality of first face sample images;
an extracting module 620, configured to extract image features of each first face sample image;
a determining module 630, configured to determine a quality score of each first face sample image based on an image feature of each first face sample image;
the training module 640 is configured to train the initial face recognition model based on the tag information and the quality score of each first face sample image, so as to obtain a face recognition model, where the tag information is used to represent user information corresponding to the first face sample image.
According to the training device for the face recognition model, firstly, a plurality of first face sample images are acquired through an acquisition module 610; then, extracting image features of the acquired plurality of first face sample images through an extracting module 620 to extract image features of each first face sample image, and determining a quality score of each first face sample image through a determining model 630 based on the image features of the first face sample image; based on the above, the training module 640 is used for training the initial face recognition model by combining the label information and the quality score corresponding to each first face sample image to obtain the face recognition model. Because the quality score can be used as a pseudo tag when the face recognition model is trained, and the network parameters of the face recognition model can be adjusted together with the tag information, the problem of poor model generalization caused by training a sample image with good image quality and a sample image with poor image quality in the same mode can be avoided because the image quality of each first face sample image is considered, the accuracy and the robustness of the face recognition model training are improved, a more accurate face recognition model is obtained, and the accuracy of face recognition is improved.
Optionally, the determining module 630 is specifically configured to:
for each first face sample image, determining a positive sample set and a negative sample set of the first face sample image, wherein a user corresponding to the positive sample image in the positive sample set is the same as a user corresponding to the first face sample image, and a user corresponding to the negative sample image in the negative sample set is different from a user corresponding to the first face sample image;
based on the image characteristics of each first face sample image, respectively determining a first distance between the first face sample image and each positive sample image in the positive sample set and a second distance between the first face sample image and each negative sample image in the negative sample set;
a quality score of the first face sample image is determined based on the first distance and the second distance.
Optionally, the training module 640 is specifically configured to:
inputting each first face sample image into an initial face recognition model, and determining a recognition probability value corresponding to each first face sample image based on tag information and characteristic information extracted through the initial face recognition model;
determining a new recognition probability value corresponding to each first face sample image based on the recognition probability value and the quality score;
Based on the new recognition probability value, training the initial face recognition model to obtain the face recognition model.
Optionally, the training module 640 is specifically configured to:
removing the first face sample image with the quality score smaller than a preset value based on the quality score of each first face sample image to obtain a removed first face sample image;
and training the initial face recognition model based on the label information and the quality score of the removed first face sample image to obtain the face recognition model.
Optionally, the apparatus further comprises:
the acquisition module is used for acquiring a plurality of second face sample images, and the image quality of the second face sample images is greater than that of the first face sample images;
the pre-training module is used for pre-training the first face recognition model based on the second face sample image to obtain a second face recognition model;
the determining module is also used for determining a first face sample image belonging to the same user;
and the fine tuning module is used for fine tuning the model parameters of the second face recognition model based on the first face sample images, the number of which is larger than a preset value, of the first face sample images belonging to the same user, so as to obtain an initial face recognition model.
The apparatus of this embodiment may be used to execute the method of any one of the training apparatus side method embodiments of the face recognition model, and its specific implementation process and technical effects are similar to those of the training apparatus side method embodiment of the face recognition model, and specific reference may be made to the detailed description of the training apparatus side method embodiment of the face recognition model, which is not repeated herein.
The following describes a face recognition device provided by the present invention, and the face recognition device described below and the face recognition method described above can be referred to correspondingly.
Further, fig. 7 is a schematic structural diagram of a face recognition device according to an embodiment of the present invention, as shown in fig. 7, where the device includes:
an acquisition module 710, configured to acquire a face image to be identified;
the recognition module 720 is configured to input the face image to be recognized into a face recognition model to obtain a face recognition result of the face image to be recognized, where the face recognition model is obtained by training based on the training method of the face recognition model described in any one of the embodiments.
In this embodiment, the obtaining module 710 obtains the face image to be identified, and inputs the face image to be identified to a more accurate face recognition model, so as to identify the face image to be identified by the identifying module 720, and obtain a more accurate face recognition result of the face image to be identified, thereby improving the accuracy of face recognition.
The apparatus of this embodiment may be used to execute the method of any one of the embodiments of the face recognition apparatus side method, and its specific implementation process and technical effects are similar to those of the face recognition apparatus side method embodiment, and specific reference may be made to the detailed description of the face recognition apparatus side method embodiment, which is not repeated herein.
Fig. 8 illustrates a physical structure diagram of an electronic device, as shown in fig. 8, which may include: processor 810, communication interface (Communications Interface) 820, memory 830, and communication bus 840, wherein processor 810, communication interface 820, memory 830 accomplish communication with each other through communication bus 840. The processor 810 may invoke logic instructions in the memory 830 to perform a training method for a face recognition model, the method comprising: acquiring a plurality of first face sample images; extracting image characteristics of each first face sample image, and determining the quality score of each first face sample image based on the image characteristics of each first face sample image; based on the label information and the quality fraction of each first face sample image, training the initial face recognition model to obtain a face recognition model, wherein the label information is used for representing the user information corresponding to the first face sample image.
Further, the logic instructions in the memory 830 described above may be implemented in the form of software functional units and may be stored in a computer-readable storage medium when sold or used as a stand-alone product. Based on this understanding, the technical solution of the present invention may be embodied essentially or in a part contributing to the prior art or in a part of the technical solution, in the form of a software product stored in a storage medium, comprising several instructions for causing a computer device (which may be a personal computer, a server, a network device, etc.) to perform all or part of the steps of the method according to the embodiments of the present invention. And the aforementioned storage medium includes: a U-disk, a removable hard disk, a Read-Only Memory (ROM), a random access Memory (RAM, random Access Memory), a magnetic disk, or an optical disk, or other various media capable of storing program codes.
In another aspect, the present invention also provides a computer program product, where the computer program product includes a computer program, where the computer program can be stored on a computer readable storage medium, where the computer program when executed by a processor can perform a training method for a face recognition model provided by the above methods, and the method includes: acquiring a plurality of first face sample images; extracting image characteristics of each first face sample image, and determining the quality score of each first face sample image based on the image characteristics of each first face sample image; based on the label information and the quality fraction of each first face sample image, training the initial face recognition model to obtain a face recognition model, wherein the label information is used for representing the user information corresponding to the first face sample image.
In still another aspect, the present invention further provides a computer readable storage medium having stored thereon a computer program which, when executed by a processor, is implemented to perform a method of training a face recognition model provided by the above methods, the method comprising: acquiring a plurality of first face sample images; extracting image characteristics of each first face sample image, and determining the quality score of each first face sample image based on the image characteristics of each first face sample image; based on the label information and the quality fraction of each first face sample image, training the initial face recognition model to obtain a face recognition model, wherein the label information is used for representing the user information corresponding to the first face sample image.
The apparatus embodiments described above are merely illustrative, wherein the elements illustrated as separate elements may or may not be physically separate, and the elements shown as elements may or may not be physical elements, may be located in one place, or may be distributed over a plurality of network elements. Some or all of the modules may be selected according to actual needs to achieve the purpose of the solution of this embodiment. Those of ordinary skill in the art will understand and implement the present invention without undue burden.
From the above description of the embodiments, it will be apparent to those skilled in the art that the embodiments may be implemented by means of software plus necessary general hardware platforms, or of course may be implemented by means of hardware. Based on this understanding, the foregoing technical solution may be embodied essentially or in a part contributing to the prior art in the form of a software product, which may be stored in a computer readable storage medium, such as ROM/RAM, a magnetic disk, an optical disk, etc., including several instructions for causing a computer device (which may be a personal computer, a server, or a network device, etc.) to execute the method described in the respective embodiments or some parts of the embodiments.
Finally, it should be noted that: the above embodiments are only for illustrating the technical solution of the present invention, and are not limiting; although the invention has been described in detail with reference to the foregoing embodiments, it will be understood by those of ordinary skill in the art that: the technical scheme described in the foregoing embodiments can be modified or some technical features thereof can be replaced by equivalents; such modifications and substitutions do not depart from the spirit and scope of the technical solutions of the embodiments of the present invention.
Claims (9)
1. A method for training a face recognition model, comprising:
acquiring a plurality of first face sample images;
extracting image characteristics of each first face sample image, and determining the quality score of each first face sample image based on the image characteristics of each first face sample image;
training an initial face recognition model based on label information of each first face sample image and the quality score to obtain a face recognition model, wherein the label information is used for representing user information corresponding to the first face sample image;
the training of the initial face recognition model based on the label information and the quality score of each first face sample image to obtain a face recognition model comprises the following steps:
inputting each first face sample image into the initial face recognition model, and determining a recognition probability value corresponding to each first face sample image based on the tag information and the feature information extracted by the initial face recognition model;
determining a new recognition probability value corresponding to each first face sample image based on the recognition probability value and the quality score;
And training an initial face recognition model based on the new recognition probability value to obtain the face recognition model.
2. The method for training a face recognition model according to claim 1, wherein determining the quality score of each first face sample image based on the image features of each first face sample image comprises:
for each first face sample image, determining a positive sample set and a negative sample set of the first face sample image, wherein a user corresponding to the positive sample image in the positive sample set is the same as a user corresponding to the first face sample image, and a user corresponding to the negative sample image in the negative sample set is different from the user corresponding to the first face sample image;
based on the image characteristics of each first face sample image, respectively determining a first distance between the first face sample image and each positive sample image in the positive sample set and a second distance between the first face sample image and each negative sample image in the negative sample set;
a quality score of the first face sample image is determined based on the first distance and the second distance.
3. The method for training a face recognition model according to any one of claims 1 or 2, wherein training the initial face recognition model based on the label information and the quality score of each of the first face sample images to obtain the face recognition model comprises:
removing the first face sample image with the quality score smaller than a preset value based on the quality score of each first face sample image to obtain a removed first face sample image;
and training the initial face recognition model based on the label information and the quality score of the removed first face sample image to obtain the face recognition model.
4. A method of training a face recognition model according to any one of claims 1 or 2, further comprising:
acquiring a plurality of second face sample images, wherein the image quality of the second face sample images is greater than that of the first face sample images;
pre-training the first face recognition model based on the second face sample image to obtain a second face recognition model;
determining a first face sample image belonging to the same user;
and fine-tuning model parameters of the second face recognition model based on the first face sample images, belonging to the same user, of which the number is larger than a preset value, so as to obtain the initial face recognition model.
5. A face recognition method, comprising:
acquiring a face image to be identified;
inputting the face image to be recognized into a face recognition model to obtain a face recognition result of the face image to be recognized, wherein the face recognition model is obtained by training according to the training method of the face recognition model of any one of claims 1-4.
6. A training device for a face recognition model, comprising:
the acquisition module is used for acquiring a plurality of first face sample images;
the extraction module is used for extracting the image characteristics of each first face sample image;
the determining module is used for determining the quality score of each first face sample image based on the image characteristics of each first face sample image;
the training module is used for training the initial face recognition model based on the label information and the quality score of each first face sample image to obtain a face recognition model, and the label information is used for representing the user information corresponding to the first face sample image;
the training module is specifically configured to:
inputting each first face sample image into the initial face recognition model, and determining a recognition probability value corresponding to each first face sample image based on the tag information and the feature information extracted by the initial face recognition model;
Determining a new recognition probability value corresponding to each first face sample image based on the recognition probability value and the quality score;
and training an initial face recognition model based on the new recognition probability value to obtain the face recognition model.
7. A face recognition device, comprising:
the acquisition module is used for acquiring the face image to be identified;
the recognition module is used for inputting the face image to be recognized into a face recognition model to obtain a face recognition result of the face image to be recognized, and the face recognition model is obtained by training according to the training method of the face recognition model of any one of claims 1-4.
8. An electronic device comprising a memory, a processor and a computer program stored on the memory and executable on the processor, characterized in that the processor implements a training method of a face recognition model according to any one of claims 1 to 4 and implements a face recognition method according to claim 5 when the program is executed by the processor.
9. A computer readable storage medium, on which a computer program is stored, characterized in that the computer program, when being executed by a processor, implements a training method of a face recognition model according to any one of claims 1 to 4, and implements a face recognition method according to claim 5.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202211702258.7A CN115953819B (en) | 2022-12-28 | 2022-12-28 | Training method, device, equipment and storage medium of face recognition model |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202211702258.7A CN115953819B (en) | 2022-12-28 | 2022-12-28 | Training method, device, equipment and storage medium of face recognition model |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN115953819A CN115953819A (en) | 2023-04-11 |
| CN115953819B true CN115953819B (en) | 2023-08-15 |
Family
ID=87288785
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202211702258.7A Active CN115953819B (en) | 2022-12-28 | 2022-12-28 | Training method, device, equipment and storage medium of face recognition model |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN115953819B (en) |
Citations (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN108960087A (en) * | 2018-06-20 | 2018-12-07 | 中国科学院重庆绿色智能技术研究院 | A kind of quality of human face image appraisal procedure and system based on various dimensions evaluation criteria |
| CN111738243A (en) * | 2020-08-25 | 2020-10-02 | 腾讯科技（深圳）有限公司 | Method, device and equipment for selecting face image and storage medium |
| CN112381782A (en) * | 2020-11-11 | 2021-02-19 | 腾讯科技（深圳）有限公司 | Human face image quality evaluation method and device, computer equipment and storage medium |
| CN113139462A (en) * | 2021-04-23 | 2021-07-20 | 杭州魔点科技有限公司 | Unsupervised face image quality evaluation method, electronic device and storage medium |
| CN113449704A (en) * | 2021-08-31 | 2021-09-28 | 北京的卢深视科技有限公司 | Face recognition model training method and device, electronic equipment and storage medium |
| CN114399491A (en) * | 2022-01-13 | 2022-04-26 | 云从科技集团股份有限公司 | Human face image quality evaluation method and device and computer readable storage medium |
| CN115496715A (en) * | 2022-09-02 | 2022-12-20 | 盛视科技股份有限公司 | Face image quality scoring method and face image quality evaluation model obtaining method |
-
2022
- 2022-12-28 CN CN202211702258.7A patent/CN115953819B/en active Active
Patent Citations (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN108960087A (en) * | 2018-06-20 | 2018-12-07 | 中国科学院重庆绿色智能技术研究院 | A kind of quality of human face image appraisal procedure and system based on various dimensions evaluation criteria |
| CN111738243A (en) * | 2020-08-25 | 2020-10-02 | 腾讯科技（深圳）有限公司 | Method, device and equipment for selecting face image and storage medium |
| CN112381782A (en) * | 2020-11-11 | 2021-02-19 | 腾讯科技（深圳）有限公司 | Human face image quality evaluation method and device, computer equipment and storage medium |
| CN113139462A (en) * | 2021-04-23 | 2021-07-20 | 杭州魔点科技有限公司 | Unsupervised face image quality evaluation method, electronic device and storage medium |
| CN113449704A (en) * | 2021-08-31 | 2021-09-28 | 北京的卢深视科技有限公司 | Face recognition model training method and device, electronic equipment and storage medium |
| CN114399491A (en) * | 2022-01-13 | 2022-04-26 | 云从科技集团股份有限公司 | Human face image quality evaluation method and device and computer readable storage medium |
| CN115496715A (en) * | 2022-09-02 | 2022-12-20 | 盛视科技股份有限公司 | Face image quality scoring method and face image quality evaluation model obtaining method |
Also Published As
| Publication number | Publication date |
|---|---|
| CN115953819A (en) | 2023-04-11 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN111488756B (en) | Face recognition-based living body detection method, electronic device, and storage medium | |
| CN111626371B (en) | Image classification method, device, equipment and readable storage medium | |
| CN102945366B (en) | A kind of method and device of recognition of face | |
| CN104992148A (en) | ATM terminal human face key points partially shielding detection method based on random forest | |
| CN105160318A (en) | Facial expression based lie detection method and system | |
| EP3644232B1 (en) | Method and device for classifying samples to be assessed | |
| CN113221086B (en) | Offline face authentication method and device, electronic equipment and storage medium | |
| CN109635625B (en) | Intelligent identity verification method, equipment, storage medium and device | |
| US20190205589A1 (en) | Latent fingerprint ridge flow map improvement | |
| CN112215780B (en) | Image evidence obtaining and resistance attack defending method based on class feature restoration fusion | |
| CN112613471A (en) | Face living body detection method and device and computer readable storage medium | |
| KR20100116404A (en) | Method and apparatus of dividing separated cell and grouped cell from image | |
| CN111259894B (en) | Certificate information identification method and device and computer equipment | |
| CN115953819B (en) | Training method, device, equipment and storage medium of face recognition model | |
| CN111163332A (en) | Video pornography detection method, terminal and medium | |
| CN112749605A (en) | Identity recognition method, system and equipment | |
| CN116206373A (en) | Living body detection method, electronic device and storage medium | |
| US20220405527A1 (en) | Target Detection Methods, Apparatuses, Electronic Devices and Computer-Readable Storage Media | |
| KR102342495B1 (en) | Method and Apparatus for Creating Labeling Model with Data Programming | |
| CN115565097A (en) | Method and device for detecting compliance of personnel behaviors in transaction scene | |
| WO2022263904A1 (en) | Target detection methods, apparatuses, electronic devices and computer-readable storage media | |
| CN114005184A (en) | Handwritten signature authenticity identification method and device based on small amount of samples | |
| CN113255472B (en) | Face quality evaluation method and system based on random embedding stability | |
| CN116259091B (en) | Method and device for detecting silent living body | |
| CN113887283A (en) | Strange face recognition score threshold determination method, face recognition method and system |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |