CN114926887A - Face recognition method and device and terminal equipment - Google Patents
Face recognition method and device and terminal equipment Download PDFInfo
- Publication number
- CN114926887A CN114926887A CN202210601012.4A CN202210601012A CN114926887A CN 114926887 A CN114926887 A CN 114926887A CN 202210601012 A CN202210601012 A CN 202210601012A CN 114926887 A CN114926887 A CN 114926887A
- Authority
- CN
- China
- Prior art keywords
- face
- image
- front face
- recognized
- head
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V40/00—Recognition of biometric, human-related or animal-related patterns in image or video data
- G06V40/10—Human or animal bodies, e.g. vehicle occupants or pedestrians; Body parts, e.g. hands
- G06V40/16—Human faces, e.g. facial parts, sketches or expressions
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
- G06N3/045—Combinations of networks
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/08—Learning methods
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/20—Image preprocessing
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
- G06V10/74—Image or video pattern matching; Proximity measures in feature spaces
- G06V10/761—Proximity, similarity or dissimilarity measures
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
- G06V10/77—Processing image or video features in feature spaces; using data integration or data reduction, e.g. principal component analysis [PCA] or independent component analysis [ICA] or self-organising maps [SOM]; Blind source separation
- G06V10/774—Generating sets of training patterns; Bootstrap methods, e.g. bagging or boosting
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
- G06V10/82—Arrangements for image or video recognition or understanding using pattern recognition or machine learning using neural networks
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V40/00—Recognition of biometric, human-related or animal-related patterns in image or video data
- G06V40/10—Human or animal bodies, e.g. vehicle occupants or pedestrians; Body parts, e.g. hands
- G06V40/16—Human faces, e.g. facial parts, sketches or expressions
- G06V40/168—Feature extraction; Face representation
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- Health & Medical Sciences (AREA)
- General Physics & Mathematics (AREA)
- General Health & Medical Sciences (AREA)
- Multimedia (AREA)
- Evolutionary Computation (AREA)
- Computing Systems (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Artificial Intelligence (AREA)
- Software Systems (AREA)
- Oral & Maxillofacial Surgery (AREA)
- Databases & Information Systems (AREA)
- Medical Informatics (AREA)
- General Engineering & Computer Science (AREA)
- Molecular Biology (AREA)
- Mathematical Physics (AREA)
- Data Mining & Analysis (AREA)
- Computational Linguistics (AREA)
- Biophysics (AREA)
- Human Computer Interaction (AREA)
- Biomedical Technology (AREA)
- Life Sciences & Earth Sciences (AREA)
- Image Analysis (AREA)
Abstract
The application is applicable to the technical field of face recognition, and provides a face recognition method, a face recognition device and terminal equipment, wherein the face recognition method comprises the following steps: inputting a face image to be recognized into a front face reconstruction model to obtain front face information capable of representing face features, wherein the front face reconstruction model is a pre-trained neural network model, and performing face recognition based on the front face information to obtain a face recognition result. The method and the device can improve the flexibility of face recognition.
Description
Technical Field
The present application belongs to the field of face recognition technology, and in particular, to a face recognition method, apparatus, terminal device, and computer-readable storage medium.
Background
The face recognition is a biological recognition technology for carrying out identity recognition based on face feature information of people, mainly uses a camera to collect images or video streams containing faces, automatically detects and tracks the faces in the images, and further carries out face recognition on the detected faces.
With the development of science and technology, face recognition has been widely applied to work and life, such as entrance guard machines, attendance machines and the like. In order to realize automatic attendance of attendance statistics of partial staff, enterprises can generally install cameras at corresponding posts to collect face information of the staff, compare the face information with staff system information, and perform face recognition, so that the statistics of attendance of the corresponding staff is realized. Among the prior art, the front face image that gathers corresponding staff at staff post place the place ahead single camera of installation usually carries out face identification to realize the statistics of automatically going out on duty, because can only carry out face identification to the front face, and the camera position is fixed, consequently, when carrying out corresponding staff's the statistics of automatically going out on duty, require the staff to openly face towards the camera, and is great to staff's restriction, influences staff's normal work.
Disclosure of Invention
The embodiment of the application provides a face recognition method, a face recognition device and terminal equipment, and the flexibility of face recognition in the prior art can be improved.
In a first aspect, an embodiment of the present application provides a face recognition method, including:
inputting a face image to be recognized into a front face reconstruction model to obtain front face information capable of representing face features, wherein the front face reconstruction model is a pre-trained neural network model;
and carrying out face recognition based on the front face information to obtain a face recognition result.
In a second aspect, an embodiment of the present application provides a face recognition apparatus, including:
the face reconstruction module is used for inputting a face image to be recognized into a front face reconstruction model to obtain front face information capable of representing face features, and the front face reconstruction model is a pre-trained neural network model;
and the face recognition module is used for carrying out face recognition based on the front face information to obtain a face recognition result.
In a third aspect, an embodiment of the present application provides a terminal device, which includes a memory, a processor, and a computer program that is stored in the memory and is executable on the processor, and when the processor executes the computer program, the face recognition method according to the first aspect is implemented.
In a fourth aspect, the present application provides a computer-readable storage medium, where a computer program is stored, and the computer program, when executed by a processor, implements the steps of the face recognition method in the first aspect.
In a fifth aspect, an embodiment of the present application provides a computer program product, which, when run on a terminal device, causes the terminal device to execute the face recognition method according to any one of the above first aspects.
Compared with the prior art, the embodiment of the application has the advantages that: the face image to be recognized is converted into the front face information capable of representing the face characteristics by adopting the pre-trained front face reconstruction model, and then the face recognition is carried out according to the front face information to obtain the face recognition result of the face image to be recognized before the conversion.
Drawings
In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings used in the embodiments or the description of the prior art will be briefly described below.
Fig. 1 is a schematic flowchart of a face recognition method according to an embodiment of the present application;
FIG. 2 is a schematic diagram of a head pose provided by an embodiment of the present application;
fig. 3 is a schematic structural diagram of a front face reconstruction network according to an embodiment of the present application;
fig. 4 is a schematic structural diagram of a face recognition apparatus according to an embodiment of the present application;
fig. 5 is a schematic structural diagram of a terminal device according to an embodiment of the present application.
Detailed Description
In the following description, for purposes of explanation and not limitation, specific details are set forth, such as particular system structures, techniques, etc. in order to provide a thorough understanding of the embodiments of the present application. It will be apparent, however, to one skilled in the art that the present application may be practiced in other embodiments that depart from these specific details. In other instances, detailed descriptions of well-known systems, devices, circuits, and methods are omitted so as not to obscure the description of the present application with unnecessary detail.
It will be understood that the terms "comprises" and/or "comprising," when used in this specification and the appended claims, specify the presence of stated features, integers, steps, operations, elements, and/or components, but do not preclude the presence or addition of one or more other features, integers, steps, operations, elements, components, and/or groups thereof.
It should also be understood that the term "and/or" as used in this specification and the appended claims refers to any and all possible combinations of one or more of the associated listed items and includes such combinations.
Furthermore, in the description of the present application and the appended claims, the terms "first," "second," "third," and the like are used for distinguishing between descriptions and not necessarily for describing or implying relative importance.
Reference throughout this specification to "one embodiment" or "some embodiments," or the like, means that a particular feature, structure, or characteristic described in connection with the embodiment is included in one or more embodiments of the present application. Thus, appearances of the phrases "in one embodiment," "in some embodiments," "in other embodiments," or the like, in various places throughout this specification are not necessarily all referring to the same embodiment, but rather "one or more but not all embodiments" unless specifically stated otherwise.
The first embodiment is as follows:
fig. 1 shows a schematic flow chart of a face recognition method provided in an embodiment of the present invention, which is detailed as follows:
step S101, inputting a face image to be recognized into a front face reconstruction model to obtain front face information capable of representing face characteristics;
the front face reconstruction model is a pre-trained neural network model. Optionally, the front face information may be a feature vector of the front face image, or may be the front face image or other information that can describe features of the front face.
In the embodiment of the application, the face image to be recognized is converted into the uniform face information capable of representing the face characteristics through the pre-trained face reconstruction model, so that face recognition is subsequently performed according to the face information, the limitation that the face image needs to be collected when the face image to be recognized is collected is avoided, and the flexibility of face recognition is improved.
And S102, carrying out face recognition based on the front face information to obtain a face recognition result.
Specifically, step S102 can be expressed as: and matching the front face information obtained in the step S101 with the pre-input standard front face information to obtain a face recognition result.
Optionally, when acquiring the front face image of the user to obtain the standard front face information of the user, based on head pose estimation or face symmetry, determining whether the front face image meets the rule of the set standard front face image, and storing the face information that can be used as the standard front face image as the standard front face information. For example, a frontal face image with a Pitch angle (Pitch), a Yaw angle (Yaw), and a Roll angle (Roll) all within 0 to 0.5 degrees may be used as a standard frontal face image, and in the collected frontal face image of a certain user, the Pitch angle, the Yaw angle, and the Roll angle of the face are all within 0 to 0.5 degrees, and then the frontal face image may be used as the standard frontal face image of the user, and standard frontal face information of the standard frontal face image is obtained and stored.
The head pose estimation is to acquire the orientation of the head in a three-dimensional space through a human face image, and generally, the human head can be modeled as an entity-free object, and according to the setting, the head is limited to three euler angles of a pitch angle, a yaw angle and a roll angle in the posture, which are shown in fig. 2, and respectively represent the angles of the head up-and-down motion, the head left-and-right motion and the neck lateral bending motion.
Specifically, the matching of the front face information obtained in step S101 with the pre-entered standard front face information includes: and calculating the similarity between the pre-input standard front face information of the user and the front face information. For example, the similarity between the standard front face information and the front face information may be calculated based on an euclidean distance and/or a cosine similarity, the standard front face information having the similarity greater than a preset threshold with the front face information is used as target face information, and the identity information corresponding to the target face information is output as a face recognition result. If the similarity between the face information and the standard face information is not greater than the preset threshold, outputting prompt information such as 'no corresponding identity information exists', indicating that the face information (the face image to be recognized) has no relevant recognition result (namely, the face recognition result is embodied as information such as 'no' or 'matching failure').
In the embodiment of the application, the similarity between the standard front face information of the user and the pre-input front face information is calculated according to the front face information to be recognized, the corresponding identity information of the standard front face information with the similarity meeting the preset threshold is used as the face recognition result, and the obtained standard front face information conforms to the front face information (the face image to be recognized) due to the fact that the standard front face information with the similarity larger than the preset threshold is used as the recognition result, so that the face recognition result has higher accuracy.
It should be noted that, when calculating the similarity between the standard front face information of the pre-entered user and the front face information, there may be a case where the number of standard front face information having a similarity greater than a preset threshold with respect to the front face information is greater than 1, at this time, the standard face information having the greatest similarity with the front face information is used as target face information, the standard front face information having a similarity greater than a preset threshold with respect to the front face information except the target face information is used as candidate face information, the identity information corresponding to the target face information is output as a face recognition result, the identity information corresponding to the candidate face information is output as a candidate result, a corresponding user interface is provided to receive a selection instruction of the user, and an actual face recognition result is determined according to the user selection instruction, the problem that the face recognition result is unclear when the similarity between the standard face information and the face information is larger than a preset threshold value is solved, and therefore the fault tolerance of face recognition is improved.
In the embodiment of the application, the face image to be recognized is converted into the uniform front face information capable of representing the face characteristics of the image to be recognized through the pre-trained front face reconstruction model, the similarity between the pre-input standard front face information of the user and the front face information is calculated according to the front face information, and the corresponding identity information of the standard front face information with the similarity meeting the preset threshold is used as the face recognition result. The front face information obtained by conversion represents the face characteristics of the face image to be recognized, so that face recognition can be directly carried out according to the front face information, the problem that the front face image needs to be collected when the face image to be recognized is collected, or the problem that the error of a face recognition result is large when the collected face image to be recognized is a face image deflected at a certain angle is solved, the limitation in collecting the face image to be recognized is reduced, and the flexibility and the accuracy of face recognition are improved.
In some embodiments, the frontal face reconstruction model may be trained by:
a1, collecting face images of N sample users under a plurality of head gestures to obtain a training set, wherein N is a positive integer greater than 1.
Specifically, N different persons are selected as sample users, face images of the N sample users in the same illumination environment and the like and under a plurality of different head postures are collected respectively to serve as a training set of a front face reconstruction model, and front face reconstruction training is carried out. Wherein N is a positive integer greater than 1.
And A2, acquiring first front face images of the N sample users to obtain a standard image set.
Specifically, the front face reconstruction model is used for reconstructing the front face image of the user according to the face image of the user, so that in the training process, the first front face image of the sample user needs to be acquired as a standard, and the error of the reconstructed front face image in the training process of the front face reconstruction model is calculated, so that the first front face images of the N sample users are acquired as a standard image set.
Optionally, when the first frontal face images of the N sample users are collected, the face images of the sample users when the fronts face the image capturing devices such as the camera are collected, and whether the face images can be used as the first frontal face images is judged based on head pose estimation or face symmetry, and whether the face images can be used as the first frontal face images of the corresponding sample users is judged according to whether pitch angles, yaw angles, and roll angles of the collected face images are within a preset deflection range, for example, if the pitch angles, the yaw angles, and the roll angles of the faces in the collected face images of a certain sample user are all within 0 to 0.5 degrees, the face images can be used as the first frontal face images of the sample users, and the first frontal face images are stored in the standard image set.
And A3, preprocessing the images in the standard image set and the training set respectively.
Specifically, the images in the training set and the standard image set that are acquired may include other irrelevant information, and factors such as illumination and expression may have a certain effect on the facial features in the facial images, so before training the face reconstruction model, the images in the training set and the standard image set are respectively preprocessed. Optionally, the image preprocessing of the training set and the standard image set includes face detection, gray processing, and the like. For example, a pre-trained mtcn (Multi-task convolutional neural network) is used to perform face region detection and face keypoint detection on images in a training set and a standard image set, respectively, output an image containing a face with a preset size, such as an image containing a face with a size of 64 × 64, and perform gray processing on the output face image with the preset size based on average graying or weighted average graying, so as to obtain a pre-processed training set and a standard image set.
The MTCNN is a face detection and face alignment method based on deep learning, and can simultaneously complete the tasks of face detection and face alignment. When carrying out MTCNN training, an original image needs to be zoomed to different scales by using an image pyramid, and then the images with different scales are sent into three sub-networks P-Net (candidate Network), R-Net (refined Network) and O-Net (Output Network) of the MTCNN for training so as to realize face detection and feature point detection of a multi-scale image.
And A4, training the front face reconstruction model based on the preprocessed training set and the standard image set.
Optionally, the model structure of the front face reconstruction model includes:
carrying out convolution processing on the face image to be recognized by adopting convolution cores with different sizes and extracting the features of the face image to be recognized;
the image recognition method comprises the following steps that sequential pooling layers are provided, the preset number of the pooling layers is 1 layer less than the preset number of the convolutional layers, except for the last convolutional layer, the input end of each pooling layer is sequentially connected with the output end of one convolutional layer, and pooling processing is conducted on facial image features to be recognized, which are output by the convolutional layers;
and the input end of the full connection layer is connected with the output end of the last convolution layer, and the facial image characteristics to be recognized output by the convolution layers are combined to obtain a second front facial image, wherein the second front facial image is a corresponding front facial image reconstructed from the facial image to be recognized.
For example, the front face reconstruction model may adopt a structure as shown in fig. 3, where the front face reconstruction model includes an input layer, 3 convolutional layers, 2 pooling layers, and a full connection layer and an output layer, an output end of the first convolutional layer is connected to an input end of the first pooling layer, an output end of the first pooling layer is connected to an input end of the second convolutional layer, and optionally, the pooling layer adopts maximal pooling to better extract face features.
Specifically, the front face reconstruction model is trained according to the preprocessed training set, and a loss function of the front face reconstruction model is calculated according to the standard image set, so that the front face reconstruction model is optimized, and an optimal solution of the front face reconstruction model is obtained. Before the training of the front face reconstruction model is carried out, the face images of part of sample users in the training set can be used as a verification set according to requirements to verify the generalization capability of the front face reconstruction model so as to determine whether to continue the training.
Optionally, an objective function of the front face reconstruction model is calculated according to a second front face image corresponding to the face image of each sample user reconstructed by the front face reconstruction network and the first front face image corresponding to each sample user in the target image set, so as to optimize the front face reconstruction model, and obtain an optimal solution of the front face reconstruction model.
Optionally, the objective function E is:
wherein, the first and the second end of the pipe are connected with each other,
for training set P N The kth human face image of the ith sample user, W is the weight parameter of the front human face reconstruction model,
a second front face image which is reconstructed for the k face image of the ith sample user reconstructed by the front face reconstruction model, Y i F is a norm function for the first frontal face image of the ith sample user in the target image set.
In the embodiment of the application, after the face image and the front face image of the collected sample user in different head postures are preprocessed, the front face reconstruction model is trained based on the obtained training set and standard image set which contain the face and have the same size, and due to the fact that preprocessing such as face detection and graying is carried out on the training set and the step image set, images in the training set are single-channel grayscale images containing the face, unnecessary characteristic information is reduced, and therefore the data processing speed in the training process is increased.
In some embodiments, the step a1 includes:
a11, for any sample user i, acquiring the face image of the sample user i under each head posture based on head posture estimation, and storing the face image into a training set.
Specifically, when the face images of each sample user are collected, the head postures, namely the face angles, of the sample users are calculated through head posture estimation, the face images of the sample users in different head postures are collected respectively according to a preset collection rule, and the face images are stored as training images of the corresponding sample users respectively to obtain a training set.
In the embodiment of the application, the user collecting each sample is carried out based on head posture estimation
In some embodiments, the step a11 includes:
and sequentially acquiring the face images of the head postures of the sample user i in the three head motion states based on head posture estimation.
Optionally, the three head movement states include: head side-to-side movement, head up-and-down movement, and neck side-to-side movement.
Optionally, an instruction input by the user is obtained, when the sample users perform head left-right movement, head up-down movement, and neck lateral flexion movement in sequence according to the user instruction, that is, as shown in fig. 2, the sample users rotate around the Y axis (Yaw), rotate around the X axis (Pitch), and rotate around the Z axis (Roll), and according to a preset acquisition rule, the face images of the sample users during the head movement are acquired as the face images of the sample users in the training set, and are stored in the training set. The user instruction is used for instructing a user to perform corresponding head movement.
For example, the preset acquisition rule is that, during the head movement of the sample user, each movement direction changes by 45 ° at most, and each movement direction changes by 1 °, a face image of the current pose is acquired.
Aiming at any sample user i, according to a first instruction input by a user, enabling the sample user i to slowly rotate the head 45 degrees leftwards and rightwards (around a Y axis), continuously acquiring the face images of the sample user i in the motion process by a camera, estimating and calculating the angle of the yaw angle of the head in the leftward and rightwards motion process through the head posture, and sequentially storing the corresponding face images when the yaw angle changes by 1 degree into an image set P corresponding to the sample user i i Performing the following steps;
after the corresponding face image is acquired according to the first instruction, according to a second instruction input by a user, enabling a sample user i to slowly rotate the head part 45 degrees upwards and downwards (around the X axis) respectively, and continuously acquiring the movement process by the cameraCalculating the angle of a pitch angle of the head in the upward and downward movement processes through head posture estimation of the face image of the sample user i in the process, and sequentially storing the corresponding face image when the pitch angle changes by 1 degree into an image set P corresponding to the sample user i i Performing the following steps;
after the corresponding face images are acquired according to the second instruction, according to a third instruction input by the user, the sample user i slowly rotates the head 45 degrees around the Z axis towards the left side and the right side respectively, the camera continuously acquires the face images of the sample user i in the motion process, the roll angles of the head in the motion process around the Z axis towards the left side and the right side are calculated through head posture estimation, and the corresponding face images when the roll angles are changed by 1 degree are stored into the image set P corresponding to the sample user i in sequence i Performing the following steps;
after the corresponding face images are acquired according to the third instruction, according to a fourth instruction input by the user, enabling the sample user i to slowly rotate the head 45 degrees to the left and the right (around the Y axis) respectively under the head posture with the pitch angle of 45 degrees, continuously acquiring the face images of the sample user i in the motion process by the camera, estimating and calculating the angle of the yaw angle of the head in the left and the right motion processes through the head posture, and sequentially storing the corresponding face images when the yaw angle changes by 1 degree into an image set P corresponding to the sample user i i Performing the following steps;
after the corresponding face images are acquired according to the fourth instruction, according to a fifth instruction input by a user, enabling a sample user i to slowly rotate the head 45 degrees leftwards and rightwards (around the Y axis) respectively under the head posture with the pitching angle of-45 degrees, continuously acquiring the face images of the sample user i in the motion process by a camera, estimating and calculating the yaw angle of the head in the leftwards and rightwards motion processes through the head posture, and sequentially storing the corresponding face images when the yaw angle changes by 1 degree into an image set P corresponding to the sample user i i In this way, the complete image set P of the sample user i is obtained i 。
In the embodiment of the application, according to an instruction input by a user, based on head pose estimation, the face images of all head poses of all sample users during head side-to-side movement, head up-and-down movement and neck side bending movement are respectively collected according to a preset collection rule to serve as a training set, and the training set comprises the face images of the sample users under all head poses, so that the face images of all head poses can be converted by a front face reconstruction model obtained by subsequent training according to the training set, face recognition can be performed according to front face information obtained by conversion, limitation of collection angles during collection of the face images to be recognized is reduced, and flexibility of the face recognition is improved.
In some embodiments, before the step S101, the method further includes:
and B1, collecting the image to be recognized through a camera.
Optionally, the image to be recognized may be an image captured by a single image capturing device, or may be an image frame in a video stream captured by a single image capturing device (e.g., a monitoring camera).
Optionally, the acquisition time or frequency of the image to be recognized is set according to requirements, for example, the image to be recognized is acquired at a fixed time point, or the images to be recognized are acquired at random for a specified number of times within a set time range.
And B2, preprocessing the collected image to be recognized to obtain the face image to be recognized.
Because the acquired image to be recognized may or may not include a face image, and when the image to be recognized includes a face image, there may also be a case where the face image is incomplete (for example, only the lower half of the face is captured in the face image), before performing face reconstruction, it is necessary to first detect whether the face image exists in the image to be recognized.
Optionally, the MTCNN based on pre-training performs face region detection and face key point detection on the image to be recognized, and if a face region and a face key point meeting preset requirements are detected, it is indicated that the image to be recognized includes a face image, outputs the face image of a preset size, and performs gray processing on the output face image of the preset size to obtain the face image to be recognized.
Of course, if the pre-trained MTCNN does not detect a face region meeting a preset requirement in the image to be recognized, it indicates that there is no user face in the image to be recognized, and an abnormal event record is generated according to the acquisition time of the image to be recognized, the position information of the camera acquiring the image to be recognized, and the like.
And if the pre-trained MTCNN detects the face area and/or the face key point in the image to be recognized but the detected face area and/or the face key point do not meet the preset requirements, outputting corresponding prompt information to re-acquire the image to be recognized.
In the embodiment of the application, because the collected image to be recognized does not necessarily have a face image or the existing face image does not meet the preset requirements, the collected image to be recognized needs to be preprocessed, the face image existing in the image to be recognized is extracted to obtain the face image to be recognized, so that the front face information corresponding to the face image to be recognized is obtained and the face is recognized, and because the image to be recognized is subjected to gray processing, the obtained face image to be recognized is a gray image, the speed of obtaining the front face information of the face image to be recognized is increased, and the face recognition efficiency is improved.
In some embodiments, after the step S102, the method further includes:
and generating a user attendance record according to the face recognition result, the position information of the target camera and the acquisition time of the corresponding face image to be recognized.
The target camera is a camera for collecting the image to be recognized corresponding to the face image to be recognized.
Optionally, the identity information of the face image to be recognized is obtained according to the face recognition result, and an attendance record corresponding to the face image to be recognized is generated by combining the acquisition time of the image to be recognized corresponding to the face image to be recognized and the position information of the camera acquiring the image to be recognized corresponding to the face image to be recognized, so that related workers can perform attendance statistics. The attendance record describes the face identity information, the acquisition time and the acquisition position of the face image to be recognized.
Optionally, the camera is preset with a unique identifier for uniquely indicating information such as a position and a model of the corresponding camera. The unique identification comprises the model, the position, the camera number and the like of the camera.
In the embodiment of the application, the attendance record only records the face recognition result, the position information of the camera and the corresponding acquisition time of the face image to be recognized, so that related workers can perform attendance statistics of corresponding users according to the attendance record.
It should be understood that, the sequence numbers of the steps in the foregoing embodiments do not imply an execution sequence, and the execution sequence of each process should be determined by its function and inherent logic, and should not constitute any limitation to the implementation process of the embodiments of the present application.
Example two:
fig. 4 shows a block diagram of a device provided in the embodiment of the present application, which corresponds to the face recognition method described in the foregoing embodiment, and only shows portions related to the embodiment of the present application for convenience of description.
Referring to fig. 4, the apparatus includes: a face reconstruction module 41 and a face recognition module 42. Wherein the content of the first and second substances,
a face reconstruction module 41, configured to input a face image to be recognized into a front face reconstruction model, so as to obtain front face information capable of representing face features, where the front face reconstruction model is a pre-trained neural network model;
and the face recognition module 42 is configured to perform face recognition based on the front face information to obtain a face recognition result.
In the embodiment of the application, the face image to be recognized is converted into the uniform front face information capable of representing the face characteristics of the image to be recognized through the pre-trained front face reconstruction model, the similarity between the pre-input standard front face information of the user and the front face information is calculated according to the front face information, and the corresponding identity information of the standard front face information with the similarity meeting the preset threshold is used as the face recognition result. The front face information obtained by conversion represents the face characteristics of the face image to be recognized, so that face recognition can be directly carried out according to the front face information, the problem that the front face image needs to be collected when the face image to be recognized is collected, or the problem that the error of a face recognition result is large when the collected face image to be recognized is a face image deflected at a certain angle is solved, the limitation in collecting the face image to be recognized is reduced, and the flexibility and the accuracy of face recognition are improved.
In some embodiments, the face recognition apparatus further includes:
the training set acquisition module is used for acquiring face images of N sample users under a plurality of head gestures to obtain a training set, wherein N is a positive integer greater than 1.
And the standard image set acquisition module is used for acquiring the first front face images of the N sample users to obtain a standard image set.
And the training image preprocessing module is used for respectively preprocessing the images in the standard image set and the training set.
And the front face reconstruction model training module is used for training the front face reconstruction model based on the preprocessed training set and the standard image set.
In some embodiments, the training set acquisition module comprises:
and the acquisition unit is used for acquiring the face images of any sample user i under each head posture based on head posture estimation and storing the face images into a training set.
In some embodiments, the above-mentioned acquisition unit comprises:
an instruction unit, configured to sequentially acquire, according to head pose estimation, face images of respective head poses of the sample user i in three head motion states, where the three head motion states include: head side-to-side movement, head up-and-down movement, and neck side-to-side movement.
In some embodiments, the face recognition apparatus further includes:
and the image acquisition device to be identified is used for acquiring an image to be identified through the camera.
And the face image to be recognized acquisition module is used for preprocessing the acquired image to be recognized so as to obtain the face image to be recognized.
In some embodiments, the face recognition apparatus further includes:
and the record generation module is used for generating a user attendance record according to the face recognition result, the position information of the target camera and the acquisition time of the corresponding face image to be recognized.
It should be noted that, for the information interaction, execution process, and other contents between the above devices/units, the specific functions and technical effects thereof based on the same concept as those of the method embodiment of the present application can be specifically referred to the method embodiment portion, and are not described herein again.
Example three:
fig. 5 is a schematic structural diagram of a terminal device according to an embodiment of the present application. As shown in fig. 5, the terminal device 5 of this embodiment includes: at least one processor 50 (only one processor is shown in fig. 5), a memory 51, and a computer program 52 stored in the memory 51 and executable on the at least one processor 50, the steps of any of the various method embodiments described above being implemented when the computer program 52 is executed by the processor 50.
Illustratively, the computer program 52 may be partitioned into one or more modules/units, which are stored in the memory 51 and executed by the processor 50 to accomplish the present application. The one or more modules/units may be a series of computer program instruction segments capable of performing specific functions, which are used to describe the execution process of the computer program 52 in the terminal device 5. For example, the computer program 52 may be divided into a face reconstruction module 41 and a face recognition module 42, and the specific functions among the modules are as follows:
a face reconstruction module 41, configured to input a face image to be recognized into a front face reconstruction model, so as to obtain front face information capable of representing face features, where the front face reconstruction model is a pre-trained neural network model;
and the face recognition module 42 is configured to perform face recognition based on the front face information to obtain a face recognition result.
The terminal device 5 may be a desktop computer, a notebook, a palm computer, a cloud server, or other computing devices. The terminal device may include, but is not limited to, a processor 50, a memory 51. Those skilled in the art will appreciate that fig. 5 is only an example of the terminal device 5, and does not constitute a limitation to the terminal device 5, and may include more or less components than those shown, or combine some components, or different components, such as an input-output device, a network access device, and the like.
The Processor 50 may be a Central Processing Unit (CPU), and the Processor 50 may be other general purpose Processor, a Digital Signal Processor (DSP), an Application Specific Integrated Circuit (ASIC), a Field Programmable Gate Array (FPGA) or other Programmable logic device, discrete Gate or transistor logic device, discrete hardware component, or the like. A general purpose processor may be a microprocessor or the processor may be any conventional processor or the like.
The memory 51 may in some embodiments be an internal storage unit of the terminal device 5, such as a hard disk or a memory of the terminal device 5. The memory 51 may also be an external storage device of the terminal device 5 in other embodiments, such as a plug-in hard disk, a Smart Media Card (SMC), a Secure Digital (SD) Card, a Flash memory Card (Flash Card), and the like, which are provided on the terminal device 5. Further, the memory 51 may also include both an internal storage unit and an external storage device of the terminal device 5. The memory 51 is used for storing an operating system, an application program, a BootLoader (BootLoader), data, and other programs, such as program codes of the computer program. The memory 51 may also be used to temporarily store data that has been output or is to be output.
It will be apparent to those skilled in the art that, for convenience and brevity of description, only the above-mentioned division of the functional units and modules is illustrated, and in practical applications, the above-mentioned function distribution may be performed by different functional units and modules according to needs, that is, the internal structure of the apparatus is divided into different functional units or modules to perform all or part of the above-mentioned functions. Each functional unit and module in the embodiments may be integrated in one processing unit, or each unit may exist alone physically, or two or more units are integrated in one unit, and the integrated unit may be implemented in a form of hardware, or in a form of software functional unit. In addition, specific names of the functional units and modules are only for convenience of distinguishing from each other, and are not used for limiting the protection scope of the present application. The specific working processes of the units and modules in the system may refer to the corresponding processes in the foregoing method embodiments, and are not described herein again.
An embodiment of the present application further provides a network device, where the network device includes: at least one processor, a memory, and a computer program stored in the memory and executable on the at least one processor, the processor implementing the steps of any of the various method embodiments described above when executing the computer program.
An embodiment of the present application further provides a computer-readable storage medium, where a computer program is stored, and when the computer program is executed by a processor, the computer program implements the steps in the foregoing method embodiments.
The embodiments of the present application provide a computer program product, which when running on a terminal device, enables the terminal device to implement the steps in the above method embodiments when executed.
The integrated unit, if implemented in the form of a software functional unit and sold or used as a stand-alone product, may be stored in a computer readable storage medium. Based on such understanding, all or part of the processes in the methods of the embodiments described above may be implemented by instructing relevant hardware by a computer program, which may be stored in a computer-readable storage medium, and when the computer program is executed by a processor, the steps of the embodiments of the methods described above may be implemented. Wherein the computer program comprises computer program code, which may be in the form of source code, object code, an executable file or some intermediate form, etc. The computer readable medium may include at least: any entity or device capable of carrying computer program code to a photographing apparatus/terminal apparatus, a recording medium, computer Memory, Read-Only Memory (ROM), Random Access Memory (RAM), an electrical carrier signal, a telecommunications signal, and a software distribution medium. Such as a usb-disk, a removable hard disk, a magnetic or optical disk, etc. In certain jurisdictions, computer-readable media may not be an electrical carrier signal or a telecommunications signal in accordance with legislative and patent practice.
In the above embodiments, the descriptions of the respective embodiments have respective emphasis, and reference may be made to the related descriptions of other embodiments for parts that are not described or illustrated in a certain embodiment.
Those of ordinary skill in the art will appreciate that the various illustrative elements and algorithm steps described in connection with the embodiments disclosed herein may be implemented as electronic hardware, or combinations of computer software and electronic hardware. Whether such functionality is implemented as hardware or software depends upon the particular application and design constraints imposed on the implementation. Skilled artisans may implement the described functionality in varying ways for each particular application, but such implementation decisions should not be interpreted as causing a departure from the scope of the present application.
In the embodiments provided in the present application, it should be understood that the disclosed apparatus/network device and method may be implemented in other ways. For example, the above-described apparatus/network device embodiments are merely illustrative, and for example, the division of the modules or units is only one logical division, and there may be other divisions when actually implementing, for example, a plurality of units or components may be combined or integrated into another system, or some features may be omitted, or not implemented. In addition, the shown or discussed mutual coupling or direct coupling or communication connection may be an indirect coupling or communication connection through some interfaces, devices or units, and may be in an electrical, mechanical or other form.
The units described as separate parts may or may not be physically separate, and parts displayed as units may or may not be physical units, may be located in one place, or may be distributed on a plurality of network units. Some or all of the units can be selected according to actual needs to achieve the purpose of the solution of the embodiment.
The above-mentioned embodiments are only used to illustrate the technical solutions of the present application, and not to limit the same; although the present application has been described in detail with reference to the foregoing embodiments, it should be understood by those of ordinary skill in the art that: the technical solutions described in the foregoing embodiments may still be modified, or some technical features may be equivalently replaced; such modifications and substitutions do not depart from the spirit and scope of the embodiments of the present application, and they should be construed as being included in the present application.
Claims (10)
1. A face recognition method, comprising:
inputting a face image to be recognized into a front face reconstruction model to obtain front face information capable of representing face features, wherein the front face reconstruction model is a pre-trained neural network model;
and carrying out face recognition based on the front face information to obtain a face recognition result.
2. The face recognition method of claim 1, wherein the frontal face reconstruction model is trained by a method comprising:
acquiring face images of N sample users under a plurality of head gestures to obtain a training set, wherein N is a positive integer greater than 1;
acquiring front face images of the N sample users to obtain a standard image set;
respectively preprocessing the images in the standard image set and the training set;
and training the front face reconstruction model based on the preprocessed training set and the standard image set.
3. The method of claim 2, wherein the acquiring of the face images of the N sample users in the plurality of head poses to obtain a training set comprises:
and for any sample user i, acquiring a face image of the sample user i under each head posture based on head posture estimation, and storing the face image into a training set.
4. The method according to claim 3, wherein the acquiring of the face image of the sample user i in each head pose based on the head pose estimation is:
based on head pose estimation, sequentially acquiring face images of each head pose of the sample user i in three head motion states, wherein the three head motion states comprise: head side-to-side movement, head up-and-down movement, and neck side flexion movement.
5. The face recognition method according to any one of claims 1 to 4, wherein before inputting the face image to be recognized into the front face reconstruction model, the method further comprises:
acquiring an image to be identified through a camera;
and preprocessing the acquired image to be recognized to obtain the face image to be recognized.
6. The face recognition method of claim 5, further comprising, after obtaining the face recognition result:
generating a user attendance record according to the face recognition result, the position information of the target camera and the acquisition time of the corresponding face image to be recognized;
the target camera is a camera for collecting corresponding face images to be recognized.
7. A face recognition apparatus, comprising:
the face reconstruction module is used for inputting a face image to be recognized into a front face reconstruction model to obtain front face information capable of representing face features, and the front face reconstruction model is a pre-trained neural network model;
and the face recognition module is used for carrying out face recognition based on the front face information to obtain a face recognition result.
8. The face recognition apparatus of claim 7, further comprising:
the training set acquisition module is used for acquiring face images of N sample users under a plurality of head gestures to obtain a training set, wherein N is a positive integer greater than 1;
the standard image set acquisition module is used for acquiring the front face images of the N sample users to obtain a standard image set;
the training image preprocessing module is used for respectively preprocessing the standard image set and the images in the training set;
and the front face reconstruction model training module is used for training the front face reconstruction model based on the preprocessed training set and the standard image set.
9. A terminal device comprising a memory, a processor and a computer program stored in the memory and executable on the processor, characterized in that the processor implements the method according to any of claims 1 to 7 when executing the computer program.
10. A computer-readable storage medium, in which a computer program is stored which, when being executed by a processor, carries out the method according to any one of claims 1 to 7.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202210601012.4A CN114926887A (en) | 2022-05-30 | 2022-05-30 | Face recognition method and device and terminal equipment |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202210601012.4A CN114926887A (en) | 2022-05-30 | 2022-05-30 | Face recognition method and device and terminal equipment |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN114926887A true CN114926887A (en) | 2022-08-19 |
Family
ID=82813161
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202210601012.4A Pending CN114926887A (en) | 2022-05-30 | 2022-05-30 | Face recognition method and device and terminal equipment |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN114926887A (en) |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN117238020A (en) * | 2023-11-10 | 2023-12-15 | 杭州启源视觉科技有限公司 | Face recognition method, device and computer equipment |
-
2022
- 2022-05-30 CN CN202210601012.4A patent/CN114926887A/en active Pending
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN117238020A (en) * | 2023-11-10 | 2023-12-15 | 杭州启源视觉科技有限公司 | Face recognition method, device and computer equipment |
| CN117238020B (en) * | 2023-11-10 | 2024-04-26 | 杭州启源视觉科技有限公司 | Face recognition method, device and computer equipment |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN109558832B (en) | Human body posture detection method, device, equipment and storage medium | |
| Lee et al. | Ensemble deep learning for skeleton-based action recognition using temporal sliding lstm networks | |
| Ji et al. | Interactive body part contrast mining for human interaction recognition | |
| CN109657533A (en) | Pedestrian recognition methods and Related product again | |
| EP3811337A1 (en) | System for predicting articulated object feature location | |
| CN109829396B (en) | Face recognition motion blur processing method, device, equipment and storage medium | |
| CN110263603B (en) | Face recognition method and device based on central loss and residual error visual simulation network | |
| CN108108711B (en) | Face control method, electronic device and storage medium | |
| CN109215131B (en) | Virtual face driving method and device | |
| CN112530019A (en) | Three-dimensional human body reconstruction method and device, computer equipment and storage medium | |
| CN112966574A (en) | Human body three-dimensional key point prediction method and device and electronic equipment | |
| Liu et al. | Attentive cross-modal fusion network for RGB-D saliency detection | |
| CN107944381B (en) | Face tracking method, face tracking device, terminal and storage medium | |
| CN113159006B (en) | Attendance checking method and system based on face recognition, electronic equipment and storage medium | |
| Weiyao et al. | Human action recognition using multilevel depth motion maps | |
| CN110414571A (en) | A kind of website based on Fusion Features reports an error screenshot classification method | |
| CN111626163A (en) | Human face living body detection method and device and computer equipment | |
| CN111680544B (en) | Face recognition method, device, system, equipment and medium | |
| CN111079536A (en) | Behavior analysis method based on human body key point time sequence, storage medium and equipment | |
| CN114550212A (en) | Goat face detection and identification method based on lightweight model | |
| CN105844204B (en) | Human behavior recognition method and device | |
| CN116977937A (en) | Pedestrian re-identification method and system | |
| CN113688804B (en) | Multi-angle video-based action identification method and related equipment | |
| CN114926887A (en) | Face recognition method and device and terminal equipment | |
| CN111488810A (en) | Face recognition method and device, terminal equipment and computer readable medium |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |