CN111597933B - Face recognition method and device - Google Patents
Face recognition method and device Download PDFInfo
- Publication number
- CN111597933B CN111597933B CN202010367239.8A CN202010367239A CN111597933B CN 111597933 B CN111597933 B CN 111597933B CN 202010367239 A CN202010367239 A CN 202010367239A CN 111597933 B CN111597933 B CN 111597933B
- Authority
- CN
- China
- Prior art keywords
- face
- image
- speckle
- sample
- parallax
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V40/00—Recognition of biometric, human-related or animal-related patterns in image or video data
- G06V40/10—Human or animal bodies, e.g. vehicle occupants or pedestrians; Body parts, e.g. hands
- G06V40/16—Human faces, e.g. facial parts, sketches or expressions
- G06V40/161—Detection; Localisation; Normalisation
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/21—Design or setup of recognition systems or techniques; Extraction of features in feature space; Blind source separation
- G06F18/214—Generating training patterns; Bootstrap methods, e.g. bagging or boosting
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/24—Classification techniques
- G06F18/241—Classification techniques relating to the classification model, e.g. parametric or non-parametric approaches
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
- G06N3/045—Combinations of networks
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/08—Learning methods
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V40/00—Recognition of biometric, human-related or animal-related patterns in image or video data
- G06V40/10—Human or animal bodies, e.g. vehicle occupants or pedestrians; Body parts, e.g. hands
- G06V40/16—Human faces, e.g. facial parts, sketches or expressions
- G06V40/168—Feature extraction; Face representation
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V40/00—Recognition of biometric, human-related or animal-related patterns in image or video data
- G06V40/10—Human or animal bodies, e.g. vehicle occupants or pedestrians; Body parts, e.g. hands
- G06V40/16—Human faces, e.g. facial parts, sketches or expressions
- G06V40/172—Classification, e.g. identification
-
- Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
- Y02T—CLIMATE CHANGE MITIGATION TECHNOLOGIES RELATED TO TRANSPORTATION
- Y02T10/00—Road transport of goods or passengers
- Y02T10/10—Internal combustion engine [ICE] based vehicles
- Y02T10/40—Engine management systems
Abstract
The embodiment of the invention provides a face recognition method and a device, wherein the face recognition method comprises the steps of obtaining a face speckle image and a reference speckle image corresponding to the face speckle image; inputting the face speckle image and the reference speckle image into a face recognition model to obtain the face characteristics of the face speckle image; the face recognition model is obtained by training a face speckle sample image serving as a sample and a predetermined face sample labeling result corresponding to the face speckle sample image serving as a sample label, and the sample face features corresponding to the face speckle sample image correspond to the face sample labeling result. The face recognition method provided by the embodiment of the invention provides an end-to-end speckle face recognition frame, can fully utilize the abundant information contained in the face speckle image, is obviously superior to the face recognition method based on depth, and can fully utilize the information in the face speckle image by carrying out joint training on a parallax network and a recognition network, thereby obviously improving the accuracy of face recognition.
Description
Technical Field
The invention relates to the technical field of computer vision, in particular to a face recognition method and device.
Background
The face recognition is to perform the identity recognition of the person according to the input signal, and most of the existing face recognition methods are to perform face recognition directly on visual images, such as RGB images, gray images and depth images. Currently, the mainstream face recognition methods include a face recognition method based on RGB and a face recognition method based on RGB-D.
In recent years, due to the strong learning ability of the convolutional neural network and the application of large-scale face data, the face recognition method based on RGB has great progress, and has strong competitive performance on popular benchmark evaluation. Although the RGB-based face recognition method achieves better recognition accuracy, the recognition performance thereof is drastically reduced when the viewing angle is changed greatly or the illumination is changed strongly, thereby limiting the application thereof to high security levels, such as financial transactions.
With the popularization of consumer-level depth sensors such as Kinect V1 and PrimeSense, depth images can be used as input signals in addition to RGB images to assist in face recognition, further improving recognition accuracy. The face recognition method based on RGB-D utilizes more depth information, improves recognition accuracy and obtains better robustness.
According to the face recognition method based on RGB-D, an original speckle image is obtained from a depth sensor, then the depth is calculated by using an internal algorithm of the speckle image, and finally the restored depth image is used for face recognition. However, this two-stage process may not fully utilize the information stored in the original signal, mainly in that some information that is not important for depth recovery but is critical for face recognition may be lost in the first stage, and such lost information may never be recovered in the second stage.
Disclosure of Invention
Embodiments of the present invention provide a face recognition method, apparatus, electronic device, and readable storage medium that overcome or at least partially solve the above-described problems.
In a first aspect, an embodiment of the present invention provides a face recognition method, including: acquiring a face speckle image and a reference speckle image corresponding to the face speckle image; inputting the face speckle image and the reference speckle image into a face recognition model to obtain face characteristics of the face speckle image; the face recognition model comprises a parallax network and a recognition network; the step of inputting the face speckle image and the reference speckle image into a face recognition model, and the step of obtaining the face characteristics of the face speckle image comprises the following steps: inputting the face speckle image and the reference speckle image into the parallax network to obtain a parallax image; obtaining a depth map based on the parallax map; inputting the depth map to the recognition network to obtain the face characteristics of the face speckle image; the face recognition model is obtained by training a face speckle sample image serving as a sample and a predetermined face sample labeling result corresponding to the face speckle sample image serving as a sample label, and the sample face features corresponding to the face speckle sample image correspond to the face sample labeling result.
In some embodiments, the determining of the parallax network comprises: and training by taking the speckle sample image as a sample and taking a predetermined parallax true value of each pixel point corresponding to the speckle sample image as a sample label.
In some embodiments, the step of determining the identification network comprises: and fixing the trained parallax network, taking the face speckle sample image as a sample, and training by taking a predetermined face sample labeling result corresponding to the face speckle sample image as a sample label.
In some embodiments, the parallax network is determined based on a smoth-L1 Loss function and the identification network is determined based on a Softmax-Loss function.
In some embodiments, the parallax network comprises: the device comprises a feature extraction module, a three-dimensional matching cost aggregation module and a parallax calculation module; the characteristic extraction module is used for extracting characteristics of each pixel point corresponding to the speckle sample image; the three-dimensional matching cost building module is used for calculating the matching cost value of each pixel point; the stereo matching cost aggregation module is used for aggregating the matching cost values; the parallax calculation module is used for calculating the parallax measurement value of each pixel point.
In some embodiments, the acquiring a face speckle image and a reference speckle image corresponding to the face speckle image comprises: acquiring an original speckle image, an original color image and the reference speckle image; acquiring a face boundary box based on the original color image; acquiring an original speckle identification image based on the original speckle image and the face boundary box; preprocessing the original speckle identification image to obtain the face speckle image.
In some embodiments, the preprocessing the original speckle identification image to obtain the face speckle image includes: and carrying out local contrast normalization processing on the original speckle identification image to obtain the human face speckle image.
In a second aspect, an embodiment of the present invention provides a face recognition apparatus including: the acquisition unit is used for acquiring the face speckle image and the reference speckle image; the identification unit is used for inputting the face speckle image and the reference speckle image into a face recognition model to acquire the face characteristics of the face speckle image; the face recognition model comprises a parallax network and a recognition network; the step of inputting the face speckle image and the reference speckle image into a face recognition model, and the step of obtaining the face characteristics of the face speckle image comprises the following steps: inputting the face speckle image and the reference speckle image into the parallax network to obtain a parallax image; obtaining a depth map based on the parallax map; inputting the depth map to the recognition network to obtain the face characteristics of the face speckle image; the face recognition model is obtained by training a face speckle sample image serving as a sample and a predetermined face sample labeling result corresponding to the face speckle sample image serving as a sample label, and the sample face features corresponding to the face speckle sample image correspond to the face sample labeling result.
In a third aspect, an embodiment of the present invention provides an electronic device comprising a memory, a processor and a computer program stored on the memory and executable on the processor, the processor implementing the steps of the face recognition method as provided in the first aspect when the program is executed.
In a fourth aspect, embodiments of the present invention provide a non-transitory computer readable storage medium having stored thereon a computer program which, when executed by a processor, implements the steps of the face recognition method as provided in the first aspect.
The embodiment of the invention provides a face recognition method, a device, electronic equipment and a readable storage medium, which provide an end-to-end speckle face recognition frame, a parallax network carries out parallax regression on a face speckle image, a parallax image is converted into a depth image, and a recognition network extracts face features from the depth image. The cascade network model architecture consisting of the parallax network and the recognition network can fully utilize rich information contained in the face speckle image, and is obviously superior to a face recognition method based on depth. Meanwhile, by carrying out joint training on the parallax network and the recognition network, the information in the face speckle image is fully utilized, and the accuracy of face recognition is obviously improved.
Drawings
In order to more clearly illustrate the embodiments of the present invention or the technical solutions of the prior art, the following description will briefly explain the drawings used in the embodiments or the description of the prior art, and it is obvious that the drawings in the following description are some embodiments of the present invention, and other drawings can be obtained according to these drawings without inventive effort for a person skilled in the art.
Fig. 1 is a flowchart of a face recognition method according to an embodiment of the present invention;
fig. 2 is a flowchart of another face recognition method according to an embodiment of the present invention;
fig. 3 is a network architecture diagram of another face recognition method according to an embodiment of the present invention;
fig. 4 is a schematic structural diagram of a face recognition device according to an embodiment of the present invention;
fig. 5 is a schematic structural diagram of a face recognition electronic device according to an embodiment of the present invention.
Detailed Description
For the purpose of making the objects, technical solutions and advantages of the embodiments of the present invention more apparent, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings in the embodiments of the present invention, and it is apparent that the described embodiments are some embodiments of the present invention, but not all embodiments of the present invention. All other embodiments, which can be made by those skilled in the art based on the embodiments of the invention without making any inventive effort, are intended to be within the scope of the invention.
The following describes a face recognition method provided by an embodiment of the present invention with reference to fig. 1.
As shown in fig. 1, the face recognition method provided by the embodiment of the present invention includes steps S100 and S200.
Step S100, a face speckle image and a reference speckle image corresponding to the face speckle image are acquired.
It can be understood that the depth camera is provided with two cameras and a projector, one of the cameras is a common camera, the other is an infrared camera, the common camera shoots color images, and the infrared camera synchronously shoots speckle images; the projector is used for projecting speckle signals. According to the embodiment of the invention, the infrared camera of the depth camera is used for acquiring the face speckle image and the reference speckle image, wherein the reference speckle image is any plane speckle image, the face speckle image and the reference speckle image are acquired by the same depth camera, and the size of the face speckle image is the same as that of the reference speckle image.
And step 200, inputting the face speckle image and the reference speckle image into a face recognition model to acquire the face characteristics of the face speckle image.
It can be appreciated that, in the application stage of the face recognition model, the face recognition model is input into the face speckle image and the reference speckle image, and the face recognition model is output as the face characteristics of the face speckle image.
Step S200 includes steps S210 to S230.
And S210, inputting the face speckle image and the reference speckle image into a parallax network to obtain a parallax image.
It can be understood that, as shown in fig. 2, the face recognition model provided by the embodiment of the invention is an end-to-end Speckle Face Recognition Net (SFR-Net) network model constructed for the speckle image, and the face recognition model includes two sub-networks, namely a parallax network and a recognition network, wherein the parallax network is a parallax regression network with small weight, and is used for obtaining a parallax image between the face speckle image and the reference speckle image, and the recognition network is a face recognition convolutional neural network, and is used for recognizing face features in the face speckle image.
In the application stage of the face recognition model, the input of the parallax network is a face speckle image and a reference speckle image, the parallax value of each corresponding pixel point between the face speckle image and the reference speckle image is calculated, the parallax values of all the pixel points form a parallax map, and the output of the parallax network is the parallax map.
Step S220, obtaining a depth map based on the parallax map.
It can be understood that the parallax value of each pixel point in the parallax image is converted into a depth value, so that a depth image corresponding to the face speckle image can be obtained. The conversion formula for converting the parallax value into the depth value is:
wherein z is a depth value, b is a length value of a base line between the infrared camera and the projector, f is a focal length of the infrared camera, and d is a disparity value.
Step S230, the depth map is input to a recognition network, and the face characteristics of the face speckle image are obtained.
It can be understood that, the identification network provided by the embodiment of the invention adopts a Sphere Face network, the number of layers of the convolutional neural network is not particularly limited, the embodiment of the invention uses the convolutional neural network with the number of layers of 20 layers as an example, a 20-layer convolutional neural network is constructed to extract the Face features, and then the cross entropy loss function is used for classifying the Face features. The network input is recognized as a depth map, and the face features of the face speckle images are output.
It should be noted that the face recognition model is obtained by training with a face speckle sample image as a sample and a predetermined face sample labeling result corresponding to the face speckle sample image as a sample label, and the sample face features corresponding to the face speckle sample image correspond to the face sample labeling result.
It can be understood that in the training stage of the face recognition model, the parallax network and the recognition network are jointly trained in an end-to-end manner, and the learning rates of the parallax network and the recognition network are independently set and adjusted. The human face speckle sample image and the reference speckle sample image are input into the human face recognition model, the sample label is a human face sample labeling result corresponding to the human face speckle sample image, and the human face sample recognition result is output from the human face recognition model.
The reference speckle sample image is an arbitrary plane speckle image, the face speckle sample image and the reference speckle sample image are obtained by adopting the same depth camera, and the size of the face speckle sample image is the same as that of the reference speckle sample image.
The face sample labeling result is automatic labeling, the face sample labeling result and the face speckle sample image are in corresponding relation, and the face sample labeling result and the sample face characteristics corresponding to the face speckle sample image are also in corresponding relation.
The face recognition method provided by the embodiment of the invention provides an end-to-end speckle face recognition frame, a parallax network carries out parallax regression on a face speckle image, a parallax image is converted into a depth image, and the recognition network extracts face features from the depth image. The cascade network model architecture consisting of the parallax network and the recognition network can fully utilize rich information contained in the face speckle image, and is obviously superior to a face recognition method based on depth. Meanwhile, by carrying out joint training on the parallax network and the recognition network, the information in the face speckle image is fully utilized, and the accuracy of face recognition is obviously improved.
In some embodiments, the step of determining the parallax network comprises: and training by taking the speckle sample image as a sample and taking a predetermined parallax true value of each pixel point corresponding to the speckle sample image as a sample label.
It will be appreciated that during the training phase of the face recognition model, the parallax network is independently pre-trained prior to the combined training of the parallax network and the recognition network.
And in the pre-training stage, the parallax network is input into a speckle sample image and a reference speckle sample image, the sample label is a parallax true value of each pixel point corresponding to the speckle sample image, and the output of the parallax network is a parallax measured value of each pixel point corresponding to the speckle sample image.
The reference speckle sample image is an arbitrary plane speckle image, the speckle sample image and the reference speckle sample image are obtained by adopting the same depth camera, and the size of the speckle sample image is the same as that of the reference speckle sample image.
The parallax true value of each pixel corresponding to the speckle sample image is a parallax true value of each corresponding pixel between the speckle sample image and the reference speckle sample image, which is automatically determined in advance.
According to the face recognition method provided by the embodiment of the invention, under the condition of parallax true value supervision, the parallax network is independently pre-trained, and the parallax network can avoid outputting some disordered features, so that the calculation accuracy of the parallax network is improved.
In some embodiments, the step of determining the identification network comprises: and fixing the trained parallax network, taking the face speckle sample image as a sample, and training by taking a predetermined face sample labeling result corresponding to the face speckle sample image as a sample label.
It will be appreciated that during the training phase of the face recognition model, the recognition network is independently pre-trained after the parallax network is independently pre-trained prior to the joint training of the parallax network and the recognition network.
And (3) a pre-training stage is carried out on the identification network, the parallax network which is already pre-trained is fixed, and only the identification network is pre-trained. The input of the recognition network is a face speckle sample image and a reference speckle sample image, the sample label is a face sample labeling result corresponding to the face speckle sample image, and the output of the recognition network is a face sample recognition result.
The reference speckle sample image is an arbitrary plane speckle image, the face speckle sample image and the reference speckle sample image are obtained by adopting the same depth camera, and the size of the face speckle sample image is the same as that of the reference speckle sample image.
The face sample labeling result is automatic labeling, the face sample labeling result and the face speckle sample image are in corresponding relation, and the face sample labeling result and the sample face characteristics corresponding to the face speckle sample image are also in corresponding relation.
The face recognition method of the embodiment of the invention is tested by adopting the general indexes of 1v.1 comparison and 1v.n comparison in the face recognition industry, and the comparison results are shown in tables 1 and 2.
Table 1 comparative tables without and with Pre-training
| Method | 1v.1 | 1v.n |
| Comparison results without Pre-training | 80.63％ | 57.59％ |
| Comparison results with Pre-training | 92.12％ | 89.66％ |
TABLE 2 comparative table for separate training and joint training
| Method | 1v.1 | 1v.n |
| Comparison results of separate training | 86.43％ | 71.71％ |
| Comparison results of joint training | 92.12％ | 89.66％ |
The face recognition method provided by the embodiment of the invention uses a method combining pre-training and combined training, and under the supervision, the parallax network and the recognition network are respectively and independently pre-trained, and then the parallax network and the recognition network are combined trained to unify the two networks, so that the respective advantages of the parallax network and the recognition network are mutually supplemented. By training the parallax network and the recognition network twice respectively, the information in the original speckle images is utilized more fully, and the face features with better effects are obtained.
In some embodiments, the parallax network is determined based on a Smooth-L1 Loss function and the identification network is determined based on a Softmax-Loss function.
It can be understood that the Loss function is one way to measure the difference between the output measured value and the actual value of the neural network, the parallax network adopts a smoth-L1 Loss function for optimizing the parallax network, and the identification network adopts a Softmax-Loss function for optimizing the identification network.
The formula of the Smooth-L1 loss function is:
wherein d is the parallax true value,
for the parallax measurement value output by the parallax model, N is the total number of pixel points in the speckle sample image, i is the pixel points in the speckle sample image, x is the absolute value of the difference between the parallax true value and the parallax measurement value, and Smooth L1 (x) Is a Smooth-L1 loss function, < ->
Is the true value of the parallax and the loss value of the parallax measurement value.
The equation for the Softmax-Loss function is:
wherein N is the total number of samples of the training recognition network, i is the label of the samples, C is the total number of training sample categories, j is the corresponding category number, F R (-) is in the identification network, space R HxW To space R K K represents F R (Z) feature dimension, Z is the depth map, H is the height of the depth map, W is the width of the depth map,
and->
Weight parameters of the corresponding class for the last full-connection layer, +.>
And b j For the bias parameters of the corresponding category of the last full-connection layer, L s Is a Softmax-Loss function.
According to the face recognition method provided by the embodiment of the invention, the parallax network is optimized and determined through the Smooth-L1 Loss function, and the recognition network is optimized and determined through the Softmax-Loss, so that the accuracy of face features is further improved.
In some embodiments, as shown in fig. 3, the parallax network includes a feature extraction module, a construction stereo matching cost module, a stereo matching cost aggregation module, and a parallax calculation module.
It will be appreciated that the feature extraction module includes a first convolution layer-a fifth convolution layer for feature extraction for each pixel point corresponding to the speckle sample image. The feature extraction refers to extracting image information of each pixel point corresponding to the speckle sample image and the reference speckle sample image, and the extracted features are used for determining whether each pixel point belongs to an image feature, so that the pixel points in the speckle sample image are classified.
The three-dimensional matching cost module is used for calculating the matching cost value of each pixel point corresponding to the speckle sample image and the reference speckle sample image.
The stereo matching cost aggregation module comprises a sixth convolution layer-a fourteenth convolution layer and is used for aggregating the matching cost values. Since the above step of matching cost value calculation only considers local correlation, but is very sensitive to noise, it cannot be directly used to calculate the disparity value. And the stereo matching cost aggregation module is adopted to aggregate the matching cost values, so that the aggregated matching cost values can more accurately reflect the correlation between the pixel points.
And (3) after readjusting and upsampling the aggregated matching cost value, adopting a parallax calculation module to calculate a parallax measurement value of each pixel point.
The face recognition method provided by the embodiment of the invention adopts the parallax regression network with small weight, and improves the accuracy of parallax calculation.
In some embodiments, step S100 includes steps S110-S140.
Step S110, acquiring an original speckle image, an original color image, and a reference speckle image.
It can be appreciated that the embodiment of the invention adopts the infrared camera of the depth camera to acquire the original speckle image and the reference speckle image, and adopts the common camera of the depth camera to acquire the original color image.
Step S120, based on the original color image, a face boundary box is obtained.
It can be appreciated that based on the original color image, a two-dimensional face detection algorithm is adopted to obtain a face bounding box in the original color image.
And step S130, acquiring an original speckle identification image based on the original speckle image and the human face boundary box.
It can be understood that the original speckle image and the original color image are aligned, and the corresponding face boundary area in the original speckle image is cut according to the face boundary box in the original color image, so that the original speckle identification image is obtained after cutting.
And step 140, preprocessing the original speckle identification image to obtain a human face speckle image.
It can be appreciated that the original speckle pattern recognition image is preprocessed to obtain a face speckle pattern that is more suitable for face recognition models.
According to the face recognition method provided by the embodiment of the invention, the face boundary box in the original color image is obtained by adopting the two-dimensional face detection algorithm, and the original speckle image is processed by using the face boundary box to obtain the face speckle image, so that the face speckle image is more suitable for a face recognition model, and the recognition effect of the face recognition model is improved.
In some embodiments, step S140 includes: and carrying out local contrast normalization processing on the original speckle recognition image to obtain a human face speckle image.
It can be appreciated that local contrast normalization is a data preprocessing method commonly used in deep learning, and not only can remove the correlation between brightness and parallax, but also the brightness change will not affect the original speckle-identified image. The calculation formula of the local contrast normalization is as follows:
wherein I is the pixel value of the pixel, mu is the average value in the neighborhood window of the pixel, sigma is the standard deviation in the neighborhood window of the pixel, epsilon is a preset small constant,
the calculated value is used to replace the pixel value of the pixel point.
According to the face recognition method provided by the embodiment of the invention, local contrast normalization processing is introduced, so that the error problem caused by inconsistent contrast and brightness in the speckle image is eliminated.
The following describes a face recognition device provided by an embodiment of the present invention, and the face recognition device described below and the face recognition method described above may be referred to correspondingly.
The following describes a face recognition device provided by an embodiment of the present invention with reference to fig. 4.
As shown in fig. 4, the apparatus includes an acquisition unit 310 and an identification unit 320.
An acquiring unit 310, configured to acquire a face speckle image and a reference speckle image corresponding to the face speckle image.
It can be understood that the embodiment of the invention adopts the infrared camera of the depth camera to acquire the face speckle image and the reference speckle image, wherein the reference speckle image is any plane speckle image, the face speckle image and the reference speckle image are acquired by adopting the same depth camera, and the size of the face speckle image is the same as that of the reference speckle image. The recognition unit 310 acquires a face speckle image and a reference speckle image.
The identifying unit 320 is configured to input the face speckle image and the reference speckle image into a face recognition model, and obtain a face feature of the face speckle image; the face recognition model comprises a parallax network and a recognition network; inputting the face speckle image and the reference speckle image into a face recognition model, and acquiring the face characteristics of the face speckle image comprises the following steps: inputting the face speckle image and the reference speckle image into a parallax network to obtain a parallax image; obtaining a depth map based on the parallax map; inputting the depth map into a recognition network to obtain the face characteristics of the face speckle image; the face recognition model is obtained by training a face speckle sample image serving as a sample and a predetermined face sample labeling result corresponding to the face speckle sample image serving as a sample label, and the sample face features corresponding to the face speckle sample image correspond to the face sample labeling result.
It will be appreciated that the recognition unit 320 is configured to input the face speckle image and the reference speckle image into the face recognition model, and output the face features of the face speckle image.
The face recognition model is obtained by training a face speckle sample image serving as a sample and a predetermined face sample labeling result corresponding to the face speckle sample image serving as a sample label, and the sample face features corresponding to the face speckle sample image correspond to the face sample labeling result.
The embodiment of the invention provides a face recognition device, which provides an end-to-end speckle face recognition frame, wherein a parallax network carries out parallax regression on a face speckle image, converts a parallax image into a depth image, and a recognition network extracts face features from the depth image. The cascade network model architecture consisting of the parallax network and the recognition network can fully utilize rich information contained in the face speckle image, and is obviously superior to a face recognition method based on depth. Meanwhile, by carrying out joint training on the parallax network and the recognition network, the information in the face speckle image is fully utilized, and the accuracy of face recognition is obviously improved.
Fig. 5 illustrates a physical schematic diagram of an electronic device, as shown in fig. 5, which may include: processor 410, communication interface (Communications Interface) 420, memory 430 and communication bus 440, wherein processor 410, communication interface 420 and memory 430 communicate with each other via communication bus 440. The processor 410 may invoke logic instructions in the memory 430 to perform a face recognition method that includes acquiring a face speckle image and a reference speckle image corresponding to the face speckle image; inputting the face speckle image and the reference speckle image into a face recognition model to obtain the face characteristics of the face speckle image; the face recognition model comprises a parallax network and a recognition network; inputting the face speckle image and the reference speckle image into a face recognition model, and acquiring the face characteristics of the face speckle image comprises the following steps: inputting the face speckle image and the reference speckle image into a parallax network to obtain a parallax image; obtaining a depth map based on the parallax map; inputting the depth map into a recognition network to obtain the face characteristics of the face speckle image; the face recognition model is obtained by training a face speckle sample image serving as a sample and a predetermined face sample labeling result corresponding to the face speckle sample image serving as a sample label, and the sample face features corresponding to the face speckle sample image correspond to the face sample labeling result.
It should be noted that, in this embodiment, the electronic device may be a server, a PC, or other devices in the specific implementation, so long as the structure of the electronic device includes a processor 410, a communication interface 420, a memory 430, and a communication bus 440 as shown in fig. 6, where the processor 410, the communication interface 420, and the memory 430 complete communication with each other through the communication bus 440, and the processor 410 may call logic instructions in the memory 430 to execute the above method. The embodiment does not limit a specific implementation form of the electronic device.
Further, the logic instructions in the memory 430 described above may be implemented in the form of software functional units and may be stored in a computer-readable storage medium when sold or used as a stand-alone product. Based on this understanding, the technical solution of the present invention may be embodied essentially or in a part contributing to the prior art or in a part of the technical solution, in the form of a software product stored in a storage medium, comprising several instructions for causing a computer device (which may be a personal computer, a server, a network device, etc.) to perform all or part of the steps of the method according to the embodiments of the present invention. And the aforementioned storage medium includes: a U-disk, a removable hard disk, a Read-Only Memory (ROM), a random access Memory (RAM, random Access Memory), a magnetic disk, or an optical disk, or other various media capable of storing program codes.
Further, an embodiment of the present invention discloses a computer program product, including a computer program stored on a non-transitory computer readable storage medium, the computer program including program instructions, which when executed by a computer, enable the computer to perform the face recognition method provided in the above-described method embodiments, the method including acquiring a face speckle image and a reference speckle image corresponding to the face speckle image; inputting the face speckle image and the reference speckle image into a face recognition model to obtain the face characteristics of the face speckle image; the face recognition model comprises a parallax network and a recognition network; inputting the face speckle image and the reference speckle image into a face recognition model, and acquiring the face characteristics of the face speckle image comprises the following steps: inputting the face speckle image and the reference speckle image into a parallax network to obtain a parallax image; obtaining a depth map based on the parallax map; inputting the depth map into a recognition network to obtain the face characteristics of the face speckle image; the face recognition model is obtained by training a face speckle sample image serving as a sample and a predetermined face sample labeling result corresponding to the face speckle sample image serving as a sample label, and the sample face features corresponding to the face speckle sample image correspond to the face sample labeling result.
In another aspect, an embodiment of the present invention further provides a non-transitory computer readable storage medium, on which a computer program is stored, the computer program being implemented when executed by a processor to perform the face recognition method provided in the above embodiments, the method including obtaining a face speckle image and a reference speckle image corresponding to the face speckle image; inputting the face speckle image and the reference speckle image into a face recognition model to obtain the face characteristics of the face speckle image; the face recognition model comprises a parallax network and a recognition network; inputting the face speckle image and the reference speckle image into a face recognition model, and acquiring the face characteristics of the face speckle image comprises the following steps: inputting the face speckle image and the reference speckle image into a parallax network to obtain a parallax image; obtaining a depth map based on the parallax map; inputting the depth map into a recognition network to obtain the face characteristics of the face speckle image; the face recognition model is obtained by training a face speckle sample image serving as a sample and a predetermined face sample labeling result corresponding to the face speckle sample image serving as a sample label, and the sample face features corresponding to the face speckle sample image correspond to the face sample labeling result.
The apparatus embodiments described above are merely illustrative, wherein the elements illustrated as separate elements may or may not be physically separate, and the elements shown as elements may or may not be physical elements, may be located in one place, or may be distributed over a plurality of network elements. Some or all of the modules may be selected according to actual needs to achieve the purpose of the solution of this embodiment. Those of ordinary skill in the art will understand and implement the present invention without undue burden.
From the above description of the embodiments, it will be apparent to those skilled in the art that the embodiments may be implemented by means of software plus necessary general hardware platforms, or of course may be implemented by means of hardware. Based on this understanding, the foregoing technical solution may be embodied essentially or in a part contributing to the prior art in the form of a software product, which may be stored in a computer readable storage medium, such as ROM/RAM, a magnetic disk, an optical disk, etc., including several instructions for causing a computer device (which may be a personal computer, a server, or a network device, etc.) to execute the method described in the respective embodiments or some parts of the embodiments.
Finally, it should be noted that: the above embodiments are only for illustrating the technical solution of the present invention, and are not limiting; although the invention has been described in detail with reference to the foregoing embodiments, it will be understood by those of ordinary skill in the art that: the technical scheme described in the foregoing embodiments can be modified or some technical features thereof can be replaced by equivalents; such modifications and substitutions do not depart from the spirit and scope of the technical solutions of the embodiments of the present invention.
Claims (8)
1. A face recognition method, comprising:
acquiring a face speckle image and a reference speckle image corresponding to the face speckle image;
inputting the face speckle image and the reference speckle image into a face recognition model to obtain face characteristics of the face speckle image;
the step of obtaining the face speckle image and the reference speckle image corresponding to the face speckle image comprises the following steps:
acquiring an original speckle image, an original color image and the reference speckle image;
acquiring a face boundary box based on the original color image;
acquiring an original speckle identification image based on the original speckle image and the face boundary box;
preprocessing the original speckle identification image to obtain the face speckle image;
the face recognition model comprises a parallax network and a recognition network;
the step of inputting the face speckle image and the reference speckle image into a face recognition model to obtain the face characteristics of the face speckle image comprises the following steps:
inputting the face speckle image and the reference speckle image into the parallax network to obtain a parallax image;
obtaining a depth map based on the parallax map;
inputting the depth map to the recognition network to obtain the face characteristics of the face speckle image;
the face recognition model is obtained by training a face speckle sample image serving as a sample and a predetermined face sample labeling result corresponding to the face speckle sample image serving as a sample label, wherein sample face features corresponding to the face speckle sample image correspond to the face sample labeling result;
the parallax network includes: the device comprises a feature extraction module, a three-dimensional matching cost aggregation module and a parallax calculation module;
the feature extraction module is used for extracting features of each pixel point corresponding to the speckle sample image;
the three-dimensional matching cost building module is used for calculating the matching cost value of each pixel point;
the stereo matching cost aggregation module is used for aggregating the matching cost values;
the parallax calculation module is used for calculating the parallax measurement value of each pixel point.
2. The face recognition method according to claim 1, wherein the determining step of the parallax network includes:
and training by taking the speckle sample image as a sample and taking a predetermined parallax true value of each pixel point corresponding to the speckle sample image as a sample label.
3. The face recognition method according to claim 2, wherein the determining step of the recognition network includes:
and fixing the trained parallax network, taking the face speckle sample image as a sample, and training by taking a predetermined face sample labeling result corresponding to the face speckle sample image as a sample label.
4. A face recognition method according to claim 3, wherein the parallax network is determined based on a Smooth-L1 Loss function and the recognition network is determined based on a Softmax-Loss function.
5. The face recognition method according to claim 1, wherein preprocessing the original speckle recognition image to obtain the face speckle image includes:
and carrying out local contrast normalization processing on the original speckle identification image to obtain the human face speckle image.
6. A face recognition device, comprising:
an acquisition unit, configured to acquire a face speckle image and a reference speckle image corresponding to the face speckle image;
the identification unit is used for inputting the face speckle image and the reference speckle image into a face recognition model to acquire the face characteristics of the face speckle image;
the acquisition unit is specifically configured to:
acquiring an original speckle image, an original color image and the reference speckle image;
acquiring a face boundary box based on the original color image;
acquiring an original speckle identification image based on the original speckle image and the face boundary box;
preprocessing the original speckle identification image to obtain the face speckle image;
the face recognition model comprises a parallax network and a recognition network;
the step of inputting the face speckle image and the reference speckle image into a face recognition model to obtain the face characteristics of the face speckle image comprises the following steps:
inputting the face speckle image and the reference speckle image into the parallax network to obtain a parallax image;
obtaining a depth map based on the parallax map;
inputting the depth map to the recognition network to obtain the face characteristics of the face speckle image;
the face recognition model is obtained by training a face speckle sample image serving as a sample and a predetermined face sample labeling result corresponding to the face speckle sample image serving as a sample label, wherein sample face features corresponding to the face speckle sample image correspond to the face sample labeling result;
the parallax network includes: the device comprises a feature extraction module, a three-dimensional matching cost aggregation module and a parallax calculation module;
the feature extraction module is used for extracting features of each pixel point corresponding to the speckle sample image;
the three-dimensional matching cost building module is used for calculating the matching cost value of each pixel point;
the stereo matching cost aggregation module is used for aggregating the matching cost values;
the parallax calculation module is used for calculating the parallax measurement value of each pixel point.
7. An electronic device comprising a memory, a processor and a computer program stored on the memory and executable on the processor, wherein the processor implements the steps of the face recognition method of any one of claims 1 to 5 when the program is executed by the processor.
8. A non-transitory computer readable storage medium having stored thereon a computer program, which when executed by a processor performs the steps of the face recognition method according to any one of claims 1 to 5.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010367239.8A CN111597933B (en) | 2020-04-30 | 2020-04-30 | Face recognition method and device |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010367239.8A CN111597933B (en) | 2020-04-30 | 2020-04-30 | Face recognition method and device |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN111597933A CN111597933A (en) | 2020-08-28 |
| CN111597933B true CN111597933B (en) | 2023-07-14 |
Family
ID=72185614
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202010367239.8A Active CN111597933B (en) | 2020-04-30 | 2020-04-30 | Face recognition method and device |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN111597933B (en) |
Families Citing this family (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN111932482B (en) * | 2020-09-25 | 2021-05-18 | 平安科技（深圳）有限公司 | Method and device for detecting target object in image, electronic equipment and storage medium |
| CN112861764B (en) * | 2021-02-25 | 2023-12-08 | 广州图语信息科技有限公司 | Face recognition living body judging method |
| CN113763450B (en) * | 2021-09-09 | 2022-05-31 | 合肥的卢深视科技有限公司 | Method and device for acquiring depth map, electronic equipment and storage medium |
| CN114117514B (en) * | 2021-10-29 | 2022-09-13 | 香港理工大学深圳研究院 | Encrypted face recognition method and system based on optical speckle |
Citations (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN106096551A (en) * | 2016-06-14 | 2016-11-09 | 湖南拓视觉信息技术有限公司 | The method and apparatus of face part Identification |
| CN107480613A (en) * | 2017-07-31 | 2017-12-15 | 广东欧珀移动通信有限公司 | Face identification method, device, mobile terminal and computer-readable recording medium |
| CN108734776A (en) * | 2018-05-23 | 2018-11-02 | 四川川大智胜软件股份有限公司 | A kind of three-dimensional facial reconstruction method and equipment based on speckle |
| CN108804895A (en) * | 2018-04-28 | 2018-11-13 | Oppo广东移动通信有限公司 | Image processing method, device, computer readable storage medium and electronic equipment |
| CN109145854A (en) * | 2018-08-31 | 2019-01-04 | 东南大学 | A kind of method for detecting human face based on concatenated convolutional neural network structure |
| CN109948573A (en) * | 2019-03-27 | 2019-06-28 | 厦门大学 | A kind of noise robustness face identification method based on cascade deep convolutional neural networks |
| CN110009691A (en) * | 2019-03-28 | 2019-07-12 | 北京清微智能科技有限公司 | Based on the matched anaglyph generation method of binocular stereo vision and system |
-
2020
- 2020-04-30 CN CN202010367239.8A patent/CN111597933B/en active Active
Patent Citations (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN106096551A (en) * | 2016-06-14 | 2016-11-09 | 湖南拓视觉信息技术有限公司 | The method and apparatus of face part Identification |
| CN107480613A (en) * | 2017-07-31 | 2017-12-15 | 广东欧珀移动通信有限公司 | Face identification method, device, mobile terminal and computer-readable recording medium |
| CN108804895A (en) * | 2018-04-28 | 2018-11-13 | Oppo广东移动通信有限公司 | Image processing method, device, computer readable storage medium and electronic equipment |
| CN108734776A (en) * | 2018-05-23 | 2018-11-02 | 四川川大智胜软件股份有限公司 | A kind of three-dimensional facial reconstruction method and equipment based on speckle |
| CN109145854A (en) * | 2018-08-31 | 2019-01-04 | 东南大学 | A kind of method for detecting human face based on concatenated convolutional neural network structure |
| CN109948573A (en) * | 2019-03-27 | 2019-06-28 | 厦门大学 | A kind of noise robustness face identification method based on cascade deep convolutional neural networks |
| CN110009691A (en) * | 2019-03-28 | 2019-07-12 | 北京清微智能科技有限公司 | Based on the matched anaglyph generation method of binocular stereo vision and system |
Non-Patent Citations (1)
| Title |
|---|
| Gernot Riegler等.Connecting the Dots: Learning Representations for Active Monocular Depth Estimation.《IEEE》.2019,第7624-7633页. * |
Also Published As
| Publication number | Publication date |
|---|---|
| CN111597933A (en) | 2020-08-28 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN111597933B (en) | Face recognition method and device | |
| CN111488756B (en) | Face recognition-based living body detection method, electronic device, and storage medium | |
| CN110674688B (en) | Face recognition model acquisition method, system and medium for video monitoring scene | |
| CN109800710B (en) | Pedestrian re-identification system and method | |
| US20220174089A1 (en) | Automatic identification and classification of adversarial attacks | |
| CN110082821B (en) | Label-frame-free microseism signal detection method and device | |
| CN111222395A (en) | Target detection method and device and electronic equipment | |
| CN111160249A (en) | Multi-class target detection method of optical remote sensing image based on cross-scale feature fusion | |
| CN110059728B (en) | RGB-D image visual saliency detection method based on attention model | |
| CN113706472B (en) | Highway pavement disease detection method, device, equipment and storage medium | |
| CN107767358B (en) | Method and device for determining ambiguity of object in image | |
| CN110532746B (en) | Face checking method, device, server and readable storage medium | |
| CN111160229A (en) | Video target detection method and device based on SSD (solid State disk) network | |
| CN111914762A (en) | Gait information-based identity recognition method and device | |
| CN111160100A (en) | Lightweight depth model aerial photography vehicle detection method based on sample generation | |
| CN111435445A (en) | Training method and device of character recognition model and character recognition method and device | |
| CN112733581A (en) | Vehicle attribute identification method and system | |
| CN114399480A (en) | Method and device for detecting severity of vegetable leaf disease | |
| CN113221842A (en) | Model training method, image recognition method, device, equipment and medium | |
| CN115620083B (en) | Model training method, face image quality evaluation method, equipment and medium | |
| CN111127327B (en) | Picture inclination detection method and device | |
| CN113554685A (en) | Method and device for detecting moving target of remote sensing satellite, electronic equipment and storage medium | |
| CN112801002A (en) | Facial expression recognition method and device based on complex scene and electronic equipment | |
| CN111860331A (en) | Unmanned aerial vehicle is at face identification system in unknown territory of security protection | |
| CN112464922B (en) | Human-vehicle weight recognition and model training method, device, equipment and storage medium thereof |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| TA01 | Transfer of patent application right |
Effective date of registration: 20220629 Address after: Room 611-217, R & D center building, China (Hefei) international intelligent voice Industrial Park, 3333 Xiyou Road, high tech Zone, Hefei City, Anhui Province Applicant after: Hefei lushenshi Technology Co.,Ltd. Address before: Room 3032, gate 6, block B, 768 Creative Industry Park, 5 Xueyuan Road, Haidian District, Beijing 100083 Applicant before: BEIJING DILUSENSE TECHNOLOGY CO.,LTD. Applicant before: Hefei lushenshi Technology Co.,Ltd. |
|
| TA01 | Transfer of patent application right | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |