WO2021003964A1

WO2021003964A1 - Method and apparatus for face shape recognition, electronic device and storage medium

Info

Publication number: WO2021003964A1
Application number: PCT/CN2019/121344
Authority: WO
Inventors: 王杉杉; 黄轩; 胡文泽; 王孝宇
Original assignee: 深圳云天励飞技术有限公司
Priority date: 2019-07-05
Filing date: 2019-11-27
Publication date: 2021-01-14
Also published as: CN110414370A; CN110414370B

Abstract

A method and apparatus for face shape recognition, an electronic device and a storage medium, the method comprising: acquiring a facial image to be recognized (S11); using a well pre-trained 3D reconstruction parameter extraction model to extract 3D reconstruction parameters and image features in the facial image (S12); reconstructing a 3D face according to the 3D reconstruction parameters (S13); extracting contour features in the 3D face (S14); constructing a joint feature according to the contour features and the image features (S15); and using a preconfigured face shape classifier to recognize the joint feature and obtain a face shape recognition result (S16). The 3D reconstruction of a face may be completed using only one facial image, and the face shape recognition result is highly reliable.

Description

Face recognition method, device, electronic equipment and storage medium

This application claims the priority of the Chinese patent application filed with the Chinese Patent Office on July 5, 2019, the application number is 201910606389.7, and the invention title is "face recognition method, device, electronic equipment, and storage medium". The entire content of the application is approved The reference is incorporated in this application.

Technical field

The present invention relates to the technical field of face recognition, in particular to a face recognition method, device, electronic equipment and storage medium.

Background technique

In recent years, with the increasing improvement of people's material living standards, people's demand for personal image design has grown rapidly. To provide a user with a personal image design usually requires first determining the user's face shape, and then selecting the appropriate hairstyle, makeup, glasses, clothing, accessories, etc. according to the user's face shape.

Based on this demand, the field of face recognition technology has also proposed some methods for detecting a person's face shape by recognizing the face in an image. However, the existing face recognition method can only obtain a better recognition result in the case of a frontal face. Due to the complexity of the real environment, often the captured images are not the face. Although the three-dimensional reconstruction of the face can solve the problem of turning the side face to the front face, the three-dimensional reconstruction requires a depth camera or shooting multiple face images under multiple viewing angles to complete, the reconstruction process is complicated, and the reliability of face recognition is low.

Therefore, how to recognize the facial shape of a human face only through a facial image has become a technical problem to be solved urgently.

Summary of the invention

In view of the above content, it is necessary to propose a face recognition, method, device, electronic device and storage medium, which can complete the 3D reconstruction of the face with only one face image, and the result of face recognition is highly reliable.

The first aspect of the present invention provides a face recognition method, the method includes:

Acquiring a face image of a person to be recognized;

Using a pre-trained 3D reconstruction parameter extraction model to extract the 3D reconstruction parameters and image features in the face image;

Reconstructing a 3D face based on the 3D reconstruction parameters;

Extracting contour features in the 3D face;

Constructing a joint feature based on the contour feature and the image feature;

The pre-set face shape classifier is used to recognize the joint feature, and the face shape recognition result is obtained.

In an optional embodiment, the extracting and extracting the 3D reconstruction parameters and image features from the face image using a pre-trained 3D reconstruction parameter extraction model includes:

Input the face image into the pre-trained 3D reconstruction parameter extraction model;

Acquiring the 3D reconstruction parameters output by the last layer of the 3D reconstruction parameter extraction model;

Obtain the image features output by the penultimate layer of the 3D reconstruction parameter extraction model.

In an optional embodiment, the 3D reconstruction parameters include: face reconstruction shape parameters and face reconstruction deformation parameters, and the reconstruction of the 3D face based on the 3D reconstruction parameters includes:

Obtain the reference vector and the average face;

Constructing a frontal face according to the face reconstruction shape parameter, the reference vector and the average face;

The frontal face is adjusted to an expressionless 3D face according to the face reconstruction deformation parameter.

In an optional embodiment, the extracting contour features in the 3D human face includes:

Acquiring geometric features in the 3D face and a first index corresponding to each geometric feature;

Filtering out a plurality of second indexes related to face contours from the first index;

Extracting geometric features corresponding to the plurality of second indexes as contour features of the 3D face.

In an optional embodiment, the constructing a joint feature based on the contour feature and the image feature includes:

Calculating the difference between the coordinate value corresponding to the contour feature and the coordinate value corresponding to the nose tip feature;

Connecting the contour feature after the difference calculation and the image feature to form a one-dimensional vector;

Use the one-dimensional vector as the joint feature.

In an optional embodiment, before acquiring the face image to be recognized, the method further includes:

Collect user images;

Detecting the face area in the user image;

Cut out the facial area of the human face to obtain a facial image of the human face.

In an optional embodiment, the recognition of the joint feature by using a preset face classifier to obtain a face recognition result includes:

Using the preset face classifier to recognize the joint feature;

Calculate the risk loss value through the gradient backhaul algorithm;

When the risk loss value reaches the minimum, the face recognition result is output as the face recognition result of the face image to be recognized.

A second aspect of the present invention provides a face recognition device, the device includes:

The acquisition module is used to acquire the face image to be recognized;

The detection module is configured to use a pre-trained 3D reconstruction parameter extraction model to extract the 3D reconstruction parameters and image features in the face image;

A reconstruction module for reconstructing a 3D face based on the 3D reconstruction parameters;

An extraction module for extracting contour features in the 3D face;

A construction module for constructing a joint feature based on the contour feature and the image feature;

The recognition module is used to recognize the joint feature by using a preset face classifier to obtain a face recognition result.

A third aspect of the present invention provides an electronic device, the electronic device includes a processor, and the processor is configured to implement the face recognition method when executing a computer program stored in a memory.

A fourth aspect of the present invention provides a computer-readable storage medium on which a computer program is stored, and the computer program is executed by a processor to realize the face recognition method.

In summary, the face recognition method, device, electronic device and storage medium of the present invention use a pre-trained 3D reconstruction parameter extraction model to extract the 3D reconstruction parameters and image features in the face image to be recognized, After reconstructing a 3D face based on the 3D reconstruction parameters, extracting contour features in the 3D face, and finally constructing a joint feature based on the contour feature and the image feature, and using a preset face classifier to recognize The combined features can obtain the face recognition result. When reconstructing a 3D face, only one face image is required, and there is no need for multiple face images under multiple viewing angles. Therefore, the process of reconstructing a 3D face is simple, with less calculation and faster recognition of the face shape. Fast; at the same time, the contour feature representing the geometric distribution information of the face and the image feature representing the texture information are connected together to construct a joint feature. The information is richer, so the result of identifying the face shape based on the joint feature is more reliable.

Description of the drawings

In order to explain the embodiments of the present invention or the technical solutions in the prior art more clearly, the following will briefly introduce the drawings that need to be used in the description of the embodiments or the prior art. Obviously, the drawings in the following description are only It is an embodiment of the present invention. For those of ordinary skill in the art, other drawings can be obtained based on the provided drawings without creative work.

FIG. 1 is a schematic diagram of the flow of face recognition according to a preferred embodiment of the present invention.

Fig. 2 is a schematic diagram of a network structure provided by an embodiment of the present invention.

FIG. 3 is a schematic diagram of the process of reconstructing a frontal and expressionless 3D human face according to a preferred embodiment of the present invention.

Fig. 4 is a structural diagram of a face recognition device provided by a preferred embodiment of the present invention.

Fig. 5 is a schematic diagram of an electronic device provided by a preferred embodiment of the present invention.

The following specific embodiments will further illustrate the present invention in conjunction with the above-mentioned drawings.

Detailed ways

Example one

Please refer to FIG. 1 to FIG. 3 at the same time. FIG. 1 is a schematic diagram of a face recognition process according to an embodiment of the present invention.

The face recognition method specifically includes the following steps. According to different requirements, the order of the steps in the flowchart can be changed, and some steps can be omitted.

S11: Acquire a face image to be recognized.

In this embodiment, if the face shape of a certain user is to be recognized, a face image of the user needs to be acquired first, and the face shape of the user is detected by recognizing the face image.

Wherein, the human face facial image is an image that only includes the human face facial area, and does not include body parts. By acquiring a face image that only includes the face area, the calculation of useless data (for example, pixels corresponding to body parts) can be reduced, which helps to improve the recognition speed of the face shape; and the face image It removes the interference of pixels corresponding to body parts, which helps to improve the accuracy of facial recognition.

In real life, the user may not actively cooperate, or need to covertly collect the user's image, the acquired image may include body parts. At this time, the acquired image needs to be processed to ensure that it is input to the pre-trained The image in the 3D reconstruction parameter extraction model is a face image that only includes the face area.

Therefore, in an optional embodiment, in order to ensure that the image input to the pre-trained 3D reconstruction parameter extraction model is a face image, before acquiring the face image to be recognized, the method further includes :

Collect user images;

Detecting the face area in the user image;

Wherein, the user image may be an image including only the face region of a person, or may be a half-length image or a full body image including other parts.

Regardless of whether the user image is a face image, a half-length image or a full-body image, a face detection algorithm, such as a face detection algorithm based on Haar-Like features, or an adaboost face detection algorithm, is used to detect the user image. And crop the detected face area from the user image as the face image.

S12, using a pre-trained 3D reconstruction parameter extraction model to extract 3D reconstruction parameters and image features in the face image.

In this embodiment, the 3D reconstruction parameter extraction model may be trained in advance based on the deep neural network. Preferably, the deep neural network is a deep separable convolutional neural network, for example, MobileNetV1, MobileNetV2, etc. The deep separable convolutional neural network is composed of deep separable convolutions. Except for the first input layer, it is fully convolutional. All layers are followed by a batchnorm (batch normalization: speed up the deep network by reducing internal covariate conversion) Training) and ReLU nonlinear activation function, the last fully connected layer without nonlinear activation function is directly sent to the softmax layer for classification.

Before training the 3D reconstruction parameter extraction model, it is necessary to obtain multiple face images and each face from an open source face database (for example, The 300 Videos in the Wild (300-VW)). 3D reconstruction parameters of the image, then the face image and the 3D reconstruction parameters are used as a data set, and the 3D reconstruction parameter extraction model is trained based on the data set. The face shape of the human face includes: square, triangle, ellipse, heart, circle, long and inverted triangle, etc. The 3D reconstruction parameters include: face reconstruction shape parameters, face reconstruction deformation parameters, and face position parameters. Wherein, the face position parameters include: face rotation matrix and face displacement. The face displacement refers to the face translation coefficient.

Since it is based on the 3D reconstruction parameter extraction model obtained by training the deep neural network based on the face image and 3D reconstruction parameters, input a face image into the 3D reconstruction parameter extraction model, and the 3D reconstruction parameter extraction model is sufficient The input face image is detected, and the 3D reconstruction parameter corresponding to the face image is output. Since the core idea of the present invention is not to train the 3D reconstruction parameter extraction model, the present invention does not specifically elaborate on the training process.

In an optional embodiment, the extraction of 3D reconstruction parameters and image features from the face image using a pre-trained 3D reconstruction parameter extraction model includes:

Generally speaking, the penultimate layer of any network model will calculate the feature map of the input penultimate layer and output the image features to the last layer for classification or detection. In this embodiment, as shown in Figure 2, the face image is input to the input layer of the pre-trained 3D reconstruction parameter extraction model, and the second-to-last layer of image feature values (located in the last The layer above the layer can be a pooling layer), the penultimate layer further calculates the input image features and outputs the image features with stronger characterization capabilities to the last layer (fully connected layer) to the input image features Perform extraction to obtain 3D reconstruction parameters. Therefore, the image features output by the penultimate layer of the 3D reconstruction parameter extraction model and the 3D reconstruction parameters output by the last layer can be obtained.

S13, reconstructing a 3D human face based on the 3D reconstruction parameters.

After the 3D reconstruction parameters are obtained, a 3D human face can be reconstructed based on the 3D reconstruction parameters. In this embodiment, a 3D Morphable model (3D Morphable model, 3DMM) and a 3D Blend Shape model (3D Blend Shape model, 3DBM) can be used to reconstruct a 3D face.

In an optional embodiment, since the 3D face reconstructed based on the 3D reconstruction parameters contains expression information, and the expression information will affect the recognition result of the face shape, in order to further reconstruct the face and expressionless For the face of, the reconstruction of the 3D face based on the 3D reconstruction parameters includes:

Obtain the reference vector and the average face;

Wherein, the reference vector includes the first feature vector of the 3D deformation model and the second feature vector of the 3D shape fusion model. Some open source 3DMMs will be released with an average face and a set of parameters used to represent the shape changes of the face under different conditions. 3DBM will be released with a set of expressions used to represent the face under different conditions. Changing parameters. The parameter representing the shape change of the human face under different conditions is defined as the first feature vector, and the parameter representing the expression change of the human face under different situations is defined as the second feature vector.

Specifically, a 3D face can be reconstructed based on the face reconstruction shape parameters, face reconstruction deformation parameters, face rotation matrix and face displacement, average face, first feature vector, and second feature vector through the following formula:

Wherein, Face _3d represents the reconstructed 3D face; R represents the face rotation matrix, which is set as the identity matrix;

Represents the average face; s _i represents the first feature vector;

3DMM_params represents the face reconstruction shape parameter; b _i represents the second feature vector, BlendShape_params represents the face reconstruction deformation parameter, set to 0; T represents the face displacement, set to 0; m represents The number of face reconstruction shape parameters, and n is the number of face reconstruction deformation parameters.

In the reconstruction process, the face rotation matrix R is set to the unit matrix, and the face displacement T is set to 0, the reconstructed 3D face can be rotated into a frontal face, and the deformation parameters of the face can be reconstructed at the same time Setting BlendShape params to 0 can eliminate the expression contained in the reconstructed 3D face, so the redirected 3D face Face _3d is a frontal and expressionless face.

Figure 3 shows the reconstruction process of a frontal and expressionless 3D face, where the picture on the left is a 3D face reconstructed based on the 3D reconstruction parameters, a face that is not frontal and contains an expression; the middle picture is Set the face rotation matrix R to the unit matrix and the face displacement T to 0 to obtain the frontal 3D face; the picture on the right is the expressionless face obtained after the face reconstruction deformation parameter BlendShape params is set to 0 3D human face.

Rotating the 3D face reconstructed based on the 3D reconstruction parameters into a frontal face solves the problem of rotating the side face to the frontal face. The frontal face can improve the recognition accuracy of the face shape; and then eliminate the frontal face. The expression solves the problem of facial expressions. The expressionless frontal face can further improve the accuracy of facial recognition, and the result of facial recognition is highly reliable.

S14: Extract contour features in the 3D face.

The face shape is reflected by the contour of the face, so it is necessary to extract the features of the cheek part of the 3D face. The features of the cheek part are called contour features.

In this embodiment, a frontal and expressionless 3D face is redirected through 3DMM and 3D BlendShape model, which contains more than 50,000 data points, and each data point contains x, y, z coordinates, and each data point is Both have an index. First, determine the key data points that need to be extracted, and then determine the target index corresponding to the key data point, and then extract the target geometric features corresponding to the target index from more than 50,000 data points, and extract the The target geometric features are used as contour features. Exemplarily, 128 key data points need to be extracted, these 128 key points are located in the cheek part of the human face, and the geometric features corresponding to the extracted 128 key data points are used as contour features of the 3D human face.

S15, constructing a joint feature based on the contour feature and the image feature.

Wherein, the joint feature refers to a feature vector obtained by connecting the contour feature and the image feature.

In an optional embodiment, in order to symmetrically distribute the cheek coordinates of a person, the constructing a joint feature based on the contour feature and the image feature includes:

Use the one-dimensional vector as the joint feature.

In this embodiment, according to the index corresponding to the tip of the nose (that is, the center point of the nose), the coordinates of the tip of the nose are extracted from the redirected 3D face without expression as the center point, and the point on the contour of the face is made difference from the center point. Value, that is, the difference between the coordinate value corresponding to the contour feature and the coordinate value corresponding to the nose tip feature to obtain the face contour point centered at 0 point. All contour features after the difference calculation are compressed into a one-dimensional feature vector, The data size is 128x3=364. If the image feature size is 1024, the joint feature constructed based on the contour feature and the image feature is a one-dimensional feature vector of 1388 columns.

The contour feature represents the geometric distribution information of the human face, and the image feature represents the texture information of the human face. Therefore, the constructed joint feature contains the geometric distribution information and texture information of the human face, and the information is more abundant.

S16: Recognizing the joint feature by using a preset face classifier to obtain a face recognition result.

In this embodiment, a face classifier can be preset. As shown in Figure 2, the face classifier includes two fully connected layers (FC1 layer and FC2 layer) and an activation layer (Activate Layer). The last layer It is the loss function layer (Softmax Loss).

Extract the 3D reconstruction parameters and image features in the face image to be recognized through the 3D reconstruction parameter extraction model, reconstruct the 3D face based on the 3D reconstruction parameters, and then extract the contour features in the 3D face, and finally the contour The feature and the image feature are connected as a joint feature and input to the face classifier to obtain the face recognition result.

Using the preset face classifier to recognize the joint feature;

Calculate the risk loss value through the gradient backhaul algorithm;

In this embodiment, when the risk loss value of the face classifier is minimized by the gradient backhaul algorithm, it indicates that the face classifier has stabilized, and the parameters of the face classifier have reached the optimal value. Value, the obtained face recognition result is the face recognition result of the face image to be recognized.

Regarding the gradient backhaul algorithm as the prior art, the present invention will not elaborate here.

It should be noted that during the recognition process, only the weights of the two fully connected layers (FC1 and FC2) in the face classifier need to be updated. The 3D reconstruction parameter extraction model is the trained model, so 3D The weights in the reconstruction parameter extraction model are not updated.

In summary, the face recognition method of the present invention uses a pre-trained 3D reconstruction parameter extraction model to extract the face image to be recognized, obtain 3D reconstruction parameters and image features, and reconstruct based on the 3D reconstruction parameters After the 3D face is extracted, the contour features in the 3D face are extracted, and finally a joint feature is constructed based on the contour feature and the image feature, and the joint feature is recognized by using a preset face classifier to obtain Face recognition result. When reconstructing a 3D face, only one face image is required, and there is no need for multiple face images under multiple viewing angles. Therefore, the process of reconstructing a 3D face is simple, with less calculation and faster recognition of the face shape. Fast; at the same time, the contour feature representing the geometric distribution information of the face and the image feature representing the texture information are connected together to construct a joint feature. The information is richer, so the result of identifying the face shape based on the joint feature is more reliable.

In addition, in the present invention, by setting the face reconstruction deformation parameter in the obtained 3D reconstruction parameters to 0, the face rotation matrix R to the identity matrix, and the face displacement T to 0, the constructed 3D face is frontal and expressionless. The face can further improve the reliability of the recognition result of the face shape.

The above-mentioned Figures 1-3 describe the face recognition method of the present invention in detail. In conjunction with Figures 4 and 5, the functional modules of the software system that implements the face recognition method and the face recognition method are respectively described The hardware system architecture is introduced.

It should be understood that the described embodiments are for illustrative purposes only, and are not limited by this structure in the scope of the patent application.

Example two

Refer to FIG. 4, which is a diagram of functional modules in a preferred embodiment of the face recognition device of the present invention.

In some embodiments, the face recognition device 40 runs in an electronic device. The face recognition device 40 may include multiple functional modules composed of program code segments. The program code of each program segment in the face recognition device 40 can be stored in the memory of the electronic device and executed by at least one processor to execute (see Figure 1 for details) the face shape Recognition function.

In this embodiment, the face recognition device 40 can be divided into multiple functional modules according to the functions it performs. The functional modules may include: an acquisition module 401, an acquisition module 402, a detection module 403, a training module 404, a reconstruction module 405, an extraction module 406, a construction module 407, and an identification module 408. The module referred to in the present invention refers to a series of computer program segments that can be executed by at least one processor and can complete fixed functions, and are stored in the memory. In this embodiment, the function of each module will be described in detail in subsequent embodiments.

The obtaining module 401 is used to obtain a face image to be recognized.

Wherein, the human face image is an image that only includes the human face and facial area, and does not include body parts. By acquiring a face image that only includes the face area, the calculation of useless data (for example, pixels corresponding to body parts) can be reduced, which helps to improve the recognition speed of the face shape; and the face image It removes the interference of pixels corresponding to body parts, which helps to improve the accuracy of facial recognition.

Therefore, in an optional embodiment, in order to ensure that the image input to the pre-trained 3D reconstruction parameter extraction model is a face image, before acquiring the face image to be recognized, the device 40 also Including: a collection module 402 for:

Collect user images;

Detecting the face area in the user image;

The detection module 403 is configured to use a pre-trained 3D reconstruction parameter extraction model to extract 3D reconstruction parameters and image features in the face image.

The training module 404 is used to train the 3D reconstruction parameter extraction model based on the deep neural network in advance. Preferably, the deep neural network is a deep separable convolutional neural network, for example, MobileNetV1, MobileNetV2, etc. The deep separable convolutional neural network is composed of deep separable convolutions. Except for the first input layer, it is fully convolutional. All layers are followed by a batchnorm (batch normalization: speed up the deep network by reducing internal covariate conversion) Training) and ReLU nonlinear activation function, the last fully connected layer without nonlinear activation function is directly sent to the softmax layer for classification.

In an optional embodiment, the detection module 403 using a pre-trained 3D reconstruction parameter extraction model to extract the 3D reconstruction parameters and image features in the face image includes:

The reconstruction module 405 is configured to reconstruct a 3D face based on the 3D reconstruction parameters.

Obtain the reference vector and the average face;

Specifically, the 3D face can be obtained by calculating the face reconstruction shape parameter, the face reconstruction deformation parameter, the face rotation matrix, the face displacement, the average face, the first feature vector, and the second feature vector through the following formula:

Represents the average face; s _i represents the first feature vector;

3DMM_params represents the face reconstruction shape parameter; b _i represents the second feature vector, BlendShape_params represents the face reconstruction deformation parameter, set to 0; T represents the face displacement, set to 0; m represents the The number of face reconstruction shape parameters, and n is the number of face reconstruction deformation parameters.

The extraction module 406 is configured to extract contour features in the 3D face.

In an optional embodiment, the extraction module 406 extracting contour features in the 3D face includes:

The construction module 407 is configured to construct a joint feature based on the contour feature and the image feature.

In an optional embodiment, in order to symmetrically distribute the cheek coordinates of a person, the construction module 407 constructing a joint feature based on the contour feature and the image feature includes:

Use the one-dimensional vector as the joint feature.

In this embodiment, according to the index corresponding to the tip of the nose (that is, the center point of the nose), the coordinates of the tip of the nose are extracted from the redirected 3D face without expression as the center point, and the point on the contour of the face is made difference from the center point. Value, that is, to make the difference between the coordinate value corresponding to the contour feature and the coordinate value corresponding to the nose tip feature to obtain a face contour point centered at 0 point and compress all contour features after the difference calculation into a one-dimensional feature vector, The data size is 128x3=364. If the image feature size is 1024, the joint feature constructed based on the contour feature and the image feature is a one-dimensional feature vector of 1388 columns.

The recognition module 408 is configured to recognize the joint feature using a preset face classifier to obtain a face recognition result.

In an optional embodiment, the recognition module 408 uses a preset face classifier to recognize the joint feature, and obtaining a face recognition result includes:

Using the preset face classifier to recognize the joint feature;

Calculate the risk loss value through the gradient backhaul algorithm;

In summary, the face recognition device of the present invention uses a pre-trained 3D reconstruction parameter extraction model to extract a face image to be recognized, obtains 3D reconstruction parameters and image features, and reconstructs based on the 3D reconstruction parameters After the 3D face is extracted, the contour features in the 3D face are extracted, and finally a joint feature is constructed based on the contour feature and the image feature, and the joint feature is recognized by using a preset face classifier to obtain Face recognition result. When reconstructing a 3D face, only one face image is required, and there is no need for multiple face images under multiple viewing angles. Therefore, the process of reconstructing a 3D face is simple, with less calculation and faster recognition of the face shape. Fast; at the same time, the contour feature representing the geometric distribution information of the face and the image feature representing the texture information are connected together to construct a joint feature. The information is richer, so the result of identifying the face shape based on the joint feature is more reliable.

Example three

Referring to FIG. 5, in a preferred embodiment of the present invention, the electronic device 5 includes a memory 51, at least one processor 52, at least one communication bus 53, and a display screen 54.

Those skilled in the art should understand that the structure of the electronic device shown in FIG. 5 does not constitute a limitation of the embodiment of the present invention. It may be a bus-type structure or a star structure. The electronic device 5 may also include a graph Show more or less other hardware or software, or different component arrangements.

In some embodiments, the electronic device 5 includes a device capable of automatically performing numerical calculation and/or information processing according to pre-set or stored instructions. The hardware of the electronic device 5 includes but is not limited to: a microprocessor, a dedicated (Application Specific Integrated Circuit, ASIC), a programmable gate array (Field-Programmable Gate Array, FPGA), and a digital processor (Digital Signal Processor, DSP) And embedded devices, etc. The electronic device 5 may also include user equipment. The user equipment includes, but is not limited to, any electronic product that can interact with the user through a keyboard, a mouse, a remote control, a touch panel, or a voice control device, for example, Personal computers, tablet computers, smart phones, digital cameras, etc.

It should be noted that the electronic device 5 is only an example. If other existing or future electronic products can be adapted to the present invention, they should also be included in the protection scope of the present invention and included here by reference. .

In some embodiments, the memory 51 is used to store program codes and various data, such as the face recognition device 40 installed in the electronic device 5, and realizes high-speed and automatic operation during the operation of the electronic device 5. Complete the program or data access. The memory 51 includes Read-Only Memory (ROM), Random Access Memory (RAM), Programmable Read-Only Memory (PROM), and erasable programmable read-only memory (Erasable Programmable Read-Only Memory, EPROM), One-time Programmable Read-Only Memory (OTPROM), Electronically-Erasable Programmable Read-Only Memory, EEPROM ), CD-ROM (Compact Disc Read-Only Memory) or other optical disk storage, magnetic disk storage, tape storage, or any other computer-readable medium that can be used to carry or store data.

In some embodiments, the at least one processor 52 may be composed, for example, may be composed of a single package, or may be composed of multiple packages with the same function or different functions, including one or more central processors. (Central Processing unit, CPU), a combination of microprocessors, digital processing chips, graphics processors, and various control chips. The at least one processor 52 is the control core (Control Unit) of the electronic device 5, which uses various interfaces and lines to connect the various components of the entire electronic device 5, and runs or executes the program stored in the memory 51 or Modules, and call the data stored in the memory 51 to perform various functions of the electronic device 5 and process data, for example, perform facial recognition functions.

In some embodiments, the at least one communication bus 53 is configured to implement connection and communication between the memory 51, the at least one processor 52, the display screen 54, and so on.

In some embodiments, the display screen 54 can be used to display information input by the viewer or information provided to the viewer and various graphical viewer interfaces of the electronic device 5. These graphical viewer interfaces can be composed of graphics, text, Icons, videos, and any combination of them. The display screen 54 may include a display panel. Optionally, the display panel may be configured in the form of a liquid crystal display (LCD), an organic light-emitting diode (OLED), etc.

The display screen 54 may also include a touch panel. If the display screen 54 includes a touch panel, the display screen 54 may be implemented as a touch screen to receive input signals from the viewer. The touch panel includes one or more touch sensors to sense touch, sliding, and gestures on the touch panel. The above-mentioned touch sensor may not only sense the boundary of the touch or sliding action, but also detect the duration and pressure related to the above-mentioned touch or sliding operation. The display panel and the touch panel can be used as two independent components to realize input and input functions, but in some embodiments, the display panel and the touch panel can be integrated to realize the input and output functions .

Although not shown, the electronic device 5 may also include a power source (such as a battery) for supplying power to various components. Preferably, the power source may be logically connected to the at least one processor 52 through a power management system, so as to be implemented through a power management system. Manage functions such as charging, discharging, and power management. The power supply may also include one or more DC or AC power supplies, recharging systems, power failure detection circuits, power converters or inverters, power supply status indicators and other arbitrary components. The electronic device 5 may also include various sensors, Bluetooth modules, communication modules, and so on. The present invention will not be repeated here.

The above-mentioned integrated unit implemented in the form of a software function module may be stored in a computer readable storage medium. The above-mentioned software function module is stored in a storage medium and includes several instructions to enable a computer device (which may be a personal computer, a client, or a network device, etc.) or a processor to execute the method described in each embodiment of the present invention part.

In a further embodiment, with reference to FIG. 1, the at least one processor 52 can execute the operating system of the electronic device 5 and various installed applications (such as the face recognition device 50), and program code Wait.

Program codes are stored in the memory 51, and the at least one processor 52 can call the program codes stored in the memory 51 to perform related functions. For example, the various modules described in FIG. 5 are program codes stored in the memory 51 and executed by the at least one processor 52, so as to realize the functions of the various modules.

In an embodiment of the present invention, the memory 51 stores a plurality of instructions, and the plurality of instructions are executed by the at least one processor 52 to realize the function of randomly generating a neural network model.

Specifically, for the specific implementation method of the at least one processor 52 on the foregoing instructions, reference may be made to the description of the relevant steps in the embodiment corresponding to FIG.

In the several embodiments provided by the present invention, it should be understood that the disclosed system, device, and method can be implemented in other ways. For example, the device embodiments described above are only illustrative. For example, the division of the modules is only a logical function division, and there may be other division methods in actual implementation.

Claims

A face recognition method, characterized in that the method includes:

Acquiring a face image of a person to be recognized;

Using a pre-trained 3D reconstruction parameter extraction model to extract the 3D reconstruction parameters and image features in the face image;

Reconstructing a 3D face based on the 3D reconstruction parameters;

Extracting contour features in the 3D face;

Constructing a joint feature based on the contour feature and the image feature;

The pre-set face shape classifier is used to recognize the joint feature, and the face shape recognition result is obtained.
5. The face recognition method according to claim 1, wherein said extracting 3D reconstruction parameters and image features from said face image using a pre-trained 3D reconstruction parameter extraction model comprises:

Inputting the human face image to the pre-trained 3D reconstruction parameter extraction model;

Acquiring the 3D reconstruction parameters output by the last layer of the 3D reconstruction parameter extraction model;

Obtain the image features output by the penultimate layer of the 3D reconstruction parameter extraction model.
The face recognition method of claim 1, wherein the 3D reconstruction parameters comprise: face reconstruction shape parameters and face reconstruction deformation parameters, and the reconstruction of the 3D face based on the 3D reconstruction parameters includes :

Obtain the reference vector and the average face;

Constructing a frontal face according to the face reconstruction shape parameter, the reference vector and the average face;

The frontal face is adjusted to an expressionless 3D face according to the face reconstruction deformation parameter.
The face recognition method of claim 1, wherein said extracting contour features in said 3D face comprises:

Acquiring geometric features in the 3D face and a first index corresponding to each geometric feature;

Filtering out a plurality of second indexes related to face contours from the first index;

Extracting geometric features corresponding to the plurality of second indexes as contour features of the 3D face.
5. The face recognition method of claim 4, wherein said constructing a joint feature based on said contour feature and said image feature comprises:

Calculating the difference between the coordinate value corresponding to the contour feature and the coordinate value corresponding to the nose tip feature;

Connecting the contour feature after the difference calculation and the image feature to form a one-dimensional vector;

Use the one-dimensional vector as the joint feature.
The face recognition method according to any one of claims 1 to 5, wherein before acquiring the face image to be recognized, the method further comprises:

Collect user images;

Detecting the face area in the user image;

Cut out the facial area of the human face to obtain a facial image of the human face.
5. The face recognition method according to any one of claims 1 to 5, wherein said adopting a preset face and face classifier to recognize said joint feature to obtain a face recognition result comprises:

Using the preset face classifier to recognize the joint feature;

Calculate the risk loss value through the gradient backhaul algorithm;

When the risk loss value reaches the minimum, the face recognition result is output as the face recognition result of the face image to be recognized.
A face recognition device, characterized in that the device comprises:

The acquisition module is used to acquire the face image to be recognized;

The detection module is configured to use a pre-trained 3D reconstruction parameter extraction model to extract the 3D reconstruction parameters and image features in the face image;

A reconstruction module for reconstructing a 3D face based on the 3D reconstruction parameters;

An extraction module for extracting contour features in the 3D face;

A construction module for constructing a joint feature based on the contour feature and the image feature;

The recognition module is used to recognize the joint feature by using a preset face classifier to obtain a face recognition result.
An electronic device, wherein the electronic device comprises a processor, and the processor is configured to implement the face recognition method according to any one of claims 1 to 7 when executing a computer program stored in a memory.
A computer-readable storage medium having a computer program stored on the computer-readable storage medium, wherein when the computer program is executed by a processor, the face shape according to any one of claims 1 to 7 is realized recognition methods.