WO2019080579A1

WO2019080579A1 - 3d face identity authentication method and apparatus

Info

Publication number: WO2019080579A1
Application number: PCT/CN2018/098442
Authority: WO
Inventors: 肖振中; 黄源浩
Original assignee: 深圳奥比中光科技有限公司
Priority date: 2017-10-26
Filing date: 2018-08-03
Publication date: 2019-05-02
Also published as: CN107633165A; CN107633165B

Abstract

Provided are a 3D face identity authentication method and apparatus. The method comprises the following steps: obtaining a depth image and a two-dimensional image which include a target face; performing registration on the depth image and a reference face 3D texture image, so as to obtain pose information of the target face; using the reference face 3D texture image to project a reference face two-dimensional image according to the pose information; performing similarity comparison on a target face two-dimensional image and the reference face two-dimensional image. The method combines 3D information and projection to obtain a complete reference face two-dimensional image in order to increase recognition accuracy. The method also comprises steps of eye line of sight detection, liveness detection and data updating in order to improve user examination, reduce false recognition rates and tackle problems such as facial changes.

Description

3D face identity authentication method and device

Technical field

The invention belongs to the technical field of computers, and more particularly to a 3D face identity authentication method and device.

Background technique

The human body has many unique features, such as faces, fingerprints, irises, human ears, etc. These features are collectively referred to as biometrics. Biometrics are widely used in security, home, intelligent hardware and many other fields. At present, more mature biometrics such as fingerprint recognition and iris recognition have been widely used in mobile phones, computers and other terminals. For the features such as face, although the related research has been very deep, the recognition of features such as face is still not popular, mainly because the existing recognition methods have limitations that result in lower recognition rate and recognition stability. These limitations mainly include the influence of ambient light intensity and illumination direction, the recognition rate of facial expressions, and the vulnerability of artificial features.

The recognition of existing features such as face is mainly based on the two-dimensional color image of the face. When the ambient light intensity is weak, the recognition effect will be seriously affected. In addition, when the direction of the light is different, there will be shadows on the face image, which will also affect the recognition effect. When the referenced face image is acquired without an expression, and the face image currently acquired under the smiling expression, the effect of face recognition also decreases. In addition, if the recognized object is not a real face but a two-dimensional face image, it can often be recognized.

For the above problems, biometric recognition based on near-infrared or thermal infrared images is generally used. The near-infrared image can be improved by the interference of ambient light, but it is difficult to solve the problem of artificial feature deception. The thermal infrared image is only The real face is imaged, so the problem of artificial feature deception can be solved. However, the thermal infrared image has low resolution, which seriously affects the recognition effect.

Based on the above description, there is still a lack of a more comprehensive biometric solution for performing tasks such as unlocking and payment.

Summary of the invention

In order to solve the problem of lacking a comprehensive face recognition scheme in the prior art, the present invention provides a task execution method based on face recognition.

In order to solve the above problems, the technical solution adopted by the present invention is as follows:

The present invention provides a 3D face identity authentication method and apparatus, including the steps of: acquiring a depth image and a two-dimensional image including a target face; and registering the depth image with a reference face 3D texture image to obtain Determining the posture information of the target human face; projecting the reference human face 3D texture image into the reference human face two-dimensional image according to the posture information; and performing the target human face two-dimensional image and the reference human face two-dimensional image Similarity comparison.

In one embodiment, the method further includes the steps of: detecting the human eye of the target face using the depth image and/or the two-dimensional image independently of steps (b)-(d), when the person When the eye line direction coincides with the preset direction, proceed to step (b) or (c) or (d).

In one embodiment, the method further includes the steps of: detecting, by the depth image and/or the two-dimensional image, whether the target face is a real face, independent of steps (b)-(d), if true The face is: continue to perform step (b) or (c) or (d), or when the similarity exceeds a preset first threshold, the authentication passes.

In an embodiment, the method further includes the step of: updating the corresponding texture information in the reference face 3D texture image by using the target face two-dimensional image when the similarity exceeds a preset second threshold.

In one embodiment, the reference face 3D texture image is obtained by acquiring a depth image sequence including a reference face and a two-dimensional image sequence; and calculating the reference face 3D texture image. The 3D texture image includes a 3D point cloud or a 3D mesh with texture information.

In one embodiment, the projection refers to projecting the 3D texture image onto a 2D plane to form a human face two-dimensional image.

In one embodiment, the face pose in the reference face two-dimensional image is consistent with the face pose in the target face two-dimensional image.

In one embodiment, the two-dimensional image comprises an infrared image.

In one embodiment, the two-dimensional image comprises a structured light image.

The present invention also provides a 3D face identity authentication device, comprising: a depth camera for acquiring a depth image including a target face; a plane camera for acquiring a two-dimensional image including the target face; Receiving the depth image and the two-dimensional image, and performing an operation of: registering the depth image with a reference face 3D texture image to acquire posture information of the target face; The face 3D texture image projects a reference face two-dimensional image according to the posture information; and compares the target face two-dimensional image with the reference face two-dimensional image.

In one embodiment, the processor further performs: detecting, by the depth image and/or the two-dimensional image, a human eye line of sight of the target face, when the human eye line of sight and preset When the directions are the same, continue with other operations.

In one embodiment, the processor further performs: using the depth image and/or the two-dimensional image, detecting whether the target face is a real face, and if it is a real face: continuing to execute Other operations, or when the similarity exceeds a preset first threshold, the authentication passes.

In one embodiment, the processor further performs an operation of updating a corresponding texture in the reference face 3D texture image by using the target face two-dimensional image when the similarity exceeds a preset second threshold information.

DRAWINGS

FIG. 1 is a schematic diagram of a 3D face identity authentication scenario according to an embodiment of the present invention.

2 is a schematic diagram of a 3D face identity entry method in accordance with one embodiment of the present invention.

3 is a schematic diagram of a 3D face identity entry and authentication method in accordance with one embodiment of the present invention.

4 is a schematic diagram of a 3D face identity authentication method according to still another embodiment of the present invention.

FIG. 5 is a schematic diagram of a 3D face identity entry and authentication method according to still another embodiment of the present invention.

FIG. 6 is a schematic diagram of a 3D face identity entry and authentication method according to another embodiment of the present invention.

7 is a schematic diagram of a 3D face identity authentication device in accordance with one implementation of the present invention.

Detailed ways

The present invention will be further described in detail below with reference to the accompanying drawings and embodiments. It is understood that the specific embodiments described herein are merely illustrative of the invention and are not intended to limit the invention.

It is to be noted that when an element is referred to as being "fixed" or "in" another element, it can be directly on the other element or indirectly. When an element is referred to as being "connected" to another element, it can be directly connected to the other element or indirectly connected to the other element. In addition, the connection can be for a fixed effect or for circuit communication.

It should be understood that the terms "length", "width", "upper", "lower", "front", "back", "left", "right", "vertical", "horizontal", "top" The orientation or positional relationship of the "bottom", "inside", "outside" and the like is based on the orientation or positional relationship shown in the drawings, and is merely for the convenience of describing the embodiments of the present invention and the simplified description, rather than indicating or implying The device or component must have a particular orientation, configuration and operation in a particular orientation, and thus is not to be construed as limiting the invention.

Moreover, the terms "first" and "second" are used for descriptive purposes only and are not to be construed as indicating or implying a relative importance or implicitly indicating the number of technical features indicated. Thus, features defining "first" or "second" may include one or more of the features either explicitly or implicitly. In the description of the embodiments of the present invention, the meaning of "a plurality" is two or more, unless specifically defined otherwise.

Face authentication technology can be used for security check and monitoring. At present, with the popularity of smart terminals such as mobile phones and tablets, face identity can also be applied to unlocking, paying, and even entertainment games. Intelligent terminal devices, such as mobile phones, tablets, computers, televisions, etc., are mostly equipped with color cameras. After capturing images containing faces using color cameras, the images are used for face detection and recognition, thereby further utilizing the results of the recognition to perform other related application. However, for mobile terminal devices such as mobile phones and tablets, the application environment often changes, and environmental changes may affect the imaging of color cameras. For example, when the light is weak, the face cannot be well imaged. On the other hand, the color camera cannot recognize whether the recognized object is a real face.

The invention provides a 3D face identity authentication method and device. The depth image and the two-dimensional image that are insensitive to ambient illumination will be used to realize the functions of inputting, detecting and recognizing the face identity, and the biometric detection based on the depth image and the two-dimensional image is used to avoid the false recognition of the false face. . The two-dimensional image here may be an infrared image, an ultraviolet image, etc., and the corresponding acquisition camera may be a flat camera such as an infrared camera or an ultraviolet camera. In the following description, an infrared image will be described as an example.

FIG. 1 is a schematic diagram of a 3D face identity authentication scenario according to an embodiment of the present invention. The user 10 holds the face authentication device 11 (mobile terminal, such as a mobile phone, a tablet, etc.), and the device 11 is internally provided with a depth camera 111 and an infrared camera 112. When the device 11 faces the face, the depth camera 111 is used to acquire the The depth image of the target face, the infrared camera 112 is used to acquire an infrared image containing the target face. The device 11 needs to record and save the reference face information into the device 11 for subsequent authentication comparison before performing the face identity authentication; in the face identity authentication phase, the device 11 will collect the depth image of the current target face. And the infrared image, and extracting feature information of the current target face based on the depth image and the infrared image, when the feature information matches the entered reference face information, the face identity authentication succeeds, otherwise it fails. The "reference face" and the "target face" mentioned above are relative to the two different stages of face identity entry and authentication. The difference is only to show that the essence of face identity authentication is verification. Whether the target face is the same as the reference face,

3D face identity entry and authentication

2 is a schematic diagram of a 3D face identity entry method in accordance with one embodiment of the present invention. The method includes the following steps:

201: Acquire a depth image sequence and an infrared image sequence.

In this step, the depth image sequence and the infrared image sequence including the reference face are acquired by the depth camera 111 and the infrared camera 112. The sequence image is because the single image cannot contain the information of the entire face, so the collection needs to be included. The sequence image of all parts of the face, the depth image and the infrared image can be collected simultaneously or in time. In the acquisition, one way is that the device 11 does not move, the face constantly changes direction to collect a sequence image containing all the face parts information; the other way is that the face is not moving, and the device 11 collects all the movements by including A sequence image of face part information, it is understood that any other means can be applied to the present invention.

When collecting the depth image and the infrared image sequence including the reference face, any one of the images in the sequence preferably has at least partially coincides with the face region included in at least one of the other images, and the image of the coincident portion is beneficial for subsequent images. Image fusion. For example, three images are respectively collected on the left side, the middle side, and the right side of the face, wherein the middle image and the left and right sides have a common face area in the image.

Generally, the captured depth image or the infrared image includes both the face and the background. Therefore, in this step, the face needs to be detected to divide the step. For example, for the depth image, the depth information may be The face is segmented, and for the infrared image, a contour recognition based method or a machine learning based method such as an Adaboost algorithm or a neural network based detection method may be utilized. It will be appreciated that any suitable method of face detection can be applied to the present invention.

In this embodiment, the depth image and the infrared image are registered images (for details, as described later), so that only one of the images may be detected in face detection, and the face in the other image may be based on Correspondence can be obtained directly. For example, using the learned neural network model to perform face detection and segmentation on the infrared image to obtain a new infrared image with some or all of the background removed, and then a new depth image can be obtained according to the correspondence between the depth image and the infrared image. In one embodiment, a detection method that combines two images more efficiently will be employed, first based on the depth value of the corresponding pixel in the depth image; secondly, based on the depth value and the lens parameter of the infrared camera, the depth value can be estimated. The size of the face region is finally selected. On the infrared image, the infrared image region of the size of the face region corresponding to the depth value is selected as the object on the infrared image as the object for face determination. Due to the face detection of infrared images in the traditional method, the size of the face region needs to be subjected to a certain number of iterations to achieve the best effect, and the method directly determines the size by using the depth information, thereby speeding up the face detection speed. .

202: Calculate a 3D texture image of the face.

In this step, the face depth image sequence obtained in the previous step is first merged into an overall face 3D point cloud model. In one embodiment, the depth image sequence is merged into a 3D image by using an ICP (Iterative Nearest Point) algorithm, which is a face 3D point cloud model, by the ICP (Iterative Nearest Point) algorithm, the paper "Kinectfusion" The Kinectfusion method described in Real-time 3D reconstruction and interaction using a moving depth camera" can be used in the present invention. In some embodiments, considering that some facial expression changes occur in the image sequence collection process, a dynamic fusion algorithm can be used to acquire a 3D point cloud model of the face, such as the paper "Dynamicfusion reconstruction and tracking of nonrigid scenes in realtime". The Dynamicfusion algorithm in " can be used in the present invention. In some embodiments, considering the 3D point cloud model is rich in noise and large amount of data, it is also necessary to convert the 3D point cloud model into a 3D mesh model, and any suitable mesh generation algorithm can be applied to the present invention. . In the following description, the 3D point cloud model or the 3D mesh model is uniformly expressed by 3D images.

Secondly, the texture information contained in the infrared image is assigned to the 3D image to obtain a 3D texture image. In the case of infrared image and depth image registration, each pixel on the depth image contains not only the pixel value representing the depth, but also the pixel value representing the texture information, so after the 3D image is obtained, each point in the 3D image is obtained ( The node) is assigned a pixel value representing the texture information to obtain a 3D texture image.

203: Projecting an infrared image of a face.

In this step, a two-dimensional human face infrared image is obtained by projecting a 3D texture image onto a two-dimensional plane. Considering that the following face contains the most feature information, in one embodiment, in order to obtain a frontal full face infrared image, the frontal orientation of the face is first obtained according to the 3D information in the 3D texture image, and then the 3D texture image is projected. A full frontal face infrared image is obtained on a 2-dimensional plane perpendicular to the orientation. It can be understood that after acquiring the 3D texture image, the complete human face infrared image at any viewing angle can be obtained by projecting to the two-dimensional plane. It should be noted that in order to distinguish the originally acquired infrared image from the projected or transformed infrared image, the latter is uniformly represented by the "face infrared image" in the present invention to distinguish the "infrared image".

204: Extract face feature information.

Using the positive face infrared image obtained in the previous step, the feature extraction algorithm is used to extract the facial feature information. In one embodiment, the face infrared image is placed into a pre-learned neural network (such as a convolutional neural network CNN, etc.) to output feature information of the face.

205: Enter face feature information.

The extracted face feature information is saved to the device 11 as an identity authentication feature of the reference face for subsequent target face identity authentication comparison.

It can be understood that the above face identity entry method can also be used for face identity authentication. FIG. 3 is a schematic diagram of a 3D face identity entry and authentication method according to an embodiment of the present invention. The authentication step includes: acquiring a depth image sequence of the target face and an infrared image sequence; calculating a 3D texture image of the target face based on the sequence; projecting the 3D texture image of the target face into the front face infrared image and based on the infrared The image extracts feature information of the target face; unlike the face feature entry, finally, the feature information of the target face is compared with the feature information of the reference face to determine whether it is the same face. In the projection step, it is not necessary to project a frontal face infrared image, as long as both are recorded in the same plane (direction) during the entry and authentication phases.

However, using this method to collect multiple images of the current user during face authentication requires more time and brings a poor user experience, so a faster speed and experience will be provided in the present invention. Better face authentication method.

4 and FIG. 5 are schematic diagrams of a 3D face identity entry and authentication method according to an embodiment of the present invention, and the face identity entry method corresponding to the 3D face identity argumentation method shown in FIG. 4 is the same as the embodiment shown in FIG. See Figure 5 for details. This certification method includes the following steps:

401: Acquire a depth image and an infrared image.

In this step, the depth camera 111 and the infrared camera 112 collect a depth image and an infrared image including the target face. Unlike the face entry method, in the face authentication phase, it is not necessary to collect multiple images. In the embodiment, only one depth image and the infrared image are collected, which can also speed up the face identity authentication and give the user a better experience. It can be understood that multiple images can also be acquired in other embodiments, but the multiple images here are still relatively small compared to the sequence containing all the information of the face in the face entry stage. In the following description, a single depth image and a single infrared image will be described.

After acquiring the depth image and the infrared image including the face, similar to step 201, this step generally includes a face detection and segmentation step, and finally obtains a depth image and an infrared image from which part or all of the background is removed.

402: Detecting the human eye.

The human eye line of sight indicates the position of the current target human eye's attention, and line of sight detection is increasingly used in many applications. In this embodiment, in order to improve the user's physical examination, human eye line of sight detection is also performed. It can be understood that the human eye line of sight detection step may also not be applied to the 3D face identity authentication. In addition, the human eye line of sight detection step may also be placed between other steps in the embodiment, that is, the human eye line of sight detection step. It is relatively independent of other steps, and can perform this step according to different application requirements and obtain the human eyesight detection result.

The use of one of the face depth image, the face infrared image, or a combination of the two can be used to detect the human eye line of sight. In the present embodiment, a combination of a depth image and an infrared image is preferably employed to detect a human eye line of sight. Firstly, the 3D information of the human face (such as a 3D point cloud) is calculated by using the depth image, and information such as face orientation and key point 3D coordinates can be acquired according to the 3D information; secondly, the details of the human eye are identified according to the infrared image. For example, the pupil center, the scintillation point (the fixed spot formed by the cornea reflected by the human eye after the infrared light is fixed in the infrared camera), the pupil, the iris, etc., further based on the 3D information of the face and the infrared image and the depth image The relationship between the two (coincidents, or the correspondence between the pixels of the two images after registration) can obtain the 3D coordinates of the human eye detail features; finally, the human eye line of sight is calculated by combining the 3D coordinates of one or more human eye detail features. direction.

A human eye line of sight detection method known in the art can also be applied to the present invention, for example, it is possible to perform line of sight detection or the like using only an infrared image.

The human eye line of sight detection can further enhance the physical examination of the user's face identity authentication. For example, in the embodiment shown in FIG. 1, when the human eye does not look at the device 11 and the face is collected by the depth camera 111 and the infrared camera 112. At this time, the certification performed is often not the subjective will of the user, but a mis-authentication. Therefore, in some applications, the human eye line of sight detection can be used as a separate step to detect the human eye line of sight, and other steps can be based on the results of the human eye detection in this step to determine whether or not to perform or perform which method.

In this embodiment, the next step is performed when the human eye line of sight is detected to be the same as the preset line of sight direction, where the preset line of sight direction generally refers to the human eye gaze direction or attention in the current 3D face identity authentication application. For example, in the device 11, the face authentication application displayed on the screen, such as unlocking, paying, etc., in some embodiments, the preset line of sight direction may also refer to other directions, such as the direction of the pointing device 11.

It will be appreciated that the human eye line of sight detection step can also be applied to the embodiment shown in FIG.

In this embodiment, since only a single acquisition is performed on the current face, a single depth image or an infrared image often includes only part of the face information, and the single image is directly extracted and matched with the reference facial feature information. Often the accuracy is not high. Therefore, in this embodiment, the posture information of the current target face is acquired by using the depth image; secondly, the target face infrared image is aligned and corrected based on the posture information, and the purpose of the correction is to acquire the current current posture of the reference human face. Infrared image of the face, which can eliminate the face image recognition error caused by different postures to the greatest extent; finally, the corrected target face image is extracted from the face feature, and the feature is compared with the feature of the reference face image. Make a comparison for certification. Let's take a closer look at these steps:

403: Calculate face posture information.

In the face entry phase, a 3D image of the reference face (such as a 3D point cloud, a 3D mesh, etc.) has been saved, so in this step, the target face depth image acquired in step 301 and the 3D image of the reference face are performed. Alignment, in one embodiment, the ICP algorithm is used to achieve the alignment of the two, and after the alignment operation, the posture information of the current target face relative to the reference face can be obtained.

In some embodiments, a 3D image of a standard human face may also be used, and the 3D image of the standard human face is used as a 3D image of the reference human face for calculating the posture information of the target human face.

404: Align the face infrared image.

After acquiring the posture information of the current target face, correcting the target facial infrared image based on the posture information to obtain the current target facial infrared image with the same reference facial infrared image posture obtained in step 203, preferably, reference The face infrared image is a frontal face image, so the purpose of the correction is to obtain a frontal face infrared image of the current target face. The gesture-based face image alignment algorithm of the prior art can be applied to the present invention, such as the method described in the paper "DeepFace Closing the Gap to Human Level performance in face verification".

405: Extract face feature information.

For the target face infrared image obtained in the previous step, the feature extraction algorithm is used to extract the facial feature information. In one embodiment, the target face infrared image is placed in the same neural grid as used in the entry phase to output feature information having a similar structure.

406: Face authentication.

The feature information of the current target face obtained in the previous step is compared with the feature information of the target face obtained in the entry phase to determine whether it is the same face. The comparison here generally outputs a similarity. When the similarity exceeds a preset threshold, such as a threshold of 80%, it is considered to be the same face, otherwise it is a different face.

FIG. 5 is a schematic diagram of the above 3D face identity entry and authentication method. It should be noted that, in addition to the obtained reference face feature information recorded and saved in the entry phase, in fact, a 3D image (3D point cloud/grid) of the reference face needs to be entered and saved, so as to be in the authentication stage. The 3D image is called when calculating the pose of the target face.

In the embodiment corresponding to FIG. 4 and FIG. 5, the accuracy of the 3D face identity authentication is greatly dependent on the alignment and correction accuracy of the face infrared image, since only a single image or a few infrared images are acquired during the authentication phase. When the acquired infrared image pose is relatively biased, such as when the head is raised or the side face is more, even if it is converted into the same target face infrared image by the alignment and correction by the alignment and correction, the alignment and correction are performed. It is impossible for the algorithm to recover the loss of features due to the side face. Based on this, the present invention also provides a more accurate 3D face identity entry and authentication method.

FIG. 6 is a schematic diagram of a 3D face identity entry and authentication method according to still another embodiment of the present invention.

In the entry phase, the depth image sequence of the reference face and the infrared image sequence are first obtained, and then the 3D texture image including the 3D point cloud/grid and the infrared texture information is calculated, and finally the 3D texture image is recorded and stored in the storage device. in.

In the authentication phase, the depth image of the target face and the infrared image are first acquired. This step often requires face detection and image segmentation to obtain the face image. Secondly, the line of sight detection is performed when the detected eye line direction is greater than the preset. When the threshold is reached, the next step is to match (or align, register) the depth image with the saved 3D point cloud/grid of the reference face to obtain the posture information of the target face; and then refer to the face according to the posture information. The 3D texture image is projected to obtain the same reference face infrared image as the target face pose; the reference face infrared image and the target face infrared image are again placed into the neural grid to extract the respective facial features. Information; finally, the face features are compared and the comparison results are output.

In this embodiment, by acquiring the posture information of the target face and projecting the 3D texture image of the reference face according to the posture, a face infrared image closest to the target face pose is obtained. Compared with the method shown in FIG. 4 and FIG. 5, the method does not need to change the posture of the infrared image by the alignment and correction algorithm. Since the 3D texture image contains all the information including the reference face, the reference person obtained by the projection is obtained. The infrared image of the face can ensure the highest similarity with the infrared image of the target face, which is beneficial to improve the authentication accuracy.

In addition, in the embodiment, since the feature images of the reference face and the target face are extracted and compared in the authentication stage, a depth learning algorithm for similarity judgment may be trained in the algorithm selection. The two images, the output of the algorithm is the similarity, which can speed up the authentication.

Live detection

The face authentication method described above is often easily "spoofed". For example, using a 2D image or a three-dimensional model of a face, the image or the three-dimensional model is used as a target face, and the above method may be used for authentication success, which is in some cases. Applications such as unlocking and payment based on face authentication are unacceptable. Based on the problem, the 3D face identity authentication method provided by the present invention may further include a living body detecting step, which is used to determine whether the current target face is a real face, and only the target face and the reference face are similar. When the preset threshold is exceeded and the target face is a real face, it will be authenticated, otherwise it will fail.

There are various methods for detecting a living body. In one embodiment, it may be determined based on the acquired target face depth image whether it is a stereoscopic object to solve the "spoofing" caused by the 2D image. In one embodiment, the extracted infrared image may be utilized. The implicit skin characteristics of the face are used to determine the skin to solve the "spoofing" caused by the general three-dimensional model. However, there is still a lack of an effective living detection method that can cope with various "spoofing" situations. The present invention will provide an algorithm to solve this problem.

The living body detection method in the present invention is based on a deep learning algorithm. In one embodiment, the model is trained by constructing a neural grid model and using a large amount of data. A large amount of data here includes depth images of real people, 2D photos, simulation masks, 3D models, and infrared images. The larger the amount of data, the more accurate the trained neural mesh model will be. The trained neural grid can very accurately find real faces from various false faces, thus achieving live detection. In one embodiment, the acquired target face depth image and the infrared image are input into the neural grid to output a result of whether it is a real face; in another embodiment, only the depth image or the infrared image may be used. Enter into the neural grid to output the result of whether it is a real face.

Therefore, in the 3D face authentication process, only the similarity between the current target face and the reference face exceeds a preset threshold and the authentication is successful by the living body detection, otherwise it fails. In some embodiments, the living body detecting step may also be performed after the depth image and the infrared image are acquired, and the correlation detecting step is performed after the living body detection passes, so the living body detecting step is relative to the acquiring the depth image and the infrared image. The external steps are also relatively independent, and the step can be performed before any step and the next step is determined based on the result of the living body detection.

It can be understood that when the similarity detection is lower than the preset threshold, the living body detecting step may not be performed. In addition, the living body detecting step can also be performed by the feature extraction and comparison steps, that is, the similarity detection of the target face is performed only when the living body detection passes.

Data Update

Considering that the face may change with time, such as changes caused by biological growth, illness, etc., the accumulation of changes over time will become more obvious, in order to cope with the accuracy impact of these changes on the 3D face identity authentication algorithm. In the present invention, the 3D face identity authentication algorithm may further include a data update step to deal with face changes.

In the various embodiments described above, the authentication is performed when the target human face and the reference face similarity exceed a certain threshold and are detected by the living body. It is conceivable that if the entered reference face information is always the same, when the target face changes with time, the similarity will become lower and lower until there is misunderstanding, that is, the current target person cannot be distinguished. The face is the original reference face. In order to cope with this problem, after the 3D face authentication is passed, when the similarity is higher than another threshold, the current target face information is used as the new reference face information, and the reference face information is continuously updated, even if Faces can be accurately authenticated after a large change in face. It should be noted that the step of updating the information indicates that the corresponding threshold should generally be higher than the threshold in the face authentication determination step.

According to different authentication methods, the meaning of the update reference face information mentioned here is also different. For example, in the embodiment shown in FIG. 3, in addition to updating the face infrared image feature information, the face 3D texture image may be updated; for the embodiment shown in FIG. 5, the feature information of the face infrared image is updated. , the target face feature information is used as the new reference face feature information to implement data update; and in the embodiment shown in FIG. 6 , the updated 3D texture image of the face is the target face 2D image. The texture information replaces the corresponding texture information in the original reference face 3D texture image.

3D face identity authentication device

FIG. 7 is a schematic diagram of a 3D face identity authentication apparatus according to an implementation of the present invention. The device 11 includes a projection module 702 for capturing an infrared structured light image to a target space, an acquisition module 707 for collecting a structured light image, and a device 11 further including a processor (in the figure) Not shown), the processor calculates a depth image of the target after receiving the structured light image. The structured light image herein includes face texture information in addition to the structured light information, so the structured light image can also participate in face identity entry and authentication as a face infrared image and a depth image. At this time, the acquisition module 707 is a part of the depth camera 111 in FIG. 1 and is also an infrared camera 112. In other words, the depth camera and the infrared camera here can be considered to be the same camera.

In some embodiments, the device 11 further includes an infrared floodlight 706 that can emit infrared light of the same wavelength as the structured light emitted by the projection module 702. In the process of performing face entry and authentication, The projection module 702 and the infrared floodlight 706 are time-division switches to respectively acquire the depth image and the infrared image of the target. The infrared image acquired at this time is a pure infrared image, and the face feature information contained in the structured light image is more obvious, and the face authentication accuracy is higher.

In some embodiments, a depth camera based on TOF (Time Flight Method) technology may be utilized, where projection module 702 is used to emit light pulses, and acquisition module 707 is used to receive pulse pulses, and the processor is used to record pulse transmissions and receptions. The time used, the depth image of the target is calculated based on the time. At this time, the acquisition module 707 can simultaneously acquire the depth image and the infrared image of the target, and there is no parallax between the two.

In some embodiments, an additional infrared camera 703 can be provided for acquiring an infrared image. When the wavelength of the beam emitted by the infrared floodlight 706 is different from the wavelength of the beam emitted by the projection module 702, the acquisition module 707 can be used synchronously. The infrared camera 703 acquires a depth image and an infrared image of the target. This device differs from the previous ones in that, since the depth image is different from the camera of the infrared image, there is a parallax between the two, and when an image having no parallax is required in the calculation processing performed by the subsequent face authentication The depth image needs to be registered with the infrared image in advance.

Device 11 may also include handset 704, ambient light/proximity sensor 705, etc. to achieve more functionality. For example, in some embodiments, considering the harm of the infrared light to the human body, when the face is too close, the proximity of the face can be detected by the proximity sensor 705, and when the face is too close, the projection module 702 is closed. Project or reduce the projection power. In some embodiments, the face authentication and the handset can be combined to implement an automatic call. For example, when the device is a communication device, after the device receives the incoming call, the face authentication application is activated and the required depth camera and the infrared camera are used to capture the depth image. Infrared image, when the authentication is passed, connect the call and open the device such as the handset to realize the call.

The device 11 can also include a screen 701 that can be used to display image content as well as for touch interaction. The function of the screen unlocking of the device can be implemented by using the face authentication method. In one embodiment, when the device 11 is in a state of sleep or the like, the user recognizes that the inertial measurement unit in the device 11 is picked up by picking up the device 11. The acceleration will light up the screen, and the screen will appear to be unlocked. At this time, the device turns on the depth camera and the infrared camera is used to capture the depth image and/or the infrared image. When the face is detected in the captured image, the initiator is activated. Face authentication application. In the human eye sight detection process in the face authentication process, the preset human eye line of sight direction may be set to the direction in which the human eye looks at the screen 701, that is, the face authentication and unlocking are further performed only when the human eye looks at the screen.

The device 11 also includes a memory (not shown) for storing feature information entered during the entry phase, and for storing applications, instructions, and the like. For example, the 3D face identity entry and authentication method described above is saved to the memory in the form of a software program. When the application needs it, the processor calls the instructions in the memory and performs the entry and authentication methods. It can be understood that the 3D face identity entry and authentication method can also be directly written into the processor in the form of instruction code, thereby improving execution efficiency. In addition, as technology continues to evolve, the boundaries between software and hardware will gradually disappear. Therefore, the 3D face identity entry and authentication method described in the present invention may be configured in the form of software or hardware. in.

The above is a further detailed description of the present invention in connection with the specific preferred embodiments, and the specific embodiments of the present invention are not limited to the description. It will be apparent to those skilled in the art that <RTIgt; </ RTI> <RTIgt; </ RTI> <RTIgt; </ RTI> <RTIgt; </ RTI> <RTIgt;

Claims

A 3D face identity authentication method, comprising the steps of:

(a) acquiring a depth image including the target face and a two-dimensional image;

(b) registering the depth image with a reference face 3D texture image to obtain posture information of the target face;

(c) projecting the reference face 3D texture image into the reference face two-dimensional image according to the posture information;

(d) comparing the similarity of the target human face two-dimensional image with the reference human face two-dimensional image.
The method of claim 1 further comprising the step of:

(e1) detecting the human eye line of the target face using the depth image and/or the two-dimensional image independently of steps (b) to (d), when the human eye line of sight direction and the preset direction If they are consistent, proceed to step (b) or (c) or (d).
The method of claim 1 further comprising the step of:

(e2) independently of steps (b) to (d), using the depth image and/or the two-dimensional image to detect whether the target face is a real face, and if it is a real face:

Continue with step (b) or (c) or (d), or

When the similarity exceeds a preset first threshold, the authentication passes.
The method of claim 1 further comprising the step of:

(e3) updating the corresponding texture information in the reference face 3D texture image by using the target face two-dimensional image when the similarity exceeds a preset second threshold.
The method according to any one of claims 1 to 4, wherein the reference face 3D texture image is obtained by the following method:

(1) acquiring a depth image sequence including a reference face and a two-dimensional image sequence;

(2) Calculate the reference face 3D texture image.
The method of claim 5 wherein the 3D texture image comprises a 3D point cloud or a 3D mesh with texture information.
The method of claim 1 wherein said projecting is to project said 3D texture image onto a 2D plane to form a two-dimensional image of a human face.
The method according to claim 1, wherein the face pose in the reference face two-dimensional image coincides with the face pose in the target face two-dimensional image.
The method of claims 1-4, wherein the two-dimensional image comprises an infrared image.
The method of claims 1-4 wherein the two-dimensional image comprises a structured light image.
A 3D face identity authentication device, comprising:

a depth camera for acquiring a depth image containing a target face;

a planar camera for acquiring a two-dimensional image containing a target face;

The processor receives the depth image and the two-dimensional image and performs the following operations:

Registering the depth image with a reference face 3D texture image to obtain posture information of the target face;

And projecting the reference face 3D texture image into the reference face two-dimensional image according to the posture information;

Comparing the target human face two-dimensional image with the reference human face two-dimensional image.
The apparatus of claim 11 wherein said processor further performs the following operations:

Using the depth image and/or the two-dimensional image, the human eye line of sight of the target face is detected, and when the line of sight direction of the human eye coincides with the preset direction, other operations are continued.
The apparatus of claim 11 wherein said processor further performs the following operations:

Using the depth image and/or the two-dimensional image, detecting whether the target face is a real face, and if it is a real face:

Continue to perform other operations, or

When the similarity exceeds a preset first threshold, the authentication passes.
The apparatus of claim 11 wherein said processor further performs the following operations:

And updating the corresponding texture information in the reference face 3D texture image by using the target face two-dimensional image when the similarity exceeds a preset second threshold.