WO2021042375A1

WO2021042375A1 - Face spoofing detection method, chip, and electronic device

Info

Publication number: WO2021042375A1
Application number: PCT/CN2019/104730
Authority: WO
Inventors: 吕萌; 潘雷雷
Original assignee: 深圳市汇顶科技股份有限公司
Priority date: 2019-09-06
Filing date: 2019-09-06
Publication date: 2021-03-11
Also published as: CN112997185A

Abstract

A face spoofing detection method, a chip, and an electronic device. The face spoofing detection method comprises: obtaining at least two three-dimensional image frames of a human face (101); determining the depth change information of the surface of the human face according to each three-dimensional images (102); and determining, according to the depth change information, whether the human face is a real human face (103). The method can accurately and easily implement face spoofing detection.

Description

Human face live detection method, chip and electronic equipment

Technical field

This application relates to the field of image recognition technology, and in particular to a method, chip, and electronic device for detecting a living body of a human face.

Background technique

Living body detection is an important function in face recognition related projects or products, mainly to distinguish whether the detected object is the face of a real person, or the face photo, mask, or 3D printed face model of the detected object.

Current face live detection usually uses live body detection based on captured face photos, and cooperates with live body detection through active response actions of the detected object.

Summary of the invention

The purpose of some embodiments of the present application is to provide a method, chip, and electronic device for detecting a human face, which can accurately and conveniently realize the detection of a human face.

An embodiment of the present application provides a method for detecting a living body of a human face, including: acquiring at least two frames of three-dimensional images of a human face; determining the depth change information of the human face surface according to each of the three-dimensional images; determining according to the depth change information Whether the human face is a real human face.

The embodiment of the present application also provides a chip, which is used to execute the above-mentioned method for detecting a living body of a human face.

An embodiment of the present application also provides an electronic device, including the above-mentioned chip.

For the prior art, the embodiments of the present application determine the depth change information of the human face surface according to the three-dimensional image of the human face, and determine whether it is a real human face according to the depth change information. On the one hand, the depth information of the face surface recorded by the three-dimensional image of the face is not affected by light and posture; on the other hand, even if the detected object is in the case of expressionless movements, subtle changes in facial muscles and tissues will also cause The depth of the face surface changes, so there is no need for the subject to take active actions to cooperate with the detection; therefore, the technical solution of the present application can accurately and conveniently perform live face detection, and the user experience is better. Moreover, the solution of this embodiment is based on the acquired three-dimensional image to determine, compared with the prior art that requires comparison with a preset image template, the dependence on pre-stored information is lower.

For example, the determining the depth change information of the human face surface according to each of the three-dimensional images includes: selecting at least one group of images from each of the three-dimensional images; wherein, each group of images includes two frames of the three-dimensional images. Image; according to the two frames of the three-dimensional images in each group of images, determine the depth change information corresponding to each group of images. In this embodiment, two frames of three-dimensional images are used as a group, and the depth change information corresponding to each group of images is determined.

For example, the number of frames of the three-dimensional image is denoted as n, and n is an integer greater than or equal to 3; the selection of at least one group of images from each of the three-dimensional images is specifically: from n frames of the three-dimensional images Select any two of the three-dimensional images as a group of images, and the number of selected image groups is

In this embodiment, it is determined according to n frames of three-dimensional images

Group images. In this way, as many image groups as possible can be obtained in the case of currently acquired 3D images, so as to fully compare the depth of each frame of 3D images, thereby improving the accuracy of the detection results.

For example, the depth change information corresponding to each group of images includes one or a combination of the following situations: feature values obtained according to the depth changes of the corresponding pixels on the two frames of the three-dimensional images in each group of images, two Frame the depth change trend of each pixel point corresponding to the three-dimensional image. This embodiment provides specific types of depth change information.

For example, the depth change information corresponding to each group of images includes feature values obtained according to the depth changes of the corresponding pixels on the two frames of the three-dimensional images in each group of images; the determining the depth change information according to the depth change information Declaring whether a human face is a real human face includes: judging whether there is a set of image feature values greater than a preset threshold; if it exists, determining that the human face is a real human face; if it does not exist, determining that the human face is non-existent Real face. This embodiment provides a specific way of judging whether it is a real human face according to the depth change information.

For example, the depth change information corresponding to each group of images further includes the depth change trends of the corresponding pixels on the two frames of the three-dimensional images; before the determining that the human face is a real human face, it further includes: judging Whether the depth change trend of a group of images matches the preset depth change trend that characterizes the face change caused by human exertion; if it does not exist, the face is determined to be a real face; if it exists, the face is determined It is a non-real face. This embodiment provides a way to recognize the change of the human face caused by the force applied by others, which can avoid the misjudgment caused by the change of the depth of the surface of the unreal human face by the artificial force as much as possible.

For example, after acquiring at least two frames of three-dimensional images of a human face, the method further includes: aligning each of the three-dimensional images with a human face; and determining the depth change information of the human face surface according to each of the three-dimensional images is specifically : Determine the depth change information of the face surface according to each of the three-dimensional images after the face is aligned. In this embodiment, performing face alignment on the three-dimensional image can improve the accuracy of the determined depth change information of the face surface, thereby improving the accuracy of face living detection.

Description of the drawings

One or more embodiments are exemplified by the pictures in the corresponding drawings. These exemplified descriptions do not constitute a limitation on the embodiments. The elements with the same reference numerals in the drawings are denoted as similar elements. Unless otherwise stated, the figures in the attached drawings do not constitute a scale limitation.

Fig. 1 is a flowchart of a method for detecting a human face in the first embodiment of the present application;

2 is a specific flow chart of the step of determining the depth change information of the human face surface according to each three-dimensional image according to the first embodiment of the present application;

FIG. 3 is a specific flowchart of the step of determining whether a human face is a real human face according to depth change information in the first embodiment of the present application;

Fig. 4 is a flowchart of a method for detecting a human face in a second embodiment of the present application;

Fig. 5 is a flowchart of an example of a method for detecting a human face in the third embodiment of the present application;

Fig. 6 is a flowchart of another example of a method for detecting a human face in the third embodiment of the present application;

Fig. 7 is a block diagram of an electronic device in a fifth embodiment according to the present application.

detailed description

In order to make the objectives, technical solutions, and advantages of the present application clearer, some embodiments of the present application will be described in further detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described here are only used to explain the application, and not used to limit the application. The following division of the various embodiments is for convenience of description, and should not constitute any limitation on the specific implementation of the present invention, and the various embodiments may be combined with each other without contradiction.

The inventor found that the prior art has at least the following problems: Face photos are easily affected by the ambient light and the posture of the detected object during shooting, which may easily lead to misjudgment; while the detected object actively responds to actions that require the cooperation of the detected object to complete. More cumbersome and poor user experience. Based on this, the inventor proposed the technical solution of this application.

The first embodiment of the present application relates to a face living detection method, which can be applied to any scene that requires the use of face recognition for identity verification, such as access control systems, payment systems, mobile phone unlocking systems, and so on.

FIG. 1 shows a flowchart of a method for detecting a human face according to a first embodiment of the application, and the details are as follows.

Step 101: Acquire at least two frames of three-dimensional images of a human face.

Step 102: Determine the depth change information of the human face surface according to each three-dimensional image.

Step 103: Determine whether the human face is a real human face according to the depth change information.

In step 101, at least two frames of three-dimensional images can be acquired through related three-dimensional imaging equipment such as dual-camera, structured light, or TOF; among them, the specific form of the three-dimensional image is not limited in this embodiment, for example, it can be a point cloud image or a depth image. , Or grid graph.

Among them, the muscles and tissues of different parts of the human face often change at different moments; the changes in facial muscles and tissues are very obvious when a person has expressions such as joy, anger, sadness, etc., even if the person is in a quiet state, The muscles and tissues of different parts of the face also undergo subtle changes. Each pixel in the three-dimensional image has a three-dimensional coordinate (x, y, z), where z represents the depth value of the pixel, even a slight change occurs in a certain place of the face, and that place in the multi-frame three-dimensional image The depth value of the pixel points will also change accordingly; and if it is a non-real face photo, mask, face model, etc., once it is formed, the depth of the surface will not change; therefore, it can be based on the face Whether the depth of the surface changes to determine whether the face is a real face. Wherein, the real human face in this embodiment refers to a living human face.

In an example, as shown in FIG. 2, step 102 includes the following sub-steps.

In sub-step 1021, at least one group of images is selected from the three-dimensional images; wherein each group of images includes two frames of three-dimensional images.

In sub-step 1022, the depth change information corresponding to each group of images is determined according to the two frames of three-dimensional images in each group of images.

Specifically, when the number of frames of the acquired three-dimensional image is two frames, the group of images selected in sub-step 1021 includes the two frames of images. When the number of frames n of the acquired three-dimensional image is greater than or equal to 3, any two three-dimensional images in each frame of the three-dimensional image can be regarded as a group, then the number of selected image groups can be

That is, if 2 frames of 3D images are selected as a combination (ie a group of images) from n frames of 3D images without repetition, the total number of combinations that can be obtained is

For example, three frames of 3D images photo1, photo2, photo3 are obtained; when any two frames of 3D images are used as a group, the number of groups that can be divided into is

That is, it can be divided into three groups, namely photo1 and photo2, photo2 and photo3, and photo1 and photo3. However, this embodiment does not limit how to group. In other examples, two adjacent frames of three-dimensional images can also be used as a group. When two adjacent frames of three-dimensional images are used as a group, they can be divided into two groups at this time. photo1 and photo2, photo2 and photo3; or it is also possible to select two frames from these three three-dimensional images as a group.

When each group of images is determined, the depth change information corresponding to each group of images is calculated according to the two frames of three-dimensional images in each group of images. Wherein, the depth change information corresponding to each group of images includes the feature value obtained according to the depth changes of the corresponding pixels on the two frames of three-dimensional images in each group of images; the feature value may be, for example, each corresponding to each of the two frames of three-dimensional images. The average value of the depth changes of the pixels, or the minimum variance of the depth changes of the corresponding pixels on two frames of three-dimensional images. Specifically, the three-dimensional coordinates of each pixel in each frame of the three-dimensional image are represented by x, y, and z, where z represents the depth value of the pixel; the corresponding pixel in the two three-dimensional images refers to x, y, and z in the two three-dimensional images. Two pixels with the same value of y; the depth change of corresponding pixels on two frames of three-dimensional images refers to the difference between the z values of two pixels with the same values of x and y in the two frames of images.

For example, the pixel point P1 (x1, y1, z1 _-1 ) on photo1 and the pixel point P1' (x1, y1, z1 _-2 ) on photo2 are corresponding pixels, then the corresponding pixel points P1, P1' The depth change between the time is z1 _-1 -z1 _-2 .

In this embodiment, the depth variation values of the corresponding pixels in the two frames of three-dimensional images are calculated, and then the average value or minimum variance is calculated; however, it is not limited to this. In other examples, you can select in advance. Multiple reference points are determined, for example, the pixel points of the nose area, glasses area, and mouth area are used as reference points, and the corresponding feature values of the depth changes of each reference point are calculated.

Wherein, each group of images determined in sub-step 1021 corresponds to a feature value of depth change.

In an example, as shown in FIG. 3, step 103 includes the following sub-steps.

Sub-step 1031, judging whether there is a group of image feature values greater than a preset threshold; if yes, go to sub-step 1032; if not, go to sub-step 1033.

In sub-step 1032, it is determined that the human face is a real human face.

In sub-step 1033, it is determined that the human face is an unreal human face.

Among them, the preset threshold value can be obtained according to the detection of real human faces in advance, that is, the characteristic value of the depth change of the real human face in the quiet state is detected, and set according to the detected characteristic value, for example, The preset threshold is set to be slightly smaller than the detected characteristic value.

If there are multiple sets of images, in sub-step 1031, the feature value of the depth change corresponding to each set of images can be compared with the preset threshold in turn, if the feature value of the depth change corresponding to the group of images currently being compared Greater than the preset threshold, it means that there is at least one set of images whose feature values are greater than the preset threshold. At this time, the face is determined to be a real face; if the feature values of the depth changes corresponding to each set of images are all less than or equal to the preset threshold, then Make sure that the face is not a real face.

Among them, step 103 can be implemented by, for example, a feature extraction algorithm, a machine learning algorithm, or a deep learning algorithm, which is not limited in this embodiment.

Compared with the prior art, this embodiment determines the depth change information of the human face surface according to the three-dimensional image of the human face, and determines whether it is a real human face according to the depth change information. On the one hand, the depth information of the face surface recorded by the three-dimensional image of the face is not affected by light and posture; on the other hand, even if the detected object is in the case of expressionless movements, subtle changes in facial muscles and tissues will also cause The depth of the face surface changes, so there is no need for the subject to take active actions to cooperate with the detection; therefore, the technical solution of the present application can accurately and conveniently perform live face detection, and the user experience is better. Moreover, the solution of this embodiment is based on the acquired three-dimensional image to determine, compared with the prior art that requires comparison with a preset image template, the dependence on pre-stored information is lower.

The second embodiment of the present application relates to a method for detecting a living body of a human face, and the specific process is shown in FIG. 4.

Step 201: Obtain at least two frames of three-dimensional images of a human face. This step is similar to step 101 in the first embodiment, and will not be repeated here.

Step 202: Determine the depth change information of the human face surface according to each three-dimensional image. This step is similar to step 102 in the first embodiment, and will not be repeated here.

Step 203: Determine whether the face is a real face according to the depth change information; Step 203 includes the following sub-steps:

Sub-step 2031, judging whether there is a group of image feature values greater than a preset threshold; if yes, go to sub-step 2032; if not, go to sub-step 2033. This step is similar to the sub-step 1031 in the first embodiment, and will not be repeated here.

In sub-step 2032, it is judged whether the depth change trend of a group of images matches the preset depth change trend that characterizes the face change caused by artificial force; if yes, go to sub-step 2033; if not, go to sub-step 2034.

In sub-step 2033, it is determined that the human face is an unreal human face. This step is similar to the sub-step 1033 in the first embodiment, and will not be repeated here.

In sub-step 2034, it is determined that the human face is a real human face. This step is similar to the sub-step 1032 in the first embodiment, and will not be repeated here.

Among them, in order to impersonate an unreal face as a real face, a malicious person may artificially force the unreal face to deform it, such as bending, squeezing, or poking the unreal face; among them, the unreal face includes, for example, Face photos, masks, face models, etc. However, the depth change of the face surface caused by artificial force has obvious characteristics. For example, when poking a non-real face, the depth change produced by the face surface is spread around the center of the poking position, and the depth change at the center position is the largest. The farther from the center position, the smaller the depth change; In the case of a real human face, the depth change may take the form of wavy lines.

In order to prevent misjudgment due to the malicious person's artificial force on the unreal face as much as possible, the depth change information corresponding to each group of images also includes the depth change of each pixel on the two three-dimensional images of each group of images. Trend; that is, the depth change information includes both the characteristic value of the depth change of each pixel on the two frames of three-dimensional images and the depth change trend of each pixel on the two frames of three-dimensional images. It should be noted that in other examples, the depth change information may also only include the depth change trend of each pixel point corresponding to the two frames of three-dimensional images.

Designers can simulate various man-made force actions on non-real faces in advance, and set the depth change trend that characterizes the changes in the face caused by man-made force based on the actual detected depth changes. As in the above example, when the artificial force action is poking, that is, when poking a non-real face, the corresponding depth change trend is to spread from the center to the surroundings, and the depth at the center changes the most. The farther from the center, the depth The smaller the change; when the artificial force action is squeezing, that is, when the unreal face is squeezed, the corresponding depth change trend is in a wave-shaped form. In addition, if the same person applies force to different non-real faces, the possible depth change trends will be different, such as poking face photos and poking face models, when the depth changes spread to the surrounding from the center position. The diffusion range or the speed of the depth change in diffusion may be different, which can be determined according to the actual detection data; therefore, it can also be classified according to the detection data, that is, not only can identify non-real faces, but also may identify which ones are. Non-real face.

In this embodiment, a method is provided for identifying the change of the human face caused by the application of force, which can avoid the misjudgment caused by the change of the depth of the surface of the unreal human face due to the artificial force.

The third embodiment of the present application relates to a method for detecting live human faces. FIG. 5 is a flowchart of an example in this embodiment, and the details are as follows.

Step 301: Obtain at least two frames of three-dimensional images of a human face. This step is similar to step 101 in the first embodiment, and will not be repeated here.

Step 302: align the faces of the three-dimensional images.

Step 303: Determine the depth change information of the face surface according to the three-dimensional images after the face alignment. This step is similar to step 102 in the first embodiment, and will not be repeated here.

Step 304: Determine whether the human face is a real human face according to the depth change information. This step is similar to step 103 in the first embodiment, and will not be repeated here.

In this example, step 302 is newly added; that is, after obtaining the three-dimensional image of the human face, the three-dimensional image is first aligned with the human face. Among them, in the process of acquiring the three-dimensional image, the detected object may have a posture change, so the angle of the face presented by each three-dimensional image may be different. In this embodiment, the five senses of the face can be aligned to align the three-dimensional images. The angle of the presented face is corrected to the same angle. Specifically, for example, an iterative closest point algorithm can be used to align faces. In this example, face alignment is performed on each three-dimensional image, and the depth change information of the face surface is determined according to each three-dimensional image after face alignment, which can improve the accuracy of the determined depth change information of the face surface, thereby improving Accuracy of live face detection.

Fig. 6 shows a flowchart of another example in this embodiment, which is specifically as follows.

Step 301: Obtain at least two frames of three-dimensional images of a human face.

Step 301-1: Perform image preprocessing on each three-dimensional image.

Step 302: Perform face alignment on each three-dimensional image after image preprocessing.

Step 303: Determine the depth change information of the face surface according to the three-dimensional images after the face alignment.

Step 304: Determine whether the human face is a real human face according to the depth change information.

Compared with the example in Figure 5, this example adds step 301-1; that is, after acquiring the three-dimensional image of the human face, first perform image preprocessing on each three-dimensional image, and then perform image preprocessing on the preprocessed three-dimensional images. The image is aligned with the face. Among them, image preprocessing includes deburring, filling holes, smoothing filtering and so on. In this example, performing image preprocessing on the three-dimensional image can improve the image quality, thereby improving the accuracy of face recognition.

The fourth embodiment of the present application relates to a chip, which is used to execute the above-mentioned face living detection method.

The fifth embodiment of the present application relates to an electronic device. As shown in FIG. 7, the electronic device includes the aforementioned chip 10; the electronic device may also include a three-dimensional imaging device 20 and a memory 30 connected to the chip. The chip 10 obtains a three-dimensional image of a human face through a three-dimensional imaging device 20; the memory 30 is used to store instructions that can be executed by the chip, and when the instructions are executed by the chip, the chip can perform the above-mentioned method for detecting live human faces. The memory 30 can also be used to store the acquired three-dimensional image and various data generated by the chip 10 executing the above-mentioned face living detection method. The electronic device may be, for example, a face recognition device in an access control system, a payment system, or a mobile phone unlocking system.

A person of ordinary skill in the art can understand that the above-mentioned embodiments are specific embodiments for realizing the present application, and in practical applications, various changes can be made to them in form and details without departing from the spirit and spirit of the present application. range.

Claims

A method for detecting live human face, which is characterized in that it comprises:

Acquiring at least two frames of three-dimensional images of the human face;

Determining the depth change information of the human face surface according to each of the three-dimensional images;

Determine whether the human face is a real human face according to the depth change information.
The method according to claim 1, wherein the determining the depth change information of the human face surface according to each of the three-dimensional images comprises:

Select at least one group of images from each of the three-dimensional images; wherein each group of images includes two frames of the three-dimensional images;

Determine the depth change information corresponding to each group of images according to the two frames of the three-dimensional images in each group of images.
The method of claim 2, wherein the number of frames of the three-dimensional image is denoted as n, and n is an integer greater than or equal to 3; and at least one group of images is selected from each of the three-dimensional images , Specifically: selecting any two of the three-dimensional images from n frames of the three-dimensional images as a group of images, and the number of selected image groups is
The method according to claim 2, wherein the depth change information corresponding to each group of images includes one or a combination of the following situations: according to each group of images corresponding to each of the two frames of the three-dimensional image The feature value obtained from the depth change of the pixel, and the depth change trend of each pixel on the two frames of the three-dimensional images in each group of images.
The method according to claim 4, wherein the depth change information corresponding to each group of images comprises feature values obtained according to the depth changes of the corresponding pixels on the two frames of the three-dimensional images in each group of images The determining whether the human face is a real human face according to the depth change information includes:

It is judged whether there is a group of image feature values greater than a preset threshold; if it exists, it is determined that the human face is a real human face; if it does not exist, the human face is determined to be an unreal human face.
The method according to claim 5, wherein the depth change information corresponding to each group of images further comprises the depth change trend of each pixel point corresponding to the two frames of the three-dimensional image in each group of images; Before determining that the human face is a real human face, it also includes:

Determine whether the depth change trend of a set of images matches the preset depth change trend that characterizes the face change caused by human exertion; if it does not exist, determine that the face is a real face; if it exists, determine the person The face is not a real human face.
The method according to claim 4, wherein the characteristic value includes one or a combination of the following situations: the average value of the depth changes of the corresponding pixels on the two frames of the three-dimensional image, and the two frames of the The minimum variance of the depth change of each pixel on the three-dimensional image.
The method according to claim 1, wherein after said obtaining at least two frames of three-dimensional images of the human face, the method further comprises: aligning each of the three-dimensional images with the human face;

The determining the depth change information of the face surface according to each of the three-dimensional images is specifically: determining the depth change information of the face surface according to each of the three-dimensional images after the face is aligned.
The method according to claim 8, wherein after said acquiring at least two frames of three-dimensional images of the human face, the method further comprises: performing image preprocessing on each of the three-dimensional images;

The performing face alignment on each of the three-dimensional images specifically includes performing face alignment on each of the three-dimensional images after image preprocessing.
The method of claim 1, wherein the three-dimensional image is a point cloud image, or a depth image, or a grid image.
A chip, which is characterized in that it is used to implement the method for detecting a living body of a human face according to any one of claims 1 to 10.
An electronic device, characterized by comprising the chip according to claim 11.