WO2022111512A1 - Facial liveness detection method and apparatus, and device - Google Patents

Abstract

Embodiments of the present application provide a facial liveness detection method and apparatus, and a device. The method comprises: acquiring an infrared image of an object to be detected; extracting a facial local region from the acquired infrared image to obtain a target facial local region image; and determining a grayscale feature of the target facial local region image, and performing liveness detection according to the determined grayscale feature and a pre-trained liveness detection model to obtain a facial liveness detection result of said object. According to the embodiments of the present application, liveness detection based on facial local features is achieved, thereby avoiding the problem of a decrease in the accuracy of the liveness detection result due to that a face is partially blocked.

Description

Face liveness detection method, device and equipment

This application claims the priority of the Chinese patent application with the application number of 202011349369.5, which was filed with the China Patent Office on November 26, 2020, and the application name is "Facial Liveness Detection Method, Apparatus and Equipment", the entire contents of which are incorporated herein by reference. middle.

technical field

The present application relates to the technical field of biometrics, and in particular, to a method, device and device for detecting a face living body.

Background technique

Liveness detection is not unfamiliar to people, and can be used to prevent criminals from maliciously using forged biometrics of others for identity authentication, such as using stolen photos, videos recorded online, and prosthetic masks made for identity authentication. In the current live detection process, the detection is usually based on the overall characteristics of the face. In the above scheme, when the detected user does not completely remove the mask, scarf, etc., or the face is partially occluded due to other factors, the detected The accuracy will decrease, so it is particularly important to perform live detection based on the local features of the face.

SUMMARY OF THE INVENTION

The purpose of the embodiments of the present application is to provide a method, device, and device for detecting a living body of a human face, so as to realize the detection of living body based on a local area of a human face.

In order to solve the above-mentioned technical problems, the embodiments of the present application are implemented as follows:

In a first aspect, an embodiment of the present application provides a face liveness detection method, including:

Obtain the infrared image of the object to be detected;

Extracting the partial area of the face on the infrared image to obtain an image of the partial area of the target face;

Determine the grayscale feature of the target face local area image;

The living body detection process is performed according to the grayscale feature and the pre-trained living body detection model, and the face living body detection result of the object to be detected is obtained.

In a second aspect, an embodiment of the present application provides a face liveness detection device, including:

an image sensor for acquiring an infrared image of the object to be detected;

a processor, configured to perform extraction processing on the infrared image of the partial region of the face, to obtain an image of the partial region of the target face; determine the grayscale feature of the image of the partial region of the target face; according to the grayscale feature and pre-training The living body detection model is used for detection processing to obtain the face living body detection result of the object to be detected.

In a third aspect, an embodiment of the present application provides an electronic device, including: a processor, a communication interface, a memory, and a communication bus; wherein the processor, the communication interface, and the memory communicate with each other through the bus; the memory is used to store A computer program; a processor for executing the program stored in the memory to realize the steps of the above-mentioned method for detecting a living body of a human face.

In a fourth aspect, an embodiment of the present application provides a computer-readable storage medium, where a computer program is stored on the computer-readable storage medium, and when the computer program is executed by a processor, implements the steps of the above-mentioned method for detecting a living body of a human face.

In a fifth aspect, an embodiment of the present application provides a face liveness detection system, the system includes: a data processor and an image sensor that supports collecting infrared images, wherein:

The image sensor is used to collect the infrared image of the object to be detected, and send the collected infrared image to the data processor;

The data processor is configured to receive the infrared image collected by the image sensor, perform extraction processing of the partial area of the face on the infrared image, and obtain an image of the partial area of the target face; determine the image of the partial area of the target face. Grayscale feature; perform liveness detection processing according to the grayscale feature and the pre-trained liveness detection model, and obtain the face liveness detection result of the object to be detected.

In the embodiment of the present application, an infrared image of the object to be detected is extracted and processed to obtain a partial face region image of the target face; and based on the grayscale features of the target face partial region image and a pre-trained living body detection model A living body detection process is performed to determine whether the object to be detected is a living body of a human face. As a result, accurate and effective living body detection is realized based on the local features of the face, and the problems such as the decrease in the accuracy of the detection results caused by the occlusion of part of the face area are effectively solved.

Description of drawings

In order to more clearly illustrate the embodiments of the present application and the technical solutions of the prior art, the following briefly introduces the drawings required in the embodiments and the prior art. Obviously, the drawings in the following description are only the For some embodiments of the application, for those of ordinary skill in the art, other embodiments can also be obtained according to these drawings without creative efforts.

1 is a schematic flowchart of a first method for detecting a human face liveness provided in an embodiment of the present application;

2 is a schematic diagram of the eye region of a living body and a prosthesis in an infrared image provided by an embodiment of the present application;

3 is a schematic diagram of the mouth region of a living body and a prosthesis in an infrared image provided by an embodiment of the present application;

4 is a schematic flowchart of a second method for detecting a human face liveness provided by an embodiment of the present application;

FIG. 5 is a schematic flowchart of a third face liveness detection method according to an embodiment of the present application;

FIG. 6 is a schematic flowchart of a fourth method for detecting a living body of a human face provided by an embodiment of the present application;

7 is a schematic flowchart of a fifth method for detecting liveness of a human face provided by an embodiment of the present application;

FIG. 8 is a schematic flowchart of a sixth method for detecting liveness of a human face provided by an embodiment of the present application;

FIG. 9 is a schematic flowchart of a seventh method for detecting a living body of a human face provided by an embodiment of the present application;

10 is a schematic flowchart of an eighth method for detecting liveness of a human face provided by an embodiment of the present application;

FIG. 11 is a schematic flowchart of a ninth face liveness detection method provided by an embodiment of the present application;

FIG. 12 is a schematic flowchart of a tenth face liveness detection method provided by an embodiment of the present application;

FIG. 13 is a schematic diagram of the module composition of a face liveness detection apparatus provided by an embodiment of the application;

FIG. 14 is a schematic diagram of the composition of an electronic device according to an embodiment of the present application.

Detailed ways

In order to make the objectives, technical solutions, and advantages of the present application more clearly understood, the present application will be described in further detail below with reference to the accompanying drawings and examples. Obviously, the described embodiments are only a part of the embodiments of the present application, but not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without creative work fall within the protection scope of the present application.

FIG. 1 is a schematic flowchart of a first face liveness detection method provided by an embodiment of the present application. Referring to FIG. 1 , the method may specifically include the following steps:

Step 102, acquiring an infrared image of the object to be detected.

The face liveness detection method provided by the embodiments of the present application can be applied to a face liveness detection apparatus, and the face liveness detection apparatus includes an image sensor, such as a 2-megapixel image sensor. The face liveness detection device can be applied to various systems that require liveness detection, such as access control systems, attendance systems, and security systems. When it is necessary to determine whether the object to be detected is a living body, the face detection device collects an infrared image of the object to be detected.

It should be pointed out that, considering that the face in the infrared image corresponding to the infrared light band is less affected by external light, the features are stable, and the information of local areas such as eyes and mouth is obvious. Therefore, the infrared light band is used in the embodiment of the present application. The infrared image below is used for face liveness detection. In practical applications, the acquisition band of the infrared image can be the infrared light band between 850nm-1200nm.

Step 104 , extracting the partial region of the face on the acquired infrared image to obtain an image of the partial region of the target face.

Wherein, the above-mentioned partial area image of the target face may be: an image of an eye area, an image of a nose area, an image of a mouth area, an image of a forehead area, and the like.

In order to better understand the difference between the local area of the human face in the infrared images of the living body and the prosthesis, the embodiments of the present application take the living body, a paper photo prosthesis, and a silicone prosthesis as examples to illustrate. Referring to FIG. 2 , a is the eye area image in the infrared image of the living body, b is the eye area image in the infrared image of the paper photo prosthesis, and c is the eye area in the infrared image of the silicone prosthesis. It can be seen from FIG. 2 that, for a living body, the pupil, iris and sclera of the eye can be clearly distinguished. For paper photo prostheses, the eye has only two parts, black and white, and the pupil and iris cannot be distinguished, and the gray distribution of the eye is quite different from that of the living eye. For silicone prostheses, although the pupil, iris and sclera can be distinguished, the gray distribution of the eye is significantly different from that of the living eye. It can be seen that there are obvious differences between the ocular regions of the living body and the prosthesis in the infrared images.

In practical applications, the applicant found that not only the eye area has obvious differences, but also the mouth area. The living body and the silicone prosthesis are used as examples for illustration, see Figure 3, where e is the infrared image of the living body. Mouth area image, f is the mouth area in the infrared image of the silicone prosthesis. As can be seen from Figure 3, the grayscale of the living mouth and the surrounding face can be clearly divided, the mouth is darker, and the surrounding face is brighter; while the silicone prosthesis is the opposite, the mouth is much brighter than the surrounding face, although this It is related to the reflective properties of silicone materials and coatings, but repeated experiments on different materials and silicone prostheses with different reflective properties have proved that the difference in grayscale is sufficient for live detection.

Based on this, in the embodiment of the present application, the obtained infrared image is subjected to extraction processing of the eye region image and the mouth region image, and the obtained eye region image and mouth region image are determined as the target face local region image.

Step 106: Determine the grayscale feature of the partial region image of the target face.

In one embodiment of the present application, a grayscale histogram of the image of the partial region of the target face can be drawn, and the grayscale feature of the image of the partial region of the target face can be determined based on the drawn grayscale histogram. It can be understood that the above-mentioned drawing of the grayscale histogram of the target face local area image refers to the statistics of the grayscale histogram information of the target face local area image, that is, in the actual application process, it is not necessary to actually To draw the above-mentioned histogram, it is only necessary to obtain the statistical information of the gray-scale histogram of the above-mentioned partial region image of the target face. In this way, when the gray level feature is subsequently determined, the gray level feature of the target face partial region image can be determined based on the above-stated statistical information of the gray level histogram of the target face partial region image.

In another embodiment of the present application, a preset feature extraction algorithm can also be used to extract the grayscale features of the partial region image of the target face, wherein the feature extraction algorithm can be SIFT (Scale-invariant feature transform, the scale is invariant Feature transformation) algorithm, LBP (Local Binary Pattern, local binary pattern) algorithm, etc.

Step 108: Perform living body detection processing according to the determined grayscale feature and the pre-trained living body detection model to obtain a face living body detection result of the object to be detected.

Specifically, the determined grayscale feature is input into a pre-trained living body detection model for living body detection processing, and a face living body detection result of the object to be detected is obtained.

In the embodiment of the present application, the infrared image of the object to be detected is subjected to the extraction processing of the partial region of the face to obtain the image of the partial region of the target face; Liveness detection processing to determine whether the object to be detected is a face living body. As a result, accurate and effective living body detection is realized based on the local features of the face, which effectively solves the problems such as the decrease in the accuracy of the detection result caused by the occlusion of the partial area of the face.

In order to accurately extract the target face local area image to improve the accuracy of living body detection, in one or more embodiments of the present application, the key point detection of the face is firstly performed, and based on the detected key points, the face local area image is performed. extraction. Specifically, as shown in FIG. 4 , step S104 may include the following steps 104-2 to 104-8:

Step 104-2: Perform key point detection processing on the infrared image according to a preset method to obtain eye key points and mouth key points.

Among them, the eye key points include left eye key points and right eye key points, and the mouth key points include left mouth corner key points and right mouth corner key points. The detection method and specific detection process of key points can be set by themselves in practical applications, such as using ASM (Active Shape Model, active shape model) algorithm, CPR (Cascaded Pose Regression, cascade pose regression) algorithm, etc. For point detection, since both the ASM algorithm and the CPR algorithm are technical means well known to those skilled in the art, they will not be described in detail here.

Step 104-4, extract the eye region image according to the obtained eye key points.

Specifically, as shown in FIG. 5 , step 104-4 may include the following steps 104-42 to 104-44:

Steps 104-42, determine the first distance between the left eye key point and the right eye key point, respectively take the left eye key point and the right eye key point as the center, and demarcate the left eye according to the first distance and the first preset ratio area and the right eye area.

Specifically, the first coordinate information of the left eye key point and the second coordinate information of the right eye key point can be determined; the first coordinate information between the left eye key point and the right eye key point can be determined according to the first coordinate information and the second coordinate information distance; calculate the length and width of the rectangular area to be demarcated according to the first distance and the first preset ratio, take the key point of the left eye as the center, and delineate the left eye area of the rectangle according to the calculated length and width; and use the right eye The key point is the center, and the right eye area of the rectangle is delineated according to the calculated length and width.

The coordinate information of each key point may be: the pixel coordinates of the key point in the image, or may be the coordinates of the key point in a preset image coordinate system.

The first distance can be considered as the interpupillary distance between the left eye and the right eye; the first preset ratio includes the first preset width ratio and the first preset length ratio; in practical applications, the above-mentioned No. A preset width ratio and a first preset length ratio. For example, denoting the interpupillary distance as IPD, the first preset width ratio may be 0.16-0.18 times the IPD, and the first preset length ratio may be 0.4-0.45 times the IPD. The first preset length ratio can be understood as: the ratio of the length of the rectangular area where the eyes are located to the distance between the left and right eyes, and the first preset width ratio can be understood as: the width of the rectangular area where the eyes are located relative to the distance between the left and right eyes. Proportion.

Steps 104-44: Determine the images corresponding to the demarcated left eye area and right eye area as eye area images.

Specifically, the image corresponding to the demarcated left-eye area may be determined as the left-eye area image, the image corresponding to the demarcated right-eye area may be determined as the right-eye area image, and the left-eye area image and the right-eye area image may be determined as the right-eye area image. Determined as the eye area image.

Considering that in the infrared image of the paper photo prosthesis, it is difficult to distinguish the pupil, iris and sclera of the eye, and based on the special structure of the eye, for a living body, as shown in the dotted circle in Figure 2a, Two circles can be fitted based on grayscale features. Based on this, in the process of obtaining the image of the eye region, the object to be detected can be detected by a circular fitting process. Specifically, as shown in FIG. 6, step 104-44 may include the following steps 104-442 to 104-446:

Step 104-442, perform edge detection processing on the images corresponding to the left eye area and the right eye area, and perform circular fitting processing based on the processing result of the edge detection processing; if the circular fitting processing fails, perform steps 104-444 ; If the circle fitting process is successful, execute steps 104-446.

Wherein, edge detection processing can be performed on the images corresponding to the left eye region and the right eye region based on edge detection operators such as Laplacian operator and Gaussian operator, so as to obtain the edge in the above image, which is used as the processing of edge detection processing As a result, a circular fit is performed based on the detected edges. Since the process of edge detection processing is in the prior art, it will not be repeated in this application.

The above-mentioned circular fitting based on the detected edge can be understood as performing curve fitting on the detected edge and judging that it can be fitted as a circle. If the above-mentioned edge can be successfully fitted as a circle, it is considered that a circle is The fitting process is successful; if it is difficult to fit the above edge into a circle, it is considered that the circle fitting process fails.

Steps 104-444, it is determined that there is no living body, and the face living body detection result of the object to be detected is that the detection fails.

Specifically, if the circle fitting fails, it means that it is difficult to distinguish the pupil, iris and sclera of the eye in the infrared image, which further means that the above infrared image may be the infrared image of the prosthesis, so it can be determined that there is no living body. Correspondingly, it is determined that the living body detection result is: the detection fails.

Steps 104-446: Determine the images corresponding to the demarcated left-eye area and right-eye area as eye area images.

Specifically, if the circle fitting is successful, it means that the pupil, the iris and the sclera of the eye can be distinguished in the infrared image, and subsequent living detection can be further performed. In this case, the eye region image can be extracted by detecting the eye key points and extracting the eye region image based on the eye key point, so as to ensure the extraction accuracy of the eye region image.

By performing the circle fitting process in the extraction process of the eye area image, the prosthesis in the form of paper photos can be identified in time, and if the circle fitting fails, it can be directly judged that there is no living body, and there is no need to perform Subsequent related processing can improve the detection efficiency.

Step 104-6, extract the mouth region image according to the obtained mouth key points.

Specifically, as shown in FIG. 7 , step 104-6 may include the following steps 104-62 to 104-66:

Steps 104-62: Determine the reference point of the mouth according to the key point of the left corner of the mouth and the key point of the right corner of the mouth.

Specifically, the third coordinate information of the key point of the left corner of the mouth and the fourth coordinate information of the key point of the right corner of the mouth can be determined, and the reference point of the mouth is determined according to the third coordinate information and the fourth coordinate information. For example, the average value of the third coordinate information and the fourth coordinate information is calculated, and the point corresponding to the calculated average value is determined as the reference point of the mouth.

In an embodiment of the present application, if the object to be detected is in a tilted state when the infrared image is acquired, an adjustment coefficient for correcting the tilted state may be set, and when the above average value is obtained, the average value is compared with the set adjustment coefficient Multiply to obtain the reference point of the mouth of the object to be detected after correction.

Steps 104-64: Determine the second distance between the key point of the left corner of the mouth and the key point of the right corner of the mouth, take the reference point as the center, and define the mouth area according to the second distance and the second preset ratio.

Specifically, according to the third coordinate information of the key point of the left corner of the mouth and the fourth coordinate information of the key point of the right corner of the mouth, the second distance between the key point of the left corner of the mouth and the key point of the right corner of the mouth is calculated; according to the second distance and The second preset ratio calculates the length and width of the rectangular area to be demarcated, takes the reference point of the mouth as the center, and demarcates the rectangular mouth area according to the calculated length and width. The second preset ratio may include a width ratio, the second distance is determined as the length of the rectangular area to be demarcated, the length of the second distance is denoted as d, and the second preset ratio may be 0.32-0.36 of d times. The above-mentioned second preset ratio can be understood as: the ratio of the width of the rectangular region where the mouth is located to the distance between the key points of the corners of the mouth on the left and right sides.

It should be pointed out that the above-mentioned first preset ratio and second preset ratio can be set by themselves according to actual needs. And when the detected key points are not just a left eye key point, a right eye key point, a left mouth corner key point and a right mouth corner key point, but many key points, it can also be delineated based on the detected key points. Eye area and mouth area. For example, taking the left-eye area as an example, in the case where there are multiple left-eye key points, the preset graphic area including the above-mentioned multiple left-eye key points can be used as the left-eye area, and the above-mentioned preset graphic area can be a circle area, rectangular area, oval area, etc.

Steps 104-66 determine the image corresponding to the demarcated mouth region as the mouth region image.

Considering that the grayscale distribution of the mouth region of the prosthesis such as silicone is significantly different from that of the living body, in the process of obtaining the mouth region image, the grayscale feature can be used to determine whether the object to be detected is for the living body. Specifically, as shown in FIG. 8, in an embodiment of the present application, steps 104-66 may include the following steps 104-662 to 104-666:

Step 104-662: Calculate the grayscale mean value of the image corresponding to the planned mouth region, and determine whether the statistical grayscale mean value meets the preset condition; if not, execute step 104-664; 666.

Specifically, determine the number of pixels of the image corresponding to the delineated mouth area and the gray value of each pixel, and calculate the gray mean value according to the determined number and gray value; determine whether the calculated gray mean value is greater than The preset grayscale value, if yes, it is determined that the statistical grayscale mean value does not meet the preset condition; if not, it is determined that the statistical grayscale mean value meets the preset condition.

Steps 104-664, it is determined that there is no living body, and the face living body detection result of the object to be detected is that the detection fails.

Specifically, if the statistical mean gray value does not meet the preset conditions, it means that the above-mentioned mouth area may be the mouth area of a prosthesis such as silicone, so it can be determined that there is no living body, and accordingly, it is determined that the living body detection result is: pass.

Steps 104-666, determining the image corresponding to the demarcated mouth region as the mouth region image.

Specifically, if the statistical gray mean value meets the preset condition, the subsequent living body detection can be further performed. In this case, by detecting the key points of the mouth and extracting the image of the mouth region based on the key points of the mouth, the accuracy of the extracted image of the mouth region is ensured. By calculating the average gray value of the mouth area and judging whether the gray average value meets the preset conditions, the prosthesis such as silicone can be detected in time. In this case, subsequent detection processing is not required, and the detection efficiency can be improved.

Step 104-8: Determine the extracted eye region image and mouth region image as the target face partial region image.

It should be noted that, the execution order of the above steps S104-4 and S104-6 may be interchanged, and may also be executed simultaneously.

Considering the different imaging distances of different imaging devices, some imaging devices are suitable for imaging in the range of 30 cm-100 cm, and some devices are suitable for imaging in the range of 30 cm-200 cm or even further; The size of the face image in the infrared image is often different. In order to avoid the detection difference caused by the difference in the size of the face image, in one or more embodiments of the present application, the extracted target face local area images are normalized. Subsequent processing is carried out after the chemical treatment. Specifically, as shown in FIG. 9 , step 106 includes the following steps 106-2 to 106-6:

Step 106-2, performing normalization processing on the partial region image of the target face to obtain an image of a preset size.

Specifically, the image of the left eye area, the image of the right eye area, and the image of the mouth area are respectively normalized to obtain an image of a preset size. Among them, the size of the normalized image can be set by itself in practical applications.

Step 106-4, drawing a grayscale histogram of an image of a preset size.

Specifically, for each preset size image, the gray value of each pixel can be determined, and the number of pixels corresponding to each gray value can be counted. The number of corresponding pixel points is the ordinate, and the gray histogram is drawn according to the determined gray value and the number of statistics.

It should be pointed out that when drawing a grayscale histogram, you can also specify the interval of grayscale values. For example, you only need to count the number of pixels corresponding to the grayscale values between [20, 220], and draw a grayscale histogram. .

It can be understood that the above-mentioned drawing of the grayscale histogram of the image of the preset size refers to the statistics of the grayscale histogram information of the image of the preset size, that is, in the actual application process, it is not necessary to actually The histogram is drawn, but only the statistical information of the grayscale histogram of the image of the preset size needs to be obtained by statistics. In this way, when the gray level feature is subsequently determined, the gray level feature of the image in the partial region of the target face can be determined based on the above-stated statistical information of the gray level histogram of the image of the preset size.

Step 106-6: Determine a grayscale vector according to the grayscale histogram, and determine the grayscale vector as the grayscale feature of the target face local area image.

Specifically, according to the drawn grayscale histograms of the left-eye area image, the right-eye area image, and the mouth area image, each one-dimensional grayscale vector is determined, and three one-dimensional grayscale vectors are obtained. Elements of different items in the one-dimensional vector correspond to different grayscale values, and each element in the above one-dimensional vector represents the number of pixels with corresponding grayscale values; the determined three one-dimensional grayscale vectors are determined as the target person Grayscale features of face local area images.

In order to improve the detection accuracy, after obtaining the grayscale feature of the target face local area image, the liveness detection process can be performed based on the pretrained liveness detection model according to the grayscale feature.

In an embodiment of the present application, the living body detection model includes a first living body detection model and a second living body detection model, wherein the first living body detection model is used to perform live body detection on the eye region, and the second live body detection model is used for the mouth region. Liveness testing in the area. Correspondingly, as shown in FIG. 10 , step 108 may include the following steps 108-2 to 108-6:

Step 108-2: Input the grayscale feature of the eye region image into the first living body detection model for detection processing to obtain a first detection result.

Specifically, the grayscale vector of the left-eye region image and the grayscale vector of the right-eye region image are input into the first living body detection model for detection processing to obtain the first detection result.

Step 108-4: Input the grayscale feature of the mouth region image into the second living body detection model for detection processing, and obtain a second detection result.

Specifically, the grayscale vector of the mouth region image is input into the second living body detection model for detection processing, and the second detection result is obtained.

Step 108-6, if both the first detection result and the second detection result indicate the existence of a living body, it is determined that the detection result of the living body of the face of the object to be detected is the detection passed.

When any one or more of the first detection result and the second detection result indicate that there is no living body, it is determined that the detection result of the living body of the face of the object to be detected is that the detection fails.

In one or more embodiments of the present application, the living body detection model may include a third detection model, where the third detection model is used to simultaneously perform live body detection on the eye region and the mouth region. Correspondingly, as shown in FIG. 11 , step 108 may include the following steps 108-8 and 108-10:

Step 108-8, combine the grayscale features of the eye region image and the grayscale features of the mouth region to obtain combined grayscale features.

Among them, the merging method can be set by itself in practical applications. For example, the gray vector of the eye area image and the gray vector of the mouth area image are spliced to obtain a combined gray vector, and the combined gray vector Determined to merge grayscale features.

In step 108-10, the combined grayscale feature is input into the third living body detection model for detection processing, and a face living body detection result of the object to be detected is obtained.

Specifically, the merged grayscale vector is input into the third living body detection model for detection processing, and a face living body detection result of the object to be detected is obtained.

It should be pointed out that, for some scenarios where the security requirements are not particularly high, it is also possible to perform liveness detection only through the above-mentioned first liveness detection model, or only through the above-mentioned second liveness detection model; After the infrared image of the subject, only the eye area image or only the mouth area image can be extracted. The number and usage of the models can be set according to actual needs.

In order to achieve high-accuracy detection of living bodies based on partial regions of faces, in one or more embodiments of the present application, as shown in FIG. 12 , before step 102 , the following steps 100-2 to 100-8 may be further included:

Step 100-2, acquiring infrared images of multiple samples, wherein the samples include positive samples and negative samples, the positive samples are living objects, and the negative samples are prosthetic objects.

Infrared images of multiple samples can be collected through an infrared image collection device, or infrared images of multiple samples can be obtained from a network. The manner of acquiring the infrared images of the multiple samples is not specifically limited in this application.

Step 100-4, extracting the partial region of the face on the infrared image of each sample to obtain the partial region image of the face to be trained.

The process of extracting the partial region image of the face is the same as the process of extracting the image of the partial region of the target face, and reference may be made to the above-mentioned related description, and the repeated parts will not be repeated here.

Step 100-6: Determine the grayscale feature of the partial region image of the face to be trained.

The manner of determining the grayscale feature is the same as the foregoing manner of determining the grayscale feature of the partial region image of the target face, and reference may be made to the foregoing related descriptions, and repeated details will not be repeated here.

Step 100-8, according to a preset training method, perform training processing based on the grayscale features of the partial region images of the face to be trained to obtain a living body detection model.

Specifically, according to the attributes of the living body and the prosthesis of each sample, labeling is performed on the grayscale features of the partial region image of the face to be trained, so as to mark the corresponding sample as a living body or a prosthesis. The grayscale features with labels are divided into training sets and test sets, and the SVM (Support Vector Machine, Support Vector Machine) algorithm is used to train the living body detection model based on the training set, and the initial living body detection model is obtained. Based on the test set, the obtained initial live detection model is tested and processed to obtain a test result. If it is determined that the test result meets the preset conditions, the initial living body detection model is determined as the final living body detection model; if it is determined that the test result does not meet the preset conditions, the training process of the living body detection model is performed again based on the training set until the test result is reached. In accordance with the preset conditions, the final living detection model is obtained. Wherein, the preset condition may be: the accuracy rate of the test result is greater than the preset accuracy rate, and the like. Since the training process of the model is a technical means well known to those skilled in the art, the training process of the model will not be described in detail in this application.

It should be pointed out that, when the living body detection model includes the first living body detection model and the second living body detection model, the training of the first living body detection model is performed according to the grayscale features of the eye region images to be trained, and the The grayscale features of the mouth region image are used to train the second in vivo detection model. The training method of the living body detection model is also not limited to the above-mentioned SVM training, and may also be trained based on a neural network. The training method of the living body detection model can be set according to the needs in practical applications.

In the embodiment of the present application, the infrared image of the object to be detected is subjected to the extraction processing of the partial region of the face to obtain the image of the partial region of the face of the target; Liveness detection processing to determine whether the object to be detected is a face living body. Therefore, accurate and effective living body detection is realized based on the local features of the face, and the problems such as the decrease in the accuracy of the detection result caused by the occlusion of a part of the face area are effectively solved.

Based on the same technical concept, one or more embodiments of the present application also provide a face liveness detection device. FIG. 13 also provides a schematic diagram of the module composition of a face liveness detection device in one or more embodiments of the present application, such as As shown in Figure 13, the device includes:

An image sensor 201, used for acquiring an infrared image of an object to be detected;

The processor 202 is configured to perform extraction processing of the partial area of the face on the infrared image to obtain an image of the partial area of the target face; determine the grayscale feature of the image of the partial area of the target face; The trained living body detection model is subjected to detection processing to obtain a face living body detection result of the object to be detected.

The face living body detection device provided by the embodiment of the present application obtains the target face local area image by extracting the face local area of the infrared image of the object to be detected; The trained live detection model performs live detection processing to determine whether the object to be detected is a live human face. As a result, accurate and effective living body detection is realized based on the local features of the face, which effectively solves the problems such as the decrease in the accuracy of the detection result caused by the occlusion of the partial area of the face.

In an embodiment of the present application, the processor 202 extracts an eye region image according to the eye key points; and,

extracting the mouth region image according to the mouth key points;

The eye region image and the mouth region image are determined as target face local region images.

In an embodiment of the present application, the eye key points include left eye key points and right eye key points;

Correspondingly, the processor 202 determines a first distance between the left-eye key point and the right-eye key point; and,

Taking the left-eye key point and the right-eye key point as the center, respectively, delimiting the left-eye area and the right-eye area according to the first distance and the first preset ratio;

The images corresponding to the demarcated left eye area and right eye area are determined as eye area images.

In an embodiment of the present application, the processor 202 performs edge detection processing on the images corresponding to the left eye region and the right eye region; and,

performing circle fitting processing based on the processing result of the edge detection processing;

If the circle fitting process is successful, then the images corresponding to the demarcated left eye area and right eye area are determined as eye area images;

If the circle fitting process fails, it is determined that there is no living body, and the face living body detection result of the object to be detected is that the detection fails.

In an embodiment of the present application, the processor 202 determines the first coordinate information of the left eye key point and the second coordinate information of the right eye key point; and,

A first distance between the left-eye keypoint and the right-eye keypoint is determined according to the first coordinate information and the second coordinate information.

In an embodiment of the present application, the key points of the mouth include a key point of the left corner of the mouth and a key point of the right corner of the mouth;

Correspondingly, the processor 202 determines the reference point of the mouth according to the key point of the left corner of the mouth and the key point of the right corner of the mouth; and,

determining a second distance between the key point of the left corner of the mouth and the key point of the right corner of the mouth;

Taking the reference point as the center, delimiting the mouth region according to the second distance and the second preset ratio;

An image corresponding to the demarcated mouth region is determined as a mouth region image.

In an embodiment of the present application, the processor 202 calculates the average gray value of the image corresponding to the predetermined mouth region; and,

determining whether the gray mean value meets a preset condition;

If yes, then determine the image corresponding to the demarcated mouth region as the mouth region image;

If not, it is determined that there is no living body, and the face living body detection result of the object to be detected is that the detection fails.

In an embodiment of the present application, the processor 202 determines the third coordinate information of the key point of the left corner of the mouth and the fourth coordinate information of the key point of the right corner of the mouth;

The reference point of the mouth is determined according to the third coordinate information and the fourth coordinate information.

In an embodiment of the present application, the processor 202 performs normalization processing on the image of the partial region of the target face to obtain an image of a preset size;

drawing a grayscale histogram of the image of the preset size;

A grayscale vector is determined according to the grayscale histogram, and the grayscale vector is determined as the grayscale feature of the target face local area image.

In an embodiment of the present application, the processor 202 inputs the grayscale feature of the eye region image into a first living body detection model for detection processing, and obtains a first detection result;

inputting the grayscale feature of the mouth region image into the second living body detection model for detection processing to obtain a second detection result;

If both the first detection result and the second detection result indicate the existence of a living body, it is determined that the detection result of the living body of the face of the object to be detected is the detection pass.

In an embodiment of the present application, the processor 202 performs a merge process on the grayscale feature of the eye region image and the grayscale feature of the mouth region to obtain a combined grayscale feature;

The combined grayscale feature is input into a third living body detection model for detection processing to obtain a face living body detection result of the object to be detected.

In an embodiment of the present application, the processor 202 acquires infrared images of multiple samples; wherein the samples include positive samples and negative samples, the positive samples are living objects, and the negative samples are prosthetic objects ;as well as,

Extracting the face local area on the infrared image of each sample to obtain the face local area image to be trained;

Determine the grayscale feature of the face local area image to be trained;

According to a preset training method, a training process is performed based on the grayscale feature of the partial region image of the face to be trained to obtain the living body detection model.

The face living body detection device provided by the embodiment of the present application obtains the target face local area image by extracting the face local area of the infrared image of the object to be detected; The trained live detection model performs live detection processing to determine whether the object to be detected is a live human face. As a result, accurate and effective living body detection is realized based on the local features of the face, which effectively solves the problems such as the decrease in the accuracy of the detection result caused by the occlusion of the partial area of the face.

In addition, for the above-mentioned apparatus embodiments, since they are basically similar to the method embodiments, the description is relatively simple, and reference may be made to some descriptions of the method embodiments for related parts. Moreover, it should be noted that, among the components of the device of the present application, the components are logically divided according to the functions to be implemented, but the present application is not limited to this, and each component can be re-configured as required. Divide or combine.

Based on the same technical concept, one or more embodiments of the present application also provide a face liveness detection system, the system includes: a data processor and an image sensor supporting infrared image acquisition, wherein:

The image sensor is used to collect the infrared image of the object to be detected, and send the collected infrared image to the data processor;

The data processor is configured to receive the infrared image collected by the image sensor, perform extraction processing of the partial area of the face on the infrared image, and obtain an image of the partial area of the target face; determine the image of the partial area of the target face. Grayscale feature; perform liveness detection processing according to the grayscale feature and the pre-trained liveness detection model, and obtain the face liveness detection result of the object to be detected.

FIG. 14 is a schematic structural diagram of an electronic device according to an embodiment of the description. Referring to FIG. 14 , the electronic device includes a processor, an internal bus, a network interface, a memory, and a non-volatile memory, and of course may also include other business required hardware. The processor reads the corresponding computer program from the non-volatile memory into the memory and runs it, forming a face liveness detection device on a logical level. Of course, in addition to software implementations, this application does not exclude other implementations, such as logic devices or a combination of software and hardware. hardware or logic device.

Network interfaces, processors and memories can be interconnected through a bus system. The bus can be an ISA (Industry Standard Architecture, industry standard architecture) bus, a PCI (Peripheral Component Interconnect, peripheral component interconnect standard) bus, or an EISA (Extended Industry Standard Architecture, extended industry standard structure) bus and the like. The bus can be divided into an address bus, a data bus, a control bus, and the like. For ease of representation, only one bidirectional arrow is shown in FIG. 14, but it does not mean that there is only one bus or one type of bus.

Memory is used to store programs. Specifically, the program may include program code, and the program code includes computer operation instructions. The memory, which may include read-only memory and random access memory, provides instructions and data to the processor. The memory may include high-speed random-access memory (Random-Access Memory, RAM), and may also include non-volatile memory (non-volatile memory), such as at least one disk memory.

a processor, configured to execute the program stored in the memory, and specifically execute:

Obtain the infrared image of the object to be detected;

Extracting the partial area of the face on the infrared image to obtain an image of the partial area of the target face;

Determine the grayscale feature of the target face local area image;

The living body detection process is performed according to the grayscale feature and the pre-trained living body detection model, and the face living body detection result of the object to be detected is obtained.

The above-mentioned method performed by the apparatus for detecting a face living body disclosed in the embodiment shown in FIG. 13 of the present application may be applied to a processor, or implemented by a processor. A processor may be an integrated circuit chip with signal processing capabilities. In the implementation process, each step of the above-mentioned method can be completed by a hardware integrated logic circuit in a processor or an instruction in the form of software. The above-mentioned processor can be a general-purpose processor, including a central processing unit (Central Processing Unit, CPU), a network processor (Network Processor, NP), etc.; it can also be a digital signal processor (Digital Signal Processor, DSP), dedicated integrated Circuit (Application Specific Integrated Circuit, ASIC), Field Programmable Gate Array (Field-Programmable Gate Array, FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components. The methods, steps, and logic block diagrams disclosed in the embodiments of this application can be implemented or executed. A general purpose processor may be a microprocessor or the processor may be any conventional processor or the like. The steps of the method disclosed in conjunction with the embodiments of the present application may be directly embodied as executed by a hardware decoding processor, or executed by a combination of hardware and software modules in the decoding processor. The software modules may be located in random access memory, flash memory, read-only memory, programmable read-only memory or electrically erasable programmable memory, registers and other storage media mature in the art. The storage medium is located in the memory, and the processor reads the information in the memory, and completes the steps of the above method in combination with its hardware.

Based on the same technical concept, the embodiments of the present application also provide a computer-readable storage medium, where the computer-readable storage medium stores one or more programs, and the one or more programs, when included as a plurality of application programs When the electronic device executes, the electronic device is made to execute the face liveness detection method provided by any of the embodiments corresponding to FIG. 1 , FIG. 4 to FIG. 12 .

Each embodiment in this application is described in a progressive manner, and the same and similar parts between the various embodiments may be referred to each other, and each embodiment focuses on the differences from other embodiments. In particular, for the system embodiments, apparatus embodiments, electronic device embodiments, computer-readable storage medium embodiments, and computer program product embodiments, since they are basically similar to the method embodiments, the descriptions are relatively simple, and the relevant parts Please refer to the partial description of the method embodiment.

The foregoing describes specific embodiments of the present application. Other embodiments are within the scope of the appended claims. In some cases, the actions or steps recited in the claims can be performed in an order different from that in the embodiments and still achieve desirable results. Additionally, the processes depicted in the figures do not necessarily require the particular order shown, or sequential order, to achieve desirable results. In some embodiments, multitasking and parallel processing are also possible or may be advantageous.

As will be appreciated by those skilled in the art, the embodiments of the present application may be provided as a method, a system, or a computer program product. Accordingly, the present application may take the form of an entirely hardware embodiment, an entirely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, the present application may take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, etc.) having computer-usable program code embodied therein.

The present application is described with reference to flowchart illustrations and/or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the present application. It will be understood that each process and/or block in the flowchart illustrations and/or block diagrams, and combinations of processes and/or blocks in the flowchart illustrations and/or block diagrams, can be implemented by computer program instructions. These computer program instructions may be provided to the processor of a general purpose computer, special purpose computer, embedded processor or other programmable data processing device to produce a machine such that the instructions executed by the processor of the computer or other programmable data processing device produce Means for implementing the functions specified in a flow or flow of a flowchart and/or a block or blocks of a block diagram.

These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing apparatus to function in a particular manner, such that the instructions stored in the computer-readable memory result in an article of manufacture comprising instruction means, the instructions The apparatus implements the functions specified in the flow or flow of the flowcharts and/or the block or blocks of the block diagrams.

These computer program instructions can also be loaded on a computer or other programmable data processing device to cause a series of operational steps to be performed on the computer or other programmable device to produce a computer-implemented process such that The instructions provide steps for implementing the functions specified in the flow or blocks of the flowcharts and/or the block or blocks of the block diagrams.

In a typical configuration, a computing device includes one or more processors (CPUs), input/output interfaces, network interfaces, and memory.

Memory may include forms of non-persistent memory, random access memory (RAM) and/or non-volatile memory in computer readable media, such as read only memory (ROM) or flash memory (flash RAM). Memory is an example of a computer-readable medium.

Computer-readable media includes both persistent and non-permanent, removable and non-removable media, and storage of information may be implemented by any method or technology. Information may be computer readable instructions, data structures, modules of programs, or other data. Examples of computer storage media include, but are not limited to, phase-change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read only memory (ROM), Electrically Erasable Programmable Read Only Memory (EEPROM), Flash Memory or other memory technology, Compact Disc Read Only Memory (CD-ROM), Digital Versatile Disc (DVD) or other optical storage, Magnetic tape cassettes, magnetic tape magnetic disk storage or other magnetic storage devices or any other non-transmission medium that can be used to store information that can be accessed by a computing device. As defined herein, computer-readable media does not include transitory computer-readable media, such as modulated data signals and carrier waves.

It should also be noted that the terms "comprising", "comprising" or any other variation thereof are intended to encompass a non-exclusive inclusion such that a process, method, article or device comprising a series of elements includes not only those elements, but also Other elements not expressly listed, or inherent to such a process, method, article of manufacture or apparatus are also included. Without further limitation, an element qualified by the phrase "comprising a..." does not preclude the presence of additional identical elements in the process, method, article of manufacture, or device that includes the element.

It will be appreciated by those skilled in the art that the embodiments of the present application may be provided as a method, a system or a computer program product. Accordingly, the present application may take the form of an entirely hardware embodiment, an entirely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, the present application may take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, etc.) having computer-usable program code embodied therein.

The above descriptions are merely examples of the present application, and are not intended to limit the present application. Various modifications and variations of this application are possible for those skilled in the art. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of this application shall be included within the scope of the claims of this application.

Claims (16)

1. A face liveness detection method, comprising:

   Obtain the infrared image of the object to be detected;

   Extracting the partial area of the face on the infrared image to obtain an image of the partial area of the target face;

   Determine the grayscale feature of the target face local area image;

   The living body detection process is performed according to the grayscale feature and the pre-trained living body detection model, and the face living body detection result of the object to be detected is obtained.
2. The method according to claim 1, wherein, performing the extraction processing of the partial region of the face on the infrared image to obtain an image of the partial region of the face of the target, comprising:

   Perform key point detection processing on the infrared image according to a preset method to obtain eye key points and mouth key points;

   extracting an eye region image according to the eye key points;

   extracting the mouth region image according to the mouth key points;

   The eye region image and the mouth region image are determined as target face local region images.
3. The method of claim 2, wherein the eye key points include a left eye key point and a right eye key point;

   The extracting the eye region image according to the eye key points includes:

   determining a first distance between the left-eye keypoint and the right-eye keypoint;

   Taking the left-eye key point and the right-eye key point as the center, respectively, delimiting the left-eye area and the right-eye area according to the first distance and the first preset ratio;

   The images corresponding to the demarcated left eye area and right eye area are determined as eye area images.
4. The method according to claim 3, wherein the determining the images corresponding to the demarcated left eye region and the right eye region as the eye region images comprises:

   performing edge detection processing on the images corresponding to the left eye area and the right eye area;

   performing circle fitting processing based on the processing result of the edge detection processing;

   If the circle fitting process is successful, then the images corresponding to the demarcated left eye area and right eye area are determined as eye area images;

   If the circle fitting process fails, it is determined that there is no living body, and the face living body detection result of the object to be detected is that the detection fails.
5. The method according to claim 3, wherein the determining the first distance between the left-eye key point and the right-eye key point comprises:

   determining the first coordinate information of the left eye key point and the second coordinate information of the right eye key point;

   A first distance between the left-eye keypoint and the right-eye keypoint is determined according to the first coordinate information and the second coordinate information.
6. The method according to claim 2, wherein the key points of the mouth include a key point of the left corner of the mouth and a key point of the right corner of the mouth;

   The extracting the mouth region image according to the mouth key points includes:

   Determine the reference point of the mouth according to the key point of the left corner of the mouth and the key point of the right corner of the mouth;

   determining a second distance between the key point of the left corner of the mouth and the key point of the right corner of the mouth;

   Taking the reference point as the center, delimiting the mouth region according to the second distance and the second preset ratio;

   An image corresponding to the demarcated mouth region is determined as a mouth region image.
7. The method according to claim 6, wherein the determining the image corresponding to the demarcated mouth region as the mouth region image comprises:

   The gray mean value of the image corresponding to the mouth region planned by the system;

   determining whether the grayscale mean value meets a preset condition;

   If yes, then determine the image corresponding to the demarcated mouth region as the mouth region image;

   If not, it is determined that there is no living body, and the face living body detection result of the object to be detected is that the detection fails.
8. The method according to claim 6, wherein the determining the reference point of the mouth according to the key point of the left corner of the mouth and the key point of the right corner of the mouth comprises:

   Determine the third coordinate information of the key point of the left corner of the mouth and the fourth coordinate information of the key point of the right corner of the mouth;

   The reference point of the mouth is determined according to the third coordinate information and the fourth coordinate information.
9. The method according to claim 1, wherein the determining the grayscale feature of the partial region image of the target face comprises:

   normalizing the image of the target face local area to obtain an image of a preset size;

   drawing a grayscale histogram of the image of the preset size;

   A grayscale vector is determined according to the grayscale histogram, and the grayscale vector is determined as the grayscale feature of the target face local area image.
10. The method according to claim 2, wherein the performing a living body detection process according to the grayscale feature and a pre-trained living body detection model to obtain a face living body detection result of the object to be detected, comprising:

    inputting the grayscale feature of the eye region image into the first living body detection model for detection processing to obtain a first detection result;

    inputting the grayscale feature of the mouth region image into the second living body detection model for detection processing to obtain a second detection result;

    If both the first detection result and the second detection result indicate the existence of a living body, it is determined that the detection result of the living body of the face of the object to be detected is the detection passed.
11. The method according to claim 2, wherein the performing a living body detection process according to the grayscale feature and a pre-trained living body detection model to obtain a face living body detection result of the object to be detected, comprising:

    Merging the grayscale feature of the eye region image and the grayscale feature of the mouth region to obtain a combined grayscale feature;

    The combined grayscale feature is input into a third living body detection model for detection processing to obtain a face living body detection result of the object to be detected.
12. The method according to claim 1, wherein the method further comprises:

    acquiring infrared images of a plurality of samples; wherein the samples include positive samples and negative samples, the positive samples are living objects, and the negative samples are prosthetic objects;

    Extracting the face local area on the infrared image of each sample to obtain the face local area image to be trained;

    Determine the grayscale feature of the face local area image to be trained;

    According to a preset training method, a training process is performed based on the grayscale feature of the partial region image of the face to be trained to obtain the living body detection model.
13. A face liveness detection device, comprising:

    an image sensor for acquiring an infrared image of the object to be detected;

    a processor, configured to perform extraction processing on the infrared image of the partial region of the face, to obtain an image of the partial region of the target face; determine the grayscale feature of the image of the partial region of the target face; according to the grayscale feature and pre-training The living body detection model is used for detection processing to obtain the face living body detection result of the object to be detected.
14. An electronic device, characterized in that it includes: a processor, a communication interface, a memory, and a communication bus; wherein, the processor, the communication interface, and the memory communicate with each other through the bus; the memory is used to store computer programs; the processor, It is used to execute the program stored in the memory to realize the steps of the method described in any one of the above claims 1 to 12.
15. A computer-readable storage medium, characterized in that a computer program is stored on the computer-readable storage medium, and when the computer program is executed by a processor, the method of any one of the preceding claims 1 to 12 is implemented. step.
16. A face living body detection system, characterized in that the system comprises: a data processor, an image sensor that supports collecting infrared images, wherein:

    The image sensor is used to collect the infrared image of the object to be detected, and send the collected infrared image to the data processor;

    The data processor is configured to receive the infrared image collected by the image sensor, perform extraction processing of the partial area of the face on the infrared image, and obtain an image of the partial area of the target face; determine the image of the partial area of the target face. Grayscale feature; perform liveness detection processing according to the grayscale feature and the pre-trained liveness detection model, and obtain the face liveness detection result of the object to be detected.

Images