CN113989870A

CN113989870A - Living body detection method, door lock system and electronic equipment

Info

Publication number: CN113989870A
Application number: CN202110858762.5A
Authority: CN
Inventors: 郑新莹; 张劲风; 高通
Original assignee: Orbbec Inc
Current assignee: Orbbec Inc
Priority date: 2021-07-28
Filing date: 2021-07-28
Publication date: 2022-01-28

Abstract

The application relates to the technical field of biological identification, in particular to a living body detection method, a door lock system and electronic equipment. The in vivo detection method comprises the following steps: acquiring at least two images of an infrared image, a color image and a depth image as target images, and identifying a plurality of human face characteristic points of a human face area of a target object in each target image; cutting a plurality of feature frames in the face area of each target image by taking the plurality of face feature points as anchor points; inputting the plurality of feature frames of each target image into a living body detection model corresponding to the target image to carry out living body judgment, and obtaining a judgment result; and determining whether the target object is a living body according to the judgment result. According to the embodiment of the application, the user does not need to cooperate to make the specified action, so that the convenience is improved; in addition, the accuracy of the detection result can be improved by performing the living body detection through the multi-modal data.

Description

Living body detection method, door lock system and electronic equipment

Technical Field

The application relates to the technical field of biological identification, in particular to a living body detection method, a door lock system and electronic equipment.

Background

Face recognition is a biometric technique for identifying an identity based on facial feature information of a person. As the face recognition technology becomes mature, the commercial application thereof is increasingly wide, for example, the face recognition technology is widely applied to the fields of financial transactions, access control systems, mobile terminals and the like.

However, the face is very easy to copy by means of photos, videos, models or masks, and the like, so that the counterfeit of the face of a legal user is an important threat to the safety of the face recognition and authentication system. In order to prevent a malicious person from forging and stealing the biometric features of another person for identity authentication, the biometric identification system needs to have a face anti-biometric (face anti-biometric) function, i.e., to determine whether the submitted biometric features are from a living individual.

At present, the current living body detection technology of the face recognition technology generally adopts a mode of matching instruction actions, such as left turning, right turning, mouth opening, blinking and the like of a face, and if the instruction matching is wrong, forgery and deception are considered. However, the instruction action coordination mode needs user coordination, which greatly limits the popularization of related products.

Disclosure of Invention

In view of this, the embodiment of the application provides a living body detection method, a door lock system and an electronic device, and a user does not need to cooperate to make a specified action, so that convenience is improved.

In a first aspect, an embodiment of the present application provides a method for detecting a living body, including:

acquiring at least two images of an infrared image, a color image and a depth image as target images, and identifying a plurality of human face characteristic points of a human face area of a target object in each target image;

cutting a plurality of feature frames in the face area of each target image by taking the plurality of face feature points as anchor points;

inputting the plurality of feature frames of each target image into a living body detection model corresponding to the target image to carry out living body judgment, and obtaining a judgment result;

and determining whether the target object is a living body according to the judgment result.

According to the living body detection method provided by the embodiment, the living body detection model is used for carrying out living body detection on the face image of the target object, and a user does not need to cooperate to make a specified action, so that convenience is improved; in addition, the accuracy of the detection result can be improved by inputting multi-modal data into the living body detection model for living body detection.

As an implementation manner of the first aspect, the living body detection method includes:

acquiring a depth image and an infrared image, and respectively identifying a plurality of human face characteristic points of a target object human face area in the infrared image and the depth image;

cutting a plurality of depth feature frames in a face area in the depth image by taking the plurality of face feature points as anchor points respectively, and cutting a plurality of infrared feature frames in the face area in the infrared image; cropping the plurality of depth feature frames in at least one size in the face area in the depth image, and cropping the plurality of infrared feature frames in at least five different sizes in the face area in the infrared image;

inputting the infrared characteristic frames with at least five different sizes into at least five infrared living body detection models for living body judgment to obtain infrared judgment results; inputting the depth feature frames with at least one size into at least one depth living body detection model for living body judgment to obtain a depth judgment result;

and when the infrared judgment result and the depth judgment result are both living bodies, determining that the current target object is a living body.

As an implementation manner of the first aspect, the acquiring the depth image and the infrared image includes: acquiring the aligned depth image and infrared image.

As an implementation manner of the first aspect, the acquiring a depth image and an infrared image, and identifying a plurality of human face feature points of a target object human face region in the infrared image and the depth image respectively includes:

the method comprises the steps of obtaining a depth image and an infrared image, and if the depth image and the infrared image are determined to meet preset quality conditions, respectively identifying a plurality of human face characteristic points of a target object human face area in the infrared image and the depth image.

As an implementation manner of the first aspect, the inputting the plurality of infrared feature boxes of at least five different sizes into at least five infrared living body detection models for living body judgment to obtain infrared judgment results includes:

inputting the infrared feature frames with at least five different sizes into at least five infrared living body detection models to perform living body judgment, obtaining living body probabilities output by the infrared living body detection models, performing weighted summation on the living body probabilities according to respective weights of the infrared living body detection models, judging whether a sum value of the weighted summation is greater than or equal to a preset threshold value, if so, determining that the current target object is a living body, and if not, determining that the current target object is a prosthesis.

acquiring a depth image and a color image, and respectively identifying a plurality of human face characteristic points of a target object human face area in the color image and the depth image;

cutting a plurality of depth feature frames in a face area in the depth image and cutting a plurality of color feature frames in the face area in the color image by taking the plurality of face feature points as anchor points; cropping the plurality of depth feature frames in at least one size in the face region in the depth image, and cropping the plurality of color feature frames in at least five different sizes in the face region in the color image;

inputting the plurality of color feature frames with at least five different sizes into at least five color in-vivo detection models for in-vivo judgment to obtain color judgment results; inputting the depth feature frames with at least one size into at least one depth living body detection model for living body judgment to obtain a depth judgment result;

and when the color judgment result and the depth judgment result are both living bodies, determining that the current target object is a living body.

As an implementation manner of the first aspect, the acquiring the depth image and the color image includes: acquiring the aligned depth image and color image.

As an implementation manner of the first aspect, the acquiring a depth image and a color image, and identifying a plurality of face feature points of a target object face region in the color image and the depth image respectively includes:

the method comprises the steps of obtaining a depth image and a color image, and if the color image and the depth image are determined to meet preset quality conditions, respectively identifying a plurality of human face characteristic points of a target object human face area in the color image and the depth image.

As an implementation manner of the first aspect, the inputting the plurality of color feature frames of at least five different sizes into at least five color living body detection models for living body judgment to obtain color judgment results includes:

inputting the multiple color feature frames with at least five different sizes into at least five color in-vivo detection models to perform in-vivo judgment, obtaining the in-vivo probability output by each color in-vivo detection model, performing weighted summation on the in-vivo probability according to the respective weight of each color in-vivo detection model, judging whether the sum of the weighted summation is greater than or equal to a preset threshold, if so, determining that the current target object is a living body according to the color judgment result, and if not, determining that the current target object is a prosthesis according to the color judgment result.

acquiring a color image, an infrared image and a depth image, and respectively identifying a plurality of human face characteristic points of a target object human face area in the color image, the infrared image and the depth image;

cutting a plurality of depth feature frames in a face area in the depth image, cutting a plurality of color feature frames in the face area in the color image, and cutting a plurality of infrared feature frames in the face area in the infrared image by taking the plurality of face feature points as anchor points; cropping the plurality of depth feature frames in at least one size at a face region in the depth image, cropping the plurality of color feature frames in at least five different sizes at a face region in the color image, and cropping the plurality of infrared feature frames in at least five different sizes at a face region in the infrared image;

inputting the plurality of color feature frames with at least five different sizes into at least five color in-vivo detection models for in-vivo judgment to obtain color judgment results; inputting the infrared characteristic frames with at least five different sizes into at least five infrared living body detection models for living body judgment to obtain infrared judgment results; inputting the depth feature frames with at least one size into at least one depth living body detection model for living body judgment to obtain a depth judgment result;

and when the color judgment result, the infrared judgment result and the depth judgment result are all living bodies, determining that the current target object is a living body.

As an implementation manner of the first aspect, the acquiring a color image, an infrared image, and a depth image includes: acquiring the aligned color image, infrared image and depth image.

As an implementation manner of the first aspect, the acquiring a color image, an infrared image, and a depth image, and identifying a plurality of human face feature points of a target object human face region in the color image, the infrared image, and the depth image respectively includes:

the method comprises the steps of obtaining a color image, an infrared image and a depth image, and if the color image, the infrared image and the depth image are determined to meet preset quality conditions, respectively identifying a plurality of human face characteristic points of a target object human face area in the color image, the infrared image and the depth image.

In a second aspect, an embodiment of the present application provides a door lock system based on living body detection, including:

the camera module is used for respectively acquiring at least two images of an infrared image, a color image and a depth image;

the acquisition module is used for acquiring at least two images of the infrared image, the color image and the depth image as target images and identifying a plurality of human face characteristic points of a human face area of a target object in each target image;

the cutting module is used for cutting a plurality of feature frames in the face area of each target image by taking the plurality of face feature points as anchor points;

the judging module is used for inputting the plurality of feature frames of each target image into a living body detection model corresponding to the target image to carry out living body judgment so as to obtain a judgment result;

and the determining module is used for determining whether the target object is a living body according to the judgment result so as to carry out identity authentication.

In a third aspect, an embodiment of the present application provides an electronic device, including: a memory, a processor and a computer program stored in the memory and executable on the processor, the processor implementing the method of living body detection according to the first aspect or any implementation manner of the first aspect when executing the computer program.

In a fourth aspect, an embodiment of the present application provides a computer-readable storage medium, which stores a computer program, and the computer program, when executed by a processor, implements the biopsy method according to the first aspect or any implementation manner of the first aspect.

In a fifth aspect, an embodiment of the present application provides a computer program product, which, when run on an electronic device, causes the electronic device to execute the living body detection method according to the first aspect or any implementation manner of the first aspect.

It can be understood that, for the beneficial effects of the second aspect to the fifth aspect, reference may be made to the description of the first aspect and any implementation manner of the first aspect, and details are not described herein again.

Drawings

In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings needed to be used in the embodiments or the prior art descriptions will be briefly described below, and it is obvious that the drawings in the following description are only some embodiments of the present application, and it is obvious for those skilled in the art to obtain other drawings based on these drawings without inventive exercise.

Fig. 1 is a schematic flow chart illustrating an implementation of a method for detecting a living body according to an embodiment of the present application;

fig. 2 is a schematic flowchart illustrating an implementation of step S110 in a method for detecting a living body according to an embodiment of the present application;

FIG. 3 is a process schematic of a method of in vivo detection provided by an embodiment of the present application;

FIG. 4 is a schematic flow chart illustrating another method for detecting a living body according to an embodiment of the present disclosure;

FIG. 5 is a schematic structural diagram of a door lock system based on liveness detection according to an embodiment of the present application;

fig. 6 is a schematic structural diagram of an electronic device according to an embodiment of the present application.

Detailed Description

In the following description, for purposes of explanation and not limitation, specific details are set forth, such as particular system structures, techniques, etc. in order to provide a thorough understanding of the embodiments of the present application. It will be apparent, however, to one skilled in the art that the present application may be practiced in other embodiments that depart from these specific details. In other instances, detailed descriptions of well-known systems, devices, circuits, and methods are omitted so as not to obscure the description of the present application with unnecessary detail.

The term "and/or" as used in this specification and the appended claims refers to and includes any and all possible combinations of one or more of the associated listed items.

Reference throughout this specification to "one embodiment" or "some embodiments," or the like, means that a particular feature, structure, or characteristic described in connection with the embodiment is included in one or more embodiments of the present application. Thus, appearances of the phrases "in one embodiment," "in some embodiments," "in other embodiments," or the like, in various places throughout this specification are not necessarily all referring to the same embodiment, but rather "one or more but not all embodiments" unless specifically stated otherwise. The terms "comprising," "including," "having," and variations thereof mean "including, but not limited to," unless expressly specified otherwise.

Further, in the description of the present application, "a plurality" means two or more. The terms "first," "second," "third," and "fourth," etc. are used merely to distinguish one description from another, and are not to be construed as indicating or implying relative importance.

In order to explain the technical solution described in the present application, the following description will be given by way of specific examples.

Fig. 1 is a schematic flow chart of an implementation of a living body detection method according to an embodiment of the present application, where the living body detection method in this embodiment can be executed by an electronic device. Electronic devices include, but are not limited to, computers, tablets, servers, cell phones, cameras, wearable devices, or the like. The server includes but is not limited to a stand-alone server or a cloud server, etc. As shown in fig. 1, the living body detecting method may include steps S110 to S140.

S110, acquiring a depth image and an infrared image, and respectively identifying a plurality of human face characteristic points of a target object human face area in the infrared image and the depth image.

The electronic equipment collects the infrared image and the depth image of the target object through the collecting equipment. Wherein the acquisition device may be a depth camera based on structured light, binocular, or time of flight (TOF) technology.

The acquisition frequency of the depth image and the acquisition frequency of the infrared image can be the same or different, and corresponding setting is carried out according to specific functional requirements. For example, the depth image and the infrared image are alternately acquired at a frequency of 60 Frames Per Second (FPS), or the depth image and the infrared image are respectively acquired at a frequency of 30 FPS.

In some implementations, the electronic device itself includes the acquisition device, and the electronic device can acquire the infrared image and the depth image of the target object in real time through the acquisition device itself.

In some other implementation manners, the electronic device does not include the acquisition device itself, but is connected with the acquisition device, and the acquisition device establishes a communication connection with the electronic device. The acquisition equipment acquires the infrared image and the depth image of the target object, and the electronic equipment receives the infrared image and the depth image transmitted by the acquisition equipment.

In some embodiments, the infrared image and the depth image acquired by the acquisition device need to be further registered (or aligned), that is, a corresponding relationship between pixels in the depth image and the infrared image is found through a registration algorithm, so as to eliminate parallax caused by a difference in spatial position between the two images. The registration may be done by a dedicated processor in the acquisition device or by an external processor. The registered depth image and infrared image can realize multiple functions, such as accelerating human face living body detection or recognition.

In some embodiments, face detection or face recognition may be performed on the infrared image and the depth image, respectively.

In other embodiments, the face detection can be performed on the infrared image, and then the face part in the depth image can be directly positioned by using the corresponding relation of pixels, so that the face detection algorithm for the depth image at one time can be reduced, and the calculation cost is reduced.

In other embodiments, the face detection may be performed on the infrared image of the previous frame first, and when the depth image of the next frame is acquired, only the depth value of the pixel at the position where the face is located is obtained, that is, only the depth image of the face portion is output, so that the amount of computation of the extraction algorithm of the depth image is reduced, and the data transmission bandwidth is reduced at the same time. On the contrary, the face detection or the recognition can be performed on the depth image first, and then the face detection or the recognition in the infrared image is accelerated by utilizing the corresponding relation of the pixels.

In some embodiments, the face feature points in the infrared image or depth image are identified, and the initial face frame and face feature points in the infrared image or depth image may be obtained by a single step inference face detector (Retinaface) model.

Taking face detection on the infrared image as an exemplary description, a process of acquiring an initial face frame and face feature points in the infrared image by using a retinafece model is described below, and as shown in fig. 2, the process may include the following steps S111 to S114. It should be understood that the process of performing face detection on a depth image or performing face detection on a color image by using the Retinaface model is similar to the process, and is not described in detail later.

And S111, transmitting the infrared image to a trunk feature extraction network, and outputting the last three first effective feature layers.

In one embodiment, the Retinaface model comprises a backbone feature extraction network, and the backbone feature extraction network is used for extracting features of different scales of the infrared image.

As a non-limiting implementation, the stem feature extraction network includes a depth separable convolution (mobilene) model or a depth residual error network (Resnet) model, preferably a mobilene model, whose parameters can be reduced.

And S112, inputting a feature map pyramid network (FPN) structure by using the three first effective feature layers to obtain three effective feature fusion layers.

In one embodiment, the Retinaface model further comprises an FPN structure, and the FPN structure is used for fusing the multi-scale features extracted by the trunk feature extraction network.

As a non-limiting implementation manner, the convolution kernel is a 1 × 1 convolution layer to adjust the number of channels of the three first effective feature layers, and the first effective feature layers with the number of channels adjusted are used to perform upsampling and feature fusion to obtain three effective feature fusion layers with different sizes, which is the constructed FPN structure. It should be understood that the convolution kernel size of the convolutional layer can be designed according to practical situations, and is not limited herein.

And S113, performing reinforced feature extraction on the obtained effective feature fusion layer, and outputting a second effective feature layer.

In one embodiment, the retanaface model further includes a Single Stage Headless Face Detector (SSH) structure, and the SSH structure is used for performing enhanced feature extraction on a plurality of fusion features of different sizes output by the FPN structure.

As a non-limiting implementation, the SSH structure is used to perform enhanced feature extraction on three different sizes of valid feature fusion layers output by the FPN structure. The SSH structure comprises three parallel convolution layer structures, wherein the three convolution layer structures can be respectively connected in parallel by 1 convolution layer 3 multiplied by 3, 2 convolution layers 3 multiplied by 3 and 3 convolution layers 3 multiplied by 3, so that the receptive field of the convolution layers is increased, and the calculation of parameters is reduced. And merging the three effective characteristic fusion layers with different sizes through a fusion (concat) function after the three effective characteristic fusion layers with different sizes pass through three parallel convolution layer structures to obtain a new effective characteristic layer. Namely, three effective feature fusion layers with different sizes can obtain three new second effective feature layers with SSH structures and different sizes through three parallel convolution layer structures.

And S114, performing face prediction by using the second effective characteristic layer to obtain an initial face frame and face key point information.

In one embodiment, the Retinaface model also includes an output layer. The output layer is used for carrying out face prediction on the second effective characteristic layer output by the SSH structure and outputting an initial face frame and face key point information.

The second effective feature layers with three different sizes of the SSH structure are equivalent to dividing the whole infrared image into grids with different sizes, each grid includes two prior frames, and each prior frame represents a certain area on the infrared image. And carrying out face probability detection on each prior frame, and predicting whether the prior frame contains a face or not by setting a confidence coefficient threshold value to be 0.5. Specifically, the probability of the prior frame is compared with a threshold, and if the probability of the prior frame is greater than or equal to the threshold, the prior frame contains the face, which is the initial face frame. It should be understood that the threshold of the confidence level may be specifically set according to actual situations, and is not limited herein.

Further, the prior frame is adjusted to obtain the face key points (or feature points), and it should be understood that the number of the face feature points may include 98 or 106, which may be designed according to the actual situation. The method selects a plurality of feature points of the human face, preferably seven feature points of the human face, each feature point of the human face needs two adjusting parameters, and x-axis coordinates and y-axis coordinates of the center of each prior frame are adjusted to obtain coordinates of the feature points of the human face.

It should be understood that the model for face detection in the present application includes, but is not limited to, a Retinaface model, and may also include a Multi-task convolutional neural network (MTCNN) model, etc., which is not limited herein. The Retinaface model and the MTCNN model can realize face region detection and face key point detection.

S120, cutting a plurality of depth feature frames in a face area in the depth image by taking a plurality of face feature points as anchor points, and cutting a plurality of infrared feature frames in the face area in the infrared image; and cropping the plurality of depth feature frames in at least one size in the face area in the depth image, and cropping the plurality of infrared feature frames in at least five different sizes in the face area in the infrared image.

In some embodiments, a plurality of feature boxes are each cropped in the depth image and the infrared image with a plurality of face feature points each being an anchor point. Note that the feature frame size for clipping is the same as the feature frame size for model training in each living body detection model in step S130.

In some implementations, the face keypoints are preferably at least 7, including any keypoint for the left eye, any keypoint for the right eye, any keypoint for the nose, any keypoint for the left cheek, any keypoint for the right cheek, a left mouth corner keypoint, and a right mouth corner keypoint. And according to the position of the face part where the selected key point is located, at least 7 key points are used as the center point, the upper left corner, the lower left corner, the upper right corner, the lower right corner, the left boundary, the right boundary, the upper boundary or the lower boundary and the like of the feature frame, and at least 7 infrared feature frames with the same size and at least 7 depth feature frames with the same size are cut. When a plurality of feature frames with different sizes are needed, at least 7 key points are used as anchor points of the feature frames, a plurality of groups of feature frames with different sizes are cut, each group of feature frames comprises at least 7 feature frames, and the size of each group of feature frames is the same.

Specifically, the depth feature frames include a depth left-eye feature frame, a depth right-eye feature frame, a depth nose feature frame, a depth left cheek feature frame, a depth right cheek feature frame, and a depth left mouth corner feature frame and a depth right mouth corner feature frame. It should be understood that the face features of the seven feature frames are more, so that living body detection can be better performed, and an accurate detection result can be obtained. In addition, the selection of the infrared feature frame is similar to the depth feature frame, which is not described herein again.

In some embodiments, the living body detection model (i.e., the depth living body detection model) for performing living body judgment based on the depth image is at least one, and when the number of the living body detection models is multiple, the sizes of the feature frames input by the multiple living body detection models may be completely the same, or partially the same, or completely different, which is not particularly limited in this application. The number of the living body detection models (i.e., the infrared living body detection models) for determining the living body based on the infrared image is at least five, and the sizes of the feature frames input by the at least five living body detection models may be completely the same, or partially the same, or completely different, which is not limited in this application.

As a non-limiting implementation, the infrared liveness detection model includes at least five, and the infrared feature boxes input by the five infrared liveness detection models include at least five different sizes. That is, at least five infrared biopsy models each input a set of infrared feature boxes of a different size. In this case, at least five sets of infrared feature frames of at least five different sizes are cropped in the face region in the infrared image, each set of infrared feature frames including 7 infrared feature frames. At least five infrared physical examination models each input a set of infrared feature boxes of one size.

S130, inputting the infrared feature frames with at least five different sizes into at least five infrared living body detection models for living body judgment to obtain infrared judgment results; and inputting the depth feature frames with at least one size into at least one depth living body detection model for living body judgment to obtain a depth judgment result.

The following describes a process of performing living body judgment using infrared living body detection models of five different input sizes, taking the infrared living body detection models of five different input sizes as an exemplary description.

Specifically, live body judgment is performed based on 7 infrared feature boxes using 5 infrared live body detection models, which are 2 lightweight models (light-weight models), 1 Central Difference Convolution Networks (CDCN) model, 1 CDC-DAN model, and 1 Sc-resext 26 model, respectively.

As a non-limiting example, one of the 2 lightweight models enters 7 infrared feature boxes of 16 × 16 size, and the other model enters 7 infrared feature boxes of 32 × 32 size. The size of the infrared feature box input by the CDC-DAN model is 48 x 48. The size of the feature box input by the CDCN model is 60 × 60. The Sc-resenext 26 model inputs 7 infrared feature boxes of size 72 x 72. Partial features can be decoupled from complete facial features by inputting feature frames with different sizes into different network models, so that the models can focus on texture features of the feature frames with different sizes, and can capture more generalized differences between a real person and a prosthesis, thereby finally obtaining more accurate in-vivo detection results.

The lightweight model includes, but is not limited to, network models such as squeezet, MobileNet, ShuffleNet, or Xception, and the output result is a probability that the target object is determined to be a living body, which is referred to as a living body probability.

The CDCN model can effectively capture the essential characteristics of the prosthesis, and is not easily influenced by the external environment, so that more robust modeling capability can be provided. The output of the method is a characteristic image, the pixel values of all pixel points on the characteristic image are obtained according to the size of the output characteristic image, and the average pixel value of the characteristic image is calculated after binarization. The calculated average pixel value is used as the living body probability output by the CDCN model.

The CDC-DAN model is the binding of CDCN to DAN, which is able to capture detailed patterns by aggregating intensity and gradient information. While DAN with a self-attention mechanism can enhance the resolution of feature characterization through spatial and channel interdependence. The combination of the two methods simulates rich context information on local strength and gradient characteristics, so that the performance of human face living body detection can be obviously improved. The method comprises the steps of outputting a characteristic image, obtaining pixel values of all pixel points on the characteristic image according to the size of the output characteristic image, calculating an average pixel value of the characteristic image, comparing the average pixel value with a preset threshold value, and judging whether a current input characteristic frame is a living body. For example, if the feature image is 15 × 15, that is, it is described that the feature image includes 225 pixel points, the pixel values of all the pixel points are binarized and then summed, and an average value is calculated, and the average value is used as the live probability output by the CDC-DAN model.

The SE-ResNeXt26 model is obtained by applying a SE module in the Squeeze-and-Excitation Networks to a residual module (residual block) of the ResNeXt model, and can improve convergence speed and accuracy. The output is the probability of determining that the target object is a living body.

When the face of the current target object is finally judged to be a living body, according to living body probabilities output by the infrared living body detection models respectively, carrying out weighted summation on the living body probabilities according to respective weights of the infrared living body detection models, judging whether the final sum is greater than or equal to a preset threshold value, if so, judging that the living body is the living body, and if not, judging that the living body is the prosthesis; or judging whether the final sum is greater than a preset threshold value, if so, determining the living body, and if not, determining the prosthesis, thereby obtaining an infrared judgment result.

It should be noted that the five network models are pre-trained living body detection models, i.e., trained living body detection models. Taking one model CDC-DAN as an example for explanation, 48 x 48 feature frames are randomly taken in a face region during training, each feature frame is independently sent to a network for learning, 7 feature frames which accord with the size of an input image of the CDC-DAN model are selected in the face region according to key points and sent to the model together during actual living body detection, and the CDC-DAN model outputs the living body probability of the face of a target object.

Further, live body detection based on the depth image. In the process, after at least 7 face key points corresponding to the infrared image in the depth image are determined, at least seven depth feature frames meeting the input size requirement of the depth living body detection model are cut out.

In one embodiment, the deep biopsy model is any one of the above network models, and is preferably a lightweight model, because the depth image contains less information, and a deep biopsy model with high accuracy can be trained by using a simple model. At the moment, at least seven depth feature frames with the same size are input into the depth living body detection model, the living body probability is output, and when the living body probability is larger than or equal to a preset threshold value, the target object face is a living body, otherwise, the target object face is a prosthesis. And further obtaining a depth judgment result.

In another embodiment, the depth biopsy models are a plurality of the network models, and the depth biopsy results with higher accuracy can be obtained by using the plurality of models. At this time, the input image sizes of the plurality of depth live body detection models may be completely the same, or may be completely different, or may be partially the same. Respectively inputting a plurality of groups of depth feature frames (each group of depth feature frames comprises at least seven depth feature frames) into a plurality of depth living body detection models, respectively outputting living body probabilities by each depth living body detection model, carrying out weighted summation on the living body probabilities of the depth living body detection models, and when the sum value obtained by the weighted summation is determined to be greater than or equal to a preset threshold value, determining that the target object face is a living body, otherwise, determining that the target object face is a prosthesis. Thus, a depth judgment result is obtained.

It should be noted that, the deep biopsy model and the infrared biopsy model have the same architecture, but the specific values of the weighting parameters and the like in the architecture are different, and the values in the architecture are obtained in the pre-training process, which is not limited herein.

Further, in some other embodiments, when feature frames (infrared feature frames or depth feature frames) corresponding to different face key points are input into different living body detection models, weights of the different feature frames can be adaptively adjusted. For example, since the realistic 3D head model is also different from a real person in terms of eye whites, the weight of the eye feature box is appropriately increased, and the model is more effective. As a non-limiting example, as shown in fig. 3, the following three target faces are all prostheses, such as a head model or a mask made of resin or silica gel, and we can determine the prostheses by increasing the weight of the eye feature frame, and no erroneous determination is generated. In other embodiments, when feature frames (infrared feature frames and/or depth feature frames) corresponding to different face key points are input into different living body detection models, weights of the different feature frames can be adjusted according to attributes of the face image. For example, when the living body examination is performed under the condition that the target object wears the mask, at least 7 feature frames need to include the left-eye feature frame and the right-eye feature frame, and the weight occupied by the left-eye feature frame and the right-eye feature frame can be properly increased, so that the result is more accurate.

And S140, determining whether the current target object is a living body according to the infrared judgment result and the depth judgment result.

In some embodiments of the present application, the infrared determination result may be obtained based on the living body probabilities output by the plurality of infrared living body detection models, and the depth living body detection model may also output the depth determination result.

In some embodiments, when both the infrared determination results of the plurality of infrared living body detection models and the depth determination result of the depth living body detection model are living bodies, it is indicated that the target object is a living body. In the embodiments, only the living body detection completely passing through the two models is considered as a real human living body, so that the accuracy of the living body detection result is greatly improved.

According to the in-vivo detection method provided by the embodiment, on one hand, infrared data and depth data are input into the in-vivo detection model for in-vivo detection, and a user does not need to cooperate to make a specified action, so that convenience is improved; on the other hand, infrared data and depth data are respectively input into two living body detection models through a method based on an image block (patch), namely a characteristic frame, the depth living body detection model can be used for judging whether the image is a video, a printed paper photo, a hole-digging bent photo and the like, the infrared living body detection model can be used for preventing a 3D head model, a mask, a facial mask and the like, in the embodiment of the application, living body detection is carried out by combining data of the two modes, and the accuracy of a living body detection algorithm is greatly improved.

Through tests, the depth living body detection model provided by the embodiment of the application has the real person passing rate of more than 99.9%, and can effectively prevent paper attack or paper face hole digging attack of more than 99.5%. The infrared living body detection model provided by the embodiment of the application has the real person passing rate of more than 99.9%, and can prevent the attack of a high-precision 3D head model or a resin mask by more than 95%.

In some embodiments, as shown in FIG. 4, another in vivo detection method is provided on the basis of the embodiment shown in FIG. 1. The living body detecting method shown in fig. 4 is improved on the basis of the embodiment shown in fig. 1, and a step of determining whether an image satisfies a preset quality condition is added. The living body detecting method shown in fig. 4 includes the following steps S110' to S140. It should be understood that the embodiment shown in fig. 4 has the same steps as the embodiment shown in fig. 1, and the description thereof is omitted.

Step S110', a depth image and an infrared image are obtained, and if the depth image and the infrared image are determined to meet preset quality conditions, a plurality of human face characteristic points of a target object human face area in the infrared image and the depth image are respectively identified.

In these embodiments, determining whether the acquired depth image and infrared image satisfy the preset quality condition includes, but is not limited to: judging whether the head posture is reasonable or not through the depth image or the infrared image; judging whether the face is shielded (the judgment can be carried out through the face contour edge detection of the depth image or the infrared image); and judging whether the illumination is normal or not (the illumination can be judged through the pixel value of the infrared image). And if any one or more than one of the images is not used, directly not entering the subsequent step, namely, the step of respectively identifying a plurality of human face characteristic points of the target object human face region in the infrared image and the depth image and the subsequent step are not entered, the images need to be collected again, and the step of respectively identifying a plurality of human face characteristic points of the target object human face region in the infrared image and the depth image and the subsequent step are entered until the newly collected images meet the preset quality condition.

In this embodiment, the quality of the image is screened, and the subsequent steps are performed only on the image meeting the preset quality condition, so that the accuracy of the detection result can be further improved.

In the foregoing embodiments, the living body detection is performed by using data of two modalities, namely, an infrared image and a depth image, and in other embodiments, the living body detection may be performed by using data of two modalities, namely, a color image and a depth image; in other embodiments, the data of two modalities, namely a color image and an infrared image, can be used for the living body detection; in other embodiments, the living body detection can be performed even by using data of three modes, namely an infrared image, a color image and a depth image, and the accuracy of the detection result can be provided by using the data of the three modes compared with the data of the two modes. It should be noted that the processes of these embodiments can be similar to the previous embodiments, and the descriptions of the same parts are omitted here.

When the living body detection method utilizes a color image, the color image is correspondingly acquired, a plurality of characteristic points of a face area in the color image are identified, and then a plurality of color characteristic frames are cut in the face area in the color image by taking the plurality of characteristic points as anchor points. The plurality of color feature frames can be in a plurality of sizes, for example, at least five sizes, correspondingly, the living body detection is carried out by utilizing at least five color living body detection models, each color living body detection model respectively outputs a living body probability, the living body probabilities respectively output by the color living body detection models are weighted and summed, the sum value obtained by weighted summation is compared with a preset threshold value, when the sum value is larger than or equal to the preset threshold value, the living body is identified, and otherwise, the prosthesis is identified. Thus, a color judgment result is obtained.

In some embodiments, if both the color determination result and the depth determination result are living bodies, the target object is a living body; and if at least one of the color judgment result and the depth judgment result is a prosthesis, the target object is the prosthesis. In some embodiments, the infrared determination result, the color determination result, and the depth determination result are all living bodies, and the target object is a living body; and if at least one of the infrared judgment result, the color judgment result and the depth judgment result is a prosthesis, the target object is the prosthesis.

In some embodiments, the color liveness detection model may be a plurality of the aforementioned infrared liveness detection models.

It should be noted that the architecture of the color biopsy model is the same as that of the infrared biopsy model, but the specific values of the weighting parameters and the like in the architecture are different, and the values in the architecture are obtained in the pre-training process, which is not limited herein.

It should be understood that, the sequence numbers of the steps in the foregoing embodiments do not imply an execution sequence, and the execution sequence of each process should be determined by its function and inherent logic, and should not constitute any limitation to the implementation process of the embodiments of the present application.

An embodiment of the present application further provides a living body detection apparatus. The details of the biopsy device not described in detail are described in the embodiments of the method described above.

Referring to fig. 5, fig. 5 is a schematic block diagram of a door lock system based on liveness detection according to an embodiment of the present application. The living body detecting apparatus includes: a camera module 50, an acquisition module 51, a cropping module 52, a judgment module 53, and a determination module 54, wherein:

a camera module 50 for respectively acquiring at least two images of an infrared image, a color image and a depth image;

an acquiring module 51, configured to acquire at least two images of an infrared image, a color image, and a depth image as target images, and identify a plurality of face feature points of a face region of a target object in each target image;

a cropping module 52, configured to crop a plurality of feature frames in the face region in each target image with the plurality of face feature points as anchor points;

a judging module 53, configured to input the feature frames of each target image into a living body detection model corresponding to the target image to perform living body judgment, so as to obtain a judgment result;

and the determining module 54 is configured to determine whether the target object is a living body according to the determination result, so as to perform identity authentication.

Further, if the target object is judged to be a living body, identifying the face information on the target object and comparing the face information with the registered face information preset on the cloud end, and if the face information on the target object is consistent with the face information on the cloud end, opening a door lock; otherwise, the door lock is not opened.

In some embodiments, the liveness detection based door lock system includes:

a camera module 50 including a depth camera and an infrared camera for respectively acquiring a depth image and an infrared image;

an obtaining module 51, configured to obtain a depth image and an infrared image, and identify a plurality of human face feature points of a target object face region in the infrared image and the depth image, respectively;

a cropping module 52, configured to crop a plurality of depth feature frames in the face region in the depth image and crop a plurality of infrared feature frames in the face region in the infrared image with each of the plurality of face feature points as an anchor point; cropping the plurality of depth feature frames in at least one size in the face area in the depth image, and cropping the plurality of infrared feature frames in at least five different sizes in the face area in the infrared image;

the judgment module 53 is configured to input the multiple infrared feature frames with at least five different sizes into at least five infrared living body detection models to perform living body judgment, so as to obtain an infrared judgment result; inputting the depth feature frames with at least one size into at least one depth living body detection model for living body judgment to obtain a depth judgment result;

and the determining module 54 is configured to determine that the current target object is a living body to perform identity authentication when the infrared determination result and the depth determination result are both living bodies.

As an implementation manner of these embodiments, the obtaining module 51 is specifically configured to: acquiring the aligned depth image and infrared image.

As an implementation manner of these embodiments, the obtaining module 51 is specifically configured to:

As an implementation manner of these embodiments, the determining module 53 is specifically configured to:

In some embodiments, the liveness detection based door lock system includes:

a camera module 50 including a depth camera and a color camera for acquiring a depth image and a color image, respectively;

an obtaining module 51, configured to obtain a depth image and a color image, and identify a plurality of face feature points of a face region of a target object in the color image and the depth image, respectively;

a cropping module 52, configured to crop a plurality of depth feature frames in the face region in the depth image and crop a plurality of color feature frames in the face region in the color image, with each of the plurality of face feature points as an anchor point; cropping the plurality of depth feature frames in at least one size in the face region in the depth image, and cropping the plurality of color feature frames in at least five different sizes in the face region in the color image;

the judgment module 53 is configured to input the plurality of color feature frames with at least five different sizes into at least five color in-vivo detection models for performing in-vivo judgment, so as to obtain a color judgment result; inputting the depth feature frames with at least one size into at least one depth living body detection model for living body judgment to obtain a depth judgment result;

and the determining module 54 is configured to determine that the current target object is a living body to perform identity authentication when the color determination result and the depth determination result are both living bodies.

As an implementation manner of these embodiments, the obtaining module 51 is specifically configured to: acquiring the aligned depth image and color image.

In some embodiments, the liveness detection based door lock system includes:

a camera module 50 including a color camera, an infrared camera, and a depth camera, and respectively configured to collect a color image, an infrared image, and a depth image;

an obtaining module 51, configured to obtain a color image, an infrared image, and a depth image, and respectively identify a plurality of human face feature points of a target object face region in the color image, the infrared image, and the depth image;

a cropping module 52, configured to crop a plurality of depth feature frames in the face region in the depth image, crop a plurality of color feature frames in the face region in the color image, and crop a plurality of infrared feature frames in the face region in the infrared image, with each of the plurality of face feature points as an anchor point; cropping the plurality of depth feature frames in at least one size at a face region in the depth image, cropping the plurality of color feature frames in at least five different sizes at a face region in the color image, and cropping the plurality of infrared feature frames in at least five different sizes at a face region in the infrared image;

the judgment module 53 is configured to input the plurality of color feature frames with at least five different sizes into at least five color in-vivo detection models for performing in-vivo judgment, so as to obtain a color judgment result; inputting the infrared characteristic frames with at least five different sizes into at least five infrared living body detection models for living body judgment to obtain infrared judgment results; inputting the depth feature frames with at least one size into at least one depth living body detection model for living body judgment to obtain a depth judgment result;

and the determining module 54 is configured to determine that the current target object is a living body to perform identity authentication when the color determination result, the infrared determination result, and the depth determination result are all living bodies.

As an implementation manner of these embodiments, the obtaining module 51 is specifically configured to: acquiring the aligned color image, infrared image and depth image.

An embodiment of the present application also provides an electronic device, as shown in fig. 6, which may include one or more processors 60 (only one is shown in fig. 6), a memory 61, and a computer program 62, e.g., a biopsy program, stored in the memory 61 and executable on the one or more processors 60. The one or more processors 60, when executing the computer program 62, may perform the various steps in the in vivo detection method embodiments. Alternatively, the one or more processors 60, when executing the computer program 62, may implement the functionality of the various modules/units of the liveness detection device embodiments, not limited herein.

Those skilled in the art will appreciate that fig. 6 is merely an example of an electronic device and is not intended to limit the electronic device. The electronic device may include more or fewer components than shown, or combine certain components, or different components, e.g., the electronic device may also include input-output devices, network access devices, buses, etc.

In one embodiment, the Processor 60 may be a Central Processing Unit (CPU), other general purpose Processor, a Digital Signal Processor (DSP), an Application Specific Integrated Circuit (ASIC), an off-the-shelf Programmable Gate Array (FPGA) or other Programmable logic device, discrete Gate or transistor logic device, discrete hardware component, or the like. A general purpose processor may be a microprocessor or the processor may be any conventional processor or the like.

In one embodiment, the storage 61 may be an internal storage unit of the electronic device, such as a hard disk or a memory of the electronic device. The memory 61 may also be an external storage device of the electronic device, such as a plug-in hard disk, a Smart Memory Card (SMC), a Secure Digital (SD) card, a flash memory card (flash card), and the like provided on the electronic device. Further, the memory 61 may also include both an internal storage unit and an external storage device of the electronic device. The memory 61 is used for storing computer programs and other programs and data required by the electronic device. The memory 61 may also be used to temporarily store data that has been output or is to be output.

It will be apparent to those skilled in the art that, for convenience and brevity of description, only the above-mentioned division of the functional units and modules is illustrated, and in practical applications, the above-mentioned function distribution may be performed by different functional units and modules according to needs, that is, the internal structure of the apparatus is divided into different functional units or modules, so as to perform all or part of the functions described above. Each functional unit and module in the embodiments may be integrated in one processing unit, or each unit may exist alone physically, or two or more units are integrated in one unit, and the integrated unit may be implemented in a form of hardware, or in a form of software functional unit. In addition, specific names of the functional units and modules are only for convenience of distinguishing from each other, and are not used for limiting the protection scope of the present application. The specific working processes of the units and modules in the system may refer to the corresponding processes in the foregoing method embodiments, and are not described herein again.

An embodiment of the present application further provides a computer-readable storage medium, which stores a computer program, and the computer program, when executed by a processor, implements the steps in the embodiment of the living body detecting method.

An embodiment of the present application provides a computer program product, which, when run on an electronic device, enables the electronic device to implement the steps in the living body detection method embodiment.

In the above embodiments, the descriptions of the respective embodiments have respective emphasis, and reference may be made to the related descriptions of other embodiments for parts that are not described or illustrated in a certain embodiment.

Those of ordinary skill in the art will appreciate that the various illustrative elements and algorithm steps described in connection with the embodiments disclosed herein may be implemented as electronic hardware or combinations of computer software and electronic hardware. Whether such functionality is implemented as hardware or software depends upon the particular application and design constraints imposed on the implementation. Skilled artisans may implement the described functionality in varying ways for each particular application, but such implementation decisions should not be interpreted as causing a departure from the scope of the present application.

In the embodiments provided in the present application, it should be understood that the disclosed apparatus/electronic device and method may be implemented in other ways. For example, the above-described apparatus/electronic device embodiments are merely illustrative, and for example, a module or a unit may be divided into only one logic function, and may be implemented in other ways, for example, a plurality of units or components may be combined or integrated into another system, or some features may be omitted or not executed. In addition, the shown or discussed mutual coupling or direct coupling or communication connection may be an indirect coupling or communication connection through some interfaces, devices or units, and may be in an electrical, mechanical or other form.

Units described as separate parts may or may not be physically separate, and parts displayed as units may or may not be physical units, may be located in one place, or may be distributed on a plurality of network units. Some or all of the units can be selected according to actual needs to achieve the purpose of the solution of the embodiment.

In addition, functional units in the embodiments of the present application may be integrated into one processing unit, or each unit may exist alone physically, or two or more units are integrated into one unit. The integrated unit can be realized in a form of hardware, and can also be realized in a form of a software functional unit.

The integrated modules/units, if implemented in the form of software functional units and sold or used as separate products, may be stored in a computer readable storage medium. Based on such understanding, all or part of the flow in the method according to the embodiments described above may be implemented by a computer program, which is stored in a computer readable storage medium and used by a processor to implement the steps of the embodiments of the methods described above. Wherein the computer program comprises computer program code, which may be in the form of source code, object code, an executable file or some intermediate form, etc. The computer readable medium may include: any entity or device capable of carrying computer program code, recording medium, usb disk, removable hard disk, magnetic disk, optical disk, computer memory, read-only memory (ROM), Random Access Memory (RAM), electrical carrier wave signals, telecommunications signals, software distribution medium, and the like. It should be noted that the computer readable medium may contain other components which may be suitably increased or decreased as required by legislation and patent practice in jurisdictions, for example, in some jurisdictions, in accordance with legislation and patent practice, the computer readable medium does not include electrical carrier signals and telecommunications signals.

The above embodiments are only used to illustrate the technical solutions of the present application, and not to limit the same; although the present application has been described in detail with reference to the foregoing embodiments, it should be understood by those of ordinary skill in the art that: the technical solutions described in the foregoing embodiments may still be modified, or some technical features may be equivalently replaced; such modifications and substitutions do not substantially depart from the spirit and scope of the embodiments of the present application and are intended to be included within the scope of the present application.

Claims

1. A method of in vivo detection, comprising:

2. The in-vivo detection method as set forth in claim 1, comprising:

3. The liveness detection method of claim 2 wherein said acquiring a depth image and an infrared image comprises: acquiring the aligned depth image and infrared image.

4. The liveness detection method of claim 2 or 3 wherein said acquiring a depth image and an infrared image and identifying a plurality of facial feature points of a facial region of a target object in said infrared image and said depth image, respectively, comprises:

5. The in-vivo detection method as set forth in claim 1, comprising:

6. The in-vivo detection method as set forth in claim 1, comprising:

7. The in-vivo detection method as set forth in claim 2 or 6, wherein the inputting of the plurality of infrared feature boxes of at least five different sizes into at least five infrared in-vivo detection models for in-vivo judgment to obtain infrared judgment results comprises:

8. A door lock system based on in vivo detection, comprising:

9. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the liveness detection method of any one of claims 1 to 7 when executing the computer program.

10. A computer storage medium storing a computer program, wherein the computer program, when executed by a processor, implements the living body detecting method according to any one of claims 1 to 7.