CN111428577B

CN111428577B - Face living body judgment method based on deep learning and video amplification technology

Info

Publication number: CN111428577B
Application number: CN202010138178.8A
Authority: CN
Inventors: 张静; 郭权浩; 刘娟秀; 刘霖; 杜晓辉; 倪光明; 刘永
Original assignee: University of Electronic Science and Technology of China
Current assignee: University of Electronic Science and Technology of China
Priority date: 2020-03-03
Filing date: 2020-03-03
Publication date: 2022-05-03
Anticipated expiration: 2040-03-03
Also published as: CN111428577A

Abstract

The invention discloses a human face living body judgment method based on deep learning and video amplification technologies, and relates to image processing, neural networks, software development and data set manufacturing and acquisition. The adopted attack face covers electronic equipment such as a tablet, a mobile phone and a computer, and also covers non-electronic equipment such as a common photo, a poster and the like and a three-dimensional model, wherein the three-dimensional model further comprises different environmental light intensities and different distances. The data set has greater generalization. The method processes data through an Euler influence amplification algorithm and a convolutional neural network, writes an interface to achieve real-time identification of input data, and displays whether the acquired face is a real face or an attack face in real time.

Description

Face living body judgment method based on deep learning and video amplification technology

Technical Field

The invention relates to an intelligent detection technology, which covers the field of optical-mechanical computer software and is particularly designed for a human face living body detection technology in biological recognition.

Background

Since the first demonstration of the faceID face unlocking technology released by apple company at 1 am, 9 and 13 months in 2017, the living body detection technology enters a high-speed development era, and the main function of the living body detection is to distinguish a real face from a false face. Because the face unlocking technology is non-contact, convenient and fast, the face unlocking technology of the user is brought on line in each field one after another, in 2019, in 10 months, the medium explodes out of an express cabinet of a domestic Hovere express company to generate a heavy BUG, and the Hovere express cabinet brings on a new function of face unlocking express, but investigators find that the machine can be cheated by printing photos to take out the express. More and more electronic devices are also equipped with a face brushing unlocking technology successively, and even mainstream payment platforms are on line with a face brushing function, so that the safety of face brushing is particularly important.

At present, living body detection technical schemes at home and abroad are numerous and comprise action matching type living body detection, off-line 3D structured light living body detection, off-line near-infrared living body detection and the like. The methods have respective advantages and disadvantages, and the matched living body detection belongs to non-silent detection, requires a user to make a specified action, and is relatively complicated.

Disclosure of Invention

In order to overcome the problems, the invention provides a method for judging the living body by acquiring a face video and an image, dividing the face video and the image into a real face image and a false face image and identifying the difference between the two types of images through a video amplification technology and a deep learning training model. The main objective is to carry on the safety equipment that needs to be carried out living body detection face identification and unlock.

The technical scheme of the invention is a human face living body judgment method based on deep learning and video amplification technology, which comprises the following steps:

step 1: obtaining a training sample; respectively acquiring picture data and video data of a large number of live faces at different distances, different light intensities, different angles and different shielding conditions by adopting high-resolution and low-resolution cameras as positive sample data;

step 2: the data obtained in the step 1 are played by a display and printed with photos and posters, and high-resolution and low-resolution cameras are used for shooting and displaying images respectively in the same way to obtain picture data and video data which serve as negative sample data;

and step 3: preprocessing the acquired positive and negative sample data;

step 3-1: classifying the data of the positive and negative samples into video data and image data, checking whether each frame of the video data contains face data or not, checking whether each image contains face data or not, and reserving the video and image data containing the face data;

step 3-2: performing primary backup on video data in the positive sample data, and processing the video data as a preprocessing data set of a video amplification technology;

step 3-3: carrying out background mask processing on a non-human face background area in a video data picture in the positive sample data, eliminating the background and only leaving a picture containing face information;

step 3-4: performing video stable and video align processing on the data processed in the step 3-1 to realize face alignment and algorithm shake elimination, wherein the video align processing enables the face of the picture to be aligned in each frame and have approximately the same size, and the video stable processing eliminates the shake influence on the face due to environmental factors and other human factors;

and 4, step 4: calculating the heart rate weight;

step 4-1: adopting facial landworks to calibrate face key points in the data obtained in the step 3;

step 4-2: locking a forehead rectangular area with sufficient facial blood flow and obvious change rule and triangular areas of two faces and faces by using key points and OpenCV polylines and fillPoly functions; one triangular area is an isosceles triangular area with the root of the nasal bridge bone as the vertex and the connecting line of two corners of the mouth as the bottom side; the other is an inverted triangle area with the two corners of the eyes and the bottom of the lower lip as three vertexes;

step 4-3: carrying out video amplification processing on the data acquired in the step 3, respectively testing heart rates of the three areas acquired in the step 4-2, recording numerical values x1, x2 and x3 as initial heart rates serving as references, and measuring the real heart rate y of the original data object by using standard medical equipment as calibration;

step 4-4: the step 4-3 is carried out for a plurality of times, batch test heart rates and standard heart rates are obtained, and fitting curves are utilized

Weights a1, a2 and a3 of the three heart rates are obtained;

and 5: taking another part of the positive and negative sample data as a preprocessing data set for deep learning; storing each frame of video data in the partial data as image data, performing batch processing on the image data by adopting a trained buffer model and an OpenCV (open channel code) haarcacade file to obtain each image face box, and finally making into a tfrecrd VOC (volatile organic compound) format data file;

step 6: training the neural network model based on the ssd _ mobilenet by adopting the file obtained in the step 5 until the training is successful;

and 7: when the video amplification technology is used for actual detection, video data of a target to be detected are obtained, and heart rates of three areas of the target to be detected are calculated by sequentially adopting the methods in the step 3-3, the step 3-4, the step 4-1, the step 4-2 and the step 4-3;

and 8: calculating a real heart rate according to the weighted value calculated in the step 4, comparing the real heart rate with a set heart rate threshold value, if the real heart rate does not meet the requirement, determining that the human face is attacked, otherwise, performing a step 9;

and step 9: and (6) identifying the face by adopting the neural network model based on the ssd _ mobilenet trained in the step 6, and judging whether the face is a living face.

The method adopts the video data to judge the heart rate characteristic of the face, adopts the heart rate characteristic and the neural network to judge whether the face is a real face or an attack face, does not need interaction of testers in the judging process, has high identification precision on the attack face, and is difficult to cheat by the existing attack mode.

Drawings

FIG. 1 is a flow chart of the video amplification technique detection process of the present invention.

FIG. 2 is a flow chart of the convolutional neural network of the present invention.

FIG. 3 is a flow chart of the data set generation process of the present invention.

Fig. 4 is an overall working flow of the present invention.

Detailed Description

For the purpose of making the objects, technical solutions and advantages of the present invention more apparent, the present invention will be described in further detail below with reference to the accompanying drawings and embodiments, it being understood that the embodiments described herein are only for the purpose of explaining the present invention, but not for the purpose of limiting the present invention.

Referring to fig. 1 to 4, the invention is a human face in-vivo detection system based on deep learning and video amplification technology, which comprises a data acquisition camera, an electronic device and a non-electronic device for attacking a face, a computer, an android mobile phone and a set of man-machine interaction UI of the android, wherein the man-machine interaction UI is used for processing human face image data acquired by a mobile phone camera, and judging whether the currently collected face data is true or false through analysis of the image data.

The data acquisition module comprises an acquisition object 10, a camera slide 20, a high resolution camera 30, a low resolution camera 40, an acquisition object position slide 50. The acquisition 10 object can be a person or a face attack object, such as an electronic device carrying an attack face: tablet, computer, cell-phone. Non-electronic device carrying attack face: poster, photo. The position of the acquisition object 10 may be slid on the position slider 50 to adjust the distance from the

acquisition cameras

30, 40. The

acquisition cameras

30, 40 may slide on the camera slide 20 to determine which camera to use.

A human face living body detection method based on deep learning and video amplification technology comprises the following steps:

step 1-1: the high resolution camera 30 in the adjustment figure 1 carries out facial image collection to the different people's faces of gathering object 10, at the in-process of gathering, the slow movement gathers object 10, make it constantly slide on the sliding strip 50 and gather the image of different distance positions, the face of gathering should constantly adjust the contained angle of gathering facial to the camera at the collection in-process, and allow to have different shelters from facial, such as wearing the cap, sunglasses, gauze mask etc., the data of gathering should have different distances, different angles, different people's faces, different light intensity, different facial shelters from. After the acquisition by the high resolution camera 30 is completed, the same procedure is performed using the low resolution camera 40.

Step 1-2: the collected data are carried on the electronic equipment to become an attack face, the high-resolution camera 30 and the low-resolution camera 40 are used for collecting, only the light intensity needs to be adjusted in the collection, and the distance of the movable sliding belt 50 is adjusted to the optimal distance only according to different attack faces carried on the electronic equipment. And collecting data of the attack face, and finally taking the total collected data as data to be processed.

Step 2-1: preprocessing the acquired positive and negative sample data before formal data processing;

step 2-2: firstly, classifying data of positive and negative samples into video data and image data respectively, checking whether each frame of the video data contains face data or not, whether each image contains face data or not, and reserving data meeting requirements;

step 2-3: carrying out primary backup on a video format data set in the positive sample data, and processing the video format data set to be used as a preprocessing data set of a video amplification technology;

step 2-4: firstly, removing the back of a non-human face in a part of video data set picture, and only leaving a picture containing face information;

step 2-5: processing the data processed in the step 2-4 to realize the face alignment and the algorithm shake elimination

Step 2-6: through the steps, the data meeting the requirement of realizing the video amplification technology is obtained, and the face key point calibration is realized on the face information in the data;

step 2-7: and a forehead rectangular area with sufficient facial blood flow and obvious change rule and triangular areas of two faces are locked by using key points;

step 2-8: performing video amplification processing on the acquired data, respectively testing heart rates of the three areas obtained in the steps 2-7, recording numerical values x1, x2 and x3 as initial heart rates serving as references, and measuring the real heart rate y of the original data object by using standard medical equipment as calibration;

step 2-9: performing the step 4-3 for multiple times, acquiring batch test heart rates and standard heart rates, and calculating the heart rate weight of each area;

step 2-10: when the video amplification technology is used for actual detection, video data of a target to be detected are obtained, and then the video data are processed by adopting the methods of the step 2-4 and the step 2-5;

step 2-11: acquiring the weight values in the steps 2-9, calculating the heart rate of the video-amplified face data by using the weight values, comparing the heart rate value with a human body heart rate threshold value, and determining whether the tested face is a living body;

step 3-1: for a non-static attack face, deep learning is used for processing, collected data to be processed are subjected to batch processing by utilizing a trained mask model and haarcacade files provided by OpenCV to obtain face boxes, the face boxes are finally manufactured into a tfrecrd VOC data file format, and the data are finally divided into three parts, namely a training set, a verification set and a test set.

Step 3-2: training input data by using a neural network model based on ssd _ mobilenet until loss is reduced to be below 1, setting a detection target threshold to be 0.6, stopping training after the test accuracy of the evaluation model reaches 97%, and establishing the model.

And 4, step 4: and (3) converting the model obtained in the step (3-2) into an android supporting model by utilizing a toco tool of tenserflow, and carrying and using the android supporting model. And compiling a human-computer interaction interface to display the identification condition in real time.

Claims

1. A human face living body judgment method based on deep learning and video amplification technology comprises the following steps:

and step 3: preprocessing the acquired positive and negative sample data;

step 3-4: performing video stable and video align processing on the data processed in the step 3-1 to realize face alignment and jitter elimination, wherein the video align processing enables the face of the picture to be aligned and have the same size in each frame, and the video stable processing eliminates the jitter influence on the face due to environmental factors and other human factors;

and 4, step 4: calculating the heart rate weight;

Weights a1, a2 and a3 of the three heart rates are obtained;

and 5: taking a part of positive and negative sample data as a preprocessing data set for deep learning; storing each frame of video data in the partial data as image data, performing batch processing on the image data by adopting a trained buffer model and an OpenCV (open channel code) haarcacade file to obtain each image face box, and finally making into a tfrechrd VOC (volatile organic compound) format data file;

and 6: training the neural network model based on the ssd _ mobilenet by adopting the file obtained in the step 5 until the training is successful;