WO2021217764A1

WO2021217764A1 - Human face liveness detection method based on polarization imaging

Info

Publication number: WO2021217764A1
Application number: PCT/CN2020/092040
Authority: WO
Inventors: 张堃博; 孙哲南; 田雨; 王乐源
Original assignee: 天津中科智能识别产业技术研究院有限公司; 中国科学院自动化研究所
Priority date: 2020-04-27
Filing date: 2020-05-25
Publication date: 2021-11-04
Also published as: CN111539348A

Abstract

A human face liveness detection method based on polarization imaging, comprising the steps of: collecting, at a plurality of angles in real time, polarization images of a target to be detected; and performing polarization analysis on the polarization images by using a polarization human face feature learning and classification network model based on a Siamese neural network structure, so as to recognize that the target to be detected is real or fake. The method uses a non-contact real time detection means, does not require active cooperation of a user, and can implement accurate detection even in a user non-cooperation state, thereby having good user experience.

Description

A living body detection method based on polarization imaging

Technical field

The present invention relates to the technical field of face recognition, in particular to a method for detecting a living body of a face based on polarization imaging.

Background technique

Face recognition technology has been widely used due to its convenient, fast, non-contact and other characteristics. However, with the continuous development of technical means such as smart phones, cameras, and social networks, facial information is becoming more and more accessible. Some criminals use photos, videos or masks containing facial information to attack the face recognition authentication system to obtain the identity permissions of legitimate users, posing a threat to the protection of personal identity and privacy information, and even seriously affecting the security of citizens’ identity and property .

There are several currently most widely used live detection solutions. The first is based on the cooperation of the user. The user performs actions such as blinking, turning the head, opening the mouth, and reading according to the instructions, and uses the algorithm to analyze the facial motion to make judgments. Whether it is a real face; the second method uses infrared or multi-spectral imaging devices to analyze the difference in reflectance characteristics of real and fake faces to the spectrum to achieve live body detection. In addition, there are live body detection methods that use texture information, depth information, and so on.

The above methods either require the active cooperation of the user, the user experience is poor, or the requirements for the environment of the collection are relatively strict, and the detection results are easily affected by the environment. In addition, methods based on deep learning have also been proposed to be used in the research of living body detection. The generalization ability is poor, and it can only play a better role in the specified data set. It performs poorly in practical applications, and at the same time, it cannot effectively respond to new attack methods.

Summary of the invention

The purpose of the present invention is to address the technical shortcomings existing in the prior art and provide a method for live detection of human faces based on polarization imaging, which can realize real-time and accurate live detection of human faces and has extremely high environmental robustness. , Can effectively deal with various deceptive attacks.

The technical solutions adopted to achieve the purpose of the present invention are:

A method for living body detection based on polarization imaging, including the steps:

Real-time collection of polarization images at multiple angles of the target to be detected;

The polarization facial feature learning and classification network model based on the twin neural network structure is used to perform polarization analysis on the polarization image to identify the authenticity of the target to be detected.

Specifically, a multi-angle polarization image acquisition device is used to acquire real-time polarization images of the target to be detected at different angles.

Wherein, the multi-angle polarization image acquisition device includes one of a micro-polarizer array type polarization camera, a split focal plane type polarization camera, and a split path type polarization camera.

Preferably, the multi-angle polarization image acquisition device integrates a polarization element containing four angles on the surface of the CMOS photosensitive element, which can achieve four polarization images of the same object with the same resolution but different polarization directions in one shot.

Preferably, a polarization collecting device is used to obtain the polarization images of the target to be detected at four angles of 0°, 45°, 90°, and 135°.

Preferably, the polarization analysis of the polarization image using the polarization facial feature learning and classification network model based on the twin neural network structure is to use a deep learning method to combine the polarization degree map of the real face and the polarization degree map of the deceptive attack They are labeled separately, sent to the convolutional neural network for learning, feature extraction of facial polarization visual information, and support vector machine SVM for classification and identification, finally realizing the distinction between true and false faces.

Aiming at the limitations of the existing living body detection methods in terms of environment and data sets, the present invention uses the physical property of polarization to classify true and false faces, which can effectively overcome the shortcomings of the existing methods and effectively resist the biological characteristics of the recognition system. A variety of presentation attacks have good user experience and detection accuracy, and at the same time, it is more robust in a variety of environments.

The invention utilizes the physical association between the polarized optical information and the face material, and by analyzing the characteristic difference of the reflected light, effectively distinguishes the face skin and deceptive attacks. The method does not require any cooperation from the user (blinking, turning head, opening mouth, etc.) , At the same time, it can automatically and deeply learn the texture information of real faces, effectively distinguish real faces and deceive fake faces.

The invention can detect a variety of face spoofing attacks including printing paper attacks, printing photo attacks, screen display attacks, rubber mask attacks, silicone mask attacks, etc., and can complete living body detection under multiple light source settings including visible light, near-infrared light, polarization Visible light and polarized near-infrared light can be used in indoor and outdoor scenes.

Description of the drawings

Fig. 1 is an implementation flow chart of a method for live detection of a face based on polarization imaging;

Figure 2 is a schematic diagram of the flow of polarized face live detection;

Figure 3 is a working principle diagram of the polarization collection device;

Fig. 4 is a schematic diagram of collecting a four-direction polarization image of a human face by using a chip integrated with a four-direction polarization sensor;

Figure 5 is a polarized image of a human face in various situations.

Detailed ways

Hereinafter, the present invention will be further described in detail with reference to the drawings and specific embodiments. It should be understood that the specific embodiments described here are only used to explain the present invention, but not used to limit the present invention.

The present invention uses polarized image information as input, and uses a trained and automated polarization facial feature knowledge learning and classification network model (face living detection algorithm model) based on the twin neural network structure to perform polarization analysis on the input polarized image to achieve The detection of the authenticity of the human face, so as to realize the detection of the living body.

As shown in FIG. 2, the method for detecting a living body of a human face based on polarization imaging of the present invention includes the following steps:

S1, real-time collection of polarization images at multiple angles of the target to be detected;

S2. Use the polarization facial feature learning and classification network model based on the twin neural network structure to perform polarization analysis on the polarization image to identify the authenticity of the target to be detected.

It should be noted that in the present invention, a multi-angle polarization image acquisition device (polarization acquisition device) can be used to collect real-time polarization images of the target to be detected at four different angles of 0 degrees, 45 degrees, 90 degrees, and 135 degrees. , And then through the analysis of these images to identify, as shown in Figure 3.

Among them, after the polarization collection device is used to obtain the polarization images of the four angles of 0°, 45°, 90°, and 135° of the face to be detected, the four polarization images are recorded as I ₀ , I ₄₅ , I ₉₀ , I ₁₃₅ . Then use the Stokes vector to characterize the polarization:

In the actual environment, the circular polarization component is less and almost negligible, so V is ignored in the calculation of the Stokes vector, and the polarization degree image I _{DOLP is} calculated with the help of formula (2),

In the present invention, the multi-angle polarization image acquisition device used may be a polarization image sensor. The polarization image sensor, as shown in FIG. 4, integrates a polarization element containing four directions and angles on the surface of a traditional CMOS photosensitive element, which can achieve four polarization images of the same object in one shot. Through the analysis of a single frame image, the camera can obtain four grayscale images with the same resolution (1224 x 1024 pixels) but with different polarization directions, as shown in Figure 4.

For example, a SONY chip integrated polarization image sensor can be used with a resolution of 1224x1024 pixels, which can obtain four-angle target polarization images in real time and analyze four-angle grayscale images at the same time, with high integration and compact size.

It should be noted that, in the present invention, the polarization image is not limited to the above-mentioned four angle polarization images of 0°, 45°, 90° and 135°. It can also be three outside, or five and other different numbers. indivual.

It should be noted that during the image acquisition process, different light sources can be used to provide illumination according to the actual acquisition environment. In an environment with a better daylight environment, no light source can be used or a conventional white light source can be used to provide illumination. In the recognition environment at night, near-infrared light sources can be used to provide illumination.

It should be noted that the existing devices that can achieve real-time multi-angle polarization image acquisition can be micro-polarizer array polarization cameras, split-focus plane polarization cameras, split-path polarization cameras, etc., all of which can be used in the present invention to perform polarization. Image acquisition to obtain the target polarization images at the four angles of 0°, 45°, 90° and 135° at the same time.

Due to the difference between the existing photo printing attacks, video playback attacks, 3D masks and other deceptive attacks and real faces in terms of material, texture, roughness and other attributes, as shown in Figure 5, they affect the reflected polarized light characteristics. Analyzing the obtained polarization degree map can detect whether the current recognition object is a real human face, thereby realizing living body detection.

Specifically, since the real face and the fake face show the difference in intensity value and intensity distribution in the polarization diagram, through deep learning or statistical analysis, the attack on photographic paper printing, ordinary white paper printing, and 3D By analyzing the polarization map of deceptive attacks such as masks and the polarization of real faces, accurate classification can be achieved.

Among them, if the deep learning method is used, the polarization map of the real face and the polarization map of the deceptive attack can be marked separately, sent to the convolutional neural network for learning, and SVM is used for classification and identification, and finally real and fake faces are realized. The distinction. For example, the twin neural network structure is used for feature extraction of face polarization visual information, and support vector machine SVM is used for feature classification to identify true and false faces. You can also use the lightweight MobileNetV2 architecture as the network infrastructure to improve overall operating efficiency.

Among them, for example, by means of statistical analysis, the true and false face polarization maps can be analyzed in terms of mean, standard deviation, kurtosis, etc., so as to realize the classification of true and false faces.

Among them, the present invention uses polarized image information as input, learns the knowledge of face physical materials by using a deep neural network, establishes correlation with the essential characteristics of the physical materials of the target to be detected, and trains an automated face detection algorithm model. As shown in Figure 1, the specific implementation steps are as follows:

Step S1: Set up four-angle polarization face image sensing equipment and light source device. The polarization sensor adopts the chip integrated type, which can obtain the polarization images of four angles (0 degree, 45 degree, 90 degree and 135 degree) synchronously in real time. Corresponding gray image can be obtained;

Step S2: Collect a polarized face data set, which contains no less than 30 persons to be collected. Paper printing, photo printing, screen display, rubber mask, silicone mask and other attack methods, visible light, infrared light, polarization Visible light, polarized infrared light and other light source settings, indoor, outdoor, daytime, night and other scenes;

Among them, in order to train and test the face authenticity detection model, 33 people's various faces were collected to attack the multi-modal face database. The database contains four types of attacks: photo printing attacks, paper printing attacks, screen display attacks, and mask attacks, as well as visible light, near-infrared light, polarized visible light, and polarized near-infrared light. A total of 4800 indoor and outdoor human face pictures.

Step S3, training a polarization facial feature learning and classification network model based on the twin neural network model;

Step S4, test the polarization face feature learning and classification network model, and have the ability to correctly distinguish between true and false faces on known and unknown data;

Step S5: Apply the trained algorithm model to the display scene and deploy it to the chip-integrated polarization perception recognition system to realize the application of polarized face live detection, which can realize the face of multiple attacks, multiple scenarios, and multiple people. Real-time accurate Lupin live face recognition.

The training and testing of the present invention use relatively small data resources. The polarization face image data set of 30 people only needs to be trained for 150 epochs. During the test, the image is down-sampled to 224×224 pixels and sent to the network. The detection of the present invention The model has the characteristics of generality under small data training conditions. Using the same polarization sensor, it can detect the authenticity of unknown face targets in a large number of unknown scenes, and achieve 100% detection accuracy.

The present invention can finally realize real-time multi-person live detection above 15fps, and the detection distance can reach 5 meters. It can resist paper printing, photo printing, screen display attacks, rubber masks, silicone masks and other types of presentation attacks. It can be used in daytime and also Indoor and outdoor scenes at night, with strong versatility and scalability.

The invention utilizes the physical property of the target's polarization characteristic to perform living body detection, has extremely high robustness to the environment, and can be effectively applied to various complex scenes, such as indoor, outdoor, daytime, night and other scenes. Since it is difficult for the forged human face to be completely consistent with the real human face in terms of material, texture, roughness, etc., regardless of the technical means used, the present invention can effectively detect a variety of known and unknown deceptive attacks. Including paper and photo printing attacks, screen display attacks, silicone face-like attacks and other attacks.

Therefore, the present invention can realize the task of face detection in daytime and night indoor and outdoor scenes for various attack types such as paper printing, photo printing, screen display, silicone mask, rubber mask, etc.

In addition, the present invention adopts a non-contact real-time detection method, does not require active cooperation of the user, and can realize accurate detection even in the state of non-cooperation of the user, and the user experience is better.

What’s important is that the present invention is different from the existing pure deep learning methods, does not need to rely on a large number of data sets, and can obtain high-accuracy detection results with small samples, which has higher recognition accuracy than traditional methods. There is no need to manually design and change the recognition algorithm for multiple attack types.

Moreover, the hardware system of the present invention has simple structure, small size, light weight, and high integration, is easy to install, and can be applied to a variety of detection environments.

The living body detection method of the present invention has the characteristics of high accuracy, strong robustness and wide versatility, and can be quickly deployed and applied under various lighting conditions after a small amount of data training.

The invention can replace the existing traditional biometric anti-spoofing technology in a variety of applications with high requirements for identity authentication, such as transportation, financial services, customs clearance certification, insurance issuance, election examination, public security, etc., thereby realizing automatic and efficient , Accurate and intelligent face detection in vivo, which improves the safety protection of the biometric system.

The above are only the preferred embodiments of the present invention. It should be pointed out that for those of ordinary skill in the art, without departing from the principle of the present invention, several improvements and modifications can be made. These improvements and Retouching should also be regarded as the protection scope of the present invention.

Claims

A method for detecting live human faces based on polarization imaging, which is characterized in that it comprises the following steps:

Real-time collection of polarization images at multiple angles of the target to be detected;

The polarization facial feature learning and classification network model based on the twin neural network structure is used to perform polarization analysis on the polarization image to identify the authenticity of the target to be detected.
The method of living body detection based on polarization imaging according to claim 1, wherein a multi-angle polarization image acquisition device is used to collect polarization images of the target to be detected at different angles in real time.
The method for detecting a human face based on polarization imaging according to claim 2, wherein the multi-angle polarization image acquisition device includes a micro-polarizer array type polarization camera, a split-focus plane type polarization camera, and a split-path type polarization camera. A sort of.
The method for detecting a human face based on polarization imaging according to claim 2, wherein the multi-angle polarization image acquisition device integrates a polarization element with four directions on the surface of the CMOS photosensitive element, which can achieve the same acquisition in one shot. Four polarization images of the same resolution but different polarization directions of an object.
The method of living body detection based on polarization imaging according to claim 1, characterized in that a polarization collection device is used to obtain polarization images at four angles of 0°, 45°, 90°, and 135° of the target to be detected.
The method for living face detection based on polarization imaging according to claim 1, wherein the polarization analysis of the polarization image is performed by using the polarization facial feature learning and classification network model based on the twin neural network structure through deep learning. The method separately annotates the polarization map of the real face and the polarization map of the deceptive attack, sends them to the convolutional neural network for learning, performs the feature extraction of the polarization visual information of the face, uses the support vector machine SVM for classification and identification, and finally Realize the distinction between true and false faces.