WO2013131407A1

WO2013131407A1 - Double verification face anti-counterfeiting method and device

Info

Publication number: WO2013131407A1
Application number: PCT/CN2013/000228
Authority: WO
Inventors: 李子青; 张志炜; 雷震; 易东
Original assignee: 无锡中科奥森科技有限公司
Priority date: 2012-03-08
Filing date: 2013-03-05
Publication date: 2013-09-12
Also published as: CN102622588A; CN102622588B

Abstract

The invention relates to a double verification face anti-counterfeiting method and device. The method comprises: step 1, carrying out bio-assay on a collected target face to judge whether the target face has biological activity; if it does, then switch to step 2; step 2, if the target face is in a face-identifying application, then calculating the similarity of the collected target face to a face corresponding to the identification result, and if the similarity is greater than a threshold value, then regarding the target face as a real and valid face; if the target face is in a face verification application, then calculating the similarity of the collected target face to a face corresponding to an identity claimed by the target face, and if the similarity is greater than a threshold value, then regarding the target face as a real and valid face. Using the method of the invention, an accurate and reliable face anti-counterfeiting detection result can be provided through the combination of the bio-assay and the authentication.

Description

Double verification face anti-counterfeiting method and device

Technical field

The invention relates to a face anti-counterfeiting method and device, in particular to a double-verification face anti-counterfeiting method and device, belonging to the technical field of image processing and pattern recognition.

Background technique

The face anti-counterfeiting technology is related to the security of the face recognition authentication and authorization system. If there is no face anti-counterfeiting function, the face recognition authentication and authorization system is vulnerable to false face attacks, which may cause serious security problems. For example, an attacker can obtain a face image of a specific target (ie, a designated person) by some means and make a photo, video, or mask, etc., presented to the system in order to obtain illegal rights. Therefore, face anti-counterfeiting technology is receiving more and more attention. At present, the existing international face anti-counterfeiting technology is mainly based on human-computer interaction strategy: The system issues specific instructions, requiring the user to make specific actions such as blinking and pronunciation, and then judge the activity of the input face. According to the common actions, it can be divided into the following three ways - the first one is based on the blink detection of the living body. The literatures that disclose the technology are: 1) Gang Pan, Lin Sun, Zhaohui Wu and Shilong Lao. Eyeblink-based Anti-Spoofmg in Face Recognition from a Generic Webcamera, International Conference on Computer Vision, 2007, 2) K. Kollreider, H. Fronthaler and J. Bigun. Verifying Liveness by Multiple Experts in Face Biometrics, IEEE Conference on Computer Vision and Pattern Recognition Workshop, 2008, 3) Patent No. ZL200710178088.6, the patent document entitled "A Living Body Detection Method and System Based on Human Physiological Motion". The second is based on moving body detection, and the related literature on the disclosure includes: 1) K. Kollreider, H. Fronthaler and J. Bigun. Evaluating Liveness by Face Images and the Structure Tensor, IEEE Workshop on Automatic Identification Advanced Technologies, 2005, 2) Wei Bao, Hong Li, Nan Li and Wei Jiang. A Liveness Detection Method for Face Recognition Based on Optical Flow Field, International Conference on Image Analysis and Signal Processing, 2009. The third is biometric detection based on speech and mouth movements. The related documents of the technology are: G. Chetty and M. Wagner. Liveness Verification in Audio-Video Speaker Authentication. In 10th Australian Int. Conference on Speech Science and Technology, 2004.

This method based on human-computer interaction requires a user to express a specific behavior, so the user is burdened, the user experience is poor, and the time required is long.

In addition, some researchers start from the perspective of multi-spectral and analyze the skin's reflectivity under different spectra for biopsy. The related literatures are: 1) loannis PavHdis, Peter Symosek, The Imaging Issue in an Automatic Face/Disguise Detection System , IEEE workshop on Computer Vision Beyond the Visible Spectrum: Methods and Applications, 2000. 2) Youngshin Kim, Jaekeun Na, Seongbeak Yoon, and Juneho Yi. Masked fake face detection using radiance measurements, J. Opt, Soc. Am, vol. 26, no.4, April 2009. However, this method is still very rough, the accuracy is not ideal, and there is still much room for improvement.

The face anti-counterfeiting method described above can also be used as a living body detection technique because they only judge whether the target face is biologically active. However, in practical applications, there may be real people going to counterfeit. Attacking the designated person's situation, the target face is indeed a real face, but still belongs to the behavior of attacking the face recognition system. Therefore, face anti-counterfeiting technology should not only include live detection. Moreover, the above methods generally have shortcomings such as heavy user burden, long human-computer interaction time, and low accuracy. Therefore, it is imperative to develop an accurate, fast, and wide-ranging face anti-counterfeiting method.

Summary of the invention

The object of the present invention is to overcome the deficiencies in the prior art and provide a dual verification face anti-counterfeiting method and device, which improves recognition accuracy, is convenient, safe and reliable.

According to the technical solution provided by the present invention, a dual verification face anti-counterfeiting method, the method comprising: - Step 1, performing a living body detection on the collected target face to determine whether the target face has biological activity, if the target face is identified With living characteristics, go to step 2;

Step 2: If it is in the face recognition application, calculate the similarity between the collected target face and the face corresponding to the recognition result. If it is greater than a certain threshold, the target face is considered to be a true and effective person. Face

If it is in the face verification application, the similarity between the collected target face and the face corresponding to the claimed identity of the target face is calculated, and if it is greater than a certain threshold, the target face is considered to be true and effective. Face,

Step 1 is not related to the designated person. Step 2 is related to the designated person. When the target face is verified by steps 1 and 2 at the same time, it can be regarded as a true and effective face, otherwise it is considered to be a false face.

If the dual verification face anti-counterfeiting method is based on visible light, step 1 further includes: Step 101: Performing a living body detection on the target face, first collecting a large number of real and false face samples, and extracting various texture features on the target face. , training the living body detection texture classifier, if the target face is recognized as a real face by the living body detection texture classifier, proceed to step 2, otherwise it is determined as a false face;

Step 102: Determine the validity of the target face by human-computer interaction, wherein the system issues an instruction, and the user is required to perform a certain action, and then the system continuously detects whether the target face performs a corresponding action, and if the action is detected within a certain time, If it occurs, it is judged that the target face is a real face, otherwise it is a false face;

Only the target face passes through steps 101 and 102 at the same time, and is considered to pass the living body detection of step 1. Step 2 further includes:

Step 201, first collecting a large number of real face images, extracting texture features for each face image; step 202, and then subtracting the feature vectors of all the collected face images by two or two, according to whether the two images belong to the same person, The subtracted feature vectors are divided into two categories: intra-class and inter-class. The machine learning algorithm is used to train a two-class classifier. The trained classifier can determine whether the two input feature vectors belong to the same person.

Step 203, if it is in the face recognition application, if the target face image and the face image corresponding to the recognition result are determined by the classifier in step 202 to belong to the same person, the target face is considered to be true and valid, otherwise it is false. human face;

If it is in the face verification application, the face image corresponding to the claimed designated person identity is identified by the classifier in step 202 as belonging to the same person, and the target face is considered to be true. Effect, otherwise it is a false face.

If the dual verification face anti-counterfeiting method is based on multi-modality, step 1 further includes: Step 101: roughly determine the biological activity of the target human face, wherein the judging is performed according to one or more of the following manners: Infrared determines the temperature of the target face, determines whether it is close to 37 degrees; determines the depth information of the face through the 3D image to determine whether the face is a 3D object; analyzes the ultrasonic reflectance of the target face by ultrasonic reflection, and determines whether the ultrasonic reflectance of the skin is Similar to the real face; multi-spectral imaging to analyze the reflectivity of the target face in different spectra, to determine whether the multi-spectral reflectance of the skin is similar to the real face, if the target face is judged by one or more of the above methods The information indicator is similar to the real face, then proceeds to step 102;

Step 102: accurately determine the biological activity of the target face, and use the mutual image algorithm to accurately determine the living body.

Only the target face passes through steps 101 and 102 at the same time, and is considered to pass the living body detection of step 1. The mutual business image algorithm comprises the following steps - step 1021, collecting multi-spectral imaging of a large number of real human faces and false human faces at different distances to form a training data set, and performing pixel-level phase on any two images of different spectra of the same individual. In addition, the mutual image group is composed, assuming that two spectra are selected arbitrarily, and the images of the same face in the two spectra are sum, and the mutual image is defined as follows:

Where ρ represents the reflectivity of the face, κ represents the intensity of the light source at the surface of the face, ζ represents the distance of the face from the light source, and (x, y) represents the coordinates on the face image;

Step 1022: For all mutual business images, divide into multiple overlapping or non-overlapping small blocks on multiple scales, extract feature vectors of each small block, and combine feature vectors of all small blocks as global features. Vector

Step 1023: Based on the statistical learning method, the classifier is trained on the training data set to distinguish between true and false faces.

Step 2 further includes: Step 201, collecting a plurality of multi-modal images of real faces, extracting texture features for each image; Step 202, subtracting the feature vectors of the images by two or two, according to whether the two images belong to the same person, The subtracted feature vectors are divided into two categories: intra-class and inter-class. The machine learning algorithm is used to train a two-class classifier. The trained classifier can determine whether the two input feature vectors belong to the same person.

If it is in the face verification application, if the face image corresponding to the claimed designated person identity is identified by the classifier in step 202 as belonging to the same person, the target face is considered to be true and effective, otherwise False face.

Each different imaging type is called a modal, imaging types include visible light imaging, near infrared Imaging, near-ultraviolet imaging, thermal infrared imaging or ultrasound imaging.

A dual verification face anti-counterfeiting device, the device comprising:

Inductive unit for sensing the presence of a human face by means of real-time monitoring using one or more of a near-infrared, ultrasonic, radio frequency or visible light camera;

a multimodal source, comprising one or more of an active source in multiple spectra, a 3D structured light for 3D imaging, or an ultrasonic generator;

Multi-modal data acquisition device for collecting multi-spectral imaging of a human face, thermal infrared light imaging by the human body itself, one or more of a 3D image of a human face or ultrasound imaging;

a multi-modal face detecting unit, configured to detect a face position in the multi-modal image, and send the detected face image to the multi-modal double-verification face anti-counterfeiting unit;

The multi-modal dual verification face anti-counterfeiting unit is used to verify whether the target face is a real and effective face; the display unit is configured to display the face anti-counterfeiting result,

The multi-modal dual-authentication face anti-counterfeiting unit further includes: a multi-modal human face detection unit for performing living body detection on the target face; and a multi-modal face verification unit for authenticating the target face .

When the multi-modal human face living body detecting unit performs the living body detection on the target face, firstly, the biological activity of the target face is roughly judged, wherein the judgment is performed according to one or more of the following modes: determining the target by thermal infrared The temperature of the face is judged to be close to 37 degrees; the depth information of the face is judged by the 3D image to determine whether the face is a 3D object; the ultrasonic reflectance of the target face is analyzed by ultrasonic reflection, and whether the ultrasonic reflectance of the skin is determined with the real person The face is similar; the multi-spectral imaging is used to analyze the reflectivity of the target face under different spectra, and it is judged whether the multi-spectral reflectance of the skin is similar to the real face. If the target information of the target face is judged by one or more of the above methods, If the real faces are similar, the biological activity of the target face will continue to be accurately determined, and the acquired multi-spectral face images will be accurately determined by using the mutual business image algorithm.

The mutual business image algorithm includes the following steps: collecting multi-spectral imaging of a large number of real human faces and false human faces at different distances to form a training data set, and performing pixel-level division of images of any two different spectra of the same person, composing Mutual quotient image group, assuming that the two spectra are arbitrarily selected and the images of the same human face in the two spectra are sum, and the mutual quotient image is defined as follows:

Where, represents the reflectivity of the face, f represents the intensity of the light source at the surface of the face, Z represents the distance of the face from the light source, (χ, y) represents the coordinates on the face image;

For all mutual business images, divided into multiple overlapping or non-overlapping small blocks on multiple scales, extract the feature vectors of each small block, and combine the feature vectors of all the small blocks as global features based on statistics. Learning method, training the classifier on the training data set to distinguish between real and false faces. When the multi-modal face verification unit authenticates the target face, firstly collect a multi-modal image of a large number of real faces, and extract texture features for each image; secondly, subtract the feature vectors of the images by two, according to Whether the two images belong to the same person, the subtracted feature vectors are divided into two categories: intra-class and inter-class. The machine learning algorithm is used to train a two-class classifier. The trained classifier can determine whether the two input feature vectors belong to The same person; if it is in the face recognition application, if the face image corresponding to the recognition result of the target face image is recognized as belonging to the same person by the above two types of classifiers, the target face is considered to be true and effective, otherwise it is a false person Face; if it is in the face verification application, if the face image corresponding to the claimed person's identity is identified as belonging to the same person by the above two types of classifiers, the target face is considered to be true and effective, otherwise For a false face.

Each of the different imaging types is referred to as a modality, and imaging types include visible light imaging, near-infrared imaging, near-ultraviolet imaging, thermal infrared imaging, or ultrasound imaging.

The advantages of the invention: By combining the living body detection and the identity verification, an accurate and reliable face anti-counterfeiting detection result is provided.

DRAWINGS

1 is a flowchart of a dual verification face anti-counterfeiting method under visible light according to the present invention;

2 is a flow chart of a dual verification face spoofing method in a multi-modal mode according to the present invention;

3 is a structural block diagram of a dual-verification face anti-counterfeiting device in a multi-modal state according to the present invention; FIG. 4 is a flowchart of a dual-verification face anti-counterfeiting device in a multi-modal mode according to the present invention; A schematic diagram of a light source coverage of a dual-verification face anti-counterfeiting device in a multi-modality proposed by the invention;

M 6 is a schematic diagram of the relationship between the gray mean value of the face region image and the distance of the face distance gathering device in the example of the dual verification face anti-counterfeiting device proposed by the present invention;

Figure 7 is a schematic diagram of the human face reflectance curves of black and white in a certain spectral range;

8 is a schematic diagram of a reflectance curve of several common fake faces in a certain spectral range; FIG. 9 is a schematic diagram of a panel of a multi-modal acquisition device in an example of a dual-verification face anti-counterfeiting device according to the present invention;

Figure 10 is a schematic diagram of face imaging of three different spectra, from left to right: visible light, 850 nm near-infrared light and 400 nm purple light;

Figure 11 is a schematic diagram of thermal infrared imaging of a human face;

Figure 12 is a schematic diagram of 3D imaging of a human face;

Figure 13 is a schematic diagram of the reflected wave of the ultrasonic wave on the human face.

detailed description

The invention will now be further described with reference to the specific drawings and embodiments.

The basic principle of the face anti-counterfeiting method proposed by the present invention is based on the idea of double verification. The so-called double verification face anti-counterfeiting includes the following two steps: Step 1: Performing a living, non-living judgment on the input face, this step has nothing to do with the identity of the person, that is, the designated person has nothing to do; Step 2, performing the input face image Face authentication, only when the input face image matches the corresponding identity, is determined to be a true and effective face, and this step is related to the designated person. Only the input face that is judged by the above two steps is considered to be a valid and true face. In step 1, the face biometric detection technology is used, that is, the living face and the non-living body are judged on the face to identify whether it is a real human face; Step 2 is actually verifying the face by a designated person or a non-designated person. In step 2, if the face recognition application is used, the recognition result is the identity corresponding to the target face, and only if the similarity between the two is greater than a certain threshold, the verification is performed by the step, and the threshold can be determined by the manager according to actual needs. Setting; if it is a face verification application, the identity corresponding to the target face is the identity claimed by the target face, and the similarity between the input face image and the corresponding identity face image must be greater than a certain threshold. Face identity authentication. Step 2 is a classification of the image of the designated person and the non-designated person. By combining the information of steps 1 and 2, reliable anti-counterfeiting purposes are achieved.

The method of the present invention uses the above steps 1 and 2 at the same time, because in practical applications, the potential false face type cannot be predicted, and the simple living body detection cannot always maintain high accuracy. On the other hand, even if the false face passes the in vivo detection, there is reason to believe that there is a certain difference between the fake face and the counterfeit designated person's face, so the face can be further enhanced by extracting and identifying the difference. The precision of anti-counterfeiting. Therefore, it is proposed that the input face image needs to pass through the face biometric detection and the face authentication at the same time, so as to be recognized as a true and effective face.

The traditional face anti-counterfeiting research still stays on the face detection, and ignores the verification of the input face. In fact, the face biometric detection in step 1 has nothing to do with the designated person; and the face identity verification in step 2 is for the identity verification of the designated person. The face anti-counterfeiting technology of the invention, combined with face authentication and living body detection, can effectively improve the reliability of face anti-counterfeiting.

In the dual verification face anti-counterfeiting method of the present invention, a specific application form under visible light and a specific application form in multi-modality are further proposed.

The dual verification face anti-counterfeiting method under visible light is suitable for the traditional visible light face recognition and face verification system, and the face anti-counterfeiting task can be completed without additional hardware. A false face image can be regarded as an image obtained by a real face image after some sort of post-processing, so that the image quality will be somewhat lost compared to the real image. By extracting multiple types of texture information from the captured target face, the apparent features of the target face on the skin texture details can be fully exploited, and further classified according to pre-set evaluation criteria. In addition, the accuracy can be further enhanced by introducing a traditional human-computer interaction process.

For the dual-verification face anti-counterfeiting method in the multi-modality proposed by the present invention, the multi-modality is further used to fully exploit the facial features. Existing face recognition and face verification techniques only rely on obtaining a face image using a modality such as visible light or near infrared. We believe that this kind of data collection method can not fully exploit the skin characteristics of human face, and can not achieve high anti-counterfeiting precision. Therefore, a multi-modality face anti-counterfeiting device under multi-modality is proposed and designed. The device includes multi-spectral imaging devices, thermal infrared imaging devices, ultrasonic imaging devices, and the like in different spectra, and fully exploits the physical physical characteristics of human skin from different levels. By carefully analyzing the characteristics of the human face and the typical false face in different modes, the appropriate modal combination is selected to provide the most discriminative features for the subsequent face anti-counterfeiting algorithm.

The dual verification face anti-counterfeiting method under visible light according to the invention can accurately judge the authenticity of the target face by analyzing the skin texture details and/or the movement of the face without relying on additional hardware. The multi-modal dual verification face anti-counterfeiting method proposed by the invention can not only defend against more attack types, but also has less time and user body than the existing face detection algorithm. Good test and high accuracy. The face information obtained by multi-modality can provide richer face information, fully exploit the essential features of the face, and increase the discrimination between the real face and the false face, which can effectively solve the anti-counterfeiting problem of the face.

FIG. 1 is a flow chart of a specific application of the dual verification face anti-counterfeiting method in visible light according to the present invention. Referring to Fig. 1, in the living body detecting step 101, a face living body detecting strategy combining skin texture and facial motion is used. In the authentication step 102, the identity corresponding to the target face (the claimed identity in the face verification application, the identity corresponding to the recognition result in the face recognition application) is verified, and if the matching similarity is greater than a certain threshold, It is considered to be a real face, otherwise it is a false face. Only the target face passed the 101 and 102 steps at the same time to identify the target face as a real face.

The living body detecting step 101 further includes a step 1011 and a step 1012: Step 1011, first extracting various texture features, such as LBP (Local Binary Pattern) > HOG (Histograms of Oriented Gradients) features, etc., and then authenticating through the collection. The face sample is trained by a machine learning algorithm (such as support vector machine SVM) to obtain a skin texture based living body detector. If it is judged to be a real face, go to step 1012.

An example of step 1011 is to filter the target face image by using LBP descriptors of different scales, for example, ^, ^,^, and then multi-scale the image, for example, into small blocks of 1 x1, 3x3, and 5x5. In each block, the histograms of the three LBP descriptors are counted, and all the histograms are linked together as the texture features of the target face.

Then, a large number of images of real and false faces are collected, for example, a real face image of 50 people is collected, and then a face image of different sizes is created using the face image, and then the photo image is acquired again. Remove the face area and extract the features as described in the previous step. Then use the SVM algorithm to train to get a classifier.

At step 1012, the biological activity of the target face is further detected using human-computer interaction. For example, an instruction to blink the user or shake the head can be given by the face recognition system. By detecting whether the target face has made a corresponding action, it is determined whether the target face is a real face. In this step, facial motion estimation can be performed using motion estimation or template matching algorithms. For example, if the form of blinking is used, the motion vector of the target human eye region can be calculated by the optical flow method to determine whether a blinking motion has occurred. Or a template matching algorithm, pre-training a blinking, closed-eye classifier, and then performing motion detection.

An example of a human-computer interaction is that the face recognition system gives instructions to ask the user to blink for a certain period of time, for example 5 seconds. Through the trained human eye state classifier, it is detected whether a blinking-closing eye-blinking process occurs during the period of time. If it appears, it is considered to be a real face, otherwise it is considered a false face, and proceeds to step 1021. The above-mentioned human eye state classifier can collect a large number of blinking and closed eye images in advance, and then use the SVM classifier to train a classifier for eye state for the above-described blink detection.

In the authentication step 102, features (eg, LBP and Gabor features) are extracted from the face image in the database, and then the feature vectors of all the collected face images are subtracted two by two, according to whether the two images belong to the same person, The subtracted feature vectors are divided into two categories: intra-class and inter-class. The machine learning algorithm is used to train a two-class classifier. The trained classifier can determine whether the two input feature vectors belong to the same person.

After the above steps, the similarity between facial features belonging to the same person should be greater than that of different people. The similarity between features. By setting a reasonable threshold, it can be used for authentication: if at step

In 102, the similarity between the target face and its claimed identity is greater than the threshold, and it is considered to have passed the authentication; otherwise it fails.

Figure 2 is a flow chart of the application of the dual verification face anti-counterfeiting method in multi-modal form. The method uses multi-modality as a carrier to collect multi-modal face images. Using the rich information provided by multi-modal images and utilizing the characteristics of different biological characteristics of different bio-features, the multi-modal information fusion is designed to be reasonable. Reliable dual verification face anti-counterfeiting algorithm.

The dual verification face anti-counterfeiting method in the multi-modal form includes a biometric verification step 201 and an authentication step 202.

In the biometric information verification 201, a two-step strategy from coarse to fine is employed.

First, in the first step 2011, using the acquired multimodal face information, a rough judgment is made on the living characteristics of the input face. An example is: First, the temperature is detected by the thermal infrared image. If the temperature range of the real human body is met (for example, whether it is 37 degrees or less), the face depth information is judged by the 3D face image, and if the input face is judged to be a three-dimensional image The object continues to use the ultrasonic reflected wave to measure the ultrasonic reflectivity of the input face. If the reflectance is similar to that of a real person's face, check whether the average image brightness of the multispectral is within a reasonable range. If it is reasonable, it is judged to be true. Face, otherwise it is a false face. In this step, the facial living characteristics as a rough judgment can be dynamically selected according to a specific face modality.

After the first step 2011 is recognized as a real human face, in the second step 2012, the multi-modal imaging for the face, the present invention proposes a face detection algorithm based on mutual business image, which gives more accurate and precise results. The detection result; if the mutual image algorithm determines that the face is a real face, the input face is biologically active. If it is judged as a real person face in the second step 2012, it is a real person face, otherwise it is a false face.

In step 2012, an accurate face detection is performed using a mutual business image algorithm. A cross-commercial image is an image obtained by dividing an image of any two spectra by dividing a pixel value at a corresponding position (Mutual Quotient Image, MQI). The mutual business image can reflect the relationship between the reflection of the face in the two bands and is independent of the shape of the face. According to the definition of the mutual business image, it is assumed that two spectra are randomly selected ^ the same person face is in two lights

Where P represents the reflectivity of the face, κ represents the intensity of the light source at the surface of the face, _Z represents the distance between the face and the light source, and (x, y) represents the coordinates on the face image.

If it is ensured that the luminous powers of the two spectra are the same, then the ratio of the intensities of the two sources is about 1 in the appropriate distance range, so (4) can be approximately equal to

1 ~ 1⁄20,

It can be seen that the mutual quotient image reflects the ratio of the reflectivity of the face in the two spectra of 44, so it is a feature that can reflect the essential characteristics of the face and can be used to design the living body detection algorithm.

In the derivation of equation (5), assume that within the appropriate distance range, ^ the intensity of the two sources The ratio is equal to 1. This assumption can be met in practice by properly designing the light source. For example, the present invention collects the variation of the gray mean value of the face image of the same person between the 480 nm and 850 nm light sources with the same luminous power, and the distance between the light source and the light source is 40 cm to 90 cm, as shown in FIG. It is reasonable to assume that the ratio of the intensities of the two sources is about one.

In the multi-modal human face detection algorithm, after acquiring the mutual commercial image of any two spectra, a reasonable feature can be designed for the detection of the living body. Feature vector extraction can use a variety of methods, such as: intensity histogram, Gabor filter, etc., Likelihood Ratio. After selecting the feature type, the mutual business image can be segmented and multi-scale processed, and the face mutual business image feature vector at different scales and different positions can be obtained, and then the real and false face samples can be collected in large quantities and utilized. The Boosting algorithm trains the biometric classifier.

In the multi-modal face detection algorithm, the difference in reflectivity between real and fake faces should be fully considered, and the light source should be selected. Figure 7 illustrates the human face reflectance curves for blacks and whites in multiple spectra. Figure 8 illustrates the reflectance curves for several common fake faces in multiple spectra, including two different silica gels and photographs. According to these two curves, it can provide a basis for spectral selection in multimodal human face detection.

The specific process of the face detection algorithm based on mutual business image is as follows:

(1) Collecting a large number of real face and fake face data at different distances to form a training data set, and performing MQI calculation on any two images of different spectra of the same person.

(2), on all MQI images, divided into multiple small blocks (overlapping or non-overlapping) on multiple scales, extract the feature vectors of each small block, and combine the feature vectors of all the small blocks as a global Feature vector.

(3) Based on the statistical learning method, the classifier is trained on the training data set, such as: SVM (Support Vector Machine), LDA (Linear Discriminant Analysis), Boosting, and the like.

The cross-commercial image algorithm of the living body detecting step 2012 is further illustrated by way of example below.

For example, 釆 imaging with two light sources of 480 nm and 940 nm, and the obtained face images are respectively Λ ₈ . , ₉₄ . . Then specify _4S . For the reference image, calculate the mutual quotient image for the two bands as

MQi _m , _m ^y) = ^y) ' i _m ^y). The present invention is hereby given by way of example only for the two bands, and the light source of any of a plurality of bands can be selected according to the actual situation.

The 128x 128 MQI image is preprocessed for multi-scale processing and is divided into five scales, which are 128x128 pixels, 64x64 pixels, 32x32 pixels, 16x16 pixels, and 8x8 pixels. Based on the probability model obtained by statistical learning on the training set, for each point on the mutual quotient image, it can be regarded as the likelihood of living and non-living, _ν, _σ ^), _Ρ( ^ _σ ^), where G represents The image comes from a living body,

5 means the image is from a non-living body, and (x, y) is the image coordinates. Dividing these two quantities, you can get the local likelihood ratio of the mutual image:

r _(X)y , _{a) =} ^wl£2 ( 6 ) For the multi-resolution cross-commercial image described above, all local likelihood ratios may constitute a living feature vector with a dimension of 21824.

In order to make the features more distinguishable and have higher computational efficiency, the in vivo feature extraction algorithm uses Boosting for feature selection, and selects the most discriminative 3000-dimensional features from the original high-dimensional features. Then collect a large number of true and false face samples, form a training database, perform feature extraction according to the above-mentioned Boosting selected feature labels, and use the Support Vector Machine (SVM) method to learn a two-class classifier for The input feature vector is used for living and non-living judgments.

In the authentication step 202, the similarity verification of the input face and the corresponding identity is required. The specific verification algorithm is similar to the verification method 102 under visible light, except that the input features are the sum of all features on the multimodal image.

Only when the biometric information verification and the authentication step determine that the input face is a real person's face, the input face is judged by the face anti-counterfeiting.

The face verification algorithm in the authentication step 202 is further illustrated by an example, wherein the face verification application is taken as an example.

Suppose each person has N different modal images. First, each face image is extracted with LBP features and Gabor features to form the feature vector of the image / _t = l _: N.

Then, the feature vectors of each image belonging to the same multimodal image combination are concatenated into a unified feature vector = [/i;...; ], which is a multimodal feature vector for each individual.

In the training process of the face verification classifier, the positive samples are the difference of the multi-spectral feature vectors F belonging to the same person, and the negative samples are the differences of the multi-spectral feature vectors F that do not belong to the same person. The Boosting algorithm is used to select features to obtain a feature subset.

For the multimodal image of each person in the training data set, feature extraction is performed according to the sample selected by Boosting, and the discriminant analysis is performed by using the LDA algorithm.

After the above steps, the similarity between facial features belonging to the same person should be greater than the similarity of facial features between different people. If the similarity between the target face and its claimed identity is greater than the threshold in step 202, then authentication is considered to have passed; otherwise, it fails.

The invention also proposes a double verification face anti-counterfeiting device. FIG. 3 is a structural block diagram of a multi-modality double face verification device according to the present invention. 4 is a flow chart showing the operation of the multi-modal dual verification face anti-counterfeiting device of the present invention.

In the multi-modal dual verification face anti-counterfeiting device of the present invention, the multi-modality includes one or more of modes such as multi-spectroscopy, 3D, and ultrasonic. Since the human skin has different reflectances under different spectra, the present invention introduces a multi-spectral face imaging system for collecting and analyzing the imaging of human faces in different spectra, and fully exploiting the essential characteristics of the human face, thereby Subsequent face anti-counterfeiting provides rich facial features. The choice of spectrum may include near-infrared light, mid-infrared light, far-infrared (thermal infrared), near-violet light, etc., to reflect the different reflection characteristics of the human face as much as possible. In particular, the thermal infrared image refers to the infrared light image emitted by the body's own heat, which is related to the individual's physical and biological characteristics, and has significant individual differences, and is suitable for use as a basis for face anti-counterfeiting. In addition to the hot infrared rays, the above light sources require a multi-spectral acquisition system to provide an active light source.

The invention simultaneously introduces a 3D face image, and ultrasonic imaging, together with the multi-spectral image, to form a multi-modal face image acquisition system. The depth information of the face part obtained by the 3D image is an important basis for the anti-counterfeiting of the face, and can resist attacks of common false faces, such as photos, videos, and the like. The method of ultrasonic imaging, by measuring the reflectivity of the human face to the ultrasonic wave, can provide another measure of the physical characteristics of the human face skin, and furthermore the need for assisting the detection of the human face.

3 and FIG. 4, the multi-modal dual verification face anti-counterfeiting device of the present invention comprises a sensing unit 301, a multi-modality generating source 302, a multi-modal data collecting device 303, a multi-modal face detecting unit 304, The multi-modal dual verification face security unit 305 (including a multi-modal face living body detecting unit 3051, a multi-modal face authentication unit 3052), a control unit 306, and a display unit 307.

The sensing unit 301 is used for biometric sensing using near-infrared, ultrasonic, or radio frequency, or real-time monitoring using a visible light camera. The unit is for sensing the presence of a human face in a specific sensing area, and if a human face is sensed, signals the presence of the object to the control unit 306. In fact, the sensing unit 301 cannot judge that the face is sensed, and as long as an object appears in the sensing area, it is considered to be a human face. The sensing unit 301 can perform face sensing using near-infrared, ultrasonic, or radio frequency, or simply use a visible light camera for real-time monitoring. The size and position of the particular sensing area is preferably set to capture the entire face.

The sensing unit 301 senses the presence of a human face, and performs the following operations: Step 1. If the face is not detected yet, the loop detection is continued; if the presence of the face is detected, the process proceeds to step 2; Step 2, wait After a certain period of time, the face is detected again. If the face still exists, it is considered to be a valid face, and a signal is sent to the control unit 306. If the face no longer exists, it is considered to be an invalid face, and the process proceeds to step 1 to restart. Detection.

In one example, sensing unit 301 is a visible light camera that performs face sensing in a monitored manner. The visible light camera loops through the image and detects the presence of a human face. If there is no face, continue to collect visible light images for face detection; if there is a face, wait 0.5 seconds to collect the image again and detect the face. If the face still exists at this time, it means that a stable and effective face appears, and then sends a signal to the control unit 306 to start the corresponding image collection work; if the face disappears after waiting, the face is likely not to be performed. The face of multimodal image acquisition is considered to be noise and ignored. Continue to collect visible light images and detect the presence of a human face.

The multimodal generation source 302 can include, but is not limited to, one or more of the following: active light sources in multiple spectra (providing the illumination required for multispectral imaging), 3D structured light required for 3D imaging, Ultrasonic generator (to emit ultrasonic waves). In a multispectral light source, the spectral combination may include visible light (in which case a visible light source is not required), but must include a combination of one or more non-visible light sources, the spectral range of the source may be in the near infrared (740mn-4000nm), or Near-ultraviolet (360-400nm) _o can also include thermal infrared imaging, where hot infrared rays are emitted by the human body, eliminating the need to erect additional light sources. However, the spectral combination should not include light that is harmful to the human body, such as medium-ultraviolet light (290-320 nm wavelength) or near-ultraviolet light (200 nm-290 nm wavelength). The 3D structured light can be configured according to actual needs, such as line laser or 3DMR structured light. The frequency of the ultrasonic generator is set according to actual needs, for example, it can be set to 50 kHz.

For the multi-spectral light source, the light emitted by the light source should conform to two principles: 1. In the proper distance range, in the plane directly in front of the multi-mode generating source 302, the light should be kept in a certain area. As shown in Fig. 5, at a certain distance (d) directly in front of the collecting device, the light intensity within a certain area (circular in the figure) should be kept uniform. 2. The luminous intensity should be kept within a reasonable range, so that the imaging device can be collected clearly. To the face image, it is not too strong for the user's discomfort.

The multi-modal data acquisition device 303 is configured to collect multi-spectral light that is irradiated on the human face by the active light source and then reflected, and is also used for collecting hot infrared light emitted by the human body, a 3D image of the human face, and an ultrasonic wave of the human face. Imaging. The collection device includes but is not limited to one or more of the following device units: a camera that responds to light from each source, a receiver or photodiode that responds to each spectral light, a thermal infrared sensor or sensor, a 3D image acquisition device, an ultrasound imaging device Or receiver.

The multi-modality data acquisition unit 303 first includes an imaging device corresponding to each spectrum in 302 for collecting multi-spectral light reflected by the face, including the imaging device and the corresponding filter, and further includes thermal infrared, 3D, ultrasonic imaging device or induction. Device. The multispectral imaging device preferably prefers a camera that responds well to the light of the multispectral source, at which point the return data type is an image. If the conditions are limited, other receiving devices, such as receivers that respond to multi-spectral light, photodiodes, etc., can be used. The return data type is the reflection intensity scalar. One of the multi-spectral sources can correspond to a single camera, or it can respond to multiple bands of multi-spectral sources with a single camera. The camera should have a high sensitivity at the spectrum it responds to. For ultrasonic imaging equipment, the frequency should be the same as that of the ultrasonic generator in 302; if the conditions do not allow, the ultrasonic receiver can also be used. For thermal infrared, a thermal infrared camera is preferred, and a sensor that senses temperature can also be used. For a 3D camera, an image reflecting the depth information of the face is acquired.

In the multi-spectral imaging device of the multi-modal data acquisition unit 303, a filter corresponding to the band is required to eliminate interference of ambient light and other bands of light to the band. The filter should be placed in front of the imaging device of the appropriate segment and attached to the camera lens or receiving device to prevent stray light from entering.

The sensing unit multi-modal face detecting unit 304 is configured to preprocess the face image collected by the multi-modal image forming device, and then detect the pre-processed face image, when all face images are detected. In the case of the face and eyes, it is considered that the face is detected.

The multi-modal dual-verification face anti-counterfeiting unit 305 includes two sub-units: multi-modal human face detection 3051 and multi-modal face verification 3052. In the multi-modal human face detection 3051, a suitable multi-modal face biometric classifier is designed by using the coarse-to-fine two-step strategy mentioned above; the multi-modal face authentication unit 3052, from more The morphological face image extracts information that can determine the target identity for face authentication. The multi-modal human face detection unit 3051 and the multi-modal face authentication unit 3052 jointly constitute an implementation unit 305 of the multi-modal dual verification face anti-counterfeiting algorithm of the present invention.

The control unit 306 is configured to control the working state of each unit, the information communication between the units, and the like. The display unit 307 is configured to display the intermediate result on the output medium, which is convenient for the user to query.

The control unit 306 is configured to implement the operating state of the multimodal generation source 302 and the control of the multimodal data acquisition unit 303. It can be controlled by a single-chip microcomputer or connected by a PC.

Referring to FIG. 3 and FIG. 4, the control unit 306 is controlled by: after receiving the face presence signal sent by the sensing unit 301, first giving a control signal, turning on the light source of the spectrum 1, and then waiting for a certain time to give the camera exposure. The image signal corresponding to the camera of Spectrum 1 is then acquired and then the source of Spectrum 1 is turned off. The signal is then given, the source of spectrum 2 is turned on, a certain exposure time is awaited, the image signal corresponding to the camera of spectrum 2 is accommodated and acquired, then the source of spectrum 2 is turned off, and so on, until the image data of all spectra is acquired. If you don’t use a camera in a certain spectrum, Instead, other receiving devices, such as receiving tubes, photodiodes, etc., are used, and the corresponding received intensity values are read. Then control the thermal infrared camera for image acquisition. After the thermal infrared, the 3D camera is controlled to perform 3D face image collection. The ultrasonic waves are then controlled to emit ultrasonic waves and imaged using an ultrasonic imaging device.

An example is: The control unit 306 is composed of PC software of the host computer. After receiving the signal sent by the sensing unit 301, the control unit 306 first gives an open command of the light source 1, waits for 50 ms and then gives an acquisition command of the camera (or receiving tube) corresponding to the light source 1, which is collected by the camera (or receiving tube). data. Then let the light source 1 go out, give the light source 2 open command, wait for 50ms, and let the light source 2 camera (or receiving tube) perform data acquisition. And so on, until the cameras of all the light sources collect data. Then, when collecting thermal infrared and 3D images, you can directly collect them without waiting. The ultrasonic transmitter is then turned on and the echo is received and imaged by the ultrasound imaging device. Then, the control unit 306 sends the image data collected by each camera to the multi-modal face detecting unit 304.

The display unit 307 is configured to display the face image collected by the multi-modal data acquisition unit 303, and give various intermediate results or feedback information to facilitate human-computer interaction.

It is worth noting that if one of the above modalities does not have a corresponding image data collection device, it can be replaced with other non-image sensing instruments.

Figure 9 shows a schematic diagram of a multimodal generation source and a multimodal data acquisition unit by way of example. Among them, the multi-modal image acquisition device panel 804 functions as a device frame. The panel is divided into upper and lower parts, the upper part is a multi-modal generation source 901 and a multi-modal data acquisition unit 902, and the lower part is a display unit 905, which is composed of an LCD screen with a visible light camera between the two parts. 903, used as a sensing unit. In the upper half, the three multimode sources are an 800 nm multispectral light source, a 3D structured light source, and an ultrasonic source. The three sources are arranged in a crosswise arrangement and form a rectangle, which ensures that each source can form a uniform hook distribution within a certain range in front of the device. In the center of the source, there are four imaging devices (or receiving devices), including multi-spectral imaging devices (the filters in the corresponding bands are covered in front of the camera to prevent interference from visible light or other spectral light), and thermal infrared cameras (for collecting heat) Infrared image), 3D and ultrasound imaging equipment. When testing, the face should face the collection device face to face. The control unit is not included in the panel of the multispectral collector, but is a separate part (either a microcontroller or a PC software) connected to the multispectral acquisition panel via a control signal line.

The multi-modal face detection unit and the multi-modal double-verification face anti-counterfeiting unit are applications of the upper computer, and after receiving the collected multi-modal face images, respectively, are sent to the above two units, and given Corresponding results.

The workflow of the above multi-modal dual verification face anti-counterfeiting device is shown in FIG. 4 . Referring to FIG. 4, the presence of the face is first sensed by the sensing unit 401; if there is no face, the loop detection is continued, and in fact, the sensing unit 401 cannot determine that the detected face is, but only detects that an object exists. At the time, it is considered that the face is sensed; if there is a face, a command is issued to the control unit 402, and the control unit 402 issues a control command to guide the multi-modal generation source 403 to turn on, off, and multi-modal data acquisition. Unit 404 collects data; then enters multimodal face detection unit 405 for face detection, If no face is detected in any of the images, the display unit 407 is signaled. Outputting the information of the detection failure, and returning to the sensing unit 401 to perform image acquisition again; if all the modal images detect the human face, enter the multi-modal double verification face anti-counterfeiting unit 406, and send a signal to the display unit 407 for output. The face detection information or the displayed face image is captured; after entering the multi-modal double verification face security unit 406, the face living body detection determination 4061 and the face identity verification 4062 are performed, and if it is a fake face, the display unit 407 is passed. The information about the failure of the corresponding living body detection is given, and the sensing unit 401 is returned to perform a new round of image acquisition; if it is a real human face, it is also given by the display unit 407, and then waits for a period of time, and returns to the sensing unit 401 to start a new round. Face detection.

In one example, the multi-modal face detection unit 405 is an application of the PC of the host computer for invoking a corresponding face detection classifier for face detection for each image acquired by the multi-modal data acquisition device 403. If a face is detected in all the images, a certain face image is sent to the display unit 407 for display (for example, a face image under visible light is selected), and the detected face images of all the spectra are input to the multi-modality. Double verification face security unit 406. If not all of the faces are detected, the result of the delivery detection failure is displayed to the display unit 407, and returned to the sensing unit 401 to restart the image sensing.

Finally, the present invention has to be pointed out that with the dual verification face anti-counterfeiting method and device thereof provided by the present invention, the user can adapt to different biological modes according to his own needs, such as a face, an iris, and the like. And the modal combination can be freely selected according to the actual situation. For example, different spectral combinations can be selected separately, or combined with thermal infrared light, 3D image or ultrasonic imaging.

The above described specific embodiments of the present invention are described in detail, and are not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and scope of the present invention are intended to be included within the scope of the present invention.

Claims

Claim

A dual verification face anti-counterfeiting method, characterized in that the method comprises:

Step 1, performing a living body detection on the collected target face to determine whether the target face has biological activity, and if the target face is determined to have a living characteristic, then proceeds to step 2;

2. The dual verification face anti-counterfeiting method according to claim 1, wherein if the double-verified face anti-counterfeiting method is based on visible light, step 1 further comprises:

Step 101: Perform living body detection on the target face, first collect a large number of real and false face samples, extract various texture features on the target face, and train the living body detection texture classifier, if the target face is identified by the living body detection texture classifier as If you have a real face, go to step 2, otherwise it will be considered a false face;

Only the target face passes through steps 101 and 102 at the same time, and is considered to pass the living body detection of step 1.

The method of claim 2, wherein the step 2 further comprises:

Step 203: If it is in the face recognition application, if the target face image and the face image corresponding to the recognition result are determined by the classifier in step 202 to belong to the same person, the target face is considered to be true and valid, otherwise it is false. human face;

If it is in the face verification application, the face image corresponding to the claimed designated person identity is determined by the classifier in step 202 to belong to the same person, and the target face is considered to be true and effective, otherwise False face.

4. The dual verification face anti-counterfeiting method according to claim 1, wherein if the double test The witness face anti-counterfeiting method is based on multi-modality, and step 1 further includes - step 101, roughly determining the biological activity of the target human face, wherein the judgment is performed according to one or more of the following manners: determining the target human face by thermal infrared The temperature is judged to be close to 37 degrees; the depth information of the face is judged by the 3D image to determine whether the face is a 3D object; the ultrasonic reflectance of the target face is analyzed by ultrasonic reflection to determine whether the ultrasonic reflectance of the skin is similar to the real face Through multi-spectral imaging to analyze the reflectivity of the target face in different spectra, to determine whether the multi-spectral reflectance of the skin is similar to the real face, if the target information of the target face is determined by one or more of the above methods and the real person If the faces are similar, proceed to step 102;

5. The dual verification face anti-counterfeiting method according to claim 4, wherein the mutual commerce image algorithm comprises the following steps:

Step 1021: collecting multi-spectral imaging of a large number of real human faces and false human faces at different distances to form a training data set, and performing pixel-level division on images of any two different spectra of the same person to form a mutual business image group. Assuming that two spectra i^ are arbitrarily selected, the images of the same human face in the two spectra are sum, and the mutual quotient image is defined as follows -

Where p is the reflectivity of the face, r is the intensity of the light source at the surface of the face, z is the distance from the face to the light source, and (x, y) is the coordinates on the face image;

The method of claim 4, wherein the step 2 further comprises:

Step 201: Collect a plurality of multi-modal images of real faces, and extract texture features for each image. Step 202: subtract the feature vectors of the images by two or two, according to whether the two images belong to the same person, and subtract the features. The vector is divided into two classes, class and class. The machine learning algorithm is used to train a two-class classifier. The trained classifier can determine whether the two feature vectors of the input belong to the same person.

7. The dual verification face anti-counterfeiting method according to claim 4, wherein each different imaging type is referred to as a modality, and the imaging types include visible light imaging, near infrared imaging, near ultraviolet imaging, and thermal infrared imaging. Or ultrasound imaging.

8. A dual verification face anti-counterfeiting device, the device comprising: a sensing unit, configured to detect the presence of a human face by means of real-time monitoring by using one or more of a near-infrared, ultrasonic, radio frequency or visible light camera ;

The multi-modal double-verification face anti-counterfeiting unit is configured to verify whether the target face is a real and effective face; the display unit is configured to display the face anti-counterfeiting result,

9. The dual-verification face anti-counterfeiting device according to claim 8, wherein when the multi-modal human face living body detecting unit performs a living body detection on the target face, firstly, the biological activity of the target face is roughly determined. It is judged according to one or more of the following ways: determining the temperature of the target face by thermal infrared to determine whether it is close to 37 degrees; determining the depth information of the face by the 3D image, determining whether the face is a 3D object; Reflective analysis of the ultrasonic reflectivity of the target face, to determine whether the ultrasonic reflectance of the skin is similar to the real face; to analyze the reflectivity of the target face in different spectra by multi-spectral imaging, to determine whether the multi-spectral reflectance of the skin is related to the real person If the face is similar, if the information indicator of the target face is similar to the real face by one or more of the above methods, then the biological activity of the target face is continuously determined accurately, and the acquired multi-spectral face image is utilized, and the mutual business image is utilized. The algorithm performs accurate living judgments.

10. The dual verification face anti-counterfeiting device according to claim 9, wherein the mutual business image algorithm comprises the following steps:

Multi-spectral imaging of a large number of real human faces and false faces collected at different distances constitutes a training data set, and pixels of any two different spectra of the same person are divided by pixels to form a mutual business image group, assuming any selection Two spectra 4 are defined, and the image of the same face in the two spectra is sum, and the mutual image is defined as follows: ρ _λ2 (χ, γ) κ _λ2 (∑)

For all mutual business images, divided into multiple overlapping or non-overlapping small blocks on multiple scales, The feature vectors of each small block are taken, and the feature vectors of all the small blocks are combined as a global feature. Based on the statistical learning method, the classifier is trained on the training data set to distinguish the real and false faces.

11. The dual-verification face anti-counterfeiting device according to claim 9, wherein when the multi-modal face verification unit performs identity verification on the target face, first acquiring a plurality of multi-modal images of the real face, for each The image is extracted from the texture features. Secondly, the feature vectors of the image are subtracted two by two. According to whether the two images belong to the same person, the subtracted feature vectors are divided into two categories: intraclass and interclass. The machine learning algorithm is used to train one. Two types of classifiers, the trained classifier can determine whether the two input feature vectors belong to the same person; if it is in the face recognition application, if the target face image and the face image corresponding to the recognition result are the above two types If the classifier is deemed to belong to the same person, the target face is considered to be true and effective, otherwise it is a false face; if it is in the face verification application, if the target face image corresponds to the face image of the claimed designated person, If the above two types of classifiers are deemed to belong to the same person, the target face is considered to be true and effective, otherwise it is a false face.

12. The dual verification face anti-counterfeiting device according to any one of claims 9-11, characterized in that: each different imaging type is called a modality, and the imaging types include visible light imaging, near infrared imaging, near ultraviolet Imaging, thermal infrared imaging or ultrasound imaging.