CN112115826A - Face living body detection method and system based on bilateral branch network - Google Patents
Face living body detection method and system based on bilateral branch network Download PDFInfo
- Publication number
- CN112115826A CN112115826A CN202010935746.7A CN202010935746A CN112115826A CN 112115826 A CN112115826 A CN 112115826A CN 202010935746 A CN202010935746 A CN 202010935746A CN 112115826 A CN112115826 A CN 112115826A
- Authority
- CN
- China
- Prior art keywords
- living body
- face
- branch
- branch network
- bilateral
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000001514 detection method Methods 0.000 title claims abstract description 146
- 230000002146 bilateral effect Effects 0.000 title claims abstract description 132
- 238000012549 training Methods 0.000 claims abstract description 74
- 238000012360 testing method Methods 0.000 claims description 43
- 238000001727 in vivo Methods 0.000 claims description 39
- 239000013598 vector Substances 0.000 claims description 27
- 238000007781 pre-processing Methods 0.000 claims description 12
- 238000012512 characterization method Methods 0.000 claims description 11
- 230000003044 adaptive effect Effects 0.000 claims description 9
- 230000001186 cumulative effect Effects 0.000 claims description 7
- 230000004931 aggregating effect Effects 0.000 claims description 5
- 238000000034 method Methods 0.000 description 27
- 238000005070 sampling Methods 0.000 description 11
- 230000008569 process Effects 0.000 description 8
- 238000012952 Resampling Methods 0.000 description 7
- 238000000605 extraction Methods 0.000 description 5
- 238000005516 engineering process Methods 0.000 description 4
- 208000016261 weight loss Diseases 0.000 description 4
- 230000009286 beneficial effect Effects 0.000 description 3
- 230000004580 weight loss Effects 0.000 description 3
- 238000004364 calculation method Methods 0.000 description 2
- 238000013527 convolutional neural network Methods 0.000 description 2
- 238000011161 development Methods 0.000 description 2
- 238000010586 diagram Methods 0.000 description 2
- 230000006870 function Effects 0.000 description 2
- 239000000463 material Substances 0.000 description 2
- 238000012986 modification Methods 0.000 description 2
- 230000004048 modification Effects 0.000 description 2
- 238000011176 pooling Methods 0.000 description 2
- 238000011160 research Methods 0.000 description 2
- 238000009825 accumulation Methods 0.000 description 1
- 230000002776 aggregation Effects 0.000 description 1
- 238000004220 aggregation Methods 0.000 description 1
- 238000004422 calculation algorithm Methods 0.000 description 1
- 238000010276 construction Methods 0.000 description 1
- 230000000694 effects Effects 0.000 description 1
- 230000005484 gravity Effects 0.000 description 1
- 230000006872 improvement Effects 0.000 description 1
- 238000005457 optimization Methods 0.000 description 1
- 238000003909 pattern recognition Methods 0.000 description 1
- 238000012545 processing Methods 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V40/00—Recognition of biometric, human-related or animal-related patterns in image or video data
- G06V40/10—Human or animal bodies, e.g. vehicle occupants or pedestrians; Body parts, e.g. hands
- G06V40/16—Human faces, e.g. facial parts, sketches or expressions
- G06V40/161—Detection; Localisation; Normalisation
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
- G06N3/045—Combinations of networks
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/08—Learning methods
- G06N3/084—Backpropagation, e.g. using gradient descent
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V40/00—Recognition of biometric, human-related or animal-related patterns in image or video data
- G06V40/10—Human or animal bodies, e.g. vehicle occupants or pedestrians; Body parts, e.g. hands
- G06V40/16—Human faces, e.g. facial parts, sketches or expressions
- G06V40/168—Feature extraction; Face representation
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V40/00—Recognition of biometric, human-related or animal-related patterns in image or video data
- G06V40/10—Human or animal bodies, e.g. vehicle occupants or pedestrians; Body parts, e.g. hands
- G06V40/16—Human faces, e.g. facial parts, sketches or expressions
- G06V40/172—Classification, e.g. identification
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V40/00—Recognition of biometric, human-related or animal-related patterns in image or video data
- G06V40/40—Spoof detection, e.g. liveness detection
- G06V40/45—Detection of the body part being alive
Abstract
The embodiment of the invention provides a face living body detection method based on a bilateral branch network, which comprises the steps of obtaining a face sample image, constructing the bilateral branch network based on living body detection, inputting the face sample image into the bilateral branch network based on living body detection, training to obtain a bilateral branch network model based on living body detection, inputting a face image to be detected into the bilateral branch network model based on living body detection, and detecting a living body face image. The embodiment of the invention also provides a face living body detection system based on the bilateral branch network. According to the embodiment of the invention, the bilateral branch network based on the living body detection is constructed, the bilateral branch network model based on the living body detection is trained and optimized, the living body face recognition can be rapidly and accurately realized, the detection accuracy of unbalanced sample data is effectively improved, the practicability is high, and the face recognition efficiency and the safety are effectively improved.
Description
Technical Field
The invention relates to the technical field of computer vision and image recognition, in particular to a human face living body detection method and system based on a Two-sided branch network (TSBN).
Background
With the intensive research and rapid development of computer vision and pattern recognition technologies, biometric identification technologies such as face identification, fingerprint identification, iris identification and the like are applied in different scenes. The face recognition technology has the advantages of convenience in use, non-contact property and the like, and is widely applied to various fields such as finance, security protection, internet and the like. Meanwhile, the human face recognition system is attacked by using photos, videos, masks and the like to disguise the living human face, the challenge is also provided for the safety of the human face recognition system, and the problem that the user generally pays attention to how to improve the safety of the human face recognition system by effectively recognizing the living human face is solved.
At present, the research of the living body face recognition method is based on ideal sample data, that is, the sample data is balance data with the same quantity of each type of sample, and the living body face recognition model is designed and tested by adopting high-quality large-scale data sets such as ImageNet, ILSVR, MS COCO and Places. However, unbalanced data often needs to be faced in a real application scene, and since the class distribution of the data set is seriously uneven, a few classes occupy a large amount of data, and most classes have little data, the sample classes of the large-scale data set are evenly distributed, and cannot accurately reflect the real scene, and the large-scale data set cannot be actually applied to the real scene. How to construct an efficient and accurate human face living body detection model, and accurately judge whether a target to be detected is a living human face in real time, so that the human face detection and identification efficiency and safety are effectively improved, and the human face detection and identification model becomes one of the technical problems to be solved urgently in the development and application processes of the human face identification technology.
Disclosure of Invention
In order to solve at least one of the above technical problems, an embodiment of a first aspect of the present invention provides a face liveness detection method based on a bilateral branch network, including the following steps: s101, acquiring a face sample image, wherein the face sample image comprises a living body face image and a non-living body face image, and preprocessing the acquired face sample image; s102, constructing a bilateral branch network based on living body detection, wherein the bilateral branch network based on living body detection comprises a first branch and a second branch, and the network structures of the first branch and the second branch are the same; s103, inputting the face sample image into the bilateral branch network based on the living body detection, and training to obtain a bilateral branch network model based on the living body detection; s104, inputting a face image to be detected into the bilateral branch network model based on living body detection, judging whether output data is larger than a preset threshold value, and if so, determining that the image to be detected is a living body face image; and if not, determining that the image to be detected is a non-living body face image.
Preferably, the step of preprocessing the acquired face sample image includes: and identifying a living body face image and a non-living body face image in the face sample image to respectively obtain a training sample set and a test sample set.
Preferably, the bilateral branch network based on living body detection further comprises: accumulating the learning strategy; and the cumulative learning strategy aggregates the feature vectors output by the first branch and the second branch.
Preferably, the step of aggregating the feature vectors output by the first branch and the second branch by the cumulative learning strategy is specifically:
aggregating the feature vectors output by the first branch and the second branch according to an adaptive weight parameter a, wherein the adaptive weight parameter a is determined by the following formula:
wherein, TmaxRepresents the total number of training and T represents the current number of training.
Preferably, network parameters of the first branch and the second branch of the liveness detection based bilateral branch network are shared.
Preferably, in step S103, specifically: and inputting the face sample image into the bilateral branch network based on the living body detection, and respectively training the characterization capability and the classification capability of the bilateral branch network based on the living body detection to obtain a bilateral branch network model based on the living body detection.
Preferably, before the step S104, the method further includes: and inputting the test sample set into the bilateral branch network model based on the in-vivo detection, and testing the bilateral branch network model based on the in-vivo detection.
In a second aspect of the present invention, a face liveness detection system based on a bilateral branch network is further provided, including: the system comprises an acquisition module, a preprocessing module and a display module, wherein the acquisition module is used for acquiring a human face sample image, the human face sample image comprises a living body human face image and a non-living body human face image, and the acquired human face sample image is preprocessed; the bilateral branch network based on the living body detection comprises a first branch and a second branch, and the network structures of the first branch and the second branch are the same; the training module is used for inputting the face sample image into the bilateral branch network based on the living body detection and training to obtain a bilateral branch network model based on the living body detection; the detection module is used for inputting a face image to be detected into the bilateral branch network model based on living body detection, judging whether output data is larger than a preset threshold value or not, and if so, determining that the image to be detected is a living body face image; and if not, determining that the image to be detected is a non-living body face image.
Preferably, the obtaining module further includes: and the preprocessing unit is used for identifying the living body face image and the non-living body face image in the face sample image to respectively obtain a training sample set and a test sample set.
Preferably, the bilateral branch network-based face in-vivo detection system further includes: a test module: the bilateral branch network model based on the living body detection is used for inputting the test sample set into the bilateral branch network model based on the living body detection and testing the bilateral branch network model based on the living body detection.
Additional aspects and advantages of the invention will be set forth in part in the description which follows, and in part will be obvious from the description, or may be learned by practice of the invention.
Drawings
The above and/or additional aspects and advantages of the present invention will become apparent and readily appreciated from the following description of the embodiments, taken in conjunction with the accompanying drawings of which:
FIG. 1 is a schematic flow chart of a face liveness detection method based on a bilateral branch network in an embodiment of the present invention;
FIG. 2 is a schematic structural diagram of a bilateral branch network based on liveness detection in an embodiment of the present invention;
FIG. 3 is a schematic structural diagram of a face in-vivo detection system based on a bilateral branch network in an embodiment of the present invention;
FIG. 4 is a graphical representation of error rates for testing a bilateral branch network based on liveness detection using a test sample set derived based on unbalanced data sets CIFAR-100 and CIFAR-10.
Detailed Description
In order that the above objects, features and advantages of the present invention can be more clearly understood, a more particular description of the invention will be rendered by reference to the appended drawings. It should be noted that the embodiments and features of the embodiments of the present application may be combined with each other without conflict.
In the following description, numerous specific details are set forth in order to provide a thorough understanding of the present invention, however, the present invention may be practiced in other ways than those specifically described herein, and therefore the scope of the present invention is not limited by the specific embodiments disclosed below.
Example one
The embodiment of the invention provides a face living body detection method based on a bilateral branch network, which comprises the following steps as shown in figure 1: s101, acquiring a face sample image, wherein the face sample image comprises a living body face image and a non-living body face image, and preprocessing the acquired face sample image; s102, constructing a bilateral branch network based on living body detection, wherein the bilateral branch network based on living body detection comprises a first branch and a second branch, and the network structures of the first branch and the second branch are the same; s103, inputting the face sample image into the bilateral branch network based on the living body detection, and training to obtain a bilateral branch network model based on the living body detection; s104, inputting a face image to be detected into the bilateral branch network model based on living body detection, judging whether output data is larger than a preset threshold value, and if so, determining that the image to be detected is a living body face image; and if not, determining that the image to be detected is a non-living body face image.
In the technical scheme, a preset number of face sample images are collected as a training data set and a testing data set, a bilateral branch network based on living body detection is constructed, the face sample images are input into the constructed bilateral branch network based on the living body detection for characterization capability and classification capability training to obtain a bilateral branch network model based on the living body detection, the face images in the testing data set are input into the bilateral branch network model based on the living body detection for detection accuracy testing and network parameter adjustment, the face images to be detected are input into the bilateral branch network model based on the living body detection, and whether the input face images to be detected are the living body face images or not is judged according to output data of the bilateral branch network model based on the living body detection.
In the above technical solution, the step of preprocessing the acquired face sample image includes: and identifying a living body face image and a non-living body face image in the face sample image to respectively obtain a training sample set and a test sample set.
In the technical scheme, a preset number of living body face images and non-living body face images are collected, wherein the preset number of living body face images and non-living body face images comprise a first preset number of living body face images collected under the conditions of different light rays, different backgrounds and different collection devices, and a second preset number of non-living body face images collected through pictures, videos and the like. And selecting sample images from the collected living body face image and the collected non-living body face image, and removing sample images which do not contain complete face images or do not meet preset conditions, such as image resolution lower than a preset threshold value. The method includes the steps of identifying a living body face image and a non-living body face image in a sample image which meets preset conditions, specifically, setting labels for the living body face image and the non-living body face image respectively, wherein the label corresponding to the living body face image is 1, and the label corresponding to the non-living body face image is 2, so that a face sample image set is obtained. Randomly extracting a third preset number of face sample images from the face sample image set to obtain a training sample set, and randomly extracting a fourth preset number of face sample images to obtain a testing sample set.
Further, the face sample images in the training sample set and/or the testing sample set are converted into tfrecrd format for multithread parallelization input of the bilateral branch network based on living body detection.
In the above technical solution, step S102 specifically includes: the bilateral Branch network based on the living body detection is constructed and comprises a first Branch and a second Branch, wherein the first Branch is a traditional Learning Branch (Conventional Learning Branch), the second Branch is a rebalancing Branch (Re-Balancing Branch), and the traditional Learning Branch and the rebalancing Branch have the same residual error network structure.
In this technical solution, further, there is parameter sharing in the residual error networks of the conventional learning branch and the rebalancing branch, specifically, except for the last residual error module, the network parameter sharing of the conventional learning branch and the rebalancing branch. By sharing the network parameters of the traditional learning branch and the rebalancing branch, on one hand, the characteristics extracted by the traditional learning branch can be beneficial to the learning of the rebalancing branch; on the other hand, the traditional learning branch and the rebalancing branch share the weight, which effectively reduces the computational complexity.
In the above technical solution, the bilateral branch network based on in-vivo detection further includes: accumulating the learning strategy; and the cumulative learning strategy aggregates the feature vectors output by the traditional learning branch and the rebalancing branch.
In the technical scheme, as shown in fig. 2, when a face sample image is input into a bilateral branch network based on living body detection, the face sample image enters into a traditional learning branch and a rebalancing branch, corresponding feature vectors are obtained through a convolutional neural network and a Global Average Pooling (GAP), and an accumulative learning strategy is used for aggregating the feature vectors output by the traditional learning branch and the rebalancing branch. Specifically, after uniform sampling is carried out on a training sample set through a uniform sampler, a traditional learning branch is input for feature extraction, and a feature vector f is obtainedc(ii) a After sampling is carried out on the training sample set through the reverse sampler, the training sample set is input into a rebalance branch to carry out feature extraction, and a feature vector f is obtainedr. Feature vector f of cumulative learning strategy to traditional learning branch outputcAnd feature vectors fr output by the rebalancing branches are aggregated, so that the traditional learning is fusedThe branch and category rebalance the characteristics of the branch and use the fused characteristics for category prediction.
Further, the accumulative learning strategy aggregates the feature vectors output by the traditional learning branch and the rebalancing branch according to an adaptive weight parameter a, and the adaptive weight parameter a is determined by the following formula:
wherein, TmaxRepresents the total number of training and T represents the current number of training. The self-adaptive weight parameter a is not directly changed from 1 to 0, but is dynamically adjusted and gradually reduced along with the increase of the training times, so that a bilateral branch network based on living body detection firstly learns a universal characterization mode from a human face sample image, a traditional learning branch and a rebalancing branch can simultaneously keep a learning state, unbalanced data are gradually concerned in the model training process, and the characteristics learned in the early stage are not damaged.
Step S103, specifically: and inputting the face sample image into the bilateral branch network based on the living body detection, and respectively training the characterization capability and the classification capability of the bilateral branch network based on the living body detection to obtain a bilateral branch network model based on the living body detection.
In the technical scheme, in the characterization capability training stage of the bilateral branch network based on the in-vivo detection, a face sample image is selected in a training sample set through a uniform sampler, a traditional learning branch of the bilateral branch network based on the in-vivo detection is input, a face sample image is selected in the training sample set through a reverse sampler, a traditional learning branch of the bilateral branch network based on the in-vivo detection is input, after two samples are input into respective corresponding branches, high-dimensional face features are extracted through a convolutional neural network, and a feature vector f is obtained after Global Average Pooling (GAP)c∈RDAnd fr∈RD. Specifically, the uniform sampler operates by applying the same probability to each class sample in the training sample set during a training periodAfter sampling once, inputting the traditional learning branch, thereby keeping the characteristic distribution of the original data and being beneficial to learning the living body characteristics based on the bilateral branch network of the living body detection; when the inverse sampler samples in the training sample set, the sampling probability of each class sample is in direct proportion to the reciprocal of the number of the class sample, that is, the greater the number of the class samples in the training sample set, the smaller the sampling probability of the class sample. For example, in a training sample set x, a class label y ∈ {1, 2, …, C } corresponding to a face sample image is obtained, where C is the total number of classes of the face sample image in the training sample set, and the face sample image is obtained by a uniform sampler (x)c，yc) Inputting the data into a traditional learning branch to perform feature extraction to obtain a feature vector fc(ii) a And obtaining a face sample image (x) by an inverse samplerr，yr) Inputting the data to a rebalance branch for feature extraction to obtain a feature vector fr。
Further, the step of constructing the inverse sampler specifically comprises: calculating the sampling probability (P) of the ith class according to the number of samplesi) (ii) a Randomly sampling (w) according to a sampling probability; replacing a sample from the class i, and repeating the inverse sampler process to obtain a small batch of training sample sets, wherein the calculation formula is as follows: wherein the number of samples of the i-th class is NiThe maximum number of samples of all classes is Nmax。
Wherein the number of samples of the i-th class is NiThe maximum number of samples in all classes is Nmax。
Specifically, in the training process of the bilateral branch network based on in-vivo detection, different samplers will affect the accuracy of the output result of the bilateral branch network based on in-vivo detection, and table 1 shows the error rate of performing bilateral branch network training based on in-vivo detection by using different samplers, where the uniform samplers maintain the distribution of the original data in the training data set; the balance sampler samples all class data in a training data set with the same probability so as to construct a small batch of training data which obeys balance distribution; the error rate of the reverse sampler is smaller than that of the uniform sampler and the balanced sampler, and the result shows that the accuracy of the bilateral branch network model input result based on the in vivo detection can be effectively improved by adjusting the unbalanced data by combining the rebalance branch with the reverse sampler.
TABLE 1
Further, in the bilateral branch network training process based on the in-vivo detection, the accumulated learning strategy transfers the learning center of gravity between the traditional learning branch and the rebalancing branch by controlling the characteristic weight and the classification loss L generated by the traditional learning branch and the rebalancing, so that the bilateral branch network based on the in-vivo detection is adjusted to firstly learn the universal characteristic according to the distribution of the face sample image and then gradually pay attention to the unbalanced data in the training sample set. Specifically, the accumulative learning strategy aggregates the feature vectors f output by the traditional learning branch according to the adaptive weight parameter acAnd rebalancing the feature vectors f of the branch outputsrFor example, feature vector f of conventional learning branch outputcMultiplying by a to rebalance the feature vector f of the branch outputrMultiplying by (1-a) to adjust the conventional learning branch output feature vector fcAnd rebalancing branch output feature vectors frWherein the parameter a is automatically generated according to the training times, and the adaptive weight parameter a is determined by the following formula:
wherein, TmaxRepresents the total number of training and T represents the current number of training.
Further, the cumulative learning strategy separately weights the feature vectors afcAnd (1-a) frRespectively input into a classifier Wc∈RD×CAnd Wr∈RD×CThen, the output results are integrated by element-by-element accumulation, and the output result (z) is determined by the following formula:
wherein z ∈ RCIs a predicted value, i.e. [ z ]1，z2，…，zC]T(ii) a For each class i e {1, 2, …, C }, the softmax function computes each class probability
The formula of (1) is:
wherein the output probability distribution is represented as
E (-) is expressed as a cross entropy loss function, and the weighted cross entropy classification loss (L) formula of the bilateral branch network model based on the living body detection is as follows:
in the above technical solution, specifically, unbalanced data sets CIFAR-10 and CIFAR-100 are selected as sample data sets, wherein the number of sample categories is 10 and 100, the sample data sets respectively include 80000 images, 70000 samples are randomly selected as training sample sets, and 10000 samples are selected as testing sample sets. By the unbalance ratio β (β ═ N)max/Nmin) Controlling the degree of data imbalance, wherein NmaxRepresents the maximum number of samples in all classes, NminRepresenting the minimum number of samples in all categories. The original human face sample image size is adjusted to 256 multiplied by 256, the learning rate is set to 0.01, and the batch processing size is 128, momentum of 0.9, weight decay of 1 × 10-4A backbone network of a bilateral branch network based on liveness detection is trained using a random gradient descent (SGD) method (ResNet 34).
Further, the face sample image is input into the bilateral branch network based on the living body detection, and the characterization capability and the classification capability of the bilateral branch network based on the living body detection are trained respectively. Specifically, training is performed using a conventional Cross-Entropy CE (Cross-Entropy) loss in a training sample set; uniformly Sampling one class in the training sample set through class resampling (Re-Sampling), then Sampling from the class through reverse Sampling, and repeating the process to obtain small-batch balance data; the sample image is weighted by weight loss RW (Re-Weighting) using the inverse of the class sample size to calculate the error rate on the test set. Fig. 4 shows that the error rates of the bilateral branch network based on the living body detection are tested by using the test sample sets obtained based on the unbalanced data sets CIFAR-100 and CIFAR-10, the error rates of the three methods are compared in the horizontal direction by using the same classifier, and the error rate of the classifier trained by using the conventional cross entropy CE is always lower than that of the classifier trained by using the weightlessness weighting RW or the class resampling RS, which indicates that the conventional cross entropy CE training can obtain a better classification effect and extract better features. Meanwhile, the error rate of weight loss weighted RW or category resampling RS is high, the recognition capability of deep features is poor, and the characterization capability of the deep features is damaged to a certain extent by the rebalance method. In addition, the error rates of the three methods in the vertical direction are compared by adopting the same feature learning method, the error rate of the classifier trained by the rebalancing method (weight loss weighted RW and class resampling RS) is lower than that of the traditional cross entropy CE method, which shows that the rebalancing method can adjust and update the weight of the network classifier, thereby effectively reducing the error rate of the bilateral branch network based on the living body detection and improving the classification capability of the bilateral branch network based on the living body detection.
In the technical scheme, the training process of the bilateral branch network based on the in-vivo detection comprises representation capability training and classification capability training, and in the stage of the representation capability training, the traditional cross entropy CE method, the class resampling RS method or the loss-weight weighted RW method is used for extracting the image characteristics of the sample and training the representation capability of the bilateral branch network based on the in-vivo detection; in the classification capability training stage, based on the network parameters determined in the characterization capability training stage, a small amount of sample data is used for fine adjustment of the network parameters at a small learning rate, the error rate of the living body model is reduced, the weight and the classification loss of the network classifier are adaptively adjusted and updated, the feature learning and the classifier learning are considered, the recognition rate of unbalanced samples can be comprehensively improved, and the error rate of the bilateral branch network model based on the living body detection on the unbalanced data detection is effectively reduced.
In the above technical solution, before step S104, the method further includes: and inputting the test sample set into the bilateral branch network model based on the in-vivo detection, and testing the bilateral branch network model based on the in-vivo detection.
In the technical scheme, specifically, a test sample set is respectively input into a traditional learning branch and a rebalancing branch to obtain a feature vector f output by the traditional learning branchc' and rebalancing the feature vector f of the branch outputr' setting the adaptive weight parameter a to 0.5 according to the network parameters determined in the network model training phase, and weighting the feature vector afc' and (1-a) fr' separately inputting classifiers W corresponding to the conventional learning branchescClassifiers W corresponding to rebalancing branchesrAnd obtaining two corresponding predicted values, and adding the two predicted values element by element for aggregation to obtain a classification result.
Further, based on unbalanced data sets CIFAR-10 and CIFAR-100 (the unbalanced ratios beta are 10, 50 and 100 respectively) with different unbalanced ratios beta, a comparison test is performed on the bilateral branch network model based on the in-vivo detection, table 2 shows the test error rate of the bilateral branch network model based on the in-vivo detection compared with other network models, the in-vivo detection network model is trained by adopting a traditional cross entropy CE method, the test error rate is high, wherein Mixup is a universal data enhancement algorithm to generate other abundant samples, the test error rate of the network model is reduced to a certain extent, and the precision of the in-vivo detection network model is improved. The CE-DRW and the CE-DRS are used for performing representation capability training of the network model by adopting a traditional cross entropy CE method, and after classification capability training of the network model is performed by combining loss-weight-loss weighting and category resampling methods, the test error rate is effectively reduced, and meanwhile, the test error rate is further reduced by the network model after bilateral branch network training based on in vivo detection.
TABLE 2
According to the face living body detection method based on the bilateral branch network, the bilateral branch network based on living body detection is constructed, sample image feature extraction and classifier learning are carried out through the traditional learning branch and the rebalance branch respectively, and a bilateral branch network model based on living body detection is obtained through training optimization, so that the characterization capability and the classification capability of the bilateral branch network model based on living body detection are effectively improved, the face sample identification accuracy is improved, the calculation complexity is further reduced, and the non-balanced face sample identification efficiency is improved.
Example two
The embodiment of the present invention also provides a face in-vivo detection system 200 based on a bilateral branch network, as shown in fig. 3, including: an obtaining module 201, configured to obtain a face sample image, where the face sample image includes a living body face image and a non-living body face image, and pre-process the obtained face sample image; a constructing module 202, configured to construct a bilateral branch network based on liveness detection, where the bilateral branch network based on liveness detection includes a first branch and a second branch, and network structures of the first branch and the second branch are the same; the training module 203 is configured to input the face sample image into the bilateral branch network based on the living body detection, and train to obtain a bilateral branch network model based on the living body detection; the detection module 204 is configured to input a face image to be detected into the bilateral branch network model based on living body detection, determine whether output data is greater than a preset threshold, and determine that the image to be detected is a living body face image if the output data is greater than the preset threshold; and if not, determining that the image to be detected is a non-living body face image.
In the technical scheme, an acquisition module 201 acquires a preset number of face sample images as a training data set and a test data set, a construction module 202 constructs a bilateral branch network based on in-vivo detection, a training module 203 inputs the face sample images into the constructed bilateral branch network based on in-vivo detection for performing characterization capability and classification capability training to obtain a bilateral branch network model based on in-vivo detection, a detection module 204 inputs the face images to be detected into the bilateral branch network model based on in-vivo detection, and whether the input face images to be detected are in-vivo face images is judged according to output data of the bilateral branch network model based on in-vivo detection.
In the foregoing technical solution, the obtaining module 201 further includes: and the preprocessing unit is used for identifying the living body face image and the non-living body face image in the face sample image to respectively obtain a training sample set and a test sample set.
In this technical solution, the obtaining module 201 collects a preset number of live body face images and non-live body face images, including a first preset number of live body face images collected under different light, different backgrounds and different collecting device conditions, and a second preset number of non-live body face images collected through pictures, videos, and the like. The preprocessing unit selects sample images from the collected living body face images and non-living body face images, and removes sample images which do not contain complete face images or do not meet preset conditions, such as image resolution lower than a preset threshold value. The method includes the steps of identifying a living body face image and a non-living body face image in a sample image which meets preset conditions, specifically, setting labels for the living body face image and the non-living body face image respectively, wherein the label corresponding to the living body face image is 1, and the label corresponding to the non-living body face image is 2, so that a face sample image set is obtained. Randomly extracting a third preset number of face sample images from the face sample image set to obtain a training sample set, and randomly extracting a fourth preset number of face sample images to obtain a testing sample set.
In the above technical solution, the face in-vivo detection system 200 based on the bilateral branch network further includes: the testing module 205 is configured to input the test sample set into the bilateral branch network model based on liveness detection, and test the bilateral branch network model based on liveness detection.
In the technical solution, the testing module 205 inputs the face image in the test data set into a bilateral branch network model based on living body detection to perform detection accuracy testing and network parameter adjustment.
The bilateral branch network-based face in-vivo detection device provided by the embodiment of the invention can execute the bilateral branch network-based face in-vivo detection method provided by the embodiment of the invention, has a functional module corresponding to the bilateral branch network-based face in-vivo detection method, and has the beneficial effects generated by realizing the bilateral branch network-based face in-vivo detection method.
In the present invention, the term "plurality" means two or more unless explicitly defined otherwise. The terms "mounted," "connected," "fixed," and the like are to be construed broadly, and for example, "connected" may be a fixed connection, a removable connection, or an integral connection; "coupled" may be direct or indirect through an intermediary. The specific meanings of the above terms in the present invention can be understood by those skilled in the art according to specific situations.
In the description herein, the description of the terms "one embodiment," "some embodiments," "specific embodiments," etc., means that a particular feature, structure, material, or characteristic described in connection with the embodiment or example is included in at least one embodiment or example of the invention. In this specification, the schematic representations of the terms used above do not necessarily refer to the same embodiment or example. Furthermore, the particular features, structures, materials, or characteristics described may be combined in any suitable manner in any one or more embodiments or examples.
The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention, and various modifications and changes may be made by those skilled in the art. Any modification, equivalent replacement, or improvement made within the spirit and principle of the present invention should be included in the protection scope of the present invention.
Claims (10)
1. A face living body detection method based on a bilateral branch network is characterized by comprising the following steps:
s101, acquiring a face sample image, wherein the face sample image comprises a living body face image and a non-living body face image, and preprocessing the acquired face sample image;
s102, constructing a bilateral branch network based on living body detection, wherein the bilateral branch network based on living body detection comprises a first branch and a second branch, and the network structures of the first branch and the second branch are the same;
s103, inputting the face sample image into the bilateral branch network based on the living body detection, and training to obtain a bilateral branch network model based on the living body detection;
s104, inputting a face image to be detected into the bilateral branch network model based on living body detection, judging whether output data is larger than a preset threshold value, and if so, determining that the image to be detected is a living body face image; and if not, determining that the image to be detected is a non-living body face image.
2. The bilateral branch network-based human face living body detection method according to claim 1, wherein the step of preprocessing the acquired human face sample image comprises:
and identifying a living body face image and a non-living body face image in the face sample image to respectively obtain a training sample set and a test sample set.
3. The bilateral branch network-based human face living body detection method according to claim 2, wherein the bilateral branch network based on living body detection further comprises: accumulating the learning strategy; and the number of the first and second groups,
the cumulative learning strategy aggregates feature vectors output by the first branch and the second branch.
4. The bilateral branch network-based human face living body detection method according to claim 3, wherein the step of aggregating the feature vectors output by the first branch and the second branch by the cumulative learning strategy is specifically as follows:
aggregating the feature vectors output by the first branch and the second branch according to an adaptive weight parameter a, wherein the adaptive weight parameter a is determined by the following formula:
wherein, TmaxRepresents the total number of training and T represents the current number of training.
5. The bilateral branch network-based face live detection method according to any one of claims 1-4, wherein network parameters of the first branch and the second branch of the bilateral branch network based on live detection are shared.
6. The bilateral branch network-based human face living body detection method according to claim 5, wherein the step S103 specifically comprises:
and inputting the face sample image into the bilateral branch network based on the living body detection, and respectively training the characterization capability and the classification capability of the bilateral branch network based on the living body detection to obtain a bilateral branch network model based on the living body detection.
7. The bilateral branch network-based human face living body detection method according to claim 6, further comprising, before the step S104:
and inputting the test sample set into the bilateral branch network model based on the in-vivo detection, and testing the bilateral branch network model based on the in-vivo detection.
8. A face living body detection system based on a bilateral branch network is characterized by comprising:
the system comprises an acquisition module, a preprocessing module and a display module, wherein the acquisition module is used for acquiring a human face sample image, the human face sample image comprises a living body human face image and a non-living body human face image, and the acquired human face sample image is preprocessed;
the bilateral branch network based on the living body detection comprises a first branch and a second branch, and the network structures of the first branch and the second branch are the same;
the training module is used for inputting the face sample image into the bilateral branch network based on the living body detection and training to obtain a bilateral branch network model based on the living body detection;
the detection module is used for inputting a face image to be detected into the bilateral branch network model based on living body detection, judging whether output data is larger than a preset threshold value or not, and if so, determining that the image to be detected is a living body face image; and if not, determining that the image to be detected is a non-living body face image.
9. The bilateral branch network-based human face in-vivo detection system according to claim 8, wherein the obtaining module further comprises:
and the preprocessing unit is used for identifying the living body face image and the non-living body face image in the face sample image to respectively obtain a training sample set and a test sample set.
10. The bilateral branch network-based face in-vivo detection system according to claim 9, further comprising:
and the testing module is used for inputting the testing sample set into the bilateral branch network model based on the living body detection and testing the bilateral branch network model based on the living body detection.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010935746.7A CN112115826A (en) | 2020-09-08 | 2020-09-08 | Face living body detection method and system based on bilateral branch network |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010935746.7A CN112115826A (en) | 2020-09-08 | 2020-09-08 | Face living body detection method and system based on bilateral branch network |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN112115826A true CN112115826A (en) | 2020-12-22 |
Family
ID=73803333
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202010935746.7A Pending CN112115826A (en) | 2020-09-08 | 2020-09-08 | Face living body detection method and system based on bilateral branch network |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN112115826A (en) |
Cited By (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN113077051A (en) * | 2021-04-14 | 2021-07-06 | 广东博智林机器人有限公司 | Network model training method and device, text classification model and network model |
| CN113221662A (en) * | 2021-04-14 | 2021-08-06 | 上海芯翌智能科技有限公司 | Training method and device of face recognition model, storage medium and terminal |
| CN113344479A (en) * | 2021-08-06 | 2021-09-03 | 首都师范大学 | Online classroom-oriented learning participation intelligent assessment method and device |
Citations (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN109255322A (en) * | 2018-09-03 | 2019-01-22 | 北京诚志重科海图科技有限公司 | A kind of human face in-vivo detection method and device |
-
2020
- 2020-09-08 CN CN202010935746.7A patent/CN112115826A/en active Pending
Patent Citations (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN109255322A (en) * | 2018-09-03 | 2019-01-22 | 北京诚志重科海图科技有限公司 | A kind of human face in-vivo detection method and device |
Non-Patent Citations (1)
| Title |
|---|
| BOYAN ZHOU，ET AL.: "BBN: Bilateral-Branch Network with Cumulative Learning for Long-Tailed Visual Recognition", 《ARXIV:1912.02413V4》, 10 March 2020 (2020-03-10) * |
Cited By (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN113077051A (en) * | 2021-04-14 | 2021-07-06 | 广东博智林机器人有限公司 | Network model training method and device, text classification model and network model |
| CN113221662A (en) * | 2021-04-14 | 2021-08-06 | 上海芯翌智能科技有限公司 | Training method and device of face recognition model, storage medium and terminal |
| CN113221662B (en) * | 2021-04-14 | 2022-09-27 | 上海芯翌智能科技有限公司 | Training method and device of face recognition model, storage medium and terminal |
| CN113077051B (en) * | 2021-04-14 | 2024-01-26 | 南京零洞数智科技有限公司 | Network model training method and device, text classification model and network model |
| CN113344479A (en) * | 2021-08-06 | 2021-09-03 | 首都师范大学 | Online classroom-oriented learning participation intelligent assessment method and device |
| CN113344479B (en) * | 2021-08-06 | 2022-01-07 | 首都师范大学 | Online classroom-oriented learning participation intelligent assessment method and device |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN110414377B (en) | Remote sensing image scene classification method based on scale attention network | |
| CN108491874B (en) | Image list classification method based on generation type countermeasure network | |
| CN112380952B (en) | Power equipment infrared image real-time detection and identification method based on artificial intelligence | |
| CN112115826A (en) | Face living body detection method and system based on bilateral branch network | |
| CN107133948B (en) | Image blurring and noise evaluation method based on multitask convolution neural network | |
| CN109858563B (en) | Self-supervision characterization learning method and device based on transformation recognition | |
| CN108921051A (en) | Pedestrian's Attribute Recognition network and technology based on Recognition with Recurrent Neural Network attention model | |
| CN110781829A (en) | Light-weight deep learning intelligent business hall face recognition method | |
| CN110992351B (en) | sMRI image classification method and device based on multi-input convolution neural network | |
| CN105303150B (en) | Realize the method and system of image procossing | |
| CN110930387A (en) | Fabric defect detection method based on depth separable convolutional neural network | |
| CN114841257B (en) | Small sample target detection method based on self-supervision comparison constraint | |
| CN108664986B (en) | Based on lpNorm regularized multi-task learning image classification method and system | |
| WO2023179099A1 (en) | Image detection method and apparatus, and device and readable storage medium | |
| CN110378393A (en) | A kind of mixing printing product acetes chinensis method based on PSO-GSA-SVM | |
| CN111126155B (en) | Pedestrian re-identification method for generating countermeasure network based on semantic constraint | |
| CN114596590A (en) | Single-monkey video motion classification method with long-tail distribution characteristic | |
| CN114663769A (en) | Fruit identification method based on YOLO v5 | |
| CN114399763A (en) | Single-sample and small-sample micro-body ancient biogenetic fossil image identification method and system | |
| CN106682604B (en) | Blurred image detection method based on deep learning | |
| CN111914758A (en) | Face in-vivo detection method and device based on convolutional neural network | |
| CN116681960A (en) | Intelligent mesoscale vortex identification method and system based on K8s | |
| CN110428012A (en) | Brain method for establishing network model, brain image classification method, device and electronic equipment | |
| Osumi et al. | Domain adaptation using a gradient reversal layer with instance weighting | |
| CN114627424A (en) | Gait recognition method and system based on visual angle transformation |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |