CN114863515A - Human face living body detection method and device based on micro-expression semantics - Google Patents

Human face living body detection method and device based on micro-expression semantics Download PDF

Info

Publication number
CN114863515A
CN114863515A CN202210406508.6A CN202210406508A CN114863515A CN 114863515 A CN114863515 A CN 114863515A CN 202210406508 A CN202210406508 A CN 202210406508A CN 114863515 A CN114863515 A CN 114863515A
Authority
CN
China
Prior art keywords
micro
face
expression
key
human face
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Pending
Application number
CN202210406508.6A
Other languages
Chinese (zh)
Inventor
高志斌
韦舒婷
张盛
黄联芬
李王明卉
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Xiamen University
Original Assignee
Xiamen University
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Xiamen University filed Critical Xiamen University
Priority to CN202210406508.6A priority Critical patent/CN114863515A/en
Publication of CN114863515A publication Critical patent/CN114863515A/en
Pending legal-status Critical Current

Links

Images

Classifications

    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F18/00Pattern recognition
    • G06F18/20Analysing
    • G06F18/21Design or setup of recognition systems or techniques; Extraction of features in feature space; Blind source separation
    • G06F18/214Generating training patterns; Bootstrap methods, e.g. bagging or boosting
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06NCOMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00Computing arrangements based on biological models
    • G06N3/02Neural networks
    • G06N3/04Architecture, e.g. interconnection topology
    • G06N3/045Combinations of networks
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06NCOMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00Computing arrangements based on biological models
    • G06N3/02Neural networks
    • G06N3/08Learning methods

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • Data Mining & Analysis (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Artificial Intelligence (AREA)
  • General Physics & Mathematics (AREA)
  • General Engineering & Computer Science (AREA)
  • Evolutionary Computation (AREA)
  • Biophysics (AREA)
  • Computational Linguistics (AREA)
  • Software Systems (AREA)
  • Mathematical Physics (AREA)
  • Health & Medical Sciences (AREA)
  • Biomedical Technology (AREA)
  • Computing Systems (AREA)
  • Molecular Biology (AREA)
  • General Health & Medical Sciences (AREA)
  • Evolutionary Biology (AREA)
  • Bioinformatics & Cheminformatics (AREA)
  • Bioinformatics & Computational Biology (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Image Analysis (AREA)

Abstract

The invention discloses a human face in-vivo detection method and a human face in-vivo detection device based on micro-expression semantics, wherein the method comprises the following steps: judging whether the human face has micro expression changes or not, if so, detecting micro expression semantics of different regions of the human face, judging whether the micro expression semantics of the different regions of the human face are consistent or not, and if so, judging that the target human face is a real human face; therefore, three mainstream in-vivo detections including printing attack, playback attack and 3D mask attack can be completed by detecting micro expression changes and human face micro expression semantics, so that the diversity of in-vivo detection methods is expanded, and the in-vivo detection accuracy is improved.

Description

Human face living body detection method and device based on micro-expression semantics
Technical Field
The invention relates to the technical field of in-vivo detection, in particular to a micro-expression-semantic-based human face in-vivo detection method and a micro-expression-semantic-based human face in-vivo detection device.
Background
In the related art, the face recognition technology is an important choice in security application, and is widely applied to scenes such as smart phone payment, border control, automatic teller machines and the like. In order to prevent illegal persons from performing identity authentication by using biological features that counterfeit faces and are stolen, a face recognition system needs to have a live body detection function, that is, to determine the authenticity of physiological features of an object.
The face recognition system can effectively resist common attack means such as photos, replayed videos and the like by verifying whether a user operates the face recognition system by a real living body. At present, a plurality of realization methods for human face living body detection exist, for example, a traditional method adopts manual characteristics to distinguish a real human face from a forged human face, a plurality of interactive living body detection methods judge whether the human face is a living body by sending an instruction to enable a user to cooperate, and along with the development of deep learning, a method for carrying out human face anti-spoofing by using a convolutional neural network also provides higher accuracy for an anti-human face spoofing method.
However, with the progress of the counterfeiting technology, many illegal persons forge faces by using 3D masks, and the faces are spoofed by wearing the masks to serve as identities of other persons, so that the face recognition system is deceived, and a great risk is brought to the safety of the face recognition system. Because the mask can forge face texture and depth features, the attack mode of wearing the 3D mask cannot be resisted by applying some traditional detection methods. For example, a pair of glasses manufactured by an artificial intelligence enterprise reiai wisdom real ai established by the artificial intelligence institute of qinghua university breaks a face recognition unlocking system of a 19-model android mobile phone by resisting sample attack. Therefore, the research on the attack of the 3D mask by using the micro expression of the face has practical application significance.
Disclosure of Invention
The present invention is directed to solving, at least to some extent, one of the technical problems in the art described above. Therefore, one object of the present invention is to provide a human face in-vivo detection method based on micro-expression semantics, which does not need to interact with an object to be detected, so that human face detection is more convenient, and meanwhile, the detection accuracy of the in-vivo detection on the mainstream attack mode is improved, and the application scenarios and range are widened.
The second purpose of the invention is to provide a human face living body detection device based on micro-expression semantics.
In order to achieve the above object, an embodiment of a first aspect of the present invention provides a human face living body detection method based on micro-expression semantics, including the following steps: acquiring a plurality of key face micro-expression images of a face to be detected; extracting the characteristics of each key face micro expression image in the plurality of key face micro expression images to obtain a plurality of corresponding face areas; acquiring optical flow values of a plurality of face areas corresponding to each key face micro-expression image, carrying out normalization processing on the optical flow values of the plurality of face areas to obtain normalized optical flow values, and acquiring optical flow value strain of the plurality of face areas corresponding to adjacent key face micro-expression images according to the normalized optical flow values; judging the micro-expression change of the face to be detected according to the optical flow value strain; if the micro expression change exists in the face to be detected, inputting a plurality of face areas corresponding to each key face micro expression image into a trained micro expression semantic recognition model to obtain the micro expression semantics of each face area; and according to the micro-expression semantics, carrying out consistency judgment on the micro-expression semantics of different face areas of the face to be detected so as to carry out living body detection on the face to be detected.
According to the human face living body detection method based on the micro expression semantics, firstly, a plurality of key human face micro expression images of a human face to be detected are obtained; then, extracting the characteristics of each key face micro expression image in the plurality of key face micro expression images to obtain a plurality of corresponding face areas; then acquiring optical flow values of a plurality of face areas corresponding to each key face micro-expression image, carrying out normalization processing on the optical flow values of the plurality of face areas to obtain normalized optical flow values, and acquiring optical flow value strain of the plurality of face areas corresponding to adjacent key face micro-expression images according to the normalized optical flow values; then, judging the micro-expression change of the face to be detected according to the optical flow value strain; if the micro expression change exists in the face to be detected, inputting a plurality of face areas corresponding to each key face micro expression image into a trained micro expression semantic recognition model to obtain the micro expression semantics of each face area; finally, according to the micro expression semantics, carrying out consistency judgment on the micro expression semantics of different face areas of the face to be detected so as to carry out in-vivo detection on the face to be detected; therefore, the human face detection method and the human face detection device do not need to interact with an object to be detected, so that the human face detection is more convenient, meanwhile, the detection accuracy of the living body detection on the mainstream attack mode is improved, and the application scenes and the application range are widened.
In addition, the human face living body detection method based on micro-expression semantics proposed by the above embodiment of the present invention may further have the following additional technical features:
optionally, the obtaining of multiple key face micro-expression images of the face to be detected includes: acquiring a plurality of continuous frames of human face micro-expression images of the human face to be detected; and extracting a plurality of key human face micro expression images from the plurality of continuous frame human face micro expression images.
Optionally, the determining the micro-representation change of the face to be detected according to the optical flow value strain includes: comparing the optical flow value strain with a preset optical flow strain threshold; presetting a proportion judgment threshold; if the optical flow value strain is larger than the proportion of the number of the micro expression images of the human face, which is equal to or larger than the optical flow strain threshold value, in the total number of the key human face images is larger than a proportion judgment threshold, judging that the human face to be detected has micro expression; and if the proportion of the number of the micro expression images of the face, which are greater than or equal to the optical flow value strain threshold value, in the total number of the key face images is smaller than a proportion judgment threshold, judging that the micro expression does not exist in the face to be detected.
Optionally, the consistency judgment of the micro-expression semantics of different face regions of the face to be detected according to the micro-expression semantics includes: calculating the sum of micro-expression average semantic distances of a plurality of face areas corresponding to each key face micro-expression image according to the micro-expression semantics; comparing the sum of the micro-expression average semantic distances with a preset average semantic correlation threshold;
if the proportion of the number of the micro expression images with the micro expression average semantic distance more than or equal to the average semantic correlation threshold in the total number of the key micro expression images of the face is more than or equal to a proportion judgment threshold, judging that the detection result of the face to be detected is a non-living body; and if the proportion of the number of the micro expression images with the micro expression average semantic distance more than or equal to the average semantic correlation threshold to the number of the key micro expression images is less than a proportion judgment threshold, judging that the detection result of the face to be detected is a living body.
Optionally, the sum of the micro-expression average semantic distances of a plurality of face regions corresponding to each key face micro-expression image is calculated according to the following formula:
Figure BDA0003602081580000031
wherein N represents that each key human face micro-expression image corresponds to N human face areas; i, j represent different ones of the N face regions, and d (i, j) represents a semantic distance between different ones of the face regions.
In order to achieve the above object, a human face living body detection device based on micro-expression semantics provided by an embodiment of a second aspect of the present invention includes an obtaining module, where the obtaining module is configured to obtain a plurality of key human face micro-expression images of a human face to be detected; the face feature extraction module is used for extracting features of each key face micro expression image in the key face micro expression images to obtain a plurality of corresponding face regions; the optical flow analysis module is used for acquiring optical flow values of a plurality of face areas corresponding to each key face micro expression image, normalizing the optical flow values of the face areas to obtain normalized optical flow values, and acquiring optical flow value strain of the face areas corresponding to adjacent key face micro expression images according to the normalized optical flow values; the first judgment module is used for judging the micro-expression change of the face to be detected according to the optical flow value strain; a micro expression semantic recognition module, configured to, if a micro expression change exists in the face to be detected, input the multiple face regions corresponding to each key face micro expression image into a trained micro expression semantic recognition model to obtain a micro expression semantic of each face region; and the second judgment module is used for carrying out consistency judgment on the micro expression semantics of different face areas of the face to be detected according to the micro expression semantics so as to carry out in-vivo detection on the face to be detected.
According to the human face living body detection device based on micro-expression semantics, an acquisition module is arranged to acquire a plurality of key human face micro-expression images of a human face to be detected, and a human face feature extraction module performs feature extraction on each key human face micro-expression image in the plurality of key human face micro-expression images to obtain a plurality of corresponding human face regions; the optical flow analysis module acquires optical flow values of a plurality of face areas corresponding to each key face micro-expression image, normalizes the optical flow values of the face areas to obtain normalized optical flow values, and acquires optical flow value strain of the face areas corresponding to adjacent key face micro-expression images according to the normalized optical flow values; the first judging module judges the micro-expression change of the face to be detected according to the optical flow value strain; when the micro expression change exists in the face to be detected, the micro expression semantic recognition module inputs a plurality of face regions corresponding to each key face micro expression image into a trained micro expression semantic recognition model to obtain the micro expression semantic of each face region; the second judgment module is used for carrying out consistency judgment on micro expression semantics of different face areas of the face to be detected according to the micro expression semantics so as to carry out in-vivo detection on the face to be detected; therefore, the human face detection method and the human face detection device do not need to interact with an object to be detected, so that the human face detection is more convenient, meanwhile, the detection accuracy of the living body detection on the mainstream attack mode is improved, and the application scenes and the application range are widened.
In addition, the human face living body detection device based on micro-expression semantics according to the above embodiment of the present invention may further have the following additional technical features:
optionally, the obtaining module is further configured to obtain multiple continuous frames of human face micro-expression images of the human face to be detected; and extracting a plurality of key human face micro-expression images from the plurality of continuous frame human face micro-expression images.
Optionally, the first determining module is further configured to compare the optical flow value strain with a preset optical flow strain threshold; if the optical flow value strain is larger than or equal to the optical flow strain threshold value, the proportion of the number of the micro expression images of the face to the total number of the key images is larger than or equal to a preset proportion judgment threshold, judging that the micro expression exists in the face to be detected; and if the proportion of the number of the micro expression images of the face, of which the optical flow value strain is greater than or equal to the optical flow strain threshold value, in the total number of the key images is smaller than a proportion judgment threshold, judging that the micro expression does not exist in the face to be detected.
Optionally, the second judging module is further configured to calculate, according to the micro-expression semantics, a sum of micro-expression average semantic distances of a plurality of face regions corresponding to each key face micro-expression image; comparing the sum of the micro-expression average semantic distances with a preset average semantic correlation threshold; if the proportion of the number of the micro expression images with the micro expression average semantic distance more than or equal to the average semantic correlation threshold in the total number of the key micro expression images of the face is more than or equal to a proportion judgment threshold, judging that the detection result of the face to be detected is a non-living body; and if the proportion of the number of the micro expression images with the micro expression average semantic distance more than or equal to the average semantic correlation threshold to the number of the key micro expression images is less than a proportion judgment threshold, judging that the detection result of the face to be detected is a living body.
Optionally, the sum of the micro-expression average semantic distances of a plurality of face regions corresponding to each key face micro-expression image is calculated according to the following formula:
Figure BDA0003602081580000041
wherein N represents that each key human face micro-expression image corresponds to N human face areas; i, j represent different ones of the N face regions, and d (i, j) represents a semantic distance between different ones of the face regions.
Drawings
FIG. 1 is a schematic flow chart of a human face in-vivo detection method based on micro expression semantics according to an embodiment of the invention;
FIG. 2 is a schematic flow chart of a human face in-vivo detection method based on micro-expression semantics according to an embodiment of the invention;
FIG. 3 is a flow diagram illustrating micro-expression semantic extraction according to one embodiment of the present invention;
fig. 4 is a block diagram of a human face liveness detection device based on micro expression semantics according to an embodiment of the invention.
Detailed Description
Reference will now be made in detail to embodiments of the present invention, examples of which are illustrated in the accompanying drawings, wherein like reference numerals refer to the same or similar elements or elements having the same or similar functions throughout. The embodiments described below with reference to the drawings are illustrative and intended to be illustrative of the invention and are not to be construed as limiting the invention.
It should be noted that the micro-expression of the face refers to an instantaneous flash facial expression, which is a subconscious expression and reveals the real feeling of the human. Because the 3D mask needs to be attached to the face of a human face, the disguising mode can cause the area of the face of the human face to move inflexibly, and thus micro-expression semantics of different face areas are inconsistent. Therefore, the living body detection is carried out based on the consistency of the micro-expression semantics of the face, so that the living body face can be identified, the attack of a 3D mask to a face identification system is effectively resisted, and the safety risk of face identification is reduced.
The invention provides a human face living body detection method based on micro-expression semantics, which comprises the steps of firstly obtaining a plurality of key human face micro-expression images of a human face to be detected; then, extracting the characteristics of each key face micro expression image in the plurality of key face micro expression images to obtain a plurality of corresponding face areas; then acquiring optical flow values of a plurality of face areas corresponding to each key face micro-expression image, carrying out normalization processing on the optical flow values of the plurality of face areas to obtain normalized optical flow values, and acquiring optical flow value strain of the plurality of face areas corresponding to adjacent key face micro-expression images according to the normalized optical flow values; then, judging the micro-expression change of the face to be detected according to the optical flow value strain; if the micro expression change exists in the face to be detected, inputting a plurality of face areas corresponding to each key face micro expression image into a trained micro expression semantic recognition model to obtain the micro expression semantics of each face area; finally, according to the micro expression semantics, carrying out consistency judgment on the micro expression semantics of different face areas of the face to be detected so as to carry out in-vivo detection on the face to be detected; therefore, the human face detection method and the human face detection device do not need to interact with an object to be detected, so that the human face detection is more convenient, meanwhile, the detection accuracy of the living body detection on the mainstream attack mode is improved, and the application scenes and the application range are widened.
In order to better understand the above technical solutions, exemplary embodiments of the present invention will be described in more detail below with reference to the accompanying drawings. While exemplary embodiments of the invention are shown in the drawings, it should be understood that the invention may be embodied in various forms and should not be limited to the embodiments set forth herein. Rather, these embodiments are provided so that this disclosure will be thorough and complete, and will fully convey the scope of the invention to those skilled in the art.
In order to better understand the technical solution, the technical solution will be described in detail with reference to the drawings and the specific embodiments.
Referring to fig. 1-2, the method for detecting a living human face based on micro-expression semantics includes the following steps:
step 101, acquiring a plurality of key face micro-expression images of a face to be detected.
It is noted that, a plurality of continuous frame human face micro expression images of the human face to be detected are obtained at first, and then a plurality of key human face micro expression images are extracted from the continuous frame human face micro expression images.
As an embodiment, an image collector is started to collect continuous frames of human face micro-expression images through a sliding window, and continuous frames of human face micro-expression images can be obtained through camera collection according to a set time sliding window, wherein the time length of the time sliding window can be automatically set through the resolution of a sensor environment sensing collection device, a photosensitive environment, the distance between a detected object and a camera and the like, and can also be determined through the self-operation of a user through a parameter setting module.
As a specific embodiment, assuming that a face to be detected is in an indoor environment, the illumination intensity is 700Lux, and when the distance between a camera and an object to be detected is 80cm, an image collector with a frame rate of 25f/s is used to collect a sliding time window length T of 3s, so as to obtain 75 continuous images; determining N from 75 continuous images collected by an image collector f 20 pieces of key face micro expression images.
And 102, extracting the characteristics of each key face micro expression image in the plurality of key face micro expression images to obtain a plurality of corresponding face areas.
According to one embodiment, the characteristics of 20 key face micro-expression images obtained by the acquisition device are extracted, and N-4 face regions are located in each key face micro-expression image, wherein the 4 face regions comprise eyes, a nose, a mouth and an eyebrow.
Step 103, acquiring optical flow values of a plurality of face areas corresponding to each key face micro-expression image, performing normalization processing on the optical flow values of the plurality of face areas to obtain normalized optical flow values, and acquiring optical flow value strains of the plurality of face areas corresponding to adjacent key face micro-expression images according to the normalized optical flow values.
As a specific embodiment, optical flow values of 4 individual face regions corresponding to 20 key face micro-expression images are acquired, and the optical flow value of the kth region of the ith key face image is set to be α i =γ i,k Normalization of optical flow values of 4 face regions within each frame in 20 images
Figure BDA0003602081580000061
And acquiring the optical flow value strain delta between every two adjacent inter-frame micro expression images according to the normalized optical flow value of 20 pictures i =α ii+1 The number of optical flow values strain was 19.
That is, optical flow calculation is performed on a plurality of face regions corresponding to each key face micro-expression image, and an optical flow value of each region is obtained: the luminance I (x, y, t) at time t at a point (x, y) on the image, the optical flow is solved by the following two-frame motion constraint equation:
Figure BDA0003602081580000062
u, v represent the horizontal and vertical components of the vector in the velocity field, and the optical flow field describes both the motion direction and the motion magnitude, so that for corresponding pixels of two consecutive frames, the dynamic information can be represented by the following formula:
Figure BDA0003602081580000063
and 104, judging the micro-expression change of the face to be detected according to the optical flow value strain.
As one example, the optical flow value strain is compared to a preset optical flow strain threshold; presetting a proportion judgment threshold, and if the proportion of the number of the face micro expression images with the optical flow value strain greater than or equal to the optical flow strain threshold to the total number of the key face micro expression images is greater than or equal to the preset proportion judgment threshold, judging that the micro expression exists in the face to be detected; and if the proportion of the number of the human face micro expression images with the optical flow value strain greater than or equal to the optical flow strain threshold value to the number of the key human face micro expression images is less than a proportion judgment threshold, judging that the micro expression does not exist in the human face to be detected.
As a specific embodiment, the optical flow value strain is compared with a preset optical flow strain threshold; setting a proportion judgment threshold as 20%; if the number of the optical flow value strains which is greater than or equal to the optical flow strain threshold value in the 19 optical flow value strains is greater than or equal to 20% of the total number of the images, namely is greater than or equal to 4, judging that the micro-expression exists in the face to be detected; and if the quantity of the optical flow value strain is more than or equal to the optical flow strain threshold value and is less than 20 percent of the total number of the images, namely less than 4, judging that the micro expression does not exist in the face to be detected.
That is, it is determined that the strain value is greater than the preset optical flow strain threshold value T h1 Whether the number of images is equal to or greater than 20% of the total number of images: if the optical flow strain is greater than or equal to a preset optical flow strain threshold T h1 If the quantity is less than 20 percent of the total quantity, the judging and alarming module gives an alarm and quits the detection in the current round; if the optical flow strain is greater than or equal to a preset optical flow strain threshold T h1 The number of the images is equal to or greater than 20% of the total number of the images, and the next step is performed.
And 105, if the micro expression change exists in the face to be detected, inputting a plurality of face areas corresponding to each key face micro expression image into the trained micro expression semantic recognition model to obtain the micro expression semantics of each face area.
As an embodiment, as shown in fig. 3, a data set of micro-expression video sequences required by neural network training is first acquired through a source camera, where M is 6 micro-expression types including happiness, sadness, surprise, disgust, fear, and anger as sample labels, acquired persons in the data set include three age groups of 18 to 45 years old, 46 to 69 years old, and over 69 years old and two genders to reduce the influence of different factors on the difference of micro-expressions, and the data set acquires real persons and prosthesis images and performs corresponding data calibration, where the ratio of the number of the images in the three age groups is 1: 1: 1, the ratio of the number of the male and female images is 1: 1, the ratio of the number of real person images to the number of prosthesis images is 1: 1; then, the human face is detected through a human face detection algorithm and the data set is cut to obtain a data set D S Data set D S Putting the micro expression semantic recognition convolutional neural network into a designed micro expression semantic recognition convolutional neural network for training to obtain a trained micro expression semantic recognition model; finally, inputting 4 human face areas of 20 key human face micro-expression images acquired by a human face collector into a trained micro-tableThe emotion semantic recognition model extracts micro-expression characteristics of 4 face regions of 20 key face micro-expression images collected by the face collector, and judges micro-expression semantics of the 4 face regions to obtain micro-expression label information corresponding to each face region.
And 106, carrying out consistency judgment on the micro expression semantics of different face areas of the face to be detected according to the micro expression semantics so as to carry out in-vivo detection on the face to be detected.
As an example, calculating the sum of micro-expression average semantic distances of a plurality of face areas corresponding to each key face micro-expression image according to micro-expression semantics; comparing the sum of the average semantic distances of the micro-expression with a preset average semantic correlation threshold; if the proportion of the number of the micro expression images with the micro expression average semantic distance more than or equal to the average semantic correlation threshold in the total number of the key micro expression images of the face is more than or equal to the proportion judgment threshold, judging that the detection result of the face to be detected is a non-living body; and if the proportion of the number of the micro expression images of the face, of which the sum of the average semantic distances of the micro expression is greater than or equal to the average semantic correlation threshold value, to the number of the micro expression images of the key face is smaller than a proportion judgment threshold, judging that the detection result of the face to be detected is a living body.
As a specific embodiment, calculating the sum of micro-expression average semantic distances of a plurality of face areas corresponding to 20 key face micro-expression images according to micro-expression semantics; comparing the sum of the average semantic distances of the 20 micro-expressions with a preset average semantic correlation threshold; setting a proportion judgment threshold as 20%; if the number of the images with the micro-expression average semantic distance sum being more than or equal to the average semantic correlation threshold exceeds 20% of the total number, namely the number of the images with the micro-expression average semantic distance sum exceeding 4 images is more than or equal to the average semantic correlation threshold, judging that the detection result of the face to be detected is a non-living body; and if the number of the images with the micro-expression average semantic distance sum being more than or equal to the average semantic correlation threshold value is not more than 20% of the total number, namely the number of the images with the micro-expression average semantic distance sum being less than 4 images is more than the average semantic correlation threshold value, judging that the detection result of the face to be detected is a living body.
As an embodiment, the sum of the micro-expression average semantic distances of a plurality of face regions corresponding to each key face micro-expression image is calculated according to the following formula:
Figure BDA0003602081580000081
wherein N represents that each key human face micro-expression image corresponds to N human face areas; i, j represent different ones of the N face regions, and d (i, j) represents a semantic distance between different ones of the face regions.
That is, the sum of the average semantic distances of 4 individual face area microexpresses is calculated from the microexpressing label information
Figure BDA0003602081580000082
And judging whether the sum of the average semantic distances of 4 face regions of the face to be detected is greater than a preset average semantic correlation threshold T or not h2 : if the sum of the average semantic distances in the 20 images is more than or equal to a preset average semantic correlation threshold T h2 If the number of the detection objects is more than or equal to 4, the detection object is a non-living body, and the judgment and alarm module gives an alarm and quits the detection in the current round; if the sum of the average semantic distances is more than or equal to a preset average semantic relevance threshold value T h2 If the number of the detection targets is less than 4, the detection target is a living body, and the detection is quitted.
In summary, according to the human face living body detection method based on micro expression semantics in the embodiment of the invention, a plurality of key human face micro expression images of a human face to be detected are obtained; then, extracting the characteristics of each key face micro expression image in the plurality of key face micro expression images to obtain a plurality of corresponding face areas; then acquiring optical flow values of a plurality of face areas corresponding to each key face micro-expression image, carrying out normalization processing on the optical flow values of the plurality of face areas to obtain normalized optical flow values, and acquiring optical flow value strain of the plurality of face areas corresponding to adjacent key face micro-expression images according to the normalized optical flow values; then, judging the micro-expression change of the face to be detected according to the optical flow value strain; if the micro expression change exists in the face to be detected, inputting a plurality of face areas corresponding to each key face micro expression image into a trained micro expression semantic recognition model to obtain the micro expression semantics of each face area; finally, according to the micro expression semantics, carrying out consistency judgment on the micro expression semantics of different face areas of the face to be detected so as to carry out in-vivo detection on the face to be detected; therefore, the human face detection method and the human face detection device do not need to interact with an object to be detected, so that the human face detection is more convenient, meanwhile, the detection accuracy of the living body detection on the mainstream attack mode is improved, and the application scenes and the application range are widened.
In order to implement the foregoing embodiment, an embodiment of the present invention further provides a human face living body detection device based on micro expression semantics, as shown in fig. 4, the human face living body detection device based on micro expression semantics includes: the system comprises an acquisition module 10, a face feature extraction module 20, an optical flow analysis module 30, a first judgment module 40, a micro expression semantic recognition module 50 and a second judgment module 60.
The acquisition module 10 is used for acquiring a plurality of key face micro-expression images of a face to be detected; the face feature extraction module 20 is configured to perform feature extraction on each of the plurality of key face micro expression images to obtain a plurality of corresponding face regions; the optical flow analysis module 30 is configured to acquire optical flow values of a plurality of face regions corresponding to each key face micro-expression image, perform normalization processing on the optical flow values of the plurality of face regions to obtain normalized optical flow values, and acquire optical flow value strains of the plurality of face regions corresponding to adjacent key face micro-expression images according to the normalized optical flow values; the first judging module 40 is used for judging the micro-expression change of the face to be detected according to the optical flow value strain; the micro expression semantic recognition module 50 is configured to, if the micro expression change exists in the face to be detected, input a plurality of face regions corresponding to each key face micro expression image into the trained micro expression semantic recognition model to obtain the micro expression semantics of each face region; the second judging module 60 is configured to perform consistency judgment on the micro-expression semantics of different face regions of the face to be detected according to the micro-expression semantics, so as to perform in-vivo detection on the face to be detected.
In some embodiments, the obtaining module 10 is further configured to obtain multiple continuous frames of human face micro-expression images of a human face to be detected; and extracting a plurality of key human face micro expression images from the human face micro expression images of the continuous frames.
In some embodiments, the first determining module is further configured to compare the optical flow value strain with a preset optical flow strain threshold; if the proportion of the number of the micro expression images of the face with the optical flow value strain greater than or equal to the optical flow strain threshold value to the total number of the key images is greater than or equal to a preset proportion judgment threshold, judging that the micro expression exists in the face to be detected; and if the proportion of the number of the micro expression images of the face with the optical flow value strain greater than or equal to the optical flow strain threshold value to the total number of the key images is smaller than a proportion judgment threshold, judging that the micro expression does not exist in the face to be detected.
In some embodiments, the second determining module is further configured to calculate, according to the micro-expression semantics, a sum of micro-expression average semantic distances of a plurality of face regions corresponding to each key face micro-expression image; comparing the sum of the average semantic distances of the micro-expression with a preset average semantic correlation threshold; if the proportion of the number of the micro expression images with the micro expression average semantic distance more than or equal to the average semantic correlation threshold in the total number of the key micro expression images of the face is more than or equal to the proportion judgment threshold, judging that the detection result of the face to be detected is a non-living body; and if the proportion of the number of the micro expression images of the human face with the sum of the average semantic distances of the micro expression being more than or equal to the average semantic correlation threshold to the number of the micro expression images of the key human face is less than a proportion judgment threshold, judging that the detection result of the human face to be detected is a living body.
In some embodiments, the sum of the micro-expression average semantic distances of a plurality of face regions corresponding to each key face micro-expression image is calculated according to the following formula:
Figure BDA0003602081580000101
wherein N represents that each key human face micro-expression image corresponds to N human face areas; i, j represent different ones of the N face regions, and d (i, j) represents a semantic distance between different ones of the face regions.
The human face living body detection device based on the micro expression semantics further comprises a parameter setting module, and is used for setting a threshold value, setting the time length of a sliding window and setting the numerical values of N and M.
It should be noted that the above description about the human face living body detection method based on micro-expression semantics in fig. 1 is also applicable to the human face living body detection device based on micro-expression semantics, and is not repeated herein.
In summary, according to the human face living body detection device based on micro-expression semantics according to the embodiment of the invention, the acquisition module is configured to acquire a plurality of key human face micro-expression images of a human face to be detected, and the human face feature extraction module performs feature extraction on each key human face micro-expression image in the plurality of key human face micro-expression images to obtain a plurality of corresponding human face regions; the optical flow analysis module acquires optical flow values of a plurality of face areas corresponding to each key face micro expression image, normalizes the optical flow values of the plurality of face areas in each image, and acquires optical flow value strain of the plurality of face areas corresponding to adjacent key face micro expression images according to the normalized optical flow values; the first judging module judges the micro-expression change of the face to be detected according to the optical flow value strain; when the micro expression change exists in the face to be detected, the micro expression semantic recognition module inputs a plurality of face areas corresponding to each key face micro expression image into the trained micro expression semantic recognition model to obtain the micro expression semantics of each face area; the second judgment module is used for carrying out consistency judgment on micro expression semantics of different face areas of the face to be detected according to the micro expression semantics so as to carry out in-vivo detection on the face to be detected; therefore, interaction with an object to be detected is not needed, the face detection is more convenient, meanwhile, the detection accuracy of the living body detection on the mainstream attack mode is improved, and the application scenes and the application range are widened.
As will be appreciated by one skilled in the art, embodiments of the present invention may be provided as a method, system, or computer program product. Accordingly, the present invention may take the form of an entirely hardware embodiment, an entirely software embodiment or an embodiment combining software and hardware aspects. Furthermore, the present invention may take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, and the like) having computer-usable program code embodied therein.
The present invention is described with reference to flowchart illustrations and/or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the invention. It will be understood that each flow and/or block of the flow diagrams and/or block diagrams, and combinations of flows and/or blocks in the flow diagrams and/or block diagrams, can be implemented by computer program instructions. These computer program instructions may be provided to a processor of a general purpose computer, special purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, create means for implementing the functions specified in the flowchart flow or flows and/or block diagram block or blocks.
These computer program instructions may also be stored in a computer-readable memory that can direct a computer or other programmable data processing apparatus to function in a particular manner, such that the instructions stored in the computer-readable memory produce an article of manufacture including instruction means which implement the function specified in the flowchart flow or flows and/or block diagram block or blocks.
These computer program instructions may also be loaded onto a computer or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer implemented process such that the instructions which execute on the computer or other programmable apparatus provide steps for implementing the functions specified in the flowchart flow or flows and/or block diagram block or blocks.
It should be noted that in the claims, any reference signs placed between parentheses shall not be construed as limiting the claim. The word "comprising" does not exclude the presence of elements or steps not listed in a claim. The word "a" or "an" preceding an element does not exclude the presence of a plurality of such elements. The invention may be implemented by means of hardware comprising several distinct elements, and by means of a suitably programmed computer. In the unit claims enumerating several means, several of these means may be embodied by one and the same item of hardware. The usage of the words first, second and third, etcetera do not indicate any ordering. These words may be interpreted as names.
While preferred embodiments of the present invention have been described, additional variations and modifications in those embodiments may occur to those skilled in the art once they learn of the basic inventive concepts. Therefore, it is intended that the appended claims be interpreted as including preferred embodiments and all such alterations and modifications as fall within the scope of the invention.
It will be apparent to those skilled in the art that various changes and modifications may be made in the present invention without departing from the spirit and scope of the invention. Thus, if such modifications and variations of the present invention fall within the scope of the claims of the present invention and their equivalents, the present invention is also intended to include such modifications and variations.
In the description of the present invention, it is to be understood that the terms "first", "second" and the like are used for descriptive purposes only and are not to be construed as indicating or implying relative importance or implying any number of technical features indicated. Thus, a feature defined as "first" or "second" may explicitly or implicitly include one or more of that feature. In the description of the present invention, "a plurality" means two or more unless specifically defined otherwise.
In the present invention, unless otherwise expressly stated or limited, the terms "mounted," "connected," "secured," and the like are to be construed broadly and can, for example, be fixedly connected, detachably connected, or integrally formed; can be mechanically or electrically connected; either directly or indirectly through intervening media, either internally or in any other relationship. The specific meanings of the above terms in the present invention can be understood by those skilled in the art according to specific situations.
In the present invention, unless otherwise expressly stated or limited, the first feature "on" or "under" the second feature may be directly contacting the first and second features or indirectly contacting the first and second features through an intermediate. Also, a first feature "on," "over," and "above" a second feature may be directly or diagonally above the second feature, or may simply indicate that the first feature is at a higher level than the second feature. A first feature being "under," "below," and "beneath" a second feature may be directly under or obliquely under the first feature, or may simply mean that the first feature is at a lesser elevation than the second feature.
In the description herein, references to the description of the term "one embodiment," "some embodiments," "an example," "a specific example," or "some examples," etc., mean that a particular feature, structure, material, or characteristic described in connection with the embodiment or example is included in at least one embodiment or example of the invention. In this specification, the schematic representations of the terms used above should not be understood to necessarily refer to the same embodiment or example. Furthermore, the particular features, structures, materials, or characteristics described may be combined in any suitable manner in any one or more embodiments or examples. Furthermore, various embodiments or examples and features of different embodiments or examples described in this specification can be combined and combined by one skilled in the art without contradiction.
Although embodiments of the present invention have been shown and described above, it is understood that the above embodiments are exemplary and should not be construed as limiting the present invention, and that variations, modifications, substitutions and alterations can be made to the above embodiments by those of ordinary skill in the art within the scope of the present invention.

Claims (10)

1. A human face living body detection method based on micro-expression semantics is characterized by comprising the following steps:
acquiring a plurality of key face micro-expression images of a face to be detected;
extracting the characteristics of each key face micro expression image in the plurality of key face micro expression images to obtain a plurality of corresponding face areas;
acquiring optical flow values of a plurality of face areas corresponding to each key face micro-expression image, carrying out normalization processing on the optical flow values of the plurality of face areas to obtain normalized optical flow values, and acquiring optical flow value strain of the plurality of face areas corresponding to adjacent key face micro-expression images according to the normalized optical flow values;
judging the micro-expression change of the face to be detected according to the optical flow value strain;
if the micro expression change exists in the face to be detected, inputting a plurality of face areas corresponding to each key face micro expression image into a trained micro expression semantic recognition model to obtain the micro expression semantics of each face area;
and carrying out consistency judgment on the micro expression semantics of different face areas of the face to be detected according to the micro expression semantics so as to carry out in-vivo detection on the face to be detected.
2. The micro-expression semantics based human face in-vivo detection method as claimed in claim 1, wherein the obtaining of a plurality of key human face micro-expression images of the human face to be detected comprises:
acquiring a plurality of continuous frames of human face micro-expression images of the human face to be detected;
and extracting a plurality of key human face micro expression images from the plurality of continuous frame human face micro expression images.
3. The method for detecting the living human face based on the micro-expression semantics as claimed in claim 2, wherein the determining the micro-expression changes of the human face to be detected according to the optical flow value strain comprises:
comparing the optical flow value strain with a preset optical flow strain threshold;
presetting a proportion judgment threshold, and if the proportion of the number of the face micro expression images with the optical flow value strain larger than or equal to the optical flow strain threshold to the total number of the key face micro expression images is larger than or equal to the preset proportion judgment threshold, judging that the face to be detected has micro expression;
and if the optical flow value strain is larger than or equal to the optical flow strain threshold value, the proportion of the number of the human face micro expression images in the number of the key human face micro expression images is smaller than a proportion judgment threshold, judging that the human face to be detected has no micro expression.
4. The micro-expression-semantic-based human face in-vivo detection method according to claim 3, wherein the consistency judgment of the micro-expression semantics of different human face regions of the human face to be detected according to the micro-expression semantics comprises:
calculating the sum of micro-expression average semantic distances of a plurality of face areas corresponding to each key face micro-expression image according to the micro-expression semantics;
comparing the sum of the micro-expression average semantic distances with a preset average semantic correlation threshold;
if the proportion of the number of the micro expression images with the micro expression average semantic distance more than or equal to the average semantic correlation threshold in the total number of the key micro expression images of the face is more than or equal to a proportion judgment threshold, judging that the detection result of the face to be detected is a non-living body;
and if the proportion of the number of the micro expression images with the micro expression average semantic distance more than or equal to the average semantic correlation threshold to the number of the key micro expression images is less than a proportion judgment threshold, judging that the detection result of the face to be detected is a living body.
5. The micro-expression-semantic-based human face living body detection method according to claim 4, characterized in that the sum of micro-expression average semantic distances of N human face regions corresponding to each key human face micro-expression image is calculated according to the following formula:
Figure FDA0003602081570000021
wherein N represents that each key human face micro-expression image corresponds to N human face areas; i, j represent different ones of the N face regions, and d (i, j) represents a semantic distance between different ones of the face regions.
6. A human face living body detection device based on micro-expression semantics is characterized by comprising:
the system comprises an acquisition module, a processing module and a display module, wherein the acquisition module is used for acquiring a plurality of key face micro-expression images of a face to be detected;
the face feature extraction module is used for extracting features of each key face micro expression image in the key face micro expression images to obtain a plurality of corresponding face regions;
the optical flow analysis module is used for acquiring optical flow values of a plurality of face areas corresponding to each key face micro expression image, normalizing the optical flow values of the face areas to obtain normalized optical flow values, and acquiring optical flow value strain of the face areas corresponding to adjacent key face micro expression images according to the normalized optical flow values;
the first judgment module is used for judging the micro-expression change of the face to be detected according to the optical flow value strain;
a micro expression semantic recognition module, configured to, if a micro expression change exists in the face to be detected, input the multiple face regions corresponding to each key face micro expression image into a trained micro expression semantic recognition model to obtain a micro expression semantic of each face region;
and the second judgment module is used for carrying out consistency judgment on the micro expression semantics of different face areas of the face to be detected according to the micro expression semantics so as to carry out in-vivo detection on the face to be detected.
7. The micro-expression-semantic-based human face in-vivo detection device according to claim 6, wherein the acquisition module is further configured to acquire a plurality of continuous frames of human face micro-expression images of the human face to be detected; and extracting a plurality of key human face micro expression images from the plurality of continuous frame human face micro expression images.
8. The micro-expression-semantic-based human face in-vivo detection device as claimed in claim 7, wherein the first judgment module is further configured to compare the optical flow value strain with a preset optical flow strain threshold; if the optical flow value strain is larger than or equal to the optical flow strain threshold value, the proportion of the number of the micro expression images of the face to the total number of the key images is larger than or equal to a preset proportion judgment threshold, judging that the micro expression exists in the face to be detected; and if the proportion of the number of the micro expression images of the face, of which the optical flow value strain is greater than or equal to the optical flow strain threshold value, in the total number of the key images is smaller than a proportion judgment threshold, judging that the micro expression does not exist in the face to be detected.
9. The micro-expression-semantics-based living human face detection device of claim 8, wherein the second judgment module is further configured to calculate a sum of micro-expression average semantic distances of a plurality of human face regions corresponding to each key human face micro-expression image according to the micro-expression semantics; comparing the sum of the micro-expression average semantic distances with a preset average semantic correlation threshold; if the proportion of the number of the micro expression images with the micro expression average semantic distance more than or equal to the average semantic correlation threshold in the total number of the key micro expression images of the face is more than or equal to a proportion judgment threshold, judging that the detection result of the face to be detected is a non-living body; and if the proportion of the number of the micro expression images with the micro expression average semantic distance more than or equal to the average semantic correlation threshold to the number of the key micro expression images is less than a proportion judgment threshold, judging that the detection result of the face to be detected is a living body.
10. The micro-expression-semantics-based human face in-vivo detection device as claimed in claim 9, wherein the sum of micro-expression average semantic distances of a plurality of human face regions corresponding to each key human face micro-expression image is calculated according to the following formula:
Figure FDA0003602081570000031
wherein N represents that each key human face micro-expression image corresponds to N human face areas; i, j represent different ones of the N face regions, and d (i, j) represents a semantic distance between different ones of the face regions.
CN202210406508.6A 2022-04-18 2022-04-18 Human face living body detection method and device based on micro-expression semantics Pending CN114863515A (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202210406508.6A CN114863515A (en) 2022-04-18 2022-04-18 Human face living body detection method and device based on micro-expression semantics

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202210406508.6A CN114863515A (en) 2022-04-18 2022-04-18 Human face living body detection method and device based on micro-expression semantics

Publications (1)

Publication Number Publication Date
CN114863515A true CN114863515A (en) 2022-08-05

Family

ID=82632162

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202210406508.6A Pending CN114863515A (en) 2022-04-18 2022-04-18 Human face living body detection method and device based on micro-expression semantics

Country Status (1)

Country Link
CN (1) CN114863515A (en)

Citations (7)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2019127262A1 (en) * 2017-12-28 2019-07-04 深圳前海达闼云端智能科技有限公司 Cloud end-based human face in vivo detection method, electronic device and program product
CN110287801A (en) * 2019-05-29 2019-09-27 中国电子科技集团公司电子科学研究院 A kind of micro- Expression Recognition algorithm
CN111274895A (en) * 2020-01-15 2020-06-12 新疆大学 CNN micro-expression identification method based on cavity convolution
CN111553189A (en) * 2020-03-30 2020-08-18 深圳壹账通智能科技有限公司 Data verification method and device based on video information and storage medium
CN111931544A (en) * 2019-05-13 2020-11-13 中国移动通信集团湖北有限公司 Living body detection method, living body detection device, computing equipment and computer storage medium
WO2021259005A1 (en) * 2020-06-23 2021-12-30 平安科技(深圳)有限公司 Video-based micro-expression recognition method and apparatus, computer device, and storage medium
CN113920571A (en) * 2021-11-06 2022-01-11 北京九州安华信息安全技术有限公司 Micro-expression identification method and device based on multi-motion feature fusion

Patent Citations (7)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2019127262A1 (en) * 2017-12-28 2019-07-04 深圳前海达闼云端智能科技有限公司 Cloud end-based human face in vivo detection method, electronic device and program product
CN111931544A (en) * 2019-05-13 2020-11-13 中国移动通信集团湖北有限公司 Living body detection method, living body detection device, computing equipment and computer storage medium
CN110287801A (en) * 2019-05-29 2019-09-27 中国电子科技集团公司电子科学研究院 A kind of micro- Expression Recognition algorithm
CN111274895A (en) * 2020-01-15 2020-06-12 新疆大学 CNN micro-expression identification method based on cavity convolution
CN111553189A (en) * 2020-03-30 2020-08-18 深圳壹账通智能科技有限公司 Data verification method and device based on video information and storage medium
WO2021259005A1 (en) * 2020-06-23 2021-12-30 平安科技(深圳)有限公司 Video-based micro-expression recognition method and apparatus, computer device, and storage medium
CN113920571A (en) * 2021-11-06 2022-01-11 北京九州安华信息安全技术有限公司 Micro-expression identification method and device based on multi-motion feature fusion

Non-Patent Citations (3)

* Cited by examiner, † Cited by third party
Title
李秋宇;张玉明;杨福猛;詹曙;: "一种视频微表情检测的改进光流算法", 图学学报, no. 03, 15 June 2018 (2018-06-15) *
许刚;赵中原;谈元鹏;: "基于差分定位与光流特征提取的微表情识别", 计算机应用与软件, no. 01, 15 January 2017 (2017-01-15) *
马思源;郑涵;郭文;: "应用深度光学应变特征图的人脸活体检测", 中国图象图形学报, no. 03, 16 March 2020 (2020-03-16) *

Similar Documents

Publication Publication Date Title
CN109583342B (en) Human face living body detection method based on transfer learning
US6661907B2 (en) Face detection in digital images
CN108140123A (en) Face living body detection method, electronic device and computer program product
CN105095829B (en) A kind of face identification method and system
CN104966070B (en) Biopsy method and device based on recognition of face
CN107403142B (en) A kind of detection method of micro- expression
Kashem et al. Face recognition system based on principal component analysis (PCA) with back propagation neural networks (BPNN)
KR101159164B1 (en) Fake video detecting apparatus and method
CN109145817A (en) A kind of face In vivo detection recognition methods
CN103514440A (en) Facial recognition
CN108764058A (en) A kind of dual camera human face in-vivo detection method based on thermal imaging effect
CN109522853A (en) Face datection and searching method towards monitor video
CN112766159A (en) Cross-database micro-expression identification method based on multi-feature fusion
Reese et al. A comparison of face detection algorithms in visible and thermal spectrums
US10915739B2 (en) Face recognition device, face recognition method, and computer readable storage medium
CN111666835A (en) Face living body detection method and device
Xie et al. A novel transient wrinkle detection algorithm and its application for expression synthesis
CN107862298B (en) Winking living body detection method based on infrared camera device
CN110705454A (en) Face recognition method with living body detection function
CN107992845A (en) A kind of face recognition the method for distinguishing and device, computer equipment
CN113627256B (en) False video inspection method and system based on blink synchronization and binocular movement detection
CN115147936A (en) Living body detection method, electronic device, storage medium, and program product
CN113011544B (en) Face biological information identification method, system, terminal and medium based on two-dimensional code
CN104615985B (en) A kind of recognition methods of human face similarity degree
CN114863515A (en) Human face living body detection method and device based on micro-expression semantics

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination