WO2022032549A1 - 基于跨模态转化辅助的人脸防伪检测方法、系统及装置 - Google Patents

基于跨模态转化辅助的人脸防伪检测方法、系统及装置 Download PDF

Info

Publication number
WO2022032549A1
WO2022032549A1 PCT/CN2020/108775 CN2020108775W WO2022032549A1 WO 2022032549 A1 WO2022032549 A1 WO 2022032549A1 CN 2020108775 W CN2020108775 W CN 2020108775W WO 2022032549 A1 WO2022032549 A1 WO 2022032549A1
Authority
WO
WIPO (PCT)
Prior art keywords
modal
face
rgb
image
training sample
Prior art date
Application number
PCT/CN2020/108775
Other languages
English (en)
French (fr)
Inventor
万军
李子青
刘阿建
Original Assignee
中国科学院自动化研究所
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 中国科学院自动化研究所 filed Critical 中国科学院自动化研究所
Publication of WO2022032549A1 publication Critical patent/WO2022032549A1/zh

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V40/00—Recognition of biometric, human-related or animal-related patterns in image or video data
    • G06V40/10—Human or animal bodies, e.g. vehicle occupants or pedestrians; Body parts, e.g. hands
    • G06V40/16—Human faces, e.g. facial parts, sketches or expressions
    • G06V40/161—Detection; Localisation; Normalisation
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F18/00—Pattern recognition
    • G06F18/20—Analysing
    • G06F18/21—Design or setup of recognition systems or techniques; Extraction of features in feature space; Blind source separation
    • G06F18/214—Generating training patterns; Bootstrap methods, e.g. bagging or boosting
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/20—Image preprocessing
    • G06V10/32—Normalisation of the pattern dimensions
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V40/00—Recognition of biometric, human-related or animal-related patterns in image or video data
    • G06V40/40—Spoof detection, e.g. liveness detection

Definitions

  • the invention belongs to the field of image processing and pattern recognition, and particularly relates to a face anti-counterfeiting detection method, system and device based on cross-modal transformation assistance.
  • the face recognition system has become an indispensable device in daily life, and is widely deployed in mobile phones or scenes equipped with face recognition systems.
  • these face recognition systems are extremely vulnerable to malicious attacks by criminals, such as displaying the face images of real users in front of the collection device, and attacking samples such as video playback or masks to disguise as legitimate users to steal private data or money. Therefore, deploying the face anti-counterfeiting detection module on the face recognition system not only has practical application significance, but also has been applied in various scenarios, such as face payment, face security check, mobile phone face unlocking and other human-computer interaction scenarios .
  • Face anti-counterfeiting detection algorithms based on RGB cameras are prone to performance fluctuations in the face of changes in shooting scenes, lighting changes, or camera settings. This phenomenon rarely occurs in face anti-counterfeiting systems based on IR cameras. Because the IR camera mainly captures the heat information radiated by the face sample, the heat information can not only effectively determine the type of the sample to be tested, but also is not easily disturbed by changes in the external environment. However, it is difficult for face anti-counterfeiting systems to be equipped with an additional IR camera in most practical application scenarios.
  • the present invention provides a cross- A face anti-counterfeiting detection method assisted by modal transformation, the face anti-counterfeiting detection method includes:
  • step S10 face detection is performed on the acquired RGB modal image with a human face, and the RGB modal image with a human face is cropped through the detected face bounding box, and is scaled to a set size to obtain the desired size.
  • Detect RGB modal images
  • Step S20 based on the RGB modal image to be detected, generate a corresponding IR modal image through the trained face modal conversion model;
  • Step S30 based on the RGB modal image to be detected and the IR modal image, obtain the probability that the RGB modal image to be detected belongs to a real face through a trained face anti-counterfeiting detection model;
  • Step S40 if the probability value is greater than the set threshold, the RGB modal image to be detected is a real face image; otherwise, the RGB modal image to be detected is an attack image.
  • the method for obtaining training samples of the face modality conversion model is:
  • Step B10 obtaining the RGB modal image set with human face and the corresponding IR modal image set
  • Step B20 performing face detection on each image in the RGB modal image set with a human face, and cutting the corresponding RGB modal image with a human face and the IR modal image by the detected face bounding box, And zoom to the set size to obtain a pair of RGB and IR modal training samples.
  • the training method of the face modality conversion model is:
  • the face modality conversion model as the generator G, construct a reverse generator F and discriminators D I and D R , and train any RGB and IR modality in the set based on the RGB and IR modality training samples
  • the sample pair is iteratively trained by the CycleGAN method until the total loss function value is lower than the set threshold, and the trained face mode conversion model is obtained;
  • the total loss function is:
  • L GAN (G e , D Ie ) and L GAN (F e , D Re ) represent the objective loss functions of feature space Ge and Fe
  • L cyc-final (G, F ) represents the cycle constraint consistency loss function
  • ⁇ and ⁇ are preset weight factors
  • c 1, 2
  • 3 are G(r), G(i) and i and F(i), F(r) and
  • the category of r, r and i represent the RGB modal training sample and the corresponding IR modal training sample, respectively
  • G(r) and G(i) represent the RGB modal training sample r and the corresponding IR modal training sample i respectively.
  • the cycle constraint consistency loss function is:
  • ⁇ ⁇ 1 represents the L1 norm
  • P r and P i are the distribution of the RGB modal training sample r and the corresponding IR modal training sample i, respectively, and respectively represent the mathematical expectation of the modal training sample r and the corresponding IR modal training sample i under the given probability distribution
  • F(i) represents the IR modal training sample i corresponding to the RGB modal training sample r after passing through F
  • the samples of , F(G(r)) represents the RGB modal training sample r after passing through G and then the sample after F
  • G(F(i)) represents the IR modal training sample i corresponding to the RGB modal training sample r Samples after F and then after G.
  • the objective loss functions of G and F include a discriminator and the adversarial loss function of the generator G and the discriminator and the adversarial loss function of the inverse generator F;
  • the objective loss functions of the feature spaces Ge and Fe are:
  • r and i represent the RGB modal training samples and the corresponding IR modal training samples, respectively
  • P r is the distribution of the IR modal training sample i
  • P le is the IR modal training sample i corresponding to the RGB modal training sample r
  • the feature distribution in the feature space P Re is the feature distribution of the RGB modal training sample r in the feature space
  • Ge ( i ) and Ge (r) represent the RGB modal training sample r and the corresponding IR modal training sample, respectively.
  • the spatial features of i in G, Fe (r) and Fe (i) represent the spatial features of the RGB modal training sample r and the corresponding IR modal training sample i in F
  • D Ie (G e ( i)) and D Ie (G e (r)) respectively represent the probability that Ge (i) and Ge (r) belong to the P Ie distribution
  • D Re (F e (r)) and D Re (F e (i)) respectively represents the probability that Fe (r) and Fe (i) belong to the P Re distribution
  • the training method of the face anti-counterfeiting detection model is:
  • Step C10 obtaining the RGB and IR modal training sample pair sets and the sample label of each RGB and IR modal training sample pair;
  • Step C20 randomly select any RGB and IR modal training sample pair of the RGB and IR modal training sample pairs, and use the Branch-R and Branch-I branches of the face anti-counterfeiting detection model to extract the RGB modal training samples respectively. Characteristics of samples and IR modality training samples;
  • Step C30 using the Shared-branch branch of the face anti-counterfeiting detection model to perform forward information fusion and feedback on the features of the RGB modal training samples and the IR modal training samples, and calculate the classification loss value in combination with the sample labels;
  • Step C40 if the classification loss value is greater than the set threshold, adjust the parameters of the face anti-counterfeiting detection model, and jump to step C20 until the classification loss value is less than or equal to the set threshold, and obtain a trained person. Face anti-counterfeiting detection model.
  • the calculation method of the classification loss value is:
  • a face anti-counterfeiting detection system based on cross-modal transformation assistance is proposed.
  • the face anti-counterfeiting detection system includes a preprocessing unit. , face mode conversion unit, face anti-counterfeiting detection unit and discrimination unit;
  • the preprocessing unit is configured to perform face detection on the acquired RGB modal image with a human face, and perform cropping of the RGB modal image with a human face through the detected face bounding box, and zoom to the setting. Determine the size to obtain the RGB modal image to be detected;
  • the face modal conversion unit is configured to generate a corresponding IR modal image through the trained face modal conversion model based on the RGB modal image to be detected;
  • the face anti-counterfeiting detection unit is configured to obtain the probability that the to-be-detected RGB modal image belongs to a real face through a trained face anti-counterfeiting detection model based on the to-be-detected RGB modal image and the IR modal image;
  • the discriminating unit is configured to, if the probability value is greater than a set threshold, the RGB modal image to be detected is a real face image; otherwise, the RGB modal image to be detected is an attack image.
  • a storage device in which a plurality of programs are stored, and the programs are adapted to be loaded and executed by a processor to realize the above-mentioned method for anti-counterfeiting detection based on cross-modal transformation assistance.
  • a processing device including a processor and a storage device; the processor is suitable for executing various programs; the storage device is suitable for storing multiple programs; the program is suitable for It is loaded and executed by the processor to realize the above-mentioned face anti-counterfeiting detection method based on cross-modal transformation assistance.
  • the present invention is based on the cross-modal transformation-assisted face anti-counterfeiting detection method, which makes full use of the IR modal image.
  • the IR modal image mainly captures the heat information radiated from the face sample, and the heat information can not only effectively judge the test to be tested
  • the category of the sample is not easily disturbed by the change of the external environment.
  • the invention uses the more discriminative anti-counterfeiting information of the IR mode to assist the learning of the RGB sample, reduces the performance fluctuation of the face anti-counterfeiting detection algorithm caused by the external environment interference, and greatly improves the performance of the anti-counterfeiting detection algorithm.
  • the accuracy and precision of RGB face anti-counterfeiting detection is based on the cross-modal transformation-assisted face anti-counterfeiting detection method, which makes full use of the IR modal image.
  • the IR modal image mainly captures the heat information radiated from the face sample, and the heat information can not only effectively judge
  • the present invention is based on the cross-modal transformation-assisted face anti-counterfeiting detection method, only IR modal images are introduced in the model training, and in the test after the model training is completed, only the RGB modal images are needed to pass the modal
  • the corresponding IR modal information is obtained by transformation, and the system does not need to configure additional IR cameras, which reduces the difficulty and cost of installation.
  • Fig. 1 is the schematic flow chart of the face anti-counterfeiting detection method based on cross-modal conversion assistance of the present invention
  • FIG. 2 is a schematic diagram of a training sample acquisition flow diagram of an embodiment of the cross-modal transformation-assisted face anti-counterfeiting detection method of the present invention
  • FIG. 3 is a schematic diagram of face modal transformation according to an embodiment of the cross-modal transformation-assisted face anti-counterfeiting detection method according to the present invention
  • FIG. 4 is a schematic diagram of modal assistance according to an embodiment of a face anti-counterfeiting detection method based on cross-modal conversion assistance of the present invention
  • FIG. 5 is an example diagram of a face anti-counterfeiting detection process according to an embodiment of the cross-modal transformation-assisted face anti-counterfeiting detection method of the present invention.
  • a face anti-counterfeiting detection method based on cross-modal transformation assistance of the present invention includes:
  • step S10 face detection is performed on the acquired RGB modal image with a human face, and the RGB modal image with a human face is cropped through the detected face bounding box, and is scaled to a set size to obtain the desired size.
  • Detect RGB modal images
  • Step S20 based on the RGB modal image to be detected, generate a corresponding IR modal image through the trained face modal conversion model;
  • Step S30 based on the RGB modal image to be detected and the IR modal image, obtain the probability that the RGB modal image to be detected belongs to a real face through a trained face anti-counterfeiting detection model;
  • Step S40 if the probability value is greater than the set threshold, the RGB modal image to be detected is a real face image; otherwise, the RGB modal image to be detected is an attack image.
  • the face anti-counterfeiting detection method based on cross-modal transformation assistance includes steps S10 to S40, and each step is described in detail as follows:
  • step S10 face detection is performed on the acquired RGB modal image with a human face, and the RGB modal image with a human face is cropped through the detected face bounding box, and is scaled to a set size to obtain the desired size. Detect RGB modal images.
  • FIG. 2 it is a schematic diagram of the training sample acquisition process of an embodiment of the face anti-counterfeiting detection method based on cross-modal conversion assistance of the present invention.
  • the training sample of the face modality conversion model needs to include both RGB modal images and IR.
  • Modal image :
  • Step B10 acquiring an RGB modal image set with a human face and a corresponding IR modal image set.
  • Step B20 performing face detection on each image in the RGB modal image set with a human face, and cutting the corresponding RGB modal image with a human face and the IR modal image by the detected face bounding box, And zoom to the set size to obtain a pair of RGB and IR modal training samples.
  • the images are processed by the preprocessing unit, and the RGB image is input to the preprocessing unit.
  • the preprocessing unit first performs face detection on the image, and if no face is detected, the image is discarded; otherwise , and then perform center cropping on the image, that is, keep the central area of the face and crop it to a fixed size.
  • the image is cropped to a size of 128*128. In other embodiments, the cropping size may be set according to different application scenarios, which is not limited in the present invention.
  • the detection can be performed.
  • the corresponding IR modal image after the preprocessing is completed to form a training sample pair.
  • Step S20 based on the RGB modal image to be detected, generate a corresponding IR modal image through the trained face modal conversion model.
  • the face mode conversion model is as follows:
  • the face modality conversion model as the generator G, construct a reverse generator F and discriminators D I and D R , and train any RGB and IR modality in the set based on the RGB and IR modality training samples
  • the sample pair is iteratively trained by the CycleGAN method until the total loss function value is lower than the set threshold, and the trained face mode conversion model is obtained.
  • the present invention performs adversarial training through the CycleGAN method, and the training process is improved in two aspects: (1) the cycle consistency constraint L cyc (G, F) is extended from the source mode to the target mode in the original pixel space; (2) ) in the subspace of the mapping function to adjust the mapping direction of the mode to ensure that the sample is converted from the original mode to the target mode in the optimal direction during the mode conversion process.
  • the total loss function includes the discriminator and generator adversarial loss function, the feature space discriminator and generator adversarial loss function, and the cycle constraint consistency loss function.
  • r represents a face sample in RGB mode (abbreviated as R), and the distribution is P r .
  • i a sample belonging to the IR mode (abbreviated as I), and the distribution is P i . Therefore, people take the sample pair composed of samples r and i as network input, and under the supervision of two discriminators D I and D R , respectively train two cyclic mapping functions G: R ⁇ I and F: I ⁇ R. Based on the GAN-based adversarial training strategy, the process is shown in formula (1):
  • the generator G fools the discriminator D I as much as possible to make it believe that the transformed sample G(r) belongs to the I mode, and the discriminator D I tries to distinguish G(r) from the real sample i belonging to the I mode.
  • the inverse generator F tries to fool the discriminator DR as much as possible into believing that the transformed sample F(i) belongs to the R mode, and the discriminator DR tries to distinguish F(i) from the real sample r belonging to the R mode.
  • ⁇ ⁇ 1 represents the L1 norm.
  • L MT-j L GAN (G, D I )+L GAN (F, D R )+ ⁇ L cyc (G, F)
  • L GAN (G, D I ) and L GAN (F, D R ) are the loss functions corresponding to the two mapping functions G and F, and the ⁇ control loop is consistent with the constraint L cyc (G, F) in the training process. proportion. All generators and discriminators are trained alternately until the network converges.
  • ⁇ ⁇ 1 represents the L1 norm
  • P r and P i are the distribution of the RGB modal training sample r and the corresponding IR modal training sample i, respectively, and respectively represent the mathematical expectation of the modal training sample r and the corresponding IR modal training sample i under the given probability distribution
  • F(i) represents the IR modal training sample i corresponding to the RGB modal training sample r after the mapping function
  • the sample after F, F(G(r)) represents the RGB modal training sample r after the mapping function G and then the sample after the mapping function F
  • G(F(i)) represents the IR corresponding to the RGB modal training sample r
  • the modal training sample i is the sample after passing through the mapping function F and then through the mapping function G.
  • D I is essentially a two-class discriminator, which classifies G(r) and i samples into category 1 and category 2, respectively.
  • discriminator Rewrite as shown in formula (9):
  • the two mapping functions G and F output samples that are indistinguishable from the target modality as much as possible regardless of the input modalities.
  • the samples generated by the generator are closer to the real samples in terms of global color.
  • the present invention further adjusts the mapping direction in the feature subspace. Because the mapping function G (similarly for F) is an Encoder-Decoder cascaded network structure, the output of sample r is adjusted after the encoder network (denoted as G e ) to be as close as possible to the distribution of the output of sample i . Samples r and i satisfy: Ge(r) ⁇ G( i ). Since the samples r and i are not strictly aligned, the distribution fitting of the two feature spaces Ge(r) and G( i ) is not suitable for L1 or L2 norm. The present invention uses an adversarial subspace learning strategy to align feature distributions.
  • r and i represent the RGB modal training samples and the corresponding IR modal training samples, respectively
  • P r is the distribution of the IR modal training sample i
  • P Ie is the IR modal training sample i corresponding to the RGB modal training sample r
  • the feature distribution in the feature space P Re is the feature distribution of the RGB modal training sample r in the feature space
  • Ge ( i ) and Ge (r) represent the RGB modal training sample r and the corresponding IR modal training sample, respectively.
  • the spatial features of i in the mapping function G, F e (r) and Fe (i) represent the spatial features of the RGB modal training sample r and the corresponding IR modal training sample i in the mapping function F
  • D Ie (G e (i)) and D Ie (G e (r)) represent the probability that Ge (i) and Ge (r) belong to the P Ie distribution
  • D Re (F e (r)) and D Re (F e ( i)) represent the probability that Fe (r) and Fe (i) belong to the P Re distribution , respectively, represents the mathematical expectation under the distribution of Ge (i) ⁇ P Ie
  • mapping direction of the mapping function G to the sample r is adjusted in the subspace to ensure the most suitable direction to convert from the original mode to the target mode.
  • L GAN (G e , D Ie ) and L GAN (F e , D Re ) represent the objective loss functions of feature space Ge and Fe
  • L cyc-final (G, F ) represents the cycle constraint consistency loss function
  • ⁇ and ⁇ are preset weight factors
  • c 1, 2
  • 3 are G(r), G(i) and i and F(i), F(r) and
  • the category of r, r and i represent the RGB modal training sample and the corresponding IR modal training sample, respectively
  • G(r) and G(i) represent the RGB modal training sample r and the corresponding IR modal training sample i respectively.
  • FIG. 3 it is a schematic diagram of face modal transformation based on an embodiment of the cross-modal transformation-assisted face anti-counterfeiting detection method: (1) Input the paired training set samples (r and i) into the modalities Transform the network for training. (2) Where the module contains two reciprocal mapping functions G: R ⁇ I and F: I ⁇ R. This unit mainly uses the mapping function G: R ⁇ I, and the two mutually inverse mapping functions are mainly used to solve the problem of misalignment of samples in different modalities. (3) Extend the cycle consistency constraint L cyc (G, F) from the source modality to the target modality in the original pixel space.
  • sample r satisfies: r ⁇ G(r) ⁇ F(G(r)) ⁇ r
  • sample i satisfies: i ⁇ F(i) ⁇ G(F(i)) ⁇ i.
  • the newly generated samples G(i) are incorporated into the discriminator D I as a separate class.
  • the original two-class discriminator D I with a three-class discriminator (4) Constrain the sample Ge ( i ) and the sample Ge (r) in the subspace Ge of the mapping function G.
  • the distribution of Ge(r) is mainly adjusted to align with the distribution of Ge ( i ) to ensure that the sample r is converted from RGB to IR mode in an optimal direction.
  • Step S30 based on the RGB modal image to be detected and the IR modal image, obtain the probability that the RGB modal image to be detected belongs to a real face through a trained face anti-counterfeiting detection model.
  • the training method is:
  • Step C10 obtaining the RGB and IR modal training sample pair sets and the sample label of each RGB and IR modal training sample pair;
  • Step C20 randomly select any RGB and IR modal training sample pair of the RGB and IR modal training sample pairs, and use the Branch-R and Branch-I branches of the face anti-counterfeiting detection model to extract the RGB modal training samples respectively. Characteristics of samples and IR modality training samples;
  • Step C30 using the Shared-branch branch of the face anti-counterfeiting detection model to perform forward information fusion and feedback on the features of the RGB modal training samples and the IR modal training samples, and calculate the classification loss value in combination with the sample labels;
  • Step C40 if the classification loss value is greater than the set threshold, adjust the parameters of the face anti-counterfeiting detection model, and jump to step C20 until the classification loss value is less than or equal to the set threshold, and obtain a trained person. Face anti-counterfeiting detection model.
  • the face anti-counterfeiting detection model includes three branches: Branch-I, Branch-R and Shared-branch.
  • the network structure of Branch-I is a ResNet network, which takes G(r) samples as input, passes through a convolutional layer and a maximum pooling layer, and then connects to 4 Resblocks, namely Res1, Res2, Res3 and Res4, followed by one
  • the global pooling layer is input to the binary classification loss function layer. Because IR modal samples contain anti-counterfeiting discrimination information that is not available in RGB samples, the output features of each Resblock of this branch are used as auxiliary information to guide the learning of RGB samples.
  • the binary loss function layer is a softmax feature classifier.
  • the face anti-counterfeiting detection task is regarded as a binary classification task, and the output of the global pooling layer in this branch is the discriminative feature. To sum up, the binary classification loss function of this branch is shown in formula (15):
  • Branch-R is similar to Branch-I, which takes sample r as input and adopts the same network structure as Branch-I.
  • this branch also uses the output of the Shared-branch corresponding to the Resblock as the input of the corresponding module.
  • L RGB is used, as shown in formula (16):
  • each Resblock of the Shared-branch branch is not only used as the input of the corresponding module of Branch-R, but also each Resblock uses the output of the corresponding module of Branch-R as the input.
  • the two-class loss function L Shared is used, as shown in formula (17):
  • y is the sample label of the training sample
  • the present invention introduces a shared branch as a feature intermediary. During the network training process, it fuses the feature output of each Resblock in a specific modal branch, performs feature selection, and inputs it into Branch-R. This process can be expressed as information forward fusion and feedback. The description of the information forward fusion stage is shown in formula (19):
  • FIG. 4 it is a schematic diagram of the modal assistance of an embodiment of the face anti-counterfeiting detection method based on the cross-modal transformation assistance of the present invention: (1) The converted IR modal sample G(r) is compared with the original sample r Enter Branch-I and Branch-R respectively. The anti-counterfeiting features of the samples are learned under the supervision of a binary classification loss function. (2) Send the output features of each Resblock in Branch-I and Branch-R into the Resblock corresponding to Shared-branch. At the same time, the outputs of the Res1 modules of Branch-I and Branch-R are summed as the input of Shared-branch. (3) Feedback the features of each Resblock in the Shared-branch to the Branch-R of the corresponding module to complete the auxiliary role of the IR modal features for the learning of RGB sample features.
  • Step S40 if the probability value is greater than the set threshold, the RGB modal image to be detected is a real face image; otherwise, the RGB modal image to be detected is an attack image.
  • FIG. 5 it is an example diagram of the face anti-counterfeiting detection process based on an embodiment of the cross-modal transformation-assisted face anti-counterfeiting detection method: (1) Collect the RGB modal sample r to be tested. (2) Perform face detection on the samples to be tested, and crop them to the same size as the training set samples. (3) Input the preprocessed sample r into the face modal conversion model, perform modal conversion, and obtain the converted sample G(r). (4) Input the sample r together with the sample G(r) into the face anti-counterfeiting detection model, and predict the probability that the sample r is judged as a real sample. (5) According to the preset threshold, make the final judgment on the sample r. If the probability value is greater than the set threshold, it is determined as a real face image, otherwise it is an attack image.
  • the face anti-counterfeiting detection system based on cross-modal transformation assistance includes a preprocessing unit, a face model state conversion unit, face anti-counterfeiting detection unit and discrimination unit;
  • the preprocessing unit is configured to perform face detection on the acquired RGB modal image with a human face, and perform cropping of the RGB modal image with a human face through the detected face bounding box, and zoom to the setting. Determine the size to obtain the RGB modal image to be detected;
  • the face modal conversion unit is configured to generate a corresponding IR modal image through the trained face modal conversion model based on the RGB modal image to be detected;
  • the face anti-counterfeiting detection unit is configured to obtain the probability that the to-be-detected RGB modal image belongs to a real face through a trained face anti-counterfeiting detection model based on the to-be-detected RGB modal image and the IR modal image;
  • the discriminating unit is configured to, if the probability value is greater than a set threshold, the RGB modal image to be detected is a real face image; otherwise, the RGB modal image to be detected is an attack image.
  • the face anti-counterfeiting detection system based on cross-modal transformation assistance provided in the above-mentioned embodiments is only illustrated by the division of the above-mentioned functional modules. That is, the modules or steps in the embodiments of the present invention are decomposed or combined. For example, the modules in the above-mentioned embodiments can be combined into one module, or can be further split into multiple sub-modules, so as to complete all the above descriptions. or some functions.
  • the names of the modules and steps involved in the embodiments of the present invention are only for distinguishing each module or step, and should not be regarded as an improper limitation of the present invention.
  • a storage device stores a plurality of programs, and the programs are adapted to be loaded and executed by a processor to implement the above-mentioned method for face anti-counterfeiting detection based on cross-modal transformation assistance.
  • a processing device includes a processor and a storage device; the processor is adapted to execute various programs; the storage device is adapted to store multiple programs; the programs are adapted to be loaded and executed by the processor In order to realize the above-mentioned face anti-counterfeiting detection method based on cross-modal transformation assistance.

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Multimedia (AREA)
  • Data Mining & Analysis (AREA)
  • Human Computer Interaction (AREA)
  • General Engineering & Computer Science (AREA)
  • Evolutionary Computation (AREA)
  • Evolutionary Biology (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Bioinformatics & Computational Biology (AREA)
  • Bioinformatics & Cheminformatics (AREA)
  • Artificial Intelligence (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Health & Medical Sciences (AREA)
  • General Health & Medical Sciences (AREA)
  • Oral & Maxillofacial Surgery (AREA)
  • Image Analysis (AREA)
  • Image Processing (AREA)

Abstract

一种基于跨模态转化辅助的人脸防伪检测方法、系统及装置，旨在解决仅配备RGB摄像头的条件下无法利用IR模态辅助进行人脸检测，导致检测性能低的问题。所述方法包括：对输入图像进行人脸检测、图像裁剪；采用改进的CycleGAN通过RGB与IR模态训练样本对进行人脸模态转换单元的训练，并将输入图像转换为IR图像；通过RGB与IR模态训练样本对和样本标签进行人脸防伪检测模型的训练，并结合输入图像和IR图像计算输入图像属于真实人脸的概率；概率大于设定阈值的为真实人脸图像，否则为攻击图像。本方法利用IR模态更具判别的防伪信息辅助RGB样本的学习，并且IR模态仅参与训练阶段，模型进行人脸防伪检测准确率高，成本低。

Description

基于跨模态转化辅助的人脸防伪检测方法、系统及装置 技术领域
本发明属于图像处理与模式识别领域，具体涉及了一种基于跨模态转化辅助的人脸防伪检测方法、系统及装置。
背景技术
人脸识别系统已经是日常生活中不可或缺的设备，并广泛部署在手机移动端或配备人脸识别系统的场景中。然而这些人脸识别系统极易受到不法分子的恶意攻击，如在采集设备前展示真实用户的人脸图像，通过视频回放或者面具等攻击样本，以伪装成合法用户窃取私人数据或金钱等。因此，在人脸识别系统上部署人脸防伪检测模块不仅具有实际的应用意义，而且已经在多种场景中得到具体应用，如人脸支付、人脸安检、手机人脸解锁等人机交互情境。
在面对拍摄场景变化，光照变化或者相机设置变化等情况下，基于RGB摄像头的人脸防伪检测算法极易出现性能波动，而基于IR摄像头的人脸防伪系统则很少出现这种现象。因为IR摄像头主要捕获人脸样本辐射出的热量信息，该热量信息不仅可以有效判断待测样本的类别，而且不易受到外界环境变化的干扰。然而，大多数实际应用场景中人脸防伪系统很难配备一个额外的IR摄像头。基于这个事实与考虑，本领域还急需一种人脸防伪检测方法，可以利用IR模态的优秀判别特征辅助基于RGB摄像头进行人脸防伪检测，且无需在测试阶段提供真实的IR模态样本。
发明内容
为了解决现有技术中的上述问题，即现有技术在仅配备RGB摄像头的条件下无法利用IR模态辅助进行人脸防伪检测，因而导致检测性能低的问题，本发明提供了一种基于跨模态转化辅助的人脸防伪检测方法，该人脸防伪检测方法包括：
步骤S10，对获取的带人脸的RGB模态图像进行人脸检测，通过检测到的人脸边界框进行所述带人脸的RGB模态图像的裁剪，并缩放至设定大小，获得待检测RGB模态图像；
步骤S20，基于所述待检测RGB模态图像，通过训练好的人脸模态转换模型生成对应的IR模态图像；
步骤S30，基于所述待检测RGB模态图像和IR模态图像，通过训练好的人脸防伪检测模型获取所述待检测RGB模态图像属于真实人脸的概率；
步骤S40，若所述概率值大于设定阈值，则所述待检测RGB模态图像为真实人脸图像；否则，所述待检测RGB模态图像为攻击图像。
在一些优选的实施例中，所述人脸模态转换模型，其训练样本获取方法为：
步骤B10，获取带人脸的RGB模态图像集和对应的IR模态图像集；
步骤B20，对所述带人脸的RGB模态图像集中每一个图像进行人脸检测，通过检测到的人脸边界框进行对应的带人脸的RGB模态图像和IR模态图像的裁剪，并缩放至设定大小，获得RGB和IR模态训练样本对集。
在一些优选的实施例中，所述人脸模态转换模型，其训练方法为：
以所述人脸模态转换模型作为生成器G，构建反向生成器F以及判别器D I与D R，基于所述RGB和IR模态训练样本对集中的任一RGB和IR模态训练样本对，通过CycleGAN方法迭代进行对抗训练，直至总损失函数值低于设定阈值，获得训练好的人脸模态转换模型；
其中，所述总损失函数为：
Figure PCTCN2020108775-appb-000001
其中，
Figure PCTCN2020108775-appb-000002
和
Figure PCTCN2020108775-appb-000003
分别代表G与F的目标损失函数，L GAN(G e，D Ie)和L GAN(F e，D Re)代表特征空间G e与F e的目标损失函数，L cyc-final(G，F)代表循环约束一致性损失函数，α、λ为预设的权重因子，c＝1，2，3分别为G(r)，G(i)与i以及F(i)，F(r)与r的类别，r和i分别代表RGB模态训练样本和对应的IR模态训练样本，G(r)和G(i)分别代表RGB模态训练样本r和对应的IR模态训练样本i经过G后的样本。
在一些优选的实施例中，所述循环约束一致性损失函数为：
Figure PCTCN2020108775-appb-000004
其中，‖·‖ 1代表L1范数，P r和P i分别为RGB模态训练样本r和对应的IR模态训练样本i的分布，
Figure PCTCN2020108775-appb-000005
和
Figure PCTCN2020108775-appb-000006
分别代表在给定的概率分布下求模态训练样本r和对应的IR模态训练样本i的数学期望，F(i)代表RGB模态训练样本r对应的IR模态训练样本i经过F后的样本，F(G(r))代表RGB模态训练样本r经过G之后再经过F后的样本，G(F(i))代表RGB模态训练样本r对应的IR模态训练样本i经过F之后再经过G后的样本。
在一些优选的实施例中，所述G与F的目标损失函数包括判别器
Figure PCTCN2020108775-appb-000007
和生成器G的对抗损失函数以及判别器
Figure PCTCN2020108775-appb-000008
和反向生成器F的对抗损失函数；
所述判别器
Figure PCTCN2020108775-appb-000009
和生成器G的对抗损失函数为：
Figure PCTCN2020108775-appb-000010
所述判别器
Figure PCTCN2020108775-appb-000011
和反向生成器F的对抗损失函数为：
Figure PCTCN2020108775-appb-000012
其中，
Figure PCTCN2020108775-appb-000013
和
Figure PCTCN2020108775-appb-000014
分别代表在给定的概率分布下求模态训练样本r和对应的IR模态训练样本i的数学期望，
Figure PCTCN2020108775-appb-000015
表示三类判别器，类别1，2，3分别为G(r)，G(i)与i所属的类别，
Figure PCTCN2020108775-appb-000016
表示三类判别器，类别1，2，3分别为F(i)，F(r)与r所属的类别。
在一些优选的实施例中，所述特征空间G e与F e的目标损失函数为：
Figure PCTCN2020108775-appb-000017
Figure PCTCN2020108775-appb-000018
其中，r和i分别代表RGB模态训练样本和对应的IR模态训练样本，P r为IR模态训练样本i的分布，P le为RGB模态训练样本r对应的IR模态训练样本i在特征空间的特征分布，P Re为RGB模态训练样本r在特征空间的特征分布，G e(i)和G e(r)分别代表RGB模态训练样本r和对应的IR 模态训练样本i在G的空间特征，F e(r)和F e(i)分别代表RGB模态训练样本r和对应的IR模态训练样本i在F的空间特征，D Ie(G e(i))和D Ie(G e(r))分别代表G e(i)和G e(r)属于P Ie分布的概率，D Re(F e(r))和D Re(F e(i))分别代表F e(r)和F e(i)属于P Re分布的概率，
Figure PCTCN2020108775-appb-000019
代表求G e(i)∈P Ie分布下的数学期望，
Figure PCTCN2020108775-appb-000020
代表求G e(r)∈P Re分布下的数学期望。
在一些优选的实施例中，所述人脸防伪检测模型，其训练方法为：
步骤C10，获取RGB和IR模态训练样本对集以及每一个RGB和IR模态训练样本对的样本标签；
步骤C20，随机选取所述RGB和IR模态训练样本对集任一RGB和IR模态训练样本对，采用所述人脸防伪检测模型的Branch-R与Branch-I分支分别提取RGB模态训练样本与IR模态训练样本的特征；
步骤C30，采用所述人脸防伪检测模型的Shared-branch分支将所述RGB模态训练样本与IR模态训练样本的特征进行信息前向融合与反馈，并结合样本标签计算分类损失值；
步骤C40，若所述分类损失值大于设定阈值，则调整所述人脸防伪检测模型的参数，并跳转步骤C20，直至所述分类损失值小于或等于设定阈值，获得训练好的人脸防伪检测模型。
在一些优选的实施例中，所述分类损失值，其计算方法为：
Figure PCTCN2020108775-appb-000021
其中，
Figure PCTCN2020108775-appb-000022
为模型预测Branch-I分支中训练样本为真实人脸图像的概率，
Figure PCTCN2020108775-appb-000023
为模型预测Branch-R分支中训练样本为真实人脸图像的概率，
Figure PCTCN2020108775-appb-000024
为模型预测Shared-branch分支中融合特征为真实人脸特征的概率，y为训练样本的样本标签，y＝1表示训练样本为真实人脸图像，y＝0表示训练样本为攻击图像。
本发明的另一方面，提出了一种基于跨模态转化辅助的人脸防伪检测系统，基于上述的基于跨模态转化辅助的人脸防伪检测方法，该人脸防伪检测系统包括预处理单元、人脸模态转化单元、人脸防伪检测单元和判别单元；
所述预处理单元，配置为对获取的带人脸的RGB模态图像进行人脸检测，通过检测到的人脸边界框进行所述带人脸的RGB模态图像的裁剪，并缩放至设定大小，获得待检测RGB模态图像；
所述人脸模态转化单元，配置为基于所述待检测RGB模态图像，通过训练好的人脸模态转换模型生成对应的IR模态图像；
所述人脸防伪检测单元，配置为基于所述待检测RGB模态图像和IR模态图像，通过训练好的人脸防伪检测模型获取所述待检测RGB模态图像属于真实人脸的概率；
所述判别单元，配置为若所述概率值大于设定阈值，则所述待检测RGB模态图像为真实人脸图像；否则，所述待检测RGB模态图像为攻击图像。
本发明的第三方面，提出了一种存储装置，其中存储有多条程序，所述程序适于由处理器加载并执行以实现上述的基于跨模态转化辅助的人脸防伪检测方法。
本发明的第四方面，提出了一种处理装置，包括处理器、存储装置；所述处理器，适于执行各条程序；所述存储装置，适于存储多条程序；所述程序适于由处理器加载并执行以实现上述的基于跨模态转化辅助的人脸防伪检测方法。
本发明的有益效果：
(1)本发明基于跨模态转化辅助的人脸防伪检测方法，充分利用了IR模态图像，IR模态图像主要捕获人脸样本辐射出的热量信息，该热量信息不仅可以有效判断待测样本的类别，而且不易受到外界环境 变化的干扰，本发明利用IR模态更具判别的防伪信息辅助RGB样本的学习，减少了外界环境干扰造成的人脸防伪检测算法的性能波动，大大提高了RGB人脸防伪检测的准确率与精度。
(2)本发明基于跨模态转化辅助的人脸防伪检测方法，仅在模型训练中引入IR模态图像，而模型训练完成后的测试中，仅需要RGB模态的图像即可通过模态转化获得对应的IR模态信息，系统无需配置额外的IR摄像头，降低了安装的难度与成本。
附图说明
通过阅读参照以下附图所作的对非限制性实施例所作的详细描述，本申请的其它特征、目的和优点将会变得更明显：
图1是本发明基于跨模态转化辅助的人脸防伪检测方法的流程示意图；
图2是本发明基于跨模态转化辅助的人脸防伪检测方法一种实施例的训练样本获取流程示意图；
图3是本发明基于跨模态转化辅助的人脸防伪检测方法一种实施例的人脸模态转化示意图；
图4是本发明基于跨模态转化辅助的人脸防伪检测方法一种实施例的模态辅助示意图；
图5是本发明基于跨模态转化辅助的人脸防伪检测方法一种实施例的人脸防伪检测过程示例图。
具体实施方式
下面结合附图和实施例对本申请作进一步的详细说明。可以理解的是，此处所描述的具体实施例仅用于解释相关发明，而非对该发 明的限定。另外还需要说明的是，为了便于描述，附图中仅示出了与有关发明相关的部分。
需要说明的是，在不冲突的情况下，本申请中的实施例及实施例中的特征可以相互组合。下面将参考附图并结合实施例来详细说明本申请。
本发明的一种基于跨模态转化辅助的人脸防伪检测方法，该人脸防伪检测方法包括：
步骤S10，对获取的带人脸的RGB模态图像进行人脸检测，通过检测到的人脸边界框进行所述带人脸的RGB模态图像的裁剪，并缩放至设定大小，获得待检测RGB模态图像；
步骤S20，基于所述待检测RGB模态图像，通过训练好的人脸模态转换模型生成对应的IR模态图像；
步骤S30，基于所述待检测RGB模态图像和IR模态图像，通过训练好的人脸防伪检测模型获取所述待检测RGB模态图像属于真实人脸的概率；
步骤S40，若所述概率值大于设定阈值，则所述待检测RGB模态图像为真实人脸图像；否则，所述待检测RGB模态图像为攻击图像。
为了更清晰地对本发明基于跨模态转化辅助的人脸防伪检测方法进行说明，下面结合图1对本发明实施例中各步骤展开详述。
本发明第一实施例的基于跨模态转化辅助的人脸防伪检测方法，包括步骤S10-步骤S40，各步骤详细描述如下：
步骤S10，对获取的带人脸的RGB模态图像进行人脸检测，通过检测到的人脸边界框进行所述带人脸的RGB模态图像的裁剪，并缩放至设定大小，获得待检测RGB模态图像。
如图2所示，为本发明基于跨模态转化辅助的人脸防伪检测方法一种实施例的训练样本获取流程示意图，人脸模态转换模型的训练样本需要同时包含RGB模态图像和IR模态图像：
步骤B10，获取带人脸的RGB模态图像集和对应的IR模态图像集。
步骤B20，对所述带人脸的RGB模态图像集中每一个图像进行人脸检测，通过检测到的人脸边界框进行对应的带人脸的RGB模态图像和IR模态图像的裁剪，并缩放至设定大小，获得RGB和IR模态训练样本对集。
模型的训练和测试阶段，图像都是由预处理单元进行处理，将RGB图像输入到预处理单元，预处理单元首先对图像进行人脸检测，若未检测到人脸则放弃该张图片；否则，再对图像进行中心裁剪，即保留人脸中心区域，并裁剪成固定的大小。本发明一个实施例中，将图像裁剪为128*128大小，在其他实施例中还可以根据不同的应用场景进行裁剪尺寸设定，本发明对此不作限定。
测试阶段的RGB模态图像预处理完成后即可进行检测，训练阶段的RGB模态图像预处理完成后与对应的预处理完成后的IR模态图像配成训练样本对。
步骤S20，基于所述待检测RGB模态图像，通过训练好的人脸模态转换模型生成对应的IR模态图像。
人脸模态转换模型，其训练方法为：
以所述人脸模态转换模型作为生成器G，构建反向生成器F以及判别器D I与D R，基于所述RGB和IR模态训练样本对集中的任一RGB和IR模态训练样本对，通过CycleGAN方法迭代进行对抗训练，直至总损失函数值低于设定阈值，获得训练好的人脸模态转换模型。
本发明通过CycleGAN方法进行对抗训练，该训练过程进行了两个方面的改进：(1)在原始像素空间将循环一致约束L cyc(G，F)从源模态扩展到目标模态；(2)在映射函数子空间对模态的映射方向进行调整，确保样本在模态转换过程中以最优的方向从原始模态向目标模态转换。
因而，总损失函数包括了判别器和生成器对抗损失函数、特征空间判别器和生成器对抗损失函数和循环约束一致性损失函数。
r表示一个RGB模态(简写为R)的人脸样本，分布为P r。同理，i表示一个属于IR模态(简写为I)的样本，分布为P i。因此，人将由样本r与i构成的样本对作为网络输入，在两个判别器D I与D R的监督下，分别训练两个循环映射函数G：R→I和F：I→R。基于GAN的对抗训练策略，将该过程如式(1)所示：
Figure PCTCN2020108775-appb-000025
对于反向生成器F与判别器D R也采用相同的原理，如式(2)所示：
Figure PCTCN2020108775-appb-000026
其中，生成器G尽可能愚弄判别器D I，使其相信转换后的样本G(r)属于I模态，而判别器D I尽力分辨G(r)与真实属于I模态的样本i。反向生成器F尽可能愚弄判别器D R，使其相信转换后的样本F(i)属于R模态，而判别器D R尽力分辨F(i)与真实属于R模态的样本r。
为了进一步规范两个映射函数G与F，将一个循环一致约束(cycle-consistency loss)引入CycleGAN框架中，确保每个样本r(或i)可以返回到原始状态，以解决不同模态样本不对齐的问题。该循环约束一致性损失函数L cyc(G，F)如式(3)所示：
Figure PCTCN2020108775-appb-000027
其中，‖·‖ 1代表L1范数。
从而，该过程的局部总损失函数如式(4)所示：
L MT-j＝L GAN(G，D I)+L GAN(F，D R)+λL cyc(G，F)
              (4)
其中，L GAN(G，D I)与L GAN(F，D R)为两个映射函数G与F对应损失函数，λ控制循环一致约束L cyc(G，F)在训练过程中所占的比重。所有生成器与判别器以交替训练的方式直至网络收敛。
基于上述模块对循环一致约束L cyc(G，F)的讨论，可以发现该约束的本质是在源模态促使两个映射函数G与F循环一致。如对于样本r来说满足：r→G(r)→F(G(r))≈r，同理对于样本i满足：i→F(i)→G(F(i))≈i。然而，该约束在目标模态的作用是缺席的。因此，需要引入一个额外的约束L cyc-tm(G)(或L cyc-tm(F))鼓励映射函数G(或F)将来自目标模态的样本i(或r)映射到本身。如对于样本i来说满足：i→G(i)≈i，同理对于样本r满足：r→F(r)≈r。约束L cyc-tm(G)如式(5)所示：
Figure PCTCN2020108775-appb-000028
综上所述，最终的循环约束一致性损失函数如式(6)所示：
Figure PCTCN2020108775-appb-000029
其中，‖·‖ 1代表L1范数，P r和P i分别为RGB模态训练样本r和对应的IR模态训练样本i的分布，
Figure PCTCN2020108775-appb-000030
和
Figure PCTCN2020108775-appb-000031
分别代表在给定的概率分布下求模态训练样本r和对应的IR模态训练样本i的数学期望，F(i)代表RGB 模态训练样本r对应的IR模态训练样本i经过映射函数F后的样本，F(G(r))代表RGB模态训练样本r经过映射函数G之后再经过映射函数F后的样本，G(F(i))代表RGB模态训练样本r对应的IR模态训练样本i经过映射函数F之后再经过映射函数G后的样本。
因为，在生成器G与反向生成器F中引入了新转换的样本G(i)与F(r)，在对应的判别器D I与D R中应将这些样本单独归为一类。通过将公式(1)根据生成器与判别器分步训练的策略拆写为L G-GAN与
Figure PCTCN2020108775-appb-000032
分别如式(7)和式(8)所示：
Figure PCTCN2020108775-appb-000033
Figure PCTCN2020108775-appb-000034
其中，D I本质一个二分类判别器，将G(r)与i样本分别归为类别1与类别2。首先用
Figure PCTCN2020108775-appb-000035
代替D I将判别器
Figure PCTCN2020108775-appb-000036
重写，如式(9)所示：
Figure PCTCN2020108775-appb-000037
类似地，将样本G(i)，G(r)与i分别归为类别1，2，3。因此，最终判别器
Figure PCTCN2020108775-appb-000038
和生成器G的对抗损失函数如式(10)所示：
Figure PCTCN2020108775-appb-000039
                   
Figure PCTCN2020108775-appb-000040
其中，
Figure PCTCN2020108775-appb-000041
和
Figure PCTCN2020108775-appb-000042
分别代表在给定的概率分布下求模态训练样本r和对应的IR模态训练样本i的数学期望，
Figure PCTCN2020108775-appb-000043
表示三类判别器，类别1，2，3分别为G(r)，G(i)与i所属的类别。
类似地，最终判别器
Figure PCTCN2020108775-appb-000044
和反向生成器F的对抗损失函数如式(11)所示：
Figure PCTCN2020108775-appb-000045
其中，
Figure PCTCN2020108775-appb-000046
和
Figure PCTCN2020108775-appb-000047
分别代表在给定的概率分布下求模态训练样本r和对应的IR模态训练样本i的数学期望，
Figure PCTCN2020108775-appb-000048
表示三类判别器，类别1，2，3分别为F(i)，F(r)与r所属的类别。
通过对目标模态像素空间的约束，两个映射函数G与F无论输入何种模态的样本，都尽可能输出与目标模态不可区分的样本。实验验证，在引入约束后，生成器生成的样本在全局颜色方面与真实样本更为接近。
除了在目标模态像素空间进行约束，本发明进一步在特征子空间对映射方向进行调整。因为映射函数G(类似的方式对于F)是一个Encoder-Decoder级联的网络结构，在Encoder网络(表示为G e)之后对样本r的输出进行调整，使其尽可能接近样本i输出的分布。样本r与i满足：G e(r)～G(i)。由于样本r与i并非严格对齐，因此两个的特征空间G e(r)与G(i)的分布拟合并不适合采用L1或L2范数。本发明采用一个对抗的子空间学习策略对齐特征分布。将一个额外的判别器D Ie引入G e之后，调整G e(r)的分布与G(i)的分布对齐。类似于生成器G与判别器D I，该部分的目标损失函数如式(12)和式(13)所示：
Figure PCTCN2020108775-appb-000049
Figure PCTCN2020108775-appb-000050
其中，r和i分别代表RGB模态训练样本和对应的IR模态训练样本，P r为IR模态训练样本i的分布，P Ie为RGB模态训练样本r对应的IR模态训练样本i在特征空间的特征分布，P Re为RGB模态训练样本r在特征空间的特征分布，G e(i)和G e(r)分别代表RGB模态训练样本r和对应的IR模态训练样本i在映射函数G的空间特征，F e(r)和F e(i)分别代表RGB模态训练样本r和对应的IR模态训练样本i在映射函数F的空间特征，D Ie(G e(i))和D Ie(G e(r))分别代表G e(i)和G e(r)属于P Ie分布的概率，D Re(F e(r))和D Re(F e(i))分别代表F e(r)和F e(i)属于P Re分布的概率，
Figure PCTCN2020108775-appb-000051
代表求G e(i)∈P Ie分布下的数学期望，
Figure PCTCN2020108775-appb-000052
代表求G e(r)∈P Re分布下的数学期望。
通过对抗损失函数的约束，不仅保留了目标模态的结构信息，而且在子空间调整了映射函数G对样本r的映射方向，确保以最合适的方向从原始模态向目标模态转换。
综上所述，人脸模态转换模型的总损失函数如式(14)所示：
Figure PCTCN2020108775-appb-000053
其中，其中，
Figure PCTCN2020108775-appb-000054
和
Figure PCTCN2020108775-appb-000055
分别代表G与F的目标损失函数，L GAN(G e，D Ie)和L GAN(F e，D Re)代表特征空间G e与F e的目标损失函数，L cyc-final(G，F)代表循环约束一致性损失函数，α、λ为预设的权重因 子，c＝1，2，3分别为G(r)，G(i)与i以及F(i)，F(r)与r的类别，r和i分别代表RGB模态训练样本和对应的IR模态训练样本，G(r)和G(i)分别代表RGB模态训练样本r和对应的IR模态训练样本i经过G后的样本。
如图3所示，为本发明基于跨模态转化辅助的人脸防伪检测方法一种实施例的人脸模态转化示意图：(1)将配对的训练集样本(r与i)输入模态转换网络中进行训练。(2)其中该模块包含两个互逆的映射函数G：R→I和F：I→R。该单元主要用到映射函数G：R→I，而采用两个互逆的映射函数主要为了解决不同模态样本不对齐的问题。(3)在原始像素空间将循环一致约束L cyc(G，F)从源模态扩展到目标模态。如样本r满足：r→G(r)→F(G(r))≈r，样本i满足：i→F(i)→G(F(i))≈i。同时，将新生成的样本G(i)作为单独类别融入判别器D I中。将原始的二分类判别器D I替换为三分类判别器
Figure PCTCN2020108775-appb-000056
(4)在映射函数G的子空间G e对样本G e(i)与样本G e(r)进行约束。主要调整G e(r)的分布与G e(i)的分布对齐，确保样本r以最优的方向从RGB模态向IR模态转换。
步骤S30，基于所述待检测RGB模态图像和IR模态图像，通过训练好的人脸防伪检测模型获取所述待检测RGB模态图像属于真实人脸的概率。
人脸防伪检测模型，其训练方法为：
步骤C10，获取RGB和IR模态训练样本对集以及每一个RGB和IR模态训练样本对的样本标签；
步骤C20，随机选取所述RGB和IR模态训练样本对集任一RGB和IR模态训练样本对，采用所述人脸防伪检测模型的Branch-R与Branch-I分支分别提取RGB模态训练样本与IR模态训练样本的特征；
步骤C30，采用所述人脸防伪检测模型的Shared-branch分支将所述RGB模态训练样本与IR模态训练样本的特征进行信息前向融合与反馈，并结合样本标签计算分类损失值；
步骤C40，若所述分类损失值大于设定阈值，则调整所述人脸防伪检测模型的参数，并跳转步骤C20，直至所述分类损失值小于或等于设定阈值，获得训练好的人脸防伪检测模型。
人脸防伪检测模型包括三个分支：Branch-I、Branch-R和Shared-branch分支。
Branch-I的网络结构是一个ResNet网络，将G(r)样本作为输入，经过一个卷积层与最大池化层后，接4个Resblock，分别为Res1，Res2，Res3与Res4，随后接一个全局池化层后输入到二分类损失函数层。因为IR模态样本包含RGB样本中不具备的防伪判别信息，因此将该分支每个Resblock的输出特征作为辅助信息指导RGB样本的学习。其中二分类损失函数层是一个softmax特征分类器。将人脸防伪检测任务视为一个二分类任务，且该分支中全局池化层的输出为判别特征。综上，该分支的二分类损失函数如式(15)所示：
Figure PCTCN2020108775-appb-000057
其中，
Figure PCTCN2020108775-appb-000058
为模型预测Branch-I分支中训练样本为真实人脸图像的概率，y为训练样本的样本标签，y＝1表示训练样本为真实人脸图像，y＝0表示训练样本为攻击图像。
Branch-R类似于Branch-I，该分支将样本r作为输入，并采用与Branch-I相同的网络结构。该分支除了将每个Resblock的输出作为Shared-branch的输入，同时将Shared-branch对应Resblock的输出作为对应模块的输入。最后采用二分类损失函数L RGB，如式(16)所示：
Figure PCTCN2020108775-appb-000059
其中，
Figure PCTCN2020108775-appb-000060
为模型预测Branch-R分支中训练样本为真实人脸图像的概率，y为训练样本的样本标签，y＝1表示训练样本为真实人脸图像，y＝0表示训练样本为攻击图像。
Shared-branch分支每个Resblock的输出不仅作为Branch-R对应模块的输入，同时每个Resblock将Branch-R对应模块的输出作为输入。最后采用二分类损失函数L Shared，如式(17)所示：
Figure PCTCN2020108775-appb-000061
其中，
Figure PCTCN2020108775-appb-000062
为模型预测Shared-branch分支中融合特征为真实人脸特征的概率，y为训练样本的样本标签，y＝1表示训练样本为真实人脸图像，y＝0表示训练样本为攻击图像。
因而，分类损失函数如式(18)所示：
L MA＝L IR+L RGB+L Shared
               (18)
在训练过程中，将IR模态样本的特征指导RGB样本的特征学习。如果将Branch-I中每个Resblock的输出直接融入Branch-R对应Resblock中，会导致最终Branch-R的性能变差。因为两个模态之间的样本分布差距(modal gap)会影响判别特征的学习。为此，本发明引入共享分支作为特征中介，在网络训练过程中，它融合特定模态分支中每个Resblock的特征输出，进行特征选择后输入Branch-R中。该过程可以表述为信息前向融合与反馈。信息前向融合阶段描述如式(19)所示：
Figure PCTCN2020108775-appb-000063
其中，
Figure PCTCN2020108775-appb-000064
为特定模态第t个Resblock的输出，
Figure PCTCN2020108775-appb-000065
为样本的角标，
Figure PCTCN2020108775-appb-000066
为Shared-branch第t+1个Resblock的输入，S t表示第t个 Resblock的输出。同时，采用Shared-branch的特征信息进行反馈，该反馈过程仅发生在Shared-branch与Branch-R之间，如式(20)所示：
Figure PCTCN2020108775-appb-000067
在特征融合之后，
Figure PCTCN2020108775-appb-000068
为Branch-R中第t+1个Resblock的输入，
Figure PCTCN2020108775-appb-000069
为Branch-R中第t个Resblock的输出，S t为Shared-branch的反馈特征
在Shared-branch与Branch-I之间不进行信息反馈是因为IR模态的样本特征作为辅助信息，尽可能地减小其被RGB样本特征的影响。除此之外，对三个分支的全局池化层输出特征进行元素相加求和操作，并将求和后的特征作为输入样本r的最终判别特征。
如图4所示，为本发明基于跨模态转化辅助的人脸防伪检测方法一种实施例的模态辅助示意图：(1)将转换后的IR模态样本G(r)与原始样本r分别输入Branch-I与Branch-R中。在二分类损失函数的监督下学习样本的防伪特征。(2)将Branch-I与Branch-R中每个Resblock的输出特征送入Shared-branch对应的Resblock中。同时将Branch-I与Branch-R的Res1模块的输出求和后作为Shared-branch的输入。(3)将Shared-branch中每个Resblock的特征反馈到对应模块的Branch-R中，完成IR模态特征对RGB样本特征学习的辅助作用。
步骤S40，若所述概率值大于设定阈值，则所述待检测RGB模态图像为真实人脸图像；否则，所述待检测RGB模态图像为攻击图像。
如图5所示，为本发明基于跨模态转化辅助的人脸防伪检测方法一种实施例的人脸防伪检测过程示例图：(1)采集RGB模态的待测样本r。(2)对待测样本进行人脸检测，并裁剪到与训练集样本相同的大小。(3)将预处理完毕的样本r输入人脸模态转换模型，进行模态转换，获得转换后的样本G(r)。(4)将样本r与样本G(r)一同输入人脸防伪检测模型，预测样本r被判定为真实样本的概率。(5)根据预先设定的 阈值，对样本r进行最终判定。如概率值大于设定阈值，则判定为真实人脸图像，否则为攻击图像。
本发明第二实施例的基于跨模态转化辅助的人脸防伪检测系统，基于上述的基于跨模态转化辅助的人脸防伪检测方法，该人脸防伪检测系统包括预处理单元、人脸模态转化单元、人脸防伪检测单元和判别单元；
所述预处理单元，配置为对获取的带人脸的RGB模态图像进行人脸检测，通过检测到的人脸边界框进行所述带人脸的RGB模态图像的裁剪，并缩放至设定大小，获得待检测RGB模态图像；
所述人脸模态转化单元，配置为基于所述待检测RGB模态图像，通过训练好的人脸模态转换模型生成对应的IR模态图像；
所述人脸防伪检测单元，配置为基于所述待检测RGB模态图像和IR模态图像，通过训练好的人脸防伪检测模型获取所述待检测RGB模态图像属于真实人脸的概率；
所述判别单元，配置为若所述概率值大于设定阈值，则所述待检测RGB模态图像为真实人脸图像；否则，所述待检测RGB模态图像为攻击图像。
所属技术领域的技术人员可以清楚地了解到，为描述的方便和简洁，上述描述的系统的具体工作过程及有关说明，可以参考前述方法实施例中的对应过程，在此不再赘述。
需要说明的是，上述实施例提供的基于跨模态转化辅助的人脸防伪检测系统，仅以上述各功能模块的划分进行举例说明，在实际应用中，可以根据需要而将上述功能分配由不同的功能模块来完成，即将本发明实施例中的模块或者步骤再分解或者组合，例如，上述实施例的模块可以合并为一个模块，也可以进一步拆分成多个子模块，以完成以 上描述的全部或者部分功能。对于本发明实施例中涉及的模块、步骤的名称，仅仅是为了区分各个模块或者步骤，不视为对本发明的不当限定。
本发明第三实施例的一种存储装置，其中存储有多条程序，所述程序适于由处理器加载并执行以实现上述的基于跨模态转化辅助的人脸防伪检测方法。
本发明第四实施例的一种处理装置，包括处理器、存储装置；处理器，适于执行各条程序；存储装置，适于存储多条程序；所述程序适于由处理器加载并执行以实现上述的基于跨模态转化辅助的人脸防伪检测方法。
所属技术领域的技术人员可以清楚地了解到，为描述的方便和简洁，上述描述的存储装置、处理装置的具体工作过程及有关说明，可以参考前述方法实施例中的对应过程，在此不再赘述。
本领域技术人员应该能够意识到，结合本文中所公开的实施例描述的各示例的模块、方法步骤，能够以电子硬件、计算机软件或者二者的结合来实现，软件模块、方法步骤对应的程序可以置于随机存储器(RAM)、内存、只读存储器(ROM)、电可编程ROM、电可擦除可编程ROM、寄存器、硬盘、可移动磁盘、CD-ROM、或技术领域内所公知的任意其它形式的存储介质中。为了清楚地说明电子硬件和软件的可互换性，在上述说明中已经按照功能一般性地描述了各示例的组成及步骤。这些功能究竟以电子硬件还是软件方式来执行，取决于技术方案的特定应用和设计约束条件。本领域技术人员可以对每个特定的应用来使用不同方法来实现所描述的功能，但是这种实现不应认为超出本发明的范围。
术语“第一”、“第二”等是用于区别类似的对象，而不是用于描述或表示特定的顺序或先后次序。
术语“包括”或者任何其它类似用语旨在涵盖非排他性的包含，从而使得包括一系列要素的过程、方法、物品或者设备/装置不仅包括那些要素，而且还包括没有明确列出的其它要素，或者还包括这些过程、方法、物品或者设备/装置所固有的要素。
至此，已经结合附图所示的优选实施方式描述了本发明的技术方案，但是，本领域技术人员容易理解的是，本发明的保护范围显然不局限于这些具体实施方式。在不偏离本发明的原理的前提下，本领域技术人员可以对相关技术特征做出等同的更改或替换，这些更改或替换之后的技术方案都将落入本发明的保护范围之内。

Claims (11)

  1. 一种基于跨模态转化辅助的人脸防伪检测方法，其特征在于，该人脸防伪检测方法包括：
    步骤S10，对获取的带人脸的RGB模态图像进行人脸检测，通过检测到的人脸边界框进行所述带人脸的RGB模态图像的裁剪，并缩放至设定大小，获得待检测RGB模态图像；
    步骤S20，基于所述待检测RGB模态图像，通过训练好的人脸模态转换模型生成对应的IR模态图像；
    步骤S30，基于所述待检测RGB模态图像和IR模态图像，通过训练好的人脸防伪检测模型获取所述待检测RGB模态图像属于真实人脸的概率；
    步骤S40，若所述概率值大于设定阈值，则所述待检测RGB模态图像为真实人脸图像；否则，所述待检测RGB模态图像为攻击图像。
  2. 根据权利要求1所述的基于跨模态转化辅助的人脸防伪检测方法，其特征在于，所述人脸模态转换模型，其训练样本获取方法为：
    步骤B10，获取带人脸的RGB模态图像集和对应的IR模态图像集；
    步骤B20，对所述带人脸的RGB模态图像集中每一个图像进行人脸检测，通过检测到的人脸边界框进行对应的带人脸的RGB模态图像和IR模态图像的裁剪，并缩放至设定大小，获得RGB和IR模态训练样本对集。
  3. 根据权利要求2所述的基于跨模态转化辅助的人脸防伪检测方法，其特征在于，所述人脸模态转换模型，其训练方法为：
    以所述人脸模态转换模型作为生成器G，构建反向生成器F以及判别器D I与D R，基于所述RGB和IR模态训练样本对集中的任一RGB和IR模态训练样本对，通过CycleGAN方法迭代进行对抗训练，直至总损失函数值 低于设定阈值，获得训练好的人脸模态转换模型；
    其中，所述总损失函数为：
    Figure PCTCN2020108775-appb-100001
    其中，其中，
    Figure PCTCN2020108775-appb-100002
    和
    Figure PCTCN2020108775-appb-100003
    分别代表G与F的目标损失函数，L GAN(G e，D Ie)和L GAN(F e，D Re)代表特征空间G e与F e的目标损失函数，L cyc-final(G，F)代表循环约束一致性损失函数，α、λ为预设的权重因子，c＝1，2，3分别为G(r)，G(i)与i以及F(i)，F(r)与r的类别，r和i分别代表RGB模态训练样本和对应的IR模态训练样本，G(r)和G(i)分别代表RGB模态训练样本r和对应的IR模态训练样本i经过G后的样本。
  4. 根据权利要求3所述的基于跨模态转化辅助的人脸防伪检测方法，其特征在于，所述循环约束一致性损失函数为：
    Figure PCTCN2020108775-appb-100004
    其中，‖·‖ 1代表L1范数，P r和P i分别为RGB模态训练样本r和对应的IR模态训练样本i的分布，
    Figure PCTCN2020108775-appb-100005
    和
    Figure PCTCN2020108775-appb-100006
    分别代表在给定的概率分布下求模态训练样本r和对应的IR模态训练样本i的数学期望，F(i)代表RGB模态训练样本r对应的IR模态训练样本i经过F后的样本，F(G(r))代表RGB模态训练样本r经过G之后再经过F后的样本，G(F(i))代表RGB模态训练样本r对应的IR模态训练样本i经过F之后再经过G后的样本。
  5. 根据权利要求3所述的基于跨模态转化辅助的人脸防伪检测方法，其特征在于，所述G与F的目标损失函数包括判别器
    Figure PCTCN2020108775-appb-100007
    和生成器G的对抗损失函数以及判别器
    Figure PCTCN2020108775-appb-100008
    和反向生成器F的对抗损失函数；
    所述判别器
    Figure PCTCN2020108775-appb-100009
    和生成器G的对抗损失函数为：
    Figure PCTCN2020108775-appb-100010
    所述判别器
    Figure PCTCN2020108775-appb-100011
    和生成器F的对抗损失函数为：
    Figure PCTCN2020108775-appb-100012
    其中，
    Figure PCTCN2020108775-appb-100013
    和
    Figure PCTCN2020108775-appb-100014
    分别代表在给定的概率分布下求模态训练样本r和对应的IR模态训练样本i的数学期望，
    Figure PCTCN2020108775-appb-100015
    表示三类判别器，类别1，2，3分别为G(r)，G(i)与i所属的类别，
    Figure PCTCN2020108775-appb-100016
    表示三类判别器，类别1，2，3分别为F(i)，F(r)与r所属的类别。
  6. 根据权利要求3所述的基于跨模态转化辅助的人脸防伪检测方法，其特征在于，所述特征空间G e与F e的目标损失函数为：
    Figure PCTCN2020108775-appb-100017
    其中，r和i分别代表RGB模态训练样本和对应的IR模态训练样本，P r为IR模态训练样本i的分布，P Ie为RGB模态训练样本r对应的IR模态训练样本i在特征空间的特征分布，P Re为RGB模态训练样本r在特征空间的特征分布，G e(i)和G e(r)分别代表RGB模态训练样本r和对应的IR模态训练样 本i在G的空间特征，F e(r)和F e(i)分别代表RGB模态训练样本r和对应的IR模态训练样本i在F的空间特征，D Ie(G e(i))和D Ie(G e(r))分别代表G e(i)和G e(r)属于P Ie分布的概率，D Re(F e(r))和D Re(F e(i))分别代表F e(r)和F e(i)属于P Re分布的概率，
    Figure PCTCN2020108775-appb-100018
    代表求G e(i)∈P Ie分布下的数学期望，
    Figure PCTCN2020108775-appb-100019
    代表求G e(r)∈P Re分布下的数学期望。
  7. 根据权利要求1所述的基于跨模态转化辅助的人脸防伪检测方法，其特征在于，所述人脸防伪检测模型，其训练方法为：
    步骤C10，获取RGB和IR模态训练样本对集以及每一个RGB和IR模态训练样本对的样本标签；
    步骤C20，随机选取所述RGB和IR模态训练样本对集任一RGB和IR模态训练样本对，采用所述人脸防伪检测模型的Branch-R与Branch-I分支分别提取RGB模态训练样本与IR模态训练样本的特征；
    步骤C30，采用所述人脸防伪检测模型的Shared-branch分支将所述RGB模态训练样本与IR模态训练样本的特征进行信息前向融合与反馈，并结合样本标签计算分类损失值；
    步骤C40，若所述分类损失值大于设定阈值，则调整所述人脸防伪检测模型的参数，并跳转步骤C20，直至所述分类损失值小于或等于设定阈值，获得训练好的人脸防伪检测模型。
  8. 根据权利要求7所述的基于跨模态转化辅助的人脸防伪检测方法，其特征在于，所述分类损失值，其计算方法为：
    Figure PCTCN2020108775-appb-100020
    其中，
    Figure PCTCN2020108775-appb-100021
    为模型预测Branch-I分支中训练样本为真实人脸图像的概率，
    Figure PCTCN2020108775-appb-100022
    为模型预测Branch-R分支中训练样本为真实人脸图像的概率，
    Figure PCTCN2020108775-appb-100023
    为 模型预测Shared-branch分支中融合特征为真实人脸特征的概率，y为训练样本的样本标签，y＝1表示训练样本为真实人脸图像，y＝0表示训练样本为攻击图像。
  9. 一种基于跨模态转化辅助的人脸防伪检测系统，其特征在于，基于权利要求1-8任一项所述的基于跨模态转化辅助的人脸防伪检测方法，该人脸防伪检测系统包括预处理单元、人脸模态转化单元、人脸防伪检测单元和判别单元；
    所述预处理单元，配置为对获取的带人脸的RGB模态图像进行人脸检测，通过检测到的人脸边界框进行所述带人脸的RGB模态图像的裁剪，并缩放至设定大小，获得待检测RGB模态图像；
    所述人脸模态转化单元，配置为基于所述待检测RGB模态图像，通过训练好的人脸模态转换模型生成对应的IR模态图像；
    所述人脸防伪检测单元，配置为基于所述待检测RGB模态图像和IR模态图像，通过训练好的人脸防伪检测模型获取所述待检测RGB模态图像属于真实人脸的概率；
    所述判别单元，配置为若所述概率值大于设定阈值，则所述待检测RGB模态图像为真实人脸图像；否则，所述待检测RGB模态图像为攻击图像。
  10. 一种存储装置，其中存储有多条程序，其特征在于，所述程序适于由处理器加载并执行以实现权利要求1-8任一项所述的基于跨模态转化辅助的人脸防伪检测方法。
  11. 一种处理装置，包括
    处理器，适于执行各条程序；以及
    存储装置，适于存储多条程序；
    其特征在于，所述程序适于由处理器加载并执行以实现：
    权利要求1-8任一项所述的基于跨模态转化辅助的人脸防伪检测方法。
PCT/CN2020/108775 2020-08-11 2020-08-13 基于跨模态转化辅助的人脸防伪检测方法、系统及装置 WO2022032549A1 (zh)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
CN202010802184.9 2020-08-11
CN202010802184.9A CN111881884B (zh) 2020-08-11 2020-08-11 基于跨模态转化辅助的人脸防伪检测方法、系统及装置

Publications (1)

Publication Number Publication Date
WO2022032549A1 true WO2022032549A1 (zh) 2022-02-17

Family

ID=73203516

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2020/108775 WO2022032549A1 (zh) 2020-08-11 2020-08-13 基于跨模态转化辅助的人脸防伪检测方法、系统及装置

Country Status (2)

Country Link
CN (1) CN111881884B (zh)
WO (1) WO2022032549A1 (zh)

Cited By (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN114694185A (zh) * 2022-05-31 2022-07-01 浪潮电子信息产业股份有限公司 一种跨模态目标重识别方法、装置、设备及介质
CN116309031A (zh) * 2023-05-16 2023-06-23 中国科学技术大学 人脸伪造主动干扰方法、系统、设备及存储介质
CN117392396A (zh) * 2023-12-08 2024-01-12 安徽蔚来智驾科技有限公司 跨模态目标状态的检测方法、设备、智能设备和介质
CN117636416A (zh) * 2023-10-24 2024-03-01 广州像素数据技术股份有限公司 基于人脸不适度的人脸伪造与活体联合检测方法和相关设备

Families Citing this family (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN112464741B (zh) * 2020-11-05 2021-11-26 马上消费金融股份有限公司 人脸分类方法、模型训练方法、电子设备及存储介质
CN112801998B (zh) * 2021-02-05 2022-09-23 展讯通信（上海）有限公司 一种印制电路板检测方法、装置、计算机设备和存储介质
CN113158854B (zh) * 2021-04-08 2022-03-22 东北大学秦皇岛分校 多模态信息融合的自动监测列车安全运行方法
CN115082299B (zh) * 2022-07-21 2022-11-25 中国科学院自动化研究所 非严格对齐的小样本不同源图像转换方法、系统及设备

Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN109284597A (zh) * 2018-11-22 2019-01-29 北京旷视科技有限公司 一种人脸解锁方法、装置、电子设备和计算机可读介质
CN109840475A (zh) * 2018-12-28 2019-06-04 深圳奥比中光科技有限公司 人脸识别方法及电子设备
CN110263621A (zh) * 2019-05-06 2019-09-20 北京迈格威科技有限公司 图像识别方法、装置及可读存储介质
CN110866470A (zh) * 2019-10-31 2020-03-06 湖北工程学院 一种基于随机图像特征的人脸防伪检测方法

Family Cites Families (14)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US9875393B2 (en) * 2014-02-12 2018-01-23 Nec Corporation Information processing apparatus, information processing method, and program
CN103886301B (zh) * 2014-03-28 2017-01-18 北京中科奥森数据科技有限公司 一种人脸活体检测方法
US9767358B2 (en) * 2014-10-22 2017-09-19 Veridium Ip Limited Systems and methods for performing iris identification and verification using mobile devices
US10198645B2 (en) * 2014-11-13 2019-02-05 Intel Corporation Preventing face-based authentication spoofing
CN108197563B (zh) * 2017-12-29 2022-03-11 百度在线网络技术（北京）有限公司 用于获取信息的方法及装置
CN108875935B (zh) * 2018-06-11 2020-08-11 兰州理工大学 基于生成对抗网络的自然图像目标材质视觉特征映射方法
US11663307B2 (en) * 2018-09-24 2023-05-30 Georgia Tech Research Corporation RtCaptcha: a real-time captcha based liveness detection system
CN109543640B (zh) * 2018-11-29 2022-06-17 中国科学院重庆绿色智能技术研究院 一种基于图像转换的活体检测方法
CN109299342B (zh) * 2018-11-30 2021-12-17 武汉大学 一种基于循环生成式对抗网络的跨模态检索方法
CN109753992B (zh) * 2018-12-10 2020-09-01 南京师范大学 基于条件生成对抗网络的无监督域适应图像分类方法
CN109919251A (zh) * 2019-03-21 2019-06-21 腾讯科技（深圳）有限公司 一种基于图像的目标检测方法、模型训练的方法及装置
CN110503626B (zh) * 2019-07-09 2021-11-23 上海交通大学 基于空间-语义显著性约束的ct图像模态对齐方法
CN111179207B (zh) * 2019-12-05 2022-04-08 浙江工业大学 一种基于并联生成网络的跨模态医学图像合成方法
CN111260741B (zh) * 2020-02-07 2022-05-10 北京理工大学 一种利用生成对抗网络的三维超声模拟方法及装置

Patent Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN109284597A (zh) * 2018-11-22 2019-01-29 北京旷视科技有限公司 一种人脸解锁方法、装置、电子设备和计算机可读介质
CN109840475A (zh) * 2018-12-28 2019-06-04 深圳奥比中光科技有限公司 人脸识别方法及电子设备
CN110263621A (zh) * 2019-05-06 2019-09-20 北京迈格威科技有限公司 图像识别方法、装置及可读存储介质
CN110866470A (zh) * 2019-10-31 2020-03-06 湖北工程学院 一种基于随机图像特征的人脸防伪检测方法

Non-Patent Citations (2)

* Cited by examiner, † Cited by third party
Title
GUAN-AN WANG; TIANZHU ZHANG. YANG YANG; JIAN CHENG; JIANLONG CHANG; XU LIANG; ZENGGUANG HOU: "Cross-Modality Paired-Images Generation for RGB-Infrared Person Re-Identification", ARXIV.ORG, CORNELL UNIVERSITY LIBRARY, 201 OLIN LIBRARY CORNELL UNIVERSITY ITHACA, NY 14853, 10 February 2020 (2020-02-10), 201 Olin Library Cornell University Ithaca, NY 14853 , XP081595141 *
HUIJIAO WANG; LI WANG; XULEI YANG; LEI YU; HAIJIAN ZHANG: "Facial Feature Embedded CycleGAN for VIS-NIR Translation", ARXIV.ORG, CORNELL UNIVERSITY LIBRARY, 201 OLIN LIBRARY CORNELL UNIVERSITY ITHACA, NY 14853, 20 April 2019 (2019-04-20), 201 Olin Library Cornell University Ithaca, NY 14853 , XP081171796 *

Cited By (8)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN114694185A (zh) * 2022-05-31 2022-07-01 浪潮电子信息产业股份有限公司 一种跨模态目标重识别方法、装置、设备及介质
CN114694185B (zh) * 2022-05-31 2022-11-04 浪潮电子信息产业股份有限公司 一种跨模态目标重识别方法、装置、设备及介质
CN116309031A (zh) * 2023-05-16 2023-06-23 中国科学技术大学 人脸伪造主动干扰方法、系统、设备及存储介质
CN116309031B (zh) * 2023-05-16 2023-08-29 中国科学技术大学 人脸伪造主动干扰方法、系统、设备及存储介质
CN117636416A (zh) * 2023-10-24 2024-03-01 广州像素数据技术股份有限公司 基于人脸不适度的人脸伪造与活体联合检测方法和相关设备
CN117636416B (zh) * 2023-10-24 2024-05-28 广州像素数据技术股份有限公司 基于人脸不适度的人脸伪造与活体联合检测方法和相关设备
CN117392396A (zh) * 2023-12-08 2024-01-12 安徽蔚来智驾科技有限公司 跨模态目标状态的检测方法、设备、智能设备和介质
CN117392396B (zh) * 2023-12-08 2024-03-05 安徽蔚来智驾科技有限公司 跨模态目标状态的检测方法、设备、智能设备和介质

Also Published As

Publication number Publication date
CN111881884B (zh) 2021-05-28
CN111881884A (zh) 2020-11-03

Similar Documents

Publication Publication Date Title
WO2022032549A1 (zh) 基于跨模态转化辅助的人脸防伪检测方法、系统及装置
WO2020186914A1 (zh) 行人再识别方法、装置及存储介质
Hsu et al. Camera response functions for image forensics: an automatic algorithm for splicing detection
Villalba et al. Smartphone image clustering
US9330312B2 (en) Multispectral detection of personal attributes for video surveillance
Smith-Creasey et al. Continuous face authentication scheme for mobile devices with tracking and liveness detection
CN111079816A (zh) 图像的审核方法、装置和服务器
Zhang et al. Face anti-spoofing detection based on DWT-LBP-DCT features
Zhang et al. Unsupervised learning-based framework for deepfake video detection
CN113642639B (zh) 活体检测方法、装置、设备和存储介质
CN111339897B (zh) 活体识别方法、装置、计算机设备和存储介质
CN113128481A (zh) 一种人脸活体检测方法、装置、设备及存储介质
US20110182497A1 (en) Cascade structure for classifying objects in an image
CN114220097A (zh) 一种基于对抗攻击的图像语义信息敏感像素域的筛选方法及应用方法与系统
CN113807237B (zh) 活体检测模型的训练、活体检测方法、计算机设备及介质
Bohong et al. Garbage detection algorithm based on YOLO v3
Al-Kawaz et al. Advanced facial recognition for digital forensics
Zhao et al. A comprehensive study on third order statistical features for image splicing detection
Zheng et al. Digital spliced image forensics based on edge blur measurement
Shekhawat et al. Realization of Smart and Highly Efficient IoTbased Surveillance System using Facial Recognition on FPGA
CN114267089B (zh) 一种伪造图像的识别方法、装置及设备
CN113516089B (zh) 人脸图像识别方法、装置、设备及可读存储介质
CN117094986B (zh) 基于小样本的自适应缺陷检测方法及终端设备
Safarzadeh et al. A Secure Face Anti-spoofing Approach Using Deep Learning
Mande et al. Smart door using face recognition

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 20949048

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 20949048

Country of ref document: EP

Kind code of ref document: A1