KR102674496B1 - Method and Apparatus for Detecting Fake Faces Using Remote Photoplethysmography Signals - Google Patents

Abstract

Obtaining a face image; calculating a remote photoplethysmographic signal (remote PPG signal) using the facial image; extracting features of the calculated remote photoplethysmographic signal; and whether the face image is a forged image by comparing the features of the forged image and the features of the remote photoplethysmographic signal using a first artificial intelligence model learned using an image containing a human face as learning data. A method for determining a counterfeit face, comprising: determining.

Description

Method and device for detecting fake faces using remote photoplethysmography signals {Method and Apparatus for Detecting Fake Faces Using Remote Photoplethysmography Signals}

The present disclosure relates to a method and device for identifying fake faces using a remote photoplethysmographic signal. More specifically, it relates to a method and device for identifying a fake face by comparing the characteristics of a fake image with the characteristics of a remote photoplethysmographic signal using a previously learned artificial intelligence model.

With the development of technology, security systems based on biometric information, including facial recognition, are being commercialized, and accordingly, attempts to forge biometric information, including the face, to attack biometric information-based security systems are increasing.

Face forgery detection is used to identify and defend against the features of a forged face, such as when a face recognition system is attacked with a forged face. However, when using conventional general identification technology, there were inconveniences to users, such as requiring expensive camera equipment or requiring specific gestures from the user.

Accordingly, there is a need for a method to identify fake faces without requiring specific actions from the user or requiring expensive equipment.

The present disclosure provides a method and device for identifying fake faces using a remote photoplethysmographic signal. The problem to be solved by the present disclosure is not limited to the problems mentioned above, and other problems and advantages of the present disclosure that are not mentioned can be understood through the following description and can be understood more clearly through the examples of the present disclosure. It will be. In addition, it will be appreciated that the problems and advantages to be solved by the present disclosure can be realized by the means and combinations thereof indicated in the patent claims.

As a technical means for achieving the above-described technical problem, a first aspect of the present disclosure includes acquiring a face image; calculating a remote photoplethysmographic signal (remote PPG signal) using the facial image; extracting features of the calculated remote photoplethysmographic signal; and whether the face image is a forged image by comparing the features of the forged image and the features of the remote photoplethysmographic signal using a first artificial intelligence model learned using an image containing a human face as learning data. A method for determining a counterfeit face may be provided, including the step of determining.

A second aspect of the present disclosure includes a memory storing at least one program; and a processor operating by executing the at least one program, wherein the processor acquires a facial image, calculates a remote PPG signal using the facial image, and calculates a remote PPG signal using the facial image. By extracting the features of the remote photoplethysmographic signal and comparing the features of the forged image with the features of the remote photoplethysmographic signal using the first artificial intelligence model learned using images containing human faces as learning data. , it is possible to provide a fake face determination device that determines whether the face image is a fake image.

A third aspect of the present disclosure may provide a computer-readable recording medium on which a program for executing the method of the first aspect of the present disclosure on a computer is recorded.

Other aspects, features and advantages in addition to those described above will become apparent from the following drawings, claims and detailed description of the invention.

According to the above-described means for solving the problem of the present disclosure, it is possible to determine whether or not the face has been forged without expensive equipment by using photoplethysmography (PPG), which measures changes in blood flow using an optical sensor.

In addition, according to another problem-solving method of the present disclosure, convenience can be provided to the user by using a remote photoplethysmography signal (remote PPG signal) without requiring any separate action from the user.

In addition, according to another problem-solving method of the present disclosure, high processing speed can be provided by maintaining high accuracy and at the same time performing a discrimination process in advance using display and photo characteristics and BPM characteristics in the stage before feature extraction.

1 is a diagram illustrating an example of a method of providing a service for determining a forged face using a counterfeit face identification device.
Figure 2 is a block diagram of a counterfeit face recognition device.
Figure 3 is a flowchart of a method for determining a fake face using a remote photoplethysmographic signal.
Figure 4 is a diagram for explaining a method of extracting a face area and a skin area.
Figure 5 is a diagram to explain a method of detecting and using features found in a photo or display in a face image.
FIG. 6 is a flowchart illustrating an example of performing a discrimination process using the characteristics of a display and a photo and performing a discrimination process using BPM estimation before extracting the characteristics of the remote photoplethysmographic signal.
Figure 7 is a diagram for explaining a method of extracting features using a remote photoplethysmographic signal and a frequency domain extracted from the remote photoplethysmographic signal.
Figure 8 is a graph to explain the distribution of main component values in the frequency domain and time domain calculated through principal component analysis.
Figure 9 is a graph showing the probability distribution of fake face discrimination for learning data of an artificial intelligence model that determines fake faces using the characteristics of a remote photoplethysmographic signal according to an embodiment.

The present disclosure can be implemented in various ways and can have various embodiments, and specific embodiments are illustrated in the drawings and described in the detailed description. The effects and features of the present disclosure, and methods for achieving them, will become clear with reference to the embodiments described in detail below along with the drawings.

However, the present disclosure is not limited to the embodiments disclosed below and may be implemented in various forms. In the following embodiments, terms such as first and second are used not in a limiting sense but for the purpose of distinguishing one component from another component. For example, without departing from the scope of the present disclosure, a first element described first may be later described as a second element, and similarly, the second element may also be described as a first element. . Additionally, singular expressions include plural expressions unless the context clearly dictates otherwise. In addition, terms such as include or have mean that the features or components described in the specification exist, and do not preclude the possibility of adding one or more other features or components.

Additionally, in the drawings, the sizes of components may be exaggerated or reduced for convenience of explanation. For example, the size and thickness of each component shown in the drawings are shown arbitrarily for convenience, and the present disclosure is not necessarily limited to what is shown.

Hereinafter, embodiments of the present disclosure will be described in detail with reference to the accompanying drawings. When describing with reference to the drawings, identical or corresponding components will be assigned the same reference numerals and redundant description thereof will be omitted. do.

Hereinafter, the present disclosure will be described in detail with reference to the attached drawings.

1 is a diagram illustrating an example of a method of providing a service for determining a forged face using a counterfeit face identification device.

Referring to FIG. 1, when providing a service for determining a forged face using a counterfeit face determination device, the service determines whether the facial image generated by photographing the subject 120 by the photographing device 110 is counterfeit. It can be confirmed that it can be provided. The counterfeit face identification device can identify photos containing a face, displays containing a face, and masks as counterfeit faces, but is not limited to this.

The fake face determination device may acquire a face image generated by the photographing device 110 photographing the subject 120.

Images of the present disclosure include still images and moving images. Meanwhile, an image that is essentially used to calculate a signal that changes with respect to time, for example, a remote photoplethysmographic signal, may be understood as a dynamic image. Additionally, a person skilled in the art can know this.

The photographing device 110 may be a part of the counterfeit face recognition device 200, or may capture a subject 120 outside the counterfeit face recognition device 200 to generate a face image and forge it using a network. It may be a device provided to the face discrimination device 200. The photographing device 110 may be positioned in front or on the side of the subject 120 and generate a face image by photographing the subject 120 .

The photographing device 110 may generate a face image using visible light reflected by the subject 120. The photographing device 110 may include, but is not limited to, any device that generates an image using visible light reflected by the subject 120.

The photographing device 110 may include a high-speed camera capable of continuous photographing at precise frame intervals, and may include a high-resolution camera capable of confirming the degree of change in RGB components of the face when photographing at precise frame intervals.

Figure 2 is a block diagram of a counterfeit face recognition device.

Referring to FIG. 2, the fake face determination device 200 may include a processor 210 and a memory 220. Since only components related to the embodiment are shown in the counterfeit face recognition device 200 in FIG. 2, it will be understood by those skilled in the art that other general-purpose components may be included in addition to the components shown in FIG. 2. If so, I can understand.

The counterfeit face recognition device 200 is applicable to smartphones, tablet PCs, PCs, smart TVs, mobile phones, personal digital assistants (PDAs), laptops, media players, micro servers, global positioning system (GPS) devices, digital broadcasting terminals, and navigation devices. , kiosks, MP3 players, home appliances, devices with cameras, and other mobile or non-mobile computing devices. Additionally, the counterfeit face determination device 200 may be a wearable device such as a watch, glasses, hair band, or ring equipped with a data processing function. However, it is not limited to this.

The processor 210 controls the overall operation of the counterfeit face determination device 200. For example, the processor 210 executes programs stored in the memory 220, such as an input unit (not shown), a display (not shown), a memory 220, a communication module (not shown), a camera (not shown), The time domain feature extraction unit (not shown), the frequency domain feature extraction unit (not shown), etc. can be generally controlled, and the operation of the counterfeit face determination device 200 can be controlled. The processor 210 may control at least some of the operations of the counterfeit face determination device 200 described in FIGS. 1 to 9 .

The processor 210 includes application specific integrated circuits (ASICs), digital signal processors (DSPs), digital signal processing devices (DSPDs), programmable logic devices (PLDs), field programmable gate arrays (FPGAs), controllers, and microcontrollers. It may be implemented using at least one of micro-controllers, microprocessors, and other electrical units for performing functions.

The memory 220 is hardware that stores various data processed within the counterfeit face recognition device 200, and can store programs for processing and control of the processor 210.

The memory 220 includes random access memory (RAM) such as dynamic random access memory (DRAM), static random access memory (SRAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), CD- It may include ROM, Blu-ray or other optical disk storage, a hard disk drive (HDD), a solid state drive (SSD), or flash memory.

Additionally, embodiments according to the present disclosure may be implemented in the form of a computer program that can be executed on a computer through various components, and such a computer program may be recorded on a computer-readable medium. At this time, the media includes magnetic media such as hard disks, floppy disks, and magnetic tapes, optical recording media such as CD-ROMs and DVDs, magneto-optical media such as floptical disks, and ROM. , RAM, flash memory, etc., may include hardware devices specifically configured to store and execute program instructions.

Meanwhile, the computer program may be specially designed and configured for the present disclosure, or may be known and usable by those skilled in the computer software field. Examples of computer programs may include not only machine language code such as that created by a compiler, but also high-level language code that can be executed by a computer using an interpreter or the like.

According to one embodiment, methods according to various embodiments of the present disclosure may be included and provided in a computer program product. Computer program products are commodities and can be traded between sellers and buyers. The computer program product may be distributed in the form of a machine-readable storage medium (e.g. compact disc read only memory (CD-ROM)) or through an application store (e.g. Play StoreTM) or between two user devices. It may be distributed in person or online (e.g., downloaded or uploaded). In the case of online distribution, at least a portion of the computer program product may be at least temporarily stored or temporarily created in a machine-readable storage medium, such as the memory of a manufacturer's server, an application store's server, or a relay server.

Figure 3 is a flowchart of a method for determining a fake face using a remote photoplethysmographic signal.

Referring to FIG. 3, in step 310, the fake face determination device 200 may acquire a face image.

In step 320, the fake face determination device 200 may calculate a remote photoplethysmography signal (remote PPG signal) using the face image.

The fake face determination device 200 can extract features of a face image.

The fake face recognition device 200 uses a second artificial intelligence model learned using images taken of photos containing faces as learning data, and compares the features of the photo with the features of the face image to determine the face image. It is possible to determine whether is a forged image.

The counterfeit face recognition device 200 compares the features of the display and the features of the face image using a third artificial intelligence model learned using images taken of a display containing a face as learning data, thereby detecting if the face image is forged. You can determine whether it is an image or not.

The fake face determination device 200 can extract the face area included in the face image.

The fake face determination device 200 can extract the skin area included in the face area.

The extracted face area and the extracted skin area may be areas extracted using at least one of deep learning and color threshold-based techniques.

The fake face determination device 200 can calculate a remote photoplethysmographic signal based on the skin area.

The counterfeit face recognition device 200 can extract RGB components from the skin area.

The counterfeit face determination device 200 may generate an RGB matrix corresponding to RGB components and QR decompose the RGB matrix.

The counterfeit face determination device 200 can calculate a remote photoplethysmographic signal using a QR decomposition matrix and the least squares method.

The counterfeit face recognition device 200 can estimate heart beats per minute using a remote photoplethysmographic signal.

The counterfeit face recognition device 200 may process a remote photoplethysmographic signal using Fourier transform to generate a transformed signal.

Based on the converted signal, the counterfeit face recognition device 200 can estimate the frequency with the largest amplitude in the remote photoplethysmographic signal as heart beats per minute.

The fake face determination device 200 can determine whether the face image is a fake image based on the heart rate per minute and a preset error range.

In step 330, the counterfeit face recognition device 200 may extract features of the calculated remote photoplethysmographic signal.

In step 340, the fake face determination device 200 compares the characteristics of the fake image and the characteristics of the remote photoplethysmographic signal using a first artificial intelligence model learned using an image containing a human face as learning data. , it is possible to determine whether the face image is a forged image.

The fake face determination device 200 may compare the features of the fake image and the features of the remote photoplethysmography signal to calculate a first similarity score in the time domain of the images and a second similarity score in the frequency domain. .

The fake face determination device 200 may determine whether the face image is a fake image based on the first similarity score, the second similarity score, and a preset threshold.

The fake face determination device 200 can also obtain the final features by combining the features in the time domain and the features in the frequency domain of the remote photoplethysmographic signal, and compare the features of the fake image and the final features, A third similarity score of the images may be calculated.

The fake face determination device 200 may determine whether the face image is a fake image based on the third similarity score and a preset threshold.

Figure 4 is a diagram for explaining a method of extracting a face area and a skin area.

Referring to FIG. 4, the counterfeit face determination device 200 extracts a face area 411 from an image 410 containing a human face and extracts a skin area 420 from the extracted face area 411. You can.

The fake face recognition device 200 determines the human face in the image 410 based on color, location, and body structure characteristics, and sets the size of the face area 411 based on the size of the determined face. It can be extracted. In addition, the counterfeit face recognition device 200 immediately detects a human face in the image 410 using OpenCV (Open-source Computer Vision library) and divides the detected face area 411 and the facial skin area 420 into You can also differentiate.

The counterfeit face recognition device 200 can extract the face area 411 from the image 410 containing a human face using a deep learning-based technique, and the skin area 420 included in the face area 411 can be extracted using a deep learning-based technique. It can be extracted using learning-based techniques. Below, an embodiment using a deep learning-based technique is disclosed.

The counterfeit face determination device 200 may include an artificial intelligence model that can learn and distinguish the human shape, human face, human facial skin, human posture, etc. from the image 410. The artificial intelligence model can repeat learning and distinguish between people, human faces, and human skin included in images through the machine learning algorithm set inside the model. Artificial intelligence models may include, but are not limited to, algorithms such as ANN (Artificial Neural Network), CNN (Convolutional Neural Network), RNN (Recurrent Neural Network), DNN (Deep Neural Network), Autoencoder, etc. .

The fake face determination device 200 may extract the skin area 420 included in the face area 411 based on a color threshold technique. Below, an embodiment using a color threshold technique is disclosed.

The acquired image 410 may be an RGB channel image. The fake face determination device 200 converts the face area 411 into the YCbCr color space, and determines that pixels with Y, Cb, and Cr values within a preset threshold range among the converted pixels are skin, thereby creating the skin area 420. The steps to extract can be performed. For example, if the Y value is 235 or less, the Cb value is 133 or more and 173 or less, and the Cr value is 77 or more and 127 or less, the corresponding pixel may be determined to be skin and the skin area 420 may be extracted.

Figure 5 is a diagram to explain a method of detecting and using features found in a photo or display in a face image.

Referring to FIG. 5, the fake face determination device 200 may first extract some features of the acquired face image before calculating a remote photoplethysmographic signal from the face image. If the features extracted from the face image are features that inevitably appear in photos or displays, the fake face determination device 200 can determine the fake face.

The fake face determination device 200 may perform a preprocessing step to determine a fake face in advance by comparing the features of the extracted face image and the features of the photo or display, such as discontinuity.

The fake face determination device 200 may include an artificial intelligence model that can learn and determine the location and arrangement of points and lines where discontinuities appear in images taken of photos containing faces. The artificial intelligence model can repeat learning and extract discontinuous features included in the image through the machine learning algorithm set inside the model.

Additionally, the fake face determination device 200 may include an artificial intelligence model that can learn and determine the location and arrangement of points and lines where discontinuities appear in an image taken of a display containing a face. The artificial intelligence model can repeat learning and extract discontinuous features included in the image through the machine learning algorithm set inside the model.

Meanwhile, the counterfeit face determination device 200 can estimate the heart rate per minute using the calculated remote photoplethysmographic signal after calculating the remote photoplethysmographic signal and before extracting the features of the remote photoplethysmographic signal. . Specific embodiments of this will be described in detail in the description of FIG. 6 below.

FIG. 6 is a flowchart illustrating an example of performing a discrimination process using the characteristics of a display and a photo and performing a discrimination process using BPM estimation before extracting the characteristics of the remote photoplethysmographic signal.

Referring to FIG. 6, in step 610, the fake face determination device 200 may determine whether a photo or display feature is detected in the acquired face image. When the above feature is detected, the fake face determination device 200 can determine that the acquired face image is a fake image.

In step 620, if the feature is not detected, the fake face determination device 200 may calculate a remote photoplethysmographic signal using the face image.

The fake face determination device 200 may extract a face area from a face image and extract a skin area included in the face area. The counterfeit face determination device 200 can calculate a remote photoplethysmographic signal for the skin area. At this time, the fake face determination device 200 can calculate a remote photoplethysmographic signal using the extracted skin area and the OMIT (Orthogonal Matrix Image Transformation) method.

According to one embodiment using the OMIT method, the counterfeit face recognition device 200 extracts RGB components from the skin area, generates an RGB matrix corresponding to the RGB components, QR decomposes the RGB matrix, and provides a QR decomposed matrix and A remote photoplethysmographic signal can be calculated using the least squares method.

Alternatively, the fake face determination device 200 may use a conventional method when calculating a remote photoplethysmographic signal for the skin area. For example, Republic of Korea Patent No. 10-2225557 discloses a technology for extracting remote photoplethysmographic signals from facial images.

In step 630, the fake face recognition device 200 may estimate the heart rate per minute using the calculated remote photoplethysmographic signal.

The counterfeit face recognition device 200 generates a converted signal by processing the calculated remote photoplethysmographic signal with Fourier Transform, and finds the frequency with the largest value in the converted signal, thereby The frequency with the largest magnitude can be estimated as heart beats per minute.

In step 640, the fake face determination device 200 may determine whether the estimated heart rate per minute is outside a preset error range. If it is outside the error range, the counterfeit face determination device 200 can determine that the acquired face image is a counterfeit image.

Meanwhile, the error range may be an error range set based on a person's average heart rate per minute. For example, the error range may be 60bpm or more and 200bpm or less, but is not limited thereto.

The error range can be set in various ways depending on age, etc. The fake face determination device 200 uses an artificial intelligence model learned using an image containing a face as learning data to estimate the age range of a subject recognized in the image, and can use an error range set differently depending on the estimated age range. .

In step 650, if the error range is not exceeded, the fake face determination device 200 may extract features of the calculated remote photoplethysmographic signal. The method of extracting features of the remote photoplethysmographic signal generated by the fake face determination device 200 will be described in detail in the description of FIG. 7 below.

In step 660, the fake face determination device 200 determines whether the face image is a fake image by comparing the characteristics of the fake image and the characteristics of the remote photoplethysmographic signal using a previously learned first artificial intelligence model. can do.

Figure 7 is a diagram for explaining a method of extracting features using a remote photoplethysmographic signal and a frequency domain extracted from the remote photoplethysmographic signal.

Referring to FIG. 7, the calculated remote photoplethysmographic signal 710 is calculated as a signal representing the time component. The fake face determination device 200 processes the remote photoplethysmographic signal 710 calculated from the acquired face image by Fourier transform to obtain the frequency domain 720 of the signal representing the frequency component.

The counterfeit face recognition device 200 can extract features in the time domain by inputting the remote photoplethysmographic signal 710 representing the time component to the time domain feature extractor 730. Additionally, the counterfeit face recognition device 200 may input the frequency domain 720 of the remote photoplethysmographic signal 710, which represents the frequency component, to the frequency domain feature extractor 740 to extract features of the frequency domain.

The time domain feature extractor 730 and the frequency domain feature extractor 740 may be included in the first artificial intelligence model learned using the image including the human face described in FIG. 3 as learning data. In addition, the time-domain feature extraction unit 730 and the frequency-domain feature extraction unit 740 extract the time-domain photoplethysmographic signal 710 and the frequency domain of the photoplethysmographic signal 720 calculated from an image containing a human face. It may include an artificial intelligence model learned with learning data. The artificial intelligence model according to one embodiment may include Support Vector Machine (SVM) and Deep Neural Network (DNN).

The time-domain feature extraction unit 730 and the frequency-domain feature extraction unit 740 may be part of the counterfeit face determination device 200, or may exist outside the counterfeit face recognition device 200 and may be used as a counterfeit face recognition device. Signals 710 and 720 and extracted features can be exchanged through communication with 200 through a network.

The counterfeit face recognition device 200 extracts time-domain and frequency-domain features of the signal 710 calculated through the principal component analysis (PCA) algorithm of the time-domain feature extractor 730 or the frequency-domain feature extractor 740. And, by comparing it with the time domain and frequency domain features of the learning data, it can be determined whether the face image is a fake image. The principal component analysis algorithm is an algorithm that obtains the eigenvectors of the covariance matrix in which the characteristics of the input signal are most prominently expressed, and expresses the high-dimensional vector as a low-dimensional vector.

The counterfeit face recognition device 200 may obtain the final feature 750 by combining the time domain features and frequency domain features of the remote photoplethysmographic signal 710. For example, the fake face determination device 200 may obtain the final feature 750 by concatenating the two vectors into one for time domain features and frequency domain features expressed as vectors. The fake face recognition device 200 compares the obtained final features 750 with the final features combining the time domain features and frequency domain features of the learning data including the fake image, so that the calculated similarity score is set to a preset threshold. It is possible to determine whether the value is greater than the value and determine whether the face image is a forged image.

Alternatively, the fake face determination device 200 may compare the time domain features of the learning data and the features of the remote photoplethysmography signal 710 to calculate a first similarity score in the time domain of the images. Additionally, the fake face determination device 200 may compare the frequency domain features of the learning data and the features in the frequency domain 720 of the remote photoplethysmography signal to calculate a second similarity score in the frequency domain of the images.

The fake face determination device 200 may determine whether the face image is a fake image based on the first similarity score, the second similarity score, and a preset threshold. For example, the fake face determination device 200 may determine whether the face image is a fake image if the sum of the first similarity score and the second similarity score exceeds a preset threshold.

Figure 8 is a graph to explain the distribution of main component values in the frequency domain and time domain calculated through principal component analysis.

Referring to FIG. 8, following FIG. 7, the fake face determination device 200 inputs the signal calculated from the acquired face image and the signal calculated from the learning data to the feature extraction units 730 and 740 to obtain a frequency principal component analysis result. (810) and time principal component analysis results (820) can be obtained. Additionally, the counterfeit face recognition device 200 may obtain a final principal component analysis result 830 by combining the obtained analysis results.

Figure 9 is a graph showing the probability distribution of fake face discrimination for learning data of an artificial intelligence model that determines fake faces using the characteristics of a remote photoplethysmographic signal according to an embodiment.

Referring to Figure 9, it can be seen that the horizontal axis represents the probability that a sample is a real face, and the vertical axis represents the number of samples with that probability value.

The probability distribution 910 of a fake face sample according to an embodiment is the probability that the artificial intelligence model used by the fake face determination device 200 determines a specific fake face sample to be a real face and the number of fake face samples with such probability. represents.

The probability distribution 920 of a real face sample according to an embodiment is the probability that the artificial intelligence model used by the counterfeit face determination device 200 determines a specific real face sample to be a real face and the number of real face samples with such probability. represents.

Meanwhile, unless the order of the steps constituting the method according to the present disclosure is clearly stated or there is no description to the contrary, the steps may be performed in an appropriate order. The present disclosure is not necessarily limited by the order of description of the steps above. The use of any examples or illustrative terms (e.g., etc.) in the present disclosure is merely to describe the present disclosure in detail, and unless limited by the claims, the scope of the present disclosure is limited by the examples or illustrative terms. It doesn't work. Additionally, those skilled in the art will recognize that various modifications, combinations and changes may be made depending on design conditions and factors within the scope of the appended claims or their equivalents.

Therefore, the spirit of the present disclosure should not be limited to the above-described embodiments, and the scope of the patent claims described below as well as all scopes equivalent to or equivalently changed from the claims are within the scope of the spirit of the present disclosure. It will be said to belong to

Claims (12)

Obtaining a face image;
Inputting the face image into a first artificial intelligence model and first determining whether the face image is a forged image based on the output of the first artificial intelligence model;
If the result of the first determination does not indicate that the facial image is fake, calculating a remote photoplethysmography signal (remote PPG signal) using the facial image;
extracting features in the time domain and features in the frequency domain of the remote photoplethysmographic signal;
Obtaining final features by combining features in the time domain and features in the frequency domain; and
Inputting the final features into a second artificial intelligence model and making a final determination as to whether the face image is a forged image based on the output of the second artificial intelligence model;
Including,
The first artificial intelligence model takes an image as an input, outputs a fake face determination result, and includes at least one of an image taken from a photo containing a face and an image taken from a display containing a face. The image is set as first learning data, and each of the first learning data includes a model learned to determine whether or not the first learning data is a forged image as a photo or display image taken,
The second artificial intelligence model takes as input the final features combining the features in the time domain and the features in the frequency domain of the remote photoplethysmographic signal, and outputs the result of determining a fake face, including a human face. The final features combining the features in the time domain and the features in the frequency domain of the remote photoplethysmographic signal calculated from the image are used as second learning data, and an image including the face of the person is created using the second learning data. A method for determining fake faces, including a model trained to determine whether each image is fake. delete delete According to claim 1,
The step of calculating the remote photoplethysmographic signal is,
extracting a facial area included in the facial image;
extracting a skin area included in the face area; and
calculating the remote photoplethysmographic signal based on the skin area;
Method, including. According to claim 4,
The method wherein the extracted face area and the extracted skin area are areas extracted using at least one of deep learning and color threshold-based techniques. According to claim 4,
The step of calculating the remote photoplethysmographic signal is,
Extracting RGB components from the skin area;
generating an RGB matrix corresponding to the RGB components and QR decomposing the RGB matrix; and
Calculating a remote photoplethysmographic signal using the QR decomposed matrix and the least squares method;
Method, including. According to claim 1,
The step of extracting features in the time domain and features in the frequency domain includes:
estimating heart beats per minute using the remote photoplethysmographic signal;
Second determining whether the face image is the forged image based on the heart rate per minute and a preset error range; and
If the result of the secondary determination does not indicate that the facial image is fake, extracting features in the time domain and features in the frequency domain of the remote photoplethysmography signal;
Method, including. According to claim 7,
The step of estimating the heart rate per minute is,
Processing the remote photoplethysmographic signal through Fourier transform to generate a transformed signal; and
Based on the converted signal, estimating the frequency with the largest magnitude in the remote photoplethysmographic signal as the heart beats per minute;
Method, including. delete delete a memory in which at least one program is stored; and
a processor that operates by executing the at least one program;
Including,
The processor,
Obtain a face image,
Input the face image into a first artificial intelligence model, and first determine whether the face image is a forged image based on the output of the first artificial intelligence model,
If the result of the first judgment does not indicate that the face image is a fake, calculating a remote photoplethysmography signal (remote PPG signal) using the face image,
Extracting features in the time domain and features in the frequency domain of the remote photoplethysmographic signal,
Obtaining final features by combining the features in the time domain and the features in the frequency domain,
Input the final features into a second artificial intelligence model, and finally determine whether the face image is a forged image based on the output of the second artificial intelligence model,
The first artificial intelligence model takes an image as an input, outputs a fake face determination result, and includes at least one of an image taken from a photo containing a face and an image taken from a display containing a face. The image is set as first learning data, and each of the first learning data includes a model learned to determine whether or not the first learning data is a forged image as a photo or display image taken,
The second artificial intelligence model takes as input the final features combining the features in the time domain and the features in the frequency domain of the remote photoplethysmographic signal, and outputs the result of determining a fake face, including a human face. The final features combining the features in the time domain and the features in the frequency domain of the remote photoplethysmographic signal calculated from the image are used as second learning data, and an image including the face of the person is created using the second learning data. A counterfeit face recognition device, comprising a model learned to determine whether each is a counterfeit image. A computer-readable recording medium recording a program for executing the method of claim 1 on a computer.