KR20220150868A - Method of motion vector and feature vector based fake face detection and apparatus for the same - Google Patents

Abstract

The present invention relates to fake face detection. According to one aspect of the present invention, a method for detecting a fake face includes the steps of: extracting a motion vector for an image obtained from an image acquisition device; extracting a feature vector for the obtained image; and performing fake face classification based on the motion vector and fake face classification based on the feature vector to determine whether the fake face is a fake face.

Description

Method and apparatus for detecting fake face based on motion vector and feature vector

The present invention relates to detecting a fake face, and more particularly, to a method, apparatus, and software for detecting a fake face based on a motion vector and a feature vector, and a recording medium storing such software.

Demand for face recognition technology is increasing for various purposes such as identification and access control. For accurate face recognition, it is required to detect whether an image acquired through a camera is a fake face. For example, whether the image acquired through the camera is a real face or a fake subject (for example, whether a face photo is taken or a display on which a face image is played) is determined. Discrimination skills are required.

Conventional counterfeit face detection technology may determine whether or not a fake face is faked based on whether heat of a real person's face is detected using equipment such as an infrared camera or a thermal infrared camera. However, since an expensive infrared camera is separately provided for this purpose, it is difficult to apply it to a general camera.

On the other hand, according to the fake face detection technology using a commonly used RGB camera, limited motion around the eyes or mouth is detected, or motion of the background and face object is detected, and a weight is applied to the detected result to digitize it, This can be used to detect forgery. However, in the technology that simply uses the movement of the eyes or mouth, if a high-resolution color printer is used to precisely print a specific part such as the eye or mouth, or if a method of modeling according to the movement of the actual eye or mouth is applied, It may not be possible to determine whether it is counterfeit. In addition, when replaying a high-resolution image taken in the same or similar background as the environment in which the face recognition device is installed, the conventional face recognition system may not be able to determine whether it is forged or not. In this way, since the conventional fake face discrimination technology is based on data on facial features such as eyes and mouth, it cannot reflect the difference between the characteristics of the real face and the fake face for each person, and thus the accuracy of fake face discrimination There is a problem with low

A technical problem of the present invention is to provide a method and apparatus for remarkably increasing the probability of detecting a fake face by combining a motion vector and a feature vector.

The technical problems to be achieved in the present invention are not limited to the technical problems mentioned above, and other technical problems not mentioned will be clearly understood by those skilled in the art from the description below. You will be able to.

According to one aspect of the present invention, a method for detecting a fake face includes extracting a motion vector for an image obtained from an image acquisition device; extracting a feature vector for the obtained image; and performing fake face classification based on the motion vector and fake face classification based on the feature vector to determine whether or not the fake face is a fake face.

According to another aspect of the present invention, an apparatus for detecting a fake face includes a motion vector extractor extracting a motion vector for an image obtained from an image acquisition apparatus; a feature vector extractor extracting a feature vector for the obtained image; a fake face classifier based on the motion vector; and a fake face classifier based on the feature vector. Whether or not a fake face is determined may be determined based on a classification result of the fake face classifier based on the motion vector and a classification result of the fake face classifier based on the feature vector.

According to another aspect of the present invention, a computer-readable medium storing software having instructions executable by a fake face detection device may be provided. The executable instructions may cause the fake face detection device to extract a motion vector for an image obtained from an image acquisition device, extract a feature vector for the obtained image, and classify a fake face based on the motion vector. and, by performing fake face classification based on the feature vector, it is possible to determine whether or not the face is fake.

In various aspects of the present invention, when the fake face is classified as a fake face in one or more of the motion vector-based fake face classification and the feature vector-based fake face classification, it may be finally determined to be a fake face.

In various aspects of the present invention, when the fake face is classified as a real face in both the fake face classification based on the motion vector and the fake face classification based on the feature vector, individual fake face classification may be additionally performed.

In various aspects of the present invention, the fake face classification for each individual may be performed by a classifier trained using one or more of real face data for each individual, real fake face data for each individual, or converted fake face data for each individual.

In various aspects of the present invention, the real fake face data for each individual may be a result of taking a photo or video of each individual's real face.

In various aspects of the present invention, the transformed fake face data is obtained by applying a transformation matrix to the real fake face data, and the transformation matrix may be adaptively updated.

In various aspects of the present invention, the step of detecting a facial landmark from the acquired image may be further included, and the feature vector may be descriptor information representing a geometric feature of the detected facial landmark. have.

In various aspects of the present invention, the method may further include detecting a facial landmark from the acquired image, and the motion vector may be information indicating the size and direction of motion of the detected facial landmark.

In various aspects of the present invention, the fake face classification based on the motion vector determines whether the obtained image is fake based on a motion vector feature model for a real face and a motion vector feature model for the fake face. steps may be included.

In various aspects of the present invention, the motion vector feature model for the fake face may include a motion vector feature model for the fake face using a photo and a motion vector feature model for the fake face using a video.

In various aspects of the present invention, the fake face classification based on the motion vector is based on a motion pattern observed between two consecutive frames of the obtained image for each of the first area, the second area, and the third area. The method may include calculating, wherein the third area includes the second area, and the second area includes the first area.

In various aspects of the present invention, the motion vector may be determined by calculating an average vector in each of the first, second and third regions, and combining the average vectors of the first, second and third regions. can

In various aspects of the present invention, the first region may be a face region, the second region may be a head region, and the third region may be a background region.

In various aspects of the present invention, the fake face classification based on the feature vector is performed on a classifier learned using one or more of one or more real face data, one or more real fake face data, or one or more converted fake face data. can be performed by

In various aspects of the present invention, each of the one or more real fake face data may correspond to each of a result of photographing a photo or video of the one or more real faces.

In various aspects of the present invention, each of the one or more transformed fake face data is obtained by applying a transformation matrix to each of the one or more real fake face data, and the transformation matrix may be adaptively updated. .

The features briefly summarized above with respect to the present invention are only exemplary aspects of the detailed description of the present invention that follows, and do not limit the scope of the present invention.

According to the present invention, a method and apparatus for remarkably increasing the probability of detecting a fake face by combining a motion vector and a feature vector may be provided.

The effects obtainable in the present invention are not limited to the effects mentioned above, and other effects not mentioned can be clearly understood by those skilled in the art from the description below. will be.

The drawings accompanying this specification are intended to provide an understanding of the present invention, show various embodiments of the present invention, and explain the principles of the present invention together with the description of the specification.
1 is a flowchart illustrating a fake face detection method based on facial feature points.
2 is a block diagram for explaining fake face detection using changes in facial feature points and background.
3 is a diagram for explaining the configuration and operation of the fake face detection device according to the present invention.
4 is a diagram for explaining a motion vector extraction area according to the present invention.
5 is a diagram for explaining the configuration and operation of an apparatus for extracting a motion vector according to the present invention.
6 to 8 are diagrams for explaining examples of motion vectors obtained according to the present invention.
9 is a diagram for explaining the configuration and operation of a motion vector fake face classifier according to the present invention.
10 is a diagram for explaining a method of discriminating a fake face using a motion vector fake face classifier according to the present invention.
11 is a diagram for explaining the configuration and operation of a general fake face classifier and an individual fake face classifier according to the present invention.
12 is a diagram for explaining a transformation matrix usable in the fake facial feature point data generation unit according to the present invention.
13 is a diagram for explaining the generation of a general fake face classifier.
14 is a diagram for explaining creation of a fake face classifier for each individual.

Hereinafter, with reference to the accompanying drawings, embodiments of the present invention will be described in detail so that those skilled in the art can easily carry out the present invention. However, the present invention may be embodied in many different forms and is not limited to the embodiments described herein. And, in order to clearly explain the present invention in the drawings, parts irrelevant to the description are omitted, and similar reference numerals are attached to similar parts throughout the specification.

1 is a flowchart illustrating a fake face detection method based on facial feature points.

In step S110, a photographed image may be acquired using a camera or the like. A facial object (eg, an eye) may be detected from the image acquired in step S120. In step S130, face image normalization may be performed to compare the amount of change. In step S140, a region (eg, an eye region) corresponding to a feature point, which is a main target of variation measurement, may be binarized and stored. In step S150, it is determined whether steps S110 to S140 are repeated N times, and if not, the steps are performed again from step S110. When an image repeated N times is obtained, step S160 may be performed.

In step S160, the amount of change in feature point areas in the N face images may be calculated based on the N normalized face images and the binarized data of feature point areas (eg, eye areas) extracted from the respective face images. If eye blinking does not occur in a fake face such as a photo, the amount of change in the feature point area may be very small compared to the real face image, and based on this, it is possible to determine whether the face is fake.

2 is a block diagram for explaining fake face detection using changes in facial feature points and background.

The image acquisition block 210 may obtain an image captured by a camera and input the obtained image to the face extraction block 220 and the background learning block 230 .

The face extraction block 220 may extract a face object from the acquired image and transmit the face image to the motion information comparison block 260 and the background comparison block 270 .

The background learning block 230 generates a plurality of models through background learning that analyzes and models the current background among a plurality of images acquired from the image acquisition block 210, and among them, the background of the model having a relatively high weight It can be selected as a learning background (or reference background) and stored in the storage block 250. That is, the background learning block 230 may update the background model having the highest probability through continuous learning.

The blink detection block 240 may detect whether the eyes blink or not through the amount of change in the facial feature point region (eg, eye region) extracted by the face extraction block 220 . For example, the method of FIG. 1 may be performed in the blink detection block 240 . The detection result of the blink detection block 240 may be transmitted to the counterfeit determination block 280 .

In the motion information comparison block 260, a motion vector component of the face area and a motion vector component of the current background may be extracted and compared. A numerical value representing a motion determined based on the motion vector of the face object and the vector of the background area may be transmitted to the fake determination block 280 .

The comparison block 270 between backgrounds compares the background of the current image received from the face extraction block 220 with the learning background stored in the storage block 250 to determine the similarity, and the result is fake determination block 280 can be forwarded to

In the forgery determination block 280, it is determined whether or not the forgery is faked based on whether there is blinking of the eyes of the face object in the acquired image, whether the background of the acquired image is similar to the actual background model, whether the motion of the background of the acquired image is abnormal, and the like. can

3 is a diagram for explaining the configuration and operation of the fake face detection device according to the present invention.

The image acquisition block 301 may obtain an image captured by an image acquisition device (eg, a camera) and may transmit the image to the facial landmark detection unit 302 .

The facial landmark detection unit 302 may detect major facial feature points (eg, eyebrows, eyes, nose, mouth, etc.). For example, one eyebrow may be divided into two feature parts and detected, and a total of four feature parts of the left and right eyebrows may be detected. One eye can be divided into 4 features and detected, and a total of 8 features can be detected for the left and right eyes. The nose can be detected by dividing the two parts on both sides. Lips can detect a total of 4 features, 2 upper left and right parts and 2 lower left and right parts. This is just an example, and the scope of the present invention is not limited to extracting 18 feature points from the face.

The facial feature point extraction unit 303 extracts features for facial landmarks (eg, 18 positions of eyebrows, eyes, nose, and mouth) and expresses them (eg, MsLBP (Multi-scale Local Binary Pattern)) can be created. This is just an example, and other descriptors other than MsLBP may be used. However, it is effective to use a descriptor that is robust to light changes in a local area and expresses geometric features well for facial feature point extraction. This is because when an image of a real face is compared with a fake face image of a photo or video, geometric information such as edge information disappears in the fake face image of flat data, so it is difficult to detect fake faces. because it is more advantageous.

The motion vector extractor 304 may extract a motion vector of each frame from a video (or a sequence composed of a plurality of frames) and calculate the magnitude and angle of the motion vector (here, the motion vector The magnitude of means the length of the vector, and the direction of the motion vector means the gradient value between 0˚ and 360˚). In particular, motion vectors can be extracted for each of the face area, head area, and background area. A detailed description of the operation of the motion vector extractor 304 will be described later with reference to FIGS. 4 to 8 .

The motion vector fake face classifier 306 may determine whether an arbitrary face image corresponds to a fake face based on the motion vector. A detailed description of the operation of the motion vector fake face classifier 306 will be described later with reference to FIGS. 9 and 10 .

The general fake face classifier 305 is distinguished from the individual fake face classifier 308 described later. A general fake face classifier 305 determines whether or not a face is fake based on facial feature points (or feature vectors), and the feature points (or feature vectors) of the face image input from the facial feature point extractor 303 are learned in advance. Comparing with a prepared criterion, it may be determined whether an arbitrary face image corresponds to a fake face. A detailed description of the operation of the general fake face classifier 305 will be described later with reference to FIGS. 11 to 14 .

If at least one of the general fake face classifier 305 and the motion vector fake face classifier 306 classifies the face as a fake face, it may be finally determined that the face is a fake face. If both of the general fake face classifier 305 and the motion vector fake face classifier 306 classify that the face is not a fake face, additional fake face detection is performed through the face recognition unit 307 and the individual fake face classifier 308. It can be.

The face recognition performer 307 may determine who the person corresponding to the face of the acquired image is. For example, identity or identification information (ie, ID information) of a person matched with the face may be extracted based on the feature of the corresponding face image. The face recognition result may be delivered to the fake face classifier 308 for each individual.

The fake face classifier 308 for each individual compares facial feature points of the face corresponding to the identified identity with standard facial feature points stored in the database in advance and facial feature points of the obtained image to determine whether the face is faked. A detailed description of the operation of the fake face classifier 308 for each individual will be described later with reference to FIGS. 11 to 14 .

One or more of the components described with respect to FIG. 3 may be implemented as a separate processor or may be implemented as a function module in the form of software or firmware of one processor.

4 is a diagram for explaining a motion vector extraction area according to the present invention.

In order to extract a motion vector for an acquired image, it can be divided into a face area, a head area (or a head and shoulder area), and a background area (or an area outside the camera frame). In the case of an image of a real face, there is movement in the face area and the head area according to the motion of the human body, but such motion hardly occurs in the background. In addition, in the face area, a lot of minute movements occur according to eye blinking, speaking, facial expression change, etc., but relatively little such minute movements occur in the head area.

In the present invention, the terms face region, head region, and background region are used for convenience of explanation, but the scope of the present invention is not limited to the dictionary meaning of the terms. That is, the present invention provides an example of dividing the obtained face image into first, second, and third areas, the third area including the second area, and the second area including the first area. include

5 is a diagram for explaining the configuration and operation of an apparatus for extracting a motion vector according to the present invention.

The image acquisition block 501 may correspond to the image acquisition block 301 of FIG. 3 .

The two consecutive frame extractor 502 extracts two consecutive frames from among the consecutive frames input by the image acquisition block 501 . The two consecutive frame extractor 502 may extract two consecutive frames of an input image after a facial object is detected in facial landmark detection (302 in FIG. 3 ).

The dense optical flow calculation unit 503 may calculate a dense optical flow between the two extracted frames (ie, a movement pattern observed for an object, surface, or edge).

The histogram generating unit 504 may further subdivide the three regions divided into a face region, a head region, and a background region as shown in the example of FIG. 4 . For example, the face region may be divided into N1 sub-blocks, the head region may be divided into N2 sub-blocks, and the background region may be divided into N3 sub-blocks. An average vector may be calculated in each of the divided regions (eg, a face region, a head region, and a background region), and a magnitude and an angle of the average vector in the corresponding region may be calculated. Using the result, a histogram (ie, a one-dimensional histogram vector) for each of the divided regions may be generated.

The vector combiner 505 generates one motion vector (or motion feature vector) by combining 1-dimensional histogram vectors derived from each of the divided regions (eg, the face region, the head region, and the background region). can

One or more of the components described with respect to FIG. 5 may be implemented as a separate processor or may be implemented as a function module in the form of software or firmware of one processor.

6 to 8 are diagrams for explaining examples of motion vectors obtained according to the present invention.

The upper two figures in FIG. 6 show examples of motion vectors (vectors marked in green) obtained from an image of a real face, and the lower figure shows a corresponding histogram.

The upper two diagrams in FIG. 7 show examples of motion vectors (vectors marked in green) obtained from an image in which a photo (eg, a face photo taken with a smartphone) is taken, and the lower diagram is a corresponding histogram. indicates

The upper two figures in FIG. 8 show examples of motion vectors (vectors marked in green) obtained from an image taken of a video (eg, a face video taken with a smartphone), and the lower figure shows a corresponding histogram. indicates

For example, if the direction of one motion vector is 70 degrees and its magnitude is 20, the count of the corresponding element in Table 1 below can be added by 1. For the remaining motion vectors, the count according to the direction and magnitude may be repeated. The sum of vector magnitudes in a specific direction is obtained by multiplying the first magnitude value by the number of vectors having the first magnitude in the corresponding direction, by multiplying the second magnitude value by the number of vectors having the second magnitude, and ... all of these It can be derived by summing.

direction
size 0 … 70˚ … 360˚ … … … … … … 20 0 … One … 0 … … … … … … MAX 0 … 0 … 0

Accordingly, since the sum of the magnitudes of vectors in a specific direction may be relatively large, it can be easily compared with the motion vectors of other images by determining the minimum and maximum values and applying scaling when expressed as a histogram graph. That is, when comparing histogram graphs of motion vectors of different images, the degree of similarity is determined by looking at the size distribution in the direction of the vector rather than comparing the size of the graph, so a relative size relationship of vector sizes is not necessary. to be.

From the histogram graph of the example of FIG. 6, it can be seen that in the case of a real face, a relatively large number of motion vector components in the left and right (or horizontal) directions occur, and from the histogram graph of the example of FIG. 7, in the case of a fake face (photo), It can be seen that since the person holding the picture shakes, the occurrence of a motionless situation is low, and motion vector components having all directions (or vertical and horizontal) are frequently generated. In the case of the fake face (video) from the histogram graph of the example of FIG. 8, the size of the motion vector is relatively small compared to other cases due to the limitation of the size of the video display that can be carried by a person, and the shape of the graph itself is similar to that of the real face. It may show a pattern, but it can be seen that the size of the vector is smaller or the pattern is relatively distorted in one direction. The graphs of the histograms of FIGS. 6 to 8 are only examples expressing average values obtained through many experiments, and do not limit the scope of the present invention.

In this way, as can be seen from the graph expressing the histogram of the size and direction based on the motion vectors obtained from the real face, fake face (photo), and fake face (video), the motion vector obtained from the real face is It shows a big difference from the motion vector obtained from the video. That is, since the feature of the normalized histogram of the real face and the motion vector of the photo or video has a significant difference, it is possible to determine whether the face is fake based on this.

9 is a diagram for explaining the configuration and operation of a motion vector fake face classifier according to the present invention.

As described above, the motion vector fake face classifier (306 in FIG. 3) according to the present invention can determine whether or not it is a fake face based on the feature of the motion vector (eg, distribution feature on the histogram) itself, but additionally trained. It also supports determining whether or not a face is fake using an image.

For example, a situation in which each of the three categories of real face, fake face (photo), and fake face (video) is actually detected using features of motion vectors trained in various ways (eg, distribution features on a histogram). In , fake face detection performance can be improved by determining which one of the three categories is close to.

In the example of FIG. 9 , the training image generating unit 901 may collect or store data for each modeling classification. For example, in order to generate a model of a real face (ie, a reference motion vector for comparison with respect to the real face), a training image for recognizing various characteristics of the real face is required. Such training image data may be provided in advance, or image data determined as a real face according to the fake face detection result of the present invention may be provided as a training image in a feedback method. In order to generate a model for a fake face (photo) (that is, a reference motion vector for comparison of a fake face using a photo), a training image for identifying various characteristics of a fake face using a photo is required. Such training image data may be provided in advance, or image data determined as a fake face using a photo according to a result of detecting a fake face according to the present invention may be provided as a training image in a feedback method. In order to create a model for a fake face (video) (that is, a reference motion vector for comparison with respect to a fake face using a video), a training image for identifying various characteristics of a fake face using a video is required. Such training image data may be provided in advance, or image data determined as a fake face using a video according to the fake face detection result of the present invention may be provided as a training image in a feedback method.

The motion vector extractor 902 can extract motion vectors for each of the modeling classifications (for example, three classifications of real face, fake face (photo), and fake face (video)), as shown in FIG. method may be used.

The model generation unit 903 uses a statistical modeling method based on the extracted motion vector, and models for each of the modeling classifications (for example, three classifications of real face, fake face (photo), and fake face (video)). can create

One or more of the components described with respect to FIG. 9 may be implemented as a separate processor or may be implemented as a function module in the form of software or firmware of one processor.

10 is a diagram for explaining a method of discriminating a fake face using a motion vector fake face classifier according to the present invention.

Motion vector extraction 1002 may be performed on the input image 1001 acquired through the image acquisition unit 301, and the method described in FIG. 5 may be applied to this. The extracted motion vectors are generated for each of the pre-generated models (eg, modeling classifications (eg, real face, fake face (photo), and fake face (video) three classifications) as shown in the example of FIG. 9). model) and can be compared (1003). That is, the extracted motion vector may be compared with a model for a real face, a model for a fake face (photo), and a model for a fake face (video). As a result of comparison 1003, a model having the highest similarity among various modeling classifications may be determined. If the motion vector extracted for the input image is most similar to the real face model, it can be determined that it is not a fake face, otherwise (for example, fake face (photo) or fake face (video) model and most similar) may be determined to be a fake face.

11 is a diagram for explaining the configuration and operation of a general fake face classifier and an individual fake face classifier according to the present invention.

The image acquisition block 1101, the facial landmark detection unit 1102, and the facial feature point extraction unit 1103 of FIG. 11 are the image acquisition block 301, the facial landmark detection unit 302, and the facial feature point extraction unit in the example of FIG. (303). That is, an image is obtained using an image acquisition device (eg, a camera), facial feature points (eg, 18 facial landmarks for eyebrows, eyes, nose, and lips) are detected, and the detected feature points are detected. A feature vector can be extracted using MsLBP at the location.

The fake facial feature point data generation unit 1104 converts the extracted facial feature data into fake facial feature data (or fake facial feature) by using a transformation matrix (eg, a transfer function or an adaptation function). vector) can be converted. An example of a transformation matrix will be described with reference to FIG. 12 .

12 is a diagram for explaining a transformation matrix usable in the fake facial feature point data generation unit according to the present invention.

A basis (G _basis ) of the real face data may be determined through principal component analysis (PCA) on a set of real face data (G ₁ , G ₂ , ..., G _N ). Meanwhile, a basis (F _basis ) of fake face data may be determined through PCA for a set of fake face data (F ₁ , F ₂ , ..., F _M ). Based on these, a conversion matrix (X) can be calculated. That is, if the result of applying a specific transformation matrix X to G _basis is called F _basis , transformation matrix X can be obtained as in Equation 1 below.

Referring back to FIG. 11, by using the real fake face generation unit 1105, a result of photographing a real person's face (ie, a photo or video) is taken again by an image acquisition device (eg, a camera) to counterfeit it. A fake face image corresponding to a face sample may be generated. One or a plurality of fake face images may be generated using the fake face generation unit 1105 . When the real fake face generation unit 1105 is applied, since it can be used as useful fake face data not only in personal fake face classifiers but also in general fake face classifiers, fake face detection performance can be improved. In particular, the application of the real fake face generation unit 1105 can be applied at the time of generating and registering a personal identification image for a fake face classifier for each individual. However, if it is not appropriate to register a personal identification image in advance, the real fake face generation unit 1105 uses the above-described conversion function to convert fake face data (ie, real face feature vectors) to fake face data (ie, real face feature vectors). , fake facial feature vectors) and use them.

The general fake face classifier generation unit 1106 can improve the performance of the general fake face classifier by learning or updating the general fake face classifier.

In the individual fake face classifier generation unit 1107, performance of the individual fake face classifier may be improved by learning or updating the individual fake face classifier.

Examples of a general fake face classifier generator 1106 and an individual fake face classifier generator 1107 will be described with reference to FIGS. 13 and 14, respectively.

13 is a diagram for explaining the creation of a general fake face classifier.

A general fake face classifier may determine a comparison criterion for a general fake face feature vector using various training data, and determine whether or not a generic face is a fake face. In order to train or update such a general fake face classifier, new data other than the previously obtained training data (eg, data on the real face of the newly registered person(s) and real fake face (i.e., real face) Add data about the result of taking a picture (ie, a picture or video) and re-photographing it with an image acquisition device (eg, a camera), or data about a fake face transformed from a real face using a transformation matrix) can be entered as Accordingly, a criterion for determining general fake faces is learned from the feature vector of the real face set of the existing registrant and the fake face set of the existing registrant (which includes at least one of the real fake face or the transformed fake face). and update the binary classifier.

14 is a diagram for explaining creation of a fake face classifier for each individual.

When an individual is identified through face recognition, the fake face classifier for each individual can learn and update data on each individual's real face and fake face to obtain a more accurate fake face detection result. The fake face classifier for each individual identifies data on the real face of the individual (eg, registrant), the actual fake face (ie, the result of photographing the real face (ie, a photo or video)) to an image acquisition device (eg, camera), or data on a fake face transformed from a real face using a transformation matrix), it learns a criterion for determining whether or not a fake face for the corresponding individual is trained and uses a binary classifier can be updated.

One or more of the components described with reference to FIGS. 11 to 14 may be implemented as a separate processor or as a function module in the form of software or firmware of one processor.

As described above, a fake face classifier that operates based on both a motion vector (eg, high-density optical flow) and a feature vector (eg, MsLBP) is more accurate and has a higher probability than conventional fake face detection methods. A fake face detection result can be provided. In addition, since the fake face data necessary for generating (or learning or updating) the fake face classifier can be obtained from a real fake face or reproduced using a trained transformation matrix, the data necessary for generating the fake face classifier can be easily provided to counteract fake face data. The performance of the face classifier can be further improved.

In addition, by using a combination of motion vectors extracted from the face area, head area, and background area, as well as motion vectors extracted only from the face area, the characteristics of motion vectors derived from real human behavior and holding and shaking photos A fake face can be detected more accurately by using a classifier that learns the characteristics of a motion vector that is induced when a video is used.

Furthermore, paying attention to the difference between the feature vector obtained from a real face and the feature vector obtained when a photo or video is taken, a fake face classifier can be generated by learning the difference between these feature vectors in advance using a support vector machine. have. The fake face classifier learns based on general fake face features and individual fake face features in parallel, so even if general fake face features are not revealed, fake face detection probability based on individual fake face features can increase In this case, when registering personal information to create a fake face classifier for each individual, the real fake face of the individual may be input together with the real face of the individual, or the fake face generated by the transformation matrix from the real face of the individual may be used.

As described above, according to the present invention, a fake face can be accurately detected only with an image of a generally used RGB camera without expensive separate equipment such as an infrared camera or a thermal infrared camera. In particular, the conventional method of detecting a fake face based on eye blinking or lip movement can be neutralized by adding motion to a specific part of a high-quality photo or video. However, according to the present invention, motion vectors and feature vectors are simultaneously considered. In addition, feature vector-based fake face detection can detect a fake face with a higher probability by considering both general features and individual features.

Matters described in various embodiments of the present invention described above may be applied independently or two or more embodiments may be simultaneously applied.

Exemplary methods described in various embodiments of the present invention described above are expressed as a series of operations for simplicity of explanation, but this is not intended to limit the order in which steps are performed, and if necessary, each step is performed simultaneously or They may be performed in a different order. In addition, not all steps illustrated in order to implement the method proposed in the present invention are necessarily required.

The scope of the present invention includes a device that processes or implements an operation according to the method proposed in the present invention.

The scope of the present invention is software (or operating system, application, firmware, program, etc.) that allows the operation according to the method proposed in the present invention to be executed on a device or computer, and such software is stored on a device or computer. Includes executable medium.

301 image acquisition block 302 facial landmark detection unit
303 Facial feature point detection unit 304 Motion vector extraction unit
305 generic fake face classifier 306 motion vector fake face classifier
307 Face recognition unit 308 Individual counterfeit face classifier

Claims (14)

A method for detecting a fake face,
Detecting facial landmarks for an image obtained from an image acquisition device;
extracting a motion vector for the obtained image, wherein the motion vector is information including magnitude and direction of motion of the detected face landmark;
extracting a feature vector for the obtained image, wherein the feature vector is information including a descriptor representing a geometric feature of the detected face landmark; and
Performing fake face classification based on the motion vector and general fake face classification based on the feature vector to determine whether or not a fake face is present,
When the fake face is classified as a fake face in at least one of the fake face classification based on the motion vector and the general fake face classification based on the feature vector, finally determining that the face is a fake face;
The fake face detection method of claim 1 , further performing fake face classification for each individual when the fake face is classified as a real face in both the fake face classification based on the motion vector and the general fake face classification based on the feature vector. According to claim 1,
Wherein the fake face classification for each individual is performed by a classifier learned using at least one of the real face data for each individual, the real fake face data for each individual, and the converted fake face data for each individual. According to claim 2,
The fake face detection method of claim 1 , wherein the real fake face data for each individual is a result of taking a photo or video of a real face of each individual. According to claim 2,
The transformed fake face data is obtained by applying a transformation matrix to the real fake face data,
Wherein the transformation matrix is adaptively updated. According to claim 1,
Fake face classification based on the motion vector,
and determining whether the obtained image is fake based on a motion vector feature model of a real face and a motion vector feature model of a fake face. According to claim 5,
The motion vector feature model for the fake face is
A motion vector feature model for a fake face using a photo;
A fake face detection method comprising a motion vector feature model for a fake face using video. According to claim 1,
Fake face classification based on the motion vector,
Calculating a movement pattern observed between two consecutive frames of the obtained image for each of the first region, the second region, and the third region;
The fake face detection method of claim 1 , wherein the third area includes the second area, and the second area includes the first area. According to claim 7,
The motion vector is
Calculate an average vector in each of the first, second and third regions;
The fake face detection method is determined by combining the mean vectors of the first, second and third regions. According to claim 7,
wherein the first area is a face area, the second area is a head area, and the third area is a background area. According to claim 1,
General fake face classification based on the feature vector,
A fake face detection method, performed by a classifier trained using one or more of one or more real face data, one or more real fake face data, or one or more converted fake face data. According to claim 10,
Each of the one or more real fake face data corresponds to each of a result of photographing a photo or video of the one or more real faces. According to claim 11,
Each of the one or more transformed fake face data is obtained by applying a transformation matrix to each of the one or more real fake face data,
Wherein the transformation matrix is adaptively updated. In the device for detecting a fake face,
a facial landmark detection unit for detecting facial landmarks for an image obtained from an image acquisition device;
a motion vector extractor extracting a motion vector of the acquired image, wherein the motion vector is information including magnitude and direction of motion of the detected face landmark;
a feature vector extraction unit extracting a feature vector of the obtained image, wherein the feature vector is information including a descriptor representing a geometric feature of the detected face landmark;
a fake face classifier based on the motion vector; and
A generic fake face classifier based on the feature vector;
Based on the classification result of the fake face classifier based on the motion vector and the classification result of the fake face classifier based on the feature vector, it is determined whether or not the face is fake.
When the fake face is classified as a fake face in at least one of the fake face classification based on the motion vector and the general fake face classification based on the feature vector, finally determining that the face is a fake face;
The fake face detection device further performs fake face classification for each individual when classified as a real face in both fake face classification based on the motion vector and general fake face classification based on the feature vector. A computer-readable medium storing software having instructions executable by a fake face detection device,
The executable instructions may cause the fake face detection device to detect a face landmark of an acquired image, extract a motion vector including a magnitude and direction of a movement of the detected face landmark, and extract the detected face landmark. After extracting a feature vector including a descriptor representing a geometric feature of a face landmark,
Performing fake face classification based on the motion vector and general fake face classification based on the feature vector to determine whether or not a fake face is present,
When the fake face is classified as a fake face in at least one of the fake face classification based on the motion vector and the general fake face classification based on the feature vector, finally determining that the face is a fake face;
and additionally performing fake face classification for each individual when classified as a real face in both fake face classification based on the motion vector and normal fake face classification based on the feature vector.

Images