KR102596897B1 - Method of motion vector and feature vector based fake face detection and apparatus for the same - Google Patents

Abstract

The present invention relates to fake face detection. According to one aspect of the present invention, a method for detecting a fake face includes extracting a motion vector for an image acquired from an image acquisition device; extracting feature vectors for the acquired image; And it may include performing fake face classification based on the motion vector and performing fake face classification based on the feature vector to determine whether the face is fake.

Description

Motion vector and feature vector based fake face detection method and device {METHOD OF MOTION VECTOR AND FEATURE VECTOR BASED FAKE FACE DETECTION AND APPARATUS FOR THE SAME}

The present invention relates to detecting fake faces, and more specifically, to a method, device, and software for detecting fake faces based on motion vectors and feature vectors, and a recording medium storing such software.

Demand for facial recognition technology is increasing for various purposes such as identity verification and access control. For accurate face recognition, it is required to detect whether an image acquired through a camera is a fake face. For example, whether an image acquired through a camera is a photograph of a real face or a fake subject (e.g., a photograph of a face or a display on which a face image is played) Discrimination technology is required.

Conventional counterfeit face detection technology uses equipment such as an infrared camera or thermal infrared camera to determine whether the face is counterfeit based on whether the heat of the actual person's face is detected. However, since an expensive infrared camera is separately provided for this, it is difficult to apply it to a general camera.

Meanwhile, according to the commonly used fake face detection technology using RGB cameras, limited movement around the eyes or mouth is detected, or movement of the background and facial objects is detected, weights are applied to the detected results and quantified, Using this, you can determine whether it is counterfeit or not. However, in technology that simply uses the movements of the eyes or mouth, if a photo with specific parts such as the eyes or mouth is precisely printed using a high-resolution color printer, or modeling according to the actual movements of the eyes or mouth is applied, It may not be possible to determine whether it is counterfeit or not. Additionally, when replaying a high-resolution image taken in the same or similar background as the environment in which the facial recognition device is installed, a conventional facial recognition system may not be able to determine whether it is forgery. As such, since the conventional fake face identification technology is based on data about facial features such as eyes and mouth, it cannot reflect the differences between the features of the real face and the fake face for each person, so the accuracy of fake face identification is low. There is a problem with low

The technical task of the present invention is to provide a method and device that dramatically increases the probability of detecting a fake face by combining motion vectors and feature vectors.

The technical problems to be achieved in the present invention are not limited to the technical problems mentioned above, and other technical problems not mentioned will be clearly understood by those skilled in the art from the description below. You will be able to.

According to one aspect of the present invention, a method for detecting a fake face includes extracting a motion vector for an image acquired from an image acquisition device; extracting feature vectors for the acquired image; And it may include performing fake face classification based on the motion vector and performing fake face classification based on the feature vector to determine whether the face is fake.

According to another aspect of the present invention, an apparatus for detecting a fake face includes: a motion vector extractor for extracting a motion vector for an image acquired from an image acquisition device; a feature vector extraction unit that extracts feature vectors for the acquired image; a fake face classifier based on the motion vector; and a fake face classifier based on the feature vector. A fake face may be determined based on the classification result of the fake face classifier based on the motion vector and the classification result of the fake face classifier based on the feature vector.

According to another aspect of the present invention, a computer-readable medium storing software having instructions executable by a counterfeit face detection device may be provided. The executable instructions cause the fake face detection device to extract a motion vector for an image acquired from an image acquisition device, extract a feature vector for the acquired image, and classify a fake face based on the motion vector. It is possible to perform fake face classification based on the feature vector and determine whether it is a fake face.

In various aspects of the present invention, if a face is classified as a fake in one or more of the fake face classification based on the motion vector or the fake face classification based on the feature vector, it may be finally determined to be a fake face.

In various aspects of the present invention, if a fake face is classified as a real face in both the motion vector-based fake face classification and the feature vector-based fake face classification, individual fake face classification may be additionally performed.

In various aspects of the present invention, the individual fake face classification may be performed by a classifier learned using one or more of individual real face data, individual real fake face data, or converted individual fake face data.

In various aspects of the present invention, the real fake face data for each individual may be the result of taking a photo or video of the real face for each individual.

In various aspects of the present invention, the transformed fake face data is obtained by applying a transformation matrix to the real fake face data, and the transformation matrix can be adaptively updated.

In various aspects of the present invention, the method further includes detecting a facial landmark from the acquired image, wherein the feature vector may be descriptor information expressing geometric features for the detected facial landmark. there is.

In various aspects of the present invention, the method further includes detecting a facial landmark from the acquired image, and the motion vector may be information indicating the magnitude and direction of movement of the detected facial landmark.

In various aspects of the present invention, the motion vector-based fake face classification determines whether the acquired image is fake based on a motion vector feature model for a real face and a motion vector feature model for a fake face. It may include steps.

In various aspects of the present invention, the motion vector feature model for the fake face may include a motion vector feature model for the fake face using a photo and a motion vector feature model for the fake face using a video.

In various aspects of the present invention, the fake face classification based on the motion vector includes, for each of the first region, the second region, and the third region, a motion pattern observed between two consecutive frames of the acquired image. A calculating step may be included, wherein the third area may include the second area, and the second area may include the first area.

In various aspects of the invention, the motion vector may be determined by calculating an average vector in each of the first, second and third regions and combining the average vectors of the first, second and third regions. You can.

In various aspects of the present invention, the first area may be a face area, the second area may be a head area, and the third area may be a background area.

In various aspects of the present invention, the fake face classification based on the feature vector includes a classifier learned using one or more of one or more real face data, one or more real fake face data, or one or more converted fake face data. It can be performed by

In various aspects of the present invention, each of the one or more real fake face data may correspond to each of a result of taking a photo or video of the one or more real faces.

In various aspects of the present invention, each of the one or more transformed fake face data is obtained by applying a transformation matrix to each of the one or more real fake face data, and the transformation matrix can be adaptively updated. .

The features briefly summarized above with respect to the present invention are merely exemplary aspects of the detailed description of the present invention that follows, and do not limit the scope of the present invention.

The present invention can provide a method and device that dramatically increases the detection probability of a fake face by combining a motion vector and a feature vector.

The effects that can be obtained from the present invention are not limited to the effects mentioned above, and other effects not mentioned can be clearly understood by those skilled in the art from the description below. will be.

The drawings attached to this specification are intended to provide an understanding of the present invention, show various embodiments of the present invention, and together with the description of the specification, explain the principles of the present invention.
1 is a flowchart illustrating a method for detecting a fake face based on facial feature points.
Figure 2 is a block diagram to explain fake face detection using changes in facial features and background.
Figure 3 is a diagram for explaining the configuration and operation of a fake face detection device according to the present invention.
Figure 4 is a diagram for explaining a motion vector extraction area according to the present invention.
Figure 5 is a diagram for explaining the configuration and operation of a device for extracting a motion vector according to the present invention.
6 to 8 are diagrams for explaining examples of motion vectors obtained according to the present invention.
Figure 9 is a diagram for explaining the configuration and operation of the motion vector forged face classifier according to the present invention.
Figure 10 is a diagram for explaining a method of identifying a fake face using a motion vector fake face classifier according to the present invention.
Figure 11 is a diagram for explaining the configuration and operation of a general fake face classifier and an individual fake face classifier according to the present invention.
Figure 12 is a diagram for explaining the transformation matrix available in the fake facial feature point data generator according to the present invention.
Figure 13 is a diagram to explain the creation of a general fake face classifier.
Figure 14 is a diagram to explain the creation of an individual fake face classifier.

Below, with reference to the attached drawings, embodiments of the present invention will be described in detail so that those skilled in the art can easily implement the present invention. However, the present invention may be implemented in many different forms and is not limited to the embodiments described herein. In order to clearly explain the present invention in the drawings, parts that are not related to the description are omitted, and similar parts are given similar reference numerals throughout the specification.

1 is a flowchart illustrating a method for detecting a fake face based on facial feature points.

In step S110, an image captured using a camera or the like can be obtained. A facial object (eg, eyes) may be detected from the image acquired in step S120. In step S130, facial image normalization can be performed so that the amount of change can be compared. In step S140, the area (for example, eye area) corresponding to the feature point that is the main target of change measurement may be binarized and stored. In step S150, it is determined whether steps S110 to S140 are repeated N times. If not, the process is performed again from step S110. If an image repeated N times is obtained, step S160 may be performed.

In step S160, the amount of change in the feature point areas in the N face images can be calculated based on the N normalized face images and the binarized data of the feature point regions (for example, eye regions) extracted from each face image. If eye blinking does not occur in a fake face such as a photo, the amount of change in the feature point area may be very small compared to an actual face image, and based on this, it can be determined whether the face is fake.

Figure 2 is a block diagram to explain fake face detection using changes in facial features and background.

The image acquisition block 210 may acquire an image captured through a camera, and input the acquired image into the face extraction block 220 and the background learning block 230.

The face extraction block 220 may extract a face object from the acquired image and transmit the face image to the motion information comparison block 260 and the background-to-background comparison block 270.

The background learning block 230 generates a plurality of models through background learning that analyzes and models the current background among the plurality of images acquired from the image acquisition block 210, and selects the background of the model with a relatively high weight among them. It can be selected as a learning background (or reference background) and stored in the storage block 250. That is, the background learning block 230 can update the background model with the highest probability through continuous learning.

The blink detection block 240 may detect whether or not an eye blinks through the amount of change in the facial feature point area (eg, eye area) extracted from the face extraction block 220. For example, in the blink detection block 240, the method shown in FIG. 1 may be performed. The detection result of the blink detection block 240 may be transmitted to the forgery detection block 280.

In the motion information comparison block 260, the motion vector component of the face area and the motion vector component of the current background can be extracted and compared. A value representing the movement determined based on the motion vector of the face object and the vector of the background area may be transmitted to the counterfeit determination block 280.

The background comparison block 270 can determine the similarity by comparing the background in the current image received from the face extraction block 220 with the learning background stored in the storage block 250, and the result is the forgery determination block 280. can be passed on.

The forgery determination block 280 determines forgery based on whether there is a blink of the face object in the acquired image, whether the background of the acquired image is similar to the actual background model, and whether the movement of the background of the acquired image is abnormal. You can.

Figure 3 is a diagram for explaining the configuration and operation of a fake face detection device according to the present invention.

The image acquisition block 301 may acquire an image captured by an image acquisition device (eg, a camera) and transmit it to the facial landmark detection unit 302.

The facial landmark detection unit 302 can detect main feature points of the face (eg, eyebrows, eyes, nose, mouth, etc.). For example, one eyebrow can be detected by dividing it into two features, and a total of four features of the left and right eyebrows can be detected. One eye can be detected by dividing it into 4 features, and a total of 8 features can be detected for the left and right eyes. The nose can be detected by distinguishing two parts on both sides. A total of 4 features can be detected for the lips: 2 upper left and right parts, and 2 lower left and right parts. This is just an example, and the scope of the present invention is not limited to extracting 18 feature points from the face.

The facial feature point extraction unit 303 extracts features for facial landmarks (e.g., 18 positions of eyebrows, eyes, nose, and mouth) and sets parameters representing them (e.g., Multi-scale Local Binary (MsLBP) Pattern) can be created. This is just an example, and other descriptors other than MsLBP may be used. However, to extract facial features, it is effective to use a descriptor that is robust against light changes in local areas and expresses geometric features well. This means that when comparing an image of a real face with a fake face image taken from a photo or video, geometric information such as edge information disappears in the fake face image of flat data, making it difficult to detect fake faces. Because it is more advantageous.

The motion vector extractor 304 can extract the motion vector of each frame from a video (or a sequence consisting of a plurality of frames) and calculate the magnitude and direction (angle) of the motion vector (here, the motion vector The size of means the length of the vector, and the direction of the motion vector means the slope value between 0˚ and 360˚). In particular, motion vectors can be extracted for each of the face area, head area, and background area. A detailed description of the operation of the motion vector extractor 304 will be described later with reference to FIGS. 4 to 8.

The motion vector fake face classifier 306 can determine whether any face image corresponds to a fake face based on the motion vector. A detailed description of the operation of the motion vector fake face classifier 306 will be described later with reference to FIGS. 9 and 10.

The general fake face classifier 305 is distinguished from the individual fake face classifier 308, which will be described later. The general fake face classifier 305 determines whether a face is fake based on facial feature points (or feature vectors), and learns the feature points (or feature vectors) of the face image input from the facial feature point extractor 303 in advance. By comparing with the prepared standard, it can be determined whether a random face image corresponds to a fake face. A detailed description of the operation of the general fake face classifier 305 will be described later with reference to FIGS. 11 to 14.

If the face is classified as a fake face by at least one of the general fake face classifier 305 or the motion vector fake face classifier 306, it may be finally determined to be a fake face. If the face is classified as not a fake in both the general fake face classifier 305 and the motion vector fake face classifier 306, additional forgery detection is performed through the face recognition performance unit 307 and the individual fake face classifier 308. It can be.

The face recognition unit 307 may determine who the person corresponding to the face in the acquired image is. For example, the identity or identification information (i.e., ID information) of a person matching the face can be extracted based on the features of the face image. The face recognition result may be transmitted to the individual counterfeit face classifier 308.

The individual counterfeit face classifier 308 can determine whether the face corresponding to the identified identity is counterfeit by comparing standard facial feature points pre-stored in the database with the facial feature points of the acquired image. A detailed description of the operation of the individual fake face classifier 308 will be described later with reference to FIGS. 11 to 14.

One or more of the components described with respect to FIG. 3 may be implemented as a separate processor, but may also be implemented as a functional module in the form of software or firmware of one processor.

Figure 4 is a diagram for explaining a motion vector extraction area according to the present invention.

In order to extract motion vectors from the acquired image, it can be divided into face area, head area (or head and shoulders area), and background area (or area outside the camera frame). In a video of an actual face, there is movement in the face area and head area depending on the movement of the human body, but such movement rarely occurs in the background. In addition, many fine movements occur in the face area due to eye blinking, speech, facial expression changes, etc., but relatively less such fine movements occur in the head area.

In the present invention, the terms face area, head area, and background area are used for convenience of explanation, but the scope of the present invention is not limited to the dictionary meaning of the terms. That is, the present invention provides an example of dividing the acquired facial image into first, second and third areas, the third area includes the second area, and the second area includes the first area. Includes.

Figure 5 is a diagram for explaining the configuration and operation of a device for extracting a motion vector according to the present invention.

The image acquisition block 501 may correspond to the image acquisition block 301 of FIG. 3 .

When consecutive frames are input by the image acquisition block 501, the two consecutive frame extractor 502 extracts two consecutive frames from them. The two consecutive frames extractor 502 may extract two consecutive frames from an image input after a face object is detected in facial landmark detection (302 in FIG. 3).

The high-density optical flow calculation unit 503 can calculate the high-density optical flow (i.e., the pattern of movement observed for an object, surface, or edge) between two extracted frames.

The histogram generator 504 can divide the three areas divided into a face area, a head area, and a background area in more detail, as shown in the example of FIG. 4 . For example, the face area can be divided into N1 sub-blocks, the head area can be divided into N2 sub-blocks, and the background area can be divided into N3 sub-blocks. The average vector can be calculated in each of the divided areas (e.g., face area, head area, background area), and the magnitude and direction (angle) of the average vector within the area can be calculated. Using the result, a histogram (i.e., a one-dimensional histogram vector) can be created for each divided area.

The vector combining unit 505 combines one-dimensional histogram vectors derived from each of the divided areas (e.g., face area, head area, background area) to generate one motion vector (or motion feature vector). You can.

One or more of the components described with respect to FIG. 5 may be implemented as a separate processor, but may also be implemented as a functional module in the form of software or firmware of one processor.

6 to 8 are diagrams for explaining examples of motion vectors obtained according to the present invention.

The upper two drawings of FIG. 6 show examples of motion vectors (vectors indicated in green) obtained from images of actual faces, and the lower drawings show the corresponding histograms.

The top two drawings of FIG. 7 show examples of motion vectors (vectors marked in green) obtained from images taken from photographs (e.g., facial photos taken with a smartphone), and the lower drawings show the corresponding histograms. represents.

The top two drawings of FIG. 8 show examples of motion vectors (vectors marked in green) obtained from images captured in a video (e.g., a face video taken with a smartphone), and the lower drawings show the corresponding histogram. represents.

For example, if the direction of one motion vector is 70° and its size is 20, the count of the corresponding element in Table 1 below can be added by 1. For the remaining motion vectors, counting according to their direction and size can be repeated. The sum of the vector magnitudes for a specific direction is the number of vectors with the first magnitude in that direction multiplied by the first magnitude value, the number of vectors with the second magnitude multiplied by the second magnitude value, and all of these. It can be derived by summing.

direction
size 0 … 70˚ … 360˚ … … … … … … 20 0 … One … 0 … … … … … … MAX 0 … 0 … 0

Accordingly, the calculated sum of vector sizes in a specific direction may result in a relatively large value, so when expressing it as a histogram graph, it can be easily compared with motion vectors of other images by applying scaling by determining the minimum and maximum values. In other words, when comparing motion vectors of different images with histogram graphs, the size of the graph is not compared, but the similarity is determined by looking at the size distribution in the vector direction, so the relative size relationship between the vector sizes is not necessary. am.

From the histogram graph of the example of FIG. 6, it can be seen that in the case of a real face, relatively many motion vector components in the left and right (or horizontal) directions occur, and from the histogram graph of the example of FIG. 7, in the case of a fake face (photo) It can be seen that the frequency of situations in which there is no movement due to the person holding the photo and causing it to shake is low, and many motion vector components in all directions (up, down, left and right (or vertical and horizontal)) occur. From the example histogram graph in FIG. 8, in the case of a fake face (video), the size of the motion vector appears relatively small compared to other cases due to restrictions on the size of the video display that can be carried by people, and the shape of the graph itself is similar to that of a real face. A pattern may be visible, but the size of the vector is smaller or the pattern is relatively distorted in one direction. The histogram graphs in FIGS. 6 to 8 are only examples representing average values obtained through many experiments and do not limit the scope of the present invention.

As can be seen from the graph expressing the histogram of the size and direction based on the motion vectors obtained from the real face, fake face (photo), and fake face (video), the motion vector obtained from the real face is either a photo or a video. It shows a significant difference from the motion vector obtained from the video. In other words, since there is a significant difference between the characteristics of the normalized histogram of the motion vector of the real face and the photo or video, it is possible to determine whether the face is fake based on this.

Figure 9 is a diagram for explaining the configuration and operation of the motion vector forged face classifier according to the present invention.

As described above, the motion vector fake face classifier (306 in FIG. 3) according to the present invention may determine whether a face is fake based on the features of the motion vector itself (e.g., distribution features on the histogram), but requires additional training. It also supports detection of fake faces using images.

For example, a situation in which each of the three categories of real faces, fake faces (photos), and fake faces (videos) is actually detected using variously trained motion vector features (e.g., distribution features on the histogram). In this case, fake face detection performance can be improved by determining which of the three categories it is closest to.

In the example of FIG. 9 , the training image generator 901 may collect or store data for each modeling classification. For example, in order to create a model of a real face (i.e., a reference motion vector for comparison with a real face), training images are required to identify various characteristics of the real face. Such training image data may be provided in advance, or image data determined to be a real face according to the fake face detection results of the present invention may be provided as a training image in a feedback manner. In order to create a model for a fake face (photo) (i.e., a motion vector based on comparison for a fake face using a photo), a training image is required to identify various characteristics of a fake face using a photo. Such training image data may be provided in advance, or image data determined to be a fake face using a photo according to the fake face detection result of the present invention may be provided as a training image in a feedback manner. In order to create a model for a fake face (video) (i.e., a motion vector based on comparison for a fake face using a video), a training image is required to identify various characteristics of a fake face using a video. Such training image data may be provided in advance, or image data determined to be a fake face using a video according to the fake face detection results of the present invention may be provided as a training image in a feedback manner.

The motion vector extraction unit 902 can extract motion vectors for each modeling classification (e.g., three classifications of real face, fake face (photo), and fake face (video)), and for this purpose, a motion vector as shown in Figure 5 method may also be used.

The model generator 903 uses a statistical modeling method based on the extracted motion vector to create a model for each modeling classification (e.g., three classifications: real face, fake face (photo), fake face (video)). can be created.

One or more of the components described with respect to FIG. 9 may be implemented as a separate processor, but may also be implemented as a functional module in the form of software or firmware of one processor.

Figure 10 is a diagram for explaining a method of identifying a fake face using a motion vector fake face classifier according to the present invention.

Motion vector extraction 1002 may be performed on the input image 1001 acquired through the image acquisition unit 301, and the method shown in FIG. 5 described above may be applied to this. The extracted motion vector is generated for each of the parasitic models (e.g., three classifications of modeling classification (e.g., real face, fake face (photo)), and fake face (video)) as in the example of Figure 9). model) can be compared (1003). That is, the extracted motion vector can be compared with a model for a real face, compared with a model for a fake face (photo), and compared with a model for a fake face (video). As a result of comparison (1003), a model with the highest similarity among several modeling classifications can be determined. If the motion vector extracted for the input image is most similar to the actual face model, it can be determined not to be a fake face, and if not (for example, a model of a fake face (photo) or fake face (video) (if it is most similar), it can be determined to be a fake face.

Figure 11 is a diagram for explaining the configuration and operation of a general fake face classifier and an individual fake face classifier according to the present invention.

The image acquisition block 1101, facial landmark detection unit 1102, and facial feature point extraction unit 1103 in FIG. 11 are the image acquisition block 301, facial landmark detection unit 302, and facial feature point extraction unit in the example of FIG. 3. (303) can be responded to. That is, an image is acquired using an image acquisition device (e.g., a camera), facial feature points (e.g., 18 facial landmarks for eyebrows, eyes, nose, and lips) are detected, and the detected feature points are A feature vector can be extracted from the location using MsLBP.

The fake facial feature point data generator 1104 uses the extracted facial feature data as fake facial feature data (or fake facial features) using a transformation matrix (e.g., transfer function or adaptation function). vector). An example of a transformation matrix will be described with reference to FIG. 12.

Figure 12 is a diagram for explaining the transformation matrix available in the fake facial feature point data generator according to the present invention.

The basis (G _basis ) of real face data can be determined through principal component analysis (PCA) on a set of real face data (G ₁ , G ₂ , ..., G _N ). Meanwhile, the basis (F _basis ) of the fake face data can be determined through PCA on the set of fake face data (F ₁ , F ₂ , ..., F _M ). Based on these, the transformation matrix (X) can be calculated. In other _words , if the _result of applying a specific transformation matrix

Referring again to FIG. 11, using the real fake face generator 1105, the result of photographing a real person's face (i.e., a photo or video) is re-photographed with an image acquisition device (e.g., a camera) to create a forgery. A fake face image corresponding to a face sample can be created. Using the fake face generator 1105, one or more fake face images can be generated. When the actual fake face generator 1105 is applied, it can be used as useful fake face data not only in a personal fake face classifier but also in a general fake face classifier, thereby improving the performance of fake face detection. In particular, the application of the actual fake face generator 1105 can be applied together when generating and registering a personal identification image for an individual fake face classifier. However, in cases where it is not appropriate to register a personal identification image in advance, the real fake face generator 1105 uses the above-described conversion function to transform real face data (i.e. real facial feature vector) into fake face data (i.e. , fake facial feature vectors) can also be created and used.

The general fake face classifier generator 1106 can improve the performance of the general fake face classifier by learning or updating the general fake face classifier.

The personal fake face classifier generator 1107 can improve the performance of the personal fake face classifier by learning or updating the personal fake face classifier.

Examples of the general fake face classifier generator 1106 and the individual fake face classifier generator 1107 will be described with reference to FIGS. 13 and 14, respectively.

Figure 13 is a diagram to explain the creation of a general fake face classifier.

A general fake face classifier can use various training data to set a comparison standard for general fake face feature vectors and determine whether it is a general fake face accordingly. In order to train or update this general fake face classifier, new data other than the previously obtained training data (e.g., data on the real face of the newly registered person(s), real fake face (i.e. real face) Add data about the captured result (i.e., a photo or video) re-photographed by an image acquisition device (e.g., a camera), or data about a fake face converted from a real face using a transformation matrix) It can be entered as . Accordingly, a judgment criterion for whether a general fake face is learned is learned from the feature vector for the real face set of the existing registrant and the fake face set of the existing registrant (which is a set containing one or more real fake faces or converted fake faces). and update the binary classifier.

Figure 14 is a diagram to explain the creation of an individual fake face classifier.

When an individual is identified through face recognition, the individual counterfeit face classifier can learn and update data on each individual's real face and counterfeit face for more accurate counterfeit detection results. The individual fake face classifier combines data about the real face of an identified individual (e.g., the registrant), the real fake face (i.e., the result of taking a picture of the real face (i.e., a photo or video)) using an image acquisition device (e.g., Based on the data (re-photographed result with a camera) or data (data about the fake face converted from the real face using a transformation matrix), a decision criterion for whether the corresponding individual is a fake face is learned and a binary classifier is used. can be updated.

One or more of the components described with reference to FIGS. 11 to 14 may be implemented as a separate processor, but may also be implemented as a functional module in the form of software or firmware of one processor.

As described above, a counterfeit face classifier that operates based on both motion vectors (e.g., high-density optical flow) and feature vectors (e.g., MsLBP) is more accurate and has a higher probability than conventional counterfeit face detection methods. It can provide fake face detection results. In addition, the fake face data needed to create (or learn, update) a fake face classifier can be obtained from an actual fake face or reproduced using a trained transformation matrix, so it is easy to provide the data needed to create a fake face classifier. The performance of the face classifier can be further improved.

In addition, by using a combination of motion vectors extracted from the face area, head area, and background area, rather than simply the motion vector extracted only from the face area, the features of the motion vector derived from the behavior of a real person Fake faces can be detected more accurately using a classifier that learns the characteristics of motion vectors triggered when using videos.

Furthermore, by paying attention to the differences between feature vectors obtained from real faces and feature vectors obtained when taking photos or videos, a fake face classifier can be created by learning the differences between these feature vectors in advance using a support vector machine. there is. The fake face classifier combines a method of learning based on the characteristics of a general fake face and a method of learning based on the characteristics of an individual fake face, so even if the general fake face features are not revealed, the probability of detecting a fake face based on the individual fake face features is high. can increase. In this case, when registering personal information to create an individual fake face classifier, the individual's real fake face can be input along with the individual's real face, or a fake face generated by a transformation matrix from the individual's real face can be used.

As such, according to the present invention, a counterfeit face can be accurately detected using images from a commonly used RGB camera without the need for expensive separate equipment such as an infrared camera or a thermal infrared camera. In particular, the conventional method of detecting a fake face based on eye blinks or lip movements can be neutralized by adding movement to a specific part of a high-quality photo or video, but according to the present invention, motion vectors and feature vectors are considered simultaneously. Additionally, feature vector-based fake face detection can detect fake faces with a higher probability by considering both general features and individual features.

Matters described in the various embodiments of the present invention described above may be applied independently, or two or more embodiments may be applied simultaneously.

The exemplary methods described in the various embodiments of the present invention described above are expressed as a series of operations for simplicity of description, but this is not intended to limit the order in which steps are performed, and if necessary, each step may be performed simultaneously or It may be performed in a different order. Additionally, not all of the exemplified steps are necessarily necessary to implement the method proposed in the present invention.

The scope of the present invention includes devices that process or implement operations according to the method proposed in the present invention.

The scope of the present invention is software (or operating system, application, firmware, program, etc.) that allows the operation according to the method proposed in the present invention to be executed on a device or computer, and such software is stored and stored on the device or computer. Includes executable medium.

301 Image acquisition block 302 Face landmark detection unit
303 Facial feature point detection unit 304 Motion vector extraction unit
305 Generic fake face classifier 306 Motion vector fake face classifier
307 Face recognition performance unit 308 Individual fake face classifier

Claims (14)

In a method for detecting a fake face,
Detecting facial landmarks on an image acquired from an image acquisition device;
extracting a motion vector for the acquired image, where the motion vector is information including the magnitude and direction of movement of the detected facial landmark;
extracting a feature vector for the acquired image, where the feature vector is information including a descriptor expressing geometric features for the detected facial landmark; and
Performing fake face classification based on the motion vector and performing general fake face classification based on the feature vector to determine whether the face is fake,
If the face is classified as a fake face in one or more of the fake face classification based on the motion vector or the general fake face classification based on the feature vector, it is finally determined that it is a fake face,
A fake face detection method that additionally performs individual fake face classification when a face is classified as a real face in both the fake face classification based on the motion vector and the general fake face classification based on the feature vector. According to claim 1,
The fake face classification for each individual is performed by a classifier learned using one or more of real face data for each individual, real fake face data for each individual, or converted fake face data for each individual. According to claim 2,
The actual fake face data for each individual is a fake face detection method that is the result of taking a photo or video of each individual's real face. According to claim 2,
The converted fake face data is obtained by applying a transformation matrix to the real fake face data,
The method of detecting a fake face, wherein the transformation matrix is adaptively updated. According to claim 1,
The fake face classification based on the motion vector is,
A method for detecting a fake face, comprising determining whether the acquired image is fake based on a motion vector feature model for a real face and a motion vector feature model for a fake face. According to claim 5,
The motion vector feature model for the fake face is,
A motion vector feature model for fake faces using photos,
A fake face detection method including a motion vector feature model for a fake face using a video. According to claim 1,
The fake face classification based on the motion vector is,
For each of the first region, second region, and third region, it further includes calculating a motion pattern observed between two consecutive frames of the acquired image,
The third area includes the second area, and the second area includes the first area. According to claim 7,
The motion vector is,
Calculating an average vector in each of the first, second and third regions,
A method for detecting a fake face, determined by combining average vectors of the first, second and third regions. According to claim 7,
The first area is a face area, the second area is a head area, and the third area is a background area. According to claim 1,
General fake face classification based on the feature vector is,
A method for detecting fake faces, performed by a classifier trained using one or more of one or more real face data, one or more real fake face data, or one or more converted fake face data. According to claim 10,
A method of detecting a fake face, wherein each of the one or more real fake face data corresponds to a result of taking a photo or video of the one or more real faces. According to claim 11,
Each of the one or more transformed fake face data is obtained by applying a transformation matrix to each of the one or more real fake face data,
The method of detecting a fake face, wherein the transformation matrix is adaptively updated. In a device for detecting a fake face,
A facial landmark detection unit that detects facial landmarks in an image acquired from an image acquisition device;
a motion vector extraction unit that extracts a motion vector for the acquired image, where the motion vector is information including the magnitude and direction of movement of the detected facial landmark;
a feature vector extraction unit for extracting a feature vector for the acquired image, where the feature vector is information including a descriptor representing a geometric feature for the detected facial landmark;
a fake face classifier based on the motion vector; and
Including a general counterfeit face classifier based on the feature vector,
A fake face is determined based on the classification result of the fake face classifier based on the motion vector and the classification result of the fake face classifier based on the feature vector,
If the face is classified as a fake face in one or more of the fake face classification based on the motion vector or the general fake face classification based on the feature vector, it is finally determined that it is a fake face,
A fake face detection device that additionally performs individual fake face classification when a face is classified as a real face in both the fake face classification based on the motion vector and the general fake face classification based on the feature vector. A computer-readable medium storing software having instructions executable by a counterfeit face detection device, comprising:
The executable instructions cause the counterfeit face detection device to detect a facial landmark for an acquired image, extract a motion vector including the magnitude and direction of movement of the detected facial landmark, and After extracting a feature vector containing a descriptor expressing the geometric features of the facial landmark,
Perform fake face classification based on the motion vector and perform general fake face classification based on the feature vector to determine whether or not the face is fake,
If the face is classified as a fake face in one or more of the fake face classification based on the motion vector or the general fake face classification based on the feature vector, it is finally determined that it is a fake face,
A computer-readable medium that additionally performs individual fake face classification if it is classified as a real face in both the fake face classification based on the motion vector and the general fake face classification based on the feature vector.