KR20210049570A - Apparatus and method for detecting forgery or alteration of the face - Google Patents

Abstract

The present invention provides a face forgery detection apparatus and method having forgery detection performance. The face forgery detection method includes the steps of: (a) acquiring a photographed image of a face part of an object through a dual camera; (b) extracting a compression feature related to a face region in the photographed image by using the photographed image; (c) extracting a difference of latent features using the photographed image; and (d) detecting whether the face region in the photographed image is forged or altered by using the difference between the compressed feature and the latent feature.

Description

Face forgery detection device and method {APPARATUS AND METHOD FOR DETECTING FORGERY OR ALTERATION OF THE FACE}

The present application relates to a face forgery detection apparatus and method. In particular, the present application relates to a face forgery detection apparatus and method using a dual camera.

With the recent increase in intelligent crime, the demand for advanced security technology is increasing. Biometric recognition technology that identifies a person using unique biometric characteristics inherent in each person is gradually increasing in importance in terms of user convenience.

Among them, face recognition is one of the most attractive technologies because it naturally recognizes people based on their faces without contacting a specific sensor. Face recognition is gradually expanding its use as its performance improves with the development of face detection, posture estimation, lighting processing, and facial feature extraction technologies.

Such face recognition technology is classified into a recognition technology based on a 2D image and a technology based on a 3D shape. Since a recognition technology based on a 3D shape requires a large amount of computation, a face recognition system based on a 2D image is currently widely used.

Meanwhile, a face recognition system based on a 2D image has a disadvantage that is vulnerable to a camouflage attack using a face photograph of a registered target. In order to solve this problem, studies on liveness detection are constantly being studied.

One of the prior art is a method of detecting a fake face by analyzing a Fourier spectrum. This takes advantage of the fact that the image captured from the real face contains more high-frequency components than the image captured from the photograph. However, this method has a problem in that the performance is not stable because the difference between the high frequency components in the two images is less pronounced than the difference between the high frequency components in the image caused by changes in the external environment.

In addition, most of the existing studies identify real faces and forged faces using images acquired from a single camera. In particular, using features such as the quality of the face image and texture pattern, the actual faces and forged faces are identified. Research to detect subtle differences has been actively conducted.

In recent years, thanks to the success of the deep neural network in the field of image recognition, attempts to apply it to the detection of forgery or alteration are increasing, but the conventional forgery detection techniques known to date include deformation of a mask output on paper or a high-resolution screen (image). The disadvantage is that it is still vulnerable to the forgery and alteration attacks used.

The technology behind the present application is disclosed in Korean Patent Publication No. 10-1064945.

The present application is to solve the problems of the prior art described above, the purpose of which is to provide a face forgery detection apparatus and method having a robust forgery detection performance against a forgery attack using a mask output on paper or a high-resolution screen (image). do.

However, the technical problem to be achieved by the embodiment of the present application is not limited to the technical problems as described above, and other technical problems may exist.

As a technical means for achieving the above technical problem, a method for detecting forgery and alteration of a face according to an embodiment of the present application includes: (a) obtaining a photographed image of a face portion of an object through a dual camera; (b) extracting a compression feature related to a face region in the captured image by using the captured image; (c) extracting a difference of latent features using the captured image; And (d) detecting whether the face region in the photographed image has been forged or altered by using the difference between the compressed feature and the latent feature.

In addition, the captured image in step (a) includes a first viewpoint image and a second viewpoint image, and in step (b), the first viewpoint image and the second viewpoint image are provided as inputs of the first neural network. Thus, the compression feature including a first view compression feature corresponding to the first view image and a second view compression feature corresponding to the second view image may be extracted from the first neural network.

In addition, in step (c), a potential feature difference between the first viewpoint image and the second viewpoint image may be extracted by using the first viewpoint image and the second viewpoint image.

In addition, in the step (c), the difference in the latent feature is the feature of the first view image and the feature of the first view image generated when the feature of the first view image and the feature of the second view image are compressed in the first neural network. It can be obtained by passing a difference feature, which is a difference between features of the second view image, through the second neural network.

In addition, in the step (d), a score for the captured image is calculated by using the first view compression feature, the second view compression feature, and the latent feature difference, and the captured image is based on the calculated score. It is possible to detect whether or not the face area is forged or altered.

In addition, in the step (d), the first view compression feature, the second view compression feature, and the latent feature difference are combined to generate an integrated combination feature, and the integrated combination calculated using the generated integrated combination feature. The forgery or alteration may be detected based on the feature-related score.

In addition, in the step (d), a first score related to a combined feature is calculated using a combined feature obtained by combining the first view compression feature and the second view compression feature, and the potential feature difference is related using the latent feature difference. A second score may be calculated, and the forgery or alteration may be detected based on a sum score obtained by adding the first score and the second score.

In addition, the summed score in step (d) may be a score obtained by an average of the first score and the second score.

Meanwhile, an apparatus for detecting face forgery and alteration according to an exemplary embodiment of the present disclosure includes: an acquisition unit for obtaining a photographed image of a face portion of an object through a dual camera; A first extracting unit for extracting a compression feature related to a face region in the captured image by using the captured image; A second extraction unit for extracting a difference of latent features using the captured image; And a detection unit that detects whether or not the face region in the photographed image is forged or altered by using the difference between the compressed feature and the latent feature.

In addition, the captured image includes a first viewpoint image and a second viewpoint image, and the first extraction unit provides the first viewpoint image and the second viewpoint image as inputs of the first neural network, and The compression feature including a first view compression feature corresponding to the first view image and a second view compression feature corresponding to the second view image may be extracted.

In addition, the second extraction unit may extract a potential feature difference between the first viewpoint image and the second viewpoint image by using the first viewpoint image and the second viewpoint image.

In addition, the latent feature difference is a feature of a first view image and a feature of the second view image occurring in a process in which the feature of the first view image and the feature of the second view image are compressed in the first neural network. It can be obtained by passing the difference feature, which is the difference between the two, through the second neural network.

In addition, the detection unit calculates a score for the captured image by using the first viewpoint feature, the second viewpoint feature, and the latent feature difference, and based on the calculated score, the face region in the photographed image is It is possible to detect whether or not forgery or alteration has occurred.

In addition, the detection unit generates an integrated combination feature by combining the first view compression feature, the second view compression feature, and the latent feature difference, and an integrated combination feature related score calculated using the generated integrated combination feature. It is possible to detect whether the forgery or alteration is based on.

In addition, the detection unit calculates a first score related to a combined feature by using a combined feature obtained by combining the first view compression feature and the second view compression feature, and uses the latent feature difference to calculate a second score related to the potential feature difference. Is calculated, and the forgery or alteration may be detected based on a sum score obtained by adding the first score and the second score.

In addition, the summed score may be a score obtained by an average of the first score and the second score.

The above-described problem solving means are merely exemplary and should not be construed as limiting the present application. In addition to the above-described exemplary embodiments, additional embodiments may exist in the drawings and detailed description of the invention.

According to the above-described problem solving means of the present application, by providing a face forgery detection device using a first viewpoint image and a second viewpoint image acquired through a dual camera, deformation of a mask output on paper or a high-resolution screen (image) is used. It can perform high-performance face forgery detection, which shows robust characteristics even in forgery attacks.

According to the above-described problem solving means of the present application, by providing a face forgery detection apparatus using a latent feature difference that is a difference between a first viewpoint image and a second viewpoint image, it is possible to implicitly learn the difference due to a three-dimensional structure, Through this, it is possible to effectively detect whether the face has been forged or altered.

However, the effect obtainable in the present application is not limited to the above-described effects, and other effects may exist.

1 is a view showing a schematic configuration of a face forgery detection apparatus according to an embodiment of the present application.
FIG. 2 is a diagram schematically illustrating a first neural network and a second neural network included in a face forgery detection apparatus according to an embodiment of the present application.
3 is a diagram for explaining a first method in which a detection unit of a face forgery detection apparatus according to an exemplary embodiment of the present disclosure detects whether there is forgery or alteration.
4 is a diagram for explaining a second method in which a detection unit of a face forgery detection apparatus according to an exemplary embodiment of the present disclosure detects whether or not forgery has been altered.
FIG. 5 is a diagram for describing a case of identifying a face region in a captured image by a face forgery detection apparatus according to an exemplary embodiment of the present disclosure.
6 is a diagram showing a performance evaluation result of a face forgery detection apparatus according to an embodiment of the present application.
7 is a flowchart illustrating an operation of a method for detecting forgery and alteration of a face according to an exemplary embodiment of the present application.

Hereinafter, embodiments of the present application will be described in detail with reference to the accompanying drawings so that those of ordinary skill in the art may easily implement the present application. However, the present application may be implemented in various different forms and is not limited to the embodiments described herein. In addition, in the drawings, parts irrelevant to the description are omitted in order to clearly describe the present application, and similar reference numerals are attached to similar parts throughout the specification.

Throughout the present specification, when a part is said to be "connected" with another part, it is not only the case that it is "directly connected", but also "electrically connected" or "indirectly connected" with another element interposed therebetween. "Including the case.

Throughout the present specification, when a member is positioned "on", "upper", "upper", "under", "lower", and "lower" of another member, this means that a member is located on another member. This includes not only the case where they are in contact but also the case where another member exists between the two members.

In the entire specification of the present application, when a certain part "includes" a certain component, it means that other components may be further included rather than excluding other components unless specifically stated to the contrary.

1 is a view showing a schematic configuration of a face forgery detection apparatus 100 according to an embodiment of the present application. 2 is a diagram schematically illustrating a first neural network 20 and a second neural network 30 included in the apparatus 100 for detecting forgery and alteration of a face according to an exemplary embodiment of the present application. 3 is a view for explaining a first method in which the detection unit 140 of the face forgery detection apparatus 100 according to an embodiment of the present disclosure detects whether there is forgery or alteration. 4 is a view for explaining a second method in which the detection unit 140 of the face forgery detection apparatus 100 according to an embodiment of the present disclosure detects whether or not forgery has been altered. FIG. 5 is a diagram for describing a case in which the face forgery detection apparatus 100 according to an exemplary embodiment of the present disclosure identifies a face area within a captured image 10.

Hereinafter, the face forgery detection apparatus 100 according to an exemplary embodiment of the present disclosure will be referred to as the present apparatus 100 for convenience of description.

1 to 5, the apparatus 100 may include an acquisition unit 110, a first extraction unit 120, a second extraction unit 130, and a detection unit 140. The apparatus 100 may be a device that detects whether a face of an object (person) included in an input image is a real face or a false face (a fake face that has been forged).

The acquisition unit 110 may acquire a photographed image 10 of a face portion of the object through the dual camera 1. The captured image 10 may include a first viewpoint image 11 and a second viewpoint image 12. Since the image acquired by the acquisition unit 110 is an image acquired using the dual camera 1, two images 11 and 12 may be included.

The dual camera 1 may be expressed differently as a stereo camera or the like. The dual camera 1 may include two sensors disposed to be spaced apart from the lower surface of the dual camera 1 at a predetermined interval (pre-set interval) with respect to a horizontal direction horizontal to the lower surface of the dual camera 1. Here, the sensor may be expressed differently as an image sensor. The distance between the two sensors may be different for each model of the manufactured dual camera 1. For example, the distance between the two sensors may be 12 cm, but is not limited thereto, and the numerical value may be applied in various ways.

The first viewpoint image 11 may be an image acquired (photographed) through any one of the two sensors. The second viewpoint image 12 may be an image acquired (photographed) using any one of the two sensors (ie, other sensors other than the sensor that acquires the first viewpoint image).

As an example, the first viewpoint image 11 may be a left image, and the second viewpoint image 12 may be a right image, but the present invention is not limited thereto. As another example, the first viewpoint image 11 may be a right image, and the second viewpoint image 12 may be a left image.

Hereinafter, the first viewpoint image 11 is the left image, and the second viewpoint image 12 is the right image. According to this, one of the two sensors in the dual camera 1 may be a left sensor, and the other sensor may be a right sensor.

The first extracting unit 120 may extract a compressed feature 40 related to a facial region within the captured image 10 by using the captured image 10 acquired by the acquisition unit 110. In this case, the compression feature 40 may be a compression feature related to the pre-processed photographed image 10 in which the photographed image 10 has been pre-processed.

The compression feature 40 may include a first view compression feature 41 that is a compression feature related to the first view image 11 and a second view compression feature 42 that is a compression feature related to the second view image 12. I can. In this case, the first view compression feature 41 may be a compression feature related to the preprocessed first view image 11 ′ in which the first view image 11 has been preprocessed. In addition, the second view compression feature 42 may be a compression feature related to the preprocessed second view image 12 ′ in which the second view image 12 has been pre-processed. A description of the pretreatment may be more easily understood with reference to FIG. 5.

Referring to FIG. 5, the acquisition unit 110 may acquire a captured image 10 including a first viewpoint image 11 and a second viewpoint image 12 through the dual camera 1. Thereafter, the first extraction unit 120 may receive the captured image 10 (11, 12) from the acquisition unit 110, at this time, the first extraction unit 120 is transmitted from the acquisition unit 110 The photographed images 10 (11, 12) may be pre-processed photographed images 10 ′, 11 ′, 12 ′ in which pre-processing has been performed. To this end, although not shown in the drawings, the apparatus 100 may include a preprocessor (not shown).

The preprocessor (not shown) may perform preprocessing on the face of the object in the captured images 10 (11, 12) acquired through the dual camera 1. In particular, the preprocessor (not shown) may perform preprocessing on the face portion s1 of the object in the first viewpoint image 11. Also, the preprocessor (not shown) may perform preprocessing on the face portion s2 of the object in the second viewpoint image 12.

The preprocessor (not shown) identifies the face regions s1 and s2 including the face of the object within the captured images 10 (11, 12) acquired from the dual camera 1, and identifies the identified face regions ( Pre-processing of normalizing the size of s1 and s2) can be performed.

Through the pre-processing of the captured image 10, a pre-processed captured image 10 ′ including only the identified face area may be obtained. Through the pre-processing of the first viewpoint image 11, a preprocessed first viewpoint image 11 ′ including only the identified face region s1 may be obtained. Through pre-processing of the second viewpoint image 12, a preprocessed second viewpoint image 12 ′ including only the identified face region s2 may be obtained.

In this case, the normalization may be, for example, normalization that is readjusted to a size of 128 × 128 pixels. However, it is not limited to this, and the number can be applied in various ways. In other words, the size of the preprocessed first view image 11 ′ and the preprocessed second view image 12 ′ in which normalization has been performed may have a pixel size of 128×128, for example.

The preprocessed first viewpoint image 11 ′ and the preprocessed second viewpoint image 12 ′ may be transmitted to the first extraction unit 120.

The first extraction unit 120 provides the first viewpoint image 11 and the second viewpoint image 12 as inputs of the first neural network 20, and the first viewpoint image 11 from the first neural network 20 A compression feature 40 including a first view compression feature 41 corresponding to and a second view compression feature 41 corresponding to the second view image 12 may be extracted. The first view compression feature 41 and the second view compression feature 42 may be outputs (output values, output results) of the first neural network 20.

In particular, the first extraction unit 120 may receive the preprocessed first viewpoint image 11 ′ and the preprocessed second viewpoint image 12 ′ as the preprocessed photographed image 10 ′ from the acquisition unit 110. have. Thereafter, the first extraction unit 120 may provide the preprocessed first viewpoint image 11 ′ and the preprocessed second viewpoint image 12 ′ as inputs of the first neural network 20. In other words, the image input to the first neural network 20 may be a preprocessed photographed image 10 ′, that is, a preprocessed first viewpoint image 11 ′ and a preprocessed second viewpoint image 12 ′.

Accordingly, the first extraction unit 120 includes the first view compression feature 41 corresponding to the first view image 11 ′ preprocessed from the first neural network 20 and the preprocessed second view image 12 ′. The compression feature 40 including the second view compression feature 41 corresponding to may be extracted.

The first neural network 20 refers to a deep learning model, an artificial intelligence (AI) algorithm model, a machine learning (machine learning) model, a neural network model (artificial neural network model), a deep neural network model, a neuro fuzzy model, etc. can do. The first neural network 20 may be, for example, a convolution neural network (CNN, a convolutional neural network). However, the present invention is not limited thereto, and various neural networks known in the art or developed in the future may be applied to the first neural network 20.

The first neural network 20 may include a first sub neural network 21 and a second sub neural network 22. The first sub neural network 21 includes first layers 21a, 21b, 21c, ..., 21x, and the second sub neural network 22 includes second layers 22a, 22b, 22c, ..., 22x) It may include.

Each of the first layers 21a, 21b, 21c, …, 21x and each of the second layers 22a, 22b, 22c, …, 22x may mean a convolutional layer (convolutional layer, CONV).

When the preprocessed first viewpoint image 11 ′ is input to the first neural network 20, the characteristics (features) of the preprocessed first viewpoint image 11 ′ are the first layers in the first neural network 20 As (21a, 21b, 21c, …, 21x) passes, it may be gradually compressed (that is, features are gradually compressed), and accordingly, the first view compression feature 41 may be output from the first neural network 20 have.

Specifically, among the first layers 21a, 21b, 21c, ..., 21x, the first layer 21a receives the preprocessed first view image 11' as an input, and the preprocessed first view image 11' The first first compressed feature 11a in which the feature of) is compressed once (once) by the first layer 21a may be output. That is, as the first feature compression process is performed by the first layer 21a, the first first compressed feature 11a may be output.

In addition, the second layer 21b positioned after the first layer 21a receives the first first compressed feature 11a as an input, and the feature of the preprocessed first view image 11' is the first layer. The second first compressed feature 11b compressed twice (twice) by the (21a) and the second layer 21b may be output. That is, as the second feature compression process is performed by the second layer 21b, the second first compressed feature 11b may be output.

As described above, the preprocessed first viewpoint image 11 ′ and its related features are compressed as they pass through the first layers 21a, 21b, 21c, …, 21x in the first neural network 20 (according to roughness). The process may be performed and gradually compressed, through which the first view compression feature 41 may be output. The first view compression feature 41 is an output result generated when the preprocessed first view image 11' passes through the first layers 21a, 21b, 21c, …, 21x among the first neural networks 20 in particular. As, it may mean a compressed feature (compressed feature) of the preprocessed first viewpoint image 11 ′.

When the preprocessed second view image 12' is input to the first neural network 20, the features (features) of the preprocessed second view image 12' are the second layers in the second neural network 30 As passing through (22a, 22b, 22c, …, 22x), it may be gradually compressed (ie, features are gradually compressed), and accordingly, the second view compression feature 42 may be output from the second neural network 30. have.

Specifically, among the second layers 22a, 22b, 22c, ..., 22x, the first layer 22a receives the preprocessed second view image 12' as an input, and the preprocessed second view image 12' The first second compressed feature 12a in which the feature of) is compressed once (once) by the first layer 22a may be output. That is, as the first feature compression process is performed by the first layer 22a, the first second compressed feature 12a may be output.

In addition, the second layer 22b positioned after the first layer 22a receives the first second compressed feature 12a as an input, and the feature of the preprocessed second view image 12' is the first layer. The second second compressed feature 12b compressed twice (twice) by the (22a) and the second layer 22b may be output. That is, as the second feature compression process is performed by the second layer 22b, the second second compressed feature 12b may be output.

As described above, the preprocessed second view image 12 ′ and its related features are compressed as they pass through the second layers 22a, 22b, 22c, …, 22x in the first neural network 20 (according to the roughness). The process may be performed and gradually compressed, through which the second view compression feature 42 may be output. The second view compression feature 42 is an output result generated by the preprocessed second view image 12' passing through the second layers 22a, 22b, 22c, …, 22x, among the first neural networks 20 in particular. As, it may mean a compressed feature (compressed feature) of the preprocessed second viewpoint image 12 ′.

In other words, the preprocessor (not shown) identifies the face region for each of the first viewpoint image 11 and the second viewpoint image 12 acquired by the acquisition unit 110 from the dual camera 1 and Pre-processing of normalizing the size of the face area can be performed. Thereafter, the preprocessor (not shown) may transfer the preprocessed first viewpoint image 11 ′ and the preprocessed second viewpoint image 12 ′ to the first extraction unit 120. Thereafter, the first extraction unit 120 extracts features from each of the two preprocessed images 11 ′ and 12 ′, and in the case of the preprocessed first viewpoint image 11 ′, the first neural network 20 In the case of the second view image 12 ′ that is provided as an input of the sub neural network 21 and preprocessed, it may be provided as an input of the second sub neural network 22 in the first neural network 20. Through this, the first extraction unit 120 includes a first view compression feature 41 corresponding to the first view image 11 ′ preprocessed from the first neural network 20 (through the output of the first neural network) and preprocessing. The second view compression feature 42 corresponding to the second view image 12 ′ may be extracted.

The second extraction unit 130 may extract a difference of latent features 50 using the captured image 10.

The second extraction unit 130 extracts a potential feature difference 50 between the first viewpoint image 11 and the second viewpoint image 12 using the first viewpoint image 11 and the second viewpoint image 12. can do. Here, the first viewpoint image 11 and the second viewpoint image 12 considered when calculating the potential feature difference 50 are preprocessed first viewpoint images 11 ′ and second viewpoint images 12 ′, respectively. I can. In other words, the second extraction unit 130 uses the preprocessed first viewpoint image 11 ′ and the preprocessed second viewpoint image 12 ′ to obtain a preprocessed first viewpoint image 11 ′ and a preprocessed second viewpoint image 11 ′. A potential feature difference 50 between the viewpoint images 12 ′ may be extracted.

The latent feature difference 50 is a first view that occurs in a process in which the features of the first view image 11 and the features of the second view image 12 are compressed in the first neural network 20 (feature compression process) It may be obtained by passing through the second neural network 30 a difference feature, which is a difference between the feature of the image 11 and the feature of the second viewpoint image 12. In particular, the latent feature difference 50 is a process in which the features of the preprocessed first viewpoint image 11' and the features of the preprocessed second viewpoint image 12' are compressed in the first neural network 20 (feature compression Process) may be obtained by passing through the second neural network 30 a difference feature that is a difference between the features of the preprocessed first viewpoint image 11 ′ and the preprocessed second viewpoint image 12 ′. .

In order to extract the potential feature difference 50, the second extraction unit 130 includes compressed features for each layer and the first neural network 20 output through the first sub-neural network 21 in the first neural network 20. ) A difference between compressed features for each layer output through the second sub neural network 22 in) may be calculated for each layer as a difference feature. Thereafter, the second extraction unit 130 provides the difference features (difference features) calculated for each layer to the second neural network 30, and obtains the latent feature difference 50 from the output of the second neural network 30. can do. A detailed description is as follows.

As mentioned above, as the preprocessed first view image 11' passes through the first layers 21a, 21b, 21c, ..., 21x of the first sub neural network 21 in the first neural network 20 , Features of the preprocessed first viewpoint image 11 ′ may be gradually compressed. Similarly, as the preprocessed second view image 12' passes through the second layers 22a, 22b, 22c, …, 22x of the second sub neural network 22 in the first neural network 20, the preprocessed The features of the second viewpoint image 12 ′ may be gradually compressed.

In this case, the second extraction unit 130 extracts the first first compressed feature 11a and the first compressed feature 11a, which is an output of the first layer 21a in the first sub-neural network 21, to extract the potential feature difference 50. The difference between the first second compressed features 12a, which is an output of the first layer 22a in the sub-neural network 22, may be calculated as the first difference feature 13a. In addition, the second extraction unit 130 includes a second first compressed feature 11b that is an output of the second layer 21b in the first sub neural network 21 and a second layer in the second sub neural network 22 ( The difference between the second second compressed features 12b, which is the output of 22b), may be calculated as the second difference feature 13b. As such, the second extraction unit 130 includes compressed features 11a, 11b, …, which are outputs of each of the layers in the first sub-neural network 21, and outputs of each of the layers in the second sub-neural network 22. The difference between the compressed features 12a, 12b, ... may be calculated as the difference features 13a, 13b, ....

The second neural network 30 includes the number of layers 31a, 31b, 31c, ..., 31x corresponding to the number of layers in the first sub neural network 21 or the number of layers in the second sub neural network 22 can do. In this case, the calculated first difference feature 13a may be provided as the first layer 31a in the second neural network 30. In addition, the calculated second difference feature 13b may be provided as the second layer 31b in the second neural network 30.

Like the first neural network 20, the second neural network 30 may be, for example, a convolutional neural network (CNN) neural network, but is not limited thereto.

The second extraction unit 130 may provide the calculated difference features 13a, 13b,… to the second neural network 30, and pass the difference features 13a, 13b,… through the second neural network 30 By doing so, the latent feature difference 50 can be calculated.

The detection unit 140 uses the compressed feature 40 extracted from the first extracting unit 120 and the latent feature difference 50 extracted from the second extracting unit 130 to determine the identified object in the captured image 10. Whether the face regions s1 and s2 are forged or altered may be detected.

The detection unit 140 detects the first view compression feature 41, the second view compression feature 42, and the latent feature difference 50 extracted from the second extraction unit 130, extracted from the first extraction unit 120. By using it, a score for the captured image 10 may be calculated. Thereafter, the detection unit 140 may detect whether the face area of the object in the captured image 10 is forged or altered based on the calculated score.

In particular, the detection unit 140 may calculate a score regarding the possibility of forgery or alteration (or possibility of a real face) for a face region in the pre-processed photographed image 10 ′ using each of the features 41, 42, and 50. The detection unit 140 may detect whether the face regions s1 and s2 in the captured image 10 are forged or altered based on the calculated score.

In this case, there may be two methods of calculating a score for detecting whether or not forgery or alteration exists. Among them, the first method may be more easily understood with reference to FIG. 3 and the second method with reference to FIG. 4.

Referring to FIG. 3, a method of calculating a first score by the detection unit 140 (a first method) is as follows. The detection unit 140 may generate an integrated combination feature 43 by combining the first view compression feature 41, the second view compression feature 42, and the latent feature difference 50. Thereafter, the detection unit 140 detects whether the captured image 10 (in particular, a face area in the captured image) is forged or altered based on the score 43s related to the integrated combination feature calculated using the generated integrated combination feature 43 can do.

The detection unit 140 may provide the combined combined feature 43 as an input of the third sub neural network 61 and calculate a score 43s related to the combined combined feature from the output of the third sub neural network 61.

Here, the third sub-neural network 61 used when calculating the integrated coupling feature related score 43s is, for example, three convolutional layers (conv) and three fully connected layers (fully connected, FC, fully connected layers). It may be a neural network consisting of. However, the present invention is not limited thereto, and the number of convolutional layers and the number of fully connected layers in the third sub neural network 61 may be applied in various ways.

In addition, the latent feature difference 50 used when the integrated combined feature 43 is generated may be a compressed latent feature difference 50 ′ in which feature compression is performed by passing through the fourth sub neural network 62.

The compressed features 40 (41, 42) output from the first neural network 20 and the latent feature difference 50 output from the second neural network 30 may have different dimensions. Therefore, in order to match the dimensions between the features that are the outputs of the two neural networks 20 and 30, the detection unit 140 determines the latent feature difference 50 output from the second neural network 30 to the fourth sub neural network 62. Can be provided as input. Through this, the potential feature difference 50 may be compressed by the fourth sub neural network 62, and the compressed potential feature difference 50 ′ may be extracted from the fourth sub neural network 62.

In this case, the fourth sub neural network 62 used to extract the compressed latent feature difference 50 ′ may be formed of three convolutional layers (conv), but is not limited thereto, The type and number of layers can be applied in various ways.

The detection unit 140 may detect forgery or alteration by using an integrated combination feature 43 that combines the compressed features 40 (41, 42) having the same dimension and the compressed latent feature difference 50.

When the calculated integrated combined feature related score 43s is equal to or greater than a preset score, the detection unit 140 may determine that the face regions s1 and s2 in the captured image 10 will not be forged or altered. That is, when the score 43s related to the integrated combination feature is equal to or greater than a preset score, the detection unit 140 indicates that the face regions s1 and s2 in the captured image 10 are the real faces of the object that has not been forged or altered. It can be judged as. In this case, the detection unit 140 may output a real face as a result 141 of whether the forgery or alteration has occurred.

On the other hand, when the calculated integrated combined feature related score 43s is less than a preset score, the detection unit 140 may determine that the face regions s1 and s2 in the captured image 10 have been forged. That is, when the integrated combination feature related score 43s is less than a preset score, the detection unit 140 is a fake face of an object in which the face regions s1 and s2 in the captured image 10 have been forged and altered. It can be determined to be. In this case, the detection unit 140 may output a false face as a result 141 of whether a forgery or alteration has occurred.

Referring to FIG. 4, a second score calculation method (second method) by the detection unit 140 is as follows. The detection unit 140 may calculate a first score 44s related to the combined feature by using the combined feature 44 that combines the first view compression feature 41 and the second view compression feature 42. In addition, the detection unit 140 may calculate a second score 50s related to the latent feature difference by using the latent feature difference 50. Thereafter, the detection unit 140, based on the sum score 55s obtained by summing the calculated first score 44s and the calculated second score 50s, the captured image 10 (especially, the face area in the captured image). It is possible to detect whether forgery or alteration of

In this case, the sum score 55s may be a score obtained by an average of the first score 44s and the second score 50s. That is, the detection unit 140 may detect whether or not forgery or alteration is based on the average score 55s obtained by averaging the first score 44s and the second score 50s.

The detection unit 140 may provide the combined feature 44 as an input of the third sub neural network 61 ′ and calculate a first score 44 s related to the combined feature from the output of the third sub neural network 61 ′. .

Here, the third sub-neural network 61' used when calculating the first score 44s is, for example, two convolutional layers (conv) and three fully connected layers (fully connected, FC, fully connected layers). It may be a constructed neural network. However, the present invention is not limited thereto, and the number of convolutional layers and the number of fully connected layers in the third sub neural network 61 ′ may be applied in various ways.

In addition, the detection unit 140 provides the latent feature difference 50 as an input of the fourth sub neural network 62 ′, and calculates a second score 50 s related to the latent feature difference from the output of the fourth sub neural network 62 ′. Can be calculated.

Here, the fourth sub neural network 62' used when calculating 2 scores (50s) is composed of 4 convolutional layers (conv) and 3 fully connected layers (fully connected, FC, fully connected layers), for example. It could be a neural network. However, the present invention is not limited thereto, and the number of convolutional layers and the number of fully connected layers in the fourth sub neural network 62 ′ may be applied in various ways.

Accordingly, as the latent feature difference 50 is provided as an input of the fourth sub-neural network 62', feature compression and compression for compressing the feature of the latent feature difference 50 through the fourth sub-neural network 62' The second score 50s may be calculated based on the specified characteristics. The detection unit 140 may calculate a second score 50s related to the latent feature difference using the fourth sub neural network 62 ′.

Thereafter, the detection unit 140 may detect whether or not forgery or alteration is based on the average score 55s obtained by summing the calculated first score 44s and the second score 50s.

When the sum score (or average score) 55s is equal to or greater than a preset score, the detection unit 140 may determine that the face regions s1 and s2 in the captured image 10 are not forged or altered. That is, when the sum score 55s is equal to or greater than a preset score, the detection unit 140 may determine that the face regions s1 and s2 in the captured image 10 are the real faces of the object that has not been forged or altered. I can. In this case, the detection unit 140 may output a real face as a result 141 of whether the forgery or alteration has occurred.

On the other hand, when the sum score 55s is less than a preset score, the detection unit 140 may determine that the face regions s1 and s2 in the captured image 10 have been forged. That is, when the sum score 55s is less than a preset score, the detection unit 140 determines that the face regions s1 and s2 in the captured image 10 are the fake faces of the forged object. can do. In this case, the detection unit 140 may output a false face as a result 141 of whether a forgery or alteration has occurred.

In this case, the fact that the forgery or alteration result 141 is a real face means that the photographed image 10 acquired through the dual camera 1 represents the face of an actual object (person) as shown in FIG. 5A. As the image is acquired by photographing, it may mean that the identified face regions s1 and s2 in the acquired captured image 10 correspond to the face of an actual object (person).

On the other hand, the fact that the forgery or alteration result 141 is a false face means that the captured image 10 acquired through the dual camera 1 is an object printed on paper as shown in FIG. ), the identified face regions (s1, s2) in the acquired photographed image 10 are forged (forged or altered) not the face of the actual object (person). It may mean that it corresponds to a fake face).

As another example, the fact that the forgery or alteration result 141 is a false face means that the captured image 10 acquired through the dual camera 1 is displayed on the screen of the mobile terminal as shown in FIG. 5(c). As the image is acquired by photographing the face of the displayed object (person), the identified face regions (s1, s2) in the acquired photographed image 10 are forged (forged or altered) other than the face of the actual object (person). It may mean that it corresponds to a false face (fake face).

Here, the mobile terminal may be a smartphone, a tablet PC, etc. as an example, but is not limited thereto, and a display (providing) of a face image or a face shape (for example, a face shape provided using a laser beam) of the object Various possible terminals can be applied.

According to this, the detection unit 140 may be configured such that the acquired photographed image 10 is an image photographing the face of the object output on paper, an image photographing the face of the object output on a high-resolution screen, or In the case of an image obtained by photographing a face model of the produced object, it may be detected that the face of the object included in the photographed image 10 is a false face rather than a real face. That is, the detection unit 140 results in a forgery or alteration result. (141) can be output as a false face.

As shown in (a) of FIG. 5, when the captured image 10 acquired through the dual camera 1 is the captured image 10 of an actual face, the object in the preprocessed first viewpoint image 11' There may be a difference between a line of sight (a line of sight direction) of and a line of sight (a line of sight direction) of an object in the preprocessed second viewpoint image 12 ′.

On the other hand, as shown in (b) or (c) of FIG. 5, when the captured image 10 acquired through the dual camera 1 is a photographed image of a false face, the gaze of the object in the preprocessed first viewpoint image There may be no difference between the (line of sight direction) and the line of sight of the object in the preprocessed second viewpoint image (gaze direction).

In other words, when the photographed image 10 acquired through the dual camera 1 is the photographed image 10 of an actual face, a potential difference exists and appears to be a potential feature difference, whereas the dual camera 1 If the photographed image 10 obtained through) is a photographed image in which a false face is photographed, it may appear that there is little potential difference and thus there is no difference in potential features.

In consideration of this, the device 100 includes a first viewpoint image 11 (left image) and a second viewpoint image 12 (right image) acquired using the dual camera 1, that is, the left/right images 11 and 12. Using, it is possible to detect whether the acquired image 10 is a face of an actual object (person) or a false face of a forged object (person).

The first neural network 20 considered herein receives a plurality of photographed images (multiple learning photographed images) obtained by photographing face portions of a plurality of objects using the dual camera 1 as inputs, and each of the plurality of photographed images It may be a pre-trained neural network to output a result of forgery or alteration matching (ie, a result of whether a real face or a fake face) is output.

Likewise, the second neural network 20 considered in the present application takes as an input the potential feature difference of each of a plurality of photographed images (multiple learning photographed images) obtained by photographing face parts of a plurality of objects using the dual camera 1. And, it may be a pre-trained neural network to output a forgery/modulation result (ie, a result value of whether it is a real face or a fake face) that matches the potential feature difference of each of the plurality of captured images.

In the case of an image photographing an actual face, the present application focuses on the fact that the object's gaze (direction of gaze) differs in each image 11 and 12, so that each image 11 and 12 acquired from the dual camera 1 is used. By using a previously learned neural network, it is possible to detect forgery or alteration.

The present application may build a large-capacity database for detecting forgery or alteration based on a larger number of objects (subjects, people), and detect forgery or alteration based on this. The present application allows the present apparatus 100 to learn a potential difference (that is, a potential feature difference) between the left and right images 11 and 12 based on the structure of the deep neural network 30, based on the structure of the deep neural network 30. Suggest.

The apparatus 100 may effectively improve detection performance by learning the degree of difference between the left/right images 11 and 12 using a latent space through learning of a deep neural network. In other words, the present apparatus 100 uses a second neural network in which a plurality of potential feature differences 50 extracted based on photographed images of a plurality of subjects are learned, thereby taking a picture acquired in real time from the dual camera 1. Whether the image 10 is forged or altered can be effectively detected.

Here, the latent space may mean a space in which features of the preprocessed first viewpoint image 11 ′ and features of the preprocessed second viewpoint image 12 ′ are compressed, that is, a space in which a feature compression process is performed. The latent feature difference 50 may mean a difference between the compressed features related to the preprocessed first view image 11 ′ and the compressed features related to the preprocessed second view image 12 ′ in such a latent space. .

The device 100 identifies (detects) a face area through a preprocessor (not shown) with respect to the left/right face images 11 and 12 acquired from the dual camera 1, and Preprocessing to normalize the image size can be performed. Thereafter, in order to extract features from the pre-processed photographed image 10, the apparatus 100 separates each image 11 12 into a separate deep neural network, that is, the first sub neural network 21 in the first neural network 20. And the second sub neural network 22 as inputs.

In addition, in order to effectively predict the difference in potential features of the left and right images 11 and 12, the present apparatus 100 uses the second neural network 30 as a separate difference information compression neural network, not a loss function-based method used in the prior art. Was constructed (prepared).

In order to combine the compressed feature 40, which is the compressed information extracted through the first neural network 20, and the latent feature difference 50, which is the potential difference information extracted through the second neural network 20, the apparatus 100 The deep neural network structure of the branch method (the first method and the second method described above) can be used.

At this time, the deep neural network structure of the first method is a structure of a method of learning by simply concatenating each compressed information (ie, a difference between a compressed feature and a latent feature), which may mean a structure as shown in FIG. 3. have. That is, FIG. 3 may refer to a structure of a deep neural network of the first method used by the detection unit 140 in the device 100 when calculating a score. This first method can be expressed differently as an early fusion method.

In this case, in the case of the first method, the device 100 additionally includes a convolutional layer and a fully connected layer (fully connected layer) in order to calculate a score for the combined information (ie, integrated combination feature). 3 A sub neural network 61 can be provided. Using the third sub-neural network 61, the apparatus 100 may calculate a score for an actual face and a false face that is a forged face with respect to the acquired captured image 10. Based on the calculated score, a forgery or alteration result 141 may be derived.

On the other hand, the deep neural network structure of the second method uses only compressed information from the left/right images 11 and 12 (i.e., using only the first view compression feature and the second view compression feature) to learn whether or not forgery is altered. Neural network (this may mean a neural network including a first neural network and a third sub neural network) and a neural network that learns a potential difference feature (a potential feature difference) (this is a neural network including a second neural network and a fourth sub neural network) It is a structure of a method of separately configuring) and using the average of the final scores (ie, the first score and the second score) of each neural network, which may mean a structure as shown in FIG. 4. That is, FIG. 4 may mean the structure of the second type of deep neural network used by the detection unit 140 in the present apparatus 100 to calculate a score. This second method may be differently expressed as a late fusion method.

In this case, in the case of the second method, the device 100 calculates the first score 44s using the first neural network 20 and the third sub neural network 61 ′, and the second neural network 30 and the second neural network 30 4 A second score 50s may be calculated using the sub neural network 62'. Thereafter, the device 100 is based on the average score obtained by averaging the calculated two scores (44s, 50s), for the acquired captured image 10, the score for the real face and the false face that is a forged face (Score ) Can be calculated. Based on the calculated score, a forgery or alteration result 141 may be derived.

According to an experimental example of the present application, in order to verify the performance of the present apparatus 100, the present application detects forgery or alteration using a database of 50 persons produced by itself. The produced database may be included in the photographed images of 29 men and 21 women, as well as the results of forgery or alteration.

In one experiment of the present application, the model oCamS-1CGN-U) is used as the dual camera 1 (or stereo camera), and the resolution of the captured image 10 obtained through this may be 1280 × 960 pixels. In one experiment of the present application, a face region in the captured image 10 obtained through the dual camera 1 is identified (detected) using a conventionally known image analysis algorithm (face identification algorithm), and the identified face region is 128 Pre-processing of normalizing to a pixel size of x 128 was performed, and the pre-processed photographed image was used as an input of the first neural network 20. 5 shows a process of detecting a face area from a captured image 10 acquired from the dual camera 1.

6 is a diagram showing a performance evaluation result of the face forgery detection apparatus 100 according to an embodiment of the present application.

In FIG. 6, the Proposed method refers to a method proposed in the present application, that is, a forgery detection technology by the present apparatus 100. In addition, as a conventional forgery detection technique [1] compared to the performance evaluation of the present apparatus 100 in FIG. 6, the conventionally known document [X. Sun, L. Huagn, and C. Liu, "Dual Camera Based Feature For Face Spoofing Detection," in Proc. Chinese Conference Pattern Recognition., Chengdu, China, Nov. 2016, pp. 332 344.] was considered.

In addition, in FIG. 6, 22-fold may mean an experiment based on 22 subjects, and 50-forld may mean an experiment based on 50 subjects. In addition, in FIG. 6, early fusion may mean an experiment based on the first method of the device 100, and late fusion may mean an experiment based on the second method of the device 100. In addition, EER (Equal Error Rate) means the same error rate, and a smaller value may mean better performance.

In one experiment of the present application, as an example, Intel Xeon CPU E5-1650 and NVIDIA TITAN Xp GPU were used to evaluate the performance of the device 100.

Referring to FIG. 6, as a result of an experiment of the present application, the performance of the technology proposed by the present application (that is, the performance of the detection result of forgery or alteration by the device) is compared to the conventional forgery detection technology. ), it can be seen that it shows excellent performance.

In addition, according to the experimental results of the present application, in the case of the device 100, the forgery detection performance using the second method (late fusion), which is a score combining method, is 0.652%. It can be seen that the performance is better than fusion).

According to the performance evaluation result according to the above-described experiment of the present application, it can be confirmed that the technique proposed by the present apparatus 100 can be effectively applied to detection of forgery or alteration of a face. That is, it can be seen that the method proposed in the present application (a method of separately learning a potential feature difference and using it to detect forgery) is more effective when detecting whether a face is forged or altered compared to the conventional technology.

The device 100 may detect whether a high-performance face is forged or altered based on the left and right images 11 and 12. In addition, the present apparatus 100 can implicitly learn the difference due to the three-word structure.

The apparatus 100 may detect whether a face has been forged or altered by using the left and right images 11 and 12 acquired from the dual camera 1 and the difference (ie, potential feature difference).

The technology proposed in the present application can be effectively applied and utilized in the field of application of security enhancement using face images in various embedded systems (eg, smartphones, kiosks, etc.).

Hereinafter, based on the details described above, the operation flow of the present application will be briefly described.

7 is a flowchart illustrating an operation of a method for detecting forgery and alteration of a face according to an embodiment of the present application.

The method for detecting face forgery and alteration illustrated in FIG. 7 may be performed by the apparatus 100 described above. Accordingly, even if the contents are omitted below, the contents described with respect to the apparatus 100 may be equally applied to the description of the method for detecting face forgery and alteration.

Referring to FIG. 7, in step S11, the acquisition unit 110 may acquire a photographed image of a face portion of an object through a dual camera.

In this case, the captured image may include a first viewpoint image and a second viewpoint image.

Next, in step S12, the first extracting unit 120 may extract a compression feature related to the face region in the captured image by using the captured image acquired in step S11.

In this case, in step S12, the first extraction unit 120 may receive the pre-processed captured image from the acquisition unit 110, and may extract a compression feature using the pre-processed captured image. Accordingly, in a description to be described later, the first viewpoint image and the second viewpoint image considered when extracting the compressed feature and the latent feature difference may be a preprocessed first viewpoint image and a preprocessed second viewpoint image, respectively.

In step S12, the first extraction unit 120 provides a first viewpoint image (especially, a preprocessed first viewpoint image) and a second viewpoint image (especially, a preprocessed second viewpoint image) as inputs of the first neural network. , Compression including a first view compression feature corresponding to a first view image (preprocessed first view image) and a second view compression feature corresponding to a second view image (preprocessed second view image) from the first neural network Features can be extracted.

Next, in step S13, the second extraction unit 130 may extract a difference of latent features by using the captured image acquired in step S11.

In this case, in step S13, the second extraction unit 130 may extract a potential feature difference between the first viewpoint image and the second viewpoint image by using the first viewpoint image and the second viewpoint image. In particular, the second extraction unit 130 may extract a latent feature difference between the preprocessed first viewpoint image and the preprocessed second viewpoint image.

Here, the latent feature difference is the feature of the first view image and the feature of the second view image occurring in a process in which the feature of the first view image and the feature of the second view image are compressed in the first neural network (feature compression process). It may be obtained by passing a difference feature, which is a difference between, through the second neural network.

Next, in step S14, the detection unit 140 may detect whether the face region in the captured image is forged or altered by using the difference between the compressed feature extracted in step S12 and the latent feature extracted in step S13.

In addition, in step S14, the detection unit 140 calculates a score for the captured image by using the first view compression feature, the second view compression feature, and the latent feature difference, and based on the calculated score, the identified face in the captured image It is possible to detect whether an area has been forged or altered.

In addition, in step S14, the detection unit 140 generates an integrated combination feature by combining the first view compression feature, the second view compression feature, and the latent feature difference, and relates the integrated combination feature calculated using the generated integrated combination feature. It is possible to detect forgery or alteration based on the score.

In addition, in step S14, the detection unit 140 calculates a first score related to the combined feature by using the combined feature that combines the first view compression feature and the second view compression feature, and uses the latent feature difference to calculate a potential feature difference related product. 2 A score is calculated, and forgery or alteration may be detected based on a sum score obtained by summing the first score and the second score.

Here, the sum score may be a score obtained by an average of the first score and the second score.

In the above description, steps S11 to S14 may be further divided into additional steps or may be combined into fewer steps, depending on the embodiment of the present application. In addition, some steps may be omitted as necessary, or the order between steps may be changed.

The method for detecting forgery and alteration of a face according to an exemplary embodiment of the present disclosure may be implemented in the form of a program command that can be executed through various computer means and recorded in a computer-readable medium. The computer-readable medium may include program instructions, data files, data structures, etc. alone or in combination. The program instructions recorded on the medium may be specially designed and configured for the present invention, or may be known and usable to those skilled in computer software. Examples of computer-readable recording media include magnetic media such as hard disks, floppy disks, and magnetic tapes, optical media such as CD-ROMs and DVDs, and magnetic media such as floptical disks. -A hardware device specially configured to store and execute program instructions such as magneto-optical media, and ROM, RAM, flash memory, and the like. Examples of program instructions include not only machine language codes such as those produced by a compiler, but also high-level language codes that can be executed by a computer using an interpreter or the like. The hardware device described above may be configured to operate as one or more software modules to perform the operation of the present invention, and vice versa.

In addition, the above-described face forgery detection method may be implemented in the form of a computer program or application executed by a computer stored in a recording medium.

The foregoing description of the present application is for illustrative purposes only, and those of ordinary skill in the art to which the present application pertains will be able to understand that it can be easily modified into other specific forms without changing the technical spirit or essential features of the present application. Therefore, it should be understood that the embodiments described above are illustrative and non-limiting in all respects. For example, each component described as a single type may be implemented in a distributed manner, and similarly, components described as being distributed may also be implemented in a combined form.

The scope of the present application is indicated by the claims to be described later rather than the detailed description, and all changes or modified forms derived from the meaning and scope of the claims and their equivalent concepts should be interpreted as being included in the scope of the present application.

100: face forgery detection device
110: acquisition unit
120: first extraction unit
130: second extraction unit
140: detection unit

Claims (14)

As a face forgery detection method,
(a) obtaining a photographed image of the face of the object through a dual camera;
(b) extracting a compression feature related to a face region in the captured image by using the captured image;
(c) extracting a difference of latent features using the captured image; And
(d) detecting whether the face region in the photographed image has been forged or altered using the difference between the compressed feature and the latent feature,
Face forgery detection method comprising a. The method of claim 1,
In the step (a), the captured image includes a first viewpoint image and a second viewpoint image,
The step (b),
A first viewpoint compression feature corresponding to the first viewpoint image and a second viewpoint corresponding to the second viewpoint image from the first neural network by providing the first viewpoint image and the second viewpoint image as inputs of a first neural network To extract the compression feature including the compression feature, facial forgery detection method. The method of claim 2,
The step (c),
And extracting a potential feature difference between the first viewpoint image and the second viewpoint image by using the first viewpoint image and the second viewpoint image. The method of claim 3,
The latent feature difference in step (c) is,
The difference feature, which is the difference between the feature of the first view image and the feature of the second view image, generated when the feature of the first view image and the feature of the second view image is compressed in the first neural network is a second difference feature. To be obtained by passing through a neural network, face forgery detection method. The method of claim 2,
The step (d),
A score for the captured image is calculated using the first view compression feature, the second view compression feature, and the latent feature difference, and whether or not the face region in the captured image is forged or altered based on the calculated score. To detect, face forgery detection method. The method of claim 5,
The step (d),
Whether the first view compression feature, the second view compression feature, and the latent feature difference are combined to generate an integrated combination feature, and whether the forgery or alteration is based on a score related to the integrated combination feature calculated using the generated integrated combination feature. To detect, face forgery detection method. The method of claim 5,
The step (d),
A first score related to a combined feature is calculated using a combined feature obtained by combining the first view compression feature and the second view compression feature,
A second score related to the latent feature difference is calculated using the latent feature difference,
The method of detecting forgery or alteration of the face based on a sum score obtained by adding the first score and the second score. The method of claim 7,
In the step (d), the summed score is a score obtained by averaging the first score and the second score. As a face forgery detection device,
An acquisition unit that acquires a photographed image of a face portion of the object through the dual camera;
A first extracting unit for extracting a compression feature related to a face region in the captured image using the captured image;
A second extraction unit for extracting a difference of latent features using the captured image; And
A detection unit that detects whether the face region in the photographed image has been forged or altered using the difference between the compression feature and the latent feature,
Face forgery detection device comprising a. The method of claim 9,
The captured image includes a first viewpoint image and a second viewpoint image,
The first extraction unit,
A first viewpoint compression feature corresponding to the first viewpoint image and a second viewpoint corresponding to the second viewpoint image from the first neural network by providing the first viewpoint image and the second viewpoint image as inputs of a first neural network To extract the compression feature including the compression feature, facial forgery detection device. The method of claim 10,
The second extraction unit,
And extracting a potential feature difference between the first viewpoint image and the second viewpoint image by using the first viewpoint image and the second viewpoint image. The method of claim 11,
The latent feature difference is,
The difference feature, which is the difference between the feature of the first view image and the feature of the second view image, generated when the feature of the first view image and the feature of the second view image is compressed in the first neural network is a second difference feature. It is obtained by passing through a neural network, facial forgery detection device. The method of claim 10,
The detection unit,
Calculating a score for the captured image using the first viewpoint feature, the second viewpoint feature, and the latent feature difference, and detecting whether the face region in the photographed image has been forged or altered based on the calculated score. That, face forgery detection device. A computer-readable recording medium storing a program for executing the method of claim 1 on a computer.

Images