KR102612941B1 - System of verifying digital synthesis/counterfeit face - Google Patents

Abstract

A digital synthetic/fake face verification system is disclosed. A facial image registration module that registers the user's facial image; a classifier generation module that pre-generates a classifier for recognizing a user using a facial image pre-stored in the facial image storage module; The classifier pre-generated by the classifier creation module constitutes a user-specific classifier storage module in which the classifier is pre-stored for each user. According to the above-described digital synthetic/fake face verification system, it is configured to accurately distinguish between real facial images and digital synthetic/fake facial images by applying a network embedding algorithm to facial recognition, thereby preventing spoofing by synthesis/falsification. It has the effect of strengthening security. In particular, it is configured to generate learning images by applying Gaussian noise, Gaussian blur, compression, etc. to learn facial images, which has the effect of further increasing the accuracy of distinguishing between real images and synthetic/fake images. In addition, it is configured to learn using blind images of specific parts of the face that cover specific parts of the face, such as the eyes, nose, and mouth, to prevent overfitting of learning due to specific patterns of the face, reduce recognition errors in facial images, and synthesize /It has the effect of being able to more accurately distinguish counterfeit images.

Description

Digital synthetic face and digital counterfeit face verification system {SYSTEM OF VERIFYING DIGITAL SYNTHESIS/COUNTERFEIT FACE}

The present invention relates to a digital facial recognition system, and specifically to a digital synthesis/falsification face verification system, and more specifically to a real face and synthesis/falsification using network embeddings technology. It is about a system that distinguishes and verifies faces (fake faces).

Facial image recognition technology is used in various devices such as access devices, laptops, and smart phones.

As the use of these technologies expands, spoofing through the use of synthetic/fake faces is also increasing.

Digital synthetic/fake facial images are becoming more sophisticated, and the need for means to accurately identify them is increasing.

Figure 1 shows an actual image (original image) and a digital composite/fake image, and shows that an image can be created by forging a part of the face or compositing the superior/inferior features.

However, as there is still no means to accurately distinguish digitally synthesized/fake faces, the risk of spoofing is increasing.

Public Patent Publication 10-2004-0037180 Public Patent Publication 10-2018-0135898

The purpose of the present invention is to provide a digital synthetic/falsified face verification system.

The digital synthetic/falsified face verification system according to the object of the present invention described above includes a facial image registration module for registering a user's facial image; a classifier generation module that pre-generates a classifier for recognizing a user using a facial image pre-stored in the facial image storage module; The classifier pre-generated by the classifier creation module may be configured to include a user-specific classifier storage module that is pre-stored for each user.

Here, it may be configured to further include a facial recognition module that verifies the user of the facial image received from the facial image receiving module using the classifier stored in the user-specific classifier storage module.

And it may be configured to further include a facial recognition result output module that outputs the user identified in the facial recognition module or the corresponding recognition result.

According to the above-described digital synthetic/fake face verification system, it is configured to accurately distinguish between real facial images and digital synthetic/fake facial images by applying a network embedding algorithm to facial recognition, thereby preventing spoofing by synthesis/falsification. It has the effect of strengthening security.

In particular, it is configured to generate learning images by applying Gaussian noise, Gaussian blur, compression, etc. to learn facial images, which has the effect of further increasing the accuracy of distinguishing between real images and synthetic/fake images.

In addition, it is configured to learn using blind images of specific parts of the face that cover specific parts of the face, such as the eyes, nose, and mouth, to prevent overfitting of learning due to specific patterns of the face, reduce recognition errors in facial images, and synthesize /It has the effect of being able to more accurately distinguish counterfeit images.

1 is an exemplary diagram of a digitally synthesized/falsified facial image.
Figure 2 is a block diagram of a digital composite/fake face verification system according to an embodiment of the present invention.
Figure 3 is a schematic diagram of digital synthesis/fake face verification according to an embodiment of the present invention.
Figure 4 is a graph showing network embedding technology.
Figure 5 is a graph showing the verification results of a digitally synthesized/fake face using a network embedding algorithm according to an embodiment of the present invention.
Figure 6 is a schematic diagram showing a digital synthesis/falsification face algorithm using a network embedding algorithm according to an embodiment of the present invention.
Figure 7 is an example diagram of a learning image according to an embodiment of the present invention.
Figure 8 is an example diagram of a compressed image according to an embodiment of the present invention.
Figure 9 is an example diagram of a facial landmark image according to an embodiment of the present invention.
Figure 10 is an example of a specific region blind facial image according to an embodiment of the present invention.

Since the present invention can make various changes and have various embodiments, specific embodiments will be illustrated in the drawings and detailed descriptions will be given for carrying out the invention. However, this is not intended to limit the present invention to specific embodiments, and should be understood to include all changes, equivalents, and substitutes included in the spirit and technical scope of the present invention. While describing each drawing, similar reference numerals are used for similar components.

Terms such as first, second, A, and B may be used to describe various components, but the components should not be limited by the terms. The above terms are used only for the purpose of distinguishing one component from another. For example, a first component may be named a second component, and similarly, the second component may also be named a first component without departing from the scope of the present invention. The term and/or includes any of a plurality of related stated items or a combination of a plurality of related stated items.

When a component is said to be "connected" or "connected" to another component, it is understood that it may be directly connected to or connected to the other component, but that other components may exist in between. It should be. On the other hand, when it is mentioned that a component is “directly connected” or “directly connected” to another component, it should be understood that there are no other components in between.

The terms used in this application are only used to describe specific embodiments and are not intended to limit the invention. Singular expressions include plural expressions unless the context clearly dictates otherwise. In this application, terms such as “comprise” or “have” are intended to designate the presence of features, numbers, steps, operations, components, parts, or combinations thereof described in the specification, but are not intended to indicate the presence of one or more other features. It should be understood that this does not exclude in advance the possibility of the existence or addition of elements, numbers, steps, operations, components, parts, or combinations thereof.

Unless otherwise defined, all terms used herein, including technical or scientific terms, have the same meaning as commonly understood by a person of ordinary skill in the technical field to which the present invention pertains. Terms defined in commonly used dictionaries should be interpreted as having a meaning consistent with the meaning in the context of the related technology, and unless explicitly defined in the present application, should not be interpreted in an ideal or excessively formal sense. No.

Hereinafter, preferred embodiments according to the present invention will be described in detail with reference to the attached drawings.

Figure 2 is a block diagram of a digital composite/fake face verification system according to an embodiment of the present invention. And Figure 3 is a schematic diagram of digital synthesis/falsification face verification according to an embodiment of the present invention, Figure 4 is a graph showing network embedding technology, and Figure 5 is a digital synthesis using a network embedding algorithm according to an embodiment of the present invention. It is a graph showing the verification results of a synthetic/fake face, Figure 6 is a schematic diagram showing a digital synthesis/fake face algorithm using a network embedding algorithm according to an embodiment of the present invention, and Figure 7 is a learning according to an embodiment of the present invention. Figure 8 is an example of an image according to an embodiment of the present invention, Figure 9 is an example of a compressed image according to an embodiment of the present invention, Figure 9 is an example of a facial landmark image according to an embodiment of the present invention, and Figure 10 is an example of a compressed image according to an embodiment of the present invention. This is an example of a specific area blind facial image according to .

Referring to FIG. 2, the digital composite/fake face verification system 100 according to an embodiment of the present invention includes a user registration module 101, a user information storage module 102, a facial image registration module 103, and a facial image. Storage module 104, classifier generation module 105, user-specific classifier storage module 106, learning image addition module 107, blind facial image addition module 108, learning image database 109 ), a classifier learning/addition module 110, a facial image reception module 111, a facial recognition module 112, and a facial recognition result output module 113.

And in Figure 3, the present invention basically includes 1) face detection, 2) feature vector detection using CNN algorithm and low-dimensional mapping using network embeddings, 3) classifier generation, and 4) real image and digital synthesis/forgery. It indicates that the images are organized in the order of distinct output.

Hereinafter, the detailed configuration of FIG. 2 will be described.

The user registration module 101 may be configured to receive and register user information. User information may include the user's name, ID, employee number, etc.

The user information storage module 102 may be configured to store user information registered by the user registration module 101.

The facial image registration module 103 may be configured to register a user's facial image. The facial image registration module 103 may be configured to register a facial image in response to user information stored in the user information storage module 102.

The facial image storage module 104 may be configured to pre-store the facial images registered in the facial image registration module 103 for each user information stored in the user information storage module 102.

The classifier creation module 105 may be configured to pre-generate a classifier for recognizing a user using a facial image pre-stored in the facial image storage module 104.

Here, the classifier may be configured to recognize the user by comparing the facial image received by the facial image receiving module 111 for login with the facial image previously stored in the facial image storage module 104.

The classifier generation module 105 may be configured to include a first CNN feature point extraction unit 105a, a first network embedding low-dimensional mapping unit 105b, and a classifier generation unit 105c.

Hereinafter, the detailed configuration will be described.

The first CNN feature point extraction unit 105a may be configured to extract feature points on the facial image stored in the facial image storage module 104 and align the face based on the extracted feature points. And it can be configured to extract 512 feature vectors from the sorted faces using a CNN algorithm.

The first network embedding low-dimensional mapping unit 105b may be configured to map the feature vector extracted from the first CNN feature point extraction unit 105a into a low-dimensional space using a network embeddings algorithm.

Figure 4 is used to classify various types of data such as people, faces, and book types by network embedding. However, in the present invention, as shown in FIG. 5, it can be configured to be used to distinguish between a real image and a digitally synthesized/falsified image.

Referring to Figure 5, when the feature vector in which the image features are quantified by the CNN algorithm is mapped into a low-dimensional space, the actual facial image and the digitally synthesized/fake facial image appear separately as shown in Figure 5, and through this, the digital synthesis is performed. /Can detect fake facial images.

Figure 6 shows the network embedding algorithm, and the learning process using network embedding can ultimately be configured to learn the Triplet Loss so that the distance between the anchor and the positive is minimized. Here, Triplet Loss is It can be defined as, and the goal is to learn so that the distance from positive becomes closer and the distance from negative becomes farther. Positive can be seen as a real image, and negative can be seen as a digitally synthesized/fake image. Previously, if Triplet Loss was a negative number, it could be configured to treat it as 0 and not learn weights. However, in the present invention, as shown on the right side of FIG. 6, when Loss is negative, the distance between positives and It can be configured to improve learning speed and prevent local minima by using .

The basic classifier generator 105c may be configured to generate a basic classifier composed of feature vectors mapped to a low-dimensional space in the first network embedding low-dimensional mapping unit 105b.

The user-specific classifier storage module 106 may be configured to pre-store the classifier pre-generated by the classifier creation module 105 for each user.

The learning image adding module 107 may be configured to add a learning image using the facial image registered by the facial image registration module 103.

The training image addition module 107 may be configured to include a Gaussian noise addition unit 107a, a Gaussian blur addition unit 107b, a Gaussian noise/blur addition unit 107c, and an image compression unit 107d.

Hereinafter, the detailed configuration will be described.

The Gaussian noise adding unit 107a may be configured to generate and add a learning image by adding Gaussian noise to the facial image registered in the facial image registration module 103.

The Gaussian blur adding unit 107b may be configured to generate and add a learning image by adding Gaussian blur to the facial image registered in the facial image registration module 103.

The Gaussian noise/blur adding unit 107c may be configured to generate and add a learning image by adding Gaussian noise and Gaussian blur to the facial image registered in the facial image registration module 103.

The image compression unit 107d may be configured to compress and process the facial image registered in the facial image registration module 103 to generate and add a learning image.

Figure 7 illustrates a training image with Gaussian noise and Gaussian blur applied, and Figure 8 illustrates a training image with compression. Through this, the accuracy of distinguishing between digitally synthesized/forged images can be increased.

The blind facial image addition module 108 may be configured to add a blind facial image using the facial image registered by the facial image registration module 103.

The blind facial image addition module 108 may be configured to include a landmark detection unit 108a and a specific blind facial image generating unit 108b.

Hereinafter, the detailed configuration will be described.

The landmark detection unit 108a may be configured to detect 68 landmarks for key parts of the face from the learning image and blind face image stored in the learning image database 109.

The specific area blind facial image generating unit 108b may be configured to generate a blind facial image covering a predetermined area of the face based on the landmark detected by the landmark detection unit 108a.

Figure 9 shows a facial landmark image and Figure 10 illustrates a specific region blind facial image. Through this, it is possible to prevent overfitting of learning by patterns of specific parts of the face during the learning process and increase the accuracy of distinguishing between digitally synthesized/falsified images.

The learning image database 109 may be configured to store the learning images added by the learning image adding module 107 and the blind facial images added by the blind facial image adding module 108.

The classifier learning/adding module 110 can be configured to learn using the learning image and blind face image stored in the learning image database 109, add a classifier for each user, and additionally store it in the user-specific classifier storage module 106. there is.

The classifier learning/addition module 110 may be configured to include a second CNN feature point extraction unit 110a, a second network embedding low-dimensional mapping unit 110b, and a synthesis/falsification classifier addition unit 110c.

Hereinafter, the detailed configuration will be described.

The second CNN feature point extraction unit 110a extracts feature points on the learning image or blind face image from the learning image database 109, sorts the face based on the extracted feature points, and uses the CNN algorithm on the aligned face 512. It can be configured to extract feature vectors.

The second network embedding low-dimensional mapping unit 110b may be configured to map the feature vector extracted from the second CNN feature point extraction unit 110a into a low-dimensional space using a network embeddings algorithm.

The synthesis/forgery classifier adding unit 110c may be configured to add a synthesis/forgery classifier for each user consisting of feature vectors mapped to a low-dimensional space in the second network embedding low-dimensional mapping unit 110b.

The facial image receiving module 111 may be configured to receive a facial image for user verification.

The facial recognition module 112 may be configured to identify the user of the facial image received from the facial image receiving module 111 using the classifier stored in the user-specific classifier storage module 106.

The face recognition module 112 may be configured to include a face recognition unit 112a using a basic classifier and a face recognition unit 112b using a synthesis/forgery classifier.

Hereinafter, the detailed configuration will be described.

First, the facial recognition unit 112a utilizing a basic classifier may be configured to recognize the facial image received from the facial image receiving module 111 using the basic classifier generated by the basic classifier generating unit 105c.

Secondly, the facial recognition unit 112b utilizing the synthesis/forgery classifier uses the synthesis/forgery classifier added in the synthesis/forgery classifier addition unit 110c to synthesize/forge the facial image received from the facial image reception module 111. It can be configured to distinguish and recognize counterfeiting.

The facial recognition result output module 113 may be configured to output the user identified in the facial recognition module 112 or the corresponding recognition result.

Although the description has been made with reference to the above examples, those skilled in the art can understand that various modifications and changes can be made to the present invention without departing from the spirit and scope of the present invention as set forth in the claims below. There will be.

101: User registration module
102: User information storage module
103: Facial image registration module
104: Facial image storage module
105: Classifier creation module
105a: First CNN feature point extraction unit
105b: First network embedding low-dimensional mapping unit
105c: Basic classifier creation unit
106: User-specific sorter storage module
107: Learning image addition module
107a: Gaussian noise addition
107b: Gaussian blur addition
107c: Gaussian noise/blur addition
107d: image compression unit
108: Blind facial image addition module
108a: Landmark detection unit
108b: Specific area blind facial image generation unit
109: Learning image database
110: Classifier training/addition module
110a: CNN feature point extraction unit
110b: Network embedding low-dimensional mapping unit
110c: Synthetic/forgery classifier addition unit
111: Facial image reception module
112: Facial recognition module
112a: Face recognition unit using basic classifier
112b: Face recognition unit utilizing synthesis/forgery classifier
113: Facial recognition result output module

Claims (3)

A user registration module that receives and registers user information;
a user information storage module that stores user information registered by the user registration module;
A facial image registration module that registers the user's facial image;
a facial image storage module in which facial images registered in the facial image registration module are pre-stored for each user information stored in the user information storage module;
a classifier generation module that pre-generates a classifier for recognizing a user using a facial image pre-stored in the facial image storage module;
a user-specific classifier storage module in which the classifier pre-generated by the classifier creation module is stored in advance for each user;
a learning image addition module that adds a learning image using the facial image registered by the facial image registration module;
a blind facial image addition module that adds a plurality of blind facial images using the facial image registered by the facial image registration module;
a learning image database storing learning images added by the learning image adding module and blind face images added by the blind face image adding module;
a classifier learning and addition module that learns using the learning images and a plurality of blind face images stored in the learning image database, adds a classifier for each user, and additionally stores the classifier in the user-specific classifier storage module;
A facial image receiving module that receives a facial image for user identification;
a facial recognition module that identifies the user of the facial image received from the facial image receiving module using a classifier stored in the user-specific classifier storage module;
It includes a facial recognition result output module that outputs the user identified in the facial recognition module or the corresponding recognition result,
The classifier creation module is,
A first CNN feature point extraction unit that extracts feature points on the facial image stored in the facial image storage module, sorts faces based on the extracted feature points, and extracts 512 feature vectors from the aligned faces using a CNN algorithm;
a first network embedding low-dimensional mapping unit that maps the feature vectors extracted from the first CNN feature point extraction unit into a low-dimensional space using a network embeddings algorithm;
It is configured to include a basic classifier generator that generates a basic classifier composed of feature vectors mapped to a low-dimensional space in the first network embedding low-dimensional mapping unit,
The learning image addition module is,
a Gaussian noise adding unit that generates and adds a learning image by adding Gaussian noise to the facial image registered in the facial image registration module;
A Gaussian blur addition unit that generates and adds a learning image by adding Gaussian blur to the facial image registered in the facial image registration module;
a Gaussian noise and blur adding unit that generates and adds a learning image by adding Gaussian noise and Gaussian blur to the facial image registered in the facial image registration module;
It is configured to include an image compression unit that generates and adds a learning image by compressing and processing the facial image registered in the facial image registration module,
The blind facial image addition module,
a landmark detection unit that detects 68 landmarks for key parts of the face from the learning images and blind face images stored in the learning image database;
It is configured to include a specific area blind facial image generating unit that generates a blind facial image covering a predetermined area of the face based on the landmark detected by the landmark detection unit,
The classifier learning and addition module is,
A second CNN feature point extraction unit that extracts feature points on the learning image or blind face image from the learning image database, sorts faces based on the extracted feature points, and extracts 512 feature vectors from the aligned faces using a CNN algorithm. ;
a second network embedding low-dimensional mapping unit that maps the feature vectors extracted from the second CNN feature point extraction unit into a low-dimensional space using a network embeddings algorithm;
It is configured to include a synthesis and forgery classifier addition unit for adding a synthesis and forgery classifier for each user, consisting of feature vectors mapped to a low-dimensional space in the second network embedding low-dimensional mapping unit,
The facial recognition module is,
A facial recognition unit utilizing a basic classifier that recognizes the facial image received from the facial image receiving module using the basic classifier generated by the basic classifier generating unit;
It is configured to include a face recognition unit utilizing a synthesis and forgery classifier that distinguishes and recognizes synthesis and forgery of the facial image received from the facial image receiving module using the synthesis and forgery classifier added in the synthesis and forgery classifier addition unit,
The first network embedding low-dimensional mapping unit and the second network embedding low-dimensional mapping unit,
When the triplet loss is calculated using the network embedding algorithm, and the triplet loss is calculated as a negative number, the triplet loss calculated as a negative number is not treated as 0, but a predetermined value is added to the calculated triplet loss and recalculated. A digital synthetic face and digital fake face verification system, characterized in that it is configured to calculate. delete delete