KR102062708B1 - Apparatus and method for detection presentaion attack for face recognition system - Google Patents

Abstract

The present invention relates to a technique for determining whether a forgery attack on a face recognition system, and more particularly, to a forgery attack detection technique using a combination of a handcrafted feature and a deep learning feature of an image. According to one embodiment of the present invention, it is possible to effectively detect the forgery attack on the face recognition system to ensure the safety of the face recognition system.

Description

Apparatus and method for detecting forgery attack in face recognition system {APPARATUS AND METHOD FOR DETECTION PRESENTAION ATTACK FOR FACE RECOGNITION SYSTEM}

The present invention relates to a technique for determining whether a forgery attack on a face recognition system, and more particularly, to a forgery attack detection technique using a combination of a handcrafted feature and a deep learning feature of an image.

Recently, biometric systems have been used in various application systems because they are difficult to steal, have high accuracy, and are convenient for users. Such biometric systems have been reported to be more suitable for improving recognition accuracy and user convenience compared to token-based methods using keys and cards or knowledge-based methods using usernames and passwords. However, biometric systems are vulnerable to fake images in the process of acquiring images.

Therefore, a presentation attack detection (PAD) method for detecting a forged attack using a fake image has been proposed. Most of the proposed PAD methods use handcrafted features based on Gabor filter, local binary pattern (LBP), local ternary pattern (LTP), or histogram of oriented gradients (HOG). However, these features reflect a limited aspect and have low accuracy, and the accuracy varies depending on the fake face image. With the recent development of deep learning technology, feature extraction method using deep learning technology has higher classification accuracy than handcrafted feature extraction method. However, the feature extraction method using the deep learning technology shows better performance than the handcrafted feature extraction method in the fingerprint recognition system of the biometric system, while the performance of the iris and face recognition system is relatively poor.

Background of the present invention is disclosed in Korean Patent Publication No. 2010-0125985.

An object of the present invention is to provide a counterfeit attack detection device and method for detecting a counterfeit attack on a face recognition system.

According to an aspect of the present invention, a forgery attack detection apparatus of a face recognition system is provided.

An apparatus for detecting forgery attack of a face recognition system according to an embodiment of the present invention includes a preprocessor for generating a face image by performing preprocessing on an input image, and handcrafted feature extraction for extracting a handcrafted feature vector from the face image. A deep feature extraction unit extracting a deep feature vector from the face image, a hybrid feature calculator configured to generate a hybrid feature vector by connecting the handcrafted feature vector and the deep feature vector, and remove noise of the hybrid feature vector. A counterfeit attack detection device is provided that includes a principal component analysis unit for reducing a dimension and an attack detection unit for generating attack detection information according to the hybrid feature vector.

The handcrafted feature extraction unit may extract the handcrafted feature vector through a modified local binary pattern (MLBP) method.

The deep feature extractor may extract the deep feature vector through a convolutional neural network (CNN).

The CCN uses a CNN structure including 16 convolutional layers, 5 pooling layers, and 2 fully connected layers, wherein kernel sizes used in each of the convolutional layers are 3 × 3 × 3, 3 × 3 × 64, and 3 ×. Any one of 3 × 128, 3 × 3 × 256 and 3 × 3 × 512, padding size is 1 × 1, stride size is 1 × 1, and the number of filters is any of 64, 128, 256 and 512 Each of the pooling layer may have a filter size of 2 × 2, a stride size of 2 × 2, and a padding size of 0.

The two fully connected layers may output 4096 feature maps, respectively.

The deep feature extractor may extract the output of the last fully connected layer among the fully connected layers as the deep feature vector.

The principal component analyzer may remove noise of the hybrid feature vector and reduce the dimension by applying a principal component analysis (PCA).

According to another aspect of the present invention, a forgery attack detection method of a face recognition system is provided.

In the forgery attack detection method of the face recognition system according to an embodiment of the present invention, generating a face image by performing a pre-processing on the input image, extracting a handcrafted feature vector from the face image, the face image Extracting a deep feature vector from the method, concatenating the handcrafted feature vector and the deep feature vector to generate a hybrid feature vector, removing noise and reducing dimensions of the hybrid feature vector, and reducing the hybrid feature vector In accordance with the present invention provides a forged attack detection method comprising the step of generating attack detection information.

According to one embodiment of the present invention, it is possible to effectively detect the forgery attack on the face recognition system to ensure the safety of the face recognition system.

1 is a block diagram illustrating a counterfeit attack detection apparatus according to an embodiment of the present invention.
2 is a diagram illustrating a preprocessing process performed by the forgery attack detection apparatus according to an embodiment of the present invention.
3 is a diagram illustrating a MLBP performed by the forgery attack detection device according to an embodiment of the present invention.
Figure 4 illustrates the general structure of a CNN used by the forgery attack detection apparatus according to an embodiment of the present invention.
5 is a diagram illustrating a specific structure of the CNN used by the forgery attack detection apparatus according to an embodiment of the present invention.
FIG. 6 is a flowchart illustrating a process of detecting a counterfeit attack by a counterfeit attack detection device according to an embodiment of the present invention. FIG.
7 is a diagram illustrating a test result of the forgery attack detection accuracy of the input image in the forgery attack detection device according to an embodiment of the present invention.

As the inventive concept allows for various changes and numerous embodiments, particular embodiments will be illustrated in the drawings and described in detail in the written description. However, this is not intended to limit the present invention to specific embodiments, it should be understood to include all modifications, equivalents, and substitutes included in the spirit and scope of the present invention. In describing the present invention, if it is determined that the detailed description of the related known technology may unnecessarily obscure the subject matter of the present invention, the detailed description thereof will be omitted. Also, the singular forms used in the specification and claims are to be interpreted as generally meaning "one or more", unless stated otherwise.

Hereinafter, preferred embodiments of the present invention will be described in detail with reference to the accompanying drawings, and in the following description with reference to the accompanying drawings, the same or corresponding components are given the same reference numerals and redundant description thereof will be omitted. Shall be.

1 is a block diagram illustrating a counterfeit attack detection device according to an embodiment of the present invention, Figure 2 is a diagram illustrating a preprocessing process performed by the counterfeit attack detection device according to an embodiment of the present invention, Figure 3 FIG. 4 is a diagram illustrating an MLBP performed by a counterfeit attack detection apparatus according to an embodiment of the present invention, and FIG. 4 is a diagram illustrating a general structure of a CNN used by a counterfeit attack detection apparatus according to an embodiment of the present invention. 5 is a diagram illustrating a specific structure of the CNN used by the forgery attack detection apparatus according to an embodiment of the present invention.

Referring to FIG. 1, the counterfeit attack detection device according to an embodiment of the present invention may include a preprocessor 110, a handcrafted feature extractor 120, a deep feature extractor 130, and a hybrid feature calculator 140. , The principal component analyzer 150 and the attack detector 160.

The preprocessor 110 receives an image (hereinafter, referred to as an input image) in which a user's face is photographed, performs a preprocessing process on the input image, detects a face region, and generates a face image that is an image including a face region. do. For example, the preprocessor 110 may detect a facial feature point using an ensemble of regression tree from an input image as shown in FIG. 2. That is, the preprocessor 110 may detect facial feature points, such as the red dot 210 of FIG. 2. The preprocessing unit 110 rotates the input image as shown in 220 of FIG. 2 so that the face faces the front using the coordinates of the feature points corresponding to both eyes among the detected facial feature points, and is positioned outside the face in the rotated input image. The facial region may be detected based on the feature point, and a facial image including the facial region (230 of FIG. 2) may be generated. The preprocessor 110 transmits the face image to the handcrafted feature extractor 120 and the deep feature extractor 130.

The handcrafted feature extractor 120 extracts a handcrafted feature vector from the face image. For example, the handcrafted feature extractor 120 may extract the handcrafted feature vector through a modified local binary pattern (MLBP) method as illustrated in FIG. 3.

The deep feature extractor 130 extracts a deep feature vector from the face image. For example, the deep feature extractor 130 inputs a face image to a CNN (Convolutional Neural Network) having a structure as illustrated in FIG. 4, and receives a deep feature vector. In detail, the CNN used by the deep feature extractor 130 may include 16 convolutional layers, 5 pulling layers, and 2 fully connected layers.

As an example of a CNN, the first convolutional layer generates a feature map using 64 filters for an input of 224 × 224 × 3 (width, height, and channel size). In the CNN, for example, the acquired feature map may output 224 × 224 × 64 through the ReLU layer of the first convolutional layer. The filter size used in the first convolutional layer may be 3 × 3 × 3, the stride size may be 1 × 1, and the padding size may be 1 × 1.

As an example of a CNN, the second convolutional layer generates a feature map using 64 filters for an input of 224 × 224 × 64 (width, height, and channel size). The generated feature map may pass through the ReLU layer of the second convolutional layer, and then 112 × 112 × 64 feature maps may be output through the maximum pulling layer. In this case, the filter size of the maximum pooling layer that receives the feature map from the second convolutional layer may be 2 × 2, and the stride size may be 2 × 2. In the second convolutional layer, the filter size may be 3 × 3 × 64, the padding size is 1 × 1, and the stride size may be 1 × 1.

As an example of a CNN, the third convolutional layer generates a feature map using 128 filters for an input of 112 × 112 × 64 (width, height, and channel size). In the CNN, for example, the acquired feature map may output 112 × 112 × 128 through the ReLU layer of the third convolutional layer. The filter size used in the first convolutional layer may be 3 × 3 × 64, the stride size is 1 × 1, and the padding size may be 1 × 1.

As an example of a CNN, the fourth convolutional layer generates a feature map using 128 filters for an input of 112 × 112 × 128 (width, height, and channel size). The generated feature map passes through the ReLU layer of the fourth convolution layer, and then a feature map of 56 × 56 × 128 may be output through the maximum pulling layer. In this case, the filter size of the maximum pooling layer that receives the feature map from the fourth convolutional layer may be 2 × 2, and the stride size may be 2 × 2. In the fourth convolutional layer, the filter size may be 3 × 3 × 128, the padding size is 1 × 1, and the stride size may be 1 × 1.

For example, in the CNN, the fifth convolutional layer generates a feature map using 256 filters for 56 × 56 × 128 (width, height, and channel size) inputs. In the CNN, for example, the acquired feature map may output 56 × 56 × 256 through the ReLU layer of the fifth convolutional layer. The filter size used in the fifth convolutional layer may be 3 × 3 × 128, a stride size of 1 × 1, and a padding size of 1 × 1.

For example, in the CNN, the sixth and seventh convolutional layers generate a feature map using 256 filters for 56 × 56 × 256 inputs (width, height, and channel size). For example, in the CNN, the obtained feature map may output 56 × 56 × 256 through the ReLU layers of the sixth and seventh convolution layers. The filter sizes used in the sixth and seventh convolution layers may be 3 × 3 × 256, the stride size is 1 × 1, and the padding size may be 1 × 1.

For example, in the CNN, the eighth convolutional layer generates a feature map using 256 filters for 56 × 56 × 256 inputs (width, height, and channel size). The generated feature map passes through the ReLU layer of the eighth convolutional layer, and then a feature map of 28 × 28 × 256 may be output through the maximum pulling layer. In this case, the filter size of the maximum pooling layer that receives the feature map from the eighth convolutional layer may be 2 × 2, and the stride size may be 2 × 2. In the eighth convolutional layer, the filter size may be 3 × 3 × 256, the padding size is 1 × 1, and the stride size may be 1 × 1.

For example, in the CNN, the ninth convolutional layer generates a feature map using 512 filters for an input of 28 × 28 × 256 (width, height, and channel size). For example, in the CNN, the obtained feature map may output 28 × 28 × 512 through the ReLU layer of the ninth convolution layer. The filter size used in the ninth convolution layer may be 3 × 3 × 256, a stride size of 1 × 1, and a padding size of 1 × 1.

For example, in the CNN, the tenth and eleventh convolutional layers generate a feature map using 512 filters for an input of 28 × 28 × 512 (width, height, and channel size). In the CNN, for example, the acquired feature map may output 28 × 28 × 512 through the ReLU layers of the tenth and eleventh convolutional layers. The filter sizes used in the tenth and eleventh convolution layers may be 3 × 3 × 512, a stride size of 1 × 1, and a padding size of 1 × 1.

As an example of a CNN, a twelfth convolutional layer generates a feature map using 512 filters for an input of 28 × 28 × 512 (width, height, and channel size). The generated feature map may pass through the ReLU layer of the twelfth convolution layer, and then a feature map of 14 × 14 × 512 may be output through the maximum pulling layer. In this case, the filter size of the maximum pooling layer that receives the feature map from the twelfth convolution layer may be 2 × 2, and the stride size may be 2 × 2. In the twelfth convolution layer, the filter size may be 3 × 3 × 512, the padding size is 1 × 1, and the stride size may be 1 × 1.

As an example of a CNN, the thirteenth to fifteenth convolution layers generate a feature map using 512 filters for an input of 14 × 14 × 512 (width, height, and channel size). In the CNN, for example, the acquired feature map may output 14 × 14 × 512 through the ReLU layer of the thirteenth to fifteenth convolution layers. The filter size used in the thirteenth and fifteenth convolution layers may be 3 × 3 × 512, a stride size of 1 × 1, and a padding size of 1 × 1.

As an example of a CNN, a sixteenth convolutional layer generates a feature map using 512 filters for an input of 14 × 14 × 512 (width, height and channel size). The generated feature map may pass through the ReLU layer of the sixteenth convolution layer, and then a feature map of 7 × 7 × 512 may be output through the maximum pulling layer. In this case, the filter size of the maximum pooling layer that receives the feature map from the sixteenth convolution layer may be 2 × 2, and the stride size may be 2 × 2. In the twelfth convolution layer, the filter size may be 3 × 3 × 512, the padding size is 1 × 1, and the stride size may be 1 × 1.

Thereafter, for example, the CNN may output 4096 feature maps through two additional fully connected layers. Here, the feature map output through the second fully connected layer may be input as a softmax function, and the softmax function may determine whether the input image is a forgery attack. The Softmax function uses Equation 1 below.

[Equation 1]

Where r is an array of output neurons.

The deep feature extractor 130 outputs the feature map output by the CNN to the hybrid feature calculator 140 as a deep feature vector.

The hybrid feature calculator 140 connects the handcrafted feature vector and the deep feature vector to calculate a hybrid feature vector. For example, the hybrid feature calculator 140 may calculate a hybrid feature vector according to Equation (2) below.

[Equation 2]

f = [f _h, f _d ]

In this case, f _h is a handcrafted feature vector, f _d is a deep feature vector, and f is a hybrid feature vector.

For example, when there are 3732 handcrafted feature vectors and 4096 deep feature vectors, the hybrid feature calculator 140 may calculate a total of 7828 hybrid feature vectors.

The hybrid feature calculator 140 transmits the hybrid feature vector to the principal component analyzer 150.

The principal component analyzer 150 removes noise of the hybrid feature vector and reduces the dimension by applying a principal component analysis (PCA) to the hybrid feature vector. The principal component analyzer 150 transmits the processed hybrid feature vector to the attack detection unit 160.

The attack detection unit 160 inputs the hybrid feature vector to the learned support vector machine (SVM) to determine whether the input image is an image for counterfeit attack, and generates and outputs attack detection information indicating the determination result.

6 is a flowchart illustrating a process of detecting a counterfeit attack by the counterfeit attack detection apparatus according to an embodiment of the present invention. For each step described below, the subject of each step is collectively referred to as a counterfeit attack detection device for a concise and clear description of a process or invention performed through each functional unit constituting the counterfeit attack detection device.

Referring to FIG. 6, in operation 610, the apparatus for detecting forgery attack receives an input image and performs a preprocessing process on the input image to generate a face image. The pretreatment process has been described above with reference to FIG. 1.

In operation 620, the forged attack detection apparatus extracts the handcrafted feature vector from the face image. For example, the counterfeit attack detection device may extract a handcrafted feature vector using Modified Local Binary Patterns (MLBP).

In operation 630, the forgery attack detection apparatus extracts a deep feature vector from the face image. For example, the counterfeit attack detection device may input a face image to a CNN (Convolutional Neural Network) and receive a deep feature vector. In this case, the structure of the CNN has been described with reference to FIGS. 5 and 6.

In operation 640, the counterfeit attack detection apparatus connects the handcrafted feature vector and the deep feature vector to calculate a hybrid feature vector.

In operation 650, the counterfeit attack detection device applies a principal component analysis (PCA) to the hybrid feature vector to remove noise of the hybrid feature vector and to reduce the dimension.

In operation 660, the counterfeit attack detection apparatus inputs the hybrid feature vector to the learned SVM to generate attack detection information indicating whether the input image is a forgery attack.

7 is a diagram illustrating a test result of the forgery attack detection accuracy of the input image in the forgery attack detection device according to an embodiment of the present invention.

At this time, the attack presentation classification error rate (APCER) is the probability of judging the image of the forgery attack as the legitimate user's image, and the bona fide presentation classification error rate (BPCER) is the probability of judging the legitimate user's image as the image of the forgery attack. ACER means the average of APCER and BPCER.

As illustrated in FIG. 7, the counterfeit attack detection device generally shows a detection error as compared to the case of using a single MLBP or a single CNN. Therefore, the counterfeit attack detection device according to an embodiment of the present invention has high detection performance because it detects a counterfeit attack through a hybrid feature vector connecting a feature vector extracted through MLBP and CNN. In addition, the counterfeit attack detection device can reduce the noise of the hybrid feature vector and reduce the dimension through the PCA, thereby reducing the load of the detection process while reducing the detection performance.

The method for determining a bill suitable for a bill according to an exemplary embodiment of the present invention may be implemented in a program instruction form that may be executed by various computer means and may be recorded in a computer readable medium. Computer-readable media may include, alone or in combination with the program instructions, data files, data structures, and the like. The program instructions recorded on the computer readable medium may be those specially designed and constructed for the present invention, or may be known and available to those skilled in the computer software arts. Examples of computer-readable recording media include magnetic media such as hard disks, floppy disks, and magnetic tape, optical media such as CD-ROMs, DVDs, and magnetic disks, such as floppy disks. Hardware devices specially configured to store and execute program instructions such as magneto-optical media and ROM, RAM, flash memory and the like. Examples of program instructions include not only machine code generated by a compiler, but also high-level language code that can be executed by a computer using an interpreter or the like. The hardware device described above may be configured to operate as one or more software modules to perform the operations of the present invention, and vice versa.

So far, the present invention has been described with reference to the embodiments. Those skilled in the art will appreciate that the present invention can be implemented in a modified form without departing from the essential features of the present invention. Therefore, the disclosed embodiments should be considered in descriptive sense only and not for purposes of limitation. The scope of the present invention is shown in the claims rather than the foregoing description, and all differences within the scope will be construed as being included in the present invention.

Claims (15)

A preprocessor configured to generate a face image by performing preprocessing on the input image;
A handcrafted feature extraction unit for extracting a handcrafted feature vector from the face image;
A deep feature extractor extracting a deep feature vector from the face image;
A hybrid feature calculator configured to calculate the hybrid feature vector by connecting the handcrafted feature vector and the deep feature vector;
A principal component analysis unit which removes noise of the hybrid feature vector and reduces the dimension; And
Including an attack detection unit for generating attack detection information according to the hybrid feature vector,
Detect a facial feature point using an ensemble of regression tree, rotate the input image to face the front using the feature point coordinates corresponding to both eyes among the detected face feature points, and rotate Detecting a face region based on a feature point located in a foreign country of the face from the input image, and generating the face image including the face region,
The handcrafted feature extraction unit,
The handcrafted feature vector is extracted through a modified local binary pattern (MLBP) method,
The dip feature extraction unit,
Extracting the deep feature vector through a CNN (Convolutional Neural Network),
And the principal component analysis unit removes noise of the hybrid feature vector and reduces its dimension by applying a principal component analysis (PCA).
delete delete The method of claim 1,
The CNN is
Use a CNN structure with 16 convolutional layers, 5 pooling layers, and 2 fully connected layers,
The kernel size used in each of the convolution layers is any one of 3 × 3 × 3, 3 × 3 × 64, 3 × 3 × 128, 3 × 3 × 256, and 3 × 3 × 512, and the padding size is 1 × 1. Stride size is 1 × 1, and the number of filters is any one of 64, 128, 256, and 512,
The filter size of each of the pooling layer is 2 × 2, the stride size is 2 × 2, the padding size is 0, characterized in that the counterfeit attack detection device.
The method of claim 4, wherein
And the two fully connected layers each output 4096 feature maps.
The method of claim 5,
The dip feature extraction unit,
And extracting the output of the last fully connected layer of the fully connected layers as the deep feature vector.
delete Generating a face image by performing preprocessing on the input image;
Extracting a handcrafted feature vector from the face image;
Extracting a deep feature vector from the face image;
Calculating and generating a hybrid feature vector by connecting the handcrafted feature vector and the deep feature vector;
Removing noise and reducing the dimension of the hybrid feature vector; And
Generating attack detection information according to the hybrid feature vector;
Generating a face image by performing preprocessing on the input image,
Detect a facial feature point using an ensemble of regression tree, rotate the input image to face the front using the feature point coordinates corresponding to both eyes among the detected face feature points, and rotate Detecting a face region based on a feature point located in a foreign country of the face from the input image, and generating the face image including the face region
Extracting the handcrafted feature vector from the face image,
The handcrafted feature vector is extracted through a modified local binary pattern (MLBP) method,
Extracting a deep feature vector from the face image,
Extracting the deep feature vector through a CNN (Convolutional Neural Network),
The removing of the noise of the hybrid feature vector and reducing the dimension may include applying a principal component analysis (PCA) to remove the noise of the hybrid feature vector and reduce the dimension.
delete delete The method of claim 8,
The CNN is
Includes a CNN structure with 16 convolutional layers, 5 pooling layers, and 2 fully connected layers,
The kernel size used in each of the convolution layers is any one of 3 × 3 × 3, 3 × 3 × 64, 3 × 3 × 128, 3 × 3 × 256, and 3 × 3 × 512, and the padding size is 1 × 1. Stride size is 1 × 1, and the number of filters is any one of 64, 128, 256, and 512,
The filter size of each of the pooling layer is 2 × 2, the stride size is 2 × 2, the padding size is 0, characterized in that the counterfeit attack detection method.
The method of claim 11,
And the two fully connected layers each output 4096 feature maps.
The method of claim 12,
Extracting a deep feature vector from the face image,
Extracting the output of the last fully connected layer from among the fully connected layers as the deep feature vector.
delete A computer program for executing the forgery attack detection method of any one of claims 8 and 11 and recorded in a computer-readable recording medium.

Images