KR20200080533A - Apparatus and Method for Recognizing Fake Face By Using Minutia Data Variation - Google Patents

Abstract

Disclosed are an apparatus for determining a forged face using a feature point variation and a method thereof. The apparatus for determining a forged face of the present invention requests at least one action to change a position of a face or prevent the face from being seen by a user attempting face authentication, and then determines whether the face attempting the face authentication is a forged face based on whether a variation exists in a feature point extracted from a photographed image. In addition, a separate hardware device is not used for determining a forged face, so that the apparatus has a very simple structure.

Description

Apparatus and Method for Recognizing Fake Face By Using Minutia Data Variation}

The present invention relates to a forged face discrimination apparatus and method for discriminating a forged face from among technologies related to biometric recognition using a face.

Face recognition, like fingerprints and irises, is one of the things that attracts attention as a means of personal authentication.

Face recognition is a method of recognizing a face by capturing it with a camera and comparing it with a registered face image. Face recognition technology is classified into a method using a two-dimensional image and a method using a three-dimensional shape. Since the three-dimensional recognition technology requires a large amount of computation, a two-dimensional face recognition system is generally used.

Fingerprint or iris recognition is excellent in immutability and relatively easy to acquire the image, whereas face recognition is an authentication technology that is not easily adopted due to various problems such as being recognized as different faces depending on lighting or facial expressions. .

Nevertheless, face recognition has the advantage that the user's body does not come into contact with the device without requiring any other manipulation by the user, such as iris recognition, and face recognition has been grafted with digital devices such as computers and mobile phones. One of the challenges face authentication has to solve is effectively distinguishing forged faces (hereinafter referred to as'falsified faces') rather than the registered users themselves. Forged faces are usually printed on paper with a face image.

When such a forged face is input to a conventional face recognition apparatus, a face image similar to a real face is obtained, and there is room for processing without distinction from the real face through a process of extracting feature points.

[Related inventions]

1. Counterfeit detection method in face recognition system (Republic of Korea Patent Publication No. 10-2011-0102073)

In the present invention, the pupil region in the image captured from the real face is very dark, whereas the image captured from the photo (forged face) reflects light in all regions, so that the pupil region is relatively bright, and thus the forged face Detects.

An object of the present invention is to provide a counterfeit face discrimination apparatus and method for discriminating a forged face and a real face by using a variation of feature points extracted from a series of face images photographed in series.

In order to achieve the above object, a method for determining a forged face according to the present invention includes: generating a face image for forgery discrimination by photographing a user attempting authentication by a first camera unit at a predetermined frame rate; Repeating a process of extracting a face portion from the plurality of face images generated by the first camera unit and extracting feature points of the face during a predetermined measurement time; When comparing the feature points extracted from the plurality of face images through the repeating step, if there is a variation in some of the feature points of the entire feature point, determining the face being authenticated as a living body, and if the change of the feature point is not detected, determining as a forged face. Includes.

Linkage of counterfeit identification and face recognition

According to an embodiment, the method of the present invention comprises the steps of confirming a user's face authentication attempt; When the authentication attempt is confirmed, the method may further include extracting a feature point from the face image obtained by the first camera unit and comparing the user registration information with the primary recognition of the user. In this case, the step of determining the face to be authenticated as a living body is the same as comparing the feature points extracted in the first recognition step with the feature points extracted at the end of the measurement time when there is a variation in some feature points of the whole feature points. It is judged as a living body.

On the other hand, in the determining step, it is preferable to detect a change in the feature point among face images generated in frames other than the first frame and the last frame of the measurement time.

User action request for forgery discrimination

According to another embodiment, the method of the present invention may further include outputting a guide to the user requesting at least one user action that makes a variation of some of the entire feature points. The repeating step is preferably performed during the measurement time after the step of displaying the guidance. For example, the guide corresponds to requesting the user to change the shape of the mouth by pronouncing a specific sound.

According to another embodiment, the method of the present invention recognizes a user's face attempting face authentication from an image provided by the first camera unit, detects a user with a proximity sensor, or detects a user's face when a specific button is operated. It may further include determining that there is an authentication attempt. When the authentication attempt is confirmed, a step of outputting the guide to the user is performed.

Save log

According to another embodiment, the method of the present invention, the second camera unit may further include the step of generating the user's face image and storing it together with the determination result of the determining.

Counterfeit face discrimination device

The present invention also extends to a discrimination device for performing the method for discriminating the forged faces. The discrimination device of the present invention includes a first camera unit, a face detection unit, a feature point extraction unit, and a counterfeit determination unit.

The first camera unit generates a face image by photographing a user attempting authentication at a preset frame rate. The face detection unit extracts a face portion from the face image, and the feature point extraction unit extracts a feature point of the face from the face portion.

When comparing the feature points extracted from a plurality of the face images during a predetermined measurement time, the counterfeit determination unit judges the face being authenticated as a living body if there is a change in some feature points of the entire feature points, and determines if the change in the feature points is not detected as a forged face. Perform counterfeiting.

The forged face discrimination apparatus according to the present invention requests a user who attempts to authenticate a face to perform an operation capable of causing a change in the feature point, and then the face being tried to authenticate is based on whether there is a change in the feature point extracted from the captured image. You can determine whether it is a face.

In addition, the method for discriminating forged faces of the present invention is very simple because the separate hardware device is not used for discriminating forged faces.

1 is a block diagram of a forged face discrimination device of the present invention,
Figure 2 is a flow chart provided in the description of the forgery face identification method of the present invention,
Figure 3 is a flow chart provided in the description of the forged face identification method according to another embodiment of the present invention, and
4 is a block diagram of a forged face discrimination device according to another embodiment of the present invention.

Hereinafter, the present invention will be described in more detail with reference to the drawings.

Referring to FIG. 1, the forged face discrimination apparatus 100 of the present invention includes a user interface 101, an image acquisition module 110, and a control module 130.

The user interface 101 is a means for outputting various guides for discriminating forged faces described below, and outputs them to the user in a visual or audible manner. For example, a display means such as LCD or OLED or a voice output means such as a speaker may correspond to the user interface 101.

The image acquisition module 110 acquires an image (hereinafter referred to as a “face image”) of a user's face, including at least one light source 111 and at least one first camera unit 113.

The light source 111 emits light of a predetermined wavelength toward the user's face in order to photograph the user's face. The wavelength of the light emitted from the light source 111 may be any band and white light may be used, but infrared light is preferred. When using the infrared light source 111, the first camera unit 113 can exclude the influence of external natural light by using an infrared filter.

The first camera unit 113 of FIG. 1 includes a lens 113a, an image sensor 113b, and an infrared filter 113c, and is a face image as an infrared image emitted from the light source 111 and reflected from the user's face. Produces The lens 113a adjusts a focal length and an angle of view so that the first camera unit 113 can photograph the entire face of the user.

The first camera unit 113 photographs the user's face at a predetermined frame rate for face recognition, and provides the obtained plurality of face images to the control module 130.

On the other hand, as described below, the first camera unit 113 periodically photographs the front to check whether the user attempts to authenticate the face, in which case the camera may take a picture at a rate lower than the predetermined frame rate.

The control module 130 includes a face detection unit 131, a feature point extraction unit 133, a face recognition unit 135, and a counterfeit determination unit 137, and controls the overall operation of the forged face discrimination device 100 of the present invention. Control.

The face detection unit 131 performs a pre-processing process for discriminating forged faces, and extracts the face portion from the face image provided by the image acquisition module 110 at a predetermined frame rate. Face extraction can be divided into knowledge-based learning methods, feature-based methods, template matching methods, and appearance-based methods, and any methods known in the art can be applied.

The appearance-based method, for example, by applying a conventional image processing method to the face image provided by the image acquisition module 110, extracts the outline of the face corresponding to the approximate circle, and then detects the eyes and mouth based on the location from the face outline Recognize facial contours.

On the other hand, a method of confirming the user's authentication attempt can be various methods including a user operating a button or confirming access using a proximity sensor, and one of the methods uses an image captured by the image acquisition module 110. will be. In this case, the face detection unit 131 checks the degree to which the face image is input from the image provided by the first camera unit 113 in the natural light state in which the light source 111 is turned off and notifies the counterfeit determination unit 137 By doing so, it is possible to check whether or not the user tries to authenticate. The image processing at this time may not be a pre-processing process for extracting feature points from the face image.

The feature point extraction unit 133 extracts information for face authentication by extracting feature point data from the face portion detected by the face detection unit 131 in the face image. The feature points extracted by the feature point extraction unit 133 are features that reflect shape information specific to the user, such as information on the shape, length, and relative position of the face, chin, eyes, eyebrows, nose, and mouth. Is included. In addition, the feature points of the face can be extracted by applying various information and extraction methods known in the art.

The face recognition unit 135 recognizes a user attempting face authentication by comparing the feature point data extracted by the feature point extraction unit 133 with pre-registered'user registration information'. The user registration information includes feature point data extracted from the user's face image and information about the user (name, etc.). The counterfeit face discrimination device 100 may perform user registration process on its own to retain user registration information, or may receive and retain user registration information from another authentication device or server.

The forgery determination unit 137 performs forgery determination based on the feature point data provided by the feature point extraction unit 133. The basic method of forgery discrimination is to check whether there are variations in feature points extracted from a series of multiple face images generated during a measurement period. Here, the variation of the feature points means that some feature point locations (for example, the position of the mouth) or the shape of the contour (for example, the shape of the mouth) changes compared to other feature points extracted during the measurement period. If a change in the feature point is detected, the face being authenticated is regarded as a real face. Conversely, if no change in the feature point is detected, the face being authenticated is regarded as a fake face. The forgery face discrimination method of the forgery determination unit 137 will be described again below with reference to FIG. 2.

How to judge counterfeit faces (Figure 2)

The basic method of determining the forged face of the present invention is that the first camera unit 113 photographs a user who attempts authentication, and then generates a face image, and the face detection unit 131 uses the first camera unit 113 at a preset frame rate. It is performed while repeating the process of extracting a face part for each of the plurality of face images to be generated, and the feature point extracting unit 133 extracting face feature point data for each of the plurality of face images.

The counterfeit determination unit 137 determines that the face being authenticated is a counterfeit face if all the feature point data extracted from the plurality of face images are the same or within the same range.

Hereinafter, a method for discriminating a forged face according to the present invention will be described with reference to FIG.

<Confirmation of face authentication attempt: S201, S203>

The first camera unit 113 periodically photographs the front side to generate an image and provides it to the control module 130. At this time, the light source 111 may be turned on, but the light source 111 may be turned off to photograph the user in a natural light state (S201).

The face detection unit 131 detects a user's face by performing image processing on an image captured by the first camera unit 113. The counterfeit determination unit 137 determines that there is an attempt to authenticate the user's face when the face detection unit 131 recognizes the user's face. The face detection unit 131 may also perform a degree of detecting the outline of the face through edge detection in the face image (S203 ).

As described above, the forged face discrimination device 100 further includes a proximity sensor or a button, and the control module 130 further includes an authentication initiator that can confirm a face authentication attempt based on the proximity sensor or a user's button operation. can do.

Meanwhile, the verification of the steps S201 and S203 may be performed by the face detection unit 131 or the face recognition unit 135 instead of the counterfeit determination unit 137.

<User guide for counterfeit identification: S205>

When the user's face is recognized in step S203, the forgery determination unit 137 provides a user guide for forgery determination to the user through the user interface 101.

As is known, since the forged face is in the form of a paper on which a face image is printed, the face shape is fixed. However, when the expression of the user's real face changes, the shape, relative position or length of the eyes, nose, and mouth may also change. When the feature point data changes a lot due to the large fluctuation, it can be recognized as different people. In view of this, the present invention determines whether counterfeit is made by requesting a facial expression change from a user to determine whether the feature point changes.

Therefore, the user guidance for counterfeit discrimination is to request the user'user action' that creates a variation (the shape, position, and relative position of a feature point) of some feature points extracted from the face. The user's motion may be any motion as long as it causes a change in the characteristic point, for example, a motion of changing the shape of the mouth or closing the eyes. It is also possible to simply change the position of the face, but it is desirable to include the movement of the eyes, nose or mouth. Also, a user action that can be performed in a particularly short time is more preferable. In the present invention, the user may be asked to pronounce at least one selected from among'A','E','I','O', and'U'.

The user guidance for forgery discrimination may be output through voice through a speaker, or may be visually displayed using a display means.

<Face recognition/Feature point extraction during measurement time: S207 to S211>

After displaying the user guidance for forgery discrimination, the forgery determination unit 137 lights up the light source 111 and causes the first camera unit 113 to capture the user's eyes at a predetermined frame rate to generate a face image. Control. The shooting speed of the first camera unit 113 may be the same as the shooting speed of step S201, but it is preferable to set it to a higher speed (S207).

The face detection unit 131 recognizes the face portion from the face image taken by the first camera unit 113, and the feature point extraction unit 133 extracts the feature points of the face (S209 ).

Steps S207 and S209 are repeated a plurality of (n) times during a predetermined measurement time, thereby generating a plurality of (n) face images, and extracting faces and extracting feature points a plurality of times (n). For example, if the shooting speed of the first camera unit 113 is set to 10 frames per second and the measurement time is set to 2 seconds, the face detection unit 131 is selected from 20 face images (n=20) provided for 2 seconds. Feature point data is extracted. Here, the measurement time is preferably set longer than the time required for the user to perform the guided operation according to the user guidance in step S205, and can be determined experimentally (S211).

<Distinguish counterfeit faces according to changes in feature points: S213 to S217>

When the measurement time has elapsed, the counterfeit determination unit 137 determines whether there is variation between the feature points extracted from the plurality of (n) face images. Here, the variation of the feature points does not mean that the entire feature points are changed into a constant pattern, but that the shape, position, and relative position of some of the feature points are changed. It is not included in the variation of the feature point that the extraction position of the entire feature point changes as the position of the face changes while maintaining the shape of the eyes, nose, and mouth (S213).

If the forgery determination unit 137 detects a face being authenticated as a living body when a feature point change is detected between a plurality of (n) face images, if the feature point change is not detected, the face being authenticated is determined as a forged face. ((S217).

If the user changes the shape of the mouth according to the guidance of step S205 to make a specific sound, a feature point extracted from the user's mouth or a feature point related to the mouth is changed from a plurality of face images taken during the measurement time (for example, 2 seconds). Since the counterfeit face in the form printed on paper or synthetic resin in high resolution cannot perform an operation according to the user's guidance, there is no change in some feature points.

On the other hand, even if it is guided to move the pupil in step S205, the user may blink the eye instead of moving the mouth, and at this time, some feature points also fluctuate. Even if the feature points of the other parts differ from the guided motion, the forgery determination unit 137 may determine the face being authenticated as a living body. Therefore, the determination criteria of the error image may be irrelevant to the guidance of step S205.

The method for discriminating forged faces of the present invention is performed by the above method.

Counterfeit identification during user authentication

The method for discriminating forged faces of the present invention may be combined with a user's face recognition process. Hereinafter, a forgery face identification method according to another embodiment of the present invention will be described with reference to FIG. 3. The method of FIG. 3 combines a user face recognition process among the methods of discriminating forged faces of FIG. 2.

<Confirmation of face authentication attempt: S201, S203>

According to steps S201 and S203 of FIG. 2, a user's face authentication attempt is confirmed.

<User recognition: S301 to S305>

When the user's face authentication attempt is confirmed, the forgery determination unit 137 lights up the light source 111 and controls the first camera unit 113 to acquire the user's face image. The face detection unit 131 recognizes the face portion from the face image provided by the first camera unit 113, and the feature point extraction unit 133 primarily extracts feature point data from the face portion. Hereinafter, for convenience of description, the feature point data extracted in step S205 is referred to as'reference feature point data' (S301).

The face recognition unit 135 first recognizes the user by comparing the reference feature point data with pre-registered user information to identify the matching user (S303).

If the registered user information identical to the reference feature point data is not found, the face recognition unit 135 processes face recognition for the user as a failure (S305).

<Measurement for discriminating forged faces: S205 to S211>

If the user is recognized in step S303, the counterfeit determination unit 137 performs step S205. Accordingly, the forgery determination unit 137 provides a user guide for forgery discrimination to the user through the user interface 101. Thereafter, while performing the steps S207 to S211 for a predetermined measurement time, a measurement for discriminating the forged face is performed.

<Distinguish counterfeit faces due to changes in feature points: S213, S217>

When the measurement time has elapsed, the counterfeit determination unit 137 determines whether there is variation between the feature points extracted from the plurality of (n) face images (S213). If the feature point variation is not detected between the plurality of (n) face images, the forgery determination unit 137 determines the face being authenticated as a forged face (S217).

<Judgment by real face: S307 to S311>

As a result of the determination in step S213, when a feature point variation is detected between a plurality of (n) face images, the forgery determination unit 137 extracts the feature point data that is taken at the end of the measurement time in steps S205 to S211 and extracts the feature point data in step S301. It is determined whether it is the same by comparing with the reference feature point data extracted in step S307.

As a result of the determination in step S307, if the two feature point data are the same and are determined to be the same user (S309), the counterfeit determination unit 137 finally determines the face being authenticated as a real face (S311).

If, as a result of the determination in step S307, the data of the two feature points are not the same, it is not the same user, and the counterfeit determination unit 137 determines that there is a possibility of another counterfeit face and determines the face being authenticated as a counterfeit face. Through steps S307 and S309, the forgery determination unit 137 blocks avoiding the forgery determination of the present invention by changing the face image during the authentication procedure.

When the steps S307 and S309 are compared with the method of FIG. 2, when a feature point variation is detected between a plurality of (n) face images, the face being authenticated is determined to be a real face without any other procedure (step S215).

Save log

Referring to FIG. 4, the image acquisition module 410 of the forged face discrimination device 400 according to another embodiment of the present invention is a second camera unit for taking a user's face image separately from the first camera module 113 (411) may be further included.

Since the second camera unit 411 photographs the face image, the second camera unit 411 is different from the first camera unit 113 that generates the face image. The second camera unit 411 may generate a color image of the user's face with natural light without infrared illumination.

The counterfeit determination unit 431 may control the second camera unit 411 to capture a user's face image when the user's face is recognized in step S203. The counterfeit determination unit 431 may store the face image photographed by the second camera unit 411 in a memory or a storage medium (not shown) together with the determination results in steps S215, S217, S305, or S311.

The face image photographed by the second camera unit 411 may be used as log data for a user who has attempted face authentication.

Although the preferred embodiments of the present invention have been illustrated and described above, the present invention is not limited to the specific embodiments described above, and it is usually in the technical field to which the present invention pertains without departing from the gist of the present invention claimed in the claims. Of course, various modifications can be implemented by a person having knowledge of these, and these modifications should not be individually understood from the technical idea or prospect of the present invention.

111: light source 113: first camera unit
113a: Lens 113b: Image sensor
113c: infrared filter 411: second camera unit

Claims (14)

Generating a face image for forgery determination by photographing a user attempting authentication by the first camera unit at a predetermined frame rate;
Repeating a process of extracting a face portion from a plurality of face images generated by the first camera unit and extracting feature points of the face during a predetermined measurement time; And
When comparing the feature points extracted from the plurality of face images through the repeating step, if there is a variation in some of the feature points of the entire feature point, determining the face being authenticated as a living body, and if the change of the feature point is not detected, determining as a forged face. It characterized in that it comprises a forged face identification method.
According to claim 1,
Confirming the user's face authentication attempt; And
When the authentication attempt is confirmed, further comprising the step of extracting a feature point from the face image acquired by the first camera unit and comparing the user registration information to the first recognition of the user,
The step of determining the face being authenticated as a living body is compared to a feature point extracted in the step of firstly recognizing the feature point extracted at the end of the measurement time when there is a change in some feature points of the entire feature point to the living body in the same case. A forgery face discrimination method characterized by determining.
The method according to claim 1 or 2,
In the determining step,
A method for discriminating a forged face, characterized in that a variation of the feature point is detected from face images generated in frames other than the first frame and the last frame of the measurement time.
The method according to claim 1 or 2,
Further comprising the step of outputting to the user a guide requesting at least one user action to make a variation of some of the entire feature points,
The repeating step is a counterfeit face discrimination method characterized in that it is performed during the measurement time after the step of displaying the guidance.
According to claim 4,
The guide, forgery face discrimination method characterized in that the user requests to change the shape of the mouth by pronunciation of a specific sound.
According to claim 4,
Further comprising the step of recognizing the face of the user attempting to authenticate the face from the image provided by the first camera unit, detecting the user with a proximity sensor or determining that the user has an attempt to authenticate the face when a specific button is operated,
And when the authentication attempt is confirmed, outputting the guide to the user.
According to claim 1,
And a second camera unit generating the user's face image and storing the result together with the determination result of the determining.
A first camera unit that photographs a user attempting authentication at a predetermined frame rate to generate a face image;
A face detection unit for extracting a face portion from the face image;
A feature point extraction unit for extracting a feature point of the face from the face portion; And
When comparing feature points extracted from a plurality of the face images during a predetermined measurement time, if there is a change in some feature points among all the feature points, the face being authenticated is judged as a living body, and if a change in the feature points is not detected, it is determined as a forged face to determine the forgery. A forgery face discrimination device comprising a forgery judgment unit to perform.
The method of claim 8,
According to the user's face authentication attempt, the facial recognition unit further recognizes the user by comparing the feature points extracted by the feature point extraction unit with the user registration information from the face image acquired by the first camera unit,
The counterfeit determining unit compares a feature point extracted at the end of the measurement time with a feature point used for the first recognition when there is a change in some feature points among the entire feature points, and determines that the counterfeit face is a living body in the same case. Discrimination device.
The method of claim 8 or 9,
The counterfeit determination unit,
A forgery face discrimination apparatus characterized by detecting a change in the feature point among face images generated in frames other than the first frame and the last frame of the measurement time.
The method of claim 7 or 8,
The method according to claim 1 or 2,
Further comprising a user interface for outputting to the user a guide requesting at least one user action to make a variation of some of the feature points under the control of the counterfeit determination unit,
The forged face discrimination unit performs the forgery discrimination based on the extraction result of the feature point extracting unit extracted during the measurement time after outputting the guidance.
The method of claim 11,
The guide, forgery face discrimination device characterized in that the user to request to change the shape of the mouth by pronunciation of a specific sound.
The method of claim 8,
The counterfeit determination unit,
The face detection unit recognizes a face of a user attempting face authentication from an image provided by the first camera unit, detects a user with a proximity sensor, or outputs the guide when a specific button is operated. Device.
The method of claim 8,
Further comprising a second camera unit for generating the face image of the user,
The counterfeit determination unit is characterized in that for storing the face image together with the result of the forgery discrimination.

Images