KR102280361B1 - Method for generating analogical face and method for recognizing face using the same - Google Patents

Abstract

The present invention is capable of generating a model graph for a face rotated in various directions and angles based on the feature points for the front face. For this purpose, a plurality of reference feature points are extracted based on the front face image and the plurality of reference feature points generating the connected front face model graph; and transforming the plurality of reference feature points in the rotated direction according to a request for generating a face model for a face rotated by a predetermined angle in an arbitrary direction based on the front face. Generating a deformation reference feature point for the rotated face; generating at least one virtual feature point in the predetermined area using the deformation reference feature point adjacent to a predetermined area in a direction opposite to the arbitrary direction; It is possible to provide a method for generating a similar face based on a specific point, comprising the step of generating a face model graph of the rotated face using a reference feature point and a virtual feature point.

Description

Feature point-based similar face generation method and face recognition method using the same {METHOD FOR GENERATING ANALOGICAL FACE AND METHOD FOR RECOGNIZING FACE USING THE SAME}

The present invention relates to a feature point-based similar face generation method and a face recognition method using the same.

In general, a method using unique biometric characteristic information that can be identified in a user's body is widely used in biometric technology applied to an existing system for user authentication. Among biometric technologies, face recognition is the most commonly used technology to verify a user's identity. In the case of the existing face recognition technology, the recognition rate for a user photographed from the front is high, but when the photograph is taken while the user is rotated by a predetermined angle, a problem occurs in that the facial recognition performance is deteriorated.

Korean Patent Registration No. 10-1414158 (Registered on June 25, 2014)

The present invention provides a feature point-based similar face generating method capable of automatically generating a face model graph of a similar face for a face rotated at various angles and directions based on the face model graph for the front face.

In addition, the present invention generates a face model graph for comparison rotated in an arbitrary direction based on the face model graph of the front face as a face image rotated in an arbitrary direction is input, A face recognition method using feature point-based similar face generation capable of performing recognition is provided.

In order to achieve the above-described problem, a feature point-based similar face generation method according to an embodiment of the present invention extracts a plurality of reference feature points based on a front face image and graphs a front face model in which the plurality of reference feature points are connected. and transforming the rotated face by transforming the plurality of reference feature points in the rotated direction according to a request for generating a face model for a face rotated by a predetermined angle in an arbitrary direction based on the front face. generating a reference feature point; generating at least one virtual feature point in the predetermined region using a deformation reference feature point adjacent to a predetermined region in a direction opposite to the arbitrary direction; It may include generating a face model graph of the rotated face.

According to an embodiment of the present invention, the deformation reference feature point and the virtual feature point may be generated using a KNN algorithm and/or a blend shape algorithm to which a K value determined by the arbitrary direction and a predetermined angle is applied.

In order to achieve the above-described problem, a face recognition method using feature point-based similar face generation according to an embodiment of the present invention extracts a plurality of reference feature points based on a front face image to obtain a front face corresponding to the front face. Generating a model graph, recognizing a face image rotated in an arbitrary direction, extracting a plurality of feature points based on this, and converting a plurality of reference feature points on the front face model graph in the arbitrary direction Generating a deformation reference feature point; generating at least one virtual feature point in the predetermined region using a deformation reference feature point adjacent to a predetermined region in a direction opposite to the arbitrary direction; generating a face model graph of the comparison image using , and performing face recognition through comparison between feature points in the generated comparison face model graph and a plurality of specific point information extracted based on the recognized face image may include the step of

According to an embodiment of the present invention, in the face recognition method, the rotation direction and rotation of the recognized face image through comparison between the distribution of a plurality of feature points of the recognized face image and the distribution of reference feature points on the front face graph The method may further include calculating an angle, and a K value of a KNN algorithm applied to generate the deformation reference feature point and the virtual feature point may be determined based on the rotation direction and the rotation angle.

According to the above-described embodiments of the present invention, feature point-based similar face generation capable of automatically generating a face model graph of a similar face for a face rotated at various angles and directions based on the face model graph for the front face By providing the method, various types of face model graphs can be generated using a face model graph corresponding to one frontal face.

In addition, according to the above-described embodiments of the present invention, as a face image rotated in an arbitrary direction is input, a face model graph for comparison rotated in an arbitrary direction is generated based on the face model graph of the front face, and then By recognizing the input face image based on the recognition rate, the recognition rate of the face image rotated in various directions and angles can be increased.

1 is a block diagram illustrating a configuration of a general computing device to which a method for generating a similar face and a method for recognizing a face through the method according to an embodiment of the present invention can be applied.
2 is a block diagram illustrating an internal configuration of a computing device to which an apparatus for generating a similar face and recognizing a face according to an embodiment of the present invention is applied.
3 is a flowchart illustrating an operation process of an apparatus for generating a feature point-based similarity face according to an embodiment of the present invention.
4A to 4D are diagrams for explaining a process of transforming and generating a feature point when generating a similar face based on a feature point according to an embodiment of the present invention.
5 is a flowchart illustrating an operation process of a face recognition apparatus using feature point-based similar face generation according to an embodiment of the present invention.

Hereinafter, specific embodiments of the present invention will be described with reference to the drawings. The following detailed description is provided to provide a comprehensive understanding of the methods, apparatus, and/or systems described herein. However, this is merely an example and the present invention is not limited thereto.

In describing the embodiments of the present invention, if it is determined that a detailed description of a known technology related to the present invention may unnecessarily obscure the gist of the present invention, the detailed description thereof will be omitted. In addition, the terms to be described later are terms defined in consideration of functions in the present invention, which may vary according to intentions or customs of users and operators. Therefore, the definition should be made based on the content throughout this specification. The terminology used in the detailed description is for the purpose of describing embodiments of the present invention only, and should in no way be limiting.

Hereinafter, a method for generating a similar face and a method for recognizing a face using the same will be described with reference to the accompanying drawings.

1 is a block diagram illustrating a configuration of a general computing device to which a method for generating a similar face and a method for recognizing a face through the method according to an embodiment of the present invention can be applied.

The computing device 100 according to an embodiment of the present invention is a means for authenticating a user based on face recognition, and includes an access device having a face recognition function, a gate, a mobile device that performs authentication based on face recognition, a personal computer, and the like. , but the present invention is not limited thereto.

As shown in FIG. 1 , a computing device 100 to which a similar face generating method and a face recognition method using the same according to an embodiment of the present invention are applied, includes a central processing unit (CPU) 110 , interfaces 140 , and a bus 130 (such as a peripheral component interconnect (PCI)). When operating under the control of appropriate software or firmware, CPU 110 may implement certain functions associated with the functions of a specifically configured computing device or machine. For example, in an embodiment of the present invention, a user's smartphone is configured to function as a program system for face generation, recognition, authentication, etc. utilizing CPU 110 , memory 120 , and interface(s) 140 . may be constructed or designed. Further, in an embodiment of the present invention, CPU 110 is controlled by one or more different types of software modules/components (including, for example, an operating system and any suitable application software, drivers, etc.) may be brought about to perform program functions and/or tasks such as facial recognition, creation, registration and authentication.

CPU 110 may include one or more processor(s) 114 such as, for example, a Motorola processor or an Intel microprocessor family or a MIPS microprocessor family. In an embodiment of the invention, the processor(s) 114 include hardware specifically designed to control the operations of the computing device 100 (eg, application-specific integrated circuits (ASICs), electrically erasable programmable (EEPROM) read-only memories), field-programmable gate arrays (FPGAs), etc.). In addition, memory 120 (such as non-volatile random access memory (RAM) and/or read-only memory (ROM)) also forms part of CPU 110 . However, there are many different ways in which a memory may couple with a system. Memory blocks 112 and 120 may be used for a variety of purposes, such as, for example, caching and/or storage of data, programming instructions, and the like.

As used herein, the term "processor" is not limited to just those integrated circuits referred to in the art as a processor, but is not limited to a microcontroller, a microcomputer, a programmable logic controller, an ASIC. (application-specific integrated circuit), and any other programmable circuit.

In an embodiment of the present invention, interfaces 140 are provided as interface cards (sometimes referred to as "line cards"), which control the transmission and reception of data packets over a computing network and are sometimes referred to as computing device 100 ) and other peripherals used with it. Among the interfaces 140 that may be provided are Ethernet interfaces, frame relay interfaces, cable interfaces, DSL interfaces, token ring interfaces, and the like. Also, e.g. Universal Serial Bus (USB), Serial, Ethernet, FireWire, PCI, Parallel, Radio Frequency (RF), Bluetooth, Near Field Communication (e.g. using Near Field Magnetics), 802.11 (WiFi) , Frame Relay, TCP/IP, ISDN, Fast Ethernet interfaces, Gigabit Ethernet interfaces, Asynchronous Transfer Mode (ATM) interfaces, High Speed Serial Interface (HSSI) interfaces, Point of Sale (POS) interfaces, FDDI (fiber) Various types of interfaces may be provided, such as data distributed interfaces).

These interfaces 140 may include ports suitable for communication with an appropriate medium. In some cases, they may also include an independent processor, and in some instances may include volatile and/or non-volatile memory (eg, RAM).

Although the system shown in FIG. 1 depicts one particular architecture for a computing device 100 for implementing the inventive techniques described herein, it is in no way intended that at least some of the features and techniques described herein may be implemented. It's not the only device architecture that can be. For example, architectures having one or any number of processor(s) 114 may be used, such processor(s) 114 may be present within one device or distributed among any number of devices. can In one embodiment, one processor 114 handles communications along with routing calculations. In various embodiments, different types of program features and/or functions include a client device (such as a smartphone running client software) and server system(s) (such as a server system to be described in greater detail below). It can be implemented in a program system such as face recognition, generation, registration and authentication.

Irrespective of the network device configuration, the system of the present invention provides data, program instructions for general-purpose network operations, and/or other functions related to the function of program techniques for implementing the similar face generating method described herein and the face recognition method using the same. One or more memories or memory modules (eg, a memory block) configured to store information may be employed. Program instructions may, for example, control operation of an operating system and/or one or more application programs. The memory or memories may also be configured to store data structures, information related to the registered face image, similar face information generated based on the registered face image, and/or other specific non-program information described herein.

Because such information and program instructions may be employed to implement the systems/methods described herein, at least some network device embodiments may include a nontransitory machine-readable storage medium, A temporary machine-readable storage medium may be configured or designed to store, for example, program instructions, state information, and the like for performing various tasks described herein. Examples of such non-transitory machine-readable media include magnetic media such as hard disks, floppy disks, and magnetic tape; optical media such as CD-ROM disks; magneto-optical media, such as floptical disks, and read-only memory devices (ROM) specially configured to store and execute program instructions, flash memory, memristor memory, random access memory (RAM) hardware devices such as, but not limited to. Examples of program instructions include both machine code, such as generated by a compiler, and files containing higher-level code that can be executed by a computer using an interpreter.

Meanwhile, as shown in FIG. 2 , the computing device 100 having the above-described configuration includes a similar face generating apparatus 200 and a face recognition apparatus 300 for generating a feature-based similar face, and an input device. 150 and an output device 160 .

In an embodiment of the present invention, the similar face generating apparatus 200 and the face recognition apparatus 300 may be implemented in a module form consisting of a plurality of instruction combinations, and may be implemented in a form executable by the processor 114 in the memory 120 . can be saved.

Meanwhile, the similar face generating apparatus 200 and the face recognition apparatus 300 may operate in connection with the input device 210 and the output device 220 in the computing device 100 . Specifically, the similar face generating apparatus 200 and the face recognition apparatus 300 may generate a similar face or perform face recognition by operating as an image is input through the input device 210 , that is, a camera.

Specific functions and operations of the similar face generating apparatus 200 according to the embodiment of the present invention as described above will be described with reference to FIGS. 3 and 4A to 4D .

As shown in FIG. 3 , first, the similar face generating apparatus 200 extracts a plurality of reference feature points as shown in FIG. 4A based on the front face image, and connects the extracted reference feature points as shown in FIG. 4B . A face model graph (FMG) for the same front face may be generated and stored in the memory 120 (S202).

Then, when there is a request for generating a face model according to the face image rotated in an arbitrary direction (eg, the left direction), the similar face generating apparatus 200 converts the plurality of reference feature points to the left direction to convert the face rotated in the left direction. It is possible to generate a deformation reference feature point for (S204). Specifically, the similar face generating apparatus 200 converts a plurality of reference feature points in a manner that converts the distance between the reference feature points on the assumption that the face is rotated by a preset angle in the left direction, and the face rotated in the left direction It is possible to create a deformation reference feature point for . For example, assuming that the face is rotated by a preset angle in the left direction, the similar face generating apparatus 200 rotates the reference feature point in the left direction by a preset angle and then calculates the distance between the reference feature points in the area corresponding to the angle. In addition to transforming (narrowing), the deformation reference feature point may be generated by moving, generating, removing, or disposing the reference feature point.

Thereafter, as shown in FIG. 4C , the similar face generating apparatus 200 may generate at least one virtual feature point by using the deformation reference feature point adjacent to the region 400 in the opposite direction to the rotated direction ( S206).

As described above, the KNN algorithm and the Blend Shape algorithm may be used as a method used for generating the deformation reference feature point and the virtual feature point. Specifically, by using the KNN algorithm to generate a feature point in a part without a feature point, and then moving and relocating the created feature point using the blend shape algorithm, the transformation reference feature point and virtual feature point are created, or the existing feature point is converted into a blend shape. After moving and relocating using an algorithm, the KNN algorithm is applied to the moved and relocated feature points to generate the feature points in an adjacent area to generate the deformation reference feature points and the virtual feature points.

The KNN algorithm selects at least one deformation reference feature point adjacent to the direction opposite to the rotated direction based on the K value, and generates a virtual feature point in a predetermined area 400 in the opposite direction to the rotated direction using the selected deformation reference feature point. can do. In this case, K is a constant related to a selection area for selecting adjacent deformation reference feature points, and may be determined by at least one parameter selected from a rotated angle and a distribution of deformation reference feature points.

The blend shape algorithm may generate a virtual feature point in the region 400 corresponding to the direction opposite to the rotation direction by moving or relocating the deformation reference feature point in the region adjacent to the rotation direction. Here, the region 400 may be determined based on a rotation angle.

Then, as shown in FIG. 4D , the similar face generating apparatus 200 generates a similar facial feature point corresponding to the face rotated by a preset angle in the left direction by using the deformation reference feature point and the virtual feature point (S208) Then, similar facial feature points may be connected to generate a similar facial model graph (S210).

By repeatedly performing the above-described steps, the similar face generating apparatus 200 may generate a similar face model graph having similar facial feature points for faces rotated in various directions and angles based on the front face model graph.

The face recognition apparatus 300 may recognize a face rotated in various directions based on the similar face generating apparatus 200 as described above. This will be described with reference to FIG. 5 .

5 is a flowchart illustrating a face recognition process using similar face generation according to an embodiment of the present invention.

As shown in FIG. 5 , the face recognition apparatus 300 extracts a plurality of reference feature points as shown in FIG. 4A based on the front face image, and connects the extracted reference feature points to the front face as shown in FIG. 4B . A face model graph (FMG) for a face may be generated and stored in the memory 120 (S302).

Then, when a face image rotated in an arbitrary direction is received from the input device 210 ( S304 ), the face recognition apparatus 300 may extract a plurality of feature points from the face image rotated in an arbitrary direction ( S304 ). S306).

Thereafter, the face recognition apparatus 300 may recognize the rotation direction and rotation angle of the face image by using the reference feature points stored in the memory 120 and the plurality of extracted feature points ( S308 ).

Specifically, the recognition of the rotation direction and rotation angle will be described. The face recognition apparatus 300 calculates the distance between a plurality of extracted feature points, calculates the face depth information using the calculated distance, and then calculates the face depth information based on the face depth information. direction of rotation can be recognized. Specifically, the face recognition apparatus 300 recognizes the rotation direction of the face to the left when the portion with the shorter distance between the plurality of feature points is on the left, and when the portion with the shorter distance between the plurality of feature points is on the right The direction of rotation of the face can be recognized to the right.

In addition, the face recognition apparatus 300 may detect a portion having similarity to the central portion of the plurality of reference feature points among the extracted feature points, calculate how much the detected portion from the central portion moves, and recognize the rotation angle.

Thereafter, the face recognition apparatus 300 may convert a plurality of reference feature points based on a rotation angle and a rotation direction to generate a modified reference feature point of the rotated face image ( S310 ). Here, the deformation reference feature point may be generated using a KNN algorithm and a blend shape algorithm to which a K value considering a rotation angle and a rotation direction is applied.

Then, the face recognition apparatus 300 may generate a virtual feature point using the deformation reference feature point adjacent to a specific region in a direction opposite to the rotation direction ( S312 ). The face recognition apparatus 300 according to an embodiment of the present invention may generate virtual feature points using the KNN algorithm and the Blend Shape algorithm as described above.

Then, the face recognition apparatus 300 may generate a similar face model graph for a face rotated in an arbitrary direction by using the transformation reference feature point and the virtual feature point ( S314 ).

Thereafter, the face recognition apparatus 300 may perform face recognition by comparing the feature points on the similar face model graph and a plurality of feature points extracted based on the recognized face image ( S316 ).

The above description of the present application is for illustration, and those of ordinary skill in the art to which the present application pertains will understand that it can be easily modified into other specific forms without changing the technical spirit or essential features of the present application. Therefore, it should be understood that the embodiments described above are illustrative in all respects and not restrictive. For example, each component described as a single type may be implemented in a distributed manner, and likewise components described as distributed may be implemented in a combined form.

The scope of the present invention is indicated by the following claims rather than the above detailed description, and all changes or modifications derived from the meaning and scope of the claims and their equivalent concepts should be construed as being included in the scope of the present application. .

100: computing device
200: similar face generating device
210: input device
220: output device
300: face recognition device
400 : area

Claims (5)

generating a front face model graph in which the plurality of reference feature points are connected by extracting a plurality of reference feature points based on the front face image;
In response to a request for generating a face model for a face rotated by a predetermined angle in an arbitrary direction based on the front face, the plurality of reference feature points are transformed in the rotated direction to generate a transformation reference feature point for the rotated face, generating the modified reference feature point by using at least one of moving, generating, removing, and rearranging the plurality of reference feature points;
generating at least one or more virtual feature points in the predetermined area by using the deformation reference feature points adjacent to the predetermined area in a direction opposite to the arbitrary direction, wherein the predetermined area is determined based on the predetermined angle;
generating a face model graph of the rotated face using the transformation reference feature point and the virtual feature point,
The deformation reference feature point is,
It is generated using the feature points generated in the part without the reference feature point through the KNN algorithm to which the K value determined by the arbitrary direction and the predetermined angle is applied,
The virtual feature point is
It is created by moving and rearranging the generated deformation reference feature point using a blend shape algorithm,
The K is for determining a selection area for selecting a deformation reference feature point when generating the virtual feature point, and is determined by at least one parameter among the predetermined angle and the distribution of the deformation reference feature point, and the reference feature point when the deformation reference feature point is generated A method for generating a similar face based on feature points for determining a selection area for selecting , which is determined by at least one of the predetermined angle and the distribution of a reference specific point.
According to claim 1,
The deformation reference feature point is,
It is created by moving and rearranging the reference feature point using a blend shape algorithm,
The virtual feature point is
A feature point-based similar face generating method generated by using the feature points generated in the selection area through the KNN algorithm to which the K value is applied.
generating a front face model graph corresponding to the front face by extracting a plurality of reference feature points based on the front face image;
Recognizing a face image rotated in an arbitrary direction and extracting a plurality of feature points based on it;
converting a plurality of reference feature points on the front face model graph in the arbitrary direction to generate a modified reference feature point;
generating at least one virtual feature point in the predetermined region by using a deformation reference feature point adjacent to a predetermined region in a direction opposite to the arbitrary direction;
generating a face model graph of an image for comparison using the transformation reference feature point and the virtual feature point;
performing face recognition through comparison between feature points in the generated comparison face model graph and a plurality of specific point information extracted based on the recognized face image,
The deformation reference feature point is,
It is generated using the feature points generated in the part without the reference feature point through the KNN algorithm to which the K value determined by the arbitrary direction and the predetermined angle is applied,
The virtual feature point is
It is created by moving and rearranging the generated deformation reference feature point using a blend shape algorithm,
The K is for determining a selection area for selecting a deformation reference feature point when generating the virtual feature point, and is determined by at least one parameter among the predetermined angle and the distribution of the deformation reference feature point, and the reference feature point when the deformation reference feature point is generated A face recognition method using feature point-based similar face generation determined by at least one of the predetermined angle and the distribution of reference specific points for determining a selection area for selecting .
4. The method of claim 3,
The deformation reference feature point is,
It is created by moving and rearranging the reference feature point using a blend shape algorithm,
The virtual feature point is
A face recognition method using the feature point-based similar face generation generated using the feature points generated in the selection area through the KNN algorithm to which the K value is applied. delete

Images