KR20130071750A - System and method for face recognition - Google Patents

Abstract

PURPOSE: A face recognition system and a method thereof are provided to control pose of a three-dimensional image, by extracting a feature point from the three-dimensional image and then using the extracted point. CONSTITUTION: An image acquisition unit (110) acquires first and second image data through a sensor. The image acquisition unit generates three-dimensional mesh data by using the second image data. A feature point unit (120) extracts a number of feature points by using the three-dimensional mesh data. An image segmenting unit (130) generates segmented mesh data showing a specific area by segmenting the three-dimensional mesh data. [Reference numerals] (110) Image acquisition unit; (120) Feature point unit; (130) Image segmenting unit; (140) Image matching unit; (150) Face recognizing unit; (160) Image DB; (170) Control unit

Description

[0001] SYSTEM AND METHOD FOR FACE RECOGNITION [0002]

The present invention relates to a face recognition system and a method thereof, and more particularly, to extract a feature point from a 3D image using a feature point extraction algorithm and to control a pose of the 3D image using the extracted extraction point. A facial recognition system for improving speed and a method thereof.

Biometrics technology refers to a technique of authenticating an individual's identity by using human characteristics such as face, fingerprint, iris, and DNA.

Recently, an automatic technology for personal authentication using images has been studied, one of which is a face recognition technique for recognizing a face by information obtained by processing an image signal.

Such face recognition technology has been studied based on two-dimensional images, and is used in various fields such as identification, access control, supervision, and database search.

In addition, a lot of research is being conducted to develop a face recognition technology that is robust against various environmental changes (for example, changes in lighting and pose changes).

In particular, various researches on feature extraction techniques for extracting feature points from images among face recognition techniques have been attempted.

Recently, interest in face recognition technology based on not only two-dimensional images but also three-dimensional images is increasing. However, research on face recognition technology using three-dimensional images is still insufficient.

In particular, in face recognition technology using a 3D image, the recognition rate and the processing speed are problematic.

The present invention is to solve the above problems, by using a feature extraction algorithm to extract a feature point from the 3D image and to improve the recognition rate and processing speed by controlling the pose of the 3D image using the extracted extraction point An object of the present invention is to provide a face recognition system and a method thereof.

A face recognition system for achieving the above object includes: an image acquisition unit for acquiring first image data and second image data through a sensor, and generating three-dimensional mesh data using the second image data; A feature point unit for extracting a plurality of feature points using the 3D mesh data; An image divider configured to divide the 3D mesh data based on the plurality of feature points and generate divided mesh data representing a specific area; And an image matching unit configured to generate the divided image data by matching the divided mesh data with the first image data.

The feature point unit may include a feature point extracting unit extracting a reference feature point and first to third feature points using the 3D mesh data; It may include a pose control unit for controlling the pose of the three-dimensional mesh data compared to the comparison image received from the image DB by using the first to third feature points.

The feature point extractor extracts a first feature point by tracking in the vertical direction (y-axis direction) with the reference feature point as a starting point, and extracts the first feature point in a first horizontal direction (x-axis positive direction) based on the first feature point. The second feature point may be extracted by tracking, and the third feature point may be extracted by tracking in a first horizontal direction (x-axis negative direction) based on the first feature point.

The reference feature point may be a nose end coordinate point, which is a coordinate point including a maximum height coordinate in the 3D mesh data.

The apparatus may further include an image matching unit configured to generate the divided image data by matching the divided mesh data with the first image data.

The apparatus may further include a face recognizing unit that recognizes a face by analyzing and comparing the plurality of comparison images received from the image DB and the divided image data received from the image matching unit.

According to an embodiment of the present invention for achieving the above object, a face recognition method obtains first image data and second image data through a sensor, and generates 3D mesh data using the second image data. Generating; Extracting a plurality of feature points using the 3D mesh data; Controlling a pose of the 3D mesh data by comparing the comparison image received from the image DB using the plurality of feature points; And dividing the image data based on the plurality of feature points to generate divided mesh data representing a specific region.

In the extracting of the plurality of feature points, a first feature point is extracted by tracking in the vertical direction (y-axis direction) with the reference feature point as a starting point, and a first horizontal direction (x axis) based on the first feature point. The second feature point can be extracted by tracking in the positive direction), and the third feature point can be extracted by tracking in the first horizontal direction (x-axis negative direction) based on the first feature point.

The method may further include generating split image data by matching the split mesh data with the first image data.

The method may further include recognizing a face by analyzing and comparing the plurality of comparison images received from the image DB and the divided image data received from the image matching unit.

As described above, in the face recognition system and method according to the present invention, the feature rate extraction algorithm extracts a feature point from the 3D image and uses the extracted extraction point to control the pose of the 3D image and the processing rate. You can improve speed.

In addition, as face recognition is performed using a specific image, a face recognition system robust to facial expression change may be provided.

1 is an internal block diagram of a face recognition system according to an embodiment of the present invention.
2 is a flowchart illustrating a face recognition process of the face recognition system according to an exemplary embodiment of the present invention.
3A to 3D are diagrams for explaining a feature point extraction process in a face recognition process of a face recognition system according to an exemplary embodiment of the present invention.
4A and 4B are views referred to for describing the first feature point extraction process in FIG. 3A.
5A and 5B are views referred to for describing the second feature point extraction process in FIG. 3B.
6 is a diagram illustrating a first segmented image used in a face recognition process according to an exemplary embodiment of the present invention.
FIG. 7 illustrates a second divided image used in a face recognition process according to an exemplary embodiment of the present invention. FIG.
8 is a diagram illustrating a third segmented image used in a face recognition process according to an embodiment of the present invention.

Hereinafter, embodiments of the present invention will be described in detail with reference to the drawings.

1 is an internal block diagram of a face recognition system according to an embodiment of the present invention.

As shown in FIG. 1, the face recognition system 100 includes an image acquirer 110, a feature point unit 120, an image divider 130, an image matcher 140, a face recognizer 150, The image DB 160, the controller 170, and the like are included.

The image acquirer 110 may acquire the first image data and the second image data through the first sensor and the second sensor.

In this case, the first sensor and the second sensor may be an image sensor and a depth sensor, respectively.

The first image data and the second image data may be two-dimensional image data for an object for which a face is to be recognized and depth image data for obtaining three-dimensional image data of the object.

That is, the image acquisition unit 110 may obtain two-dimensional image data of an object for which a face is to be recognized through a first sensor and obtain depth image data for obtaining three-dimensional image data of the object through a second sensor. have.

The image acquisition unit 110 may generate 3D mesh data using depth image data, which is second image data.

Here, the 3D mesh data may be, for example, 3D image data including a plurality of coordinate points including various coordinate values obtained by a triangulation method in the sense of depth image data.

In this case, the coordinates may be x coordinates indicating horizontal coordinates, y coordinates indicating vertical coordinates, and z coordinates indicating height coordinates.

The feature point unit 120 includes a feature point extractor (not shown) and a pose control unit (not shown). The feature point unit 120 includes a reference feature point and first to third feature points using 3D mesh data received from the image acquisition unit 110. Extract and control the pose of the 3D mesh data using the first to third feature points.

In this case, controlling the pose of the 3D mesh data may be, for example, controlling the position (direction) so that the 3D mesh data becomes the image data in the front direction.

Here, the reference feature point is a feature point that is a reference point for feature point extraction, the first feature point is a feature point extracted by the feature point extraction algorithm using the reference feature point, and the second feature point and the third feature point are feature point extraction algorithms using the first feature point. It is a feature point extracted by.

For example, the reference feature point may be a feature point corresponding to a nose end coordinate point, which is the most prominent coordinate point in the 3D mesh data.

That is, the feature point extraction algorithm may extract another feature point from the nose point coordinate point, which is a coordinate point including the maximum height coordinate, in the 3D mesh data.

The feature point extraction algorithm may extract the first feature point, for example, by tracking in a vertical direction (y-axis direction) using the reference feature point as a starting point.

The feature point extractor may extract the second feature point by tracking in the first horizontal direction (the x-axis positive direction) based on the first feature point.

Similarly, the feature point extractor may extract the third feature point by tracking in a first horizontal direction (x-axis negative direction) based on the first feature point.

The pose controller may control the pose of the 3D mesh data by comparing the comparison image received from the image DB 160 using the first to third feature points.

For example, the center of gravity of the triangle including the reference feature point of the comparison image and the center of gravity of the triangle including the reference feature point of the 3D mesh data may be matched and then rotated to match the triangle.

That is, the pose controller controls the pose of the 3D mesh data before face recognition to increase the face recognition rate of the face recognition system.

The image divider 130 may generate divided mesh data representing a specific region by dividing the 3D mesh data based on the first to third feature points extracted by the feature point extractor.

In this case, the divided mesh data may be, for example, 3D mesh data including a nose area of a face.

The image matching unit 140 matches the divided mesh data received from the image dividing unit 130 with the 3D mesh data received from the image obtaining unit 110 and the first image data (2D image data). The divided image data (three-dimensional image data) which are shown can be produced | generated.

In the face recognition system according to the present invention, face recognition is performed using split image data.

The face recognition unit 150 may perform face recognition by analyzing and comparing the plurality of comparison images received from the image DB 160 and the divided image data received from the image matching unit 140.

In this case, in more detail, an area including a nose area of a human face may be divided into face image data including an area including a nose and an eye.

As described above, in the process of performing face recognition using the divided image data, for example, meaningful pixels may be extracted from the divided image data including a plurality of pixels (coordinate points) by using a vector value.

The meaningful pixels may be used to compare whether there is an image matching the divided image data among the plurality of comparison images.

The face recognition system according to the present invention can increase the recognition rate by performing face recognition using the split image data.

In addition, since the face recognition system according to the present invention uses divided image data having a smaller size of image data used for face recognition, the processing speed may be improved.

The controller 170 may control the overall operation of the face recognition system.

2 is a flowchart illustrating a face recognition process of the face recognition system according to an exemplary embodiment of the present invention, and FIGS. 3A to 3D illustrate a feature point extraction process of the face recognition process of the face recognition system according to an exemplary embodiment of the present invention. It is a figure referred to for. This will be described with reference to FIG. 1 further.

As shown in FIG. 2, the image acquisition unit 110 of the face recognition system 100 obtains image data (first image data and second image data) through sensors (first sensor and second sensor). (S100).

That is, the image acquisition unit 110 may obtain two-dimensional image data of an object for which a face is to be recognized through a first sensor and obtain depth image data for obtaining three-dimensional image data of the object through a second sensor. have.

Here, the image acquirer 110 may generate 3D mesh data using depth image data, which is second image data.

Then, the feature point unit 120 extracts a plurality of feature points using the 3D mesh data received from the image acquisition unit 110 (S110).

In this case, the plurality of feature points may be reference feature points and first to third feature points.

Here, the reference feature point is a feature point that is a reference point for feature point extraction, the first feature point is a feature point extracted by the feature point extraction algorithm using the reference feature point, and the second feature point and the third feature point are feature point extraction algorithms using the first feature point. It is a feature point extracted by.

For example, the reference feature point may be a feature point corresponding to a nose end coordinate point, which is the most prominent coordinate point in the 3D mesh data.

A feature point extraction process will be described with reference to FIGS. 3A to 3D.

First, as illustrated in FIG. 3A, the feature point extractor may track in a vertical direction (y-axis direction) using the reference feature point as a starting point to extract the first feature point.

In this case, the reference feature point may be a feature point corresponding to a nose end coordinate point, which is the most prominent coordinate point in the 3D mesh data.

As shown in FIG. 3B, the feature point extractor may track in the first horizontal direction (the x-axis positive direction) based on the first feature point to extract the second feature point.

Next, as shown in FIG. 3C, the feature point extractor may track in the first horizontal direction (the x-axis negative direction) based on the first feature point to extract the third feature point.

As a result, as illustrated in FIG. 3D, the first to third feature points may be extracted from the 3D mesh data.

After extracting the first to third feature points as described above, the pose controller controls the pose of the image data (3D mesh data) by comparing the comparison images received from the image DB using the first to third feature points (S120). )

For example, the center of gravity of the triangle including the reference feature point of the comparison image and the center of gravity of the triangle including the reference feature point of the 3D mesh data may be matched and then rotated to match the triangle.

Next, image data may be divided based on the first to third feature points (S130) to generate divided mesh data representing a specific area.

In this case, the divided mesh data may be, for example, 3D mesh data including a nose area of a face.

The image matching unit 140 matches the second image data (two-dimensional image data) and the image data (three-dimensional mesh data) received from the image dividing unit 130 (S140).

As a result of the matching, split image data (3D image data) representing a specific region may be generated.

In addition, the face recognition unit 150 analyzes and compares a plurality of comparison images received from the image DB 160 and image data (split image data) received from the image matching unit 140 (S150 and S160).

In this case, in more detail, an area including a nose area of a human face may be divided into face image data including an area including a nose and an eye.

As described above, in the process of performing face recognition using the divided image data, for example, meaningful pixels may be extracted from the divided image data including a plurality of pixels (coordinate points) by using a vector value.

The meaningful pixels may be used to compare whether there is an image matching the divided image data among the plurality of comparison images.

The face recognition system according to the present invention can increase the recognition rate by performing face recognition using the split image data.

In addition, since the face recognition system according to the present invention uses divided image data having a smaller size of image data used for face recognition, the processing speed may be improved.

4A and 4B are views referred to for explaining the first feature point extraction process in FIG. 3A, and FIGS. 5A and 5B are views for explaining the second feature point extraction process in FIG. 3B.

4A and 4B, the feature point extracting unit tracks the first triangle and the second hypotenuse including the first hypotenuse l1 by tracking in the vertical direction (y-axis direction) with the reference feature point A as a starting point. A second triangle including l2) may be set, and the contact point B of the first triangle and the second triangle may be extracted as the first feature point.

In this case, the first triangle has a first hypotenuse l1 and a first height h1, and the second triangle has a second hypotenuse L2 and a second height h2.

The values of the first hypotenuse l1, the first height h1, the second hypotenuse l2, and the second height h2 may be determined by coordinate values of the respective vertices.

That is, the feature point extractor sets the first triangle and the second triangle, and extracts the point B as the first feature point B such that the first angle θ1 and the second angle θ2 are within a specific range. can do.

In this case, since the first angle θ1 and the second angle θ2 may be different for each person, a specific range of values, which are values previously set as experimental data through a plurality of people, is used for feature point extraction.

As shown in FIGS. 5A and 5B, the feature point extracting unit includes a third triangle including a third hypotenuse l3 by tracking in a first horizontal direction (x-axis positive direction) with respect to the first feature point B; A fourth triangle including the fourth hypotenuse l4 may be set, and the contacts D of the third and fourth triangles may be extracted as the second feature point.

In this case, the third triangle has a third hypotenuse l3 and a third height h3, and the fourth triangle has a fourth hypotenuse l4 and a fourth height h4.

The values of the third hypotenuse l3, the third height h3, the fourth hypotenuse l4, and the fourth height h4 may be determined by coordinate values of the respective vertices.

That is, the feature point extractor sets the third and fourth triangles, and extracts the point D as the second feature point D such that the third angle θ3 and the fourth angle θ4 are within a specific range. can do.

Similarly, since the third angle θ3 and the fourth angle θ4 may be different for each person, a specific range of values, which are values previously set as experimental data through a plurality of people, is used for feature point extraction.

Although not shown, the feature point extracting unit tracks in a first horizontal direction (x-axis negative direction) based on the first feature point B, and includes a fifth triangle including a fifth hypotenuse and a sixth hypotenuse. A triangle may be set and the contacts of the fifth and sixth triangles may be extracted as the third feature point.

At this time, the fifth triangle has a fifth hypotenuse and a fifth height, and the sixth triangle has a sixth hypotenuse and a sixth height.

The fifth hypotenuse, fifth height, sixth hypotenuse, and sixth height may be determined by the coordinates of the respective vertices.

That is, the feature point extracting unit sets the fifth triangle and the sixth triangle, and extracts a point for allowing the fifth and sixth angles to be within a specific range as the third feature point.

6 is a diagram illustrating a first segmented image used in a face recognition process according to an embodiment of the present invention, and FIG. 7 is a diagram illustrating a second segmented image used in a face recognition process according to an embodiment of the present invention. 8 is a diagram illustrating a third segmented image used in a face recognition process according to an embodiment of the present invention. This will be described with reference to FIG. 4A further.

As illustrated in FIG. 6, the image segmentation unit 130 of FIG. 1 may generate divided mesh data by dividing a rectangular area around the reference feature point A in the 3D mesh data.

As illustrated in FIG. 7, the image divider 130 may generate divided mesh data by dividing a rectangular area around the point C in the 3D mesh data.

As shown in FIG. 8, the image divider 130 may generate divided mesh data by dividing a rectangular area around the first feature point B in the 3D mesh data.

In this case, face recognition using segmented image data generated by using segmented mesh data by dividing a rectangular region centering on point C in the 3D mesh data has the highest recognition rate and the fastest processing speed.

The recognition rate and CPU time before applying the feature extraction algorithm were 79.6% and 2531.2 (s), respectively.

On the other hand, applying the feature extraction algorithm according to the present invention and performing face recognition using segmented image data, the recognition rate and CPU time were 84.42% and 1594.4 (s), respectively.

That is, in the face recognition system according to the present invention, the recognition rate and the processing speed may be improved by extracting the feature point from the 3D image using the feature point extraction algorithm and controlling the pose of the 3D image using the extracted extraction point.

In addition, as face recognition is performed using a specific image, a face recognition system robust to facial expression change may be provided.

The embodiments of the present invention as described above are merely illustrative, and those skilled in the art can make modifications without departing from the gist of the present invention. Accordingly, the protection scope of the present invention includes modifications of the present invention within the scope of the appended claims and equivalents thereof.

100: face recognition system 110: image acquisition unit
120: feature point 130: image segmentation
140: image matching unit 150: face recognition unit
160: image DB 170: control unit

Claims (10)

An image acquisition unit obtaining first image data and second image data through a sensor, and generating 3D mesh data using the second image data;
A feature point unit for extracting a plurality of feature points using the 3D mesh data;
An image divider configured to divide the 3D mesh data based on the plurality of feature points and generate divided mesh data representing a specific area;
Face recognition system comprising a.
The method of claim 1,
The feature point,
A feature point extracting unit extracting a reference feature point and first to third feature points using the 3D mesh data;
Pose control unit for controlling the pose of the three-dimensional mesh data compared to the comparison image received from the image DB by using the first to third feature points
Face recognition system comprising a.
The method of claim 2,
The feature point extracting unit
The first feature point is extracted by tracking in the vertical direction (y-axis direction) using the reference feature point as a starting point,
Extracting a second feature point by tracking in a first horizontal direction (x-axis positive direction) based on the first feature point,
And a third feature point is extracted by tracking in a first horizontal direction (x-axis negative direction) based on the first feature point.
The method of claim 3,
The reference feature point is a face recognition system, characterized in that the nose end coordinate point that is the coordinate point including the maximum height coordinate in the three-dimensional mesh data.
The method of claim 1,
And a video matching unit configured to generate the split image data by matching the split mesh data with the first image data.
The method of claim 5,
And a face recognizing unit which analyzes and compares the plurality of comparison images received from the image DB and the divided image data received from the image matching unit to recognize a face.
Acquiring first image data and second image data through a sensor, and generating 3D mesh data using the second image data;
Extracting a plurality of feature points using the 3D mesh data;
Controlling a pose of the 3D mesh data by comparing the comparison image received from the image DB using the plurality of feature points;
Generating split mesh data representing a specific area by dividing image data based on the plurality of feature points.
And a face recognition step of recognizing a face of the face.
The method of claim 7, wherein
In the extracting of the plurality of feature points,
The first feature point is extracted by tracking in the vertical direction (y-axis direction) using the reference feature point as a starting point,
Extracting a second feature point by tracking in a first horizontal direction (x-axis positive direction) based on the first feature point,
And a third feature point is extracted by tracking in a first horizontal direction (x-axis negative direction) based on the first feature point.
The method of claim 7, wherein
And matching the split mesh data with the first image data to generate split image data.
10. The method of claim 9,
And recognizing a face by analyzing and comparing the plurality of comparison images received from the image DB and the divided image data received from the image matching unit.

Images