JP3089605B2 - Face reference point extraction method - Google Patents

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a method for extracting, from an image, a reference point used for aligning a face in the image.

[0002]

2. Description of the Related Art Conventionally, recognition and classification of an individual using a two-dimensional face image is performed by using a feature point extracted from a target face image, performing registration and normalization, and then using a pattern matching method or the like. Has been done. At this time, extraction of features necessary for alignment and normalization is performed by obtaining edge information by binarization or by applying various operators, and analyzing the edge information.

[0003]

However, according to the conventional method, necessary features are extracted by binarizing or obtaining edge information by applying various operators and analyzing the edge information. Feature extraction cannot be performed stably. Positioning and normalization cannot be performed accurately without performing stable feature extraction.
There was a problem that classification could not be performed well.

SUMMARY OF THE INVENTION The present invention has been made to solve the above problems, and an object of the present invention is to provide a method for extracting a reference point used for aligning a face in an image from the image. It is an object of the present invention to provide a technique capable of stably extracting a reference point by using the method.

The above and other objects and novel features of the present invention will become apparent from the description of the present specification and the accompanying drawings.

[0006]

To SUMMARY OF THE INVENTION To achieve the above object, a face reference point extraction method of the present invention, the color information from the input image
As the I and Q components of the YIQ display system, and H
Extracting the H component of the VS display system (1);
Using the I component extracted in (1), from the input image
Step (2) of extracting a face region and extraction in the step (2)
Generating a face mask area from the extracted face area (3)
And the face mask area generated in the step (3);
Using the H component extracted in the process (1), a lip candidate of the face
Step (4) of extracting an area, and the area extracted in the step (4).
From the extracted lip candidate area, the Q component extracted in the step (1).
The process of extracting the lip region and extracting the mouth reference point using the minute
(5) and the face mask area generated in the step (3)
And the face area extracted in the step (2), or
Using the H component extracted in step (1), face eye candidates
Step (6) of extracting an area, and the area extracted in the step (6).
Lip region extracted in step (5) from the selected eye candidate region
Extracting eye reference point candidates based on the area (7);
Step (8) of examining whether the eye reference point candidate extracted in step (7) is a correct reference point, and extracting an eye reference point
And characterized in that:

In other words, in order to stably extract a reference point from a face image, attention is paid not to edge information indicating the features of the eyes and mouth but to color information. Extract. The correct candidate is selected from the candidate regions by comparing the extracted candidate region with the template.

[0008]

According to the above-mentioned means, when performing personal identification using a human face image, edge information conventionally used for extracting a face reference point required for normalization of position and size is used. Performs stable position and size normalization by obtaining from areas corresponding to facial features, such as eyes and mouth, which can be extracted more stably, rather than based on feature point extraction based on them. be able to.

[0009]

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS One embodiment of the present invention will be described below in detail with reference to the drawings. Here, as an example of an image recognition system for recognizing a person's face, reference [1] << Sasaki, Akamatsu, Suenaga, "Face Image Collation Device", Japanese Patent Application No. 2-21
9675, filed on Aug. 20, 1990 >>, and using the reference point extraction unit of the face image recognition system as an example.

FIG. 1 is a block diagram showing a schematic functional configuration of an embodiment of the present invention. Reference numeral 1 denotes a target face image whose reference point is to be extracted, 2 denotes a density conversion processing unit, and 3 denotes a color system conversion. Processing unit, 4 a reference point candidate extraction processing unit, 5 a test processing unit, 6
Denotes a processing result output, and 7 denotes a control unit that manages the progress of the entire processing. Here, the above-mentioned 2,3,4, ... 7
Is a component that can be constructed in the same computer.

First, a target face image 1 of a person photographed in color by a television camera or the like is sent to a density conversion processing unit 2.

The density conversion processing unit 2 performs density conversion processing such as luminance value correction (luminance value normalization), noise removal, and difference with a previously stored background on the target face image. Then, processing such as background separation is performed.

The color system conversion processor 3 obtains hue components and color difference components from the color face image sent through the density converter 2 by converting the color system. As the hue component (Hue: hereinafter referred to as H component), for example, the H component of the HVS color system can be used. The color difference component is, for example, Y
I and Q components of the IQ color system can be used. here,
A description will be given of a case where a general HVS color system and a YIQ color system are used, but it goes without saying that any color system having a hue component and a color difference component can be applied. HSV color system and Y
For the conversion method for converting to the IQ color system, see [2] << ARSm
ith, "Color Gamut Transform Pairs," Computer Graphi
cs, Vol. 12, pp. 12-19, 1978 (Proc. SIGGRAPH78)> and the like. Here, the HSV and YIQ color components are accumulated until the next input, and the accumulated information is provided as needed.

Next, as a specific example, the reference point candidate extraction processing section 4 processes the H component and the I component obtained by the three color system conversion processing sections, and refers to the Q component while referring to the Q component. The process of extracting a reference point candidate will be described. But,
It can be composed of the H component and the Q component excluding the I component,
The reason why the I component is used here is to enhance the complementary effect and ensure the processing result. The process here is
This will be described with reference to FIG.

FIG. 2 is a block diagram showing a functional configuration of the reference point candidate extraction processing unit 4, wherein reference numeral 41 denotes an I component processing unit;
Is an H component processing unit, 43 is a Q component storage memory, 44 is a face mask creating unit, 45 is a lip region extracting unit, and 46 is an eye region candidate extracting unit.

The I component processing unit 41 utilizes the property of the I component having a strong peak with respect to human skin, for example, using the method of Otsu (reference [4] << Otsu, " Automatic Threshold Selection Method Based on Information Technology, ”IEICE (D), Vol.J6
3-D, pp. 349-356, (1980-4) >>) and the like, and binarization is performed to obtain a face area. Even when there are a plurality of obtained regions, a face region can be easily obtained by using the shape of a person's face or by comparing the areas of the regions.
The obtained face area is sent to the face mask creating unit 44. Here, the convenience F ₀ that of the face region. FIG. 3 shows a processing example.

The H component processing section 42 obtains a value in a range where a human skin color exists, that is, a skin area by threshold processing, and sends it to the lip area extracting section 45. This skin area is referred to as F for convenience.
_{Assume 3} . The threshold value at the time of processing varies depending on the target race, etc., but is a value that can be obtained by learning or statistically using a neural network. "Basics of Color Engineering", Asakura Shoten. >, A value in the range of 1.3 YR to 7.4 YR may be used. FIG. 4 shows a processing example.

The Q component storage memory 43 stores a color system conversion process.
This is where the Q component of the face image obtained by the unit 3 is stored, and is referred to when the lip region extracting unit 45 performs processing.

In the face mask creating section 44, the I component processing section 4
The face mask area Fm is obtained by performing, for example, the boundary tracking of the area inside the F ₀ area sent from 1 and painting the inside of the boundary. FIG. 5 shows a processing example.

In the lip region extraction unit 45, the H component processing unit 4
2 and the Fm obtained by the face mask creation unit 44
Between the regions

[0021]

(Equation 1)

The lip candidate area 45 is calculated by the above operation (see FIG. 6).

Next, a region having the highest Q component intensity is selected as a lip region from the lip candidate region group by referring to the Q component storage memory 43 (see FIG. 7).
It is also conceivable to consider the area and flatness of the region in addition to the intensity of the Q component, or to weight the evaluation.

Next, the mouth reference point is obtained by finding the center of gravity of the selected lip region, or by finding both end points of the region and then finding the middle point.

In the eye region candidate extraction unit 46, between the F ₀ region obtained by the I component processing unit 41 and the Fm region obtained by the face mask creation unit 44,

[0026]

(Equation 2)

An eye area candidate is obtained by performing the following operation (see FIG. 8). At this time, since the same result can be obtained by using Fs obtained by the H component processing unit 42 instead of F ₀ , a convenient one may be appropriately selected.

Next, in the eye candidate area, an axis obtained from the lip area obtained by the lip area extraction unit 45 toward the vertex, for example, an axis obtained as the main moment axis in the entire eye candidate area is determined. Is the symmetry axis of the face. A region having almost the same area across the axis of symmetry and having a small angle with the axis of symmetry is searched for, and this region is referred to as an eye candidate pair region (see FIG. 9). By finding the center of gravity or finding both end points of the area and then finding the middle point, a reference point candidate for the eye is obtained, and the reference point of the mouth is added to the candidate. In the example of FIG. 9, two eye candidates are obtained, and two sets of face reference point candidates are obtained.

The area evaluated at this time may be, for example, about 0.5 as an area ratio between the target areas, but may be smaller if the processing speed of the apparatus has a margin. The same applies to the evaluation of the angle formed with the axis of symmetry. For example, an integer multiple of the variance of the angle between the principal moment axes of the eye candidate regions can be considered.

If there is only one reference point candidate for the face, the test processing section 5 does nothing and sends it to the output section 6 as it is.
When a plurality of reference point candidates are obtained as shown in FIG. 9, the test processing unit 5 uses the face template created using the structural knowledge of the face and the reference point candidates obtained by the reference point candidate extraction processing unit 4. Compare pattern groups obtained by normalizing face images,
The reference point candidate to which the pattern with the smallest difference is given is set as a reference point to be obtained and sent to the output unit 6.

A specific example of the test processing will be described with reference to FIG. FIG. 10 is a block diagram showing a functional configuration of the test processing unit 5.
Denotes a pattern matching unit, and 53 denotes a template storing memory for matching.

The face image cutout section 51 sets the right eye point, left eye point, and mouth point of the n sets of reference point candidates sent from the reference point candidate extraction section 4 to Er, El, and M points, respectively, as shown in FIG.
As shown in (Example of Normalization of Face Image), the target face image 1 is normalized by the following procedure. (1) Let O be a foot of a perpendicular drawn from M to a straight line connecting Er and El. (2) Parallel / rotational movement is performed so that the line segment OM coincides with the vertical line of the screen, and a fixed subdivision point of the line OM is located at a fixed position. (3) The image is isotropically enlarged or reduced so that the length of the line segment OM becomes a constant value. For example, an area surrounded by a frame in FIG. 11 is cut out from the normalized face image as a pattern for comparison. This is performed for n sets of reference point candidates, and n
Get matching patterns.

The pattern matching unit 52 compares the n matching patterns obtained by the face image cutout unit 51 with the patterns in the matching template storage memory 53, and determines the reference point candidate that has given the least difference. It is sent to the output unit 6 as a reference point.

As a comparison method, for example, there is the following method. The matching pattern is Xi (i = 1, 2,..., N)
And the matching template is M,

[0035]

(Equation 3)

The error Ei is obtained by Emin = m
In this method, a reference point candidate giving in (E ₁ , E ₂ ,..., En) is set as a correct reference point.

The collation template accumulation memory 53 is storing a pattern referred to by the pattern collation unit 52.

As an example of creating the collation template, there is a method in which a large number of face images from which reference points have been correctly extracted are cut out as face patterns by the same method as the face image cutout unit 51, and are added and averaged.

As another method, a pattern matching unit 52
After the matching pattern Xi obtained in the above is developed by the unique face by KL expansion, an image X′i restored as a weighted sum of all unique faces using the expansion coefficient is used as the matching template M. (Reference [5] << Akamatsu, Sasaki,
Fukamachi and Suenaga, "Evaluation of Application of Pattern Description Method by KL Expansion to Face Image Discrimination," IEICE Technical Report, PRU90-152, (1991
-3) >>).

The output unit 6 receives and outputs the reference points obtained by the test processing unit 5.

As can be understood from the above description, according to the present embodiment, when performing personal identification using a human face image, extraction of a face reference point required for normalization of position and size is performed. Extracts color information required for processing from input images instead of using feature points based on edge information, which has been used in the past
And, based on the color information, after extracting the existence region of the eye reference point and the mouth reference point of the face, eye groups of the face from the area
A reference point candidate and a mouth reference point candidate are taken out, and based on the reference point candidates, it is tested whether or not they are correct reference points .
Specifically, first, YI is used as color information from an input image.
The I and Q components of the Q display system and the HVS display system
After extracting the H component, using the extracted I component,
A face area is extracted from the input image, and the extracted face area is extracted.
A face mask area is generated from the area. And the generated
Using the extracted face mask region and the extracted H component,
After extracting the lip candidate area of the face, the extracted lip candidate area
Extract lip region from the area using the extracted Q component
Then, the mouth reference point is extracted. In addition, the generated face cell
And the extracted face area or H component
After extracting the eye candidate area using the face, the extracted
Based on the lip region extracted from the eye candidate region, an eye reference point
Candidates are extracted, and the extracted eye reference point candidates are
It tests whether it is a reference point and extracts an eye reference point. Book
According to the embodiment, it is possible to stably normalize the position and size of a face image input from the front, and as a result, it is possible to stably perform personal identification based on the face image.

Although the present invention has been described in detail with reference to the embodiments, it is needless to say that the present invention is not limited to the above-described embodiments, and various changes can be made without departing from the scope of the invention. . For example, if head three-dimensional surface information and surface color information can be obtained, extension to three-dimensional is easy.

[0043]

As described above, according to the present invention, it is possible to stably normalize the position and size of a face image input from the front, and as a result, individual identification based on the face image Can be performed stably.

[Brief description of the drawings]

FIG. 1 is a block diagram showing a schematic functional configuration of an embodiment of the present invention;

FIG. 2 is a block diagram illustrating a functional configuration of a reference point candidate extraction processing unit according to the embodiment;

FIG. 3 is a diagram illustrating a processing example of an I component processing unit according to the embodiment;

FIG. 4 is a diagram illustrating a processing example of an H component processing unit according to the embodiment;

FIG. 5 is a diagram showing a processing example of the embodiment;

FIG. 6 is a diagram illustrating a lip candidate region according to the present embodiment;

FIG. 7 is a diagram showing a lip region according to the present embodiment;

FIG. 8 is a diagram showing eye region candidates according to the present embodiment;

FIG. 9 is a diagram showing an eye candidate pair area according to the present embodiment;

FIG. 10 is a block diagram illustrating a functional configuration of a test processing unit according to the present embodiment;

FIG. 11 is an explanatory diagram for describing a face reference point extraction method according to the present embodiment.

[Explanation of symbols]

DESCRIPTION OF SYMBOLS 1 ... Target face image, 2 ... Density conversion processing part, 3 ... Color system conversion processing part, 4 ... Reference point candidate extraction processing part, 5 ... Test processing part,
6 output unit 7 control unit 41 I component processing unit 42
H component processing unit, 43 Q component storage memory, 44 face mask creation unit, 45 lip region extraction unit, 46 eye region candidate extraction unit, 51 face image cutout unit, 52 pattern match unit,
53: collation template storage memory

Continuation of the front page (56) References JP-A-1-314385 (JP, A) JP-A-63-223974 (JP, A) Seiji Kobayashi, "Method for extracting feature points of lips from facial image", Electronic Information Communication Proceedings of the National Convention of the Society, Published in 1990, 1990, Spring, Pt. 7, p-7.26 (58) Field surveyed (Int. Cl. ⁷ , DB name) G06T 1/00 G06T 7/00 JICST file (JOIS)

Claims (1)

(57) [Claims] 1. YIQ display as color information from an input image
I and Q components of the system, and H component of the HVS display system
A process for extracting (1), using the I component extracted by said step (1), the input
Extracting a face area from the image (2), and extracting a face mask area from the face area extracted in the step (2).
Generating (3), the face mask area generated in step (3),
Using the H component extracted in (1), the lip candidate area of the face
A process of extracting (4), the lip candidate region extracted in the step (4), said process
Using the Q component extracted in (1), a lip region is extracted,
Extracting a mouth reference point (5), the face mask area generated in the step (3),
The face area extracted in (2) or in the above-mentioned process (1)
Extract eye candidate area of face using extracted H component
Step (6), and extracting the eye candidate area from the eye candidate area extracted in the step (6).
Extract eye reference point candidates based on the lip region extracted in (5)
Process and step (7), to test whether the eye reference point candidates extracted by the above process (7) is the correct reference point, it extracts the eye reference point
(8) A face reference point extracting method characterized by comprising: