JPH08287216A - In-face position recognizing method - Google Patents

Abstract

PURPOSE: To obtain a noncontact, markerless in-face position recognizing method and recognize feature positions of the face of an objective person without making the object person aware specially. CONSTITUTION: A human area is determined from a human candidate area obtained by analyzing an picked-up infrared image and a skin color area obtained by analyzing a color image which is picked up almost at the same time S7, a face area is extracted through horizontal and vertical projection operation for the human area S10, and template matching is performed for this face area to recognize positions in the face S11. With the 1st extracted face area, a template is registered and when respective positions in a newly extracted face area image are matched with the template, plural search areas which are different in size are set in the image, while the sizes of the minimum search area - maximum search area are increased stepwise, positions matching the template are recognized preferably in terms of the processing speed.

Description

Detailed Description of the Invention

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to an image processing technique for a person, and more particularly to a method for recognizing parts such as eyes and mouth in the face.

[0002]

2. Description of the Related Art In conventional image processing technology, when recognizing a person's face, a marker such as a tape is attached to the face in order to make a feature point in the face, and this marker is recognized as a feature point. In this way, each part of the face was recognized and the movement of the face was tracked.

In this case, there is a problem in that the subject feels awkward and uncomfortable to put the marker on, and that he or she is being measured.

[0004]

SUMMARY OF THE INVENTION The present invention has been made in view of the above problems of the prior art, and an object of the present invention is to obtain a non-contact and markerless in-face part recognition method.

It is another object of the present invention to provide a facial part recognition method capable of recognizing a characteristic part of the face without making the subject aware of the measurement and further shortening the time required for image processing. .

[0006]

According to the present invention, a person area is determined from a person candidate area obtained by analyzing an imaged infrared image and a skin color area obtained by analyzing a color image taken at approximately the same time, and the person area is determined. In this method, a face area is extracted by projection calculation in the horizontal and vertical directions, and template matching is performed on the face area to recognize a part in the face.

In this case, when a template is registered in the first extracted face area and the registered template is matched with each part in the newly captured face area image, a large size is included in the image. The processing speed is to set a plurality of search areas with different sizes and to recognize the part that matches the template while gradually increasing the size of the area from the minimum search area to the maximum search area. Is preferred.

Particularly, it is preferable that the search area is slightly larger than the registered template and the maximum is slightly larger than the face area.

[0009]

The image with a person in the background is captured by both the infrared image and the color image, and the person region is determined from the person candidate region by the analysis of the infrared image and the skin color region by the analysis of the color image.

Next, the face area is extracted by the horizontal and vertical projection calculation for the determined person area. Then, template matching is performed while stepwise enlarging a plurality of search areas having different sizes in the extracted face area to recognize a part in the face area image.

[0011]

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS An embodiment of the facial part recognition method of the present invention will be described in detail below with reference to the drawings.

FIG. 1 shows a flowchart of the main operation. In the figure, first, in step S1, an infrared image is captured by an infrared imaging device (not shown) such as an infrared camera, and in step S2, a color image is captured by a color imaging device (not shown) such as a visible camera. To do.

Next, in step S3, an environmental temperature threshold value for extracting a human candidate area from the infrared image (see FIG. 2A) is determined, and in step S4, the environmental temperature threshold value is used for imaging. A candidate region for a person is extracted from the infrared image.

One captured color image (FIG. 2B)
Extraction of the skin temperature region for determining the skin color region is performed in step S5. When extracting the skin temperature region, information on the temperature region from the infrared image is also referred to. Then, the skin color region in the color image is determined in step S6 from the obtained information on the skin temperature region.

In step S7, the person area (see FIG. 2C) in the image is determined from the person candidate area and the skin color area obtained by the sensor fusion technique using the infrared camera and the visible camera as described above.

Next, in order to extract the face area from the obtained person area, the oblique addition of the luminance values in the horizontal and vertical directions is performed on the person area image in steps S8 and S9.

FIG. 3 is a diagram showing the manner of the slant addition in steps S8 and S9, in which the distribution of the sum of the brightness values in the horizontal and vertical directions appears in a curved line. Therefore, the horizontal threshold value is about 10% of the peak value, the vertical threshold value is about several% of the peak value at the first rising part, and a large depression appears on the jaw. The depression is used as a threshold.

From the brightness characteristic curves in the horizontal and vertical directions thus obtained, processing with threshold values is performed, and the area between the threshold values in each direction is determined as a face area in step S10, and the face area in the human image is extracted. I do.

Then, in step S11, the characteristic part in the face is recognized by the gray-scale template matching. In this embodiment, an image processing board for template matching manufactured by NED was used, and a normal correlation method was used as a template matching method.

FIG. 4 shows a flowchart of the template matching process. First, when the initial setting is completed in step S21, the template that is the basis of template matching is registered in step S22.

At this time, in FIG. 5, the horizontal direction is x coordinate,
When the vertical direction is the y coordinate, (1) the template with the smallest x coordinate is the right eyebrow (x ₀ ,
y ₀ ) (2) Select the template with the second smallest x coordinate as mouth (x
₂ , y ₂ ) (3) The template with the largest x coordinate is set to the left eyebrow (x ₁ ,
Recognize as y ₁ ) and register.

FIG. 6 shows a template of each part (both eyebrows and mouth) registered for the image of the face area obtained in step S10. Using these registered templates, the facial part is recognized by matching the images captured thereafter.

The recognition processing method is as follows. In step S23, the center coordinates of the search area in the recognized image are set. Hereinafter, the method of performing the correlation calculation on all the pixels in the search area is called the main search, and the method of performing the correlation calculation on every few pixels instead of performing the correlation calculation on all the pixels in the search area is a rough search. I will decide.

Next, in step S24, a rough search for template matching is performed using the smallest search area A (see FIG. 7). Then, in step S25, it is determined whether or not there is a match, and if not, the process proceeds to step S26 and a coarse search is similarly performed using the search area B (see FIG. 7) that is one size larger.

Further, in step S27, it is determined whether or not there is a match, and if not, the process proceeds to step S28 and a coarse search is performed using a search area C (see FIG. 7) of a larger size. In this embodiment, three search areas A to C are prepared. Therefore, if it is determined in step S29 that no match is found, ERROR is displayed in step S30.

When it is determined that the match is found in any of the steps S25, S27 and S29, it is determined in step S31 whether the positional relationship between the face and the mouth is correct. . A method of determining the positional relationship between the eyebrows and the mouth will be described below. (4) First, the reference point for determining the positional relationship is the center of gravity of the right eyebrow (see FIG. 6). (5) Rough search for the entire search area (entire face area)
Done in. (6) A board (hardware) for image processing is set for each part in initial registration. At this time, since it is known which board is in charge of which board, the board in charge of the right eyebrow is used as a reference to make a combination according to the positional relationship of each part.

That is, an arbitrary combination is selected as the right eyebrow i-th candidate
(X_0i, Y_0i), The left eyebrow i-th candidate (x _1i, Y_1i), Mouth i
Th candidate (x_2i, Y_2i),

[0028]

[Equation 1]

The following limiting conditions are satisfied. (7) The main search is performed on the search areas of the combination to narrow down the candidates for the facial parts. (8) The positional relationship of the combinations remaining in the main search is determined again.

That is, in the X direction, a combination satisfying the limiting condition such as the expression 2 is selected for the positional relationship between both eyebrows.

[0031]

[Equation 2]

On the other hand, in the Y direction, a combination satisfying the limiting condition as shown in Expression 3 is selected for the positional relationship between the eyebrows and the mouth and the right eyebrows and the left eyebrows.

[0033]

(Equation 3)

If it is determined in step S31 that the positional relationship between the eyebrows and the mouth is correct, the process proceeds to step S32, the center coordinates of the search area are reset, and the processes from step S24 are repeated. If it is judged to be incorrect, the above step S
Return to 30 and display ERROR.

FIG. 8 is a diagram conceptually showing the operation of the flowchart of FIG. Further, as shown in FIG. 9, it is apparent that it is possible to deal with the change in the orientation of the face by performing the processing according to the above-mentioned flowchart and considering the positional relationship between the eyebrows and the mouth.

Further, as is apparent from FIG. 8, the coarse search of the recognition site is carried out while the search area is expanded. Therefore, if the recognition site is found, it is natural to perform the main search by reducing the search area.

[0037]

As described above, the present invention can provide a non-contact and markerless intra-face part recognition method.

Further, it is possible to provide a facial part recognition method capable of recognizing the characteristic part of the face without making the subject aware of the measurement and further shortening the time required for the image processing.

Moreover, it is possible to follow changes in the orientation of the face by taking into consideration the positional relationship between the parts.

[Brief description of drawings]

FIG. 1 is a main flowchart of a method for recognizing an in-face part of the present invention.

FIG. 2 is a conceptual diagram of sensor fusion technology.

FIG. 3 is a conceptual diagram of a method of extracting a face area by adding a shadow.

FIG. 4 is a flowchart of template matching processing.

FIG. 5 is a conceptual diagram of a template to be registered.

FIG. 6 is a conceptual diagram showing reference points in the face and initial information to be set.

FIG. 7 is a diagram showing a search area applied to template matching.

FIG. 8 is a conceptual diagram of template matching processing.

FIG. 9 is a conceptual diagram showing a change in the orientation of the face and the positional relationship between both eyebrows and the mouth.

 ─────────────────────────────────────────────────── ─── Continuation of the front page (72) Inventor Kazuo Matsumoto 2-5-5 Keihan Hondori, Moriguchi City, Osaka Sanyo Electric Co., Ltd.

Claims (3)

[Claims] 1. A person area is determined from a person candidate area obtained by analyzing an imaged infrared image and a skin color area obtained by analyzing a color image taken at substantially the same time, and a projection operation in horizontal and vertical directions is performed on the person area. A facial part recognition method for recognizing a facial part by extracting a facial region and performing template matching on the facial region. 2. A template is registered in the first extracted face area, and when the registered template and each part in the newly captured face area image are matched, a size is calculated in the image. A plurality of search areas different from each other are set, and the site matching the template is recognized while gradually increasing the size of the area from the minimum search area to the maximum search area. Item 1. A method for recognizing an in-face part according to Item 1. 3. The intra-face part recognition method according to claim 2, wherein the search area has a size slightly larger than the registered template and a size slightly larger than the face area.