JP2003281539A - Facial component searching device and facial component searching method - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To accurately search a facial component from an image portion of a face included in a color image. <P>SOLUTION: The facial component searching device 101 is a device for searching a facial component image from the color image. It is provided with an image acquiring part 10 acquiring the color image, a facial area extracting part 20 detecting a facial area corresponding to a facial color pattern that is a pattern relevant to colors of the face from the color image and extracting the image portion of the facial area, a reference point setting part 40 detecting gradation of the image portion from the extracted image portion of the facial area and setting a reference point indicating a reference position for searching the facial component image on the basis of the gradation, and a facial component detecting part 50 detecting the facial component image on the basis of the set reference point and determining a position of the facial component image. <P>COPYRIGHT: (C)2004,JPO

Description

Detailed Description of the Invention

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to an apparatus and a face part searching method for searching for face parts such as eyes and mouths from a face image portion included in a color image obtained by a camera or the like, and more particularly to a face part The present invention relates to an apparatus and method suitable for searching for a face part in a color image in which the illumination environment to the face or the direction and inclination of the face are not uniform.

[0002]

2. Description of the Related Art When a person reflected in an image captured by a camera or the like is automatically identified based on, for example, the face of the person, an image portion corresponding to the face and face parts is searched for in the image. To be done. A conventional technique regarding such a face search is disclosed in, for example, the Institute of Electronics, Information and Communication Engineers, D-II Vol. J-77-DII No. 4p, published in April 1994.
p.672-681, "Finding and Positioning Face in Scene Using Multiple Pyramids for Individual Identification".

In the disclosed technique, a series of mosaic images for performing pattern matching on an unknown image for which a face part is to be searched for is prepared in advance. The series of mosaic images show a typical gray pattern for the head region including the face, at successive multi-resolutions. By sequentially matching each of the series of mosaic images with the unknown image, the head region is gradually narrowed down from the unknown image. Then, the position of each face part is calculated based on the histogram of the face area obtained from the head area.

[0004]

By the way, for example, when the lighting environment for a face changes between frame images of a moving image, the state of light and shade appearing in the image portion of the face also changes according to the change. However, since the mosaic image of the disclosed technique is created based on a general grayscale pattern acquired in advance, when the grayscale change occurs between the frame images to be matched,
The mosaic image cannot cope with the change, and it is difficult to properly narrow down the head region.

Further, in the above disclosed technique, each event is dealt with in order to properly search for face parts even when the face to be searched has a tilt or when the face is facing other than the front. It is conceivable to create the mosaic image. However, for that purpose, a huge number of mosaic images are required, and it is expected that the processing time for the matching will remarkably increase. Therefore, it is not easy to deal with the inclination and orientation of the face.

An object of the present invention is to provide a face part searching apparatus capable of appropriately searching for a face part by coping with changes in the lighting environment to the face, the direction and inclination of the face, etc. when searching for a face part from a color image. It is to provide a facial part search method.

[0007]

<Structure 1> A facial part searching device according to the present invention is a device for searching for a facial part image from a color image containing a facial part image representing a facial part. An image acquisition unit that acquires an image, and from the acquired color image, detects an image area corresponding to a face color pattern, which is a pattern related to the color of the face, as a face area, and extracts the image area of the face area. Unit and a reference point setting unit that detects a shade of the image portion from the extracted image portion of the face area and sets a reference point indicating a reference position for searching the face part image based on the shade. And a face part detection unit that detects the face part image based on the set reference points and obtains the position of the face part image.

In the facial part searching apparatus according to the present invention, the facial area extracting section extracts a facial area from the color image acquired by the image acquiring section using the facial color pattern described above. The reference point setting unit sets the reference point of the face component image on the basis of the light and shade shown in the image portion of the face region in order to detect a rough position of the face component image in the color image. The face part detection unit detects a face part image based on the reference point and obtains its position.

In the face area extraction unit, a face color pattern generation unit that generates the face color pattern based on the color image acquired by the image acquisition unit, and a storage unit that holds the generated face color pattern. And an extraction unit that performs the extraction of the face area based on the face color pattern of the storage unit.

The face color pattern generation unit can generate a color pattern of the color image in a predetermined color space, and corresponds to a preset color pattern model in the generated color pattern. A portion can be adopted as the facial color pattern.

The image acquisition unit can sequentially acquire a series of color images, and the color pattern generation unit generates the face color pattern based on a predetermined color image of the series of color images. can do. The extraction unit can determine the success or failure of the extraction of the face area based on a preset determination criterion, the face color pattern generation unit, when the determination that the extraction was not successful, It is possible to set a new face color pattern for another color image subsequent to the color image.

The reference point setting unit can be set so as to detect peak values in two directions which are orthogonal to each other in the shade of the face area, and the shade corresponding to each peak value in the two directions. It is desirable to set the above portion as the reference point.

The reference point setting section detects the shading from the face area by using a low frequency filter corresponding to a frequency band preset based on the position of the facial part for the shading of the facial part. You can

The face part detecting section includes face part information which is information relating to a preset image of the face part and feature point information indicating a predetermined position in the face part corresponding to the face part information. Face part data can be given, and the face part detecting section is set with a scanning range setting section for setting a scanning range of the face part data based on the reference points set by the reference point setting section. The feature point information when the face part data is scanned within the scanning range and a predetermined criterion regarding the consistency between the face part information of the face part data and the image portion corresponding to the reference point is satisfied is shown. A scanning unit that detects the position, and a position determination unit that selects a position that is close to a predetermined position from among the detected positions, and that determines the selected position as the position of the facial part image in the face area. To It can be obtained.

Further, the face part detecting unit further detects other face part images excluding the predetermined face part image based on the position determined by the position determining unit for the predetermined face part image in the color image. It is possible to provide an associated scanning range setting unit that sets the scanning range for setting the scanning range and supplies the set scanning range to the scanning unit.

The image acquisition unit can sequentially acquire a series of color images, and the face component detection unit sets the position of the face component image in the color image by the reference point setting unit for the color image. It is possible to detect based on the position of the face component image detected by the face component detecting unit with respect to the reference point that has been generated or the color image immediately before the color image. At this time, the facial part search device may include a tracking determination section that determines whether or not the facial part detection section should perform the detection based on the reference point of the color image.

The color image including the face part image may be an image including a plurality of face part images including a pair of face part images corresponding to both eyes. The inclination angle of the face area with respect to the horizontal direction is obtained based on the two reference points corresponding to the pair of face part images of the eyes set by the reference point setting unit, and the face angle is calculated by the inclination angle. A rotation processing unit that rotates the face region to correct the inclination of the region in the horizontal direction, and supplies the reference point of each face component image in the rotated face region to the face component detection unit. it can.

The image acquisition unit can sequentially acquire a series of color images, and a plurality of face part images including a pair of face part images corresponding to both eyes are added to the color image including the face part image. Can be included. At this time, in the face part search device, the two reference points corresponding to the pair of face part images of the eyes set by the reference point setting unit, or the color image immediately before the color image, the face Based on the positions of the face part images of both eyes detected by the part detection unit, the tilt angle of the face area of the color image with respect to the horizontal direction is obtained, and the tilt of the face area is set in the horizontal direction by the tilt angle. A rotation processing unit that rotates the face region to correct and supplies the reference point of each face component image in the rotated face region to the face component detection unit, and the rotation processing unit determines the tilt angle. A tracking determination unit that determines whether or not the color image should be obtained based on the reference point of the color image.

The color image including the face part image can be an image including a plurality of face part images including a pair of face part images corresponding to both eyes. Two distances between each of the pair of face part images of both eyes and each end part that defines the face width of the face area and is the closest end part of each of the pair of face part images. And a face orientation detection unit that detects the orientation of the face of the color image based on the difference between the two distances. <Structure 2> A facial part search method according to the present invention is a method for searching for a facial part image from a color image including a facial part image representing a facial part, and an image acquisition step of acquiring a color image, From the acquired color image, an image area corresponding to a face color pattern, which is a pattern relating to the color of the face, is detected as a face area, and a face area extraction step of extracting an image portion of the face area, and the extracted face area A reference point setting step of detecting a shade given to the image portion from the image portion, and setting a reference point indicating a reference position for searching the face part image based on the shade. A face part detecting step of detecting the face part image based on the reference point and obtaining the position of the face part image.

[0020]

BEST MODE FOR CARRYING OUT THE INVENTION Embodiments of the present invention will be described below with reference to specific examples. <Specific Example 1> FIG. 1 is a specific example 1 for explaining the present invention.
3 is a block diagram showing the configuration of the face part search device 101 of FIG. As shown in FIG. 1, the face part search apparatus 101 includes an image acquisition unit 10 such as a camera that acquires a color image, and a face area that extracts a face area described later from the color image obtained by the image acquisition unit. An extraction unit 20, a reference point setting unit 40 that sets a reference point that is a reference position for searching a face part such as an eye or a mouth in the face area, and a face in the face area based on the reference point. And a face part extraction unit 50 for obtaining a detailed position of the part. The reference point indicates a rough position of the face part in the face area.

The color image handled in this example is each frame image of a moving image, and the image acquisition unit 10 sequentially supplies the series of frame images to the face area extraction unit 20. The face area extraction unit 20 searches the given frame image for a face area indicating an area in which a face is represented. For the search of the face area, a face color pattern to be described later regarding a skin color pattern that is the color of the face on the image is used, and the face area extracting unit 20 extracts an image portion indicating a pattern corresponding to the face color pattern. It is extracted as a face area. The above-described face color pattern handled in this example is represented by a two-dimensional table in a predetermined color space.

FIG. 2 is a block diagram showing the configuration of the above-mentioned face area extracting section 20 of this specific example. Face area extraction unit 20
Is a face color pattern generation unit 21 that generates a face color pattern based on a predetermined frame image, and a face color pattern storage unit 2 that stores the face color pattern defined by the definition unit.
2, a face area detection unit 23 that detects a face area from the frame image based on the face color pattern of the storage unit, and a face area detection unit that controls the operations of the face color pattern generation unit 21 and the face area detection unit 23. And a control unit 24.

The face color pattern generation section 21 is a color space conversion section 25 for converting RGB values indicated by a given frame image into values in a predetermined color space, and a histogram for creating a histogram based on the values in the color space. A creation unit 26, a maximum value detection unit 27 that finds a maximum value of the histogram, and a skin color model 21a described later that is a model of a face color pattern preset for the above-described skin color pattern.
(See FIG. 4), and a facial color pattern setting unit 28 for extracting a temporary facial color pattern from the histogram.

Further, the face color pattern generation unit 21 discriminates the color of the image so as to detect a face area candidate from the frame image based on the temporary face color pattern.
A skin-color determining unit 29, a first face-likeness determining unit 30 that obtains a face-likeness matching degree, which will be described later, for the candidate from the features of the face area candidates represented by the colors determined by the determining unit,
Based on the obtained face-likeness matching degree, an optimal face area candidate is selected as a face, and a temporary facial color pattern including the optimal face area candidate is adopted as a facial color pattern for the facial area. It has an optimum face color pattern generation unit 31 which is supplied to the face color pattern storage unit 22.

In the face color pattern storage unit 22, in addition to the storage of the face color pattern from the optimum face color pattern generation unit 31 described above, when the face color pattern generation unit 21 cannot properly create the face color pattern. For example, a face color pattern as a default value is stored in advance.

The face area detecting section 23, as shown in FIG.
A second skin color determination unit 32 that determines the color of the image based on the face color pattern stored in the face color pattern storage unit 22 in order to detect a face region candidate from the subsequent frame image; The second face-likeness determination unit 33 that obtains the same face-likeness matching degree as described above for the candidates of No. 1 and selects the optimum face area candidate as a face based on the face-likeness matching degree, and the selected optimum face. A face area cutout unit 34 that removes an area showing an image portion other than a face image such as a background image captured by erroneous detection from the area candidates, and detects the area after the removal as a face area of the color image. Have.

The operation of the face area extraction unit 20 will be described.
In the example shown below, a facial color pattern is generated based on the first frame image from the image acquisition unit 10. When the frame image is supplied from the image acquisition unit 10, the face color pattern generation unit 21 causes the color space conversion unit 25 to perform the above-described color space conversion. As this color space, the (a, b) color space, the HSV color space, the YIQ color space, or the like can be used, but in this specific example, the (a, b) color space is adopted. The color space conversion unit 25 obtains the value of the (a, b) color space from the RGB value of the frame image according to the equation (1) shown in FIG.

When the color space conversion unit 25 obtains the value of the color space, the face area detection control unit 24 changes the obtained value to
The color space conversion unit 25 is controlled to supply the histogram creation unit 26.

The histogram creating section 26 creates a histogram from the color space values from the color space converting section 25 according to a general method, and supplies the histogram to the maximum value detecting section 27.

The maximum value detecting section 27 finds the maximum value of the color space coordinates in the histogram. This maximum value can be obtained using, for example, the skin color model 21a described above. As described above, the skin color model 21a is a default model of the facial color pattern, and each coordinate value in the (a, b) color space is represented by a binary value of "0" and "1". The skin color model 21a can be configured, for example, as shown in FIG. In the illustrated example, an a-coordinate axis having a scale of −2 to +2 is provided on the horizontal axis, and a b-coordinate axis having a scale of −2 to +2 is provided on the vertical axis, and the coordinates (0, 0, 0) is the point of interest. “1” of each coordinate value indicates a color space coordinate value to be detected, that is, a coordinate value corresponding to the face area to be detected, and “0” indicates a coordinate value outside the detection target. Therefore, the flesh color model 21a shown in the drawing defines the pattern formed by the coordinate group having the coordinate value of "1" as the color pattern of the face area in the (a, b) color space. The skin color model 21a
The setting of the coordinate value in is not limited to the example shown in the figure, and can be set appropriately.

The maximum value detecting unit 27 defines the skin color model 21a shown in FIG. 2 as, for example, smodel (5, 5), and in the histogram, the maximum value (x, y) that satisfies the expression (2) in FIG. Ask for. At this time, it is possible to preliminarily estimate a range in which there is a high possibility of showing a skin color pattern in the histogram and obtain the maximum value within the range.

A plurality of maximum values can be detected as long as the maximum value satisfies the expression (2).
Supplies each detected value to the facial color pattern setting unit 28 as a maximum value candidate.

The facial color pattern setting section 28 assigns the maximum color candidate in the histogram to the skin color model 21a.
So that the points of interest (0, 0) of the skin color model match, the skin color model is sequentially applied, and the histogram portion corresponding to the pattern portion of “1” of the skin color model is determined as the first skin color determination as the temporary facial color pattern. Output to the unit 29. FIG. 5 is a histogram in which one of the maximum values is (55, 28), and shows an example in which this skin color model 21a is fitted so that the target point of the skin color model 21a matches the maximum value.
At this time, the coordinate group showing the coordinate value of "1" in the figure is adopted for the temporary facial color pattern.

The first skin color judging section 29 extracts an image portion corresponding to the temporary face color pattern from the facial color pattern setting section 28 in the frame image to be searched. Figure 6
And two types of images (A) and (B) corresponding to the two types of temporary facial color patterns extracted from the histogram.
Indicates. From each of these images, the image portion (a1 to b) represented based on the corresponding temporary facial color pattern
2) is detected.

The first skin color judging section 29 performs a predetermined labeling process on the image portion of each of the images (A) and (B) corresponding to each temporary facial color pattern. As a result, as shown in FIG. 6, each image portion is identified by the label areas a1 and b2. The first skin color determination unit 29 supplies the respective images (A) and (B) that have been subjected to the above-described labeling processing to the first facialness determination unit 30.

The first face-likeness determining section 30 finds the values of the characteristics described below for the label areas a1 to b2 of the images (A) and (B), and uses the obtained values. Find the degree of facial matching. In this specific example, as the characteristics, the degree of unevenness SF2, which is the reciprocal of the circularity of each label area, the position of the center of gravity GD, and the symmetry SYM are used.

First, in order to obtain the degree of unevenness SF2, the number of pixels in each of the label areas a1 to b2 is counted, and the area AREA is obtained for each area. In addition, after performing a trace process on the contour of each label area, the perimeter PERI of the area is obtained. Further, the degree of unevenness SF2 is calculated according to the equation (3) shown in FIG. 3 by using these obtained values.

The center-of-gravity position GD is obtained by the equation (5) in the figure using the centroid coordinates (gx, gy) obtained from the equation (4) in FIG. The symmetry SYM can be obtained by the equation (6) in FIG. 3 when the RGB value “R” is used.

Using each value obtained in accordance with the above-mentioned procedure, the face-likeness matching degree LF of the j-th label area in the i-th tentative facial color pattern is calculated by the equation (7) in FIG.
Find (i, j). As for the face-likeness coincidence degree, the smaller the value is, the more the label area seems to be a face, and conversely,
When the label area does not look like a face, the degree of matching of the face looks like a large value. The first face likeness determination unit 30 displays the image (A).
And the face-likeness matching degree of each label area in (B) is supplied to the optimum face color pattern generation unit 31.

The optimum face color pattern generation unit 31 detects the minimum value of the face-likeness matching degree in each of the images (A) and (B) in order to determine the label area having the most facial-like features. To do. Further, among the face-likeness coincidences detected from each image, the minimum face-likeness coincidence is obtained. For example, when the label area a1 of the image (A) and the label area b1 of the image (B) show the minimum value of the face-likeness matching degree in each of the images, the smaller value of the label areas a1 or b1 is used. One label area indicating the degree of face-like matching is detected.

The optimum face color pattern generation unit 31 extracts a temporary face color pattern corresponding to the image having the detected one label area, and the temporary face color pattern is
It is supplied to the face color pattern storage unit 22 to be adopted as an optimum face color pattern.

The face color pattern storage unit 22 stores the face color pattern from the optimum face color pattern generation unit 31. This completes a series of processes for defining a face color pattern for detecting a face area from a subsequent frame image.

Subsequently, when a new frame image is supplied from the image acquisition unit 10 to the face color pattern generation unit 21, the color space conversion unit 25 performs the same (a, b) as described above for the frame image. Color space conversion is performed. The face area detection control unit 24 instructs the color space conversion unit 25 to supply the converted frame image to the second skin color determination unit 32 of the face area detection unit 23.

The second skin color judging section 32 uses the facial color pattern stored in the facial color pattern storage section 22 from the coordinate values of the color space from the color space converting section 25 and the relevant frame image, An image corresponding to the face color pattern is detected. This image refers to an image such as the image (A) or (B) shown in FIG. 6, for example. Second
Upon detecting the image, the skin color determination unit 32 performs the same labeling processing as described above on each image portion of the image, and then supplies the image to the second face-likeness determination unit 33.

The second face-likeness determining section 33 performs the same procedure as that in the first face-likeness determining section 30 for each label area represented by the image from the second skin color determining section 32. Features such as the degree of unevenness SF2 are obtained, and the face-likeness matching degree FLF (j) of the j label area is calculated by the equation (8) in FIG. 3 using the obtained feature values.

Furthermore, the calculated face-likeness agreement degree FLF
The face-likeness coincidence degree min that is the minimum value in (j)
(FLF (j)) is compared with a preset threshold FACE_THL1 regarding the face-likeness matching degree. In this comparison, the label area satisfying the expression (9) in FIG. 3 is determined to be an area having facial features, and the image having the label area is supplied to the face area cutout unit 34.

The absence of the label area satisfying the expression (9) indicates that it is determined that the frame image does not include a portion representing a face, and predetermined information indicating this is detected, for example, face area detection. From the unit 23 to the face area detection control unit 24
Can be output to.

The face area cutout unit 34 obtains the X-axis and Y-axis projections in the label area of the image from the second face-likeness determining unit 33. The X axis represents an axis extending in the horizontal direction of the image, and the vertical axis thereof is the Y axis. FIG. 7 shows an example of the image and the projection data obtained along the Y-axis direction thereof. In the image portion showing the face, in general, skin color does not easily appear on the eyes and mouth. Therefore, the portion showing a relatively small projection amount shown near the center of the projection data in FIG. It is considered to correspond to the part.

The face area slicing unit 34 obtains the average value m, the standard deviation σ and the maximum projection amount maxv from the projection data, and the threshold value thl in the projection data is calculated based on the equation (10) in FIG. To set. Using each of these values, X that defines the left edge of the face area (left toward the image)
L is obtained by the equation (11) in FIG. 3, and XR that defines the right end of the face area (right toward the image) is obtained by the equation (12) in the figure.

According to the above equation (11), in the projection data of FIG. 7, the portion that substantially matches LX1 is the left end XL.
On the other hand, when searching for the right end XR, p is calculated in the search range (m + k · σ to LX2) based on the equation (12).
A portion in which (x) falls below the threshold thl is adopted, but p (x) in the search range is a curve that once falls and then rises above the threshold as shown in FIG. Indicates. Originally, p (x) near the edge of the face region should converge at an almost zero value after falling below the threshold value thl, but as in the example shown in the figure, it does not rise once after falling. It is considered that the label area of this raised portion includes other image portions other than the face image, such as a part of the background.

From this, as the right end XR, as shown in FIG. 7, a portion in which p (x) is equal to or less than the threshold value thl before showing the above-mentioned bulge is adopted. As a result, most of the part of the background included in the label area can be excluded.

Further, from the projection data obtained along the other X-axis direction, the upper and lower ends of the face area are set to the above-mentioned Y.
It can be detected almost in the same way as at each axial end. The face area cutout unit 34 extracts the image portion of the face area defined by each end detected according to the procedure described above, and supplies this to the reference point setting unit 40.

In the above description of the operation relating to the face area extracting unit 20, the face area is detected from the subsequent frame image using the face color pattern defined based on the first frame image. The following procedure, which is described along the flowcharts of FIGS. 8 to 10, is adopted in consideration of the case where the operation at the time of rising is unstable, the case where the face is not displayed in the first frame image, and the like. You can

In the example shown in the flowchart of FIG. 8, the face color pattern is defined by each of the 1st to nth frame images in the moving image. First, when the frame image is supplied from the image acquisition unit 10 (step S1), the optimum facial color pattern generation unit 31 determines whether or not the optimum facial color pattern has already been defined (step S2). If the face color pattern is already defined, the face area detection unit 23 detects the face area using the table in the same manner as described above (step S3). When the face color pattern is not yet defined, the face color pattern generation unit 21 determines which frame image in the moving image is the frame image (step S4), and the frame image is 1
If any of the nth to nth images, an optimum facial color pattern is defined from the frame image (step S
5).

If the face color pattern is undefined when the frame image is received (step S2, No), and the frame image is the (n + 1) th and subsequent images (step S4, No), 1 to This indicates that the face color pattern could not be defined in the n-th frame image. In this case, the default face color pattern previously stored in the face color pattern storage unit 22 is adopted (step S6).

In the example shown in the flowchart of FIG.
Of the face color patterns defined in the nth frame image, the face color pattern with the highest definition frequency is n + 1.
It is used for the second and subsequent frame images. After the frame image is input (step S11), the face color pattern generation unit 21 determines the order of the frame image (step S12), and if the frame image is the 1st to nth images, the face color pattern is generated. Is defined (step S13). Further, the number of times the face color pattern is defined is counted (step S14). For frame images exceeding the n-th frame image, the face color pattern with the highest definition frequency is adopted from the face color patterns defined in each image before that (step S15).
The face area is detected in the same manner as described above (step S1).
6).

In the example shown in the flowchart of FIG. 10, steps S21 to S25 operate in the same manner as steps S1 to S5 in the example shown in FIG. However, in step S23, the face area detection unit 23 outputs to the face area detection control unit 24 the degree of coincidence of face-likeness with respect to the face area and whether or not the face area is cut out.

The face area detection control unit 24 includes the face area detection unit 2
When the content of the output from 3 satisfies the condition of any of the expressions (13) in FIG. 3, the illumination environment of the frame image has changed from the previous frame image, the face orientation has changed, or the like. It is determined that the environment has changed between the frame images, and the face color pattern generation unit 21 is instructed to define a new face color pattern (step S26). The face color pattern generation unit 21 generates a new face color pattern from the frame image according to the instruction from the face area detection control unit 24. As a result, it is possible to automatically define the facial color pattern corresponding to the change in the environment displayed in the image.

Using the examples according to the above-mentioned flow charts, the histogram creating section 26 creates and creates a cumulative histogram for a plurality of frame images 1 to n or a histogram showing its average. It can be set so as to generate an optimum facial color pattern based on the created histogram.

FIG. 11 is a flow chart showing the operation procedure of the reference point setting unit 40 of the face part searching apparatus 101. The reference point setting unit 40 performs a low-frequency filter process, which will be described later, on the image portion of the face area detected by the face area extracting unit 20 described above, and from the response image 41 obtained by the processing, a rough image of each face part is obtained. The above-mentioned reference point that is at a different position is detected.

The reference point setting unit 40 executes the low-frequency filtering process in step S31 shown in FIG. 11 to select each face part at a preset low frequency in the shade pattern of the color indicated by the image portion of the face area. The part where the shade appears at the position along the positional relationship is extracted. For example, in the response image 41, "dark"
A portion in which a pixel indicating “?” And a pixel indicating “light” frequently appear alternately is not extracted in the filter processing,
A portion in which any one of the pixels continuously appears is extracted at a position along the positional relationship between the face parts described above.
FIG. 12 shows the response image 4 obtained by the low frequency filtering process.
1 is shown.

The reference point setting section 40 executes the step S31.
After acquiring the response image 41 of the low frequency filtering by
In step S32, the center line 42 that divides the response image into right and left, as shown in FIG. 13, is detected.

In order to detect the above-mentioned center line 42, the reference point setting section 40 defines an arbitrary line extending in the Y-axis direction of the response image 41 as a center-line candidate 42 ', and sequentially sets this in the X-axis direction. The parallel movement is performed, and the gray level similarity of a pair of pixels located symmetrically with respect to the center line candidate is obtained for each movement. In the calculation of the degree of similarity, as shown in FIG. 14, the pixel (x, y) on the centerline candidate 42 'is a pair of pixels located symmetrically with respect to the centerline candidate.
For example, the gray level similarity of (x-1, y) and (x + 1, y) is obtained. The pair of pixels can be extracted from the entire area of the response image 41 or an arbitrary area ((xs, ys) to (xe, ye)) as shown in FIG. As a result, the centerline candidate 42 'in which the pair of pixels having a high degree of similarity is most present is set as the centerline 42 in the response image. When the face of the frame image has a tilt, it is desirable to set an angle corresponding to the tilt angle as the center line candidate 42 '.

After the center line 42 is detected in step S32, the reference point setting unit 40 takes projections in the X-axis direction and the Y-axis direction in step S33, and detects a peak portion from the projection data. When the projection in the X-axis direction is taken from the response image, the projection data in the X-axis direction as shown in FIG. 13 is obtained. In the illustrated example, two peak parts appear in the projection data in the X-axis direction. In step S33, after the projection in the X-axis direction is acquired, Y is generated in the response image areas 41a and 41b corresponding to the respective peak portions.
Get the axial projection. As a result, one projection data shown above the response image 41 in FIG. 13 and the other projection data shown below the image are obtained.
In the one projection data corresponding to the response image area 41a, two peak portions symmetrical with respect to the center line 42 appear, and in the other projection data corresponding to the response image area portion 41b, the center is included in the other projection data. A single peak portion appears near the line 42.

The reference point setting section 40 uses the step S33.
In step S34, the rough position of the eyes in the response image 41 is detected based on the peak portion appearing in each of the X-axis direction projection data and the Y-axis direction projection data detected in step S34. In detecting the eye position, a peak width exceeding a predetermined threshold is detected in the peak portion in the X-axis direction, and the peak portion in the Y-axis direction is selected from the response image portion corresponding to the peak width. Of the peak portions thus obtained in the Y-axis direction, two peak portions that are symmetrical to each other with respect to the center line 42 are selected, and two peak portions corresponding to the two peak portions are selected.
One response image portion is determined as a rough position of the eye, that is, the reference point of the eye. In the example shown in FIG. 13, the response image area 4
The two response image portions corresponding to the eye reference points appear in 1a.

Further, the reference point setting section 40 determines in step S3.
At 4, the rough position of the mouth in the response image 41 is detected. In detecting the position of the mouth, the step S33 is performed.
A single peak in the projection data portion including the position of the center line 42 among the peak portion in the Y-axis direction selected in step 1, that is, the peak portion in the Y-axis direction corresponding to the peak width exceeding the predetermined threshold in the X-axis direction. A portion is selected, and the response image portion indicated by the single peak portion is determined as the rough position of the mouth, that is, the reference point of the mouth. In the example shown in FIG. 13, the reference point of the mouth appears in the response image area 41b.

The reference point setting unit 40 follows the procedure described above.
The eye and mouth reference points in the response image 41 are set, and the coordinates corresponding to the respective reference points are supplied to the face part extraction unit 50. Although the reference points of the eyes and mouth are detected in the above description, the reference points of other face parts such as the nose and eyebrows can be detected using the procedure described above.

FIG. 15 is a block diagram showing the configuration of the face part extraction section 50 of the face part search device 101. As shown in FIG. 15, the facial part extraction unit 50, based on the coordinates of the reference point from the reference point setting unit 40, sets a search area setting unit 51 that sets a search range of facial feature points, which will be described later, and the search. Within the range, a candidate extracting unit 52 for extracting the candidates of the facial feature points by using a template having information on face parts set in advance, and a candidate selection for selecting an optimal facial feature point from the candidates of the facial feature points. And a section 53.

The above-mentioned facial feature points are positions set in advance for each face part. For example, the facial feature points corresponding to "eyebrows", "eyes", "nose" and "mouths" are referred to as "eyebrows". "Bottom center", "iris center", "nose tip" and "upper, lower, left and right edges"
Can be

The face part extraction section 50 is previously provided with data regarding a standard face image of an arbitrary size. The template of this specific example is data including the face part image of this standard face image and the relative positions of the above-mentioned feature points in the face part image. The candidate extraction unit 52 is provided with a template storage unit 52a that stores the template for each corresponding face part.

The search area setting section 51, as described above,
The range for searching the facial feature points of each facial part from the frame image is set. The search range can be a range having an arbitrary distance in the vertical and horizontal directions from the position coordinates of each face part supplied from the reference point setting unit 40. However, the position coordinates are rough for each face part. Since the position is indicated, it is desirable to set the search range in consideration of an error from the actual position.

By setting the search range, the search area setting unit 5
When 1 sets the search range of the "iris center" which is the feature point based on the rough position coordinates of the "eyes", for example, the position coordinates of the "eyes", that is, the pair of position coordinates corresponding to both eyes. Then, the inter-pupil distance, which is the distance between the two position coordinates, is obtained, and the ratio between the obtained inter-pupil distance and the inter-pupil distance in the standard face image included in the template is obtained. Further, the range defined by the coordinates at a predetermined distance from the position coordinates of the "eyes" in the standard face image is transformed by the above-mentioned ratio, and the rough "eyes" are placed within the transformed range. The range is fitted so as to include the position coordinates of ". The range thus obtained is set as the search range of the "center of the iris" which is the feature point of the "eye".

In accordance with the search range setting procedure described above,
Similarly, for other face parts such as "eyebrows" and "mouths", each search range can be set.

The candidate extracting unit 52 stores the template corresponding to the facial parts of the feature points to be extracted in the template storing unit 52.
The template is scanned within the search range acquired from a and set by the search area setting unit 51 to detect candidates for position coordinates of each facial feature point. For example, when detecting the position coordinate candidate of "center of iris", the face part image included in the template of "eyes" is deformed by the ratio calculated by the search area setting unit 51, and in accordance with this deformation, The relative position coordinates of the face part image are converted. Subsequently, the template is scanned within the search range of the frame image, and for each scanning position, each pixel of the face part image of the template and the area in the frame image to be matched with the template are scanned. The luminance difference from each pixel is sequentially obtained.

Further, the total sum is obtained from the respective brightness differences obtained for each scanning, and the scanning position where the value of the total becomes smaller than a preset threshold value is detected. When the sum total value of the respective brightness differences is the minimum value “0”, it means that the brightness of the both pixels completely matches, and the higher the total value, the lower the degree of coincidence between the two. When the threshold value set in advance for this total sum value is set to, for example, “10”, when the total sum value indicates any of “0” to “9” smaller than this threshold value, the scanning position is detected. To be done.

The candidate extracting section 52 uses the position coordinates of the frame image corresponding to the relative position coordinates of the template at each detected scanning position as the position coordinate candidates of the "iris center" of the frame image.

Further, the candidate extracting section 52 divides the total value (“0” to “9”) smaller than the threshold into a plurality of stages, and the divided stages are matched with each other between the template and the frame image. Can be expressed as a certainty factor. For example, the level of the certainty factor is “A” in descending order of certainty factor.
~ If set to five levels of "E", in the above example,
The sum total value of "0" and "1" corresponds to "A" which is the maximum level of the certainty factor, and the total sum value "8" and "9" corresponds to "E" which is the minimum level of the certainty factor. .

When the position coordinate candidates of the facial feature points for each face part are extracted by the candidate extracting section 52, subsequently, the candidate selecting section 53 selects the optimum candidate for the facial feature points from these candidates. It For example, when selecting an optimal candidate from the candidates regarding the “iris center”, the candidate selection unit 53 obtains average position coordinates from the position coordinates indicated by these candidates, and is at the closest distance to the obtained average position coordinates. Candidates
Select it as the optimum position coordinate of the "iris center". The optimum position coordinate of the "iris center" selected in this way is
It is the detailed position coordinates of the “eye” in the frame image.

Using the procedure described with reference to FIG. 15, the face part extraction section 50 can be constructed as shown in FIGS. 16 and 17. In the example shown in FIG. 16, the face part extraction section 50 is further provided with a reference point input search area setting section 54, which will be described later, and based on the position coordinates of any face part supplied from the reference point setting section 40. Then, the face feature points of other face parts including the face part are searched.

In the face part extraction section 50 of FIG. 16, for example,
When the above-mentioned arbitrary face parts are “eyes”, the search area setting unit 51 sets the search range “center of the iris” in the same manner as described above, and then the candidate extraction unit 52 extracts the template storage unit 52a from the template storage unit 52a. The acquired template of “eyes” is scanned within the search range to detect facial feature point candidates, and the candidate selection unit 53 selects the optimum candidate from the candidates.

The reference point input search area setting unit 54 sets an area for searching the face feature points of the remaining face parts based on the position coordinates of the "iris center" detected by the candidate selection unit 53. In the example described with reference to FIG. 15, the rough position coordinates from the reference point setting unit 40 are used, but here, the position coordinates already determined by the candidate selection unit 53 (“iris center”) are used. It is considered that this coordinate has little error from the actual position coordinate. Therefore, the search range set by the reference point input search area setting unit 54 based on the determined position coordinates can be set smaller than the search range set by the search area setting unit 51.

For example, when searching for the "center under the eyebrow" from the fixed position coordinates of the "iris center", the interpupillary distance is obtained from the fixed position coordinates, and as described above, the "eye" template is used. The ratio is calculated. Then, in the template, the search range of the "center under the eyebrows" located at a predetermined distance from the position coordinate of the "center of the iris",
Transform and move. Further, the “iris center” of the template obtained as a result is collated with the “iris center” determined in the frame image, and based on the search range of the “browside center” of the template, the frame image of the frame image is searched. Set the search range of "center under the eyebrows". With respect to the other face parts, the search range can be set in the same manner as the procedure for the “center under the eyebrows”. As a result, the facial feature points can be scanned within a relatively small search range.

Further, in the example shown in FIG. 17, the face part extraction section 50 is provided with a continuous processing search area setting section 55, and the setting section 55 and the search area setting section 51 are operated by a predetermined selection mechanism 56. , Selectively operated. The continuous processing search area setting unit 55 sets the search range of the facial feature points of the frame image based on the fixed position coordinates of the facial feature points extracted in the frame image immediately before the frame image to be processed. This search range can be set to a relatively small range, as in the reference point input search area setting unit 54 described above. As a result, the respective search ranges of all face feature points are set by using those of the immediately preceding frame image, so that the face feature points can be searched more efficiently.

In the face part searching apparatus 101 of the first specific example, as described above, in the frame image acquired by the image acquiring section 10, after the face area extracting section 20 detects the face area based on the face color pattern, The reference point setting unit 40 uses the response image (41) of the low frequency filter to detect the rough position of the face part in the face area. Then, the facial part extraction unit 50 extracts a detailed position of the facial part based on the rough position.

Therefore, the face part searching apparatus 101 according to the first embodiment.
According to the search for the face area in the frame image,
Since the facial color pattern generated based on the related frame image is used, it is possible to appropriately deal with a change in the illumination environment for the face to be searched.

Further, in the process of searching the face area, since the mutually different data of the color pattern data of the face color pattern and the projection data of the response image of the low frequency filter are used, a single data is used. It is possible to perform a search with higher reliability than a search that depends on.

Further, since the rough position of the face part detected by the reference point setting unit 40 is used when searching the detailed position of the face part, the position of the face part can be efficiently searched. You can

In the above description, each frame image of the moving image is used, but the present invention is not limited to this. For example, the image acquisition unit 1
0 is a scanner, and the face part search apparatus 101 can be applied to search for a face part in a still image from the scanner.

<Embodiment 2> FIG. 18 is a block diagram showing the configuration of the facial part searching apparatus 102 according to Embodiment 2 for explaining the present invention. As shown in FIG. 18, the face part search device 102 includes a tracking determination unit 60 that determines the tracking process described below, in addition to the configuration of the face part search device 101 of the specific example 1 described above.

When the face part of the frame image is searched, the tracking determination section 60 uses the detailed position of the face part extracted in the frame image immediately before that as the rough position of the frame image. It is determined whether or not the tracking process is executed. For this determination, a predetermined determination standard described later is used.

An example of the operation of the face part search device 102 is shown in FIG.
A description will be given according to the flowchart shown in FIG. Image acquisition unit 1
When a frame image is supplied from 0 (step S4
1), the tracking determination unit 60 determines whether or not to perform the tracking processing described above on this frame image (step S42). If the frame image from the image acquisition unit 10 is, for example, the first frame image in the moving image, there is no frame image immediately before it, and thus it is determined that the tracking process is not executed.

When the tracking process is not performed, the face area extraction unit 20, the reference point setting unit 40, and the face component extraction unit 50 perform the position of the face component on the frame image according to the same procedure as in the first specific example. Is extracted (steps S43 to 45), the face area detection control unit 24 determines whether or not there is a subsequent frame image (step S4).
6). If there is no subsequent frame image, the process of the face part search device 102 ends.

When a frame image subsequent to the frame image is supplied (step S41), the tracking determination section 60 determines the tracking process of the frame image based on the following determination criteria (step S42). . For example, using the certainty factor regarding the facial component position obtained by the facial component extraction unit 50, if the certainty factor for the immediately previous frame image is equal to or greater than a preset threshold value, it is determined to perform the tracking process.
Further, for example, when the number of loops for performing the series of processing in steps S43 to 45 reaches a preset number of times,
It can be set to perform a tracking process.

The tracking judgment unit 60 judges the frame image according to the above-mentioned judgment criteria, and when it judges that the tracking process is to be executed, the detailed face part position of the immediately preceding frame image is judged. Is instructed to the facial part extraction unit 50 so as to be treated as a rough position of the facial part in the frame image. The face part extraction unit 50 extracts detailed face part positions in the frame image according to the instruction from the tracking determination unit 60 (step S45).

According to the facial part searching apparatus 102 of the second specific example, when searching for a facial part of a frame image, the data obtained in the immediately preceding frame image can be used by the tracking processing described above, so that the facial part can be efficiently searched. Can be searched for. This is particularly useful when there is no significant change in the face image between the adjacent frame images of the moving image.

When the tracking determination unit 60 determines that the tracking process is to be executed, the process is executed by the same components as those in the specific example 1, so that a new mechanism for executing the tracking process is provided. There is no need.

Using the tracking process of this example,
For example, the rough facial part position or the detailed facial part position of the immediately previous frame image can be used for the generation of the facial color pattern in the facial area extraction unit 20. For example, after detecting the detailed face part position in the immediately preceding frame image, the face area of the next frame image is detected based on the detailed position, and the face color pattern is generated from this face area. This makes it possible to acquire a more appropriate facial color pattern.

<Embodiment 3> FIG. 20 is a block diagram showing the configuration of the face part search device 103 according to Embodiment 3 for explaining the present invention. In addition to the configuration of the face part searching device 102 of the specific example 2 described above, the face part searching device 103 includes a rotation processing unit 70 that performs a rotation process to be described later regarding the inclination of the face in the frame image.

The rotation processing unit 70 obtains the tilt angle of the face based on the positions of both eyes of the face component, and the position of the face component in which the tilt of the face is corrected by the obtained tilt angle is roughly calculated. The position is supplied to the facial part extraction unit 50. In the present specific example, the positions of the eyes described above are the positions of the detailed “eyes” of the frame image immediately before being referred to during the tracking process. However, when the tracking process is not performed, the reference point setting unit 40 is used. The detected rough "eye" position can be used.

The rotation processing unit 70 determines that the position coordinates of the "eyes" are, for example, left eye (x1, y1) and right eye (x2, y).
2), r = arctan ((y2-y1) / (x2-x1)) Equation (14) The tilt angle r with respect to the two coordinates is obtained from Equation (14),
The calculated tilt angle r is set as the tilt angle of the face. This tilt angle can be the tilt angle of the entire frame image. Furthermore, in order to correct the inclination of the face horizontally,
The position of the face part that is reversely rotated (-r) by the tilt angle is detected and supplied to the face part extraction unit 50.

FIG. 21 shows the operation of the face part search device 103.
A description will be given according to the flowchart shown in FIG. As described above, in the illustrated example, the rotation process is executed when the tracking process is executed. When it is determined that the frame image supplied from the image acquisition unit 10 is not subjected to the tracking process, the face part in the frame image is searched for in the same procedure as described in the specific example 1 (step S51). ~ S56).

A frame image is supplied from the image acquisition unit 10 (step S51), and the tracking determination unit 60
When it is determined that the tracking process is performed on the frame image (step S52), the rotation processing unit 70 obtains the tilt angle based on the detailed position of the face part obtained from the frame image immediately before the frame image. The rotation processing described above is applied to the frame image (step S57).
The rough position of the face part detected after the rotation processing described above is supplied to the face part extracting section 50, and the face part extracting section is based on the rough position adjusted by the rotation processing section 70. The detailed position of the face part is extracted in the same manner as in (step S55).

According to the face part search device 103 of the third specific example, even if the face of the frame image for which the face part is to be searched has a tilt, the above-described rotation processing in the rotation processing unit 70 causes the tilt to be substantially horizontal. Since the corrected information is supplied to the facial part extraction unit 50 that receives this information, the facial part extraction unit 50 can appropriately detect the detailed position of the facial part.

The face part searching apparatus 103 described above is provided with the tracking determining section 60. However, the face part searching apparatus 101 of the first specific example is provided with the rotation processing section 70 without the tracking determining section 60. It can be configured. In that case, all the frame images in which the face parts are to be searched are the targets of the rotation processing described above.

<Fourth Embodiment> FIG. 22 is a block diagram showing the arrangement of a facial part searching apparatus 104 according to the fourth embodiment for explaining the present invention. The face part search device 104 includes a face direction extraction unit 80 that detects the face direction of the frame image in addition to the configuration of the face part search device 101 of the first specific example.

The face orientation extraction unit 80 detects the orientation of the face in the frame image based on the rough position of the "eye" detected by the reference point setting unit 40. FIG. 23 shows a frame image portion (I) in which the face faces the front and a frame image portion (II) in which the face faces to the left.
In each image portion shown, the distance between the left edge of the face area and the rough position coordinates of the left eye is the left edge width, and the distance between the right edge of the face area and the rough position coordinates of the right eye is the right edge width. In this case, when the face is facing the front as in the image portion (I), the left end width and the right end width have substantially the same size. Also, for example, when the face is facing left as in image part (II), the right end width is larger than the left end width, and conversely, when the face is facing right (not shown). If so, the left end width is considered to be larger than the right end width. The face orientation extraction unit 80 detects the orientation of the face of the frame image based on the above-described relationship between the left edge width and the right edge width.

According to the face part searching apparatus 104 of the fourth specific example, the face direction of the frame image can be detected, and the information indicating the detected face direction can be obtained, for example, together with the rough position coordinates of the face part. By supplying the face part extraction unit 50, the face part extraction unit 50 can search for a detailed face part in consideration of the orientation of the face. Further, it can be output from the face part search device 104 together with the detailed positions of the face parts extracted by the face part extraction unit 50.

Conventionally, there has been known a method of detecting the orientation of a face based on a change in the distance between both eyes when the face is facing left and right from the front, a relative positional relationship between the eyes and the nose, and the like. But,
When the face is facing left and right, the difference between the left end width and the right end width described above is relatively likely to appear. Therefore, by paying attention to the relative position of the eyes in the face area as in the face part search device 104. , The orientation of the face can be easily detected.

In the specific example 4 described above, the reference point setting unit 40
The orientation of the face is detected based on the rough position coordinates of the face part extracted in step 1. However, by providing the tracking determination unit 60 in the face part search device 104, details of the face part in the immediately preceding frame image are obtained. Based on the position
The orientation of the face can be detected. As a result, a more accurate face orientation can be detected. Further, it is possible to detect the face orientation of the frame image based on the detailed positions of the face parts in the frame image, and use the detected face orientation for the subsequent frame image.

<Usage Form> The facial part searching apparatus according to the present invention is not limited to the above-described specific example, and the following usage forms can be adopted. In the above-mentioned specific examples 3 and 4, the inclination of the face or the direction of the face is detected based on the position of the eyes, but instead of this, for example, the positions of both ends (mouth corners) of the mouth can be used.

The face part extraction unit 50 for extracting the detailed positions of the face parts is not limited to the above-described example for setting the search range of the face feature points, and for example, the distance of rough position coordinates for both eyes is set between the pupils. A distance can be set, and an arbitrary region in consideration of this interpupillary distance can be set as the search range.

The information contained in the template handled by the face part extraction section 50 is not limited to the standard face part image described above, but other information may be used as long as it can extract facial feature point candidates from the face area. be able to. For example, an image portion showing an average face part of a plurality of face images, or a template including information such as a filter or a color simulating the shape of the face part can be used. For example, in the case of a filter simulating the shape of an eye, a high output can be obtained in the eye part by providing this filter with a circularity calculation function. Also, a plurality of templates can be set for each face part.

Further, the face part extraction section 50, for example, in order to ease the matching condition between the template and the frame image in scanning the template, for example, the edge positions of eyes and mouth,
The consistency between the two can be determined based on the color of the edge and the like. The average position coordinates of the candidates were used when selecting the optimum candidate from the facial feature point candidates, but instead of this, for example, the statistical positional relationship prepared in advance between the face parts is used. be able to.

The certainty factor relating to each facial feature point can be corrected based on the above-mentioned statistical positional relationship between each facial component, and the certainty factor obtained for each facial component is only used. Instead, it is possible to use the average reliability calculated from the reliability of all face parts in the face area.

[0115]

According to the facial part search device of the present invention,
Since the color pattern is used for searching the face area, even if there is a change in the illumination environment for the face to be searched, it is possible to cope with the change. Further, in the process of searching the face area, since a plurality of mutually different data such as the color pattern and the shade of the color described above are used, a more reliable and appropriate search can be performed than a search that depends on a single data. It can be carried out.

[Brief description of drawings]

FIG. 1 is a block diagram showing a configuration of a facial part search device according to a specific example 1 of the present invention.

FIG. 2 is a block diagram showing a configuration of a face area extraction unit.

FIG. 3 is an explanatory diagram illustrating a face area extraction unit.

FIG. 4 is an explanatory diagram for explaining a skin color model.

FIG. 5 is an explanatory diagram for explaining a face color pattern.

FIG. 6 is an explanatory diagram illustrating a face color pattern generation unit of a face area extraction unit.

FIG. 7 is an explanatory diagram illustrating a face area detection unit of a face area extraction unit.

FIG. 8 is a flowchart (part 1) showing the operation of the face area extracting unit.

FIG. 9 is a flowchart (No. 2) showing the operation of the face area extracting unit.

FIG. 10 is a flowchart (No. 3) showing the operation of the face area extracting unit.

FIG. 11 is a flowchart showing an operation of a reference point setting unit.

FIG. 12 is an explanatory diagram for explaining a low frequency filter process of a reference point setting unit.

FIG. 13 is an explanatory diagram (part 1) for explaining a reference point setting unit.

FIG. 14 is an explanatory diagram (Part 2) for explaining the reference point setting unit.

FIG. 15 is a block diagram (No. 1) showing a configuration of a face part extraction unit.

FIG. 16 is a block diagram (No. 2) showing the configuration of a face part extraction unit.

FIG. 17 is a block diagram (No. 3) showing the configuration of a face part extraction unit.

FIG. 18 is a block diagram showing a configuration of a facial part search device according to a second specific example of the present invention.

FIG. 19 is a flowchart showing the operation of the facial part searching device in the second specific example.

FIG. 20 is a block diagram showing a configuration of a facial part search device according to a specific example 3 of the present invention.

FIG. 21 is a flowchart showing the operation of the facial part searching device of the third specific example.

FIG. 22 is a block diagram showing the configuration of a facial part search device in a specific example 4 according to the present invention.

FIG. 23 is an explanatory diagram for explaining the operation of the face orientation extraction unit.

[Explanation of symbols]

101, 102, 103, 104 Face Part Searching Device 10 Image acquisition section 20 Face Area Extraction Unit 21 Face color pattern generator 22 Face color pattern storage 23 Face Area Detection Unit 24 Face Area Detection Control Unit 40 Reference point setting section 50 face parts extraction unit 51 Search Area Setting Section 52 Candidate extraction unit 52a template storage section 53 Candidate Selection Department 60 Tracking judgment section 70 Rotation processing unit 80 Face orientation extraction section

─────────────────────────────────────────────────── ─── Continuation of front page (51) Int.Cl. ⁷ identification code FI theme code (reference) G06T 7/60 150 G06T 7/60 150B (72) Inventor Hiroyuki Masai 1-7 Toranomon, Minato-ku, Tokyo No. 12 Oki Electric Industry Co., Ltd. (72) Inventor Takahiro Watanabe 1-7-12 Toranomon, Minato-ku, Tokyo F-term inside Oki Electric Industry Co., Ltd. (reference) 5B057 CA01 CA08 CA12 CA16 CB12 CB16 CE05 CE06 DA08 DB02 DB06 DB09 DC16 DC23 DC25 5L096 AA02 AA06 BA18 CA04 EA14 FA06 FA37 FA69 GA38 GA40 GA55 HA02 JA18

Claims (14)

[Claims] 1. An apparatus for searching the face part image from a color image including a face part image representing a face part, the image acquiring section acquiring the color image, and acquiring the face image from the acquired color image. A face area extraction unit that detects an image area corresponding to a face color pattern that is a color-related pattern as a face area and extracts an image portion of the face area; and the image portion from the image portion of the extracted face area. And a reference point setting unit that sets a reference point indicating a reference position for searching the face part image based on the gray level, and detects the face part image based on the set reference point. In addition, a face part search device comprising: a face part detection unit that obtains the position of the face part image. 2. The face area extraction unit, a face color pattern generation unit that generates the face color pattern based on the color image acquired by the image acquisition unit, and a storage that holds the generated face color pattern. The facial part searching device according to claim 1, further comprising: a section and an extracting section that performs the extraction of the face area based on the facial color pattern of the storage section. 3. The face color pattern generation unit generates a color pattern of the color image in a predetermined color space, and a portion of the generated color pattern corresponding to a model of a preset color pattern. 3. The facial part searching device according to claim 2, wherein is generated as the facial color pattern. 4. The image acquisition unit sequentially acquires a series of color images, and the face color pattern generation unit generates the face color pattern based on a predetermined color image of the series of color images. The face part search device according to claim 2. 5. The extraction unit determines success or failure of the extraction of the face area based on a preset determination criterion, and the face color pattern generation unit determines that the extraction has not succeeded. The facial part searching device according to claim 4, wherein a new facial color pattern is set for another color image subsequent to the color image. 6. The reference point setting unit detects peak values in two directions of the shade of the face area in two directions orthogonal to each other, and defines the shade portion corresponding to each peak value in the two directions as described above. The face part search device according to claim 1, wherein the face part search device is set as a reference point. 7. The reference point setting unit detects the shading from the face area by using a low frequency filter corresponding to a frequency band preset based on the position of the facial part for the shading of the facial part. The facial part search device according to claim 6. 8. The face part detection unit is face part information that is information related to a preset image of a face part,
Face part data including feature point information indicating a predetermined position in a face part corresponding to the face part information is given, and based on the reference point set by the reference point setting unit,
A scan range setting unit that sets a scan range of the face part data; and a scan part that scans the face part data within the set scan range, and the face part information of the data and an image portion corresponding to the reference point. A scanning unit that detects the position indicated by the characteristic point information when satisfying a predetermined criterion regarding consistency, and a position that is close to a predetermined position is selected from among the detected positions, and the selected position is set. The face part search device according to claim 1, further comprising a position determination unit that determines a position of the face part image in the face area. 9. The face part detection unit further includes other face parts other than the predetermined face part image based on the position determined by the position determination unit for the predetermined face part image in the color image. 9. The facial part searching device according to claim 8, further comprising a related scan range setting unit that sets the scan range for an image and supplies the set scan range to the scanning unit. 10. The image acquisition unit sequentially acquires a series of color images, and the face component detection unit sets the position of the face component image in the color image by the reference point setting unit for the color image. The reference point or the color image immediately before the color image is detected based on the position of the face component image detected by the face component detecting unit, and further, the face component detecting unit detects the detection by the color image. The facial part search device according to claim 1, further comprising a tracking determination unit that determines whether or not to perform the determination based on the reference point. 11. The color image including the face part image includes a plurality of face part images including a pair of face part images corresponding to both eyes, and further set by the reference point setting unit, Based on the two reference points corresponding to the pair of face part images of both eyes, the inclination angle of the face area with respect to the horizontal direction is obtained,
The face area is rotated to correct the inclination of the face area in the horizontal direction by the inclination angle, and the reference point of each face part image in the rotated face area is supplied to the face part detection unit. The facial part search device according to claim 1, further comprising: 12. The image acquisition unit sequentially acquires a series of color images, and the color image including the face part image includes a plurality of face part images including a pair of face part images corresponding to both eyes. Further, the face part detection section sets the two reference points corresponding to the pair of face part images of the eyes set by the reference point setting section or the color image immediately before the color image in the face part detection section. Based on the positions of the detected face part images of both eyes, the tilt angle of the face area of the color image with respect to the horizontal direction is obtained, and the tilt angle of the face area is corrected in the horizontal direction by the tilt angle. A rotation processing unit that rotates a face area and supplies the reference point of each face component image in the rotated face area to the face component detection unit, and the rotation processing unit sets the tilt angle to the tilt angle of the color image. Based on reference point The facial part search device according to claim 1, further comprising a tracking determination unit that determines whether or not to obtain it. 13. The color image including the face part image includes a plurality of face part images including a pair of face part images corresponding to both eyes, and further includes: The two distances between each of them and each of the end portions that define the face width of the face area and that are the closest to each of the pair of face part images are obtained, and the difference between the two distances is obtained. The face part search device according to claim 1, further comprising a face direction detection unit that detects the direction of the face of the color image based on. 14. A method of searching for a face part image from a color image including a face part image representing a face part, the image acquiring step of acquiring a color image, the step of acquiring a face image from the acquired color image. A face area extraction step of detecting an image area corresponding to a face color pattern that is a color-related pattern as a face area, and extracting an image portion of the face area; and extracting the image portion from the image portion of the extracted face area. A reference point setting step of setting a reference point indicating a reference position for searching the face part image based on the gray level; and detecting the face part image based on the set reference point. And a face part detecting step of obtaining the position of the face part image.