JP4219521B2 - Matching method and apparatus, and recording medium - Google Patents

Description

[0001]
BACKGROUND OF THE INVENTION
The present invention relates to a matching method and apparatus for performing template matching between an image including a human face and a template, an image processing method and apparatus using these methods and apparatuses, and a computer to execute the matching method and image processing method. The present invention relates to a computer-readable recording medium on which a program is recorded.
[0002]
[Prior art]
In an image such as a photograph obtained by taking a human image using a negative film, a reversal film, or the like, the most noticeable part is the face of the person. Therefore, in order to improve the appearance of the image, it is necessary to perform image processing on the image so that the human face has an appropriate color, density, and gradation. On the other hand, in order to perform image processing so that a person's face has an appropriate color, density, and gradation, it is necessary to appropriately extract the person's face area from the image. For this reason, for example, skin color area candidates are extracted from an image including a person, a skin color area is determined by defining a skin color color space based on a chroma or luminance histogram of each candidate, and the skin color area is defined as a face area. Method (Japanese Patent Laid-Open No. 6-309433), after preparing a plurality of face templates and performing template matching between the face templates and images to extract face area candidates, based on the skin color distribution in the face area candidates It has been proposed to extract a face area from an image by a method of extracting a face area (Japanese Patent Laid-Open No. 8-63597) or the like and perform image processing.
[0003]
[Problems to be solved by the invention]
However, the method described in JP-A-6-309433 extracts a region in a predetermined skin color space as a skin color region. Therefore, since the skin color density in the face image is distributed over a wide range when the contrast of the face image is large, there is a high possibility that an area having a skin color not included in the defined skin color space exists in the face area. As a result, it may not be possible to extract all the face regions. Further, when the color of the background of the face is included in the defined skin color space, the background is also extracted together with the face image. On the other hand, although the method described in the above-mentioned Japanese Patent Application Laid-Open No. 8-63597 can determine a certain face area by template matching using a face template, a face based on the skin color distribution from the extracted face area candidates. When extracting a region, the same problem as the method described in JP-A-6-309433 occurs.
[0004]
The present invention has been made in view of the above circumstances, and a matching method and apparatus capable of obtaining information for appropriately extracting a face region in consideration of the contrast of a face included in a face image, the matching method and An object of the present invention is to provide a computer-readable recording medium in which a program for causing a computer to execute an image processing method and apparatus using the apparatus, a matching method, and an image processing method is provided.
[0005]
[Means for Solving the Problems]
The matching method according to the present invention includes a face image including a plurality of face templates and a person's face, which include face contours and face components, and at least different face types, face orientations, and directions of light sources applied to the face. Template matching to find the matching degree with and determine the face template with the highest matching degree,
Based on the face template having the highest matching degree, information on the face direction in the face area including the face on the face image and the direction of the light source irradiated on the face is acquired.
[0006]
Here, the “face template” includes facial components such as the face outline, eyes, mouth, nose and ears of a human face, and the face type, face orientation and face are irradiated. This is obtained by changing at least the direction of the light source. In the “face template” of the present invention, the shape of the face outline and the arrangement position of the face components differ depending on the orientation of the face. In addition, when the direction of the light source is changed, the density distribution of the region surrounded by the outline of the face is different. For example, if the light source direction is the front of the face, the density distribution is uniform, but if the light source direction is the right side of the template, the right side of the area surrounded by the face outline is bright and the left side is dark. It becomes. The “face template” may be a three-dimensional representation of the face or a two-dimensional representation of the face. Examples of the “face type” include adults, children, women, men, yellow races, white races, black races, and the like. In addition to face types, face orientations, and light source directions, face templates having different face sizes may be used.
[0007]
Note that the face orientation can be known if the shape of the face contour and the positions of the facial components such as eyes and nose are known. Therefore, the “information about the orientation of the face” refers to information about the contour shape of the face and the position of the constituent parts of the face in the face template having the highest matching degree.
[0008]
Further, when the direction of the light source that irradiates the face is changed, the brightness distribution in the face is changed variously, and as a result, the density distribution of the face is changed variously. Therefore, “information on the direction of the light source” refers to information on the density distribution of the face region in the face template having the highest matching degree.
[0009]
In the matching method according to the present invention, a predetermined area including the face area in the face image is set as a face area candidate,
The template matching is preferably performed within the face area candidate.
[0010]
Here, the “region of a predetermined range including the face area” may be set by displaying the face image on a display unit such as a monitor and surrounding the face area with a rectangle, a circle, or the like. It may be set by dividing a plurality of areas and extracting an area including a face area from the divided areas.
[0011]
In the matching method according to the present invention, the plurality of face templates may be created in advance.
[0012]
Further, the plurality of face templates are created by changing a face direction and a light source direction in a plurality of reference templates created in advance according to the type of the face,
The template matching may be performed while acquiring a face template in which the face direction and the light source direction are changed.
[0013]
Furthermore, in the matching method according to the present invention, it is preferable that the template matching is performed by hierarchically obtaining matching degrees for the plurality of face templates.
[0014]
Here, “determining the matching level hierarchically” means that one of the three elements of the face type, the face direction, and the light source direction is fixed to two elements (for example, the face direction and the light source direction). This means that template matching is performed by changing only one element with (face type) as a variable. For example, when one element as a variable is set as a face type, first, template matching is performed for the face type, and a template having the highest matching degree is obtained to determine the face type. Next, the face type is fixed, template matching is performed using another element (for example, the face orientation) as a variable, and the template having the highest matching degree is obtained, whereby the face type and the face direction are determined. Then, the face type and face direction are fixed, and template matching is performed using another element (light source direction) as a variable, and a template having the highest matching degree is obtained. And the direction of the light source are determined.
[0015]
The image processing method according to the present invention obtains a skin color density distribution in the face region based on the information on the face direction and the light source direction obtained by the matching method according to the present invention,
Defining a skin color space in the face image based on the density distribution;
An area corresponding to the skin color space in the face image is extracted as the face area.
[0016]
Here, “determining the skin color density distribution in the face area based on the information on the face direction and the light source direction” means that the contour of the face area is estimated from the information on the face direction and the information on the face direction is used. It refers to obtaining the distribution of the high density area and the low density area within the outline of the face area from the information on the position of the facial component and the direction of the light source.
[0017]
“Define the skin color space based on the density distribution” means that the skin color space is defined so that all the skin color areas of the high density region and the low density region existing in the face region are included based on the density distribution. That means.
[0018]
In the image processing method according to the present invention, it is preferable to perform predetermined image processing on the skin color area.
[0019]
Here, as the “image processing”, various processes such as a color conversion process for converting the skin color area into a desired skin color, a gradation conversion process for converting the gradation of the face area, and a sharpness enhancement process can be used. .
[0020]
The matching device according to the present invention includes a face image including a plurality of face templates and a person's face that include at least a face type, a face direction, and a direction of a light source applied to the face, including a face outline and a face component. Template matching means for performing a template matching to obtain a matching degree with the template and determining a face template having the highest matching degree;
Information acquisition means for acquiring information relating to the face direction in the face region including the face on the face image and the direction of the light source irradiated to the face based on the face template having the highest matching degree; It is a feature.
[0021]
The matching device according to the present invention further includes region setting means for setting a predetermined region including the face region as a face region candidate in the face image,
The template matching means is preferably means for performing the template matching in the face area candidate.
[0022]
The matching device according to the present invention may further include template storage means for storing the plurality of face templates.
[0023]
Furthermore, in the matching device according to the present invention, reference template storage means for storing a plurality of reference templates created in advance according to the type of the face,
A template creating means for creating the plurality of face templates by changing a face direction and a light source direction in the plurality of reference templates;
The template matching unit may be a unit that performs the template matching while acquiring face templates having different face orientations and light source directions created by the template creation unit.
[0024]
Furthermore, in the matching device according to the present invention, it is preferable that the template matching unit is a unit that performs the template matching by hierarchically obtaining matching degrees for the plurality of face templates.
[0025]
An image processing apparatus according to the present invention comprises: means for obtaining a skin color density distribution in the face area based on information on the face direction and the light source direction acquired by the matching apparatus according to the present invention;
Means for defining a skin color space in the face image based on the density distribution;
Means for extracting, as the face area, an area corresponding to the skin color space in the face image.
[0026]
The image processing apparatus according to the present invention preferably further includes means for performing predetermined image processing on the skin color area.
[0027]
The matching method and the image processing method according to the present invention may be provided by being recorded on a computer-readable recording medium as a program for causing a computer to execute the matching method and the image processing method.
[0028]
【The invention's effect】
According to the matching method and apparatus of the present invention, template matching between a plurality of face templates and a face image is performed, a face template having the highest matching degree with a face region included in the face image is determined, and based on this face template Thus, information on the direction of the face in the face area on the face image and the direction of the light source irradiated on the face is acquired. Here, it is possible to know the shape of the face outline and the position of the components of the face from the information about the face direction, and the density of the face region that varies depending on the direction of the light source irradiated to the face from the information about the direction of the light source You can know the distribution. Therefore, by using the information on the face direction and the light source direction, various processes such as the extraction of the skin color area and the extraction of the face components can be easily performed.
[0029]
In addition, by setting a predetermined area including the face area in the face image as a face area candidate, and performing template matching within the face area candidate, compared with the case where template matching is performed on the entire face image, matching is performed. Therefore, the calculation time can be shortened.
[0030]
Furthermore, by creating a plurality of face templates in advance, template matching can be performed by simply using each face template without any modification to the face template.
[0031]
Furthermore, a plurality of face templates can be created by changing the face direction and the light source direction in the reference template created in advance, and saved by performing template matching while changing the face direction and the light source direction. It is possible to reduce the number of templates to be stored, thereby reducing the capacity of the storage means for storing the templates.
[0032]
In addition, by obtaining the matching level hierarchically, it is not necessary to perform template matching for all face templates, so that the calculation for template matching can be performed at high speed.
[0033]
In the image processing method and apparatus according to the present invention, the skin color density distribution in the face area in the face image is obtained based on the information about the face direction and the light source direction extracted by the matching method and apparatus according to the present invention. That is, the contour of the face area is estimated from the information related to the face direction, and the high density region and the low density in the contour of the face area are determined based on the information related to the position of the face component such as the nose and the light source direction. A density range distribution is obtained, which is obtained as a skin color density distribution. This density distribution is information that shows how the high density area and low density area in the face area are distributed to form the contrast of the face area. Based on this density distribution, this density distribution exists in the face area. The skin color space is defined so as to include all the skin color regions of the high density region and the low density region. Therefore, the skin color space can be defined so that all the skin colors of the face area are extracted even when the contrast of the face area is large, and thus the skin color area can be accurately extracted from the face image.
[0034]
Further, by performing image processing on the extracted skin color area, it is possible to perform image processing that makes the face area in the face image desired.
[0035]
DETAILED DESCRIPTION OF THE INVENTION
Embodiments of the present invention will be described below with reference to the drawings.
[0036]
FIG. 1 is a schematic block diagram showing a configuration of an image processing apparatus to which a matching apparatus according to an embodiment of the present invention is applied. As shown in FIG. 1, the image processing apparatus according to the present embodiment performs processing on image data S0 representing an image including a person's face captured by a digital camera or read by a scanner, and processed image data. S1 is obtained, and the image data S0 is read from the medium on which the image data S0 is recorded, or the reading means 1 that reads the image data S0 transferred via the network, and the image represented by the read image data S0 A face extracting unit 2 for extracting a person's face from the image, an image processing unit 3 for performing image processing on the image data S 0 to obtain processed image data S 1, An output means 4 for outputting to a monitor, a monitor 5 for displaying image data S0, etc., a face extracting means 2, an image processing means 3 and keyboard for performing various inputs to the output unit 4, and an input unit 6 consisting of a mouse or the like.
[0037]
The face extracting unit 2 includes an area specifying unit 11 that specifies a face area candidate F, which will be described later, in the image represented by the image data S0, and an area extracting unit 12 that extracts the face area candidate F specified by the area specifying unit 11. The template storage unit 13 storing the template data TD representing the plurality of templates T, and the template T and the face region candidate F extracted by the region extraction unit 12 are subjected to template matching to determine the face direction and the face region candidate F Matching means 14 for obtaining information I regarding the direction of the light source, and skin color area extracting means 15 for extracting the skin color area H from the face area candidate F based on the information I obtained by the matching means 14.
[0038]
The area designating unit 11 displays the image data S0 on the monitor 5, and in response to an instruction from the input unit 6 to the image displayed on the monitor 5 (hereinafter also referred to as S0) S0, for example, FIG. An area including a face is designated as a face area candidate F as shown in FIG. The face area candidate F is not limited to a rectangle, and may be a circle or an ellipse. Alternatively, the face region candidate F may be specified by dividing the image S0 into a plurality of regions, assigning numbers for specifying each region, and specifying the number of the region including the face. For example, as shown in FIG. 3, when the image S0 is divided into 5 × 5 areas, the number of the face area candidate F is 22.
[0039]
Note that the face area candidate F is set in advance when the image data S0 is acquired, and the information of the set face area candidate F (for example, the position of the face area candidate and the number of the area including the face) is used as the tag information. It may be added in advance to the data S0. Thereby, the area designating unit 11 can display the image S0 in which the face area candidate F is designated on the monitor 5 on the basis of the tag information of the image data S0 without an instruction from the input unit 6. .
[0040]
The area extracting unit 12 extracts the face area candidate F designated by the area designating unit 11, and at the time of extraction, area designation is performed so that the area other than the face area is minimized in the face area candidate F. The size of the face area candidate F specified in the means 11 may be slightly changed. Note that only the extracted face area candidate F may be displayed on the monitor 5 by extracting the face area candidate F in the area extracting means 12, but the face area candidate is displayed on the image S0 as shown in FIG. The boundary of F may be displayed.
[0041]
In the template storage means 13, template data TD representing a plurality of templates T having different sizes, orientations, and directions of the light source to be radiated for a plurality of types of faces is created and stored by live action or computer graphics. Has been. Here, it is more preferable to create the template data TD by computer graphics because the template T is not affected by noise or blurring during photographing. The template T includes facial contours according to the type, size, and orientation of the face, and facial components such as eyes, nose, and eyebrows.
[0042]
Here, the types of faces include adults, children, babies, elderly people, women, men, yellow races, white races or black races. Also, in order to make the face direction different, for example, the state where the face is facing the front is set to 0 degree, and the face contour and configuration are changed according to the face direction which is different every 15 degrees from 0 to 360 degrees. What is necessary is just to change the arrangement | positioning of components and to make it the template T. Also, in order to make the direction of the light source different, the state where the light source is in front of the face is 0 degree, and the direction is different every 15 degrees within a range of ± 90 degrees for a total of four directions of up, down, left and right and oblique directions The template T may be an image of a face that is irradiated with light from the face toward the face. Note that the template data TD may be created without considering the face size. In this case, in the matching means 14 described later, the face area candidate F and the template T are normalized and matching is performed.
[0043]
The template data TD is preferably stored in the template storage unit 13 in a hierarchical manner. For example, as shown in FIG. 4, a baby face having a different face direction is associated with a lower hierarchy of the baby's face, and the shadow position on the face, that is, the light source It is preferable to associate baby faces with different positions. In this way, by storing the template data TD in a hierarchical manner, first the face type is determined, then the face direction corresponding to the type is determined, and then the direction of the light source is further determined. Template matching can be performed. Therefore, the template matching between the face area candidate F and the template T in the matching means 14 can be performed more efficiently than when performing template matching using all the templates T.
[0044]
Note that a live-action or computer graphics three-dimensional shape model representing a plurality of types of faces may be stored as the template data TD. FIG. 5 is a diagram showing template data TD representing a three-dimensional shape model of a male face. In this way, when the template data TD is a three-dimensional shape model, the three-dimensional shape model is displayed on the monitor 5 and the face means and the direction of the irradiated light source are variously changed by the input means 6 and matching means 4 described later. Template matching with the face area candidate F can be performed. As described above, the template data TD is used as a three-dimensional shape model, and the template T is matched with the face region candidate F by changing the face direction and the light source direction in the template T represented by the template data TD. The template data TD stored in the storage unit 13 can be reduced, whereby the capacity of the template storage unit 13 can be reduced.
[0045]
Also in this case, template matching can be performed by hierarchically changing the direction of the face and the direction of the illuminating light source, thereby efficiently performing template matching between the face region candidate F and the template T. it can.
[0046]
Further, the template data TD may represent a binarized template T.
[0047]
When the data representing the face area candidate F extracted by the area extracting means 12 is input, the matching means 14 sequentially reads the template data TD from the template storage means 13 and performs template matching between the face area candidate F and the template T. Is to do. Specifically, the face area candidate F and the template T are binarized, and the degree of matching with the binarized face area candidate F and the template T is obtained. When the template T is binarized, the template T is used as it is. Then, the face direction and the light source direction in the template T having the maximum matching degree are obtained as information I representing the face direction and the light source direction in the face region candidate F. Specifically, assuming that the data in the binarized face area candidate F is F ′, and the binarized template T data is TD ′, the following equation (1) is applied each time the position and type of the template T are changed. As shown, the degree of matching M between the data F ′ and the data TD ′ is calculated.
[0048]
M = (Σ | Fij′−TDij ′ |) / n (1)
Where Fij ′: the pixel value of the image represented by the data F ′
TDij ′: the pixel value of the image represented by the data TD ′ corresponding to the data F ′
n: number of pixels of the image represented by the data Fij ′ or the data TDij ′
Then, the face direction and the light source direction in the template T with the calculated matching degree M being maximized are obtained as information I representing the face direction and the light source direction in the face region candidate F. Note that template matching may be performed without binarizing the template T and the face area candidate F.
[0049]
Here, the orientation of the face can be known if the shape of the face contour and the positions of the facial components such as eyes and nose are known. Therefore, the information related to the orientation of the face refers to information related to the position of the face component such as the face contour shape, eyes, and nose in the template T having the highest degree of matching.
[0050]
Further, when the direction of the light source that irradiates the face is changed, the brightness distribution in the face is changed variously, and as a result, the density distribution of the face is changed variously. Therefore, the information regarding the direction of the light source refers to information regarding the density distribution of the face in the template T having the highest matching degree.
[0051]
If the template data TD is hierarchized in the template storage means 13 as shown in FIG. 4, after matching the face type, matching is performed only for the face direction in the lower hierarchy of the face of the matched type. Further, after matching the face direction, matching may be performed only for the direction of the light source in the lower hierarchy of the face direction.
[0052]
In addition, when the area designating unit 11 designates an area, the type of face included in the image S0 (for example, a female face, a baby face, etc.) is inputted in advance, so that the matching means 14 inputs it. The template T representing the different types of faces can be used for matching preferentially, and this also enables efficient template matching.
[0053]
Further, template matching may be performed by Log-Polar conversion of the face area candidate F and the template T. Here, since the human face has a nearly circular shape, for example, a face image irradiated with light from the direction of 90 degrees to the right as shown in FIG. When Log-Polar conversion is performed using the Y axis in FIG. 6 as a reference, the contour of the face extends in a substantially horizontal direction on polar coordinates as shown in FIG. Therefore, by performing the Log-Polar conversion on the face area candidate F and the template T, it is possible to make template matching easier than in the case of performing template matching between areas having a shape close to a circle.
[0054]
The skin color area extracting unit 15 selects the face area candidate F as a bright area, that is, a low density area and a dark area, that is, a high density, based on the information I indicating the face direction and the light source direction in the face area candidate F obtained by the matching means 14. A skin color area is extracted by defining a skin color space for each area. The skin color region H is extracted as follows. First, it is possible to know the outline of the face in the face area candidate F from the information on the face direction, and further to know the distribution of the bright and dark areas in the area surrounded by the outline, that is, the density distribution from the information indicating the light source direction. it can. At this time, the position of the facial components can be known from the information on the face orientation, but the cheeks under the nose and eyes that are brighter than other areas on the face, especially based on the information on the nose and eye positions. You can know the position of. Then, a skin color space is defined based on the density distribution and the positions of the nose and cheeks, and the skin color region H is extracted.
[0055]
Specifically, as shown in FIG. 6, when light is irradiated from the right side of the face facing forward, the right half area of the face (hereinafter referred to as the right area) is bright but the left half area (hereinafter left). The area is dark. Further, since the nose and cheeks protrude in the human face, the right side of the nose and the cheeks are brighter in the right region than in the other regions. In the left area, the nose and cheeks are brighter than other areas.
[0056]
Therefore, for the right region, the skin color space is defined so as to include the brightest portion, that is, the right side of the nose and the cheek portion, and the right half region of the face is extracted. On the other hand, the skin color space is defined so that the left half of the face also includes the left side of the nose and the cheek part, and the left half region of the face is extracted.
[0057]
The image processing unit 3 performs image processing such as gradation processing, color conversion processing, and sharpness processing on the image data S0 of the skin color region H so that the extracted skin color region H has a desired density and color. To obtain processed image data S1. Note that image processing may be performed on regions other than the skin color region H. Here, as the sharpness processing, the degree of sharpness enhancement is reduced when the face is large, and the degree of sharpness enhancement is increased when the face is small, according to the size of the skin color region H, that is, the size of the face. Can be processed as follows.
[0058]
Next, the operation of this embodiment will be described. FIG. 8 is a flowchart showing the operation of this embodiment. First, the image data S0 is read by the reading means 1 (step S1) and input to the face area extracting means 2. The image data S0 is displayed on the monitor 5 (step S2), and the face area candidate F is designated by the input from the input means 6 (step S3). Then, the area extraction means 12 extracts the face area candidate F (step S4). The extracted face area candidate F is input to the matching unit 14, and the template matching between the template T represented by the template data TD and the face area candidate F is performed using the template data TD stored in the template storage unit 13. Is performed (step S5). When the template matching is completed, information I representing the face direction and the light source direction is generated based on the template T having the largest matching degree (step S6).
[0059]
The skin color area extracting means 15 extracts the skin color area H from the face area candidate F as described above based on the information I (step S7). Then, the image processing means 3 performs image processing such as gradation processing, color conversion processing, and sharpness processing on the skin color region H in the image represented by the image data S0 to obtain processed image data S1 (step S1). S8). Then, the processed image data S1 is output from the output means 4 (step S9), and the process is terminated.
[0060]
As described above, in the present embodiment, the face area candidate F is subjected to template matching using the plurality of templates T to obtain the information I indicating the face direction and the light source direction in the face area candidate F. From the information, it is possible to know the face contour shape and the position of the components of the face, and from the information about the direction of the light source, it is possible to know the density distribution of the face region candidate F that varies according to the direction of the light irradiated to the face. . Then, the contour of the face region is estimated from information related to the face orientation in the information I, and the density distribution in the contour of the face is determined from the information related to the position of the components of the face such as the nose and the light source direction in the information related to the face orientation. Is obtained. Therefore, by defining the skin color space based on this density distribution so as to include all the skin color areas of the high density area and the low density area existing in the area within the face contour in the face area candidate F, The skin color space can be defined so that all the skin colors of the face region candidate F can be extracted even if the contrast of the face region is large, whereby the skin color region H can be accurately extracted.
[0061]
In addition, since the area designating unit 11 sets a predetermined area including the face as the face area candidate F, and template matching is performed on the set face area candidate F, the template matching is performed on the entire face image. In comparison, the calculation time for matching can be shortened.
[0062]
In the above embodiment, information on the face direction and information on the direction of the light source is obtained as the information I, and the skin color region H is extracted based on these information. The position of the facial component such as the nose can be known, and the facial component from the face region candidate F can be easily extracted using the information I.
[Brief description of the drawings]
FIG. 1 is a block diagram showing a configuration of an image processing apparatus according to an embodiment of the present invention.
FIG. 2 is a diagram for explaining designation of a face area candidate F
FIG. 3 is a diagram for explaining designation of a face area candidate F
FIG. 4 is a diagram showing a hierarchical template
FIG. 5 is a diagram showing an example of a three-dimensional template
FIG. 6 is a diagram for explaining skin color region extraction;
FIG. 7 is a diagram showing face area candidates subjected to Log-Polar conversion.
FIG. 8 is a flowchart showing the operation of this embodiment.
[Explanation of symbols]
1 Reading means
2 Face extraction means
3 Image processing means
4 Output means
5 Monitor
6 Input means
11 Area designation means
12 region extraction means
13 Template storage means
14 Matching means
15 Skin color area extraction means

Claims (33)

Includes facial contours and facial components, and irradiates at least adult, child, baby, elderly, female, male, yellow, white or black face types, face orientation and face While acquiring a plurality of face templates having at least different light source directions by changing the face direction and the light source direction in a plurality of reference templates created in advance according to the type of the face, Perform template matching to obtain a matching degree with a face image including a human face, determine a face template with the highest matching degree,
A matching method comprising: obtaining information on a face direction in a face region including a face on the face image and a light source direction irradiated on the face based on a face template having the highest matching degree. A region of a predetermined range including the face region in the face image is set as a face region candidate,
The matching method according to claim 1, wherein the template matching is performed within the face area candidate.   3. The matching method according to claim 1, wherein the template matching is performed by obtaining a matching level hierarchically for the plurality of face templates. Based on the information on the face direction and the light source direction acquired by the matching method according to any one of claims 1 to 3, a skin color density distribution in the face region is obtained,
Defining a skin color space in the face image based on the density distribution;
An image processing method, wherein an area corresponding to the skin color space in the face image is extracted as the face area.   5. The image processing method according to claim 4, wherein predetermined image processing is performed on the skin color area. Multiple criteria created in advance according to face type of at least adult, child, baby, elderly, female, male, yellow race, white race or black race, including facial contours and facial components Reference template storage means for storing a template;
A template creating means for creating the plurality of face templates by changing a face direction and a light source direction in the plurality of reference templates;
The degree of matching between the face template and a face image including a human face is obtained while acquiring a plurality of face templates having different face types, face orientations and light source directions created by the template creation means. Template matching means for performing template matching and determining a face template having the highest matching degree;
Information acquisition means for acquiring information relating to the face direction in the face region including the face on the face image and the direction of the light source irradiated to the face based on the face template having the highest matching degree; A unique matching device. A region setting means for setting a region of a predetermined range including the face region in the face image as a face region candidate;
7. The matching apparatus according to claim 6, wherein the template matching means is means for performing the template matching in the face area candidate.   8. The matching apparatus according to claim 6, wherein the template matching unit is a unit that performs the template matching by hierarchically obtaining a matching degree for the plurality of face templates. Means for obtaining a skin color density distribution in the face area based on information on the face orientation and the light source orientation obtained by the matching device according to any one of claims 6 to 8;
Means for defining a skin color space in the face image based on the density distribution;
An image processing apparatus comprising: means for extracting an area corresponding to the skin color space in the face image as the face area.   The image processing apparatus according to claim 9, further comprising means for performing predetermined image processing on the skin color area. Includes facial contours and facial components, and irradiates at least adult, child, baby, elderly, female, male, yellow, white or black face types, face orientation and face While acquiring a plurality of face templates having at least different light source directions by changing the face direction and the light source direction in a plurality of reference templates created in advance according to the type of the face, Performing a template matching to obtain a matching degree with a face image including a human face, and determining a face template having the highest matching degree;
A matching method having a procedure for acquiring information on a face direction in a face region including a face on the face image and a direction of a light source irradiated on the face based on a face template having the highest matching degree. A computer-readable recording medium on which a program for execution is recorded. The method further comprises a step of setting a predetermined area including the face area as a face area candidate in the face image,
The computer-readable recording medium according to claim 11, wherein the template matching procedure is a procedure for performing the template matching in the face area candidate.   The computer-readable recording medium according to claim 11 or 12, wherein the template matching procedure is a procedure in which the template matching is performed by hierarchically obtaining matching degrees for the plurality of face templates. . A procedure for obtaining a skin color density distribution in the face region based on information on the face direction and the light source direction acquired by the matching method according to claim 1;
Defining a skin color space in the face image based on the density distribution;
A computer-readable recording medium comprising: a step of extracting an area corresponding to the skin color space in the face image as the face area.   15. The computer-readable recording medium according to claim 14, further comprising a procedure for performing predetermined image processing on the skin color area. Includes facial contours and facial components, and irradiates at least adult, child, baby, elderly, female, male, yellow, white or black face types, face orientation and face Performing template matching to obtain a matching degree between a plurality of face templates having different light source directions and a face image including a human face, and determining a face template having the highest matching degree;
Based on the face template having the highest matching degree, obtain information on the face direction in the face area including the face on the face image and the direction of the light source irradiated to the face,
Based on the information on the face direction and the light source direction, obtain a skin color density distribution in the face region;
Defining a skin color space in the face image based on the density distribution;
An image processing method, wherein an area corresponding to the skin color space in the face image is extracted as the face area. A region of a predetermined range including the face region in the face image is set as a face region candidate,
The image processing method according to claim 16, wherein the template matching is performed within the face area candidate.   The image processing method according to claim 16 or 17, wherein the plurality of face templates are created in advance. The plurality of face templates are created by changing a face direction and a light source direction in a plurality of reference templates created in advance according to the type of the face,
The image processing method according to claim 16 or 17, wherein the template matching is performed while acquiring a face template in which the face direction and the light source direction are changed.   20. The image processing method according to claim 16, wherein the template matching is performed by obtaining matching levels hierarchically for the plurality of face templates.   21. The image processing method according to claim 16, wherein predetermined image processing is performed on the skin color area. Includes facial contours and facial components, and irradiates at least adult, child, baby, elderly, female, male, yellow, white or black face types, face orientation and face Template matching means for performing template matching to obtain a matching degree between a plurality of face templates having different light source directions and a face image including a human face, and determining a face template having the highest matching degree;
Based on the face template having the highest matching degree, information acquisition means for acquiring information on the face direction in the face region including the face on the face image and the direction of the light source irradiated to the face;
Means for obtaining a skin color density distribution in the face region based on information on the face direction and the light source direction;
Means for defining a skin color space in the face image based on the density distribution;
An image processing apparatus comprising: means for extracting an area corresponding to the skin color space in the face image as the face area. A region setting means for setting a region of a predetermined range including the face region in the face image as a face region candidate;
The image processing apparatus according to claim 22, wherein the template matching means is means for performing the template matching in the face area candidate.   24. The image processing apparatus according to claim 22, further comprising template storage means for storing the plurality of face templates. Reference template storage means for storing a plurality of reference templates created in advance according to the type of the face;
A template creating means for creating the plurality of face templates by changing a face direction and a light source direction in the plurality of reference templates;
The template matching means is means for performing the template matching while acquiring face templates having different face orientations and light source directions created by the template creation means. Image processing device.   The image processing apparatus according to any one of claims 22 to 25, wherein the template matching unit is a unit that performs the template matching by hierarchically obtaining a matching degree for the plurality of face templates. .   27. The image processing apparatus according to claim 22, further comprising means for performing predetermined image processing on the skin color area. Includes facial contours and facial components, and irradiates at least adult, child, baby, elderly, female, male, yellow, white or black face types, face orientation and face Performing a template matching to obtain a matching degree between a plurality of face templates having different light source directions and a face image including a human face, and determining a face template having the highest matching degree;
A procedure for obtaining information on the face direction in the face area including the face on the face image and the direction of the light source irradiated on the face based on the face template having the highest matching degree;
A procedure for obtaining a skin color density distribution in the face region;
Defining a skin color space in the face image based on the density distribution;
A computer-readable recording medium storing a program for causing a computer to execute an image processing method, comprising: extracting a region corresponding to the skin color space in the face image as the face region. The method further comprises a step of setting a predetermined area including the face area as a face area candidate in the face image,
29. The computer-readable recording medium according to claim 28, wherein the template matching procedure is a procedure for performing the template matching in the face area candidate.   30. The computer-readable recording medium according to claim 28 or 29, wherein the plurality of face templates are created in advance. A step of creating the plurality of face templates by changing a face direction and a light source direction in a plurality of reference templates created in advance according to the type of the face;
30. The computer-readable method according to claim 28, wherein the template matching procedure is a procedure for performing the template matching while acquiring a face template in which the face direction and the light source direction are changed. recoding media.   32. The computer according to claim 28, wherein the template matching procedure is a procedure in which the template matching is performed by hierarchically obtaining matching degrees for the plurality of face templates. A readable recording medium.   The computer-readable recording medium according to any one of claims 28 to 32, further comprising a procedure of performing predetermined image processing on the skin color area.

Images