JP2009237614A - Setting of face area corresponding to face image in target image - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To attain coordination between accuracy and processing time of face area setting process. <P>SOLUTION: An image processing apparatus includes: a face area detecting unit for detecting an area corresponding to an image of a face in a target area as an original face area on the basis of image data; an organ area detecting unit for detecting areas corresponding to facial organ images in the original face area as organ areas; and a face area determining unit for setting a determined face area corresponding to the image of the face in a target image. The face area determining unit has a first processing mode for setting the original face area as the determined face area, and a second processing mode for setting an area where the original face area is adjusted based on the detection result of the organ areas as the determined face area. <P>COPYRIGHT: (C)2010,JPO&INPIT

Description

  The present invention relates to setting of a face area corresponding to a face image in a target image.

  A technique for setting a face area corresponding to a face image by sequentially cutting out partial images from an image represented by image data and determining whether or not the cut out partial image is an image corresponding to a face is known. (For example, Patent Documents 1 to 4).

JP 2007-241477 A JP 2007-94633 A JP 2007-193404 A JP 2007-193740 A

  In setting the face area in the target image, it has been desired to adjust the setting accuracy of the face area and the processing time.

  SUMMARY An advantage of some aspects of the invention is that it provides a technique capable of adjusting the accuracy and processing time of face area setting processing.

  In order to solve at least a part of the above problems, the present invention can be realized as the following forms or application examples.

Application Example 1 An image processing apparatus,
A face area detection unit that detects an area corresponding to a face image in the target image based on image data as an original face area;
An organ region detection unit that detects a region corresponding to an image of a facial organ in the original face region as an organ region;
A face area determination unit that sets a determined face area corresponding to a face image in the target image, the first processing mode for setting the original face area as the determined face area, and the detection result of the organ area An image processing apparatus comprising: a face area determination unit having a second processing mode in which an area obtained by adjusting the original face area is set as the determined face area.

  In this image processing apparatus, in the first processing mode, the original face area detected based on the image data is set as the determined face area, and in the second processing mode, the original face area is detected based on the image data. An organ area in the face area is detected, and an area obtained by adjusting the original face area based on the detection result of the organ area is set as the determined face area. Therefore, in this image processing apparatus, it is possible to adjust the accuracy and processing time of the face area setting process.

Application Example 2 The image processing apparatus according to Application Example 1, further comprising:
An accuracy setting unit for setting the accuracy when setting the determined face area;
The face area determination unit sets the determined face area in the first processing mode when the accuracy is within a predetermined range, and the second processing mode when the accuracy is higher than the predetermined range. An image processing apparatus for setting the determined face area by using

  In this image processing apparatus, since the processing mode for setting the determined face area is selected according to the set accuracy, the accuracy of the face area setting process and the processing time are adjusted according to the set accuracy. be able to.

[Application Example 3] The image processing apparatus according to Application Example 1 or Application Example 2,
The accuracy setting unit is an image processing apparatus that acquires usage information specifying a usage of the determined face area, and sets the accuracy based on the usage information.

  In this image processing apparatus, it is possible to adjust the accuracy and processing time of the face area setting process according to the use of the determined face area.

[Application Example 4] The image processing apparatus according to any one of Application Examples 1 to 3,
In the second mode, the face area determination unit sets the determined face area by adjusting an inclination of the original face area based on a positional relationship between the plurality of detected organ areas. .

  In this image processing apparatus, it is possible to adjust the accuracy and processing time related to the inclination of the face area setting process.

Application Example 5 The image processing apparatus according to Application Example 1, further comprising:
A detection inclination setting unit for setting the inclination of the face image to be detected from the target image;
A face inclination estimator that estimates the inclination of the face image represented in the target image based on the positional relationship between the plurality of detected organ regions;
Detection determination for determining whether the face image represented in the target image is a face image to be detected based on the inclination of the face image to be detected and the estimated inclination of the face image An image processing apparatus.

  In this image processing apparatus, the inclination of the face image to be detected is set from the target image, and the inclination of the face image represented in the target image is estimated and detected based on the positional relationship between the plurality of detected organ regions. It is determined whether the face image represented in the target image is the face image to be detected based on the inclination of the face image to be detected and the estimated inclination of the face image. It is possible to accurately determine whether or not the detected face image is a face image to be detected.

Application Example 6 The image processing apparatus according to any one of Application Example 1 to Application Example 5,
The face area detection unit
A determination target setting unit that sets a determination target image region that is an image region on the target image;
A memory for storing evaluation data for calculating an evaluation value indicating the probability that the determination target image area is an image area corresponding to a face image having one of a plurality of predetermined discrete slopes And
An evaluation value calculation unit that calculates the evaluation value based on the evaluation data and image data corresponding to the determination target image region;
A determination unit that determines whether or not the determination target image region is an image region corresponding to a face image based on the evaluation value;
And an area setting unit that sets the original face area based on the position and size of the determination target image area determined to be an image area corresponding to a face image.

  In this image processing apparatus, it is possible to adjust the accuracy and processing time related to the inclination of the face area setting process.

[Application Example 7] The image processing apparatus according to any one of Application Example 1 to Application Example 6,
The type of facial organ is at least one of a right eye, a left eye, and a mouth.

  In this image processing apparatus, it is possible to adjust the accuracy and processing time of the face area setting process by using detection of an organ area corresponding to at least one of the right eye, the left eye, and the mouth.

  The present invention can be realized in various modes. For example, an image processing method and apparatus, a face area detection method and apparatus, a computer program for realizing the functions of these methods or apparatuses, and the computer The present invention can be realized in the form of a recording medium recording the program, a data signal including the computer program and embodied in a carrier wave, and the like.

Next, embodiments of the present invention will be described in the following order based on examples.
A. First embodiment:
A-1. Configuration of image processing device:
A-2. Face area detection processing:
B. Second embodiment:
C. Variations:

A. First embodiment:
A-1. Configuration of image processing device:
FIG. 1 is an explanatory diagram schematically showing the configuration of a printer 100 as an image processing apparatus according to the first embodiment of the present invention. The printer 100 of this embodiment is an ink jet color printer that supports so-called direct printing, in which an image is printed based on image data acquired from a memory card MC or the like. The printer 100 includes a CPU 110 that controls each unit of the printer 100, an internal memory 120 configured by a ROM and a RAM, an operation unit 140 configured by buttons and a touch panel, a display unit 150 configured by a liquid crystal display, and a printer. An engine 160 and a card interface (card I / F) 170 are provided. The printer 100 may further include an interface for performing data communication with other devices (for example, a digital still camera or a personal computer). Each component of the printer 100 is connected to each other via a bus.

  The printer engine 160 is a printing mechanism that performs printing based on print data. The card interface 170 is an interface for exchanging data with the memory card MC inserted into the card slot 172. In this embodiment, an image file including image data is stored in the memory card MC.

  The internal memory 120 stores an image processing unit 200, a display processing unit 310, and a print processing unit 320. The image processing unit 200 is a computer program for executing face area detection processing described later under a predetermined operating system. The display processing unit 310 is a display driver that controls the display unit 150 to display processing menus, messages, images, and the like on the display unit 150. The print processing unit 320 is a computer program for generating print data from image data, controlling the printer engine 160, and printing an image based on the print data. The CPU 110 implements the functions of these units by reading and executing these programs from the internal memory 120.

  The image processing unit 200 includes an area detection unit 210, an accuracy setting unit 220, and an information addition unit 230 as program modules. The region detection unit 210 detects an image region corresponding to a predetermined type of subject image (a face image and a face organ image) in the target image represented by the target image data. The region detection unit 210 includes a determination target setting unit 211, an evaluation value calculation unit 212, a determination unit 213, a region setting unit 214, and a face region determination unit 216. The face area determination unit 216 includes a face inclination estimation unit 217. The functions of these units will be described in detail in the description of the face area detection process described later. As will be described later, the area detection unit 210 detects the original face area corresponding to the face image and the organ area corresponding to the face organ image. It functions as an area detection unit. Further, the face inclination estimation unit 217 and the face region determination unit 216 function as a face region determination unit in the present invention.

  The accuracy setting unit 220 sets the accuracy when determining the face area corresponding to the face image in the target image. The information adding unit 230 adds predetermined information (for example, information indicating the positions of the face area and the organ area) to the image file including the image data.

  The internal memory 120 also stores a plurality of preset face learning data FLD and a plurality of face organ learning data OLD. The face learning data FLD and the facial organ learning data OLD are used for detection of a predetermined image area by the area detection unit 210. FIG. 2 is an explanatory diagram showing types of face learning data FLD and face organ learning data OLD. FIGS. 2A to 2F show the types of face learning data FLD or facial organ learning data OLD and image regions detected using the type of face learning data FLD or facial organ learning data OLD. An example is shown.

  The contents of the face learning data FLD will be described in detail in the description of the face area detection process described later. The face learning data FLD of the present embodiment is set in association with the face inclination. Here, the face inclination means the inclination (rotation angle) of the face image in the target image plane (in-plane). Further, in this embodiment, the inclination of the image or the image area is changed from the reference state when the state in which the upward direction of the image or the image area coincides with the upward direction of the target image is set as the reference state (inclination = 0 degree). This is expressed by the clockwise rotation angle. For example, for the face inclination, a state where the face image is positioned along the vertical direction of the target image (a state where the top of the head is facing upward and the chin is facing downward) is referred to as a reference state (face inclination = 0 degrees). In this case, it is represented by the clockwise rotation angle of the face image from the reference state.

  In the internal memory 120, two face learning data FLD shown in FIGS. 2A and 2B, that is, face learning data FLD corresponding to 0 degree face inclination shown in FIG. The face learning data FLD corresponding to the face inclination of 30 degrees shown in 2 (b) is stored. As will be described later, the face learning data FLD corresponding to a certain face inclination is set by learning so that a face image having a face inclination value in a range of plus or minus 15 degrees around the face inclination can be detected. . In addition, the human face is substantially symmetrical. Therefore, if face learning data FLD corresponding to 0 degree face inclination (FIG. 2A) and face learning data FLD corresponding to 30 degree face inclination (FIG. 2B) are prepared in advance. By rotating these two face learning data FLD in units of 90 degrees, it is possible to obtain face learning data FLD that can detect face images of any face inclination.

  The facial organ learning data OLD is set in association with the combination of the facial organ type and the organ inclination. In this embodiment, eyes (right eye and left eye) and mouth are set as types of facial organs. The organ inclination means the inclination (rotation angle) of the image of the facial organ in the image plane (in-plane), similar to the face inclination described above. Similar to the face inclination, the organ inclination is the facial organ from the reference state when the state in which the facial organ image is positioned along the vertical direction of the target image is the reference state (organ inclination = 0 degrees). It is expressed by the clockwise rotation angle of the image.

  The internal memory 120 stores the four facial organ learning data OLD shown in FIGS. 2 (c) to 2 (f), that is, the face corresponding to the combination of the eyes and the 0 ° organ inclination shown in FIG. 2 (c). The organ learning data OLD, the facial organ learning data OLD corresponding to the combination of the eyes and the 30-degree organ inclination shown in FIG. 2D, and the mouth and 0-degree organ inclination shown in FIG. 2 and the facial organ learning data OLD corresponding to the combination of the mouth and the 30-degree organ inclination shown in FIG. Since the eyes and mouth are different types of subjects, it can be expressed that the facial organ learning data OLD is set corresponding to the combination of the type of subject and the tilt of the subject.

  Similar to the face learning data FLD, the facial organ learning data OLD corresponding to a certain organ inclination is obtained by learning so that an image of an organ having an organ inclination value in the range of plus or minus 15 degrees around the organ inclination can be detected. Is set. The eyes and mouth of the person are substantially symmetrical. Therefore, for eyes, facial organ learning data OLD (FIG. 2 (c)) corresponding to an organ inclination of 0 degrees and facial organ learning data OLD (FIG. 2 (d)) corresponding to an organ inclination of 30 degrees. If one is prepared in advance, the face organ learning data OLD that can detect the images of the eyes of any organ inclination can be obtained by rotating these two face organ learning data OLD in units of 90 degrees. Similarly for the mouth, the facial organ learning data OLD (FIG. 2 (e)) corresponding to an organ inclination of 0 degrees and the facial organ learning data OLD corresponding to an organ inclination of 30 degrees (FIG. 2 (f)). Are prepared in advance, it is possible to obtain facial organ learning data OLD capable of detecting mouth images of any organ inclination. In this embodiment, it is assumed that the right eye and the left eye are the same type of subject, and the right eye region corresponding to the right eye image and the left eye region corresponding to the left eye image are detected using the common facial organ learning data OLD. However, assuming that the right eye and the left eye are different types of subjects, dedicated face organ learning data OLD may be prepared for right eye region detection and left eye region detection, respectively.

A-2. Face area detection processing:
FIG. 3 is a flowchart showing the flow of face area detection processing in the first embodiment. The face area detection process in this embodiment is a process for determining a face area corresponding to a face image in an image represented by image data.

  In step S110, the accuracy setting unit 220 (FIG. 1) sets the accuracy required when determining the face area corresponding to the face image in the target image. In the present embodiment, the accuracy in determining the face area refers to the inclination of the determined face area (determined face area FAf described later) (hereinafter also referred to as “face area inclination”) and the face represented in the target image. Means the average value of the difference between the actual inclination of the image (hereinafter also referred to as “actual inclination”). That is, high accuracy means that the average value of the difference between the face area inclination and the actual inclination is small, and low precision means that the average value of the difference between the face area inclination and the actual inclination is large. To do. In this embodiment, an image area (for example, a face area) and a window (described later) that defines the image area include information for specifying the direction, and the above-described inclination of the face area refers to the upward direction of the face area. This means a clockwise rotation angle from the reference state when the state coincident with the upper direction of the target image is set as the reference state (tilt = 0 degree).

  The accuracy setting unit 220 acquires usage information specifying the usage of the determined face area, and sets the accuracy based on the usage information. FIG. 4 is an explanatory diagram illustrating an example of the use of the determined face area. FIG. 4 shows a state where a list of image processing menus provided by the image processing unit 200 (FIG. 1) is displayed on the display unit 150. In this embodiment, two types of skin color correction and face deformation are set as image processing menus, and the determined face area is used for two image processes of skin color correction and face deformation.

  Skin color correction is an image process that corrects the skin color of a person in a face area or an image area set based on the face area to a preferred skin color, and the effect of the difference between the face area inclination and the actual inclination on the processing result is Relatively small. Therefore, when performing skin color correction, it is relatively easy to accept that the accuracy in determining the face area is low, that is, that the average value of the difference between the face area inclination and the actual inclination is large. Therefore, when skin color correction is selected by the user via the operation unit 140, the accuracy setting unit 220 acquires usage information indicating that the usage of the face area is skin color correction, and sets a relatively low value as the accuracy. To do.

  On the other hand, face deformation is image processing for deforming a face area or an image in an image area set based on the face area. Here, since the face of a person is substantially symmetrical, if the difference between the face area inclination and the actual inclination is large, the result of the face deformation process may be unnatural. Therefore, when performing face deformation, it is desired that accuracy in determining a face area is high, that is, an average value of differences between the face area inclination and the actual inclination is small. Therefore, when face deformation is selected by the user via the operation unit 140, the accuracy setting unit 220 acquires use information that the use of the face area is face deformation, and sets a relatively high value as the accuracy. To do.

  In step S120 (FIG. 3), the image processing unit 200 (FIG. 1) acquires image data representing an image to be subjected to face area detection processing. In the printer 100 of this embodiment, when the memory card MC is inserted into the card slot 172, thumbnail images of the image files stored in the memory card MC are displayed on the display unit 150. The user selects one or more images to be processed via the operation unit 140 while referring to the displayed thumbnail images. The image processing unit 200 acquires an image file including image data corresponding to one or more selected images from the memory card MC and stores it in a predetermined area of the internal memory 120. The acquired image data is referred to as original image data, and the image represented by the original image data is referred to as original image OImg.

  In step S130 (FIG. 3), the area detection unit 210 (FIG. 1) performs an original face area detection process. The original face area detection process is a process for detecting an image area corresponding to the face image as the original face area FAo. FIG. 5 is a flowchart showing the flow of the original face area detection process. FIG. 6 is an explanatory diagram showing an outline of the original face area detection process. An example of the original image OImg is shown at the top of FIG.

  In step S310 in the original face area detection process (FIG. 5), the area detection unit 210 (FIG. 1) generates face detection image data representing the face detection image FDImg from the original image data representing the original image OImg. In the present embodiment, as shown in FIG. 6, the face detection image FDImg is an image having a size of 320 horizontal pixels × 240 vertical pixels. The area detection unit 210 generates face detection image data representing the face detection image FDImg by performing resolution conversion of the original image data as necessary. Note that the face detection image FDImg in this embodiment corresponds to the target image in the present invention, and the face detection image data corresponds to the target image data in the present invention.

  In step S320 (FIG. 5), the determination target setting unit 211 (FIG. 1) sets the size of the window SW used for setting the determination target image area JIA (described later) to an initial value. In step S330, the determination target setting unit 211 places the window SW at an initial position on the face detection image FDImg. In step S340, the determination target setting unit 211 determines whether the image area defined by the window SW arranged on the face detection image FDImg is an image area corresponding to the face image (hereinafter referred to as “face”). It is also set in a determination target image area JIA that is a target of determination. The middle part of FIG. 6 shows a state in which a window SW having an initial value size is arranged at an initial position on the face detection image FDImg, and an image area defined by the window SW is set as the determination target image area JIA. Yes. In this embodiment, as will be described later, the determination target image area JIA is sequentially set while changing the size and position of the square-shaped window SW, but the initial value of the size of the window SW is the horizontal size 240. The initial position of the window SW is such that the upper left vertex of the window SW overlaps the upper left vertex of the face detection image FDImg. Further, the window SW is arranged with the inclination of 0 degree. Note that, as described above, the inclination of the window SW is a reference when the upper direction of the window SW coincides with the upper direction of the target image (face detection image FDImg) as a reference state (inclination = 0 degree). It means the clockwise rotation angle from the state.

  In step S350 (FIG. 5), the evaluation value calculation unit 212 (FIG. 1) calculates a cumulative evaluation value Tv used for face determination for the determination target image area JIA based on the image data corresponding to the determination target image area JIA. In the present embodiment, face determination is executed for each specific face inclination set in advance. That is, for each specific face inclination, it is determined whether or not the determination target image area JIA is an image area corresponding to a face image inclined by the specific face inclination. Therefore, the cumulative evaluation value Tv is also calculated for each specific face inclination. Here, the specific face inclination means a preset face inclination value. In the present embodiment, a total of 12 face inclinations (0 degrees, 30 degrees, 60 degrees,..., 330 degrees) obtained by increasing the inclination from 0 degrees to 30 degrees are set as specific face inclinations.

  FIG. 7 is an explanatory diagram showing an outline of a method of calculating the cumulative evaluation value Tv used for face determination. In this embodiment, N filters (filter 1 to filter N) are used to calculate the cumulative evaluation value Tv. The external shape of each filter has the same aspect ratio as that of the window SW (that is, has a square shape), and a positive region pa and a negative region ma are set for each filter. The evaluation value calculation unit 212 calculates the evaluation values vX (that is, v1 to vN) by sequentially applying the filter X (X = 1, 2,..., N) to the determination target image area JIA. Specifically, the evaluation value vX is determined from the sum of the luminance values of pixels located in the area on the determination target image area JIA corresponding to the plus area pa of the filter X, based on the determination target image area JIA corresponding to the minus area ma. This is a value obtained by subtracting the sum of the luminance values of the pixels located in the upper region.

  The calculated evaluation value vX is compared with a threshold thX (that is, th1 to thN) set corresponding to each evaluation value vX. In this embodiment, when the evaluation value vX is equal to or greater than the threshold thX, it is determined that the determination target image area JIA is an image area corresponding to the face image for the filter X, and the value “ 1 "is set. On the other hand, if the evaluation value vX is smaller than the threshold thX, it is determined that the determination target image area JIA is not an image area corresponding to the face image with respect to the filter X, and the value “0” is set as the output value of the filter X Is done. Weight coefficients WeX (that is, We1 to WeN) are set for each filter X, and the sum of products of output values and weight coefficients WeX for all filters is calculated as a cumulative evaluation value Tv.

  Note that the aspect of the filter X used for face determination, the threshold thX, the weighting coefficient WeX, and the threshold TH described later are the face learning data FLD (FIG. 2A and FIG. 2) for each of the 12 specific face inclinations. b) is set in advance as a reference). Since the face learning data FLD in the present embodiment is data for calculating an evaluation value indicating the certainty that the determination target image area JIA is an image area corresponding to the face image, the evaluation data in the present invention. It corresponds to.

  The face learning data FLD is generated by learning using a sample image. FIG. 8 is an explanatory diagram illustrating an example of a sample image used for learning. For learning, a face sample image group composed of a plurality of face sample images that are known in advance to be images corresponding to face images, and a plurality of information that is known in advance to be images that do not correspond to face images. A non-face sample image group composed of non-face sample images.

  Since generation of the face learning data FLD by learning is executed for each specific face inclination, as shown in FIG. 8, a face sample image group corresponding to each of 12 specific face inclinations is prepared. For example, the generation of the face learning data FLD for the specific face inclination of 0 degree is executed using the face sample image group and the non-face sample image group corresponding to the specific face inclination of 0 degree. The face learning data FLD is generated using a face sample image group and a non-face sample image group corresponding to a specific face inclination of 30 degrees.

  The face sample image group corresponding to each specific face inclination includes a plurality of face sample images in which the ratio of the size of the face image to the image size is within a predetermined value range and the inclination of the face image is equal to the specific face inclination. (Hereinafter also referred to as “basic face sample image FIo”). The face sample image group is an image obtained by enlarging and reducing the basic face sample image FIo with a predetermined magnification in a range from 1.2 times to 0.8 times with respect to at least one basic face sample image FIo (for example, FIG. 8). And images obtained by changing the face inclination of the basic face sample image FIo within a range of plus or minus 15 degrees (for example, the images FIc and FId in FIG. 8).

  Learning using a sample image is executed by, for example, a method using a neural network, a method using boosting (for example, adaboost), a method using a support vector machine, or the like. For example, when learning is performed by a method using a neural network, a face sample image group corresponding to a specific face inclination (see FIG. 8) for each filter X (that is, filters 1 to N, see FIG. 7). And the evaluation value vX (that is, v1 to vN) is calculated using all the sample images included in the non-face sample image group, and a threshold thX (that is, th1 to thN) that achieves a predetermined face detection rate is set. . Here, the face detection rate is the ratio of the number of face sample images determined to be an image corresponding to a face image by threshold determination based on the evaluation value vX with respect to the total number of face sample images constituting the face sample image group. Means.

  Next, the weight coefficient WeX (that is, We1 to WeN) set for each filter X is set to an initial value, and cumulative evaluation is performed on one sample image selected from the face sample image group and the non-face sample image group. A value Tv is calculated. As will be described later, in the face determination, when the cumulative evaluation value Tv calculated for a certain image is equal to or greater than a predetermined threshold TH, the image is determined to be an image corresponding to the face image. In learning, the value of the weighting coefficient WeX set for each filter X is corrected based on the correctness of the threshold determination result based on the cumulative evaluation value Tv calculated for the selected sample image (face sample image or non-face sample image). Is done. Thereafter, the selection of the sample image, the threshold value determination based on the cumulative evaluation value Tv calculated for the selected sample image, and the correction of the value of the weighting coefficient WeX based on the correctness of the determination result are the face sample image group and the non-face sample image. It is repeatedly executed for all the sample images included in the group. By executing the above processing for each specific face inclination, face learning data FLD for each specific face inclination is generated.

  When the cumulative evaluation value Tv is calculated for each specific face inclination for the determination target image area JIA (step S350 in FIG. 5), the determination unit 213 (FIG. 1) sets the cumulative evaluation value Tv for each specific face inclination. The threshold value TH is compared (step S360). When the cumulative evaluation value Tv is greater than or equal to the threshold value TH for a specific face inclination, the area detection unit 210 determines that the determination target image area JIA is an image area corresponding to a face image inclined by the specific face inclination. The position of the determination target image area JIA, that is, the coordinates of the currently set window SW and the specific face inclination are stored (step S370). On the other hand, if the cumulative evaluation value Tv is smaller than the threshold value TH for any specific face inclination, the process of step S370 is skipped.

  In step S380 (FIG. 5), the area detection unit 210 (FIG. 1) determines whether or not the entire face detection image FDImg has been scanned by the window SW having the currently set size. If it is determined that the entire face detection image FDImg has not been scanned yet, the determination target setting unit 211 (FIG. 1) moves the window SW in a predetermined direction by a predetermined movement amount (step S390). The lower part of FIG. 6 shows how the window SW has moved. In this embodiment, in step S390, the window SW is moved to the right by a movement amount corresponding to 20% of the horizontal size of the window SW. If the window SW is arranged at a position where it cannot move further to the right, the window SW returns to the left end of the face detection image FDImg in step S390, and the window SW has a size of 2 in the vertical direction. It is assumed that it moves downwards by the amount of movement of the percentage. When the window SW is arranged at a position where it cannot move further downward, the entire face detection image FDImg is scanned. After the movement of the window SW (step S390), the processes after the above-described step S340 are executed for the moved window SW.

  If it is determined in step S380 (FIG. 5) that the entire face detection image FDImg has been scanned by the window SW having the currently set size, it is determined whether or not all the predetermined sizes of the window SW have been used. (Step S400). In this embodiment, as the size of the window SW, in addition to the initial value (maximum size) of 240 horizontal pixels × vertical 240 pixels, horizontal 213 pixels × vertical 213 pixels, horizontal 178 pixels × vertical 178 pixels, horizontal 149 pixels × 149 pixels vertically, 124 pixels horizontally × 124 pixels vertically, 103 pixels horizontally × 103 pixels vertically, 86 pixels wide × 86 pixels high, 72 pixels wide × 72 pixels high, 60 pixels wide × 60 pixels high, 50 pixels wide × 50 vertical Total of 15 pixels: 41 pixels wide x 41 pixels wide, 35 pixels wide x 35 pixels wide, 29 pixels wide x 29 pixels wide, 24 pixels wide x 24 pixels high, 20 pixels wide x 20 pixels high (minimum size) The size of is set. If it is determined that there is a size of the window SW that is not yet used, the determination target setting unit 211 (FIG. 1) changes the size of the window SW to the next smaller size than the currently set size ( Step S410). That is, the size of the window SW is first set to the maximum size, and then changed to a smaller size in order. After the change of the size of the window SW (step S410), the processing after step S330 described above is executed for the window SW having the changed size.

  If it is determined in step S400 (FIG. 5) that all of the predetermined size of the window SW has been used, the area setting unit 214 (FIG. 1) executes an original face area setting process (step S420). 9 and 10 are explanatory diagrams showing an outline of the original face area setting process. The area setting unit 214 determines that the cumulative evaluation value Tv is equal to or greater than the threshold value TH in step S360 of FIG. 5 and determines the coordinates of the window SW (that is, the position and size of the window SW) and the specific face inclination stored in step S370. Based on this, the original face area FAo is set. Specifically, when the specific face inclination is 0 degree, the image area defined by the window SW (that is, the determination target image area JIA) is set as the original face area FAo as it is. On the other hand, when the specific face inclination is other than 0 degrees, the inclination of the window SW is matched with the specific face inclination (that is, the window SW is clocked by the specific face inclination around a predetermined point (for example, the center of gravity of the window SW). The image area defined by the window SW after the inclination is changed is set as the original face area FAo. For example, as shown in FIG. 9A, when it is determined that the cumulative evaluation value Tv is greater than or equal to the threshold value TH for a specific face inclination of 30 degrees, as shown in FIG. Is changed to 30 degrees, and the image area defined by the window SW after the inclination change is set as the original face area FAo.

  In step S370, when a plurality of windows SW that partially overlap each other with respect to a specific face inclination are stored, the region setting unit 214 sets a predetermined point in each window SW (for example, the center of gravity of the window SW). One new window (hereinafter also referred to as “average window AW”) having an average size of each window SW is set with the average coordinate of the coordinates as the center of gravity. For example, as shown in FIG. 10A, when four windows SW (SW1 to SW4) that partially overlap each other are stored, as shown in FIG. One average window AW having an average size of the sizes of the four windows SW is defined with the average coordinate of the coordinates of the center of gravity of the four windows SW as the center of gravity. At this time, as described above, when the specific face inclination is 0 degree, the image area defined by the average window AW is set as the original face area FAo as it is. On the other hand, when the specific face inclination is other than 0 degrees, the inclination of the average window AW is made to coincide with the specific face inclination (that is, the specific face inclination is centered on a predetermined point (for example, the center of gravity of the average window AW)). The image area defined by the average window AW after the inclination is changed is set as the original face area FAo (see FIG. 10C).

  As shown in FIG. 9, even when one window SW not overlapping with other windows SW is stored, a plurality of windows SW partially overlapping each other shown in FIG. 10 are stored. Similarly, one window SW itself can be interpreted as the average window AW.

  In the present embodiment, an image obtained by enlarging and reducing the basic face sample image FIo to a face sample image group (see FIG. 8) used for learning at a predetermined magnification in a range from 1.2 times to 0.8 times. (For example, images FIa and FIb in FIG. 8), the face image size relative to the window SW size is slightly larger or smaller than the basic face sample image FIo. A face area FAo can be detected. Therefore, in the present embodiment, only the 15 discrete sizes described above are set as the size of the window SW, but the original face area FAo can be detected for facial images of any size. Similarly, in this embodiment, images obtained by changing the face inclination of the basic face sample image FIo in a range of plus or minus 15 degrees (for example, images FIc and FId in FIG. 8) are used as the face sample image group used for learning. Therefore, even when the inclination of the face image with respect to the window SW is slightly different from the basic face sample image FIo, the original face area FAo can be detected. Therefore, in the present embodiment, only the 12 discrete inclination values described above are set as the specific face inclination, but the original face area FAo can be detected for face images having any inclination.

  If the original face area FAo is not detected in the original face area detection process (step S130 in FIG. 3) (step S140: No), the face area detection process ends. On the other hand, when at least one original face area FAo is detected (step S140: Yes), the image processing unit 200 (FIG. 1) performs subsequent processing to be executed based on the accuracy set in step S110. judge. That is, when the use of the face area is skin color correction and the set accuracy is a relatively low value (step S150: No), the process proceeds to step S160.

  In step S160 (FIG. 3), the face area determination unit 216 (FIG. 1) sets the determined face area FAf. The determined face area FAf is a face area set as a final detection result of the face area detection process. In step S160, the face area determination unit 216 sets the original face area FAo as the determined face area FAf. As described above, the original face area FAo is an image area defined by the average window AW after the inclination of the average window AW is changed to match the specific face inclination. Therefore, when the original face area FAo is set as the determined face area FAf, the inclination of the determined face area FAf (face area inclination) is 12 discrete inclinations (0 degrees, 30 degrees, 60 set as the specific face inclination). Degrees,..., 330 degrees). Therefore, the difference between the inclination of the determined face area FAf (face area inclination) and the actual inclination (actual inclination) of the face image represented in the target image is a value in the range of plus or minus 15 degrees. Therefore, in this case, the average value of the difference between the face area inclination and the actual inclination is relatively large, and the accuracy in determining the face area is relatively low. The method for setting the determined face area FAf in step S160 corresponds to the method for setting the determined face area FAf in the first processing mode of the present invention.

  On the other hand, when the use of the face area is face deformation and the set accuracy is a relatively high value (step S150: Yes in FIG. 3), the process proceeds to step S170. In step S170, the area detection unit 210 (FIG. 1) selects one of the detected original face areas FAo.

  In step S180 (FIG. 3), the region detection unit 210 (FIG. 1) performs an organ region detection process. The organ area detection process is a process of detecting an image area corresponding to the facial organ image in the selected original face area FAo as an organ area. As described above, in this embodiment, three types of the right eye, the left eye, and the mouth are set as the types of facial organs. In the organ region detection process, the right eye region EA (r) corresponding to the right eye image is set. The left eye area EA (l) corresponding to the left eye image and the mouth area MA corresponding to the mouth image are detected.

  FIG. 11 is a flowchart showing the flow of the organ region detection process. FIG. 12 is an explanatory diagram showing an outline of the organ region detection process. An example of the face detection image FDImg (see FIG. 6) used in the face detection process is shown in the uppermost part of FIG.

  The detection of the organ area from the face detection image FDImg is performed in the same manner as the detection of the original face area FAo. That is, as shown in FIG. 12, a rectangular window SW is arranged on the face detection image FDImg while changing its size and position (steps S520, S530, S580 to S610 in FIG. 11), and the arranged windows An image area defined by SW is set as a determination target image area JIA that is a target of determination of whether or not the image area is an organ area corresponding to an image of a facial organ (hereinafter also referred to as “organ determination”) (FIG. 11). Step S540). Note that the possible size and position of the window SW are determined based on the size and position of the selected original face area FAo. For example, the value that can be taken as the horizontal length of the window SW is set to a predetermined number of steps between the maximum size and the minimum size obtained by multiplying the horizontal length of the original face area FAo by a predetermined number. Is done. Further, for example, the position that the window SW can take is set to a range in which the center of the window SW is located in the original face area FAo. Further, the window SW is arranged in a state where the inclination is 0 degree (a reference state in which the upper direction of the window SW coincides with the upper direction of the face detection image FDImg).

  When the determination target image area JIA is set, a cumulative evaluation value Tv used for organ determination is calculated for each organ (right eye, left eye, mouth) using the facial organ learning data OLD (FIG. 1) (FIG. 1). 11 step S550). The aspect of the filter X used for calculation of the cumulative evaluation value Tv and organ determination, the threshold thX, the weighting coefficient WeX, and the threshold TH (see FIG. 7) are defined in the facial organ learning data OLD. Note that the learning for setting the facial organ learning data OLD is performed by using a plurality of organ sample images that are known in advance to be images corresponding to the facial organs, similarly to the learning for setting the facial learning data FLD. It is executed using the constructed organ sample image group and the non-organ sample image group composed of a plurality of non-organ sample images that are known in advance not to correspond to facial organs.

  In the original face area detection process (FIG. 5), calculation of the cumulative evaluation value Tv and face determination are executed for all specific face inclinations, whereas in the organ area detection process (FIG. 11), cumulative evaluation is performed. The calculation of the value Tv and the organ determination are performed using the facial organ learning data OLD (see FIGS. 2C to 2F) corresponding to the same organ inclination as the specific face inclination of the selected original face area FAo. This is executed only for the same organ inclination as the specific face inclination of the original face area FAo. However, also in the organ region detection process, the calculation of the cumulative evaluation value Tv and the organ determination may be executed for all the specific organ inclinations.

  When the calculated cumulative evaluation value Tv is equal to or greater than the predetermined threshold TH, the determination target image area JIA is regarded as an image area corresponding to the image of the organ of the face, that is, the position of the determination target image area JIA, that is, the current The coordinates of the set window SW are stored (step S570 in FIG. 11). On the other hand, when the cumulative evaluation value Tv is smaller than the threshold value TH, the process of step S570 is skipped.

  After the entire range in which the window SW can be located is scanned for all the predetermined sizes of the window SW, an organ region setting process by the region setting unit 214 (FIG. 1) is executed (step S620 in FIG. 11). FIG. 13 is an explanatory diagram showing an outline of the organ region setting process. The organ area setting process is the same as the original face area setting process (see FIGS. 9 and 10). The area setting unit 214 determines that the cumulative evaluation value Tv is greater than or equal to the threshold value TH in step S560 of FIG. 11, and stores the coordinates of the window SW stored in step S570, the specific face inclination corresponding to the original face area FAo, Based on the above, an organ region is set as an image region corresponding to the facial organ image. Specifically, when the specific face inclination is 0 degree, the image area defined by the window SW (that is, the determination target image area JIA) is set as an organ area as it is. On the other hand, when the specific face inclination is other than 0 degrees, the inclination of the window SW is matched with the specific face inclination (that is, the window SW is clocked by the specific face inclination around a predetermined point (for example, the center of gravity of the window SW). The image region defined by the window SW after the inclination is changed is set as the organ region. For example, as shown in FIG. 13A, for a specific face inclination of 30 degrees, in a window SW (er) corresponding to the right eye, a window SW (el) corresponding to the left eye, and a window SW (m) corresponding to the mouth When it is determined that the cumulative evaluation value Tv is equal to or greater than the threshold value TH, as shown in FIG. 13B, the inclination of each window SW is changed to 30 degrees and is defined by each window SW after the inclination change. Image areas are set as organ areas (right eye area EA (r), left eye area EA (l), mouth area MA).

  Similarly to the original face area setting process, when a plurality of windows SW that partially overlap each other are stored, the average coordinates of the coordinates of a predetermined point (for example, the center of gravity of the window SW) in each window SW are calculated. When one new window (average window AW) having the center of gravity and the average size of each window SW is set and the specific face inclination is 0 degree, the image area defined by the average window AW is When the organ area is set as it is and the specific face inclination is other than 0 degree, the inclination of the average window AW is matched with the specific face inclination (that is, the average window AW is set to a predetermined point (for example, the center of gravity of the average window AW)). The image area defined by the average window AW after changing the inclination is rotated clockwise by a specific face inclination as the center. It is set as a region.

  When the organ area is not detected in the organ area detection process (step S180 in FIG. 3) (step S190: No), the original face area FAo selected in step S170 is an image area that truly corresponds to the face image. It is determined that the non-image area is erroneously detected as the original face area FAo (step S200). In this case, setting of the determined face area FAf based on the selected original face area FAo (step S210 described later) is not executed. In the present embodiment, the case where the organ area is not detected means that at least one of the organ areas (the right eye area EA (r), the left eye area EA (l), and the mouth area MA) corresponding to the three types of organs. It means the case where it was not detected.

  In the organ area detection process (step S180 in FIG. 3), when all of the organ areas (right eye area EA (r), left eye area EA (l), and mouth area MA) corresponding to the three types of organs are detected ( In step S190: Yes, the face area determining unit 216 (FIG. 1) sets the determined face area FAf (step S210). FIG. 14 is an explanatory diagram showing a method for setting the determined face area FAf. First, as shown in FIG. 14A, the face inclination estimation unit 217 of the face area determination unit 216 calculates the inclination of the straight line CL connecting the center of gravity of the right eye area EA (r) and the center of gravity of the left eye area EA (l). The actual inclination (actual inclination) of the face image represented in the target image is estimated to be equal to the inclination of the calculated straight line CL. That is, the actual inclination is estimated based on the positional relationship between the right eye area EA (r) and the left eye area EA (l). Next, as shown in FIG. 14B, the face area determination unit 216 changes the inclination of the average window AW that defines the original face area FAo so as to match the estimated actual inclination (that is, the average window AW). Is rotated clockwise by the actual inclination estimated around the center of gravity of the average window AW), and the image area defined by the average window AW after the inclination change is set as the determined face area FAf. At this time, the determined face area FAf is a rectangular image area whose two outer edges are parallel to the straight line CL.

  In the setting of the determined face area FAf in step S210 (FIG. 3), the inclination of the determined determined face area FAf (face area inclination) is an estimate of the actual inclination (actual inclination) of the face image represented in the target image. Matches the value. In addition, the value that can be taken as the estimated value of the actual slope is not a discrete value but a continuous value. Therefore, in this case, the average value of the difference between the face area inclination and the actual inclination is relatively small, and the accuracy in determining the face area is relatively high. The method for setting the determined face area FAf in step S210 corresponds to the method for setting the determined face area FAf in the second processing mode of the present invention.

  In step S220 (FIG. 3), area detection unit 210 (FIG. 1) determines whether or not there is an original face area FAo that has not yet been selected in step S170. When it is determined that there is an unselected original face area FAo (step S220: No), the process returns to step S170, and one of the unselected original face areas FAo is selected, and the processes after step S180 are executed. Is done. On the other hand, when it is determined that all the original face areas FAo have been selected (step S220: Yes), the process proceeds to step S230.

  In step S230 (FIG. 3), the information adding unit 230 (FIG. 1) adds information indicating the set position (coordinates) of the determined face area FAf as attached information to the image file including the original image data. Thereby, an image file including original image data representing the original image OImg and information indicating the position (coordinates) of the determined face area FAf in the original image OImg is generated. In addition to the information indicating the position (coordinates) of the determined face area FAf, information indicating the position (coordinates) of the organ area (right eye area EA (r), left eye area EA (l), mouth area MA) or step S110 Information indicating the accuracy set in step 1 may be added as attached information to the image file.

  As described above, in the face area detection process by the printer 100 of the first embodiment, the original face area FAo in the face detection image FDImg is detected based on the image data, and the accuracy set in step S110 is a relatively low value. In this case, the original face area FAo itself is set as the determined face area FAf. At this time, the organ region detection process is not executed. At this time, the setting accuracy of the determined face area FAf is relatively low, but the processing time is relatively short. On the other hand, if the accuracy set in step S110 is a relatively high value, an organ area in the original face area FAo is detected, and an area obtained by adjusting the original face area FAo based on the detection result of the organ area is a determined face area. Set as FAf. At this time, the processing time is relatively long, but the setting accuracy of the determined face area FAf is relatively high. For this reason, in this embodiment, it is possible to adjust the setting accuracy and the processing time when setting the determined face area FAf.

B. Second embodiment:
FIG. 15 is an explanatory diagram schematically showing the configuration of a printer 100a as an image processing apparatus according to the second embodiment of the present invention. The printer 100a of the second embodiment is different from the printer 100 of the first embodiment shown in FIG. 1 in the configuration of the image processing unit 200. That is, the printer 100a of the second embodiment includes a detection inclination setting unit 240 and a detection determination unit 250 instead of including the accuracy setting unit 220 (FIG. 1). The other configuration of the printer 100a is the same as that of the printer 100 of the first embodiment shown in FIG.

  The detection inclination setting unit 240 sets the inclination of the face image to be detected from the target image. The detection determination unit 250 determines the face represented in the target image based on the set inclination of the face image to be detected and the estimated value of the actual inclination (actual inclination) of the face image represented in the target image. It is determined whether the image is a face image to be detected.

  FIG. 16 is a flowchart showing the flow of face area detection processing in the second embodiment. The face area detection process in the second embodiment is a process for determining a face area corresponding to the face image in the image represented by the image data, similarly to the face area detection process in the first embodiment shown in FIG.

  In step S112 (FIG. 16), the detected tilt setting unit 240 (FIG. 1) sets the tilt of the face image to be detected from the target image (hereinafter also referred to as “detected tilt range”). The detected inclination setting unit 240 acquires application information specifying the application of the determined face area, and sets a detected inclination range based on the application information. FIG. 17 is an explanatory diagram illustrating an example of the use of the face area to be determined. FIG. 17 shows a state where a list of image processing menus provided by the image processing unit 200 (FIG. 1) is displayed on the display unit 150. In this embodiment, ID photo detection is set as the image processing menu, and the determined face area is used for image processing for ID photo detection.

  ID photo detection is processing for detecting a photo image having a composition suitable for an ID photo. Specifically, the photographic image having a composition suitable for the ID photo is an image in which the inclination of the face image in the target image is relatively small. When the ID photo detection is selected by the user via the operation unit 140, the detection inclination setting unit 240 acquires usage information that the usage of the face area is ID photo detection. An inclination range of 340 degrees to 360 degrees is set as a detected inclination range. FIG. 18 is an explanatory diagram illustrating an example of the detected inclination range.

  The processing contents of steps S120 and S130 of the face area detection process (FIG. 16) in the second embodiment are the same as the processing contents of steps S120 and S130 of the face area detection process (FIG. 3) in the first embodiment. However, the calculation (step S350) of the cumulative evaluation value Tv in the original face detection process (FIG. 5) in step S130 is executed only for the specific face inclination that covers the set detection inclination range. As described above, the specific face inclination is a total of 12 face inclinations (0 degrees, 30 degrees, 60 degrees,..., 330 degrees) obtained by increasing the inclination from 0 degrees to 30 degrees. The face learning data FLD corresponding to the inclination (FIG. 1) is set so that a face image having a face inclination value in the range of plus or minus 15 degrees around the specific face inclination can be detected. For example, as shown in FIG. 18, the face learning data FLD corresponding to a specific face inclination of 0 degrees has a face inclination value in the range of plus or minus 15 degrees centering on 0 degrees (“R (0)” in FIG. 18). Can be detected. Similarly, the face learning data FLD corresponding to the specific face inclination of 30 degrees has the face inclination value in the range of plus or minus 15 degrees centered on 30 degrees (shown as “R (30)” in FIG. 18). The face learning data FLD that can detect an image and corresponds to a specific face inclination of 330 degrees has a face inclination value in the range of plus or minus 15 degrees centered on 330 degrees (shown as “R (330)” in FIG. 18). ) Face image can be detected. Therefore, as shown in FIG. 18, when the inclination ranges of 0 degrees to 20 degrees and 340 degrees to 360 degrees are set as the detected inclination ranges, the calculation of the cumulative evaluation value Tv covers the detected inclination range. It is executed only for three specific face inclinations of 0 degrees, 30 degrees, and 330 degrees that are specific face inclinations.

  In the face area detection process (FIG. 16) in the second embodiment, the original face area FAo is not detected in the original face area detection process (step S130) as in the face area detection process (FIG. 3) in the first embodiment. If it is detected (step S140: No), the face area detection process ends. On the other hand, when at least one original face area FAo is detected (step S140: Yes), the process proceeds to step S170. The processing content from step S170 to S210 is the same as the processing content in the face area detection processing (FIG. 3) in the first embodiment.

  In step S212 (FIG. 16), the detection determination unit 250 (FIG. 15) sets the actual inclination (actual inclination) (see FIG. 14A) of the face image estimated in step S210 in step S112. It is determined whether it is within the detected inclination range. When it is determined that the estimated actual inclination is within the detected inclination range (step S212: Yes), the detection determination unit 250 is the face image to be detected from the face image represented in the target image. Is determined. In this case, the information adding unit 230 (FIG. 15) adds information indicating the set position (coordinates) of the determined face area FAf as attached information to the image file including the original image data (step S214). On the other hand, when it is determined that the estimated actual inclination is not within the detected inclination range (step S212: No), the detection determination unit 250 detects the face image to be detected from the face image represented in the target image. It is determined that it is not. In this case, the process of step S214 is skipped.

  Thereafter, in step S170, it is determined whether or not there is an unselected original face area FAo (step S220). If it is determined that there is an unselected original face area FAo (step S220: No) The process returns to step S170, and if it is determined that all the original face areas FAo have been selected (step S220: Yes), the process ends.

  As described above, in the face area detection processing by the printer 100a of the second embodiment, the inclination of the face image to be detected from the target image is set, and the target image is based on the detected positional relationship between the plurality of organ areas. The inclination of the face image represented in (2) is estimated, and the face image represented in the target image is to be detected based on the inclination of the face image to be detected and the inclination of the estimated face image. It is determined whether or not. Therefore, in the face area detection process by the printer 100a of the second embodiment, it is possible to accurately determine whether or not the face image represented in the target image is a face image to be detected.

C. Variations:
The present invention is not limited to the above-described examples and embodiments, and can be implemented in various modes without departing from the gist thereof. For example, the following modifications are possible.

C1. Modification 1:
In each of the above embodiments, the accuracy means the degree of the average value of the difference between the inclination of the determined face area FAf (face area inclination) and the actual inclination (actual inclination) of the face image represented in the target image. However, the accuracy may also mean an average degree of difference between the position and size of the determined face area FAf and the actual position and size of the face image represented in the target image. In this case, when high accuracy is set as the accuracy, even if the position and size of the original face region FAo are adjusted based on the detection result of the organ region in the setting of the determined face region FAf (step S210 in FIG. 3). Good. The position of the original face area FAo is adjusted, for example, by the midpoint of the line segment connecting the center of gravity of the right eye area EA (r) and the center of gravity of the left eye area EA (l), and the horizontal and vertical directions of the determined face area FAf. This is done by translating the original face area FAo so that the points whose size is divided at a predetermined ratio coincide. The size of the original face area FAo is adjusted, for example, by determining whether the horizontal size of the determined face area FAf is a line segment connecting the center of gravity of the right eye area EA (r) and the center of gravity of the left eye area EA (l). The vertical size of the determined face area FAf is a predetermined multiple of the length, and connects the line segment connecting the center of gravity of the right eye area EA (r) and the center of gravity of the left eye area EA (l) and the center of gravity of the mouth area MA. This is performed by enlarging or reducing the original face area FAo so as to be a predetermined multiple of the length of the line segment.

C2. Modification 2:
The use of the face area (see FIGS. 4 and 17) in each of the above embodiments is merely an example, and various other uses of the face area can be set. In addition, the accuracy and the detection inclination range according to the use of the face area in each of the above embodiments are merely examples, and the accuracy and the detection inclination range can be variously modified according to the use of the face area.

  In the first embodiment, the accuracy may be set directly without setting the use of the face area. In the second embodiment, the detected inclination range may be directly set without setting the use of the face area.

  In the first embodiment, the selection of the processing mode, that is, the selection of whether or not the organ area detection process is executed for setting the determined face area FAf is directly performed without setting the accuracy. It is good.

C3. Modification 3:
In each of the embodiments described above, in the organ region detection process (step S180 in FIG. 3), in any organ detection image ODImg, organ regions (right eye region EA (r) and left eye region EA (l) corresponding to three types of organs). And at least one of the mouth areas MA), it is determined that the organ area is not detected. However, if at least one of the three kinds of organ areas is detected, the organ area is detected. It may be determined that Alternatively, regardless of the detection result of the mouth area MA, when both the right eye area EA (r) and the left eye area EA (l) used for estimating the face inclination are detected, it is determined that the organ area has been detected, When at least one of the right eye area EA (r) and the left eye area EA (l) is not detected, it may be determined that the organ area is not detected.

  If it is determined that an organ area has been detected when at least one of the three types of organ areas is detected, one of the right eye area EA (r) and the left eye area EA (l) or It is possible that both are not available. In such a case, the inclination of the organ detection image ODImg in which the organ area is detected may be estimated as the face inclination.

C4. Modification 4:
In each of the above embodiments, only the face learning data FLD and the facial organ learning data OLD corresponding to the front-facing image are prepared. Detection of a face area or an organ area corresponding to an image of the above or an image of a facial organ may be performed.

C5. Modification 5:
The aspects of the original face area detection process (FIG. 5) and the organ area detection process (FIG. 11) in each of the above embodiments are merely examples, and various changes can be made. For example, the size of the face detection image FDImg (see FIG. 6) is not limited to 320 pixels × 240 pixels, and may be other sizes, or the original image OImg itself may be used as the face detection image FDImg. . Further, the size of the window SW used, the moving direction and the moving amount (moving pitch) of the window SW are not limited to those described above. Further, in each of the embodiments described above, the size of the face detection image FDImg is fixed, and a plurality of types of sizes of window SW SW are arranged on the face detection image FDImg, thereby setting a determination target image area JIA having a plurality of sizes. However, a plurality of sizes of face detection images FDImg are generated, and a fixed-size window SW is arranged on the face detection image FDImg, so that a determination target image area JIA having a plurality of sizes is set. Also good.

  In each of the above embodiments, the face evaluation and the organ determination are performed by comparing the cumulative evaluation value Tv with the threshold value TH (see FIG. 7), but the face determination and the organ determination are performed using a plurality of discriminators. Other methods may be used. The learning method used for setting the face learning data FLD and the facial organ learning data OLD is also changed according to the face determination and organ determination methods. Further, the face determination and the organ determination are not necessarily performed by a determination method using learning, and may be performed by other methods such as pattern matching.

  In each of the above embodiments, 12 types of specific face inclinations in increments of 30 degrees are set. However, more types of specific face inclinations may be set, or fewer types of specific face inclinations may be set. May be. In addition, the specific face inclination does not necessarily need to be set, and face determination may be performed for a 0 degree face inclination. In each of the above embodiments, the face sample image group includes an image obtained by enlarging or reducing the basic face sample image or a rotated image. However, the face sample image group does not necessarily include such an image. Absent.

  In each of the above embodiments, the face determination (or organ determination) for the determination target image area JIA defined by the window SW of a certain size is determined to be an image area corresponding to the face image (or facial organ image). If the window SW having a size smaller than the size by a predetermined ratio or more is arranged, the window SW is arranged avoiding the determination target image area JIA determined to be the image area corresponding to the face image. It is good. In this way, the processing speed can be increased.

  In each of the above embodiments, the image data stored in the memory card MC is set as the original image data. However, the original image data is not limited to the image data stored in the memory card MC, and is acquired via a network, for example. It may be image data.

  In each of the above embodiments, the right eye, the left eye, and the mouth are set as the types of facial organs, and detection of the right eye area EA (r), the left eye area EA (l), and the mouth area MA is performed as the organ areas. However, it is possible to change which organ of the face is set as the type of facial organ. For example, only one or two of the right eye, the left eye, and the mouth may be set as the types of facial organs. In addition to the right eye, left eye, and mouth, or instead of at least one of the right eye, left eye, and mouth, other organ types of the face (for example, nose or eyebrows) are set as the facial organ types, An area corresponding to such an organ image may be detected as the organ area.

  In each of the above embodiments, the original face area FAo and the organ area are rectangular areas, but the original face area FAo and the organ area may be areas having a shape other than a rectangle.

  In each of the above embodiments, the face area detection processing by the printer 100 as the image processing apparatus has been described. However, part or all of the processing is performed by another type of image processing apparatus such as a personal computer, a digital still camera, or a digital video camera. It may be executed. The printer 100 is not limited to an ink jet printer, and may be another type of printer, such as a laser printer or a sublimation printer.

  In each of the above embodiments, a part of the configuration realized by hardware may be replaced with software, and conversely, a part of the configuration realized by software may be replaced by hardware. .

  In addition, when part or all of the functions of the present invention are realized by software, the software (computer program) can be provided in a form stored in a computer-readable recording medium. In the present invention, the “computer-readable recording medium” is not limited to a portable recording medium such as a flexible disk or a CD-ROM, but an internal storage device in a computer such as various RAMs and ROMs, a hard disk, and the like. An external storage device fixed to the computer is also included.

1 is an explanatory diagram schematically showing the configuration of a printer 100 as an image processing apparatus in a first embodiment of the present invention. FIG. It is explanatory drawing which shows the kind of face learning data FLD and face organ learning data OLD. It is a flowchart which shows the flow of the face area detection process in 1st Example. It is explanatory drawing which shows an example of the use of the face area determined. It is a flowchart which shows the flow of an original face area | region detection process. It is explanatory drawing which shows the outline | summary of an original face area | region detection process. It is explanatory drawing which shows the outline | summary of the calculation method of accumulated evaluation value Tv used for face determination. It is explanatory drawing which shows an example of the sample image used for learning. It is explanatory drawing which shows the outline | summary of an original face area | region setting process. It is explanatory drawing which shows the outline | summary of an original face area | region setting process. It is a flowchart which shows the flow of an organ area | region detection process. It is explanatory drawing which shows the outline | summary of an organ area | region detection process. It is explanatory drawing which shows the outline | summary of an organ area | region setting process. It is explanatory drawing which shows the setting method of the determination face area | region FAf. It is explanatory drawing which shows roughly the structure of the printer 100a as an image processing apparatus in 2nd Example of this invention. It is a flowchart which shows the flow of the face area detection process in 2nd Example. It is explanatory drawing which shows an example of the use of the face area determined. It is explanatory drawing which shows an example of the detection inclination range.

Explanation of symbols

100 ... Printer 110 ... CPU
DESCRIPTION OF SYMBOLS 120 ... Internal memory 140 ... Operation part 150 ... Display part 160 ... Printer engine 170 ... Card interface 172 ... Card slot 200 ... Image processing part 210 ... Area | region detection part 211 ... Determination object setting part 212 ... Evaluation value calculation part 213 ... Determination part 214 ... Area setting unit 216 ... Face region determining unit 217 ... Face inclination estimating unit 220 ... Accuracy setting unit 230 ... Information adding unit 240 ... Detection inclination setting unit 250 ... Detection determination unit 310 ... Display processing unit 320 ... Print processing unit

Claims (9)

An image processing apparatus,
A face area detection unit that detects an area corresponding to a face image in the target image based on image data as an original face area;
An organ region detection unit that detects a region corresponding to an image of a facial organ in the original face region as an organ region;
A face area determination unit that sets a determined face area corresponding to a face image in the target image, the first processing mode for setting the original face area as the determined face area, and the detection result of the organ area An image processing apparatus comprising: a face area determination unit including: a second processing mode that sets an area obtained by adjusting the original face area as the determined face area. The image processing apparatus according to claim 1, further comprising:
An accuracy setting unit for setting the accuracy when setting the determined face area;
The face area determination unit sets the determined face area in the first processing mode when the accuracy is within a predetermined range, and the second processing mode when the accuracy is higher than the predetermined range. An image processing apparatus for setting the determined face area by using The image processing apparatus according to claim 1 or 2,
The accuracy setting unit is an image processing apparatus that acquires usage information specifying a usage of the determined face area, and sets the accuracy based on the usage information. An image processing apparatus according to any one of claims 1 to 3,
In the second mode, the face area determination unit sets the determined face area by adjusting an inclination of the original face area based on a positional relationship between the plurality of detected organ areas. . The image processing apparatus according to claim 1, further comprising:
A detection inclination setting unit for setting the inclination of the face image to be detected from the target image;
A face inclination estimation unit that estimates an inclination of an image of a face represented in the target image based on a positional relationship between the plurality of organ regions detected;
Detection determination for determining whether the face image represented in the target image is a face image to be detected based on the inclination of the face image to be detected and the estimated inclination of the face image An image processing apparatus. An image processing apparatus according to any one of claims 1 to 5,
The face area detection unit
A determination target setting unit that sets a determination target image region that is an image region on the target image;
A memory for storing evaluation data for calculating an evaluation value indicating the probability that the determination target image area is an image area corresponding to a face image having one of a plurality of predetermined discrete slopes And
An evaluation value calculation unit that calculates the evaluation value based on the evaluation data and image data corresponding to the determination target image region;
A determination unit that determines whether or not the determination target image region is an image region corresponding to a face image based on the evaluation value;
And an area setting unit that sets the original face area based on the position and size of the determination target image area determined to be an image area corresponding to a face image. An image processing apparatus according to any one of claims 1 to 6,
The type of facial organ is at least one of a right eye, a left eye, and a mouth. An image processing method comprising:
(A) detecting an area corresponding to the face image in the target image based on the image data as an original face area;
(B) detecting an area corresponding to an image of a facial organ in the original face area as an organ area;
(C) a step of setting a determined face area corresponding to a face image in the target image, the first processing mode in which the original face area is set as the determined face area, and the detection result of the organ area And a second processing mode for setting, as the determined face area, an area obtained by adjusting the original face area based on the image processing method. A computer program for image processing,
A face area detection function for detecting, as an original face area, an area corresponding to a face image in a target image based on image data;
An organ area detection function for detecting an area corresponding to an image of a facial organ in the original face area as an organ area;
A face area determination function for setting a determined face area corresponding to a face image in the target image, the first processing mode for setting the original face area as the determined face area, and the detection result of the organ area A computer program that causes a computer to realize a face area determination function having a second processing mode in which an area obtained by adjusting the original face area is set as the determined face area.

Images