JP2009237613A - Detection of face area corresponding to face image from target image - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To improve detection accuracy related to a position and an inclination of a face area while speeding up detection process of the face area. <P>SOLUTION: An image processing apparatus includes: a face area detecting unit for detecting a face area corresponding to an image of a face in a target image represented by target image data; an organ area detecting unit for detecting an organ area corresponding to a facial organ image in the detected face area; and a face inclination estimating unit for estimating an inclination in a face image plane indicated by the face area on the basis of the detection result of the organ area. <P>COPYRIGHT: (C)2010,JPO&INPIT

Description

  The present invention relates to detection of a face area corresponding to a face image from a target image.

  A technique for detecting a face area corresponding to a face image from an image by sequentially cutting out partial images from an image represented by image data and determining whether or not the cut out partial image is an image corresponding to a face. Known (for example, Patent Documents 1 to 4).

JP 2003-256834 A JP 2003-281539 A JP 2006-065640 A JP 2005-025748 A

  In detecting a face area from a target image, it has been desired to improve the detection accuracy regarding the position and inclination of the face area while increasing the processing speed.

  The present invention has been made to solve the above-described problems, and provides a technique capable of improving the detection accuracy related to the position and inclination of the face area while increasing the speed of the face area detection process. For the purpose.

  In order to solve at least a part of the above problems, the present invention can be realized as the following forms or application examples.

Application Example 1 An image processing apparatus,
A face area detection unit that detects a face area corresponding to a face image in the target image represented by the target image data;
An organ region detector for detecting an organ region corresponding to an image of a facial organ in the detected face region;
An image processing apparatus comprising: a face inclination estimation unit that estimates an inclination in an image plane of a face represented in the face area based on a detection result of the organ area.

  In this image processing apparatus, a face area corresponding to the face image in the target image represented by the target image data is detected, and an organ area corresponding to the facial organ image in the detected face area is detected, Since the inclination in the image plane of the face represented in the face area is estimated based on the detection result of the organ area, the detection accuracy related to the position and inclination of the face area is improved while speeding up the face area detection process. Can do.

[Application Example 2] The image processing apparatus according to Application Example 1,
The face area detection unit
A determination target setting unit that sets a determination target image region that is an image region on the target image;
A storage unit for storing evaluation data for calculating an evaluation value indicating the certainty that the determination target image region is an image region corresponding to a face image;
An evaluation value calculation unit that calculates the evaluation value based on the evaluation data and image data corresponding to the determination target image region;
A determination unit that determines whether or not the determination target image region is an image region corresponding to a face image based on the evaluation value;
And an area setting unit that sets the face area based on the position and size of the determination target image area determined to be an image area corresponding to a face image.

  In this image processing apparatus, a determination target image region that is an image region on the target image is set, and an evaluation value that represents the probability that the determination target image region is an image region corresponding to a face image is calculated. Evaluation data is stored, an evaluation value is calculated based on the evaluation data and image data corresponding to the determination target image area, and whether or not the determination target image area is an image area corresponding to the face image Speeding up the face area detection process performed by setting the face area based on the position and size of the determination target image area determined to be an image area corresponding to the face image, The detection accuracy regarding the position and inclination of the face area can be improved.

[Application Example 3] The image processing apparatus according to Application Example 2,
The evaluation data is generated by learning using a face image group including a plurality of face images having arbitrary different inclinations in an image plane.

  With this image processing apparatus, in detecting a face area, it is possible to detect face images having arbitrary different inclinations in the image plane, and it is possible to increase the speed of the face area detection process and to improve the face area position and inclination. Detection accuracy can be improved.

[Application Example 4] The image processing apparatus according to Application Example 3,
The face image group includes an image processing apparatus including a plurality of face images whose inclinations in an image plane differ from each other by 30 degrees or more.

  In this image processing apparatus, in detecting a face area, it is possible to detect face images whose inclinations in the image plane are different from each other by 30 degrees or more, and it is possible to increase the speed of the face area detection process and relate to the position and inclination of the face area. Detection accuracy can be improved.

[Application Example 5] The image processing apparatus according to Application Example 3 or Application Example 4,
The image processing apparatus represents the probability that the evaluation target image area is an image area corresponding to a face image having the same inclination as that of a face image included in the face image group.

  In this image processing device, in the detection of the face area, it is possible to detect face images having arbitrary different inclinations in the image plane using the evaluation value, and the face area detection process can be performed at high speed. The detection accuracy related to the position and inclination of can be improved.

Application Example 6 The image processing apparatus according to any one of Application Example 1 to Application Example 5,
The organ region detection unit detects the organ region corresponding to at least two types of organs,
The image processing apparatus, wherein the face inclination estimation unit estimates the inclination based on a positional relationship between the organ regions corresponding to the at least two types of organs.

  In this image processing apparatus, it is possible to improve the detection accuracy regarding the inclination of the face region by estimating the inclination based on the positional relationship between the organ regions corresponding to at least two types of organs.

Application Example 7 The image processing apparatus according to Application Example 6, further comprising:
Based on the detection result of the organ region, a reliability calculation unit that calculates an organ region reliability that represents the probability that the detected organ region is an image region that truly corresponds to an image of a facial organ,
The face inclination estimation unit selects the organ area to be used for the inclination estimation based on the organ area reliability when a plurality of the organ areas corresponding to a specific type of organ are detected. Processing equipment.

  In this image processing apparatus, the detection accuracy related to the inclination of the face area can be improved by selecting the organ area used for the estimation of the inclination based on the organ area reliability.

[Application Example 8] The image processing apparatus according to Application Example 7,
The organ region detection unit is
A determination target setting unit that sets a determination target image region that is an image region on the face region;
A storage unit for storing evaluation data for calculating an evaluation value representing the probability that the determination target image area is an image area corresponding to an image of a facial organ;
An evaluation value calculation unit that calculates the evaluation value based on the evaluation data and image data corresponding to the determination target image region;
A determination unit that determines whether the determination target image region is an image region corresponding to an image of a facial organ based on the evaluation value;
An area setting unit configured to set the organ area based on the position and size of the determination target image area partially overlapping each other as one or a plurality of image areas determined to be image areas corresponding to facial organ images; Including
The image processing apparatus, wherein the reliability calculation unit calculates the organ area reliability based on at least one of the number of the determination target image areas and the evaluation value referred to when the organ area is set.

  In this image processing apparatus, it is possible to accurately calculate the organ region reliability indicating the certainty that the detected organ region is an image region that truly corresponds to the facial organ image.

[Application Example 9] The image processing apparatus according to any one of Application Examples 1 to 8, further comprising:
An image processing apparatus comprising: a face area resetting unit that resets the face area based on the estimated inclination.

  In this image processing apparatus, it is possible to reset a face area with improved accuracy regarding position and inclination.

Application Example 10 The image processing apparatus according to any one of Application Example 1 to Application Example 9,
The type of facial organ is at least one of a right eye, a left eye, and a mouth.

  Note that the present invention can be realized in various modes. For example, the image processing method and apparatus, the face area detection method and apparatus, the face inclination estimation method and apparatus, and the functions of these methods or apparatuses are realized. For example, a recording medium on which the computer program is recorded, a data signal that includes the computer program and is embodied in a carrier wave, and the like.

Next, embodiments of the present invention will be described in the following order based on examples.
A. Example:
A-1. Configuration of image processing device:
A-2. Face area detection processing:
B. Variations:

A. Example:
A-1. Configuration of image processing device:
FIG. 1 is an explanatory diagram schematically showing the configuration of a printer 100 as an image processing apparatus according to an embodiment of the present invention. The printer 100 of this embodiment is an ink jet color printer that supports so-called direct printing, in which an image is printed based on image data acquired from a memory card MC or the like. The printer 100 includes a CPU 110 that controls each unit of the printer 100, an internal memory 120 configured by a ROM and a RAM, an operation unit 140 configured by buttons and a touch panel, a display unit 150 configured by a liquid crystal display, and a printer. An engine 160 and a card interface (card I / F) 170 are provided. The printer 100 may further include an interface for performing data communication with other devices (for example, a digital still camera or a personal computer). Each component of the printer 100 is connected to each other via a bus.

  The printer engine 160 is a printing mechanism that performs printing based on print data. The card interface 170 is an interface for exchanging data with the memory card MC inserted into the card slot 172. In this embodiment, an image file including image data is stored in the memory card MC.

  The internal memory 120 stores an image processing unit 200, a display processing unit 310, and a print processing unit 320. The image processing unit 200 is a computer program for executing face area detection processing described later under a predetermined operating system. The display processing unit 310 is a display driver that controls the display unit 150 to display processing menus, messages, images, and the like on the display unit 150. The print processing unit 320 is a computer program for generating print data from image data, controlling the printer engine 160, and printing an image based on the print data. The CPU 110 implements the functions of these units by reading and executing these programs from the internal memory 120.

  The image processing unit 200 includes an area detection unit 210, a reliability calculation unit 220, and an information addition unit 230 as program modules. The region detection unit 210 detects an image region corresponding to a predetermined type of subject image (a face image and a face organ image) in the target image represented by the target image data. The region detection unit 210 includes a determination target setting unit 211, an evaluation value calculation unit 212, a determination unit 213, a region setting unit 214, a face inclination estimation unit 215, and a face region determination unit 216. The functions of these units will be described in detail in the description of the face area detection process described later. As will be described later, the area detection unit 210 detects the face area corresponding to the face image and the organ area corresponding to the face organ image. Functions as a detection unit.

  The reliability calculation unit 220 includes image regions (face regions and organ regions) detected by the region detection unit 210 as image regions corresponding to predetermined types of subject images (face images and facial organ images). A reliability level is calculated that represents the probability that the image area truly corresponds to the image of the subject of the predetermined type. The information adding unit 230 adds predetermined information (for example, information indicating the positions of the face area and the organ area) to the image file including the image data.

  The internal memory 120 also stores preset face learning data FLD and face organ learning data OLD. The face learning data FLD and the facial organ learning data OLD are used for detection of a predetermined image area by the area detection unit 210. FIG. 2 is an explanatory diagram showing face learning data FLD and face organ learning data OLD. 2A to 2C show types of face learning data FLD and facial organ learning data OLD, and image regions detected using the types of face learning data FLD and face organ learning data OLD. An example is shown.

  The contents of the face learning data FLD will be described in detail in the description of the face area detection process described later, but the face learning data FLD of this embodiment corresponds to a face inclination from 0 degrees to 360 degrees. Here, the face inclination means the inclination (rotation angle) of the face image in the image plane (in-plane). In this embodiment, the face inclination is defined as a state in which the face image is positioned along the vertical direction of the image (a state in which the top of the head is facing upward and the chin is facing downward). In degrees), the angle of the face image in the clockwise direction. Also, the face learning data FLD corresponding to a certain face inclination is face learning data FLD that can detect a face image having the face inclination in the detection of an image region using the face learning data FLD. Therefore, the face learning data FLD of the present embodiment is face learning data FLD that can detect face inclinations from 0 degrees to 360 degrees, that is, face images having all face inclinations (see FIG. 2A). The face learning data FLD is set by learning to be described later so as to correspond to a face inclination from 0 degrees to 360 degrees.

  The facial organ learning data OLD is set for each type of facial organ. In this embodiment, eyes (right eye and left eye) and mouth are set as types of facial organs. The internal memory 120 stores facial organ learning data OLD corresponding to the eyes shown in FIG. 2B and facial organ learning data OLD corresponding to the mouth shown in FIG.

  The facial organ learning data OLD corresponding to the eyes and mouth corresponds to an organ inclination of 0 degrees. Here, the organ inclination means the inclination (rotation angle) of the image of the facial organ in the image plane (in-plane), similar to the face inclination described above. Similar to the face tilt, the organ tilt is the clock of the facial organ image when the facial organ image is positioned along the vertical direction of the image as the reference organ tilt (organ tilt = 0 degrees). Expressed by the angle around. The facial organ learning data OLD of the present embodiment is an image of a facial organ with an inclination of 0 degrees in the detection of an image region using the facial organ learning data OLD (see FIGS. 2B and 2C). Can be detected face organ learning data OLD. As will be described later, the facial organ learning data OLD of the present embodiment includes organ images with an organ tilt value of plus or minus 15 degrees centering on 0 degrees, in addition to an image of an organ having an organ tilt of 0 degrees. It is set by learning so that an image can also be detected.

  In this embodiment, it is assumed that the right eye and the left eye are the same type of subject, and the right eye region corresponding to the right eye image and the left eye region corresponding to the left eye image are detected using the common facial organ learning data OLD. However, assuming that the right eye and the left eye are different types of subjects, dedicated face organ learning data OLD may be prepared for right eye region detection and left eye region detection, respectively.

A-2. Face area detection processing:
FIG. 3 is a flowchart showing the flow of face area detection processing. The face area detection process in this embodiment is a process for detecting a face area corresponding to a face image in an image represented by image data.

  In step S110, the image processing unit 200 (FIG. 1) acquires image data representing an image to be subjected to face area detection processing. In the printer 100 of this embodiment, when the memory card MC is inserted into the card slot 172, thumbnail images of the image files stored in the memory card MC are displayed on the display unit 150. The user selects one or more images to be processed via the operation unit 140 while referring to the displayed thumbnail images. The image processing unit 200 acquires an image file including image data corresponding to one or more selected images from the memory card MC and stores it in a predetermined area of the internal memory 120. The acquired image data is referred to as original image data, and the image represented by the original image data is referred to as original image OImg.

  In step S120 (FIG. 3), the area detection unit 210 (FIG. 1) performs an original face area detection process. The original face area detection process is a process for detecting an image area corresponding to the face image as the original face area FAo. The original face area FAo is an area corresponding to a face image having an arbitrary inclination in the image plane. FIG. 4 is a flowchart showing the flow of the original face area detection process. FIG. 5 is an explanatory diagram showing an outline of the original face area detection process. An example of the original image OImg is shown at the top of FIG.

  In step S310 in the original face area detection process (FIG. 4), the area detection unit 210 (FIG. 1) generates face detection image data representing the face detection image FDImg from the original image data representing the original image OImg. In the present embodiment, as shown in FIG. 5, the face detection image FDImg is an image having a size of horizontal 320 pixels × vertical 240 pixels. The area detection unit 210 generates face detection image data representing the face detection image FDImg by performing resolution conversion of the original image data as necessary. Note that the face detection image FDImg in this embodiment corresponds to the target image in the present invention, and the face detection image data corresponds to the target image data in the present invention.

  In step S320 (FIG. 4), the determination target setting unit 211 (FIG. 1) sets the size of the window SW used for setting the determination target image area JIA (described later) to an initial value. In step S330, the determination target setting unit 211 places the window SW at an initial position on the face detection image FDImg. In step S340, the determination target setting unit 211 determines whether the image area defined by the window SW arranged on the face detection image FDImg is an image area corresponding to the face image (hereinafter referred to as “face”). It is also set in a determination target image area JIA that is a target of determination. The middle part of FIG. 5 shows a state in which a window SW having an initial value size is arranged at an initial position on the face detection image FDImg, and an image area defined by the window SW is set as the determination target image area JIA. Yes. In this embodiment, as will be described later, the determination target image area JIA is sequentially set while changing the size and position of the square-shaped window SW, but the initial value of the size of the window SW is the horizontal size 240. The initial position of the window SW is such that the upper left vertex of the window SW overlaps the upper left vertex of the face detection image FDImg.

  In step S350 (FIG. 4), the evaluation value calculation unit 212 (FIG. 1) calculates a cumulative evaluation value Tv used for face determination for the determination target image area JIA. FIG. 6 is an explanatory diagram showing an outline of a method of calculating the cumulative evaluation value Tv used for face determination. In this embodiment, N filters (filter 1 to filter N) are used to calculate the cumulative evaluation value Tv. The external shape of each filter has the same aspect ratio as that of the window SW (that is, has a square shape), and a positive region pa and a negative region ma are set for each filter. The evaluation value calculation unit 212 calculates the evaluation values vX (that is, v1 to vN) by sequentially applying the filter X (X = 1, 2,..., N) to the determination target image area JIA. Specifically, the evaluation value vX is determined from the sum of the luminance values of pixels located in the area on the determination target image area JIA corresponding to the plus area pa of the filter X, based on the determination target image area JIA corresponding to the minus area ma. This is a value obtained by subtracting the sum of the luminance values of the pixels located in the upper region.

  The calculated evaluation value vX is compared with a threshold thX (that is, th1 to thN) set corresponding to each evaluation value vX. In this embodiment, when the evaluation value vX is equal to or greater than the threshold thX, it is determined that the determination target image area JIA is an image area corresponding to the face image for the filter X, and the value “ 1 "is set. On the other hand, if the evaluation value vX is smaller than the threshold thX, it is determined that the determination target image area JIA is not an image area corresponding to the face image with respect to the filter X, and the value “0” is set as the output value of the filter X Is done. Weight coefficients WeX (that is, We1 to WeN) are set for each filter X, and the sum of products of output values and weight coefficients WeX for all filters is calculated as a cumulative evaluation value Tv.

  Note that the aspect of the filter X used for face determination, the threshold thX, the weighting coefficient WeX, and the threshold TH described later are set in advance as face learning data FLD (see FIG. 2A). Since the face learning data FLD in the present embodiment is data for calculating an evaluation value indicating the certainty that the determination target image area JIA is an image area corresponding to the face image, the evaluation data in the present invention. It corresponds to.

  The face learning data FLD is generated by learning using a sample image. FIG. 7 is an explanatory diagram illustrating an example of a sample image used for learning. For learning, a face sample image group composed of a plurality of face sample images that are known in advance to be images corresponding to faces, and a plurality of non-face samples that are known in advance to be images that do not correspond to faces And a non-face sample image group constituted by images.

  As shown in FIG. 7, the face sample image group includes a plurality of face sample images having arbitrary different inclinations in the image plane. Specifically, the face sample image group is composed of a plurality of face sample sub-image groups corresponding to a specific face inclination. Here, the specific face inclination is a total of 12 face inclinations (0 degrees and 30 degrees) including a reference face inclination (face inclination = 0 degrees) and a face inclination obtained by sequentially increasing the face inclination from the reference face inclination by 30 degrees. , 60 degrees,..., 330 degrees).

  The face sample sub-image group corresponding to each specific face inclination includes a plurality of face samples in which the ratio of the face image size to the image size is within a predetermined value range and the face image inclination is equal to the specific face inclination. Images (hereinafter also referred to as “basic face sample images”). Each face sample sub-image group is an image obtained by enlarging and reducing the basic face sample image at a predetermined magnification in a range from 1.2 times to 0.8 times with respect to at least one basic face sample image (for example, FIG. 7). Images FIa and FIb) and images obtained by changing the face inclination of the basic face sample image in a range of plus or minus 15 degrees (for example, images FIc and FId in FIG. 7). Therefore, the face sample image group includes a face image having a specific face inclination and a face image having an arbitrary face inclination within a range between two adjacent specific face inclinations instead of the specific face inclination. Become.

  Learning using a sample image is executed by, for example, a method using a neural network, a method using boosting (for example, adaboost), a method using a support vector machine, or the like. For example, when learning is performed by a method using a neural network, a face sample image group (see FIG. 7) and a non-face sample image group for each filter X (that is, filter 1 to filter N, see FIG. 6) Evaluation values vX (that is, v1 to vN) are calculated using all the sample images included in the threshold value thX (that is, th1 to thN) that achieves a predetermined face detection rate. Here, the face detection rate is the ratio of the number of face sample images determined to be an image corresponding to a face image by threshold determination based on the evaluation value vX with respect to the total number of face sample images constituting the face sample image group. Means.

  Next, the weight coefficient WeX (that is, We1 to WeN) set for each filter X is set to an initial value, and cumulative evaluation is performed on one sample image selected from the face sample image group and the non-face sample image group. A value Tv is calculated. As will be described later, in the face determination, when the cumulative evaluation value Tv calculated for a certain image is equal to or greater than a predetermined threshold TH, the image is determined to be an image corresponding to the face image. In learning, the value of the weighting coefficient WeX set for each filter X is corrected based on the correctness of the threshold determination result based on the cumulative evaluation value Tv calculated for the selected sample image (face sample image or non-face sample image). Is done. Thereafter, the selection of the sample image, the threshold value determination based on the cumulative evaluation value Tv calculated for the selected sample image, and the correction of the value of the weighting coefficient WeX based on the correctness of the determination result are the face sample image group and the non-face sample image. It is repeatedly executed for all the sample images included in the group. Face learning data FLD is set by such processing.

  As described above, the face learning data FLD includes a face image having a specific face inclination and a face image having an arbitrary face inclination within a range between the specific face inclinations instead of the specific face inclination. Since it is set by learning using the sample image group, the face learning data FLD can be detected from 0 degrees to 360 degrees in the face area detection process, that is, all face tilted face images can be detected. The cumulative evaluation value Tv represents the probability that the determination target image area JIA is an image area corresponding to a face image having the same face inclination as that of the face sample image included in the face sample image group. It is an indicator.

  When the cumulative evaluation value Tv is calculated for the determination target image area JIA (step S350 in FIG. 4), the determination unit 213 (FIG. 1) compares the cumulative evaluation value Tv with a preset threshold value TH (step S360). . When the cumulative evaluation value Tv is equal to or greater than the threshold value TH, the area detection unit 210 determines that the determination target image area JIA is an image area corresponding to the face image, that is, the position of the determination target image area JIA, that is, currently set. The coordinates of the current window SW are stored (step S370). On the other hand, when the cumulative evaluation value Tv is smaller than the threshold value TH, the process of step S370 is skipped.

  In step S380 (FIG. 4), the area detection unit 210 (FIG. 1) determines whether or not the entire face detection image FDImg has been scanned by the window SW having the currently set size. If it is determined that the entire face detection image FDImg has not been scanned yet, the determination target setting unit 211 (FIG. 1) moves the window SW in a predetermined direction by a predetermined movement amount (step S390). The lower part of FIG. 5 shows how the window SW has moved. In this embodiment, in step S390, the window SW is moved to the right by a movement amount corresponding to 20% of the horizontal size of the window SW. If the window SW is arranged at a position where it cannot move further to the right, the window SW returns to the left end of the face detection image FDImg in step S390, and the window SW has a size of 2 in the vertical direction. It is assumed that it moves downwards by the amount of movement of the percentage. When the window SW is arranged at a position where it cannot move further downward, the entire face detection image FDImg is scanned. After the movement of the window SW (step S390), the processes after the above-described step S340 are executed for the moved window SW.

  If it is determined in step S380 (FIG. 4) that the entire face detection image FDImg has been scanned by the window SW having the currently set size, it is determined whether or not all the predetermined sizes of the window SW have been used. (Step S400). In this embodiment, as the size of the window SW, in addition to the initial value (maximum size) of 240 horizontal pixels × vertical 240 pixels, horizontal 213 pixels × vertical 213 pixels, horizontal 178 pixels × vertical 178 pixels, horizontal 149 pixels × 149 pixels vertically, 124 pixels horizontally × 124 pixels vertically, 103 pixels horizontally × 103 pixels vertically, 86 pixels wide × 86 pixels high, 72 pixels wide × 72 pixels high, 60 pixels wide × 60 pixels high, 50 pixels wide × 50 vertical Total of 15 pixels: 41 pixels wide x 41 pixels wide, 35 pixels wide x 35 pixels wide, 29 pixels wide x 29 pixels wide, 24 pixels wide x 24 pixels high, 20 pixels wide x 20 pixels high (minimum size) The size of is set. If it is determined that there is a size of the window SW that is not yet used, the determination target setting unit 211 (FIG. 1) changes the size of the window SW to the next smaller size than the currently set size ( Step S410). That is, the size of the window SW is first set to the maximum size, and then changed to a smaller size in order. After the change of the size of the window SW (step S410), the processing after step S330 described above is executed for the window SW having the changed size.

  If it is determined in step S400 (FIG. 4) that all of the predetermined size of the window SW has been used, the area setting unit 214 (FIG. 1) executes an original face area setting process (step S420). FIG. 8 is an explanatory diagram showing an outline of the original face area setting process. The area setting unit 214 determines that the cumulative evaluation value Tv is equal to or greater than the threshold value TH in step S360 of FIG. 4, and sets the original face area FAo based on the coordinates of the window SW stored in step S370. Specifically, when the window SW stored in step S370 does not overlap with a part of other stored windows SW, the image area defined by the window SW (that is, the determination target image area JIA (see FIG. 5). )) Is set as the original face area FAo as it is.

  On the other hand, when a plurality of windows SW that partially overlap each other are stored in step S370 (FIG. 4), the area setting unit 214 (FIG. 1) determines a predetermined point (for example, the window SW of each window SW). One new window (hereinafter also referred to as “average window AW”) having an average size of the sizes of the windows SW is set with the average coordinate of the coordinates of the center of gravity) being defined by the average window AW. The image area is set as the original face area FAo. For example, as shown in FIG. 8A, when four windows SW (SW1 to SW4) that partially overlap each other are stored, as shown in FIG. 8B, each of the four windows SW is stored. An average window AW having an average size of the sizes of the four windows SW is defined with the average coordinate of the coordinates of the center of gravity of each of the four windows SW being defined, and an image area defined by the average window AW is defined as an original face area FAo Is set.

  Even when one window SW that does not overlap with other windows SW is stored, one window SW itself is the average window AW, as in the case where a plurality of windows SW that partially overlap each other are stored. It can also be interpreted as being.

  In the present embodiment, an image obtained by enlarging and reducing a basic face sample image at a predetermined magnification in a range from 1.2 times to 0.8 times in a face sample image group (see FIG. 7) used for learning (see FIG. 7). For example, since the images FIa and FIb in FIG. 7 are included, the original face region is also obtained when the size of the face image relative to the size of the window SW is slightly larger or smaller than the basic face sample image. FAo can be detected. Therefore, in the present embodiment, only the 15 discrete sizes described above are set as the size of the window SW, but the original face area FAo can be detected for facial images of any size.

  If the original face area FAo is not detected in the original face area detection process (step S120 in FIG. 3) (step S130: No), the face area detection process ends. On the other hand, when at least one original face area FAo is detected (step S130: Yes), the area detection unit 210 (FIG. 1) selects one of the detected original face areas FAo (step S140). .

  In step S150, the region detection unit 210 performs an organ region detection process. The organ area detection process is a process of detecting an image area corresponding to the facial organ image in the selected original face area FAo as an organ area. As described above, in this embodiment, the types of facial organs mean the right eye, the left eye, and the mouth, and in the organ area detection process, the right eye area EA (r) corresponding to the right eye image. The left eye area EA (l) corresponding to the left eye image and the mouth area MA corresponding to the mouth image are detected.

  FIG. 9 is a flowchart showing the flow of the organ region detection process. In step S510 of the organ region detection process, the region detection unit 210 (FIG. 1) generates organ detection image data representing the organ detection image ODImg. FIG. 10 is an explanatory diagram showing an example of the organ detection image ODImg. FIG. 10A shows the original face area FAo defined by the average window AW set on the face detection image FDImg. As shown in FIG. 10 (b), the organ detection image ODImg corresponds to 12 angles at a 30 degree pitch from 0 degree to 330 degrees centering on the center of gravity of the average window AW on the face detection image FDImg. An image obtained by rotating twelve image areas defined by each average window AW after rotating the average window AW clockwise (changing the inclination of the average window AW) counterclockwise by the angle. is there. FIG. 10B shows the rotation angle of the average window AW and an example of the organ detection image ODImg corresponding to the rotation angle. In this embodiment, the organ detection image ODImg is an image having a size of horizontal 60 pixels × vertical 60 pixels. Therefore, the region detection unit 210 generates 12 organ detection images ODImg by performing trimming, rotation, and resolution conversion of the face detection image FDImg as necessary.

  In step S520 (FIG. 9), the region detection unit 210 (FIG. 1) selects one of the generated organ detection images ODImg. After the selection of the organ detection image ODImg, the detection of the organ area from the selected organ detection image ODImg is performed in the same manner as the detection of the original face area FAo from the face detection image FDImg described above. FIG. 11 is an explanatory diagram showing an outline of the organ region detection process. An example of the selected organ detection image ODImg is shown in the uppermost part of FIG.

  In the detection of the organ region from the selected organ detection image ODImg, as shown in FIG. 11, the rectangular window SW is arranged on the organ detection image ODImg while its size and position are changed (FIG. 9). Steps S530, S540, S590 to S620) for determining whether the image area defined by the arranged window SW is an organ area corresponding to the facial organ image (hereinafter also referred to as “organ determination”). The determination target image area JIA is set (step S550 in FIG. 9). For the set determination target image area JIA, a cumulative evaluation value Tv used for organ determination is calculated for each organ (right eye, left eye, mouth) using the facial organ learning data OLD (FIG. 1) (FIG. 9). Step S560). The aspect of the filter X used for calculation of the cumulative evaluation value Tv and organ determination, the threshold thX, the weighting coefficient WeX, and the threshold TH (see FIG. 6) are defined in the facial organ learning data OLD.

  Note that the learning for setting the facial organ learning data OLD is performed by using a plurality of organ sample images that are known in advance to be images corresponding to the facial organs, similarly to the learning for setting the facial learning data FLD. It is executed using the constructed organ sample image group and the non-organ sample image group composed of a plurality of non-organ sample images that are known in advance not to correspond to facial organs. However, the face sample image group (FIG. 7) used for learning for setting the face learning data FLD corresponds to each of specific face inclinations (0 degrees, 30 degrees, 60 degrees,..., 330 degrees). In contrast to the plurality of face sample sub-image groups, the organ sample image group used for learning for setting the facial organ learning data OLD is not so configured. The organ sample image group includes a plurality of organ sample images (hereinafter referred to as “the organ image of face image”) in which the ratio of the size of the facial organ image to the image size is within a predetermined value range and the inclination of the facial organ image is 0 degrees. A basic organ sample image), an image obtained by enlarging and reducing the basic organ sample image at a predetermined magnification in a range from 1.2 times to 0.8 times with respect to at least one basic organ sample image; And an image in which the inclination of the sample image is changed in a range of plus or minus 15 degrees. Therefore, the facial organ learning data OLD is set so that an organ image having an organ inclination value in the range of plus or minus 15 degrees around 0 degrees can be detected in the organ region detection process. Note that the facial organ learning data OLD in the present embodiment is data for calculating an evaluation value representing the certainty that the determination target image region JIA is an image region corresponding to the facial organ image. This corresponds to the evaluation data in the invention.

  If the calculated cumulative evaluation value Tv is equal to or greater than the predetermined threshold TH, the determination target image area JIA is regarded as an image area corresponding to the facial organ image, that is, the position of the determination target image area JIA, that is, the current setting. The coordinates of the window SW being stored are stored (step S580 in FIG. 9). On the other hand, when the cumulative evaluation value Tv is smaller than the threshold value TH, the process of step S580 is skipped.

  For all the predetermined sizes of the window SW, the entire organ detection image ODImg is scanned by the window SW, and then an organ region setting process is executed (step S630 in FIG. 9). FIG. 12 is an explanatory diagram showing an outline of the organ region setting process. The organ area setting process is the same as the face area setting process (see FIG. 8). The region setting unit 214 sets an organ region based on the coordinates of the window SW determined in step S570 in FIG. 9 that the cumulative evaluation value Tv is greater than or equal to the threshold value TH and stored in step S580. Specifically, when the window SW stored in step S580 does not overlap with a part of other stored windows SW, the image area defined by the window SW (that is, the determination target image area JIA) remains as it is. Set as organ area. On the other hand, when a plurality of windows SW partially overlapping each other are stored, the average coordinates of the coordinates of a predetermined point (for example, the center of gravity of the window SW) in each window SW are used as the center of gravity, and the size of each window SW is determined. One new window (average window AW) having an average size is set, and an image area defined by the average window AW is set as an organ area. For example, as shown in FIG. 12A, when three windows SW (SW1 to SW3) that partially overlap each other are stored, as shown in FIG. An average window AW having an average size of the sizes of the three windows SW is defined with an average coordinate of the coordinates of the center of gravity of each of the three windows SW, and an image region defined by the average window AW is an organ region (right eye region) EA (r)).

  In step S640 (FIG. 9), region detection unit 210 (FIG. 1) determines whether all organ detection images ODImg have been selected. If it is determined that there is an organ detection image ODImg that has not yet been selected (step S640: No), the process returns to step S520, and one of the unselected organ detection images ODImg is selected, and the processing after step S530 is performed. Is executed. On the other hand, when it is determined that all the organ detection images ODImg have been selected (step S640: Yes), the organ region detection process is completed.

  By the organ area detection processing described above, in the organ detection image ODImg, the right eye area EA (r) that is an image area corresponding to the right eye image and the left eye area EA (l) that is an image area corresponding to the left eye image. And the mouth area MA which is an image area corresponding to the mouth image. FIG. 13 is an explanatory diagram illustrating an example of a result of the organ region detection process.

  When the organ area is not detected in the organ area detection process (step S150 in FIG. 3) (step S160: No), the original face area FAo selected in step S140 is an image area that truly corresponds to the face image. It is determined that an image area that is not is erroneously detected as the original face area FAo (step S170). In this case, setting of the determined face area FAf based on the selected original face area FAo (step S200 described later) is not executed. In this embodiment, the case where no organ area is detected means that in any organ detection image ODImg (see FIG. 10B), organ areas corresponding to three types of organs (right eye area EA ( It means that at least one of r), left eye area EA (l) and mouth area MA) is not detected.

  In the organ region detection process (step S150 in FIG. 3), organ regions (right eye region EA (r), left eye region EA (l), and mouth region MA) corresponding to three types of organs in at least one organ detection image ODImg. Are detected (step S160: Yes), the reliability calculation unit 220 (FIG. 1) calculates the organ region reliability for each detected organ region (step S180). The organ region reliability is an index calculated based on the result of the organ region detection process, and an organ region for a certain type of organ detected by the organ region detection process truly corresponds to an image of the organ of that type. This is an index representing the certainty of being an image region. In the organ region detection process, an image region that does not correspond to the facial organ image, that is, an image region that does not include the facial organ image at all or a part of the facial organ image, is included in the facial organ image. An image area that is not a corresponding image area may be erroneously detected as an organ area. The organ region reliability represents the certainty that the organ region detection process is not a false detection but a correct detection.

  In this embodiment, a value obtained by dividing the number of overlapping windows by the maximum number of overlapping windows is used as the organ region reliability. Here, the number of overlapping windows is the number of determination target image areas JIA referred to when setting the organ area, that is, the number of windows SW defining the determination target image area JIA. For example, when setting the right eye area EA (r) shown in FIG. 12B, the three windows SW (SW1 to SW3) partially overlapping each other shown in FIG. 12A are referred to. Therefore, the number of overlapping windows is 3. The maximum number of overlapping windows is the window SW in which at least a part of all the windows SW arranged on the organ detection image ODImg in the organ region detection process overlaps with the average window AW (see FIG. 12B). Is the number of The maximum number of overlapping windows is uniquely determined by the movement pitch of the window SW and the size change pitch.

  When the detected organ region is an image region that truly corresponds to the facial organ image, the determination target image region JIA corresponds to the facial organ image for a plurality of windows SW whose positions and sizes approximate each other. There is a high possibility that the image area is determined to be an image area. On the other hand, if the detected organ area is not an image area corresponding to the facial organ image but an erroneous detection, an image corresponding to the determination target image area JIA erroneously corresponds to the facial organ image for a certain window SW. Even if it is determined as an area, there is a possibility that the determination target image area JIA is not an image area corresponding to the facial organ image for another window SW whose position and size approximate to the window SW. Is expensive. Therefore, in this embodiment, a value obtained by dividing the number of overlapping windows by the maximum number of overlapping windows is used as the organ region reliability.

  In step S190 (FIG. 3), the face inclination estimation unit 215 (FIG. 1) estimates the inclination of the face represented in the original face area FAo (FIG. 10A) based on the detection result of the organ area. In step S200, the face area determination unit 216 (FIG. 1) sets the determined face area FAf based on the estimated face inclination. FIG. 14 is an explanatory diagram showing a method for estimating the face inclination and a method for setting the determined face area FAf. As shown in FIG. 14A, the face inclination estimation unit 215 represents the inclination of the straight line CL connecting the centroid of the right eye area EA (r) and the centroid of the left eye area EA (l) in the original face area FAo. It is estimated that the face is tilted. That is, the face inclination is estimated based on the positional relationship between the right eye area EA (r) and the left eye area EA (l).

  In the organ area detection process (FIG. 9), when an organ area is detected from a plurality of organ detection images ODImg, each organ area (right eye area EA (r), left eye area EA (l), mouth area) is detected. Face tilt is estimated based on the organ region with the highest total organ region reliability for MA). As described above, the facial organ learning data OLD is set so as to be able to detect an image of an organ whose organ inclination value is in the range of plus or minus 15 degrees around 0 degrees in the organ region detection process. On the other hand, the inclinations of the plurality of organ detection images ODImg (FIG. 10B) are different from each other by at least 30 degrees. Therefore, an organ region should be detected only from one organ detection image ODImg, but an organ region may be detected from a plurality of organ detection images ODImg due to erroneous detection or the like. In such a case, the organ region used for estimating the face inclination is selected based on the organ region reliability.

  Further, as shown in FIG. 14B, the face area determination unit 216 rotates the average window AW that defines the original face area FAo by the face inclination estimated around the center of gravity of the average window AW, and after the rotation The image area defined by the average window AW is set as the determined face area FAf. That is, the determined face area FAf is a rectangular image area whose two outer sides are parallel to the straight line CL. Note that the face area determination unit 216 of the present embodiment corresponds to a face area resetting unit in the present invention.

  In step S210 (FIG. 3), area detection unit 210 (FIG. 1) determines whether or not there is an original face area FAo that has not yet been selected in step S140. If it is determined that there is an unselected original face area FAo (step S210: No), the process returns to step S140, and one of the unselected original face areas FAo is selected, and the processing from step S150 is executed. Is done. On the other hand, when it is determined that all the original face areas FAo have been selected (step S210: Yes), the process proceeds to step S220.

  In step S220 (FIG. 3), the information adding unit 230 (FIG. 1) adds information indicating the set position (coordinates) of the determined face area FAf as attached information to the image file including the original image data. Thereby, an image file including original image data representing the original image OImg and information indicating the position (coordinates) of the determined face area FAf in the original image OImg is generated. In addition to information indicating the position (coordinates) of the determined face area FAf, information indicating the position (coordinates) of the organ area (right eye area EA (r), left eye area EA (l), mouth area MA) or organ area Information indicating the reliability may be added to the image file as attached information.

  Various kinds of information included in the generated image file can be used for various purposes. For example, a predetermined correction process can be performed using the face area FA or an image area set based on the face area FA as a target image area. Examples of the predetermined correction process include image deformation and skin color correction in the target image area.

  As described above, in the face area detection process performed by the printer 100 according to the present embodiment, an image corresponding to the face image in the face detection image FDImg is detected by detecting the original face area FAo in the face detection image FDImg. The position of the region is specified. In addition, an organ area (right eye area EA (r), left eye area EA (l), mouth area MA) in the original face area FAo is detected, and is represented in the original face area FAo based on the detection result of the organ area. By estimating the inclination of the face in the image plane (face inclination), the inclination of the face image in the face detection image FDImg is specified. Here, when the organ area is detected, the determination target image area JIA is set by the window SW for the original face area FAo. The time required for this processing is the entire face detection image FDImg. Compared to the time required for setting the determination target image area JIA by the window SW, the time is extremely short. Therefore, in the face area detection process performed by the printer 100 according to the present embodiment, it is possible to improve the detection accuracy related to the position and inclination of the face area while increasing the speed of the face area detection process.

  In particular, in the face area detection processing by the printer 100 of this embodiment, the face learning data FLD is generated by learning using a face sample image group including a plurality of face sample images having arbitrary different inclinations in the image plane. In other words, the learning data can detect all face inclination images from 0 degrees to 360 degrees. Therefore, in the original face area detection process (FIG. 4), the calculation of the cumulative evaluation value Tv and the threshold TH determination need only be executed once for one determination target image area JIA. Therefore, in the face area detection process performed by the printer 100 according to the present embodiment, the processing speed can be sufficiently increased.

B. Variations:
The present invention is not limited to the above-described examples and embodiments, and can be implemented in various modes without departing from the gist thereof. For example, the following modifications are possible.

B1. Modification 1:
In the embodiment described above, in any organ detection image ODImg in the organ region detection process (step S150 in FIG. 3), organ regions (right eye region EA (r) and left eye region EA (l)) corresponding to three types of organs. If at least one of the mouth areas MA) is not detected, it is determined that the organ area is not detected. If at least one of the three types of organ areas is detected, the organ area is detected. It may be determined. Or, regardless of the detection result of the mouth area MA, when both the right eye area EA (r) and the left eye area EA (l) used for face inclination estimation (step S190) are detected, the organ area is detected. If at least one of the right eye area EA (r) and the left eye area EA (l) is not detected, it may be determined that the organ area is not detected.

  If it is determined that an organ region has been detected when at least one of the three types of organ regions is detected, the right-eye region EA (r) and the left-eye region EA (l) are used for face inclination estimation (step S190). Or one or both of them may not be available. In such a case, the inclination of the organ detection image ODImg in which the organ area is detected may be estimated as the face inclination.

  In the above embodiment, when an organ region is detected in the organ region detection process (step S150 in FIG. 3), the organ region reliability is calculated (step S180). Only when it is detected from the organ detection image ODImg, the organ region reliability is calculated in order to determine the organ region used for the estimation of the face inclination (step S190), and the organ region has one organ detection image. When it is detected only from ODImg, the calculation of the organ region reliability may not be executed.

  If the calculated organ area reliability is lower than a predetermined threshold, the original face area FAo selected in step S140 is a true face as in the case where the organ area is not detected in the organ area detection process. It may be determined that an image area that is not an image area corresponding to the first image is erroneously detected as the original face area FAo.

B2. Modification 2:
In the above embodiment, 12 organ detection images ODImg (FIG. 10B) having different inclinations at intervals of 30 degrees are used for the organ region detection process (FIG. 9). This is because the facial organ learning data OLD in the example is set so that an image of an organ having an organ inclination value in the range of plus or minus 15 degrees around 0 degrees can be detected. The difference in inclination of the organ detection image ODImg (that is, the number of generated organ detection images ODImg) is variously modified according to the range of the organ inclination that can be detected in the facial organ learning data OLD. For example, when the facial organ learning data OLD is set so as to be able to detect an image of an organ with a value of the organ inclination in the range of plus or minus 10 degrees centering on 0 degree, the inclinations are mutually spaced at intervals of 20 degrees. 18 different organ detection images ODImg are used in the organ region detection process.

B3. Modification 3:
In the above embodiment, the setting of the determined face area FAf (step S200) is performed after the estimation of the face inclination (step S190 in FIG. 3). However, the setting of the determined face area FAf is not necessarily performed. That is, the output of the face area detection process may be only the position of the original face area FAo and the estimated face inclination.

B4. Modification 4:
In the above embodiment, only the face learning data FLD and the facial organ learning data OLD corresponding to the front-facing image are prepared, but those corresponding to the right and left directions are prepared, and the face learning data FLD and the face organ learning data OLD are prepared. Detection of a face area or an organ area corresponding to an image or an image of a facial organ may be performed.

B5. Modification 5:
The above-described organ region reliability calculation method can be variously modified. For example, the cumulative evaluation value Tv calculated in step S560 of the organ region detection process (FIG. 9) may be used as the organ region reliability. If the organ region is an image region that truly corresponds to an image of a facial organ, the cumulative evaluation value Tv is likely to be a large value, so the cumulative evaluation value Tv can be used as the organ region reliability. is there. When a plurality of windows SW are considered during the organ area setting process (step S630 in FIG. 9), the average of the cumulative evaluation values Tv calculated for the determination target image area JIA corresponding to each window SW is calculated. What is necessary is just to use as reliability.

  Further, the organ region reliability may be calculated using the evaluation value vX for each filter X calculated for calculating the cumulative evaluation value Tv. That is, when calculating the cumulative evaluation value Tv, the evaluation value vX has been converted to a value of 1 or 0 by comparison with the threshold thX and then multiplied by the weighting coefficient WeX. The weight coefficient WeX may be multiplied and the sum of the products may be used as the reliability. If the organ region is an image region that truly corresponds to an image of a facial organ, the sum of the products is likely to be a large value. Therefore, the sum of the products can be used as the reliability. When a plurality of windows SW are taken into account when setting the organ area, the average of the sum of products of the evaluation value vX calculated for the determination target image area JIA corresponding to each window SW and the weighting coefficient WeX. May be used as the reliability.

  Further, as in the above embodiment, when a value obtained by dividing the number of candidate overlapping windows by the maximum number of overlapping windows is used as the organ region original reliability, when the number of candidate overlapping windows is a predetermined number or more, the reliability May be regarded as the maximum value, and thereafter, the window SW may not be arranged at an overlapping position. In this way, the processing speed can be increased.

  In the above embodiment, the organ region reliability is calculated for each type of facial organ (right eye, left eye, mouth), but one organ region reliability corresponding to all types of facial organs is used. It may be calculated.

B6. Modification 6:
The aspects of the original face area detection process (FIG. 4) and the organ area detection process (FIG. 9) in the above embodiment are merely examples, and various changes can be made. For example, the size of the face detection image FDImg (see FIG. 5) is not limited to 320 pixels × 240 pixels, and may be other sizes, or the original image OImg itself can be used as the face detection image FDImg. . Further, the size of the window SW used, the moving direction and the moving amount (moving pitch) of the window SW are not limited to those described above. In the above-described embodiment, the size of the face detection image FDImg is fixed, and a plurality of sizes of window SW are arranged on the face detection image FDImg, so that the determination target image area JIA having a plurality of sizes is set. However, a plurality of types of face detection images FDImg are generated, and a fixed-size window SW is arranged on the face detection image FDImg so that a determination target image area JIA having a plurality of sizes is set. Good.

  In the above embodiment, the face evaluation and the organ determination are performed by comparing the cumulative evaluation value Tv with the threshold value TH (see FIG. 6), but the face determination and the organ determination are performed using a plurality of discriminators. Other methods may be used. The learning method used for setting the face learning data FLD and the facial organ learning data OLD is also changed according to the face determination and organ determination methods. Further, the face determination and the organ determination are not necessarily performed by a determination method using learning, and may be performed by other methods such as pattern matching.

  In the above embodiment, 12 types of specific face inclinations in increments of 30 degrees are set. However, more types of specific face inclinations may be set, or fewer types of specific face inclinations are set. May be. In addition, the specific face inclination does not necessarily need to be set, and face determination may be performed for a 0 degree face inclination. In the above embodiment, the face sample image group includes an image obtained by enlarging or reducing the basic face sample image or a rotated image. However, the face sample image group does not necessarily include such an image. .

  In the above embodiment, the face determination (or organ determination) for the determination target image area JIA defined by the window SW of a certain size is determined to be an image area corresponding to the face image (or facial organ image). In the case where the window SW having a size smaller than the size by a predetermined ratio or more is arranged, it is assumed that the window SW is arranged avoiding the determination target image area JIA determined to be the image area corresponding to the face image. Also good. In this way, the processing speed can be increased.

  In the above embodiment, the image data stored in the memory card MC is set as the original image data. However, the original image data is not limited to the image data stored in the memory card MC, and is acquired via a network, for example. It may be image data.

  In the above embodiment, the right eye, the left eye, and the mouth are set as the types of facial organs, and the right eye area EA (r), the left eye area EA (l), and the mouth area MA are detected as organ areas. However, it is possible to change which organ of the face is set as the type of facial organ. For example, only one or two of the right eye, the left eye, and the mouth may be set as the types of facial organs. In addition to the right eye, left eye, and mouth, or instead of at least one of the right eye, left eye, and mouth, other organ types of the face (for example, nose or eyebrows) are set as the facial organ types, An area corresponding to such an organ image may be detected as the organ area.

  In the above embodiment, the original face area FAo and the organ area are rectangular areas, but the original face area FAo and the organ area may be areas having shapes other than the rectangle.

  In the above embodiment, the face area detection process by the printer 100 as the image processing apparatus has been described. However, part or all of the process is executed by another type of image processing apparatus such as a personal computer, a digital still camera, or a digital video camera. It is good also as what is done. The printer 100 is not limited to an ink jet printer, and may be another type of printer, such as a laser printer or a sublimation printer.

  In the above embodiment, a part of the configuration realized by hardware may be replaced with software, and conversely, a part of the configuration realized by software may be replaced by hardware.

  In addition, when part or all of the functions of the present invention are realized by software, the software (computer program) can be provided in a form stored in a computer-readable recording medium. In the present invention, the “computer-readable recording medium” is not limited to a portable recording medium such as a flexible disk or a CD-ROM, but an internal storage device in a computer such as various RAMs and ROMs, a hard disk, and the like. An external storage device fixed to the computer is also included.

1 is an explanatory diagram schematically illustrating a configuration of a printer 100 as an image processing apparatus according to an embodiment of the present invention. It is explanatory drawing which shows the face learning data FLD and the facial organ learning data OLD. It is a flowchart which shows the flow of a face area | region detection process. It is a flowchart which shows the flow of an original face area | region detection process. It is explanatory drawing which shows the outline | summary of an original face area | region detection process. It is explanatory drawing which shows the outline | summary of the calculation method of accumulated evaluation value Tv used for face determination. It is explanatory drawing which shows an example of the sample image used for learning. It is explanatory drawing which shows the outline | summary of an original face area | region setting process. It is a flowchart which shows the flow of an organ area | region detection process. It is explanatory drawing which shows an example of the image ODImg for organ detection. It is explanatory drawing which shows the outline | summary of an organ area | region detection process. It is explanatory drawing which shows the outline | summary of an organ area | region setting process. It is explanatory drawing which shows an example of the result of an organ area | region detection process. It is explanatory drawing which shows the estimation method of the face inclination, and the setting method of the determination face area | region FAf.

Explanation of symbols

100 ... Printer 110 ... CPU
DESCRIPTION OF SYMBOLS 120 ... Internal memory 140 ... Operation part 150 ... Display part 160 ... Printer engine 170 ... Card interface 172 ... Card slot 200 ... Image processing part 210 ... Area | region detection part 211 ... Determination object setting part 212 ... Evaluation value calculation part 213 ... Determination part 214 ... Area setting unit 215 ... Face inclination estimation unit 216 ... Face region determination unit 220 ... Reliability calculation unit 230 ... Information addition unit 310 ... Display processing unit 320 ... Print processing unit

Claims (12)

An image processing apparatus,
A face area detection unit that detects a face area corresponding to a face image in the target image represented by the target image data;
An organ region detector for detecting an organ region corresponding to an image of a facial organ in the detected face region;
An image processing apparatus comprising: a face inclination estimation unit that estimates an inclination in an image plane of a face represented in the face area based on a detection result of the organ area. The image processing apparatus according to claim 1,
The face area detection unit
A determination target setting unit that sets a determination target image region that is an image region on the target image;
A storage unit for storing evaluation data for calculating an evaluation value indicating the certainty that the determination target image region is an image region corresponding to a face image;
An evaluation value calculation unit that calculates the evaluation value based on the evaluation data and image data corresponding to the determination target image region;
A determination unit that determines whether or not the determination target image region is an image region corresponding to a face image based on the evaluation value;
And an area setting unit that sets the face area based on the position and size of the determination target image area determined to be an image area corresponding to a face image. The image processing apparatus according to claim 2,
The evaluation data is generated by learning using a face image group including a plurality of face images having arbitrary different inclinations in an image plane. The image processing apparatus according to claim 3,
The face image group includes an image processing apparatus including a plurality of face images whose inclinations in an image plane differ from each other by 30 degrees or more. The image processing apparatus according to claim 3 or 4, wherein:
The image processing apparatus represents the probability that the evaluation target image area is an image area corresponding to a face image having the same inclination as that of a face image included in the face image group. An image processing apparatus according to any one of claims 1 to 5,
The organ region detection unit detects the organ region corresponding to at least two types of organs,
The image processing apparatus, wherein the face inclination estimation unit estimates the inclination based on a positional relationship between the organ regions corresponding to the at least two types of organs. The image processing apparatus according to claim 6, further comprising:
Based on the detection result of the organ region, a reliability calculation unit that calculates an organ region reliability that represents the probability that the detected organ region is an image region that truly corresponds to an image of a facial organ,
The face inclination estimation unit selects the organ area to be used for the inclination estimation based on the organ area reliability when a plurality of the organ areas corresponding to a specific type of organ are detected. Processing equipment. The image processing apparatus according to claim 7,
The organ region detection unit is
A determination target setting unit that sets a determination target image region that is an image region on the face region;
A storage unit for storing evaluation data for calculating an evaluation value representing the probability that the determination target image region is an image region corresponding to an image of a facial organ;
An evaluation value calculation unit that calculates the evaluation value based on the evaluation data and image data corresponding to the determination target image region;
A determination unit that determines whether the determination target image region is an image region corresponding to an image of a facial organ based on the evaluation value;
An area setting unit configured to set the organ area based on the position and size of the determination target image area partially overlapping each other as one or a plurality of image areas determined to be image areas corresponding to facial organ images; Including
The image processing apparatus, wherein the reliability calculation unit calculates the organ area reliability based on at least one of the number of the determination target image areas and the evaluation value referred to when the organ area is set. The image processing apparatus according to claim 1, further comprising:
An image processing apparatus comprising: a face area resetting unit that resets the face area based on the estimated inclination. An image processing apparatus according to any one of claims 1 to 9, wherein
The type of facial organ is at least one of a right eye, a left eye, and a mouth. An image processing method comprising:
(A) detecting a face area corresponding to a face image in the target image represented by the target image data;
(B) detecting an organ region corresponding to an image of a facial organ in the detected face region;
(C) based on the detection result of the organ region, estimating an inclination in the image plane of the face represented in the face region. A computer program for image processing,
A face area detection function for detecting a face area corresponding to a face image in the target image represented by the target image data;
An organ region detection function for detecting an organ region corresponding to an image of a facial organ in the detected face region;
A computer program for causing a computer to realize a face inclination estimation function for estimating an inclination in an image plane of a face represented in the face area based on a detection result of the organ area.

Images