JP2009217607A - Calculation for reliability in detecting face region in image - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To more precisely calculate reliability in detecting a face area in an image. <P>SOLUTION: An image processor is provided with: a face region detection part for detecting a face region as an image region corresponding to the image of the face in an object image shown by object image data; and an organ region detection part for detecting an organ region as an image region corresponding to the image of the organ of the face in the detected face region. The image processor is provided with: a reliability calculation part for calculating face region reliability showing probability that the face region is truly an image region corresponding to the image of the face based on the detection result of the face region and the detection result of the organ region for the detected face region. <P>COPYRIGHT: (C)2009,JPO&INPIT

Description

  The present invention relates to calculation of reliability in detecting a face area in an image.

  A technique for detecting a face area corresponding to a face image from an image by sequentially cutting out partial images from an image represented by image data and determining whether or not the cut out partial image is an image corresponding to a face. Known (for example, Patent Documents 1 to 3).

JP2000-149018 JP2007-108990 JP2000-137788

  When a face area is detected, there is a case where an erroneous detection that erroneously detects an image area that does not correspond to a face image as a face area may occur. In Patent Document 1, a plurality of face area candidates are extracted from the target image, the degree of overlap with other candidates is calculated for each candidate, and a candidate with a large degree of overlap with other candidates is detected as a face area. It is disclosed. However, there is room for more accurately calculating an index that represents the reliability of face area detection, that is, the probability that the detected face area is not a false detection but an image area that truly corresponds to the face image. .

  The present invention has been made to solve the above-described problem, and an object of the present invention is to provide a technique capable of calculating the reliability in detecting a face area in an image with higher accuracy.

  In order to solve at least a part of the above problems, the present invention can be realized as the following forms or application examples.

Application Example 1 An image processing apparatus,
A face area detection unit that detects a face area that is an image area corresponding to a face image in the target image represented by the target image data;
An organ region detector that detects an organ region that is an image region corresponding to an image of a facial organ in the detected face region;
For the detected face area, based on the detection result of the face area and the detection result of the organ area, the face area reliability that represents the probability that the face area is an image area that truly corresponds to a face image An image processing apparatus comprising: a reliability calculation unit that calculates the degree.

  In this image processing apparatus, since the face area reliability is calculated based not only on the detection result of the face area but also on the detection result of the organ area, the reliability in detecting the face area in the image can be calculated with higher accuracy. .

[Application Example 2] The image processing apparatus according to Application Example 1,
The reliability calculation unit includes:
A face area original reliability calculation unit for calculating a face area original reliability indicating the certainty that the detected face area is an image area that truly corresponds to a face image based on the detection result of the face area; ,
An organ area original reliability calculation unit that calculates an organ area original reliability that represents the probability that the detected organ area is an image area that truly corresponds to an image of the organ based on the detection result of the organ area; And having
Based on the calculated face area original reliability, detection success / failure in detection of the organ area, and calculated organ area original reliability when the detection of the organ area is successful, the face area reliability An image processing apparatus for calculating

  In this image processing apparatus, the calculation is based on the face area original reliability calculated based on the detection result of the face area, the success or failure of detection in the detection of the organ area, and the detection result of the organ area when the detection of the organ area is successful. Since the facial region reliability is calculated based on the original organ region reliability, the reliability in detecting the facial region in the image can be calculated more accurately than when only the original facial region reliability is used. Can do.

[Application Example 3] The image processing apparatus according to Application Example 2,
The reliability calculation unit is configured so that the face area reliability when the detection of the organ area is not successful is smaller than the face area reliability when the detection of the organ area is successful. An image processing apparatus that calculates the degree.

  In this image processing apparatus, regarding an organ region that is normally detected if the face region is an image region that truly corresponds to a face image, the reliability of the face region when the detection of the organ region is not successful is achieved. Since the face area reliability is calculated to be smaller than the face area reliability when the organ area is successfully detected, the reliability in detecting the face area in the image can be calculated with higher accuracy.

Application Example 4 The image processing apparatus according to Application Example 2 or Application Example 3,
The reliability calculation unit is an image processing device that calculates the face area reliability so that the face area reliability decreases as the organ area original reliability decreases.

  In this image processing apparatus, regarding the organ area original reliability, which is considered to be usually a large value if the face area is an image area that truly corresponds to the face image, the smaller the organ area original reliability, the more the face area reliability. Since the face area reliability is calculated so as to be small, the reliability in detecting the face area in the image can be calculated with higher accuracy.

Application Example 5 The image processing apparatus according to any one of Application Example 2 to Application Example 4,
The face area detection unit
A determination target setting unit that sequentially sets a plurality of different image areas in the target image as a determination target image area;
An evaluation value calculation unit that calculates at least one evaluation value representing the certainty that the determination target image region is an image region corresponding to a face image;
A determination unit that determines whether the determination target image region corresponds to a face image based on the evaluation value;
A face area setting unit that sets the face area based on the position and size of the determination target image area that partially overlaps one or more determined to correspond to a face image;
The face area original reliability calculation unit calculates the face area original reliability based on at least one of the number of the determination target image areas and the evaluation value referred to when the face area is set. Image processing device.

  In this image processing apparatus, the face area original reliability can be calculated as an index value representing the probability that the detected face area is an image area that truly corresponds to the face image.

Application Example 6 The image processing apparatus according to any one of Application Example 2 to Application Example 5,
The organ region detection unit is
A determination target setting unit that sequentially sets a plurality of different image areas in the face area as a determination target image area;
An evaluation value calculation unit that calculates at least one evaluation value representing the probability of being an image region corresponding to an image of a facial organ for the determination target image region;
A determination unit that determines whether the determination target image region corresponds to an image of an organ based on the evaluation value;
An organ region setting unit that sets the organ region based on the position and size of the determination target image region that partially overlaps one or more determined to correspond to the image of the organ;
The organ region original reliability calculation unit calculates the organ region original reliability based on at least one of the number of the determination target image regions and the evaluation value referred to when the organ region is set. Image processing device.

  In this image processing apparatus, the organ region original reliability can be calculated as an index value representing the probability that the detected organ region is an image region that truly corresponds to an image of a facial organ.

Application Example 7 The image processing apparatus according to any one of Application Examples 1 to 6, further comprising:
An image processing apparatus, comprising: an information adding unit that adds face position information indicating the position of the detected face area and the face area reliability to the target image data.

  In this image processing apparatus, since the position of the face area and the reliability in the face area detection calculated with higher accuracy are specified by the information added to the target image data, the face in the image represented by the target image data The result of the correction process for the image area set based on the area can be made more natural.

[Application Example 8] The image processing apparatus according to Application Example 7,
The information adding unit further includes at least one of organ position information indicating the position of the detected organ region, the face region original reliability, and the organ region original reliability in the target image data. An image processing apparatus to be added.

Application Example 9 The image processing apparatus according to any one of Application Example 1 to Application Example 7,
The image processing apparatus, wherein the facial organ is at least one of a right eye, a left eye, and a mouth.

  In this image processing apparatus, the face area reliability is based not only on the detection result of the face area but also on the detection result of the organ area as an image area corresponding to the facial organ that is at least one of the right eye, the left eye, and the mouth. Since it is calculated, the reliability in detecting the face area in the image can be calculated with higher accuracy.

  Note that the present invention can be realized in various modes, for example, an image processing method and apparatus, a face area detection method and apparatus, a reliability calculation method and apparatus, and the functions of these methods or apparatuses. For example, a recording medium on which the computer program is recorded, a data signal that includes the computer program and is embodied in a carrier wave, and the like.

Next, embodiments of the present invention will be described in the following order based on examples.
A. First embodiment:
A-1. Configuration of image processing device:
A-2. Face area / organ area detection processing:
A-3. Other methods of calculating reliability:
B. Second embodiment:
C. Variations:

A. First embodiment:
A-1. Configuration of image processing device:
FIG. 1 is an explanatory diagram schematically showing the configuration of a printer 100 as an image processing apparatus according to the first embodiment of the present invention. The printer 100 of this embodiment is an ink jet color printer that supports so-called direct printing, in which an image is printed based on image data acquired from a memory card MC or the like. The printer 100 includes a CPU 110 that controls each unit of the printer 100, an internal memory 120 configured by a ROM and a RAM, an operation unit 140 configured by buttons and a touch panel, a display unit 150 configured by a liquid crystal display, and a printer. An engine 160 and a card interface (card I / F) 170 are provided. The printer 100 may further include an interface for performing data communication with other devices (for example, a digital still camera or a personal computer). Each component of the printer 100 is connected to each other via a bus.

  The printer engine 160 is a printing mechanism that performs printing based on print data. The card interface 170 is an interface for exchanging data with the memory card MC inserted into the card slot 172. In this embodiment, an image file including image data is stored in the memory card MC.

  The internal memory 120 stores an image processing unit 200, a display processing unit 310, and a print processing unit 320. The image processing unit 200 is a computer program for executing processing such as face region / organ region detection processing, which will be described later, under a predetermined operating system. The display processing unit 310 is a display driver that controls the display unit 150 to display processing menus, messages, images, and the like on the display unit 150. The print processing unit 320 is a computer program for generating print data from image data, controlling the printer engine 160, and printing an image based on the print data. The CPU 110 implements the functions of these units by reading and executing these programs from the internal memory 120.

  The image processing unit 200 includes an area detection unit 210, a reliability calculation unit 220, and an information addition unit 230 as program modules. The area detection unit 210 detects an image area corresponding to a predetermined image (for example, a face image or a face organ image) in the target image represented by the target image data. The region detection unit 210 includes a determination target setting unit 212, an evaluation value calculation unit 214, a determination unit 216, and a region setting unit 218. The functions of these parts will be described in detail in the description of the face area / organ area detection processing described later. As will be described later, the area detection unit 210 detects the face area corresponding to the face image and the organ area corresponding to the face organ image. It corresponds to a detection unit.

  The reliability calculation unit 220 is an image in which an image region detected by the region detection unit 210 as an image region corresponding to a predetermined image (for example, a face image or a facial organ image) truly corresponds to the predetermined image. A reliability representing the certainty of being an area is calculated. The reliability calculation unit 220 includes an original reliability calculation unit 222. The function of the original reliability calculation unit 222 will be described in detail in the description of the face area / organ area detection process described later. Note that the original reliability calculation unit 222 calculates the face region original reliability and the organ region original reliability described later, so that the face region original reliability calculation unit and the organ region original reliability calculation unit according to the present invention are used. Equivalent to.

  The information adding unit 230 adds predetermined information (for example, information indicating the positions of the face area and the organ area) described later to the image file including the target image data.

  The internal memory 120 also stores face learning data FLD and face organ learning data OLD used for region detection by the region detection unit 210. The contents of the face learning data FLD and the face organ learning data OLD will be described in detail in the description of the face area / organ area detection processing described later.

A-2. Face area / organ area detection processing:
FIG. 2 is a flowchart showing the flow of face area / organ area detection processing. The face area / organ area detection process detects a face area, which is an image area corresponding to a face image in the target image represented by the target image data, and an image corresponding to the face organ image in the detected face area. This is a process for detecting an organ region which is a region. In the present embodiment, the facial organ means three organs of the right eye, the left eye, and the mouth.

  In step S110, the image processing unit 200 (FIG. 1) acquires target image data representing the target image. In the printer 100 of this embodiment, when the memory card MC is inserted into the card slot 172, thumbnail images of the image files stored in the memory card MC are displayed on the display unit 150. The user selects one or more images to be processed via the operation unit 140 while referring to the displayed thumbnail images. The image processing unit 200 sets image data corresponding to one or more selected images as target image data, acquires an image file including the target image data from the memory card MC, and stores a predetermined area in the internal memory 120. To store.

  In step S120 (FIG. 2), the area detection unit 210 (FIG. 1) detects a face area that is an image area corresponding to the face image in the target image represented by the target image data. FIG. 3 is a flowchart showing the flow of face area detection processing. FIG. 4 is an explanatory diagram showing an outline of the face area detection process. FIG. 4 shows an example of the target image TImg represented by the target image data.

  In step S310 in the face area detection process (FIG. 3), the area detection unit 210 (FIG. 1) generates face detection image data representing the face detection image FDImg from the target image data. In the present embodiment, as shown in FIG. 4, the face detection image FDImg is an image having a size of horizontal 320 pixels × vertical 240 pixels. The area detection unit 210 generates face detection image data representing the face detection image FDImg by performing resolution conversion of the target image data as necessary.

  In step S320 (FIG. 3), the determination target setting unit 212 (FIG. 1) sets the size of the window SW used for setting the determination target image (described later) to an initial value. In step S330, the determination target setting unit 212 arranges the window SW at the initial position on the face detection image FDImg. In step S340, the determination target setting unit 212 determines whether the image area defined by the window SW arranged on the face detection image FDImg is a face area corresponding to the face image (hereinafter referred to as “face”). It is also set in a determination target image area JIA that is a target of determination. The middle part of FIG. 4 shows a state in which a window SW having an initial value size is arranged at an initial position on the face detection image FDImg, and an image area defined by the window SW is set as the determination target image area JIA. Yes. In this embodiment, as will be described later, the determination target image area JIA is sequentially set while changing the size and position of the square-shaped window SW, but the initial value of the size of the window SW is the horizontal size 240. The initial position of the window SW is such that the upper left vertex of the window SW overlaps the upper left vertex of the face detection image FDImg.

  In step S350 (FIG. 3), the evaluation value calculation unit 214 (FIG. 1) calculates a cumulative evaluation value Tv used for face determination for the determination target image area JIA. In the present embodiment, face determination is executed for each specific face inclination set in advance. That is, for each specific face inclination, it is determined whether or not the determination target image area JIA is an image area corresponding to a face image inclined by the specific face inclination. Therefore, the cumulative evaluation value Tv is also calculated for each specific face inclination. Here, the specific face inclination means a rotation angle of the face image in the image plane (in-plane). In this embodiment, the specific face inclination is based on the state where the face image is positioned along the vertical direction of the image (the state where the head faces upward and the jaw faces downward) (specific face inclination = 0). 12 degrees (0 degrees, 30 degrees, 60 degrees,..., 330 degrees) are set by increasing the inclination of the face image by 30 degrees clockwise.

  FIG. 5 is an explanatory diagram showing an outline of a method of calculating the cumulative evaluation value Tv used for face determination. In this embodiment, N filters (filter 1 to filter N) are used to calculate the cumulative evaluation value Tv. The external shape of each filter has the same aspect ratio as that of the window SW (that is, has a square shape), and a positive region pa and a negative region ma are set for each filter. The evaluation value calculation unit 214 applies the filter X (X = 1, 2,..., N) in order to the determination target image area JIA to calculate the evaluation value vX (that is, v1 to vN). Specifically, the evaluation value vX is determined from the sum of the luminance values of pixels located in the area on the determination target image area JIA corresponding to the plus area pa of the filter X, based on the determination target image area JIA corresponding to the minus area ma. This is a value obtained by subtracting the sum of the luminance values of the pixels located in the upper region.

  The calculated evaluation value vX is compared with a threshold thX (that is, th1 to thN) set corresponding to each evaluation value vX. In this embodiment, when the evaluation value vX is equal to or greater than the threshold thX, it is determined that the determination target image area JIA is an image area corresponding to the face image for the filter X, and the value “ 1 "is set. On the other hand, when the evaluation value vX is smaller than the threshold thX, it is determined that the determination target image area JIA is an image area that is not considered to correspond to the face image with respect to the filter X, and the value “ “0” is set. Weight coefficients WeX (that is, We1 to WeN) are set for each filter X, and the sum of products of output values and weight coefficients WeX for all filters is calculated as a cumulative evaluation value Tv.

  Note that the aspect of the filter X used for face determination, the threshold thX, the weighting coefficient WeX, and the threshold TH described below are set in advance for each of the 12 specific face inclinations, and are used as face learning data FLD (FIG. 1). Stored in the internal memory 120. The face learning data FLD is set by learning using a sample image. FIG. 6 is an explanatory diagram illustrating an example of a sample image used for learning. For learning, a face sample image group composed of a plurality of face sample images that are known in advance to be images corresponding to face images, and a plurality of information that is known in advance to be images that do not correspond to face images. A non-face sample image group composed of non-face sample images.

  Since the setting of the face learning data FLD by learning is executed for each specific face inclination, as shown in FIG. 6, a face sample image group corresponding to each of 12 specific face inclinations is prepared. For example, the setting of the face learning data FLD for the 0 degree specific face inclination is executed using the face sample image group and the non-face sample image group corresponding to the 0 degree specific face inclination. The face learning data FLD is set using a face sample image group and a non-face sample image group corresponding to a specific face inclination of 30 degrees.

  The face sample image group corresponding to each specific face inclination includes a plurality of face sample images in which the ratio of the face image size to the image size is within a predetermined value range and the face image inclination is equal to the specific face inclination. (Hereinafter also referred to as “basic face sample image”). The face sample image group is an image obtained by enlarging and reducing the basic sample image at a predetermined magnification in the range of, for example, 1.2 to 0.8 times (for example, the image FIa in FIG. 6) for at least one basic sample image. And FIb) and images (for example, images FIc and FId in FIG. 6) obtained by rotating the basic face sample image clockwise and counterclockwise by a predetermined angle within a range of, for example, 15 degrees.

  Learning using a sample image is executed by, for example, a method using a neural network, a method using boosting (for example, adaboost), a method using a support vector machine, or the like. For example, when learning is performed by a method using a neural network, for each filter X (that is, filter 1 to filter N, see FIG. 5), a face sample image group and a non-face sample image corresponding to a specific face inclination. An evaluation value vX (that is, v1 to vN) is calculated using all sample images included in the group, and a threshold value thX (that is, th1 to thN) that achieves a predetermined face detection rate is set. Here, the face detection rate is the ratio of the number of face sample images determined to be an image corresponding to a face image by threshold determination based on the evaluation value vX with respect to the total number of face sample images constituting the face sample image group. Means.

  Next, the weight coefficient WeX (that is, We1 to WeN) set for each filter X is set to an initial value, and cumulative evaluation is performed on one sample image selected from the face sample image group and the non-face sample image group. A value Tv is calculated. As will be described later, in the face determination, when the cumulative evaluation value Tv calculated for a certain image is equal to or greater than a predetermined threshold TH, the image is determined to be an image corresponding to the face image. In learning, the value of the weighting coefficient WeX set for each filter X is corrected based on the correctness of the threshold determination result based on the cumulative evaluation value Tv calculated for the selected sample image (face sample image or non-face sample image). Is done. Thereafter, the selection of the sample image, the threshold value determination based on the cumulative evaluation value Tv calculated for the selected sample image, and the correction of the value of the weighting coefficient WeX based on the correctness of the determination result are the face sample image group and the non-face sample image. It is repeatedly executed for all the sample images included in the group. By executing the above processing for each specific face inclination, face learning data FLD for each specific face inclination is set.

  When the cumulative evaluation value Tv is calculated for each specific face inclination for the determination target image area JIA (step S350 in FIG. 3), the determination unit 216 (FIG. 1) sets the cumulative evaluation value Tv for each specific face inclination. The threshold value TH is compared (step S360). When the cumulative evaluation value Tv is greater than or equal to the threshold value TH for a specific face inclination, the area detection unit 210 determines that the determination target image area JIA is an image area corresponding to a face image inclined by the specific face inclination. The position of the determination target image area JIA, that is, the coordinates of the currently set window SW and the specific face inclination are stored (step S370). On the other hand, if the cumulative evaluation value Tv is smaller than the threshold value TH for any specific face inclination, the process of step S370 is skipped.

  In step S380 (FIG. 3), the area detection unit 210 (FIG. 1) determines whether or not the entire face detection image FDImg has been scanned by the window SW having the currently set size. If it is determined that the entire face detection image FDImg has not been scanned yet, the determination target setting unit 212 (FIG. 1) moves the window SW in a predetermined direction by a predetermined movement amount (step S390). The lower part of FIG. 4 shows how the window SW has moved. In this embodiment, in step S390, the window SW is moved to the right by a movement amount corresponding to 20% of the horizontal size of the window SW. If the window SW is arranged at a position where it cannot move further to the right, the window SW returns to the left end of the face detection image FDImg in step S390, and the window SW has a size of 2 in the vertical direction. It is assumed that it moves downwards by the amount of movement of the percentage. When the window SW is arranged at a position where it cannot move further downward, the entire face detection image FDImg is scanned. After the movement of the window SW (step S390), the processes after the above-described step S340 are executed for the moved window SW.

  If it is determined in step S380 (FIG. 3) that the entire face detection image FDImg has been scanned by the window SW having the currently set size, it is determined whether or not all the predetermined sizes of the window SW have been used. (Step S400). In this embodiment, as the size of the window SW, in addition to the initial value (maximum size) of 240 horizontal pixels × vertical 240 pixels, horizontal 213 pixels × vertical 213 pixels, horizontal 178 pixels × vertical 178 pixels, horizontal 149 pixels × 149 pixels vertically, 124 pixels horizontally × 124 pixels vertically, 103 pixels horizontally × 103 pixels vertically, 86 pixels wide × 86 pixels high, 72 pixels wide × 72 pixels high, 60 pixels wide × 60 pixels high, 50 pixels wide × 50 vertical Total of 15 pixels: 41 pixels wide x 41 pixels wide, 35 pixels wide x 35 pixels wide, 29 pixels wide x 29 pixels wide, 24 pixels wide x 24 pixels high, 20 pixels wide x 20 pixels high (minimum size) The size of is set. If it is determined that there is a size of the window SW that is not yet used, the determination target setting unit 212 (FIG. 1) changes the size of the window SW to the next smaller size than the currently set size (see FIG. 1). Step S410). That is, the size of the window SW is first set to the maximum size, and then changed to a smaller size in order. After the change of the size of the window SW (step S410), the processing after step S330 described above is executed for the window SW having the changed size.

  If it is determined in step S400 (FIG. 3) that all of the predetermined size of the window SW has been used, the area setting unit 218 (FIG. 1) executes a face area setting process (step S420). 7 and 8 are explanatory diagrams showing an outline of the face area setting process. The area setting unit 218 determines that the cumulative evaluation value Tv is equal to or greater than the threshold value TH in step S360 of FIG. 3, and based on the coordinates of the window SW and the specific face inclination stored in step S370, an image corresponding to the face image. A face area FA as an area is set. Specifically, when the specific face inclination is 0 degree, the image area defined by the window SW (that is, the determination target image area JIA (see FIG. 4)) is set as the face area FA as it is. On the other hand, when the specific face inclination is other than 0 degrees, the specific face inclination is clockwise around a predetermined point (for example, the center of gravity of the window SW) about the image area defined by the window SW (determination target image area JIA). The image area rotated by the amount is set as the face area FA. For example, as shown in FIG. 7A, when it is determined that the cumulative evaluation value Tv is greater than or equal to the threshold value TH for a specific face inclination of 30 degrees, it is defined by the window SW as shown in FIG. An image area obtained by rotating the image area to be rotated 30 degrees clockwise around the center of gravity of the window SW is set as the face area FA.

  In step S370, when a plurality of windows SW that partially overlap each other with respect to a specific face inclination are stored, the region setting unit 218 determines a predetermined point (for example, the center of gravity of the window SW) of each window SW. One new window (hereinafter also referred to as “average window AW”) having an average size of the coordinates of each window SW is set with the average coordinate of the coordinates as the center of gravity. For example, as shown in FIG. 8A, when four windows SW (SW1 to SW4) that partially overlap each other are stored, as shown in FIG. 8B, each of the four windows SW is stored. One average window AW having an average size of the sizes of the four windows SW is defined with the average coordinate of the coordinates of the center of gravity of the four windows SW as the center of gravity. At this time, as described above, when the specific face inclination is 0 degree, the image area defined by the average window AW is set as the face area FA as it is. On the other hand, when the specific face inclination is other than 0 degrees, the image area defined by the average window AW is rotated clockwise by a specific face inclination around a predetermined point (for example, the center of the average window AW). The image area is set as the face area FA (see FIG. 8C).

  As shown in FIG. 7, even when one window SW not overlapping with other windows SW is stored, a plurality of windows SW partially overlapping each other shown in FIG. 8 are stored. Similarly, one window SW itself can be interpreted as the average window AW.

  In the present embodiment, an image obtained by enlarging and reducing the basic face sample image at a predetermined magnification in a range from 1.2 times to 0.8 times, for example, in the face sample image group (see FIG. 6) used for learning. (For example, the images FIa and FIb in FIG. 6), the face area FA is also obtained when the size of the face image relative to the size of the window SW is slightly larger or smaller than the basic sample image. Can be detected. Therefore, in the present embodiment, only the 15 discrete sizes described above are set as the size of the window SW, but the face area FA can be detected for face images of any size. Similarly, in this embodiment, an image obtained by rotating the basic face sample image clockwise and counterclockwise by a predetermined angle within a range of, for example, 15 degrees (for example, FIG. 6) is used for the face sample image group used for learning. Therefore, even when the inclination of the face image with respect to the window SW is slightly different from the basic sample image, the face area FA can be detected. Therefore, in the present embodiment, only the 12 discrete angles described above are set as the specific face inclination, but the face area FA can be detected for the face images of all angles.

  When the face area FA is not detected in the face area detection process (step S120 in FIG. 2) (step S130: No), the face area / organ area detection process ends. On the other hand, when at least one face area FA is detected (step S130: Yes), the original reliability calculation unit 222 (FIG. 1) of the reliability calculation unit 220 determines the face area for the detected face area FA. The original reliability is calculated (step S140). The face area original reliability is an index calculated based on the result of the face area detection process, and it is certain that the face area FA detected by the face area detection process is an image area that truly corresponds to the face image. It is an index that represents the uniqueness. In face area detection processing, an image area that does not correspond to a face image, that is, an image area that does not include a face image at all, or an image area that includes a part of a face image but is not an image area that truly corresponds to a face image. May be erroneously detected as the face area FA. The face area original reliability indicates the certainty that the face area detection process is not a false detection but a correct detection.

  In this embodiment, a value obtained by dividing the number of overlapping windows by the maximum number of overlapping windows is used as the face area original reliability. Here, the number of overlapping windows is the number of determination target image areas JIA referred to when setting each face area FA, that is, the number of windows SW defining the determination target image area JIA. For example, when setting the face area FA shown in FIG. 8C, the four windows SW (SW1 to SW4) partially overlapping each other shown in FIGS. 8A and 8B are referred to. Therefore, the number of overlapping windows is 4. The maximum number of overlapping windows is such that at least a part of all the windows SW arranged on the face detection image FDImg (see FIG. 4) in the face area detection process is an average window AW (see FIG. 8B). Is the number of windows SW that overlap. The maximum number of overlapping windows is uniquely determined by the movement pitch of the window SW and the size change pitch.

  When the detected face area FA is an image area that truly corresponds to a face image, the determination target image area JIA corresponds to the face image for a plurality of windows SW whose positions and sizes approximate each other. Is likely to be determined. On the other hand, when the detected face area FA is not an image area corresponding to the face image but a false detection, it is determined that the determination target image area JIA is a face area corresponding to the face image for a certain window SW. Even if this is done, there is a high possibility that the determination target image area JIA is not a face area corresponding to the face image for another window SW whose position and size approximate to the window SW. Therefore, in this embodiment, a value obtained by dividing the number of overlapping windows by the maximum number of overlapping windows is used as the face area original reliability.

  In step S150 (FIG. 2), the area detection unit 210 (FIG. 1) selects one of the face areas FA detected in step S120. In step S160, the area detection unit 210 detects an organ area that is an image area corresponding to an image of a facial organ in the selected face area FA. As described above, in this embodiment, the facial organ means three organs of the right eye, the left eye, and the mouth. FIG. 9 is a flowchart showing the flow of the organ region detection process. FIG. 10 is an explanatory diagram showing an outline of the organ region detection process. 10 shows an example of the face detection image FDImg (see FIG. 4) used in the face detection process.

  In step S510 in the organ region detection process (FIG. 9), the region detection unit 210 (FIG. 1) generates organ detection image data representing the organ detection image ODImg from the face detection image data representing the face detection image FDImg. To do. In this embodiment, as shown in FIG. 10, the organ detection image ODImg is an image corresponding to the face area FA in the face detection image FDImg, and is an image having a size of 60 pixels wide × 60 pixels vertical. The area detection unit 210 generates organ detection image data representing the organ detection image ODImg by performing face detection image data trimming, affine transformation, and resolution conversion as necessary.

  The detection of the organ area from the organ detection image ODImg is performed in the same manner as the detection of the face area FA from the face detection image FDImg described above. That is, as shown in FIG. 10, a square-shaped window SW is arranged on the organ detection image ODImg while changing its size and position (steps S520, S530, S580 to S610 in FIG. 9), and the arranged windows An image area defined by SW is set as a determination target image area JIA that is a target of determination of whether or not the image area is an organ area corresponding to an image of a facial organ (hereinafter also referred to as “organ determination”) (FIG. 9). Step S540). For the set determination target image area JIA, a cumulative evaluation value Tv used for organ determination is calculated for each organ (right eye, left eye, mouth) using the facial organ learning data OLD (FIG. 1) (FIG. 9). Step S550). The facial organ learning data OLD is data that defines the form of the filter X used for calculation of the cumulative evaluation value Tv and the organ determination, the threshold thX, the weighting coefficient WeX, and the threshold TH (see FIG. 5). Note that the learning for setting the facial organ learning data OLD is composed of a plurality of organ sample images that are known in advance to include facial organ images, as in the learning for setting the facial learning data FLD. The organ sample image group and the non-organ sample image group composed of a plurality of non-organ sample images that are known in advance not to include facial organ images.

  In the detection of the face area FA (FIG. 3), the cumulative evaluation value Tv is calculated for each specific face inclination and the face determination is performed for each specific face inclination, whereas the organ area from the organ detection image ODImg is determined. In the detection, as the cumulative evaluation value Tv, only one value corresponding to the inclination of 0 degrees is calculated for one determination target image area JIA, and only the organ determination is performed for the organ image corresponding to the inclination of 0 degrees. This is because the inclination of the organ of the face is considered to substantially match the inclination of the entire face. However, in the detection of the organ region, the cumulative evaluation value Tv may be calculated for each predetermined inclination, and the organ determination may be performed for each predetermined inclination.

  When the cumulative evaluation value Tv calculated for each organ is equal to or greater than a predetermined threshold TH, the determination target image area JIA is determined to be an image area corresponding to the image of the organ of the face, and the position of the determination target image area JIA That is, the coordinates of the currently set window SW are stored (step S570 in FIG. 9). On the other hand, when the cumulative evaluation value Tv is smaller than the threshold value TH, the process of step S570 is skipped. For all the predetermined sizes of the window SW, after the entire organ detection image ODImg is scanned by the window SW, an organ region setting process is executed (step S620 in FIG. 9). Similar to the face area setting process (see FIG. 8), the organ area setting process is a process of setting an average window AW and setting an image area defined by the average window AW as an organ area.

  With the above processing, in the organ detection image ODImg, the right eye area EA (r) which is an image area corresponding to the right eye image, the left eye area EA (l) which is an image area corresponding to the left eye image, and the mouth The mouth area MA, which is an image area corresponding to the image, is detected. FIG. 11 is an explanatory diagram showing an example of the detection result of the organ region.

  When at least one organ region is detected in step S160 (FIG. 2) (step S170: Yes), the original reliability calculation unit 222 (FIG. 1) of the reliability calculation unit 220 detects the detected organ region (FIG. 1). The organ area original reliability for the right eye area EA (r), left eye area EA (l), mouth area MA) is calculated (step S180). The organ region original reliability is an index that is calculated based on the result of the organ region detection process, and an image in which the organ region of a certain organ detected by the organ region detection process truly corresponds to the image of the organ concerned. It is an index representing the certainty of being an area. In the organ region detection process, an image region that does not correspond to the facial organ image, that is, an image region that does not include the facial organ image at all or a part of the facial organ image, is included in the facial organ image. An image area that is not a corresponding image area may be erroneously detected as an organ area. The organ region original reliability indicates the certainty that the organ region detection process is not a false detection but a correct detection.

  In this embodiment, similarly to the face area original reliability described above, a value obtained by dividing the number of candidate overlapping windows by the maximum number of overlapping windows is used as the organ area original reliability. In addition, when an organ area | region is not detected in step S160 (step S170: No), the process of step S180 is skipped.

  In step S190 (FIG. 2), the reliability calculation unit 220 (FIG. 1) causes the face area detection result, that is, the face area original reliability calculated in step S140, and the organ area detection result, that is, the organ in step S160. The face area total reliability is calculated based on the success or failure of the area detection and the organ area original reliability calculated in step S180. The face area total reliability is calculated in consideration of the detection result of the organ area (success / failure of detection of the organ area and the original reliability of the organ area) in addition to the detection result of the face area (original reliability of the face area). This is an index representing the certainty that the area FA is an image area that truly corresponds to a face image. The face area total reliability corresponds to the face area reliability in the present invention.

  In this embodiment, the face area total reliability is obtained by adding the organ area original reliability for each organ area (right eye area EA (r), left eye area EA (l), mouth area MA) to the face area original reliability. Value. At this time, the organ region original reliability for the organ region not detected in step S160 is set to zero. That is, when organ regions for all organs (right eye, left eye, mouth) are not detected, the face region total reliability is equal to the face region original reliability.

  If the detected face area FA is an image area that truly corresponds to the face image, the organ area is detected in the face area FA because the face area FA is likely to contain an organ image. The organ region original reliability is considered high. On the other hand, when the detected face area FA is not an image area corresponding to the face image but an erroneous detection, the organ area is detected in the face area FA because the face area FA does not include an organ image. Even if an organ region is detected, the detection of the organ region is a false detection, so the organ region original reliability is considered to be small. Therefore, by using the value obtained by adding the organ area original reliability for each organ area to the face area original reliability as the face area total reliability, the reliability in detecting the face area FA in the target image can be calculated more accurately. can do.

  After completion of calculation of the face area total reliability for the face area FA selected in step S150 (FIG. 2) (step S190), faces that have not been selected in step S150 for the face area FA detected in step S120 It is determined whether or not the area FA exists (step S200). If it is determined that there is an unselected face area FA (step S200: No), the process returns to step S150, and one of the unselected face areas FA is selected, and the processes after step S160 are executed. . On the other hand, if it is determined that all the detected face areas FA have been selected (step S200: Yes), the process proceeds to step S210.

  In step S210 (FIG. 2), the information addition unit 230 (FIG. 1) performs the detection of the detected face area FA and organ area (right eye area EA (r), left eye area EA (l), mouth area MA) in the target image. Information indicating the position (coordinates) and the calculated face area original reliability, organ area original reliability, and face area total reliability are added to the image file including the target image data as attached information. Thereby, the image file including the target image data, the information indicating the position (coordinates) of the face region and the organ region in the target image, the face region original reliability, the organ region original reliability, and the face region total reliability. Is generated.

  Various kinds of information included in the generated image file can be used for various purposes. For example, when a predetermined correction process is performed using the face area FA or an image area set based on the face area FA as a target image area, it is possible to set the correction process content based on the face area total reliability. For example, when the predetermined correction processing is deformation of the image in the target image area, the deformation amount when the face area total reliability is relatively small is larger than the deformation amount when the face area total reliability is relatively large. Set to a small value. Alternatively, when the face area total reliability is smaller than a predetermined threshold, the image is not deformed (that is, the deformation amount is set to zero). In this way, the result of the deformation process for the target image area can be made more natural. Generally, when the face area total reliability is relatively small, it is preferable that the correction amount in the predetermined correction process is set to a smaller value than when the face area total reliability is relatively large.

  As described above, in the printer 100 of the first embodiment, the detection result of the face area (that is, the original reliability of the face area calculated in step S140) and the detection result of the organ area (that is, detection of the organ area in step S160). , The organ area original reliability calculated in step S180), and the face area total reliability that is an index representing the likelihood that the face area FA is an image area that truly corresponds to the face image. Is calculated. Therefore, in the printer 100 of the first embodiment, the reliability in detecting the face area FA in the target image can be calculated with higher accuracy.

A-3. Other methods of calculating reliability:
The calculation method of the face area original reliability and the organ area original reliability (hereinafter collectively referred to as “original reliability”) can be variously modified. For example, the cumulative evaluation value Tv (see FIG. 5) calculated in step S350 of the face area detection process (FIG. 3) or step S550 of the organ area detection process (FIG. 9) is used as the original reliability (face area original reliability, organ The original area reliability may be used. If the face region (or organ region) is an image region that truly corresponds to a face image (or facial organ image), the cumulative evaluation value Tv is likely to be a large value, and therefore the cumulative evaluation value Tv Can be used as the original reliability. When a plurality of windows SW are considered in setting the face area and the organ area (see FIG. 8), the average of the cumulative evaluation values Tv calculated for the determination target image area JIA corresponding to each window SW. May be used as the original reliability.

  Alternatively, the original reliability may be calculated using the evaluation value vX (see FIG. 5) for each filter X calculated for calculating the cumulative evaluation value Tv. That is, when calculating the cumulative evaluation value Tv, the evaluation value vX has been converted to a value of 1 or 0 by comparison with the threshold thX and then multiplied by the weighting coefficient WeX. The weight coefficient WeX may be multiplied and the sum of the products may be used as the original reliability. If the face region (or organ region) is an image region that truly corresponds to the face image (or facial organ image), the sum of the products is likely to be a large value. Can be used as the original reliability. When a plurality of windows SW are taken into account when setting the face area and the organ area, the product of the evaluation value vX calculated for the determination target image area JIA corresponding to each window SW and the weighting coefficient WeX. The average of the total may be used as the original reliability.

  As for the face area original reliability, the number of face areas FA set for other specific face inclinations overlapping with the face area FA set for a specific face inclination may be used as the face area original reliability. In this case, when the difference in specific face inclination is greater than or equal to a predetermined value, it is preferable not to include it in the overlapping number.

  Further, as in the above embodiment, when the value obtained by dividing the number of candidate overlapping windows by the maximum number of overlapping windows is used as the original reliability (facial area original reliability, organ area original reliability), the number of candidate overlapping windows is predetermined. If the number is greater than or equal to the number, the original reliability is regarded as the maximum value, and thereafter, the window SW may not be arranged at the overlapping position. In this way, the processing speed can be increased.

  The calculation method of the face area total reliability described above can be variously modified. For example, a value (organ area original reliability) set in accordance with the organ area original reliability for each organ area (right eye area EA (r), left eye area EA (l), mouth area MA) in the face area original reliability A value obtained by subtracting a value that is set to be smaller as the value is larger may be used as the face region total reliability. At this time, if an organ area for at least one facial organ is not detected in step S160, the total reliability of the face area may be set to zero. The average value of the face area original reliability and the organ area original reliability for each organ area may be used as the face area total reliability.

  In the above embodiment, the organ region original reliability is calculated for each facial organ (right eye, left eye, mouth), but one organ region original reliability corresponding to all facial organs is calculated. It may be.

B. Second embodiment:
FIG. 12 is an explanatory diagram schematically showing the configuration of a printer 100a as an image processing apparatus in the second embodiment of the present invention. The printer 100a of the second embodiment is different from the printer 100 of the first embodiment in the configuration of the image processing unit 200 shown in FIG. That is, the image processing unit 200 of the printer 100a of the second embodiment does not include the reliability calculation unit 220 (see FIG. 1), and the original reliability calculation unit 222 is an element independent of the reliability calculation unit 220. . Further, the image processing unit 200 of the printer 100a according to the second embodiment does not include the information adding unit 230 (see FIG. 1) but includes a correction processing unit 240. Other configurations of the printer 100a are the same as those of the printer 100 of the first embodiment.

  The correction processing unit 240 performs a predetermined correction process on the target image area set based on the face area FA (see FIGS. 7 and 8) detected by the area detection unit 210. Specifically, the correction processing unit 240 performs skin color correction that corrects the color of the image area corresponding to the skin image to a preferable skin color using the face area FA itself as the target image area.

  FIG. 13 is a flowchart showing the flow of the face area correction process. The face area correction process is a process of detecting a face area that is an image area corresponding to a face image in the target image represented by the target image data, and performing skin color correction using the detected face area as the target image area. is there. The face area correction process is started in response to selection of one or a plurality of images to be processed by the user via the operation unit 140.

  The processing contents of steps S110 to S200 of the face area correction process (FIG. 13) are the same as the processing contents of steps S110 to S200 of the face area / organ area detection process (FIG. 2) in the first embodiment. However, in the face area correction process of the second embodiment, the face area original reliability calculation step (step S140 in FIG. 2) and the face area total reliability calculation step in the face area / organ area detection process of the first embodiment. (Step S190 in FIG. 2) is not executed. That is, in the face area correction process of the second embodiment, the calculated reliability is only the organ area original reliability calculated in step S180.

  In the face area detection (step S120 in FIG. 13) in the face area correction process of the second embodiment, at least one of the facial organs (right eye, left eye, mouth) is covered with a shield (for example, a mask or sunglasses). Even an image in which at least one of the facial organs is not represented as in the image can be detected as the face area FA. 14 and 15 are explanatory diagrams illustrating an example of an image in which at least one of facial organs is not represented. FIG. 14A shows an example of the face area FA detected corresponding to the face image whose mouth is shielded by the mask, and FIG. 14B shows the face area FA detected in the face area FA. Examples of the right eye area EA (r) and the left eye area EA (l) are shown. In the example of FIG. 14B, the mouth area MA is not detected. FIG. 15A shows an example of a face area FA detected corresponding to an image of a face whose eyes are shielded by sunglasses. FIG. 15B shows a detection in the face area FA. An example of the mouth area MA is shown. In the example of FIG. 15B, the right eye area EA (r) and the left eye area EA (l) are not detected. In the second embodiment, a face sample image group (see FIG. 6) used for learning for setting the face learning data FLD (FIG. 1) includes, for example, a sample image of a face whose mouth is shielded by a mask and both eyes are sunglasses. Since the sample image of the face shielded by is included in the face area detection, even an image in which at least one of the facial organs is not expressed can be detected as the face area FA.

  In step S230 (FIG. 13) in the face area correction process, the correction processing unit 240 (FIG. 12) performs skin color correction as face area correction using the detected face area FA as a target image area. At this time, the correction processing unit 240 sets the correction content in the skin color correction based on the detection result of the organ region. Specifically, the correction processing unit 240 corrects the skin color correction when at least one of the organ areas (the right eye area EA (r), the left eye area EA (l), and the mouth area MA) is not successfully detected. Set the amount to zero (ie no correction). On the other hand, when the detection of all the organ areas is successful, the correction processing unit 240 sets the correction amount for the skin color correction based on the pixel values in the face area FA.

  As described above, in the face area correction process by the printer 100a of the second embodiment, the correction contents in the skin color correction are set based on the detection result of the organ area. Specifically, when the detection of at least one of the organ areas (right eye area EA (r), left eye area EA (l), mouth area MA) is not successful, the correction amount in the skin color correction is set to zero. , Skin color correction is not executed. If at least one of the organ areas is not successfully detected in the detected face area FA, the face area FA is an image area that truly corresponds to the face image, but an undetected face organ is present. For example, there is a possibility that it is shielded by a shielding object and is not represented on the image. If correction processing such as skin color correction is performed on the face area FA in such a case, the correction result may be unnatural due to the influence of the image of the shielding object in the image. In the face area correction process according to the second embodiment, skin color correction is not executed if detection of at least one of the organ areas is not successful, so that occurrence of a situation in which the correction result becomes unnatural can be suppressed. it can.

C. Variations:
The present invention is not limited to the above-described examples and embodiments, and can be implemented in various modes without departing from the gist thereof. For example, the following modifications are possible.

C1. Modification 1:
In the first embodiment, the position (coordinates) of the detected face area FA and organ area (right eye area EA (r), left eye area EA (l), mouth area MA) in the target image by the information adding unit 230 are determined. The information shown and the calculated face area original reliability, organ area original reliability, and face area total reliability are added as attached information to the image file including the target image data. It is not necessary to add all the information, and only a part of such information may be added. For example, only the information indicating the position of the face area FA and the face area total reliability may be added to the image file as attached information. Further, in addition to the information indicating the position of the face area FA and the calculated face area total reliability, at least one of the information indicating the position of the organ area, the face area original reliability, and the organ area original reliability However, it may be added to the image file as attached information. The information indicating the positions of the face area FA and the organ area, the face area original reliability, the organ area original reliability, and the face area total reliability are stored in the internal memory 120 as one or a plurality of data files, and are stored in an image file. It may be associated with

C2. Modification 2:
In the second embodiment, if the detection of at least one of the organ areas (right eye area EA (r), left eye area EA (l), mouth area MA) is not successful, the correction amount in skin color correction is zero. Although it is set and skin color correction is not executed, even if at least one detection of the organ area is not successful, the correction is smaller than the correction amount of the skin color correction when the detection of all the organ areas is successful The skin color correction may be executed by the amount.

  In the second embodiment, the correction amount of the skin color correction in the face area correction process is set based on the detection result of the organ area. However, the present invention can be applied to the setting of correction contents in other correction processes. is there. For example, when an image area obtained by enlarging or reducing the face area FA at a predetermined magnification is set as a deformation area, and the image deformation of the deformation area is performed as correction processing, the deformation amount of the image deformation is based on the detection result of the organ area. It may be set. For example, when at least one of the organ regions is not successfully detected, the image deformation may be performed with a small deformation amount compared to the deformation amount when all the organ regions are successfully detected. The amount may be set to zero and no image deformation may be performed.

  In the second embodiment, when at least one detection of the organ area is not successful, the correction amount in the skin color correction is set to zero and the skin color correction is not executed. It is also possible that the skin color correction is not executed only when the above is not successful. Alternatively, the skin color correction may not be executed only when both the right eye area EA (r) and the left eye area EA (l) are not detected.

  In the second embodiment, the correction amount in the skin color correction may be set based on the organ area original reliability of the detected organ area. For example, the correction amount may be set so that the correction amount becomes smaller as the organ region original reliability of each organ region is smaller even when all the organ regions are successfully detected.

  In the second embodiment, the calculation of the organ region original reliability (step S180 in FIG. 13) is not necessarily executed.

C3. Modification 3:
The aspects of the face area detection process (FIG. 3) and the organ area detection process (FIG. 9) in each of the above embodiments are merely examples, and various changes can be made. For example, the size of the face detection image FDImg (see FIG. 4) is not limited to 320 pixels × 240 pixels, and may be other sizes, or the target image TImg itself can be used as the face detection image FDImg. . Similarly, the size of the organ detection image ODImg (see FIG. 10) is not limited to 60 × 60 pixels, and may be any other size, and the size of the face area FA in the face detection image FDImg itself may be detected. It can also be used as the size of the image ODImg for use. Further, the size of the window SW used, the moving direction and the moving amount (moving pitch) of the window SW are not limited to those described above. In each of the above embodiments, the sizes of the face detection image FDImg and the organ detection image ODImg are fixed, and a plurality of types of windows SW are arranged on the face detection image FDImg and the organ detection image ODImg. A plurality of sizes of determination target image areas JIA are set, but a plurality of types of face detection images FDImg and organ detection images ODImg are generated, and a fixed size window SW is used for the face detection image FDImg and organ detection The determination target image area JIA having a plurality of sizes may be set by being arranged on the image ODImg.

  In each of the above embodiments, the face evaluation and the organ determination are performed by comparing the cumulative evaluation value Tv with the threshold value TH (see FIG. 5), but the face determination and the organ determination are performed using a plurality of discriminators. Other methods may be used. The learning method used for setting the face learning data FLD and the facial organ learning data OLD is also changed according to the face determination and organ determination methods. Further, the face determination and the organ determination are not necessarily performed by a determination method using learning, and may be performed by other methods such as pattern matching.

  In each of the above embodiments, 12 types of specific face inclinations (see FIG. 6) in increments of 30 degrees are set. However, more types of specific face inclinations may be set, or fewer types of specific face inclinations may be set. A specific face inclination may be set. In addition, the specific face inclination does not necessarily need to be set, and face determination may be performed for a 0 degree face inclination. In each of the above embodiments, the face sample image group includes an image obtained by enlarging or reducing the basic sample image or a rotated image. However, the face sample image group does not necessarily include such an image. .

  In each of the above embodiments, the face determination (or organ determination) for the determination target image area JIA defined by the window SW of a certain size is determined to be an image area corresponding to the face image (or facial organ image). If the window SW having a size smaller than the size by a predetermined ratio or more is arranged, the window SW is arranged avoiding the determination target image area JIA determined to be the image area corresponding to the face image. It is good. In this way, the processing speed can be increased.

  In each of the above embodiments, the calculation of the face area original reliability (step S140 in FIG. 2) is executed after the face area detection process (step S120 in FIG. 2) is completed. May be executed during the face area detection process. For example, the face area setting process (step S420 in FIG. 3) is performed for each size of the window SW in the face area detection process, and the calculation of the face area original reliability for the set face area FA may be executed. . The calculation of the organ region original reliability (step S180 in FIG. 2) can be similarly modified.

  In each of the above embodiments, the image data stored in the memory card MC is set as the target image data. However, the target image data is not limited to the image data stored in the memory card MC, and is acquired via, for example, a network. It may be image data.

  In each of the above embodiments, the right eye, the left eye, and the mouth are set as the facial organs, and the right eye area EA (r), the left eye area EA (l), and the mouth area MA are detected as the organ areas. However, it is possible to change which organ of the face is set as the facial organ. For example, only one or two of the right eye, the left eye, and the mouth may be set as the facial organ. In addition to the right eye, the left eye, and the mouth, or in place of at least one of the right eye, the left eye, and the mouth, other organs of the face (for example, the nose and eyebrows) are set as the organs of the face. A region corresponding to such an organ image may be detected.

  In each of the above embodiments, the face area FA and the organ area are rectangular areas, but the face area FA and the organ area may be areas having a shape other than a rectangle.

  In the above embodiments, the face area / organ area detection process (or face area correction process) by the printer 100 (or 100a) as the image processing apparatus has been described. However, part or all of the process may be performed by a personal computer or a digital still. It may be executed by another type of image processing apparatus such as a camera or a digital video camera. The printer 100 (or 100a) is not limited to an ink jet printer, and may be another type of printer, such as a laser printer or a sublimation printer.

  In each of the above embodiments, a part of the configuration realized by hardware may be replaced with software, and conversely, a part of the configuration realized by software may be replaced by hardware. .

  In addition, when part or all of the functions of the present invention are realized by software, the software (computer program) can be provided in a form stored in a computer-readable recording medium. In the present invention, the “computer-readable recording medium” is not limited to a portable recording medium such as a flexible disk or a CD-ROM, but an internal storage device in a computer such as various RAMs and ROMs, a hard disk, and the like. An external storage device fixed to the computer is also included.

1 is an explanatory diagram schematically showing the configuration of a printer 100 as an image processing apparatus in a first embodiment of the present invention. FIG. It is a flowchart which shows the flow of a face area | region / organ area | region detection process. It is a flowchart which shows the flow of a face area | region detection process. It is explanatory drawing which shows the outline | summary of a face area | region detection process. It is explanatory drawing which shows the outline | summary of the calculation method of accumulated evaluation value Tv used for face determination. It is explanatory drawing which shows an example of the sample image used for learning. It is explanatory drawing which shows the outline | summary of a face area | region setting process. It is explanatory drawing which shows the outline | summary of a face area | region setting process. It is a flowchart which shows the flow of an organ area | region detection process. It is explanatory drawing which shows the outline | summary of an organ area | region detection process. It is explanatory drawing which shows an example of the detection result of an organ area | region. It is explanatory drawing which shows roughly the structure of the printer 100a as an image processing apparatus in 2nd Example of this invention. It is a flowchart which shows the flow of a face area correction process. It is explanatory drawing which shows an example of the image in which at least 1 of the organ of the face is not expressed. It is explanatory drawing which shows an example of the image in which at least 1 of the organ of the face is not expressed.

Explanation of symbols

100 ... Printer 110 ... CPU
DESCRIPTION OF SYMBOLS 120 ... Internal memory 140 ... Operation part 150 ... Display part 160 ... Printer engine 170 ... Card interface 172 ... Card slot 200 ... Image processing part 210 ... Area | region detection part 212 ... Determination target setting part 214 ... Evaluation value calculation part 216 ... Determination part 218: Area setting unit 220 ... Reliability calculation unit 222 ... Original reliability calculation unit 230 ... Information addition unit 240 ... Correction processing unit 310 ... Display processing unit 320 ... Print processing unit

Claims (11)

An image processing apparatus,
A face area detection unit that detects a face area that is an image area corresponding to a face image in the target image represented by the target image data;
An organ region detector that detects an organ region that is an image region corresponding to an image of a facial organ in the detected face region;
For the detected face area, based on the detection result of the face area and the detection result of the organ area, the face area reliability that represents the probability that the face area is an image area that truly corresponds to a face image An image processing apparatus comprising: a reliability calculation unit that calculates the degree. The image processing apparatus according to claim 1,
The reliability calculation unit includes:
A face area original reliability calculation unit for calculating a face area original reliability indicating the certainty that the detected face area is an image area that truly corresponds to a face image based on the detection result of the face area; ,
An organ region original reliability calculation unit for calculating an organ region original reliability representing the certainty that the detected organ region is an image region that truly corresponds to an image of the organ based on the detection result of the organ region; And having
Based on the calculated face area original reliability, detection success / failure in detection of the organ area, and calculated organ area original reliability when the detection of the organ area is successful, the face area reliability An image processing apparatus for calculating The image processing apparatus according to claim 2,
The reliability calculation unit is configured so that the face area reliability when the detection of the organ area is not successful is smaller than the face area reliability when the detection of the organ area is successful. An image processing apparatus that calculates the degree. The image processing apparatus according to claim 2, wherein:
The reliability calculation unit is an image processing device that calculates the face area reliability so that the face area reliability decreases as the organ area original reliability decreases. An image processing apparatus according to any one of claims 2 to 4,
The face area detection unit
A determination target setting unit that sequentially sets a plurality of different image areas in the target image as a determination target image area;
An evaluation value calculation unit that calculates at least one evaluation value representing the certainty that the determination target image region is an image region corresponding to a face image;
A determination unit that determines whether the determination target image region corresponds to a face image based on the evaluation value;
A face area setting unit that sets the face area based on the position and size of the determination target image area that partially overlaps one or more determined to correspond to a face image;
The face area original reliability calculation unit calculates the face area original reliability based on at least one of the number of the determination target image areas and the evaluation value referred to when the face area is set. Image processing device. An image processing apparatus according to any one of claims 2 to 5,
The organ region detector is
A determination target setting unit that sequentially sets a plurality of different image areas in the face area as a determination target image area;
An evaluation value calculation unit that calculates at least one evaluation value representing the probability of being an image region corresponding to an image of a facial organ for the determination target image region;
A determination unit that determines whether the determination target image region corresponds to an image of an organ based on the evaluation value;
An organ region setting unit configured to set the organ region based on the position and size of one or more of the determination target image regions determined to correspond to an image of an organ, and
The organ region original reliability calculation unit calculates the organ region original reliability based on at least one of the number of the determination target image regions and the evaluation value referred to when the organ region is set. Image processing device. The image processing apparatus according to any one of claims 1 to 6, further comprising:
An image processing apparatus, comprising: an information adding unit that adds face position information indicating the position of the detected face area and the face area reliability to the target image data. The image processing apparatus according to claim 7,
The information adding unit further includes at least one of organ position information indicating the position of the detected organ region, the face region original reliability, and the organ region original reliability in the target image data. An image processing apparatus to be added. An image processing apparatus according to any one of claims 1 to 8,
The image processing apparatus, wherein the facial organ is at least one of a right eye, a left eye, and a mouth. An image processing method comprising:
(A) detecting a face area which is an image area corresponding to a face image in the target image represented by the target image data;
(B) detecting an organ region that is an image region corresponding to an image of a facial organ in the detected face region;
(C) The probability that the detected face area is an image area that truly corresponds to a face image is represented based on the detection result of the face area and the detection result of the organ area. Calculating a face area reliability. A computer program for image processing,
A face area detection function for detecting a face area that is an image area corresponding to a face image in the target image represented by the target image data;
An organ region detection function for detecting an organ region which is an image region corresponding to an image of a facial organ in the detected face region;
For the detected face area, based on the detection result of the face area and the detection result of the organ area, the face area reliability that represents the probability that the face area is an image area that truly corresponds to a face image A computer program that causes a computer to realize a reliability calculation function for calculating a degree.

Images