JP2009237615A - Detection of image of photographic subject of specific kind in target image - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To attain coordination between accuracy and processing time of detection process of an image corresponding to a photographic subject of a specific kind in a target image. <P>SOLUTION: An image processing apparatus includes: a detection photographic subject setting unit for setting a detection photographic subject that is a kind of a photographic subject to be detected from the target image; an image data group acquiring unit for acquiring an image data group including a plurality of image data having mutually different sizes for representing a target image; an image selecting unit for selecting one piece of image data included in the image data group on the basis of the set detected photographic subject; and an image detecting unit for detecting an image corresponding to the set detected photographic subject using the selected image data. <P>COPYRIGHT: (C)2010,JPO&INPIT

Description

  The present invention relates to detection of an image of a specific type of subject in a target image.

  A technique for detecting a face region corresponding to a face image by sequentially cutting out partial images from an image represented by image data and determining whether or not the cut-out partial image is an image corresponding to a face is known. (For example, Patent Documents 1 to 4).

JP 2001-167277 A JP 2006-18805 A JP 2006-190106 A JP 2006-179030 A

  In the detection of the face area in the target image, it has been desired to achieve harmony between detection accuracy and processing time. Such a problem is not limited to the detection of the face area, but is a common problem when detecting an image corresponding to a specific type of subject in the target image.

  The present invention has been made in order to solve the above-described problem, and provides a technique that makes it possible to harmonize the accuracy and processing time of image detection processing corresponding to a specific type of subject in a target image. The purpose is to provide.

  In order to solve at least a part of the above problems, the present invention can be realized as the following forms or application examples.

Application Example 1 An image processing apparatus,
A detection subject setting unit that sets a detection subject that is a type of subject to be detected from the target image;
An image data group acquisition unit for acquiring an image data group including a plurality of image data of different sizes representing the target image;
An image selection unit that selects one of the image data included in the image data group based on the set detection subject;
An image detection apparatus comprising: an image detection unit that detects an image corresponding to the set detection subject using the selected image data.

  In this image processing device, a detection subject that is a type of subject to be detected from the target image is set, and a plurality of image data groups including image data having different sizes representing the target image are acquired, and the set detection subject Since one image data included in the image data group is selected and an image corresponding to the detected subject set using the selected image data is detected, an image corresponding to a specific type of subject in the target image The accuracy of the detection process and the processing time can be harmonized.

[Application Example 2] The image processing apparatus according to Application Example 1,
The detected subject includes a human face and a human face component;
The image detection unit detects an image corresponding to a constituent element of a person's face for a face area that is an image area corresponding to the detected person's face on the target image;
The image selection unit includes size defining information that associates a component of a person's face with a necessary face region size that is a size of the face region necessary for detecting an image corresponding to the component of the person's face. And an image processing device that selects the image data used for detecting an image corresponding to a constituent element of a person's face based on the size defining information.

  In this image processing apparatus, an image corresponding to a human face component is detected for a face area which is an image area corresponding to a detected human face on the target image, and the human face component is detected. Is used to detect the image corresponding to the component of the human face based on the size regulation information that associates the required face region size that is the size of the face region necessary for detecting the image corresponding to the component of the human face Since the image data to be selected is selected, it is possible to achieve harmony between the accuracy of the detection processing of the image corresponding to a specific type of subject in the target image and the processing time.

[Application Example 3] The image processing apparatus according to Application Example 2,
The image selection unit selects the image data having a first size as the image data used for detecting an image corresponding to a person's face, and on the target image represented by the image data having the first size. When the size of the face area is smaller than the required face area size associated with a human face component, the image data used for detection of an image corresponding to the human face component is the first image data. An image processing apparatus that selects the image data having a second size larger than the size of one.

  In this image processing apparatus, image data of the first size is selected as image data used for detection of an image corresponding to a human face, and the size of the face area on the target image represented by the image data of the first size is set. When the face size is smaller than the required face area size associated with the human face component, the image data used for detecting the image corresponding to the human face component has a second size larger than the first size. Since the image data is selected, it is possible to achieve harmony between the accuracy of the detection processing of the image corresponding to a specific type of subject in the target image and the processing time.

[Application Example 4] The image processing apparatus according to Application Example 1,
The detected subject includes a human face and a human face component;
The image selection unit selects the image data having a first size as the image data used for detecting an image corresponding to a person's face, and is used for detecting an image corresponding to a constituent element of the person's face. An image processing apparatus that selects the image data having a second size larger than the first size as image data.

  In this image processing apparatus, image data of the first size is selected as image data used for detecting an image corresponding to a person's face, and image data used for detecting an image corresponding to a constituent element of the person's face. Since image data having a second size larger than the first size is selected, it is possible to achieve harmony between the accuracy and processing time of image detection processing corresponding to a specific type of subject in the target image.

Application Example 5 The image processing apparatus according to any one of Application Example 2 to Application Example 4,
The human face component is at least one of a facial organ, a facial contour, and a facial expression.

  With this image processing apparatus, it is possible to achieve harmony between the accuracy and processing time of image detection processing corresponding to at least one of facial organs, facial contours, and facial expressions in the target image.

[Application Example 6] The image processing apparatus according to Application Example 5,
The image processing apparatus, wherein the facial organ is at least one of a right eye, a left eye, and a mouth.

  In this image processing apparatus, it is possible to achieve harmony between the accuracy and processing time of the detection processing of the image corresponding to at least one of the right eye, the left eye, and the mouth.

  The present invention can be realized in various modes. For example, an image processing method and apparatus, an image detection method and apparatus, a computer program for realizing the functions of these methods or apparatuses, and the computer program Can be realized in the form of, for example, a recording medium on which data is recorded, a data signal including the computer program and embodied in a carrier wave.

Next, embodiments of the present invention will be described in the following order based on examples.
A. Example:
A-1. Configuration of image processing device:
A-2. Subject detection processing:
B. Variations:

A. Example:
A-1. Configuration of image processing device:
FIG. 1 is an explanatory diagram schematically showing the configuration of a printer 100 as an image processing apparatus according to an embodiment of the present invention. The printer 100 of this embodiment is an ink jet color printer that supports so-called direct printing, in which an image is printed based on image data acquired from a memory card MC or the like. The printer 100 includes a CPU 110 that controls each unit of the printer 100, an internal memory 120 configured by a ROM and a RAM, an operation unit 140 configured by buttons and a touch panel, a display unit 150 configured by a liquid crystal display, and a printer. An engine 160 and a card interface (card I / F) 170 are provided. The printer 100 may further include an interface for performing data communication with other devices (for example, a digital still camera or a personal computer). Each component of the printer 100 is connected to each other via a bus.

  The printer engine 160 is a printing mechanism that performs printing based on print data. The card interface 170 is an interface for exchanging data with the memory card MC inserted into the card slot 172. In this embodiment, an image file including image data is stored in the memory card MC.

  The internal memory 120 stores an image processing unit 200, a display processing unit 310, and a print processing unit 320. The image processing unit 200 is a computer program for executing subject detection processing to be described later under a predetermined operating system. The display processing unit 310 is a display driver that controls the display unit 150 to display processing menus, messages, images, and the like on the display unit 150. The print processing unit 320 is a computer program for generating print data from image data, controlling the printer engine 160, and printing an image based on the print data. The CPU 110 implements the functions of these units by reading and executing these programs from the internal memory 120.

  The image processing unit 200 includes an image detection unit 210, a detected subject setting unit 220, a detection image data group generation unit 230, an image selection unit 240, and an information addition unit 250 as program modules. The image detection unit 210 detects an image corresponding to a specific type of subject in the image represented by the image data. The image detection unit 210 includes a determination target setting unit 211, an evaluation value calculation unit 212, a determination unit 213, and a region setting unit 214. The functions of these units will be described in detail in the description of subject detection processing described later.

  The detected subject setting unit 220 sets a detected subject that is the type of subject to be detected from the target image represented by the target image data. The detection image data group generation unit 230 generates a detection image data group including a plurality of image data having different sizes representing the target image. Since the detection image data group generation unit 230 generates and acquires the detection image data group, it corresponds to the image data group acquisition unit in the present invention. Based on the set detection subject, the image selection unit 240 selects one image data included in the detection image data group as image data used for detection of an image corresponding to the detection subject. The information adding unit 250 adds predetermined information (for example, information indicating the position of the face area) to the image file including the image data.

  The internal memory 120 also stores a size definition table SPT. FIG. 2 is an explanatory diagram showing an example of the contents of the size defining table SPT. As shown in FIG. 2, the size definition table SPT includes information (hereinafter also referred to as “size definition information”) that associates a detection target (detected subject) to be detected from a target image with a necessary face region size Sn. Here, the detection subject defined in the size definition table SPT of the present embodiment includes a person's face and human face components (an organ of the person's face (for example, eyes and mouth), a contour of the person's face (for example, cheek) Or facial expression of a person (for example, a facial part representing a smile)). Further, as will be described later, the detection of the image of the constituent element of the human face in the target image is executed for the image area corresponding to the detected human face (hereinafter referred to as “face area”). The necessary face area size Sn means the size of the face area necessary for detecting an image corresponding to a constituent element of a human face. In general, as the size of the face area increases, the detection accuracy of an image corresponding to a constituent element of a person's face as a detection subject improves, but the processing time required for detection increases. Conversely, the smaller the face area size, the shorter the processing time required for detection but the lower the detection accuracy. The necessary face area size Sn is experimentally set as the minimum face area size necessary to achieve a predetermined detection rate in the detection process of the image corresponding to the constituent elements of the human face as the detection subject. It is defined in the regulation table SPT.

  In the present embodiment, the face area is detected as a square-shaped image area. In the size defining table SPT, as the required face area size Sn, the length of one side of the face area is defined in units of the number of pixels. For example, in this embodiment, detection of an image corresponding to a facial organ needs to be performed on a face area having a size of 60 pixels × 60 pixels or more.

  The internal memory 120 also stores face learning data FLD, face organ learning data OLD, and facial expression data ED. The face learning data FLD, the facial organ learning data OLD, and the facial expression data ED are used for detection of the detected subject by the image detection unit 210. The contents of these data will be described in detail in the description of subject detection processing described later.

A-2. Subject detection processing:
FIG. 3 is a flowchart showing the flow of subject detection processing in the present embodiment. The subject detection processing in the present embodiment is processing for detecting an image corresponding to a detected subject set as the type of subject to be detected from the target image represented by the target image data.

  In step S110 (FIG. 3), the detected subject setting unit 220 (FIG. 1) sets the detected subject. The detected subject setting unit 220 sets a detected subject based on the type of image processing executed by the image processing unit 200. FIG. 4 is an explanatory diagram illustrating an example of the type of image processing executed by the image processing unit 200. FIG. 4 shows a state where a list of image processing menus provided by the image processing unit 200 is displayed on the display unit 150. In the present embodiment, three types of skin color correction, face deformation, and smile detection are set as the types of image processing executed by the image processing unit 200.

  Skin color correction is image processing for correcting the skin color of a person in a face region or an image region set based on the face region to a preferable skin color. In this embodiment, it is set that the detected subject when the skin color correction is executed is a human face, and when the user selects the skin color correction image processing via the operation unit 140, the detected subject setting unit A person's face 220 is set as a detection subject.

  Face deformation is image processing for deforming an image in a face area or an image area set based on the face area. If the detection accuracy of the face area is poor (that is, there is a large shift in the position, size, and tilt between the detected face area and the actual face image), the result of the face deformation process may be unnatural. There is. Therefore, in this embodiment, when face deformation is executed, an image corresponding to a facial organ is detected, and the facial area is adjusted based on the organ area corresponding to the detected organ, thereby improving the accuracy of the facial area. To improve. Accordingly, the detection subject when the face deformation is executed is set to be a human face and a face organ, and when the user selects face deformation image processing via the operation unit 140, the detection subject setting is performed. The unit 220 sets a human face and facial organs as detection subjects.

  Smile detection is image processing for detecting a smile image by detecting the outline of a face or facial organ within a face area or an image area set based on the face area. In this embodiment, the detection subject when smile detection is executed is set to be a human face, facial organs, facial contour and facial expression, and image processing for smile detection by the user via the operation unit 140 is performed. Is selected, the detected subject setting unit 220 sets a human face, facial organs, facial contour and facial expression as the detected subject.

  In the following description, it is assumed that face deformation is selected as image processing executed by the image processing unit 200, and a human face and facial organs are set as detection subjects.

  In step S120 (FIG. 3), the image processing unit 200 (FIG. 1) acquires image data representing an image to be subjected to subject detection processing. In the printer 100 of this embodiment, when the memory card MC is inserted into the card slot 172, thumbnail images of the image files stored in the memory card MC are displayed on the display unit 150. The user selects one or more images to be processed via the operation unit 140 while referring to the displayed thumbnail images. The image processing unit 200 acquires an image file including image data corresponding to one or more selected images from the memory card MC and stores it in a predetermined area of the internal memory 120. The acquired image data is referred to as original image data, and the image represented by the original image data is referred to as original image OImg.

  In step S130 (FIG. 3), the detection image data group generation unit 230 (FIG. 1) generates a detection image data group. FIG. 5 is an explanatory diagram illustrating an example of a detection image data group. As shown in FIG. 5, the detection image data group has higher resolution than the image data representing the face detection image FDImg used for face area detection (step S140 in FIG. 3) described later and the face detection image FDImg ( Image data representing two high-resolution images Iv and Ix). In this embodiment, the size of the face detection image FDImg is QVGA (horizontal 320 pixels × vertical 240 pixels), and the size of the high resolution image Iv is VGA (horizontal 640 pixels × vertical 480 pixels). The size of XGA is XGA (1024 horizontal pixels × 768 vertical pixels). The detection image data group generation unit 230 generates image data constituting the detection image data group by performing resolution conversion of the image data representing the original image OImg. The original image OImg, the face detection image FDImg, the high resolution image Iv, and the high resolution image Ix are all the same images except for the resolution (size), and correspond to the target image in the present invention.

  In step S140 (FIG. 3), the image detection unit 210 (FIG. 1) performs face area detection processing. The face area detection process is a process for detecting a face area FA corresponding to a face image in the target image. As described above, in this embodiment, a human face and a facial organ are set as detection subjects, and the image detection unit 210 first performs a face area detection process. FIG. 6 is a flowchart showing the flow of face area detection processing. FIG. 7 is an explanatory diagram showing an outline of the face area detection process. An example of the face detection image FDImg is shown at the top of FIG.

  In step S320 of the face area detection process (FIG. 6), the determination target setting unit 211 (FIG. 1) sets the size of the window SW used for setting the determination target image area JIA (described later) to an initial value. In step S330, the determination target setting unit 211 places the window SW at an initial position on the face detection image FDImg. In step S340, the determination target setting unit 211 determines whether the image area defined by the window SW arranged on the face detection image FDImg is an image area corresponding to the face image (hereinafter referred to as “face”). It is also set in a determination target image area JIA that is a target of determination. The middle part of FIG. 7 shows a state in which a window SW having an initial value size is arranged at the initial position on the face detection image FDImg, and an image area defined by the window SW is set as the determination target image area JIA. Yes. In this embodiment, as will be described later, the determination target image area JIA is sequentially set while changing the size and position of the square-shaped window SW, but the initial value of the size of the window SW is the horizontal size 240. The initial position of the window SW is such that the upper left vertex of the window SW overlaps the upper left vertex of the face detection image FDImg. Further, the window SW is arranged with the inclination of 0 degree. Note that the inclination of the window SW is clockwise from the reference state when the state in which the upward direction of the window SW coincides with the upward direction of the target image (face detection image FDImg) is set as the reference state (inclination = 0 degree). Means the rotation angle.

  In step S350 (FIG. 6), the evaluation value calculation unit 212 (FIG. 1) calculates a cumulative evaluation value Tv used for face determination for the determination target image area JIA based on the image data corresponding to the determination target image area JIA. In the present embodiment, face determination is executed for each specific face inclination set in advance. That is, for each specific face inclination, it is determined whether or not the determination target image area JIA is an image area corresponding to a face image inclined by the specific face inclination. Therefore, the cumulative evaluation value Tv is also calculated for each specific face inclination. Here, the specific face inclination means a preset face inclination value. The face inclination means the inclination (rotation angle) of the face image in the target image plane (in-plane). In the present embodiment, the clock from the reference state in the case where the inclination of the image or the image area is the reference state (inclination = 0 degree) when the upward direction of the image or image area coincides with the upward direction of the target image. It is assumed to be expressed by the rotation angle around. For example, for the face inclination, a state where the face image is positioned along the vertical direction of the target image (a state where the top of the head is facing upward and the chin is facing downward) is referred to as a reference state (face inclination = 0 degrees). In this case, it is represented by the clockwise rotation angle of the face image from the reference state. In the present embodiment, a total of 12 face inclinations (0 degrees, 30 degrees, 60 degrees,..., 330 degrees) obtained by increasing the inclination from 0 degrees to 30 degrees are set as specific face inclinations.

  FIG. 8 is an explanatory diagram showing an outline of a method of calculating the cumulative evaluation value Tv used for face determination. In this embodiment, N filters (filter 1 to filter N) are used to calculate the cumulative evaluation value Tv. The external shape of each filter has the same aspect ratio as that of the window SW (that is, has a square shape), and a positive region pa and a negative region ma are set for each filter. The evaluation value calculation unit 212 calculates the evaluation values vX (that is, v1 to vN) by sequentially applying the filter X (X = 1, 2,..., N) to the determination target image area JIA. Specifically, the evaluation value vX is determined from the sum of the luminance values of pixels located in the area on the determination target image area JIA corresponding to the plus area pa of the filter X, based on the determination target image area JIA corresponding to the minus area ma. This is a value obtained by subtracting the sum of the luminance values of the pixels located in the upper region.

  The calculated evaluation value vX is compared with a threshold thX (that is, th1 to thN) set corresponding to each evaluation value vX. In this embodiment, when the evaluation value vX is equal to or greater than the threshold thX, it is determined that the determination target image area JIA is an image area corresponding to the face image for the filter X, and the value “ 1 "is set. On the other hand, if the evaluation value vX is smaller than the threshold thX, it is determined that the determination target image area JIA is not an image area corresponding to the face image with respect to the filter X, and the value “0” is set as the output value of the filter X Is done. Weight coefficients WeX (that is, We1 to WeN) are set for each filter X, and the sum of products of output values and weight coefficients WeX for all filters is calculated as a cumulative evaluation value Tv.

  Note that the aspect of the filter X used for face determination, the threshold thX, the weighting coefficient WeX, and the threshold TH described later are preset as face learning data FLD (FIG. 1) for each of the 12 specific face inclinations. . The face learning data FLD in the present embodiment is data for calculating an evaluation value indicating the certainty that the determination target image area JIA is an image area corresponding to a face image.

  The face learning data FLD is generated by learning using a sample image. FIG. 9 is an explanatory diagram illustrating an example of a sample image used for learning. For learning, a face sample image group composed of a plurality of face sample images that are known in advance to be images corresponding to face images, and a plurality of information that is known in advance to be images that do not correspond to face images. A non-face sample image group composed of non-face sample images.

  Since generation of the face learning data FLD by learning is executed for each specific face inclination, as shown in FIG. 9, a face sample image group corresponding to each of 12 specific face inclinations is prepared. For example, the generation of the face learning data FLD for the specific face inclination of 0 degree is executed using the face sample image group and the non-face sample image group corresponding to the specific face inclination of 0 degree. The face learning data FLD is generated using a face sample image group and a non-face sample image group corresponding to a specific face inclination of 30 degrees.

  The face sample image group corresponding to each specific face inclination includes a plurality of face sample images in which the ratio of the size of the face image to the image size is within a predetermined value range and the inclination of the face image is equal to the specific face inclination. (Hereinafter also referred to as “basic face sample image FIo”). The face sample image group is an image obtained by enlarging and reducing the basic face sample image FIo with a predetermined magnification in a range from 1.2 times to 0.8 times with respect to at least one basic face sample image FIo (for example, FIG. 9). And images obtained by changing the face inclination of the basic face sample image FIo within a range of plus or minus 15 degrees (for example, the images FIc and FId in FIG. 9).

  Learning using a sample image is executed by, for example, a method using a neural network, a method using boosting (for example, adaboost), a method using a support vector machine, or the like. For example, when learning is performed by a method using a neural network, a face sample image group corresponding to a specific face inclination (see FIG. 9) for each filter X (that is, filters 1 to N, see FIG. 8). And the evaluation value vX (that is, v1 to vN) is calculated using all the sample images included in the non-face sample image group, and a threshold thX (that is, th1 to thN) that achieves a predetermined face detection rate is set. . Here, the face detection rate is the ratio of the number of face sample images determined to be an image corresponding to a face image by threshold determination based on the evaluation value vX with respect to the total number of face sample images constituting the face sample image group. Means.

  Next, the weight coefficient WeX (that is, We1 to WeN) set for each filter X is set to an initial value, and cumulative evaluation is performed on one sample image selected from the face sample image group and the non-face sample image group. A value Tv is calculated. As will be described later, in the face determination, when the cumulative evaluation value Tv calculated for a certain image is equal to or greater than a predetermined threshold TH, the image is determined to be an image corresponding to the face image. In learning, the value of the weighting coefficient WeX set for each filter X is corrected based on the correctness of the threshold determination result based on the cumulative evaluation value Tv calculated for the selected sample image (face sample image or non-face sample image). Is done. Thereafter, the selection of the sample image, the threshold value determination based on the cumulative evaluation value Tv calculated for the selected sample image, and the correction of the value of the weighting coefficient WeX based on the correctness of the determination result are the face sample image group and the non-face sample image. It is repeatedly executed for all the sample images included in the group. By executing the above processing for each specific face inclination, face learning data FLD for each specific face inclination is generated.

  When the cumulative evaluation value Tv is calculated for each specific face inclination for the determination target image area JIA (step S350 in FIG. 6), the determination unit 213 (FIG. 1) sets the cumulative evaluation value Tv for each specific face inclination. The threshold value TH is compared (step S360). When the cumulative evaluation value Tv is greater than or equal to the threshold value TH for a specific face inclination, the image detection unit 210 determines that the determination target image area JIA is an image area corresponding to a face image inclined by the specific face inclination. The position of the determination target image area JIA, that is, the coordinates of the currently set window SW and the specific face inclination are stored (step S370). On the other hand, if the cumulative evaluation value Tv is smaller than the threshold value TH for any specific face inclination, the process of step S370 is skipped.

  In step S380 (FIG. 6), the image detection unit 210 (FIG. 1) determines whether or not the entire face detection image FDImg has been scanned by the window SW having the currently set size. If it is determined that the entire face detection image FDImg has not been scanned yet, the determination target setting unit 211 (FIG. 1) moves the window SW in a predetermined direction by a predetermined movement amount (step S390). The lower part of FIG. 7 shows how the window SW has moved. In this embodiment, in step S390, the window SW is moved to the right by a movement amount corresponding to 20% of the horizontal size of the window SW. If the window SW is arranged at a position where it cannot move further to the right, the window SW returns to the left end of the face detection image FDImg in step S390, and the window SW has a size of 2 in the vertical direction. It is assumed that it moves downwards by the amount of movement of the percentage. When the window SW is arranged at a position where it cannot move further downward, the entire face detection image FDImg is scanned. After the movement of the window SW (step S390), the processes after the above-described step S340 are executed for the moved window SW.

  When it is determined in step S380 (FIG. 6) that the entire face detection image FDImg has been scanned by the window SW having the currently set size, it is determined whether or not all the predetermined sizes of the window SW have been used. (Step S400). In this embodiment, as the size of the window SW, in addition to the initial value (maximum size) of 240 horizontal pixels × vertical 240 pixels, horizontal 213 pixels × vertical 213 pixels, horizontal 178 pixels × vertical 178 pixels, horizontal 149 pixels × 149 pixels vertically, 124 pixels horizontally × 124 pixels vertically, 103 pixels horizontally × 103 pixels vertically, 86 pixels wide × 86 pixels high, 72 pixels wide × 72 pixels high, 60 pixels wide × 60 pixels high, 50 pixels wide × 50 vertical Total of 15 pixels: 41 pixels wide x 41 pixels wide, 35 pixels wide x 35 pixels wide, 29 pixels wide x 29 pixels wide, 24 pixels wide x 24 pixels high, 20 pixels wide x 20 pixels high (minimum size) The size of is set. If it is determined that there is a size of the window SW that is not yet used, the determination target setting unit 211 (FIG. 1) changes the size of the window SW to the next smaller size than the currently set size ( Step S410). That is, the size of the window SW is first set to the maximum size, and then changed to a smaller size in order. After the change of the size of the window SW (step S410), the processing after step S330 described above is executed for the window SW having the changed size.

  If it is determined in step S400 (FIG. 6) that all of the predetermined size of the window SW has been used, the region setting unit 214 (FIG. 1) executes a face region setting process (step S420). 10 and 11 are explanatory diagrams showing an outline of the face area setting process. The area setting unit 214 determines that the cumulative evaluation value Tv is equal to or greater than the threshold value TH in step S360 of FIG. 6 and determines the coordinates of the window SW (that is, the position and size of the window SW) and the specific face inclination stored in step S370. Based on this, the face area FA is set. Specifically, when the specific face inclination is 0 degree, the image area defined by the window SW (that is, the determination target image area JIA) is set as the face area FA as it is. On the other hand, when the specific face inclination is other than 0 degrees, the inclination of the window SW is matched with the specific face inclination (that is, the window SW is clocked by the specific face inclination around a predetermined point (for example, the center of gravity of the window SW). The image area defined by the window SW after the inclination is changed is set as the face area FA. For example, as shown in FIG. 10A, when it is determined that the cumulative evaluation value Tv is greater than or equal to the threshold value TH for a specific face inclination of 30 degrees, as shown in FIG. Is changed to 30 degrees, and the image area defined by the window SW after the inclination change is set as the face area FA.

  In step S370, when a plurality of windows SW that partially overlap each other with respect to a specific face inclination are stored, the region setting unit 214 sets a predetermined point in each window SW (for example, the center of gravity of the window SW). One new window (hereinafter also referred to as “average window AW”) having an average size of each window SW is set with the average coordinate of the coordinates as the center of gravity. For example, as shown in FIG. 11A, when four windows SW (SW1 to SW4) that partially overlap each other are stored, as shown in FIG. One average window AW having an average size of the sizes of the four windows SW is defined with the average coordinate of the coordinates of the center of gravity of the four windows SW as the center of gravity. At this time, as described above, when the specific face inclination is 0 degree, the image area defined by the average window AW is set as the face area FA as it is. On the other hand, when the specific face inclination is other than 0 degrees, the inclination of the average window AW is made to coincide with the specific face inclination (that is, the specific face inclination is centered on a predetermined point (for example, the center of gravity of the average window AW)). The image area defined by the average window AW after the inclination is changed is set as the face area FA (see FIG. 11C).

  As shown in FIG. 10, even when one window SW not overlapping with other windows SW is stored, a plurality of windows SW partially overlapping each other shown in FIG. 11 are stored. Similarly, one window SW itself can be interpreted as the average window AW.

  In this embodiment, an image obtained by enlarging and reducing the basic face sample image FIo at a predetermined magnification in the range from 1.2 times to 0.8 times in the face sample image group (see FIG. 9) used for learning. (For example, the images FIa and FIb in FIG. 9), the face image size is slightly larger or smaller than the basic face sample image FIo relative to the window SW size. The area FA can be detected. Therefore, in the present embodiment, only the 15 discrete sizes described above are set as the size of the window SW, but the face area FA can be detected for face images of any size. Similarly, in this embodiment, images obtained by changing the face inclination of the basic face sample image FIo within a range of plus or minus 15 degrees (for example, images FIc and FId in FIG. 9) are included in the face sample image group used for learning. Therefore, the face area FA can be detected even when the inclination of the face image with respect to the window SW is slightly different from the basic face sample image FIo. Therefore, in the present embodiment, only the 12 discrete inclination values described above are set as the specific face inclination, but the face area FA can be detected for face images having any inclination.

  If the face area FA is not detected in the face area detection process (step S140 in FIG. 3) (step S150: No), the subject detection process ends. On the other hand, when at least one face area FA is detected (step S150: Yes), the detected face area FA is set as the detection subject in step S110 and is also set as the detection subject. For a facial organ that is a constituent element of a person's face, an organ region corresponding to the facial organ is detected (step S200 described later). The image processing unit 200 (FIG. 1) selects one face area FA as an organ area detection target (step S160).

  In step S170 (FIG. 3), the image selection unit 240 (FIG. 1) uses the size of the selected face area FA (hereinafter also referred to as “face area size Sd”) and the size definition table SPT (FIG. 2) to determine the face shape. The required face area size Sn associated with the organ is compared. As shown in FIGS. 10B and 11C, the face area size Sd is the length of one side of a square-shaped window SW that defines the face area FA. Further, as shown in FIG. 2, the necessary face area size Sn associated with the facial organ is 60 pixels. When the face area size Sd is equal to or larger than the necessary face area size Sn (step S170: Yes), the image selection unit 240 detects a facial organ image in the face area FA on the face detection image FDImg. Therefore, the face detection image FDImg is selected as an image used for organ region detection (step S180).

  On the other hand, when the face area size Sd is smaller than the required face area size Sn (step S170: No), the image selection unit 240 detects the image of the facial organ in the face area FA on the face detection image FDImg. Since this is impossible, one of the high-resolution images larger than the face detection image FDImg is selected as an image used for organ region detection (step S190). Specifically, the image selection unit 240 selects an area (hereinafter referred to as an area on the high resolution image) corresponding to the face area FA on the face detection image FDImg from among the high resolution images included in the detection image data group (FIG. 5). As long as the size of “corresponding face area FAc” (hereinafter also referred to as “corresponding face area size Sdc”) is equal to or larger than the required face area size Sn, a high-resolution image of the minimum size is selected. For example, when the face area size Sd on the face detection image FDImg having the size of QVGA shown in FIG. 5 is 40 pixels, the face area size Sd is the necessary face area size Sn ( = 60 pixels, see FIG. 2). At this time, the size of the corresponding face area FAc (corresponding face area size Sdc) on the high resolution image Iv (FIG. 5) having the size of VGA is 80 pixels, which is equal to or larger than the necessary face area size Sn. Therefore, in this case, the image selection unit 240 selects a high-resolution image Iv having a VGA size as an image used for organ region detection.

  In step S200 (FIG. 3), the image detection unit 210 (FIG. 1) performs an organ region detection process. The organ area detection process is a process for detecting an image area corresponding to a facial organ in the face area FA or the corresponding face area FAc as an organ area. As described above, in this embodiment, three types of the right eye, the left eye, and the mouth are set as the types of facial organs. In the organ region detection process, the right eye region EA (r) corresponding to the right eye, The left eye area EA (l) corresponding to the left eye and the mouth area MA corresponding to the mouth are detected.

  FIG. 12 is a flowchart showing the flow of the organ region detection process. FIG. 13 is an explanatory diagram showing an outline of the organ region detection process. 13 shows an example of the face detection image FDImg selected in step S180 of FIG. 3 and an example of the high resolution image Iv selected in step S190 of FIG.

  In step S510 of the organ region detection process (FIG. 12), the image detection unit 210 (FIG. 1) generates an organ detection image ODImg. As illustrated in FIG. 13, when the face detection image FDImg is selected in step S180, the image detection unit 210 performs trimming and affine transformation of the image data of the face detection image FDImg, thereby performing the face area FA. An image with the inclination of 0 set to 0 degree is generated as an organ detection image ODImg. When the high-resolution image Iv is selected in step S190, the image detection unit 210 performs the trimming and affine transformation of the image data of the high-resolution image Iv, thereby setting the inclination of the corresponding face area FAc to 0 degrees. The set image is generated as an organ detection image ODImg. Since the organ detection image ODImg is generated in this way, the size (length of one side of the image) of the organ detection image ODImg is associated with the facial organ defined in the size definition table SPT (FIG. 2). The required face area size Sn or larger is obtained.

  The detection of the organ area from the organ detection image ODImg is performed in the same manner as the detection of the face area FA from the face detection image FDImg described above. That is, as shown in FIG. 13, a rectangular window SW is arranged on the organ detection image ODImg while changing its size and position (steps S520, S530, S580 to S610 in FIG. 12), and the arranged windows An image area defined by SW is set as a determination target image area JIA that is a target of determination as to whether or not it is an organ area corresponding to a facial organ (hereinafter also referred to as “organ determination”) (step of FIG. 12). S540). Note that the possible size and position of the window SW are determined based on the size of the organ detection image ODImg. For example, the value that can be taken as the horizontal length of the window SW is a value in a predetermined number of steps from the maximum size to the minimum size obtained by multiplying the horizontal length of the organ detection image ODImg by a predetermined number. Is set.

  When the determination target image area JIA is set, a cumulative evaluation value Tv used for organ determination is calculated for each organ (right eye, left eye, mouth) using the facial organ learning data OLD (FIG. 1) (FIG. 1). 12 step S550). The aspect of the filter X, the threshold thX, the weighting coefficient WeX, and the threshold TH (see FIG. 8) used for calculation of the cumulative evaluation value Tv and organ determination are defined in the facial organ learning data OLD. Note that the learning for setting the facial organ learning data OLD is performed by using a plurality of organ sample images that are known in advance to be images corresponding to the facial organs, similarly to the learning for setting the facial learning data FLD. It is executed using the constructed organ sample image group and the non-organ sample image group composed of a plurality of non-organ sample images that are known in advance not to correspond to facial organs.

  In the face area detection process (FIG. 6), calculation of the cumulative evaluation value Tv and face determination are executed for all specific face inclinations, whereas in the organ area detection process (FIG. 12), the cumulative evaluation value The calculation of Tv and the organ determination are performed only for the 0 degree organ inclination using the facial organ learning data OLD corresponding to the 0 degree organ inclination. This is because the inclination of the organ of the face is considered to substantially match the inclination of the entire face. However, in the detection of the organ region, the cumulative evaluation value Tv may be calculated for each predetermined inclination, and the organ determination may be performed for each predetermined inclination.

  When the cumulative evaluation value Tv calculated for each organ is equal to or greater than a predetermined threshold TH, the determination target image area JIA is determined to be an image area corresponding to the image of the organ of the face, and the position of the determination target image area JIA That is, the coordinates of the currently set window SW are stored (step S570 in FIG. 12). On the other hand, when the cumulative evaluation value Tv is smaller than the threshold value TH, the process of step S570 is skipped. After the entire organ detection image ODImg is scanned by the window SW for all the predetermined sizes of the window SW, the organ region setting process (step S620 in FIG. 12) by the region setting unit 214 (FIG. 1) is executed. Similar to the face area setting process (FIG. 11), the organ area setting process uses the average coordinates of the coordinates of predetermined points in each window SW when a plurality of windows SW that partially overlap each other are stored. In this process, one new window (average window AW) having an average size of each window SW is set, and an image area defined by the average window AW is set as an organ area.

  Through the above processing, in the organ detection image ODImg, the right eye area EA (r) that is an image area corresponding to the right eye, the left eye area EA (l) that is an image area corresponding to the left eye, and an image area corresponding to the mouth , Which is a mouth area MA.

  In step S210 (FIG. 3), the image detection unit 210 (FIG. 1) determines whether there is a face area FA that has not yet been selected in step S160. If it is determined that there is an unselected face area FA (step S210: No), the process returns to step S160, and one of the unselected face areas FA is selected, and the processes after step S170 are executed. . On the other hand, if it is determined that all the face areas FA have been selected (step S210: Yes), the process proceeds to step S220.

  In step S220 (FIG. 3), the information adding unit 250 (FIG. 1) determines the position (coordinates) of the detected face area FA and organ area (right eye area EA (r), left eye area EA (l), mouth area MA). ) Is added as attached information to the image file including the original image data. Thereby, an image file including original image data representing the original image OImg and information indicating the positions (coordinates) of the face area FA and the organ area in the original image OImg is generated.

  In the above description, it is assumed that face deformation is selected as the image processing executed by the image processing unit 200, and a human face and a facial organ are set as detection subjects. The subject detection process when the process (for example, smile detection) is selected is executed in the same manner. That is, when smile detection is selected as the image processing, a human face, facial organs, facial contours and facial expressions are set as detection subjects. Also in this case, the face area size Sd for the detected face area FA and the required face area size Sn (FIG. 2) associated with each detected object are compared, and the detected object is detected from the detection image data group. Image data to be used for detection is selected.

  Smile detection uses facial organ learning data OLD (FIG. 1) from facial area FA to detect the organ area, and also detects facial and facial organ contours and uses facial expression data ED (FIG. 1). Then, it can be performed by comprehensively evaluating the degree of opening of the mouth corner, the presence or absence of the nasal lip, the degree of lowering of the corners of the eyes, and the like. Techniques necessary for executing smile detection are described in Japanese Patent Application Laid-Open No. 2004-178593 and Yoshiki Soejima's “Research on Tracking of Moving Objects Considering Scene Change” February 15, 1998, and the like.

  As described above, in the subject detection process by the printer 100 of this embodiment, a detection subject that is the type of subject to be detected from the target image is set, and a plurality of pieces of image data having different sizes representing the target image are included. An image data group is generated, one image data included in the detection image data group is selected based on the set detected subject, and an image corresponding to the detected subject is detected using the selected image data. The Therefore, in the subject detection process by the printer 100 of the present embodiment, image data suitable for detection of the detected subject can be selected and executed, so that an image corresponding to a specific type of subject in the target image can be detected. Harmony between the accuracy of the detection process and the processing time can be achieved.

  More specifically, in subject detection processing by the printer 100 of this embodiment, detection of an organ area corresponding to a facial organ that is a constituent element of a human face is performed on the face area FA, and the detected subject (face Image data used for organ region detection is selected based on a size defining table SPT that associates a necessary face region size Sn, which is a size of a face region necessary for detecting an image corresponding to a detected subject. The For example, when the image data of the face detection image FDImg is used for the detection of the face area FA, and the size of the face area FA on the face detection image FDImg is smaller than the required face area size Sn associated with the facial organ. The image data of a high-resolution image (Iv or Ix in FIG. 5) having a size larger than the size of the face detection image FDImg is used for detection of the organ region. Therefore, in the subject detection process by the printer 100 of the present embodiment, image data suitable for detection of the detected subject can be selected and executed, so that an image corresponding to a specific type of subject in the target image can be detected. Harmony between the accuracy of the detection process and the processing time can be achieved.

B. Variations:
The present invention is not limited to the above-described examples and embodiments, and can be implemented in various modes without departing from the gist thereof. For example, the following modifications are possible.

B1. Modification 1:
The aspect of the size defining table SPT in the above embodiment is merely an example, and the aspect of the size defining table SPT can be variously modified. FIG. 14 is an explanatory diagram showing a modification of the size defining table. The size defining table SPTa shown in FIG. 14 associates the detected subject (detection target) with the size of the image data used for detecting the detected subject. Even when such a size defining table SPTa is used, it is possible to select image data suitable for detection of the detected subject and execute the detection processing. Therefore, detection processing of an image corresponding to a specific type of subject in the target image The accuracy and the processing time can be harmonized.

B2. Modification 2:
The example of the detected subject and the necessary face area size Sn (see FIG. 2) in the above embodiment are merely examples, and other types of subjects (for example, animals, plants, buildings, and vehicles) may be set as the detected subject. However, other values may be set as the necessary face area size Sn. Further, the detected subject is not necessarily set based on the image processing to be executed, and the detected subject may be set according to a direct instruction from the user, for example. Further, the relationship between the image processing and the detected subject described in the above embodiments is merely an example, and various modifications can be made. For example, when face deformation is selected as the image processing to be executed, in addition to the human face and the facial organ, the face and the contour of the facial organ may be set as the detection subject.

B3. Modification 3:
The configuration of the detection image data group in the above embodiment is merely an example, and the detection image data group may include image data of four or more sizes, or may include image data of only two types of sizes. Good. The size of the image data included in the detection image data group is not limited to the example shown in FIG. 5 and can be variously modified.

B4. Modification 4:
In the above embodiment, only the face learning data FLD and the facial organ learning data OLD corresponding to the front-facing image are prepared, but those corresponding to the right and left directions are prepared, and the face learning data FLD and the face organ learning data OLD are prepared. Detection of a face area or an organ area corresponding to an image or an image of a facial organ may be performed.

B5. Modification 5:
The aspects of the face area detection process (FIG. 6) and the organ area detection process (FIG. 12) in the above embodiment are merely examples, and various changes can be made. For example, the size of the face detection image FDImg (see FIG. 7) is not limited to 320 pixels × 240 pixels, and may be other sizes, or the original image OImg itself can be used as the face detection image FDImg. . Further, the size of the window SW used, the moving direction and the moving amount (moving pitch) of the window SW are not limited to those described above. In the above-described embodiment, the size of the face detection image FDImg is fixed, and a plurality of sizes of window SW are arranged on the face detection image FDImg, so that the determination target image area JIA having a plurality of sizes is set. However, a plurality of types of face detection images FDImg are generated, and a fixed-size window SW is arranged on the face detection image FDImg so that a determination target image area JIA having a plurality of sizes is set. Good.

  In the above-described embodiment, the face evaluation and the organ determination are performed by comparing the cumulative evaluation value Tv with the threshold value TH (see FIG. 8), but the face determination and the organ determination are performed using a plurality of discriminators. Other methods may be used. The learning method used for setting the face learning data FLD and the facial organ learning data OLD is also changed according to the face determination and organ determination methods. Further, the face determination and the organ determination are not necessarily performed by a determination method using learning, and may be performed by other methods such as pattern matching.

  In the above embodiment, 12 types of specific face inclinations in increments of 30 degrees are set. However, more types of specific face inclinations may be set, or fewer types of specific face inclinations are set. May be. In addition, the specific face inclination does not necessarily need to be set, and face determination may be performed for a 0 degree face inclination. In the above embodiment, the face sample image group includes an image obtained by enlarging or reducing the basic face sample image or a rotated image. However, the face sample image group does not necessarily include such an image. .

  In the above embodiment, the face determination (or organ determination) for the determination target image area JIA defined by the window SW of a certain size is determined to be an image area corresponding to the face image (or facial organ image). In the case where the window SW having a size smaller than the size by a predetermined ratio or more is arranged, it is assumed that the window SW is arranged avoiding the determination target image area JIA determined to be the image area corresponding to the face image. Also good. In this way, the processing speed can be increased.

  In the above embodiment, the image data stored in the memory card MC is set as the original image data. However, the original image data is not limited to the image data stored in the memory card MC, and is acquired via a network, for example. It may be image data.

  In the above embodiment, the right eye, the left eye, and the mouth are set as the types of facial organs, and the right eye area EA (r), the left eye area EA (l), and the mouth area MA are detected as organ areas. However, it is possible to change which organ of the face is set as the type of facial organ. For example, only one or two of the right eye, the left eye, and the mouth may be set as the types of facial organs. In addition to the right eye, left eye, and mouth, or instead of at least one of the right eye, left eye, and mouth, other organ types of the face (for example, nose or eyebrows) are set as the facial organ types, An area corresponding to such an organ image may be detected as the organ area.

  In the above embodiment, the face area FA and the organ area are rectangular areas, but the face area FA and the organ area may be areas having shapes other than the rectangle.

  In the above embodiment, the subject detection process by the printer 100 as the image processing apparatus has been described. However, part or all of the process is performed by another type of image processing apparatus such as a personal computer, a digital still camera, or a digital video camera. It is good also as a thing. The printer 100 is not limited to an ink jet printer, and may be another type of printer, such as a laser printer or a sublimation printer.

  In the above embodiment, a part of the configuration realized by hardware may be replaced with software, and conversely, a part of the configuration realized by software may be replaced by hardware.

  In addition, when part or all of the functions of the present invention are realized by software, the software (computer program) can be provided in a form stored in a computer-readable recording medium. In the present invention, the “computer-readable recording medium” is not limited to a portable recording medium such as a flexible disk or a CD-ROM, but an internal storage device in a computer such as various RAMs and ROMs, a hard disk, and the like. An external storage device fixed to the computer is also included.

1 is an explanatory diagram schematically illustrating a configuration of a printer 100 as an image processing apparatus according to an embodiment of the present invention. It is explanatory drawing which shows an example of the content of the size prescription | regulation table SPT. It is a flowchart which shows the flow of the object detection process in a present Example. 6 is an explanatory diagram illustrating an example of a type of image processing executed by an image processing unit 200. FIG. It is explanatory drawing which shows an example of the image data group for a detection. It is a flowchart which shows the flow of a face area | region detection process. It is explanatory drawing which shows the outline | summary of a face area | region detection process. It is explanatory drawing which shows the outline | summary of the calculation method of accumulated evaluation value Tv used for face determination. It is explanatory drawing which shows an example of the sample image used for learning. It is explanatory drawing which shows the outline | summary of a face area | region setting process. It is explanatory drawing which shows the outline | summary of a face area | region setting process. It is a flowchart which shows the flow of an organ area | region detection process. It is explanatory drawing which shows the outline | summary of an organ area | region detection process. It is explanatory drawing which shows the modification of a size prescription | regulation table.

Explanation of symbols

100 ... Printer 110 ... CPU
DESCRIPTION OF SYMBOLS 120 ... Internal memory 140 ... Operation part 150 ... Display part 160 ... Printer engine 170 ... Card interface 172 ... Card slot 200 ... Image processing part 210 ... Image detection part 211 ... Evaluation target setting part 212 ... Evaluation value calculation part 213 ... Determination part 214 ... Area setting unit 220 ... Detected subject setting unit 230 ... Detection image data group generation unit 240 ... Image selection unit 250 ... Information addition unit 310 ... Display processing unit 320 ... Print processing unit

Claims (8)

An image processing apparatus,
A detection subject setting unit that sets a detection subject that is a type of subject to be detected from the target image;
An image data group acquisition unit for acquiring an image data group including a plurality of image data of different sizes representing the target image;
An image selection unit that selects one of the image data included in the image data group based on the set detection subject;
An image detection apparatus comprising: an image detection unit that detects an image corresponding to the set detection subject using the selected image data. The image processing apparatus according to claim 1,
The detected subject includes a human face and a human face component;
The image detection unit detects an image corresponding to a constituent element of a person's face for a face area that is an image area corresponding to the detected person's face on the target image;
The image selection unit includes size defining information that associates a component of a person's face with a necessary face region size that is a size of the face region necessary for detecting an image corresponding to the component of the person's face. And an image processing device that selects the image data used for detecting an image corresponding to a constituent element of a person's face based on the size defining information. The image processing apparatus according to claim 2,
The image selection unit selects the image data having a first size as the image data used for detecting an image corresponding to a person's face, and on the target image represented by the image data having the first size. When the size of the face area is smaller than the required face area size associated with a human face component, the image data used for detection of an image corresponding to the human face component is the first image data. An image processing apparatus that selects the image data having a second size larger than the size of one. The image processing apparatus according to claim 1,
The detected subject includes a human face and a human face component;
The image selection unit selects the image data having a first size as the image data used for detecting an image corresponding to a person's face, and is used for detecting an image corresponding to a constituent element of the person's face. An image processing apparatus that selects the image data having a second size larger than the first size as image data. An image processing apparatus according to any one of claims 2 to 4,
The human face component is at least one of a facial organ, a facial contour, and a facial expression. The image processing apparatus according to claim 5,
The image processing apparatus, wherein the facial organ is at least one of a right eye, a left eye, and a mouth. An image processing method comprising:
(A) setting a detected subject that is a type of subject to be detected from the target image;
(B) obtaining a group of image data including a plurality of image data having different sizes representing the target image;
(C) selecting one of the image data included in the image data group based on the set detected subject;
(D) using the selected image data, detecting an image corresponding to the set detection subject, and an image processing method. A computer program for image processing,
A detection subject setting function for setting a detection subject that is a type of subject to be detected from the target image;
An image data group acquisition function for acquiring an image data group including a plurality of image data of different sizes representing the target image;
An image selection function for selecting one of the image data included in the image data group based on the set detection subject;
The computer program which makes a computer implement | achieve the image detection function which detects the image corresponding to the set said to-be-detected object using the selected said image data.

Images