JP2009230557A - Object detection device, object detection method, object detection program, and printer - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To reduce the time for detecting an object, and to more reduce processing and increase speed than before. <P>SOLUTION: An object detection device for detecting a prescribed object from an input image is provided with: an edge acquisition part for setting a detection window on the input image, and for acquiring the edge quantity of each of a plurality of regions in the set detection window; and an object decision part for comparing the acquired edge quantity of each region between the prescribed regions, and for, when the comparison result satisfies prescribed conditions, deciding the presence/absence of the object by using the image in the set detection window. <P>COPYRIGHT: (C)2010,JPO&INPIT

Description

  The present invention relates to an object detection apparatus, an object detection method, an object detection program, and a printing apparatus.

A technique for detecting a target image (object) from an input image is known.
Also, a peripheral region is set for the red-eye candidate region, and an average value of the edge of each pixel in the set region (an output value when the Sobel filter is applied to each pixel) is calculated, and this average value is a threshold value There is known an image processing apparatus that determines whether or not a red-eye candidate area is a red-eye area based on whether or not it is larger (see Patent Document 1).
JP 2007-4455 A

  Conventionally, when executing detection processing of an object such as a face image from an input image, an attempt has been made to acquire a detection result without omission by attempting to detect the object for all locations in the input image. However, in the above detection processing, there is a demand for reduction and speeding up of the processing along with the accuracy of the detection, and all the locations in the input image are set as object detection targets in the same manner as in the past. The purpose of reducing processing and speeding up cannot be sufficiently achieved. Note that the above document 1 uses the average value of the edges of each pixel in the peripheral area when specifying whether or not the red-eye candidate area is a red-eye area. It did not present the usage method of.

  The present invention has been made in view of the above problems, and an object detection device and an object capable of reducing processing and speeding up processing compared to conventional methods while ensuring high-precision detection when detecting an object from an input image. It is an object to provide a detection method, an object detection program, and a printing apparatus.

  In order to achieve the above object, the present invention provides an object detection device for detecting a predetermined object from an input image, wherein a detection window is set on the input image and a plurality of regions in the set detection window are set. An edge acquisition unit that acquires the edge amount of each region and the acquired edge amount of each region are compared between predetermined regions, and when the result of the comparison satisfies a predetermined condition, the set detection window And an object determination unit that executes the determination of the presence / absence of the object for the image inside. According to the present invention, depending on the comparison result of the edge amount between predetermined regions in the detection window set in the input image, execution of the presence / absence determination of the object is avoided for the detection window. That is, as a result of the edge amount comparison, the presence / absence determination is not performed for a detection window that is estimated to have no object-like image, so that the processing amount and the processing time can be reduced without degrading the accuracy of object detection from the input image. Is reduced.

  The edge acquisition unit is an area set in the detection window, and when the detection window includes a face image, a first area set as an area corresponding to a predetermined organ of the face image, and set in the detection window When the detection window includes a face image, an edge amount is acquired for each of the second region set as a region corresponding to a predetermined skin portion other than the organ of the face image, and the object determination is performed When the edge amount of the first region is larger than the edge amount of the second region, the unit may determine the presence or absence of a face image as an object for the image in the set detection window. According to the configuration, the face image is compared with the image in the detection window only when it is estimated from the result of the edge amount comparison between the first area and the second area that a face-like image exists in the detection window. The presence / absence determination is performed. Therefore, it can be prevented that the presence / absence determination of a face image having a large amount of processing is performed in vain.

  The first area includes an area preliminarily estimated to correspond to the eyes of the face image when the detection window includes a face image and an area previously estimated to correspond to the mouth of the face image when the detection window includes the face image. It may be included. The eyes and mouth in the face image have a large amount of edges. Therefore, according to the said structure, it can be judged appropriately whether the presence or absence determination of a face image should be performed based on the comparison result of the edge amount of a 1st area | region and a 2nd area | region.

  If the edge amount of the first region is less than a certain value, the image in the detection window is considered to be an image with a small luminance difference in the first place, and in this case, the detection window actually includes a face image. Even in such a case, the edge amount of the first region> the edge amount of the second region may not hold. Therefore, when the edge amount of the first region is equal to or smaller than the predetermined threshold value, the object determination unit sets the face image for the image in the set detection window regardless of the edge amount of the second region. It may be determined whether or not there is. According to this configuration, it is possible to prevent omission of object detection due to the fact that the determination based on the comparison of the edge amounts does not substantially function.

  The edge acquisition unit rotates the detection window a plurality of times by a predetermined angle at a position on the input image where the detection window is set while maintaining the position and size of each region with respect to the detection window. The edge amount of each region is acquired for each state, and the object determination unit performs the comparison for each state in which the detection window is rotated, and based on the result of each comparison, It may be determined whether or not to perform the determination of the presence / absence of an object for the image. According to this configuration, the possibility of existence of an image that seems to be an object that can exist at various angles on the input image is estimated, and the presence / absence determination of the object is performed in the detection window in which the image that seems to be an object exists. be able to.

  The edge acquisition unit repeatedly sets the detection window on the input image while changing at least one of the position and size of the detection window in the input image while maintaining the position and size of each region with respect to the detection window. The edge amount of each area is acquired every time a detection window is set, and the object determination unit may perform the comparison for each set detection window. According to this configuration, the possibility of existence of an image that seems to be an object that can exist at various sizes at various positions on the input image is estimated, and the position and size of the detection window that is presumed that an image that seems to be an object exists. The presence / absence determination of the object can be executed based on the above.

  The technical idea of the present invention is that, in addition to the above-described invention of the object detection device, the invention of the object detection method including each processing step performed by each unit included in the above-described object detection device, and each unit included in the above-described object detection device It can also be understood as an invention of an object detection program for causing a computer to execute a function corresponding to the above. A printing apparatus that detects a predetermined object from an input image and performs printing based on the input image, sets a detection window on the input image, and sets a plurality of regions in the set detection window An edge acquisition unit that acquires the edge amount of each region and the acquired edge amount of each region are compared between predetermined regions, and when the result of the comparison satisfies a predetermined condition, the set detection window An object determination unit that determines whether or not the object exists for an image in the image, and the input image according to correction information determined based on an image in the detection window determined by the object determination unit to have an object It is possible to grasp a configuration including a print control unit that corrects at least a part of the image and performs printing based on the corrected input image. .

Embodiments of the present invention will be described in the following order.
1. General printer configuration:
2. Processing by printer:
2-1. Determining whether or not an object exists:
2-2. From object presence determination to printing:
3. Variations:

1. General printer configuration:
FIG. 1 schematically shows a configuration of a printer 10 corresponding to an example of an object detection apparatus and a printing apparatus of the present invention. The printer 10 is a color inkjet printer that supports so-called direct printing, in which an image is printed based on image data acquired from a recording medium (for example, a memory card MC). The printer 10 includes a CPU 11 that controls each unit of the printer 10, an internal memory 12 configured by, for example, a ROM and a RAM, an operation unit 14 configured by buttons and a touch panel, a display unit 15 configured by a liquid crystal display, A printer engine 16, a card interface (card I / F) 17, and an I / F unit 13 for exchanging information with an external device such as a PC, a server, or a digital still camera are provided. Each component of the printer 10 is connected to each other via a bus.

  The printer engine 16 is a printing mechanism that performs printing based on print data. The card I / F 17 is an I / F for exchanging data with the memory card MC inserted into the card slot 172. Image data is stored in the memory card MC, and the printer 10 can acquire the image data stored in the memory card MC via the card I / F 17. In addition to the memory card MC, various media can be used as recording media for providing image data. Of course, in addition to the recording medium, the printer 10 can also input image data from the external device connected via the I / F unit 13. The printer 10 may be a printing device for consumers, or may be a business printing device for DPE (so-called minilab machine). The operation unit 14 and the display unit 15 may be an input operation unit (such as a mouse or a keyboard) or a display separate from the main body of the printer 10. The printer 10 can also input print data from a PC or server connected via the I / F unit 13.

  The internal memory 12 stores an object detection unit 20, an image correction unit 30, a display processing unit 40, and a print processing unit 50. The object detection unit 20 and the image correction unit 30 are computer programs for executing object detection processing, image correction processing, and the like described below under a predetermined operating system. The display processing unit 40 is a display driver that controls the display unit 15 to display a processing menu and a message on the display unit 15. The print processing unit 50 is a computer program for generating print data from image data, controlling the printer engine 16 and printing an image based on the print data. The CPU 11 implements the functions of these units by reading and executing these programs from the internal memory 12.

  The object detection unit 20 includes a detection window setting unit 21, an edge amount calculation unit 22, a necessity determination unit 23, and a detection execution unit 24 as program modules. The image correction unit 30 includes a correction information determination unit 31 and a correction execution unit 32 as program modules. The detection window setting unit 21 and the edge amount calculation unit 22 correspond to an edge acquisition unit in the claims. The necessity determination part 23 and the detection execution part 24 correspond to the object determination part said to a claim. The image correction unit 30 and the print processing unit 50 correspond to a print control unit in the claims. The functions of these units will be described later. Further, the internal memory 12 stores various data and programs such as the edge amount calculation region definition filter 14b, the edge detection filters 14c and 14d, and the neural network NN. The printer 10 may be a so-called multifunction machine having various functions such as a copy function and a scanner function in addition to the print function.

2. Processing by printer:
2-1. Determining whether or not an object exists:
FIG. 2 is a flowchart showing processing executed by the printer 10 in this embodiment. In step S (hereinafter, step notation is omitted) 100, the object detection unit 20 obtains image data D representing an image (input image) to be subjected to image processing from a predetermined recording medium such as a memory card MC. get. That is, the object detection unit 20 acquires an input image. Of course, if the printer 10 has a hard disk drive (HDD), the object detection unit 20 can acquire the image data D stored in the HDD, and the I / F unit 13 can be used as described above. The image data D can be acquired from the external device connected via the network. That is, the user operates the operation unit 14 while referring to the user interface (UI) screen displayed on the display unit 15 to arbitrarily select the image data D as the input image and print the selected image data D. When the instruction is given, the object detection unit 20 acquires the image data D related to the selection from a recording medium or the like.

  The image data D is bitmap data composed of a plurality of pixels, and each pixel is expressed by a combination of gradations of RGB channels (for example, 256 gradations of 0 to 255). The image data D may be compressed when recorded on a recording medium or the like, or the color of each pixel may be expressed in another color space. In these cases, the object detection unit 20 executes the development of the image data D and the conversion of the color space to acquire the image data D as RGB bitmap data.

  In S200, the object detection unit 20 reduces the image data D. When the object detection process described later is performed on the image data D with the original image size as a target, the processing load is large. Therefore, the object detection unit 20 reduces the image size by reducing the number of pixels of the image data D, and acquires the reduced image data. For example, the object detection unit 20 acquires image data DR obtained by reducing the image data D to a QVGA (Quarter Video Graphics Array) size (320 pixels × 240 pixels). In the present embodiment, the image data DR is also referred to as an input image as appropriate.

In S300, the object detection unit 20 converts the image data DR into a gray image. That is, the object detection unit 20 converts the RGB data of each pixel of the image data DR into a luminance value Y (0 to 255), and generates image data DR as a monochrome image having one luminance value Y for each pixel. The luminance value Y can generally be obtained by adding R, G, and B with a predetermined weight.
In this embodiment, S200 is not essential. Therefore, when S200 is not executed, the object detection unit 20 executes S300 and further S400 and S500 described later for the image data D. Further, S300 (conversion of image data DR or image data D to a gray image) is a process that is performed in advance in consideration of the convenience of an object detection process to be described later. However, it is not essential to perform S300 in advance. You may skip.

  In S400, the object detection unit 20 executes an object detection process. Schematically, the object detection unit 20 sets a detection window SW in the image data DR (or image data D), acquires edge amounts in the areas for a plurality of areas in the detection window SW, and When the comparison result of the edge amount between the two satisfies the predetermined condition, the process of determining the presence / absence of the object for the image in the detection window SW is repeated for each detection window SW. In the present embodiment, as an example, the description will be made assuming that the object is a human face image. However, objects that can be detected using the configuration of the present invention are not limited to human face images, and various objects such as artifacts, living things, natural objects, and landscapes can be detected as objects. .

FIG. 3 is a flowchart showing details of S400.
In S410, the detection window setting unit 21 of the object detection unit 20 sets one detection window SW in the image data DR. The setting method of the detection window SW is not particularly limited, but the detection window setting unit 21 sets the detection window SW as follows as an example.
FIG. 4 shows how the detection window SW is set in the image data DR. In the first S410, the detection window setting unit 21 has a rectangular detection window SW (two-dot chain line) of a predetermined size including a plurality of pixels at the head position in the image (for example, the upper left corner of the image). Set. The detection window setting unit 21 moves the detection window SW from the position where the detection window SW has been set up to a predetermined distance (a predetermined number of pixels) in the horizontal direction and / or the vertical direction of the image every time S410 is performed for the second time and thereafter. And one new detection window SW is set at the position of the movement destination. When the detection window setting unit 21 repeatedly sets the detection window SW while moving the detection window SW to the final position of the image data DR (for example, the lower right corner position of the image) while maintaining the size of the detection window SW. Returning to the head position, the detection window SW is set.

  When the detection window SW is returned to the head position, the detection window setting unit 21 sets the detection window SW having a smaller rectangular size than before. Thereafter, in the same manner as described above, the detection window setting unit 21 sets the detection window SW at each position while moving the detection window SW to the final position of the image data DR while maintaining the size of the detection window SW. The detection window setting unit 21 repeats such movement and setting of the detection window SW while stepwise reducing the size of the detection window SW by a predetermined number of times. In this manner, every time one detection window SW is set in S410, the processing after S420 is performed.

  In S420, the edge amount calculation unit 22 calculates the edge amount in each region (first region and second region) in the detection window SW set in the latest S410. First, the edge amount calculation unit 22 reads the edge amount calculation region definition filter 14b from the internal memory 12, and applies the edge amount calculation region definition filter 14b to the image data in the set detection window SW. The edge amount calculation region definition filter 14b is a rectangular filter similar to the detection window SW, and defines a first region and a second region that are targets of edge amount calculation.

  FIG. 5 shows an example of the edge amount calculation region definition filter 14b. As shown in FIG. 5, the edge amount calculation area definition filter 14b defines a first area A1 and a second area A2 in the filter. When the edge amount calculation region definition filter 14b is applied to the detection window SW, the first region A1 is estimated to include a predetermined organ of the face image if the detection window SW includes the face image. This is an area preset in position and size. In the present embodiment, the first area A1 has left and right eyes (or left and right eyes and right and left eyebrows) when the detection window SW to which the edge amount calculation area definition filter 14b is applied includes a face image. And the mouth region estimated to include the mouth of the face image when the detection window SW to which the edge amount calculation region definition filter 14b is applied includes the face image. On the other hand, if the detection window SW includes a face image when the edge amount calculation area definition filter 14b is applied to the detection window SW, the second area A2 is a predetermined skin portion other than the above organ of the face image. This is a region preset to a position and a size estimated to be included. In the present embodiment, the second area A2 includes left and right cheek eyes that are estimated to correspond to the left and right cheeks of the face image when the detection window SW to which the edge amount calculation area definition filter 14b is applied includes the face image. Consists of regions.

  The definition form of 1st area | region A1 and 2nd area | region A2 is not restricted to what was shown in FIG. 5, The eye area | region in 1st area | region A1 may be the area | region isolate | separated into two, 1st area | region A1 is It may consist of either the eye area or the mouth area. The second area A2 may be an area estimated to correspond to the position of the forehead of the face image, for example, instead of the left and right cheek areas. The edge amount calculation area definition filter 14b defines these areas so that the total area of the first area A1 is equal to the total area of the second area A2. Further, the positional relationship and the size of each of the first area A1 and the second area A2 with respect to the rectangle of the edge amount calculation area definition filter 14b are constant. Therefore, when the edge amount calculation area definition filter 14b is reduced or enlarged The first area A1 and the second area A2 are similarly reduced or enlarged.

  The edge amount calculation unit 22 enlarges or reduces the edge amount calculation region definition filter 14b as necessary so that the edge amount calculation region definition filter 14b matches the size of the detection window SW set in the latest S410. Above, the edge amount calculation region definition filter 14b is superimposed on the set detection window SW. As a result, the first area A1 and the second area A2 are set in the detection window SW. Accordingly, the positions and sizes of the first area A1 and the second area A2 with respect to the detection window SW are always constant even if the detection window SW is moved and reduced as described above. Next, the edge amount calculation unit 22 reads the edge detection filter 14c from the internal memory 12, applies the edge detection filter 14c to each pixel of the image data DR belonging to the first area A1, and each pixel in the first area A1. Detect the edge amount.

  FIG. 6 shows a state in which the edge detection filter 14c is applied to a partial region of the image data DR included in the first region A1. The edge detection filter 14c is, for example, a 3 × 3 matrix filter. The edge amount calculation unit 22 applies the center value of the edge detection filter 14c to the pixel of interest, and integrates the results of multiplying each value of the filter 14c and each pixel value (luminance value Y) of the image data DR. Thus, the edge amount of the target pixel is detected. The edge amount of each pixel belonging to the first area A1 is detected by applying the edge detection filter 14c to all the pixels in the first area A1 sequentially as the target pixel. In FIG. 6, the edge amount is shown only for the central nine pixels in a partial region of the image data DR. The edge amount calculation unit 22 calculates the sum of the magnitudes (absolute values) of the edge amounts of the pixels in the first region A1, and acquires the sum as the edge amount of the first region A1.

  The edge amount calculation unit 22 similarly applies the edge filter 14c to each pixel of the image data DR belonging to the second region A2 to detect the edge amount of each pixel in the second region A2, and the second region A2 The sum of the edge amounts (absolute values) of each pixel is acquired as the edge amount of the second area A2. As apparent from FIG. 6, the edge detection filter 14c shown in FIG. 6 is a filter that can detect the edge amount according to the luminance change in the vertical direction of the image. Here, in human face images, the area of eyes, eyebrows, and mouth is mainly formed by lines that face in the horizontal direction of the face (the left-right direction of the face), and thus the luminance change in the vertical direction of the face is abundant. On the other hand, the luminance change of the skin part such as the cheek is poor compared to the eye, eyebrows and mouth area. Therefore, if a face image exists in the detection window SW set in the image data DR, a large amount of edge is detected from the first area A1 in the detection window SW, while from the second area A2 It is expected that only a small amount of edge will be detected.

  Therefore, in S430 (FIG. 3), the necessity determining unit 23 compares the edge amount of the first region A1 acquired in the latest S420 with the edge amount of the second region A2, and the edge amount of the first region A1> When the edge amount of the second area A2 is established, the process proceeds to S440. That is, if the edge amount of the first area A1 is larger than the edge amount of the second area A2, the image in the detection window SW has a high facial appearance, and it is necessary to determine whether or not there is a face image. The determination unit 23 proceeds to S440. On the other hand, if the edge amount of the first region A1 ≦ the edge amount of the second region A2, the necessity determination unit 23 skips S440 and S450 and proceeds to S460. That is, when the condition that the edge amount of the first region A1 is larger than the edge amount of the second region A2 is not satisfied, the face-likeness of the image in the detection window SW is low, and it is necessary to determine the presence or absence of the face image. Since it can be said that there is not, the necessity determination unit 23 proceeds to S460. In FIG. 6, the edge detection filter 14d is also illustrated, but a method of using the edge detection filter 14d will be described later.

2-2. From object presence determination to printing:
In S440, the detection execution unit 24 determines the presence / absence of a face image (face determination) for the image in the detection window SW set in the latest S410. If it is determined that the face image exists, the process proceeds to S450. If it is determined that the face image does not exist, S450 is skipped and the process proceeds to S460. The detection execution unit 24 can employ any method in S440 as long as it can determine whether or not a face image exists. In this embodiment, for example, the detection execution unit 24 performs determination using a neural network NN.

  FIG. 7 is a flowchart showing details of S440 executed by the detection execution unit 24. In S441, the detection execution unit 24 obtains image data (window image data) XD composed of pixels in the detection window SW set in the latest S410, and in S442, calculates a plurality of feature amounts from the window image data XD. To do. For these feature amounts, various filters are applied to the window image data XD, and feature amounts (average value, maximum value, minimum value, standard deviation, etc.) indicating image features such as luminance average and contrast in the filter are applied. ) Is obtained.

  FIG. 8 shows how the feature amount is calculated from the window image data XD. In the same figure, a number of filters FT having different relative sizes and positions with respect to the image data XD are prepared, and each filter FT is sequentially applied to the window image data XD, and image characteristics in each filter FT are obtained. Based on this, a plurality of feature amounts CA, CA, CA... Are calculated. In FIG. 8, each rectangle in the window image data XD is called a filter FT. If the feature quantities CA, CA, CA... Can be calculated, the detection execution unit 24 inputs the feature quantities CA, CA, CA... Into the prepared neural network NN in S443, and a face image exists as an output thereof. A determination result of whether or not is calculated is calculated.

  FIG. 9 shows an example of the structure of the neural network NN. The neural network NN has a basic structure in which the value of the unit U in the subsequent layer is determined by a linear combination of the values of the unit U in the previous layer (the suffix i is the identification number of the unit U in the previous layer). Further, the value obtained by the linear combination may be used as the value of the unit U of the next layer as it is, but the value obtained by the linear combination is converted by a non-linear function such as a hyperbolic tangent function, for example. By determining the value of U, non-linear characteristics may be provided. The neural network NN is composed of an outermost input layer and output layer, and an intermediate layer sandwiched between them. Each feature quantity CA, CA, CA... Can be input to the input layer of the neural network NN, and an output value K (value normalized to 0 to 1) can be output from the output layer. Yes. In S444, for example, if the output value K of the neural network NN is 0.5 or more, the detection execution unit 24 determines that the value indicates that a face image exists in the window image data XD, and proceeds to S450. On the other hand, if the output value K is less than 0.5, the detection execution unit 24 determines that the face image data does not exist in the window image data XD, and proceeds to S460.

  FIG. 10 schematically shows how the neural network NN is constructed by learning. In the present embodiment, by learning the neural network NN by the error back propagation method, the number of units U, the size of the weight w at the time of linear combination between the units U, and the bias b are determined. The value is optimized. In learning by the back propagation method, first, the magnitude of the weight w and the value of the bias b at the time of linear combination between the units U are initially set to appropriate values. Then, for learning image data for which it is known whether or not a face image exists, feature amounts CA, CA, CA... Are calculated in the same procedure as S442 and S443, and the feature amounts CA, CA, CA. Input to the set neural network NN and obtain the output value K. In this embodiment, it is desirable that 1 is output as the output value K for learning image data in which a face image exists, and 0 is output as the output value K for learning image data in which no face image exists. It is desirable to be done. However, since the weight w and the value of the bias b at the time of linear combination between the units U are merely set to appropriate values, there is an error between the actual output value K and the ideal value. Will occur. The weight w and the bias b for each unit U that minimizes such an error are calculated using a numerical optimization method such as a gradient method. The error as described above is propagated from the subsequent layer to the previous layer, and the weight w and the bias b are sequentially optimized for the subsequent unit U.

A face image exists in the window image data XD by preparing in advance in the internal memory 12 a neural network NN that has been optimized by performing such learning using a plurality of learning image data. It can be determined based on the feature quantities CA, CA, CA.
In S450 (FIG. 3), the detection execution unit 24 determines the position of the detection window SW (for example, the center position of the detection window SW on the image data DR) determined that the face image exists in the latest S440, and the detection window SW. Are recorded in a predetermined area of the internal memory 12. Thus, the act of recording the position and size of the detection window SW corresponds to an example of the face image detection act.

  In S460, the detection window setting unit 21 still sets the detection window SW by moving the detection window SW and further reducing its size under the concept of the detection window SW setting method described with reference to FIG. If there is room to do so, the process returns to S410, and one new detection window SW is set on the image data DR. On the other hand, when the reduction of the detection window SW is overlapped by the predetermined number of times and all the possible detection window SW settings are completed, the object detection unit 20 ends the process of S400.

  In S500 (FIG. 2), the correction information determination unit 31 of the image correction unit 30 determines correction information used for correction of the input image. Examples of correction for an input image include brightness correction, contrast correction, saturation correction, and correction for a specific memory color. In the present embodiment, when a face image is detected in S400, the correction information determination unit 31 determines correction information based on at least the face image. Specifically, when the position and size information of the detection window SW detected as a face image is recorded in the internal memory 12, the correction information determination unit 31 uses the detection window SW from the image data DR. The image data in the range indicated by the position and size information (referred to as face image data) is extracted. The image data DR from which face image data is extracted may be image data DR after conversion to a gray image, or image data DR immediately after S200 before conversion to a gray image. The correction information determination unit 31 calculates correction information (correction parameters) based on the face image data. For example, the correction information determination unit 31 calculates an average value of luminance in the face image data, calculates a difference between the average value and a predetermined target value, and uses the calculated difference as correction information.

  In S600, the correction execution unit 32 corrects at least a part of the image data D acquired in S100 based on the correction information determined in S500. For example, if the correction information is the difference between the average value of the luminance in the face image data and the target value as described above, the luminance corresponding to the difference is set in an area (image corresponding to the face image data on the image data D). The position and the size with respect to the data D are added to the respective pixels in the region where the relationship between the position and the size of the face image data with respect to the image data DR is equal. As a result, the brightness of the face image on the image data D can be improved. Further, the curve degree of the tone curve may be determined based on the magnitude of the difference, and each pixel value of the image data D may be corrected using the tone curve. Of course, the type of correction information determined in S500 and the type of correction performed in S600 are not limited to those described above.

  In S700, the print processing unit 50 controls the printer engine 16 to print the input image. That is, the print processing unit 50 performs necessary processes such as resolution conversion process, color conversion process, and halftone process on the corrected image data D to generate print data. The generated print data is supplied from the print processing unit 50 to the printer engine 16, and the printer engine 16 executes printing based on the print data. Thereby, the printing of the input image is completed.

  As described above, according to the present embodiment, when a detection window SW is set for an input image and the presence or absence of an object (face image) is determined for each detection window SW, a face image exists in the detection window SW. Sets the region estimated to correspond to the organ position such as eyes and mouth of the face image as the first region A1 in the detection window SW, and when a face image exists, the region such as the eyes and mouth of the face image exists. A region estimated to correspond to a predetermined skin portion other than the organ is set as a second region A2 in the detection window SW. Then, the edge amount in the first region A1 is compared with the edge amount in the second region A2. Then, face determination is performed on the image in the detection window SW on the condition that the edge amount of the first region A1 is larger than the edge amount of the second region A2. On the other hand, when the condition is not satisfied, the detection window SW is newly set at another position on the input image without performing face determination for the detection window SW. That is to say, of the locations on the input image, locations where there is no need to determine the presence or absence of an object are excluded from the subject of the determination. Therefore, it is possible to greatly reduce the overall amount of processing for repeating the setting of the detection window SW and the determination of the presence / absence of an object in the input image without degrading the object detection accuracy. As a result, the object detection processing is very fast. It becomes.

3. Variations:
FIG. 11 is a flowchart showing details of S400 (FIG. 2) executed by the object detection unit 20, and shows an example different from FIG. The object detection unit 20 may perform the processing of the flowchart of FIG. 11 instead of the follow chart of FIG. Since S810 and S850 to S880 in FIG. 11 are the same as S410 and S430 to S460 in FIG. In S820, the edge amount calculation unit 22 acquires the edge amount of the first region A1 in the detection window SW set in the latest S810. In S830, the necessity determination unit 23 determines whether or not the edge amount of the first area A1 acquired in the latest S820 is larger than a predetermined threshold Th, and the edge amount is larger than the threshold Th. In step S840, the flow advances to step S840. On the other hand, if the edge amount is equal to or smaller than the threshold value Th, the flow advances to step S860. In S840, the edge amount calculation unit 22 acquires the edge amount of the second region A2 in the detection window SW set in the latest S810.

  In this way, in the example of FIG. 11, before comparing the edge amount of the first region A1 and the edge amount of the second region A2, whether or not the edge amount of the first region A1 is larger than the threshold value Th. If the edge amount of the first area A1 is equal to or less than the threshold value Th, the face determination (S860) is performed regardless of the edge amount of the second area A2. When the edge amount of the first region A1 does not reach a certain amount, this is considered to be an image with poor luminance change in the entire detection window SW (or the entire input image), for example, a backlight image. Because. That is, in an image with a generally small change in brightness, even if a face image exists, the amount of edges in the eyes, mouth, and the like is also small. Therefore, the edge amount of the first region A1 and the edge amount of the second region A2 This is because it is difficult to determine whether the image in the detection window SW is a face based on the comparison result. As described above, when the edge amount of the first area A1 is equal to or less than the threshold value Th, face determination is performed, and even an input image with poor luminance change such as a backlight image is a target for face determination. The face image can be detected. The threshold value Th used for comparison with the edge amount of the first area A1 is recorded in advance in the internal memory 12, for example. The threshold value Th is determined in advance based on, for example, the edge amount detected from the eye part and the mouth part in a plurality of sample face images photographed in a backlight state.

The setting of the detection window SW in S410 (FIG. 3) and S810 (FIG. 11) is performed a plurality of times while repeating the movement and size change (reduction) of the detection window SW in the image data DR.
Further, in S420 and S820, the edge amount calculation unit 22 applies (superimposes) the edge amount calculation region definition filter 14b to the detection window SW set in the latest S410 and S810, and sets the detection window SW to the image data DR. The edge amount of the first region A1 and the edge amount of the second region A2 may be acquired for each rotated state (including a state where the rotation angle is 0 degree) by rotating the image several times at a predetermined angle. . If the edge amount of the second area A2 is also acquired in S820, the process of S840 is substantially unnecessary.

  FIG. 12 shows a state where the edge amount calculation unit 22 rotates one detection window SW set in the image data DR a plurality of times. FIG. 12 shows each state when the detection window SW is rotated by 90 degrees with the center position maintained. With this rotation, the edge amount calculation area definition filter 14b also rotates, so that the relative positional relationship and size of the first area A1 and the second area A2 with respect to the detection window SW are always maintained. The edge amount calculation unit 22 acquires the edge amount of the first region A1 and the edge amount of the second region A2 for each rotation angle (0 degree, 90 degrees, 180 degrees, 270 degrees) of the detection window SW. When the edge amount is acquired from the first area A1 and the second area A2 in the detection window SW rotated 90 degrees, the image data including the pixels included in each of the first area A1 and the second area A2 is the 90. The edge detection filter 14c is applied to the image data of the first area A1 and the image data of the second area A2 after rotating 90 degrees in the direction to cancel the rotation of the degree, and the edge amount of each pixel is detected. Similarly, when acquiring the edge amount from each area of the detection window SW rotated 180 degrees, the edge detection is performed after rotating the image data included in each area so as to cancel the rotation of 180 degrees. When the edge amount is acquired from each area of the detection window SW rotated by 270 degrees by applying the filter 14c, the image data included in each area is rotated so as to cancel the rotation of 270 degrees. The edge detection filter 14c is applied to detect the edge amount of each pixel.

  If the edge amount of the first region A1 and the edge amount of the second region A2 are obtained for each rotation angle of the detection window SW as described above, in S430 and S850, the necessity determining unit 23 sets the first amount for each rotation angle. The edge amount of the area A1 is compared with the edge amount of the second area A2. In any of the comparisons for each rotation angle, if the edge amount of the first area A1> the edge amount of the second area A2 holds, the process proceeds to S440 and S860. That is, if the edge amount of the first region A1 is larger than the edge amount of the second region A2 in any of the rotation states (0 degree, 90 degrees, 180 degrees, 270 degrees) of the detection window SW, the detection window SW is within the detection window SW. It can be said that there is something that looks like a face image. In that case, it is determined that it is necessary to perform face determination, and face determinations in S440 and S860 are performed. In addition to the 90 degrees described above, various angles such as 30 degrees, 45 degrees, and 60 degrees are conceivable as the angle per rotation when the detection window SW is rotated.

  As shown in FIG. 13, by adopting a configuration that determines the necessity of face determination by comparing the edge amount of the first region A1 and the edge amount of the second region A2 for each rotation angle of the detection window SW as described above. Even if the angle of the face image on the image data DR is various, such as when the top and bottom of the face image faces the left and right of the image data DR, or the top and bottom of the face image is opposite to the top and bottom of the image data DR. Thus, the possibility of the presence of such a face image can be estimated, which can lead to execution of face determination. In S830, the necessity determination unit 23 determines “No” when the edge amount of each first region A1 obtained for each rotation angle as described above is equal to or less than the threshold value Th, and performs S860. On the other hand, if any of the edge amounts of the first regions A1 obtained for each rotation angle is larger than the threshold value Th, a determination of “Yes” is made (S840 is substantially skipped). Proceed to S850.

  When performing face determination according to the comparison result of the edge amount for each rotation angle of the detection window SW, the detection execution unit 24 calculates each of the filters for calculating the feature amount CA from the window image data XD. The FT is also appropriately rotated around the center position of the window image data XD. Then, the detection execution unit 24 determines the presence / absence of the face image using the neural network NN for each rotation state based on the feature amount CA obtained from the filter FT for each rotation state. As a result, it is possible to accurately detect face images that may exist at various angles in the detection window SW.

  FIG. 14 shows an example of the edge amount calculation region definition filter 14b, which is an example different from the configuration shown in FIG. In the edge amount calculation area definition filter 14b in FIG. 14, a nose area AN is defined in addition to the eye area, the mouth area, and the left and right cheek areas. When the edge amount calculation region definition filter 14b is applied to the detection window SW, the nose region AN is assumed to include the nose of the face image if the detection window SW includes the face image. This is an area set in advance. The edge amount calculation unit 22 reads the edge amount calculation region definition filter 14b defining the nose region NA from the internal memory 12 in place of the edge amount calculation region definition filter 14b shown in FIG. 5, and detects the detection window on the image data DR. It can be applied to SW. In this case, the edge amount calculation unit 22 applies the edge detection filter 14c to the image data of the eye region, the mouth region, and the left and right cheek regions as described above to acquire the edge amount, and the image data in the nose region NA. The edge amount is also obtained from.

  The edge amount calculation unit 22 detects the edge amount by applying the edge detection filter 14d (see FIG. 6) to each pixel belonging to the nose area AN, and the edge amount of each pixel in the nose area AN is large. The sum of the absolute values (absolute values) is acquired as the edge amount of the nose area AN. As is apparent from FIG. 6, the edge detection filter 14d is a filter capable of detecting an edge amount corresponding to a luminance change in the left-right direction of the image. In the vicinity of the nose of the face image, there are edges of the nose extending in the vertical direction of the face, lines of the left and right noses, etc., and therefore the luminance change in the horizontal direction of the face is also rich. Therefore, if a face image exists in the detection window SW set in the image data DR, it is predicted that a large amount of edge is detected from the nose region AN in the detection window SW.

  As described above, when the edge amount is acquired from the eye region, the mouth region, the left and right cheek regions, and the nose region AN, the necessity determination unit 23 determines whether or not to perform the face determination in consideration of the edge amount of the nose region AN. Judge. For example, the necessity determining unit 23 determines the edge amount of the first area A1 (eye area + mouth area)> the edge amount of the second area A2 (left and right cheek areas) and the edge amount of the nose area AN> predetermined. When the threshold value is satisfied, it is determined that face determination is performed (proceeds to S440 and S860). It is assumed that threshold value data used for comparison with the edge amount of the nose area AN is also recorded in the internal memory 12 or the like in advance. Alternatively, the nose area AN may be a part of the first area A1. That is, the sum of the edge amounts acquired from the eye region, mouth region, and nose region by the edge amount calculation unit 22 is used as the edge amount of the first region A1, and the necessity determination unit 23 determines that the edge amount of the first region A1> the first amount. It is determined whether the edge amount of the two areas A2 holds. Alternatively, when the area of the first area A1 = the area of the second area A2 does not hold due to including the nose area AN, the necessity determining unit 23 determines the edge amount of the first area A1 including the nose area AN. The average value may be compared with the average value of the edge amounts of the second area A2, and it may be determined that the face determination is performed when the average value applied to the first area A1 is larger. In this way, by considering the edge amount of the nose region AN in addition to the edge amount of the skin region other than the eye region, the mouth region, and the facial organ, the facial appearance of the image of the detection window SW is evaluated, Only the detection window SW having an image having a high facial appearance can be set as a face determination target.

A method other than the method using the neural network NN, which is face determination that can be executed by the detection execution unit 24 in S440 and S860, will be described.
FIG. 15 schematically illustrates an example of a face determination technique performed by the detection execution unit 24. In the example shown in FIG. 15, determination means in which a plurality of determination devices J, J. The determination means including the plurality of determination devices J mentioned here may be a substantial device or a program having the following determination functions corresponding to the plurality of determination devices J. Each of the determiners J, J,... Inputs one or more feature quantities CA, CA, CA,... Of different types (for example, different filters FT) from the window image data XD subjected to face determination, Outputs a positive or negative judgment, respectively. Each of the determiners J, J... Has a determination algorithm such as a feature size comparison between CA, CA, CA... And a threshold determination, and the window image data XD is a face (positive) or not a face ( Execute the original determination of NO). Each of the next stage determiners J, J... Is connected to the positive output of the previous stage determiners J, J..., And the output of the previous stage determiners J, J. Only when this is the case, the next stage decision devices J, J... In any stage, the face determination is terminated at the point of time when no is output, and a determination that no face image exists is output (“No” in S440 and S860). On the other hand, when all of the determination devices J, J... At each stage output a positive value, the face determination is terminated and a determination that a face image exists is output (“Yes” in S440 and S860).

  FIG. 16 shows the determination characteristics of the determination means. This figure shows a feature space defined by the axes of the feature values CA, CA, CA,... Used in each of the above-described determiners J, J ..., and finally determines that a face image exists. Coordinates in the feature amount space represented by a combination of feature amounts CA, CA, CA... Obtained from the window image data XD. Since the window image data XD determined that the face image exists has a certain feature, it can be considered that a distribution is seen in a certain region in the feature amount space. Each of the determiners J, J... Generates a boundary plane in such a feature amount space, and among the spaces partitioned by the boundary plane, the determination target feature amounts CA, CA, CA. If the coordinate of exists, positive is output. Therefore, by connecting the determination devices J, J... In a cascade, it is possible to gradually narrow down the space in which the positive output is made. According to the plurality of boundary planes, it is possible to accurately determine the distribution having a complicated shape.

  In the above, an example in which the object detection device and the object detection method of the present invention are embodied by the printer 10 has been described. For example, the object detection device and the object detection method may be a computer, a digital still camera, a scanner, or the like. It may be realized in the image equipment. Further, the present invention can be realized not only in a printer that outputs image processing results on a printing sheet but also in an apparatus that outputs image processing results on a display such as a photo viewer. Furthermore, the present invention can also be applied to an ATM (Automated Teller Machine) that performs person authentication. Furthermore, the face determination performed by the detection execution unit 24 can use various determination methods in the feature amount space of the feature amount described above. For example, a support vector machine may be used.

FIG. 2 is a block diagram illustrating a schematic configuration of a printer. 4 is a flowchart illustrating processing executed by a printer. It is a flowchart which shows the detail of an object detection process. It is a figure which shows a mode that a detection window is set. It is a figure which shows an example of an edge amount calculation area | region definition filter. It is a figure which shows the mode of edge detection. It is a flowchart which shows the face determination using a neural network. It is a figure which shows a mode that a feature-value is calculated from window image data. It is a figure which shows an example of the structure of a neural network. It is a figure which shows typically a mode that a neural network is learned. It is a flowchart which shows the other example of the detail of an object detection process. It is a figure which shows a mode that the detection window was rotated. It is a figure which shows an example of an input image. It is a figure which shows the other example of an edge amount calculation area | region definition filter. It is a figure which shows typically the face determination process concerning a modification. It is a figure which shows the determination characteristic of the face determination process concerning a modification.

Explanation of symbols

DESCRIPTION OF SYMBOLS 10 ... Printer, 11 ... CPU, 12 ... Internal memory, 14b ... Edge amount calculation area definition filter, 14c, 14d ... Edge detection filter, 16 ... Printer engine, 17 ... Card I / F, 20 ... Object detection part, 21 ... Detection window setting unit, 22 ... Edge amount calculation unit, 23 ... Necessity determination unit, 24 ... Detection execution unit, 30 ... Image correction unit, 31 ... Correction information determination unit, 32 ... Correction execution unit, 50 ... Print processing unit, 172 ... Card slot

Claims (9)

An object detection device for detecting a predetermined object from an input image,
An edge acquisition unit that sets a detection window on the input image and acquires an edge amount of each area for a plurality of areas in the set detection window;
The obtained edge amount of each area is compared between predetermined areas, and when the result of the comparison satisfies a predetermined condition, the presence / absence of the object is determined for the image in the set detection window. An object detection apparatus comprising: an object determination unit to be executed.   The edge acquisition unit is an area set in the detection window, and when the detection window includes a face image, a first area set as an area corresponding to a predetermined organ of the face image, and set in the detection window When the detection window includes a face image, an edge amount is acquired for each of the second region set as a region corresponding to a predetermined skin portion other than the organ of the face image, and the object determination is performed The unit determines whether or not there is a face image as an object for the image in the set detection window when the edge amount of the first region is larger than the edge amount of the second region. The object detection device according to 1.   The first area includes an area preliminarily estimated to correspond to the eyes of the face image when the detection window includes a face image and an area previously estimated to correspond to the mouth of the face image when the detection window includes the face image. The object detection apparatus according to claim 2, further comprising:   When the edge amount of the first area is equal to or less than a predetermined threshold value, the object determination unit is configured to detect the face image for the image within the set detection window regardless of the edge amount of the second area. The object detection apparatus according to claim 2, wherein presence or absence is determined.   The edge acquisition unit rotates the detection window a plurality of times by a predetermined angle at a position on the input image where the detection window is set while maintaining the position and size of each region with respect to the detection window. The edge amount of each region is acquired for each state, and the object determination unit performs the comparison for each state in which the detection window is rotated, and based on the result of each comparison, The object detection apparatus according to claim 1, wherein it is determined whether or not to determine whether or not there is an object for the image.   The edge acquisition unit repeatedly sets the detection window on the input image while changing at least one of the position and size of the detection window in the input image while maintaining the position and size of each region with respect to the detection window. 6. Each time the detection window is set, the edge amount of each region is acquired, and the object determination unit performs the comparison for each set detection window. An object detection device according to claim 1. An object detection method for detecting a predetermined object from an input image,
An edge acquisition step of setting a detection window on the input image and acquiring an edge amount of each area for a plurality of areas in the set detection window;
The obtained edge amount of each area is compared between predetermined areas, and when the result of the comparison satisfies a predetermined condition, the presence / absence of the object is determined for the image in the set detection window. An object detection method comprising: an object determination step to be executed. An object detection program for causing a computer to execute processing for detecting a predetermined object from an input image,
An edge acquisition function for setting the detection window on the input image and acquiring the edge amount of each area for a plurality of areas in the set detection window;
The obtained edge amount of each area is compared between predetermined areas, and when the result of the comparison satisfies a predetermined condition, the presence / absence of the object is determined for the image in the set detection window. An object detection program for executing an object determination function to be executed. A printing apparatus that detects a predetermined object from an input image and executes printing based on the input image,
An edge acquisition unit that sets a detection window on the input image and acquires an edge amount of each area for a plurality of areas in the set detection window;
The obtained edge amount of each area is compared between predetermined areas, and when the result of the comparison satisfies a predetermined condition, the presence / absence of the object is determined for the image in the set detection window. An object determination unit to be executed;
At least a part of the input image is corrected according to the correction information determined based on the image in the detection window determined that the object is determined by the object determination unit, and printing is performed based on the corrected input image. A printing apparatus comprising: a printing control unit.

Images