JP4478087B2 - Image processing apparatus and method, and image processing program - Google Patents

Description

  The present invention relates to an image processing apparatus and method and an image processing program, and more particularly to an apparatus, method and program for estimating a color temperature of a photographing light source based on given image data.

  In recent years, images recorded on photographic film using digital exposure, that is, photographic film such as negative film and color reversal film, are photoelectrically read, and the read image is converted into digital data and modulated according to this image data. Digital photo printers that expose a photosensitive material with the recording light and print an image (latent image) on photographic paper have been put into practical use. Image data obtained by a digital camera or camera-equipped mobile phone is also printed on photographic paper by a digital photo printer.

  The state of image data handled by a digital photo printer is not necessarily uniform. The image is obtained by being photographed under various photographing light sources such as daylight, a fluorescent lamp, and tungsten light. Therefore, when creating a print from image data, if printing is performed without image processing, the color based on the photographing light source may be reflected in the print.

  White balance correction (gray balance correction) is a correction process for obtaining an image print that is not affected by the photographic light source.

  White balance correction expresses a subject image in an appropriate color by controlling the color balance of the image represented by the image data (for example, a skin-colored subject image (for example, a person representing a white subject image in white) This is a process of correcting the image data so that the face portion is expressed by skin color). White balance correction is generally performed at the time of shooting. At the time of shooting, a color temperature (or light source type) corresponding to the shooting situation is set in the digital camera. White balance correction is performed according to the set color temperature or light source type, and the image data subjected to white balance correction is stored in a memory card or the like.

  When correcting the white balance of an image represented by digital image data, not at the time of shooting, the color temperature at the time of shooting of the image represented by the digital image data is determined based on information obtained from the digital image data. (Estimate) is necessary.

Patent Document 1 uses a black body locus (a skin black body locus and a gray black body locus) on a chromaticity diagram for determination (estimation) of a light source color temperature. The R and B components of the RGB values of the image obtained by the digital camera are multiplied by predetermined coefficients α1 and α2, and the values are converted into chromaticity values r and b. Pixels that are in the vicinity of the skin black body locus represented on the chromaticity diagram are detected as skin color candidate pixels, and pixels that are in the vicinity of the gray black body locus are detected as gray candidate pixels. The predetermined coefficients α1 and α2 are set so that the number of skin color candidate pixels entering the vicinity of the skin black body locus and the number of gray candidate pixels entering the vicinity of the gray black body locus are maximized, or the average color temperature of the skin color candidate pixel group. It is optimized so that the difference in the average color temperature of the gray pixel group is minimized. The light source color temperature is determined (estimated) using the color temperature of the skin pixel detected based on the skin black body locus and the color temperature of the gray pixel detected using the gray black body locus.
JP 2003-209856 A

  As described above, in Patent Document 1, the R component and the B component of the RGB value of an image obtained by a digital camera are multiplied by predetermined coefficients α1, α2, and the values are converted into chromaticity values r, b. If the value obtained in this way is a pixel that is in the vicinity of the skin black body locus represented on the chromaticity diagram, the pixel is a skin color pixel, and if the pixel is in the vicinity of the gray black body locus, that pixel. Are gray pixels. However, the image before the white balance correction is often not in color balance. For this reason, there is a limit in detecting flesh color pixels and gray pixels using only color information as in Patent Document 1. For example, even if there is no skin pixel and gray pixel in the image, if there is a color pair (for example, pink and purple) that is in a relative color relationship between skin and gray, this is treated as a skin color pixel or gray pixel. It is conceivable that a false detection will occur. As a result, the light source color temperature obtained according to the skin black body locus and the gray black body locus may not be an appropriate light source color temperature, and there is a possibility that proper white balance correction may fail.

  Accordingly, the present invention has been made in view of the above circumstances, and has an object to accurately estimate a light source color temperature by accurately detecting a gray pixel included in an image represented by given image data. And

  According to a first aspect of the present invention, there is provided a skin image area detecting means for detecting a skin image area included in an image represented by given image data based on an image shape or an image structure, and the skin image area detecting means. Based on the color information obtained from the pixels constituting the skin image area detected by the above, gray pixel detection means for detecting gray pixels included in the image represented by the image data, and detection by the gray pixel detection means It is characterized by comprising color temperature estimating means for estimating the color temperature of the photographic light source based on the gray pixels.

  The image processing method according to the first aspect of the present invention detects a skin image area included in an image represented by given image data based on an image shape or an image structure, and obtains it from pixels constituting the detected skin image area. A gray pixel included in the image represented by the image data is detected based on the color information, and the color temperature of the photographing light source is estimated based on the detected gray pixel.

  A program for image processing according to a first invention includes a skin image region detection process for detecting a skin image region included in an image represented by image data given to a computer based on an image shape or an image structure, A gray pixel detection process for detecting a gray pixel included in the image represented by the image data based on color information obtained from the pixels constituting the skin image area detected by the skin image area detection process; Based on the gray pixel detected by the pixel detection process, the color temperature estimation process for estimating the color temperature of the photographing light source is executed.

  The image processing apparatus and method and the image processing program according to the first invention are suitable for white balance correction (gray balance correction, color balance correction). White balance correction is to correct the ratio of RGB values of pixels constituting an image so that a white or gray image has a predetermined color, and a light source at the time of image data acquisition (that is, at the time of shooting). By estimating the color temperature, a value for white balance correction (white balance correction value or white balance correction coefficient) can be calculated (determined). When used as a white balance correction device, the image processing device according to the first aspect of the invention calculates a white balance correction value (correction coefficient) based on the color temperature of the photographing light source estimated by the color temperature estimation means. A white balance correction unit that performs white balance correction on the image data using the correction value calculated by the balance correction value calculation unit and the white balance correction value calculation unit is further provided.

  If the gray pixel can be detected directly based on the image data, the light source color temperature can be estimated based on the detected gray pixel, and the white balance correction value is calculated based on the estimated light source color temperature. can do. However, it is difficult to detect gray pixels directly based on image data. Therefore, the image processing apparatus according to the first invention first detects a skin image area based on given image data, and detects gray pixels based on color information based on the skin image area. Note that a gray pixel is a pixel having a value (chromaticity value) on or near a black body locus (gray black body locus) of gray if it has an appropriate white balance.

  The skin image region means an image region (image portion) having a specific shape or structure including skin color, which is included in an image represented by given image data. The image processing apparatus of the first invention focuses on the shape or structure of an image and detects a skin image region in an image represented by given image data.

  Examples of the skin image area detected based on the image shape or the image structure include an image representing a person's face, body, hand, foot, and the like. Since these image regions have a specific shape or structure, they can be detected (partitioned or extracted) from given image data (in image data representing the entire image).

  Preferably, the skin image area is detected based on the face image area of the person. The face image area of a person is generally an image area including skin color. Further, a subject captured by a digital still camera or the like often includes a human face. By detecting the face image area of a person based on the shape or structure, the skin image area can be reliably acquired.

  Each of the pixels constituting the skin image area includes color information. For example, 8-bit data for each of R, G, and B exists for each pixel. Based on the color information obtained from the pixels constituting the skin image area, the gray pixels included in the image represented by the image data are detected.

  In one embodiment, the gray pixel detecting means multiplies a representative value (for example, an average value, a mode value, or a median value) of color information obtained from each of the pixels constituting the skin image region by a predetermined coefficient. Obtained by the first chromaticity value converting means, the first chromaticity value converting means for converting the coefficient multiplied skin region representative value calculated by the first coefficient multiplying means, and the first chromaticity value converting means. Determining means for determining whether the coefficient-multiplied skin region representative chromaticity value is in the vicinity range of the skin-black body locus, and the determining means determines that the coefficient-multiplied skin region representative chromaticity value is in the vicinity of the skin-black body locus. Temporary color temperature estimation that estimates the color temperature corresponding to the point on the skin black body locus closest to the coefficient-multiplied skin region representative chromaticity value as the provisional color temperature when it is determined to exist in the range Means, the image represented by the image data Second coefficient multiplication means for multiplying the color information obtained from the formed pixels by the predetermined coefficient that causes the coefficient multiplication skin region representative chromaticity value to be present in the vicinity of the skin black body locus, and the second coefficient multiplication Second chromaticity value conversion means for converting a plurality of coefficient multiplication pixel values obtained by the means into chromaticity values, and among the pixels constituting the image represented by the image data, the second chromaticity A coefficient multiplication pixel chromaticity value obtained by the value conversion means identifies a pixel that falls within a range near a point on a gray black body locus corresponding to the provisional color temperature estimated by the provisional color temperature estimation means; The identified pixel is detected as a gray pixel.

  Since the skin image area detected by the skin image area detecting means is detected based on the image shape or the image structure, if the white balance (color balance) of the given image data is appropriate, the skin image area is detected. The obtained value (chromaticity value) exists on or near the skin black body locus corresponding to the predetermined color temperature. When the white balance of the given image data is not appropriate, the value (chromaticity value) obtained from the skin image area deviates from or near the skin black body locus. When the coefficient multiplication skin region representative chromaticity value obtained by the first coefficient multiplication means is multiplied by a predetermined coefficient and obtained by the first chromaticity value conversion means is in the vicinity range of the skin black body locus, before the coefficient multiplication Pixels (virtual pixels) having a representative value (average value, mode value, or median value) of color information obtained from pixels constituting the skin image area of the skin image region are multiplied by a predetermined coefficient. It exists on the black body locus or in the vicinity thereof.

  When it is determined that the coefficient-multiplied skin region representative chromaticity value exists in the vicinity range of the skin-black body locus, it corresponds to the point on the skin-black body locus that is closest to the coefficient-multiplied skin region representative chromaticity value. The color temperature is estimated to be a provisional color temperature. The above-mentioned predetermined coefficient (predetermined coefficient for causing the coefficient-multiplied skin region representative chromaticity value to exist in the vicinity of the skin black body locus) is multiplied by the color information obtained from the pixels constituting the image represented by the image data. , And then converted into chromaticity values (calculation of coefficient multiplication pixel chromaticity values). Among the pixels constituting the image represented by the image data, the coefficient multiplication pixel chromaticity value is in the vicinity of a point on the gray black body locus corresponding to the provisional color temperature estimated by the provisional color temperature estimation means Pixels that fall within the range are specified, and the specified pixels are detected as gray pixels.

  Based on the detected gray pixel, the color temperature of the photographing light source is estimated. For example, the color temperature of each of the detected identified gray pixels is calculated, and the average value of the calculated color temperatures is set as the light source color temperature.

  According to the present invention, the gray pixel is specified by using the color information of the skin image area included in the input image data. By specifying the gray pixel, the color temperature of the photographing light source can be estimated with relatively high accuracy. Since the skin image area is detected based on the shape or structure of the image, the color balance of the image represented by the given image data is detected regardless of the color balance. be able to. Compared to detection of a skin image area based on color information, the skin image area can be detected more reliably. As described above, the gray pixel is detected based on the color information of the skin image area detected based on the image shape or the image structure. Since gray pixels are detected according to the skin image area detected with relatively high accuracy, gray pixels can be detected with higher accuracy than gray pixel detection based only on color information. The erroneous detection of gray pixels can be prevented relatively reliably, and the color temperature of the photographing light source can be estimated with relatively high accuracy.

  An image represented by the image data to be processed is prepared in advance (stored in the storage device) representing the black body locus for each race (for example, for white people, yellow people, and black people). You may make it use the skin-black-body locus | trajectory according to the race of the person contained in it. In this case, the image processing apparatus is provided with a race designation means for designating the race included in the image represented by the image data to be processed. Skin black body trajectory data is used for subsequent processing.

  For the acquisition of the image data to be processed, data representing the skin black body locus and data representing the gray body locus for each imaging device model (camera model, etc.) are prepared in advance (stored in the storage device). You may make it use the skin black body locus | trajectory and gray black body locus | trajectory according to the used imaging device. In this case, the image processing device is provided with an imaging device designating unit for designating the model of the imaging device used for obtaining the image data to be processed. The skin of the imaging device designated by the imaging device designating unit is provided. Black body locus data and gray black body locus data are used for subsequent processing.

  When tag information is attached to the image data to be processed and the tag information includes data for specifying the camera model, the skin black body is based on the camera model specifying data in the tag information. Trajectory data and gray black body trajectory data may be selected.

  In a preferred embodiment, the image processing apparatus searches for a predetermined coefficient that causes the coefficient-multiplied skin region representative chromaticity value to be present in the vicinity of the skin black body locus, in which the number of gray pixels is maximized. And the first coefficient multiplication means and the second coefficient multiplication means use the predetermined coefficient searched by the optimization coefficient search means. By using a predetermined coefficient that maximizes the detected gray pixels, it is possible to detect a large number of gray pixels contained in an image represented by given image data. The color temperature of the photographing light source estimated based on the detected gray pixels can be made more accurate.

  In a preferred embodiment, the image processing apparatus has a color temperature of a photographing light source to be included in a gray pixel to be detected based on color information obtained from pixels constituting the skin image region detected by the skin image region detecting unit. Color temperature range narrowing means for narrowing down the range of the color temperature range, and the gray pixel detecting means detects a pixel having a color temperature included in the color temperature range narrowed down by the color temperature range narrowing means as a detection target candidate. A gray pixel is detected from the target candidates.

  Prior to the gray pixel detection process, the range of the color temperature of the photographing light source that the gray pixel to be detected should have is narrowed based on the color information obtained from the pixels constituting the skin image region. The color information obtained from the skin image area detected by the skin image area detection means alone is not sufficient to estimate the color temperature of the photographic light source, but determines the range of color temperature that has a certain extent. Is effective. In addition, when the image data to be processed is obtained by imaging under a light source that is not suitable for measurement of black body radiation such as a fluorescent lamp (a light source that is not a black body radiation light source), Inappropriate gray pixels are detected by narrowing down the range of the color temperature of the photographic light source that the gray pixel to be detected should have based on the color information obtained from (the color of the inappropriate photographic light source) It is possible to prevent the temperature from being estimated).

  In one embodiment, the color temperature range narrowing means multiplies a representative value of color information obtained from pixels constituting the skin image area detected by the skin image area detection means by each of a plurality of predetermined coefficients. 3 coefficient multiplication means, third chromaticity value conversion means for converting a plurality of coefficient multiplication skin region representative values calculated by the third coefficient multiplication means to chromaticity values, and the third chromaticity value The skin area representative chromaticity obtained by converting the representative value of the color information obtained from the pixels constituting the skin image area out of the plurality of coefficient-multiplied skin area representative chromaticity values obtained by the conversion means. Coefficient range search means for searching for the range of the predetermined coefficient in which the coefficient-multiplied skin region representative chromaticity value is calculated such that the distance from the value on the skin black body locus closest to the value is within the predetermined distance.

  If the white balance (color balance) of the given image data is appropriate and the photographic light source at the time of acquisition of the given image data is a light source suitable for measuring blackbody radiation (for example, sunlight), the skin The color information obtained from the image area, for example, the representative chromaticity value (the average value, the mode value, or the median value of the chromaticity values obtained from the pixels constituting the skin image area) is the skin black body locus corresponding to the predetermined color temperature. Present at or near the top. Gray pixels included in the image represented by the image data exist on or near the gray black body locus corresponding to the predetermined color temperature.

  If the white balance (color balance) of the given image data is not appropriate, the representative color obtained from the skin image area even if the photographic light source at the time of acquisition of the given image data is a light source suitable for the measurement of black body radiation The degree value deviates from or near the skin black body locus, and the chromaticity value of the gray pixel also deviates from or near the gray black body locus. Also, if the imaging light source at the time of obtaining the given image data is a light source that is not suitable for measuring blackbody radiation (for example, a fluorescent lamp), the skin blackbody locus and gray blackbody locus are suitable for measuring blackbody radiation. Therefore, the detection of the gray pixel using the skin black body locus and the gray black body locus becomes low in accuracy.

  Therefore, even if the imaging light source at the time of obtaining the given image data is a light source that is not suitable for measurement of black body radiation (a light source that is not a black body radiation light source) (for example, a fluorescent lamp), the skin black body locus and gray black Detection based on color information obtained from the pixels constituting the skin image area detected by the skin image area detection means in order to suppress a decrease in detection accuracy of gray pixels using the body locus. A process of narrowing down the range of the color temperature of the photographing light source to be possessed by the gray pixel to be performed is performed. Since the skin image area is detected based on the image shape or structure, the skin image area can be detected relatively accurately even if the photographing light source is a fluorescent lamp. The skin image area includes a large number of skin color pixels almost certainly. Using this fact, processing for narrowing down the range of the color temperature of the photographing light source that the gray pixel to be detected should have is performed.

  The range of the color temperature of the photographing light source that the gray pixel to be detected should have corresponds to the range of the predetermined coefficient described above. Coefficient multiplication skin region representative chromaticity value obtained by multiplying a representative value of color information obtained from the skin image region by a predetermined coefficient and converting it to a chromaticity value is within the range on or near the skin black body locus. When entering, such a predetermined coefficient is set as a candidate for a predetermined coefficient for detecting gray pixels. Predetermined coefficients belonging to the narrowed down coefficient range (a part of a plurality of predetermined coefficients in the initial state) are used for subsequent processing.

  The gray pixel detection means configures the skin image area detected by the skin image area detection means, with gray pixels having a color temperature included in the color temperature range narrowed down by the color temperature range narrowing means as detection target candidates. Based on the color information obtained from the pixel to be detected, a gray pixel included in the image represented by the image data is detected.

  According to the present invention, based on the color information obtained from the skin image area detected based on the image shape or structure, the range of the color temperature of the photographing light source that the gray pixel to be detected should have is narrowed down in advance (predetermined) A coefficient range is set), and a gray pixel having a narrowed color temperature range is set as a detection target candidate (using a predetermined coefficient within the set predetermined coefficient range), and the gray pixel is detected. Even if the type of light source at the time of image data acquisition is something like a fluorescent lamp, it is possible to identify (detect) gray pixels relatively accurately.

  The predetermined coefficient may be divided into two groups of a predetermined coefficient belonging to the search range and a predetermined coefficient not belonging to the search range, and a predetermined coefficient to be used for gray pixel detection may be determined from the predetermined coefficients belonging to the search range. . A skin black body closest to a skin area representative chromaticity value obtained by converting a plurality of coefficient multiplication skin area representative chromaticity values and a representative value of color information obtained from pixels constituting the skin image area into a chromaticity value A distance from a value on the trajectory may be calculated, and each of the plurality of predetermined coefficients may be weighted according to the calculated distance. As the weight, for example, a larger value is assigned as the calculated distance is shorter. This is because the predetermined coefficient that makes the skin region representative chromaticity value closer to the skin black body locus is considered to be a predetermined coefficient suitable for gray pixel detection.

  A color temperature range narrowing unit using weighting is a third unit that multiplies each of a plurality of predetermined coefficients by a representative value of color information obtained from the pixels constituting the skin image region detected by the skin image region detecting unit. Coefficient multiplying means, a third chromaticity value converting means for converting a plurality of coefficient multiplied skin region representative values calculated by the third coefficient multiplying means into chromaticity values, and the third chromaticity value converting means. To the skin area representative chromaticity value obtained by converting the representative value of the color information obtained from the plurality of coefficient multiplication skin area representative chromaticity values obtained from the above and the pixels constituting the skin image area into chromaticity values. Weighting means for calculating a distance from a value on a close skin black body locus and weighting each of the plurality of predetermined coefficients according to the calculated distance is included. The gray pixel detecting means is a coefficient calculated by a fourth coefficient multiplying means for multiplying a representative value of color information obtained from pixels constituting the skin image region by the predetermined coefficient, and a coefficient calculated by the fourth coefficient multiplying means. A fourth chromaticity value converting means for converting the multiplied skin area representative value into a chromaticity value, and the coefficient multiplied skin area representative chromaticity value obtained by the fourth chromaticity value converting means is in the vicinity of the skin black body locus. When the determination means for determining whether or not the coefficient multiplication skin area representative chromaticity value exists in the vicinity range of the skin black body locus is determined by the determination means for determining whether or not it exists in the range, the coefficient multiplication skin area The color temperature corresponding to the value on the skin black body locus closest to the representative chromaticity value is obtained from provisional color temperature estimation means for estimating the color temperature as the provisional color temperature, and the pixels constituting the image represented by the image data. In the color information, the above coefficient multiplication skin area cost A fifth coefficient multiplier that multiplies a predetermined coefficient that causes the chromaticity value to be present in the vicinity range of the skin body locus, and a plurality of coefficient multiplication pixel values obtained by the fifth coefficient multiplier as the chromaticity value. 5th chromaticity value conversion means for converting, among the pixels constituting the image represented by the image data, the coefficient-multiplied pixel chromaticity value calculated by the fifth chromaticity value conversion means is A pixel that falls in the vicinity of the value on the gray black body locus corresponding to the provisional color temperature estimated by the provisional color temperature estimation means is specified, and the specified pixel is detected as a gray pixel. Further, the image processing apparatus has such an evaluation function that the value decreases as the number of gray pixels detected by the gray pixel detection means increases, and decreases as the weight for the predetermined coefficient increases. And means for calculating an evaluation function value based on the number of gray pixels detected by the gray pixel detecting means and the weights of predetermined coefficients used in the fourth and fifth coefficient multiplying means, and the evaluation Optimization coefficient searching means for searching for a predetermined coefficient that minimizes the function value is provided. In the fourth and fifth coefficient multiplication means, the predetermined coefficient searched by the optimization coefficient search means is used.

  The smaller the number of detected gray pixels and the greater the weight, the smaller the evaluation function value is obtained. Therefore, the predetermined coefficient capable of obtaining a small evaluation function value is suitable for estimating the light source color temperature. Conceivable. The color temperature of the photographing light source estimated based on the detected gray pixels can be made more accurate.

  Using such an evaluation function, the value increases as the number of gray pixels detected by the gray pixel detection means increases, and the value increases as the weight for the predetermined coefficient increases. The evaluation function value may be calculated based on the weight of the predetermined coefficient. In this case, a predetermined coefficient that maximizes the evaluation function value is searched.

  In a preferred embodiment, when the difference between the representative value (for example, representative chromaticity value) of the skin image area subjected to white balance correction by the white balance correcting means and the skin image reference value is larger than a predetermined value, the white balance is selected. First lower means for correcting the white balance correction value so as to weaken the correction effect of the correction is further provided. If the difference between the representative value of the skin image area after white balance correction and the skin image reference value is larger than a predetermined value, gray pixel detection (white balance correction value calculation) may not be performed properly. There is. In such a case, by correcting the white balance correction value so as to weaken the correction effect, adverse effects on the corrected image data can be reduced.

  When the number of gray pixels detected by the gray pixel detection means is smaller than a predetermined value, the second further means for correcting the white balance correction value so as to weaken the correction effect of the white balance correction is further provided. May be. You may make it use a 2nd reward means together with the 1st reward means mentioned above.

(1) Blackbody locus The blackbody locus will be briefly described. FIG. 1 shows a black body locus (hereinafter referred to as a gray black body locus G) for gray on chromaticity coordinates (the horizontal axis is r and the vertical axis is b).

  In this specification, the black body locus means the black body radiant energy distribution at the color temperature T as P (λ), the spectral reflectance distribution of the subject as ρ (λ), and the spectral sensitivity distribution of a CCD sensor such as a digital camera. When Si (λ) (where i = R, G, and B, where λ is the wavelength), Ei calculated by the following equation 1 is converted into a chromaticity value (r, b) to obtain chromaticity. Plotted on the diagram and indicates the locus when the color temperature T is moved.

  Ei = ∫P (λ) ρ (λ) Si (λ) dλ Formula 1

  Note that the conversion of RGB values into chromaticity values (r, b) is performed by the following equation 2.

r = R / (R + G + B)
b = B / (R + G + B) Equation 2

  As is clear from Equation 1, the black body locus draws a different locus for each spectral reflectance distribution ρ (λ) of the subject. The spectral reflectance distribution ρ (λ) varies depending on the color of the subject, and a predetermined value is used for the spectral reflectance ρ (λ) for gray. Further, the black body locus is different depending on the spectral sensitivity distribution Si (λ) of a CCD sensor such as a digital camera. The BT709 ideal spectral sensitivity distribution can be used when the spectral sensitivity distribution unique to the CCD sensor is unknown, or when an image captured by a CCD sensor having a spectral sensitivity distribution with different characteristics is to be processed.

  A color obtained by converting the RGB values of pixels constituting an image obtained by using a gray black body (complete radiator) (an ideal object that completely absorbs energy) as a subject to chromaticity values (r, b) When plotted on the degree chart, it is plotted at any point on the gray body locus G. As described above, since the black body locus is a locus obtained according to the color temperature T, the color temperature can be calculated according to the plot points.

  A subject that is imaged using a digital camera in a general usage situation is not a black body (complete radiator). For this reason, the chromaticity value obtained from the digital image data obtained by using the digital camera is different from the value on the gray black body locus G even if a gray subject image is included in the image. However, on the chromaticity diagram, a pixel plotted within a certain range around the gray black body locus G can be handled as a gray pixel.

  If the gray pixel has a chromaticity value near the gray black body locus G corresponding to a predetermined color temperature on the chromaticity diagram, the gray pixel can be expressed in an appropriate gray color. That is, it means that the white balance (gray balance) is appropriate image data.

  However, in the case of an image that is not color-balanced, a gray pixel that should enter on or near the gray body locus G has a chromaticity value (in the range shown by symbol A in the chromaticity diagram shown in FIG. r, b). For example, if the white balance correction processing of a digital camera is inappropriate, an image obtained from digital image data has a chromaticity value of a gray pixel that should be in a range indicated by reference sign B on the chromaticity diagram. It will have a chromaticity value in the range shown.

  In the chromaticity diagram, the white balance correction is performed by changing the gray pixels in the range indicated by the symbol A to the range indicated by the symbol B (a portion near the gray black body locus G and corresponding to a predetermined color temperature). This means that processing for moving is performed.

  In order to perform appropriate white balance correction, it is first necessary to accurately detect a gray pixel to be expressed in gray from pixels represented by the digital image data to be processed.

  In addition, in order to perform appropriate white balance correction, a correction value (correction coefficient) for converting the chromaticity value of the gray pixel in the range indicated by the symbol A into the chromaticity value indicated by the range indicated by the symbol B It is necessary to ask.

  In the digital printing system described below, instead of detecting gray pixels based on only the color information of the image represented by the digital image data and estimating the light source color temperature, the human skin area is represented by an image shape or image. It detects accurately based on the structure, uses the information obtained from this skin area, detects gray pixels indirectly, and estimates (determines) the light source color temperature.

(2) Digital Print System FIG. 2 is a block diagram schematically showing an embodiment of a digital photo print system. The digital photo print system performs white balance correction on digital image data representing a subject image photographed under various different light sources (color temperatures).

  The digital photo print system accurately detects a skin region included in an image represented by digital image data, and detects gray pixels included in the digital image based on information obtained from the detected skin region. Is. That is, among subjects to be photographed, human skin can be cited as a subject that has a high photographing frequency and is effective as auxiliary information when detecting gray pixels. The spectral reflectance of human skin varies depending on race (white, yellow, black, etc.), but the relative relationship for each wavelength is close. In other words, the skin value of a person differs depending on the race, but the RGB values themselves are close to each other when the chromaticity value is obtained. The digital photo print system uses such human skin properties. In addition, the digital print system uses the color temperature obtained from the detected skin area information to estimate the light source color temperature.

  Hereinafter, the digital print system will be described in detail.

  Referring to FIG. 2, the digital photo print system includes an image correction device 1 and peripheral devices (input device 2, display device 3, storage device 4 and printer 5) connected to the image correction device 1. .

  An input device 2 (keyboard, mouse, etc.) connected to the image correction device 1 is used for designating image data to be processed. On the display screen of the display device 3, a screen for selecting image data (image file) to be processed, an image represented by image data before and after correction, and the like are displayed. Image data is stored in a storage device (hard disk, memory card, CD-ROM, DVD, etc.) 4, and white balance correction processing is performed on the image data read from the storage device 4 in the image correction device 1. Done. The printer 5 prints an image represented by the corrected image data on photographic paper or the like.

  FIG. 3 is a block diagram showing the electrical configuration of the image correction apparatus 1, which is the core apparatus of the digital photo print system, together with the storage device 4.

  Each circuit constituting the image correction apparatus 1 can be classified into the following four parts from the viewpoint of its function.

  One of them is a skin region representative value calculation unit 10. The skin area representative value calculation unit 10 detects a skin area included in the image represented by the image data to be processed read from the storage device 4, and representative values (for example, RGB values) obtained from the detected skin area. ) Is calculated. The skin area representative value calculation unit 10 includes a reduced image generation circuit 11, a skin area detection circuit 12, and a skin area representative value calculation circuit 13.

  The second is the gray pixel detection unit 20. The gray pixel detection unit 20 uses the skin region representative value obtained by the skin region representative value calculation unit 10 to detect gray pixels included in the image represented by the image data to be processed. The gray pixel detection unit 20 includes a coefficient multiplication circuit 21, a chromaticity conversion circuit 22, a gray candidate pixel detection circuit 23, a gray candidate pixel number calculation circuit 24, and a coefficient optimization circuit 25.

  The third is a white balance correction coefficient calculation unit 30. The white balance correction coefficient calculation unit 30 performs correction for white balance correction based on the color temperature of each gray pixel included in the image represented by the image data to be processed detected by the gray pixel detection unit 20. A coefficient is calculated. The white balance correction coefficient calculation unit 30 includes a color temperature calculation circuit 31 and a correction coefficient calculation circuit 32.

  The fourth is the white balance correction unit 40. The white balance correction unit 40 uses the white balance correction coefficient calculated by the white balance correction coefficient calculation unit 30 to correct the white balance of the image data to be processed. The white balance correction unit 40 includes a reduced image generation circuit 41, a white balance correction circuit 42, and an enlarged image generation circuit 43.

  The image processing apparatus 1 is further provided with a memory 26 in which data (table) representing a skin black body locus and data (table) representing a gray black body locus are stored. As will be described later, the data representing the skin black body locus includes data (table) defining the relationship between the b value and the color temperature T for the skin black body locus. , Data (table) defining the relationship between the b value and the color temperature T for the gray black body locus is included. These data stored in the memory 26 are used in processing in the gray candidate pixel detection circuit 23 and the color temperature calculation circuit 31 (details will be described later).

  Hereinafter, each of the skin region representative value calculation unit 10, the gray pixel detection unit 20, the white balance correction coefficient calculation unit 30, and the white balance correction unit 40 will be described in order.

(I) Skin Area Representative Value Calculation Unit 10 As described above, the skin area representative value calculation unit 10 includes a reduced image generation circuit 11, a skin area detection circuit 12, and a skin area representative value calculation circuit 13.

  In this embodiment, the image represented by the image data stored in the storage device 4 includes a human face image and a gray subject image. The image data read from the storage device 4 is given to the reduced image generation circuit 11. The reduced image generation circuit 11 generates reduced image data (with a reduced number of pixels) by thinning out the given image data. The reduced image generation circuit 11 is for shortening the processing time of the image correction apparatus 1, and the processing of the reduced image generation circuit 11 may be turned on / off as necessary.

  The reduced image data output from the reduced image generation circuit 11 is used for subsequent processing. The reduced image data is input to the skin region detection circuit 12 and the coefficient multiplication circuit 21 (the description of the coefficient multiplication circuit 21 will be described later).

  The skin region detection circuit 12 is a circuit that detects (divides, demarcates, identifies) a skin region in a face image included in a reduced image represented by reduced image data. Since a person's face has a predetermined shape and structure, the face image detection process can be performed with relatively high accuracy by using the shape and structure. In addition, the face of a person surely contains skin. The skin area detection circuit 12 pays attention to these facts, detects the face image in the reduced image, and reliably detects the skin area included therein.

  For the process of detecting (sectioning) the face image included in the reduced image, various conventional or novel detection (sectioning) techniques can be used. For example, a method of detecting a face image portion by pattern matching (Japanese Patent Laid-Open No. 8-122944) can be used. Detect skin candidate regions (including hands, feet, etc.) in the input image, and detect those that contain major facial features (eyes, eyebrows, hair, nose and mouth). Accordingly, a method of detecting a face image portion from the skin candidate region (Japanese Patent Laid-Open No. 2002-203239) or the like can also be employed. In any case, the skin region detection circuit 12 detects an image portion representing a person's face included in the reduced image based on the shape or structure of the person's face.

  The face of a person has a part (for example, eyes, eyebrows, lips, etc.) containing a color other than the skin color. Pixels having these colors (white pixels and black pixels for eyes, black pixels for eyebrows, red pixels for lips, etc.) may be removed from the detected face image portion image. Good.

  The skin region detection circuit 12 outputs the RGB values of each of a plurality of pixels included in the detected face image portion. The RGB values for each pixel constituting the face image portion are input to the skin region representative value calculation circuit 13.

  The skin area representative value calculation circuit 13 calculates an average value (tfR, tfG, tfB) of each color of R, G, B for each pixel included in the face image portion given from the skin area detection circuit 12. This average value is a representative value obtained from the skin area (hereinafter referred to as a skin area representative value). In addition, a virtual pixel having this representative value is hereinafter referred to as a representative skin pixel. Instead of the average value, the median value or the mode value may be used as the representative value. The skin region representative values (tfR, tfG, tfB) are given to the coefficient multiplication circuit 21 included in the gray pixel detection unit 20.

(II) Gray Pixel Detection Unit 20 As described above, the gray pixel detection unit 20 includes the coefficient multiplication circuit 21, the chromaticity conversion circuit 22, the gray candidate pixel detection circuit 23, the gray candidate pixel number calculation circuit 24, and the coefficient optimization. The circuit 25 is configured.

  The gray pixel detection unit 20 detects gray pixels included in the reduced image. For the detection of the gray pixels, the skin area representative values (tfR, tfG, tfB) calculated by the skin area representative value calculation circuit 13, the skin black body locus H, and the gray black body locus G are used.

  FIG. 4 is a flowchart showing the flow of processing in the gray pixel detection unit 20. FIGS. 5 to 8 illustrate the processing in the gray pixel detection unit 20 using a chromaticity diagram including a skin black body locus H and a gray black body locus G. FIG. Hereinafter, the respective processes in the coefficient multiplication circuit 21, the chromaticity conversion circuit 22, the gray candidate pixel detection circuit 23, the gray candidate pixel number calculation circuit 24, and the coefficient optimization circuit 25 constituting the gray pixel detection unit 20 will be described with reference to FIG. This will be described with reference to FIG.

  The coefficient multiplication circuit 21 includes the skin area representative values (tfR, tfG, tfB) obtained by the above-described skin area representative value calculation circuit 13 and the reduced image data generated by the reduced image generation circuit 11 (which constitutes a reduced image). RGB values for each of a plurality of pixels to be input.

The coefficient multiplication circuit 21 multiplies the R component and G component of the skin region representative values (tfR, tfG, tfB) by predetermined coefficients (α2, α1), respectively (step 51). That is, the coefficient multiplication skin region representative values ( tfR , tfG , tfB ) shown in the following expression 3 are calculated in the coefficient multiplication circuit 21.

tfR = α2 ・ tfR
tfG = α1 · tfG Equation 3
tfB = tfB

  In Equation 3, only the R component and the G component of the RGB values are multiplied by a predetermined coefficient (α2, α1). The white balance correction corrects the ratio of the RGB values. This is because the ratio of RGB values can be controlled by a predetermined coefficient (α2, α1) multiplied by the R component and G component. Of course, it goes without saying that the B component may be multiplied by a predetermined coefficient. Alternatively, only the R component and the B component, and only the B component and the G component may be multiplied by a coefficient.

In the chromaticity conversion circuit 22, the coefficient multiplication skin region representative values ( tfR , tfG , tfB ) obtained in the coefficient multiplication circuit 21 are converted into chromaticity values (see the above equation 2 for conversion processing) ( Step 52). Hereinafter, this chromaticity value is referred to as a coefficient multiplication skin region representative chromaticity value (r1, b1).

  Subsequently, in the gray candidate pixel detection circuit 23, processing using the coefficient multiplication skin region representative chromaticity values (r1, b1), the skin black body locus H, the gray black body locus G, and the like stored in the memory 26 is performed. Done. As described above, the memory 26 connected to the gray candidate pixel detection circuit 23 stores the skin black body locus data defining the skin black body locus H, the gray black body locus data defining the gray black body locus G, and the skin black. Data storing the relationship between the chromaticity value b and the color temperature T for the body locus H (hereinafter referred to as “chromaticity value b / color temperature T relationship data for the skin black body locus H”), and the gray black body locus Data storing the relationship between the chromaticity value b and the color temperature T for G (hereinafter referred to as “the chromaticity value b / color temperature T relationship data for the gray black body locus G”) is stored.

  First, the skin black body locus H and the chromaticity value b / color temperature T relation data for the skin black body locus H are used, and based on the coefficient multiplication skin region representative chromaticity values (r1, b1) described above, A temporary color temperature Th is calculated. The provisional color temperature Th calculation process will be described with reference to FIGS. The skin black body trajectory H is a trajectory defined by the above-described equation 1 in the same manner as the gray black body trajectory G described above. Only the spectral reflectance distribution ρ (λ) in Equation 1 differs from the gray blackbody locus G.

  FIG. 5 shows plot points A (r0, b0) of chromaticity values (skin region representative chromaticity values) obtained from skin region representative values (tfR, tfG, tfB) on the chromaticity diagram and the above-mentioned coefficient multiplication. The plot point B (r1, b1) of the skin region representative chromaticity value is shown. FIG. 6 shows how the provisional chromaticity value Th is calculated.

  As described above, the coefficient-multiplied skin region representative chromaticity value B (r1, b1) is a predetermined coefficient (α2, α1) for the R component and G component of the representative skin pixel (skin region representative value) (tfR, tfG, tfB). ) Are converted into chromaticity values. Therefore, the skin area representative chromaticity value A (r0, b0) and the coefficient multiplied skin area representative chromaticity value B (r1, b1) are plotted at different positions on the chromaticity diagram (FIG. 5).

  The gray candidate pixel detection circuit 23 calculates a point (r value, b value) on the skin black body locus H closest to the coefficient multiplication skin region representative chromaticity value B (r1, b1). The coefficient multiplication skin region representative chromaticity value B (r1, b1) and the point (r value, b value) on the skin black body locus H closest to the coefficient multiplication skin region representative chromaticity value B (r1, b1) The distance L1 is calculated (FIG. 6) (step 53).

  If the distance L1 between the coefficient multiplication skin region representative chromaticity value B (r1, b1) and the closest point on the skin black body locus H is a predetermined value (for example, 0.01) or more, a new Another predetermined coefficient (α2, α1) is adopted (NO in step 54, step 55, NO in step 61, step 62). This is when the distance between the coefficient-multiplied skin region representative chromaticity value B (r1, b1) and a point on the skin black body locus H closest thereto is a predetermined value (for example, 0.01) or more. The representative skin pixel having the coefficient multiplied skin area representative chromaticity value B (r1, b1) obtained by multiplying the predetermined coefficient (α2, α1) is multiplied by the predetermined coefficient (α2, α1). However, it does not enter the skin black body locus H or the vicinity thereof, and as a result, as described later, gray pixels cannot be detected with high accuracy.

  That is, since the representative skin pixel is an appropriate skin pixel obtained from the face image portion of the person included in the input image based on the image shape or image structure, the white balance (color balance) of the input image is appropriate. If so, the skin region representative chromaticity value A (r0, b0) obtained from the representative skin pixel falls on or near the skin black body locus H corresponding to the predetermined color temperature. However, if the white balance of the input image is not appropriate, the skin region representative chromaticity value A (r0, b0) does not fall on the skin black body locus H or in the vicinity thereof (see FIG. 5). The coefficient multiplication skin region representative chromaticity value B (r1, b1) obtained by multiplying the R component and G component of the skin representative pixel by a predetermined coefficient (α2, α1) and converting it to a chromaticity value is the skin black body. If the predetermined coefficient (α2, α1) is used on the locus H or in the vicinity thereof, the provisional color temperature Th (color temperature) obtained by using the skin-black body locus H is also obtained for the gray pixel. This means that Th is in the vicinity range of the gray black body locus G corresponding to (described below). Using this, gray pixels can be detected with relatively high accuracy. That is, if the skin region representative chromaticity value A (r0, b0) can be moved (converted) to a point on the skin black body locus H in the vicinity of the point corresponding to the provisional color temperature Th of the photographing light source, The principle that gray pixels are also moved (converted) in the vicinity of the gray black body locus G corresponding to the same color temperature Th by the predetermined coefficients (α2, α1) used for the movement is used.

  If the distance between the coefficient-multiplied skin region representative chromaticity value B (r1, b1) and the point on the skin black body locus H closest thereto is a predetermined value (for example, 0.01) or more, it is predetermined. This means that even if the coefficients (α2, α1) are used, gray pixels cannot be detected accurately. For this reason, predetermined coefficients (α2, α1) that cause the coefficient-multiplied skin region representative chromaticity value B (r1, b1) to exist on the skin black body locus H or in the vicinity thereof are obtained.

  Although the details will be described later, the above-described predetermined coefficients (α2, α1) are changed to other values by the coefficient optimization circuit 25. The predetermined coefficient (α2, α1) after being changed by the coefficient optimizing circuit 25 is also the coefficient multiplication skin region representative chromaticity value B (r1, b1) and the point on the skin black body locus H closest to this. The distance L1 is made smaller than a predetermined value (for example, 0.01).

  When the distance L1 between the coefficient multiplication skin region representative chromaticity value B (r1, b1) and the point on the skin black body locus H closest thereto is smaller than a predetermined value (for example, 0.01) (in step 54) YES), the color temperature corresponding to the point on the skin black body locus H is set as the provisional color temperature Th (step 56, FIG. 6). For the calculation of the color temperature Th, the chromaticity value b / color temperature T relation data for the skin black body locus H stored in the memory 26 is used.

When the provisional color temperature Th is determined, the R and G components of the RGB values for each of the plurality of pixels included in the reduced image are respectively multiplied by the predetermined coefficients (α2, α1) ( Step 57) (The coefficient multiplication circuit 21 may be used for this multiplication). Values ( Rj , Gj , Bj ) shown in the following equation 4 are calculated.

Rj = α2 ・ Rj
Gj = α1 · Gj ・ ・ ・ Equation 4
Bj = Bj

  In Expression 4, the variable j represents that each pixel constituting the reduced image.

The ( Rj , Gj , Bj ) for each pixel of the reduced image multiplied by the predetermined coefficient (α2, α1) is converted into a chromaticity value (step 58). May be used). This chromaticity value is referred to as a coefficient multiplication pixel chromaticity value (rj, bj).

  Next, points (r value and b value) on the gray blackbody locus G corresponding to the same color temperature as the determined provisional chromaticity value Th are calculated, and coefficient multiplication is performed among the pixels constituting the reduced image. A pixel having a pixel chromaticity value (rj, bj) in the range of the color temperature Th ± 500 degrees and a distance from the gray black body locus G to 0.01 is determined as a gray candidate pixel. (Step 59).

  FIG. 7 shows the points on the gray black body locus G corresponding to the color temperature Th plotted on the chromaticity diagram. FIG. 8 shows coefficient multiplication pixel chromaticity values (rj, bj) plotted on a chromaticity diagram (plot points are indicated by “x”). Further, in FIG. 8, a range in which the color temperature is in the range of ± 500 ° C. and the distance from the gray black body locus G is 0.01 is shown surrounded by a chain line (hereinafter, this range is referred to as “ Gray pixel candidate range S). Pixels within the gray pixel candidate range S are determined as gray candidate pixels.

  As described above, the gray candidate pixel is obtained by multiplying each of the R component and the G component of the RGB values for each of the plurality of pixels included in the reduced image by a predetermined coefficient (α2, α1). The value (rj, bj) obtained by converting the value into a chromaticity value is determined depending on whether or not the value is within the gray pixel candidate range S. The pixel determined as the gray candidate pixel is a neighborhood range of the gray black body locus G when the R component and the B component of each of the pixels constituting the input image (reduced image) are respectively multiplied by predetermined coefficients (α2, α1). It means a pixel that enters.

  As will be described later, the finally determined light source color temperature is determined based on the color temperature of each of the pixels determined as gray pixels in the gray candidate pixel detection circuit 23. The greater the number of pixels determined as gray pixels in the gray candidate pixel detection circuit 23, the higher the calculation accuracy of the finally determined light source color temperature. In order to increase the accuracy of the finally determined light source color temperature, a process for searching for a predetermined coefficient (α2, α1) in which the number of gray candidate pixels N takes a maximum value is performed.

  The gray candidate pixel number calculating circuit 24 and the coefficient optimizing circuit 25 are used for searching for a predetermined coefficient (α2, α1) in which the gray candidate pixel number N has the maximum value. First, the number N of gray candidate pixels detected by the gray candidate pixel detection circuit 23 is calculated by the gray candidate pixel number calculation circuit 24 (step 60).

  The gray candidate pixel count N calculated by the gray candidate pixel count calculation circuit 24 is given to the coefficient optimization circuit 25.

  In the coefficient optimization circuit 25, the gray candidate pixel number N calculated by the gray candidate pixel number calculation circuit 24 is maximized, that is, the number of pixels having a chromaticity value that falls within the gray pixel candidate range S is maximized. Is a circuit for obtaining predetermined coefficients (α2, α1). The coefficient optimization circuit 25 determines the above-described predetermined coefficient (α2, α1) (hereinafter referred to as an optimization coefficient) that maximizes the number of gray candidate pixels N based on, for example, the simplex method (NO in step 61, step 62). Of course, in the process of determining the optimization coefficient, it goes without saying that the coefficient multiplication circuit 21, the chromaticity conversion circuit 22, the gray candidate pixel detection circuit 23, and the gray candidate pixel number calculation circuit 24 also operate a plurality of times.

  When the optimization coefficient (α2, α1) that maximizes the number of gray candidate pixels N is calculated (YES in step 61), the gray candidate pixel obtained by using the optimization coefficient (α2, α1) is the final Gray pixel. The determined gray pixel address is given from the gray candidate pixel detection circuit 23 to the color temperature calculation circuit 31 of the white balance correction coefficient calculation unit 30 (step 63).

(III) White Balance Correction Coefficient Calculation Unit 30 The white balance correction coefficient calculation unit 30 includes a color temperature calculation circuit 31 and a correction coefficient calculation circuit 32.

  Based on the addresses of the plurality of gray pixels given from the gray candidate pixel detection circuit 23, the color temperature calculation circuit 31 calculates the color temperature tempT of each gray pixel having the address as the gray black body locus G (gray black body locus). This is a circuit that calculates an average value aveT of a plurality of color temperatures tempT calculated based on the chromaticity value b / color temperature T related data for G). For example, the color temperature tempT of each gray pixel corresponds to the point on the gray black body locus G that is closest to the plot point of each gray pixel on the chromaticity diagram when multiplied by the optimization coefficient (α2, α1). Color temperature. The average value aveT calculated by the color temperature calculation circuit 31 is determined (estimated) as the light source color temperature for the reduced image.

  As described above, the provisional color temperature Th is already obtained by using the skin black body locus H, but this provisional color temperature Th is not adopted as the light source color temperature. This is because the white balance (gray balance) correction coefficient is finally obtained, and the light source color temperature aveT obtained using gray pixels is considered to be a more accurate light source color temperature.

  The light source color temperature aveT calculated by the color temperature calculation circuit 31 is given to the correction coefficient calculation circuit 32.

  The correction coefficient calculation circuit 32 is a circuit that calculates a coefficient (white balance correction coefficient) to be multiplied by each of the RGB values for each pixel constituting the reduced image. The correction coefficient calculation circuit 32 performs the following processes (A) to (C).

(A) On the gray black body locus G corresponding to the color temperature aveT based on the color temperature aveT calculated based on the gray pixel and the chromaticity value b / color temperature T correspondence data for the gray blackbody locus G The chromaticity value of is calculated. The chromaticity value on the gray black body locus G corresponding to the color temperature aveT is expressed as (rave, bave).

(B) A target r value and a target b value for a gray pixel are set to (rtarget, btarget) (set values) (so-called gray pixel color is set in advance). Target difference correction coefficients (β1, β2, β3) for the R component, G component, and B component are calculated by the following equations.

β1 = rtarget / rave
β2 = (1-rtarget-btarget) / (1-rave-bave) Equation 5
β3 = btarget / bave

  The calculated β1, β2, and β3 are values that make the color tone of the detected gray pixel a set color (close to the set color).

  Relationship between RGB values (Rinput, Ginput, Binput) of pixels constituting an input image (reduced image) and RGB values (Routput, Goutput, Boutput) of pixels constituting an output image after white balance correction Is expressed by the following equation using optimization coefficients (α2, α1) and target difference correction coefficients (β1, β2, β3).

Routput = α2, β1, Rinput
Goutput = α1, β2, Ginput (6)
Boutput = β3 ・ Binput

According to (C) Equation 6, the brightness of the image greatly fluctuates. Therefore, for example, the above Equation 6 is normalized to Equation 7 so as not to change the brightness of the G component.

Routput = (α2 ・ β1) / (α1 ・ β2) ・ Rinput
Goutput = Ginput Equation 7
Boutput = β3 / (α1 ・ β2) ・ Binput

  In the correction processing according to Equation 7, the gray pixel is expressed by a gray color (realized by multiplication of α2 and α1 described above), and the color is expressed by a set color (β1 described above). , Β2, β3). According to Equation 7, white balance correction is realized.

  The correction coefficient calculation circuit 32 supplies correction coefficients ((α2 · β1) / (α1 · β2), 1, β3 / (α1 · β2)) for the R component, B component, and G component to the white balance correction unit. Forty white balance correction circuit 42 is provided.

(IV) White Balance Correction Unit 40 The white balance correction unit 40 includes a reduced image generation circuit 41, a white balance correction circuit 42, and an enlarged image generation circuit 43.

  The image data read from the storage device 4 (the same image data as the image data given to the reduced image generating circuit 11) is input to the reduced image generating circuit 41. The reduced image generation circuit 41 is a circuit that reduces the input image when the size of the input image is larger than the image size to be output (predetermined size). When the input image size is the same as the predetermined size or smaller than the predetermined size, the reduced image generation circuit 41 does not perform special processing.

  The white balance correction circuit 42 is provided with the correction coefficients ((α2 · β1) / (α1 · β2), 1, β3 / (α1 · β2)) calculated by the correction coefficient calculation circuit 32 described above. The white balance correction circuit 42 corrects the RGB value for each pixel constituting the reduced image using this correction coefficient (white balance correction). The reduced image data subjected to the white balance correction is input to the enlarged image generation circuit 43. When the input image size is smaller than the predetermined size, the enlarged image generation circuit 43 performs an enlargement process for matching the white balance corrected image data with the predetermined size. When the input image size is the same as or larger than the predetermined size, the enlarged image generation circuit 43 does not perform special processing.

  The white balance of the input image by the correction coefficients ((α2 ・ β1) / (α1 ・ β2), 1, β3 / (α1 ・ β2)) for each of the R, B, and G components used for white balance correction Correction (color balance correction) is realized. Image data after white balance correction can be obtained.

  The result of white balance correction of the representative skin pixel using the calculated white balance coefficient ((α2 · β1) / (α1 · β2), 1, β3 / (α1 · β2)) is used as a reference. When the color range is not entered (for example, when a detected gray pixel is erroneous), it is preferable to weaken the correction.

  For example, the distance L3 between the chromaticity value of the representative skin pixel in the image data after the white balance correction and the reference skin chromaticity value (r2, b2) is calculated, and the lowward coefficient c is calculated based on the calculated distance L3. calculate. FIG. 9 is a graph (lookup table) showing the relationship between the calculated distance L3 and the lowward coefficient c (0 ≦ c ≦ 1). The white balance correction coefficient (gain r, gain g, gain b) for each of the R, G, and B components whose correction effect has been weakened using the lowward coefficient c is expressed by the following equation (8).

  Furthermore, even when the determined gray candidate pixel number N is small, there is a possibility that an erroneous gray pixel has been detected. Also in this case, it is preferable to weaken the effect of white balance correction using the lowward coefficient c (0 ≦ c ≦ 1). FIG. 10 shows a graph (lookup table) representing the relationship between the number of gray candidate pixels N and the lowward coefficient c.

  Note that the two Lowward coefficients c described above may be used in combination. In this case, a coefficient obtained by multiplying two lowward coefficients c is used.

Modification 1
FIG. 11 is a block diagram illustrating a modification of the image correction apparatus 1. The block diagram of the image correction apparatus 1 shown in FIG. 3 is based on the fact that a plurality of types of skin black body locus data are stored in the memory 26 and the racial designation data given from the input device 2. The difference is that skin black body locus data to be read is specified.

  In the modification shown in FIG. 11, the memory 26 stores three types of skin black body trajectory data: white skin black body locus data, yellow human skin black body locus data, and black skin black body locus data. Yes. FIG. 12 shows a white skin black body locus H1 drawn by white skin black body locus data, a yellow skin black body locus H2 drawn by yellow human skin black body locus data, and a black skin black body locus data. The black body locus H3 for black people drawn by the above is shown on the chromaticity diagram together with the gray body locus G.

  As described above, the human skin has different RGB values depending on the race, but when the chromaticity value is obtained, it becomes a close value, so one skin black body locus data (see FIGS. 5 to 8) is used. As a result, gray pixels can be detected with relatively high accuracy, and the light source color temperature can be calculated with relatively high accuracy. In the modification shown in FIG. 11, in order to further improve the accuracy of gray pixel detection and the estimation of the light source color temperature, the skin black according to the race of the person existing in the image represented by the image data to be processed. The body locus data is used to estimate the light source color temperature and perform gray pixel detection processing.

  The input device 2 uses the race (white, yellow or black) (caucasian, mongoloid or neggroid) of the person present in the image represented by the image data to be processed by the user (operator) of the digital photo print system. Specified. Skin black body locus data (white skin body locus data for white, skin skin body locus data for yellow, or black body locus data for black people) corresponding to the specified race is read from the memory 26 and gray candidate pixels It is used for processing in the detection circuit 23. For each race, skin black body trajectory data suitable for the race is used to detect gray pixels and calculate the light source color temperature. Therefore, the gray pixels can be detected with high accuracy, and the light source color temperature can be detected. Can be calculated. White balance correction can be executed more appropriately.

  When the image represented by the image data to be processed includes a plurality of persons of different races, the white balance correction coefficient is calculated for each face image (for each race), and the average value is calculated. May be used to perform white balance correction.

Modification 2
FIG. 13 is a block diagram illustrating still another modification of the image correction apparatus 1. The block diagram of the image correction apparatus 1 shown in FIG. 3 is that the memory 26 stores a set of a plurality of types of skin black body locus data and gray black body locus data, and the camera model given from the input device 2. The difference is that skin black body locus data and gray black body locus data to be read from the memory 26 are designated based on the designated data.

  In the modification shown in FIG. 13, the memory 26 stores skin black body locus data and gray black body locus data for camera A, and skin black body locus data and gray black body locus data for camera B. . FIG. 14 shows skin black body locus H4 and gray black body locus G4 drawn by the skin black body locus data and gray black body locus data for camera A, and skin black body locus data and gray black body locus data for camera B. The skin black body locus H5 and the gray black body locus G5 to be drawn are shown on the chromaticity diagram.

  As described above, the black body locus is different depending on the spectral sensitivity distribution Si (λ) of a CCD sensor such as a digital camera (see Equation 1). In the image correction apparatus 1 shown in FIG. 13, a black body locus (skin body locus and gray black body locus) corresponding to the CCD sensor included in the digital camera used to acquire the image data to be processed is detected as a gray pixel. And used to calculate the light source color temperature.

  A user (operator) of the digital photo print system designates the model of the digital camera used to create the image data to be processed using the input device 2. Skin black body locus data and gray body locus data corresponding to the designated camera model are read from the memory 26 and used for processing in the gray candidate pixel detection circuit 23 and processing in the color temperature calculation circuit 31. Skin black body trajectory data and gray black body trajectory data suitable for the digital camera model (type of CCD sensor) used to create the image data to be processed are used to detect gray pixels and to determine the light source color temperature. Since it is calculated, the accuracy of gray pixel detection and the accuracy of light source color temperature calculation are improved.

  In the above-described modification 2, the user (operator) designates the camera model using the input device 2, but the image data to be processed includes tag information (for example, Exif) including information identifying the camera model. Tag), the camera model used to acquire the image data to be processed is automatically specified based on the tag information, and the black body locus data and gray black corresponding to the specified camera model are identified. The body locus data may be read from the memory 26. In this case, the specification of the camera model using the input device 2 is not necessarily required.

  FIG. 15 is a block diagram showing the electrical configuration of the image correction apparatus 1A in the second embodiment together with the storage device 4. The image correction apparatus 1 according to the first embodiment shown in FIG. 3 differs from the image correction apparatus 1 in the configuration of the skin region representative value calculation unit 10A and the processing of the gray pixel detection unit 20A. In FIG. 15, the same circuits as those included in the image correction apparatus 1 shown in FIG.

  The skin area representative value calculation unit 10A in the image correction apparatus 1A includes a reduced image generation circuit 11, a skin area detection circuit 12, a skin area representative value calculation circuit 13, and a search target coefficient range setting circuit 71. Similar to the skin region representative value calculation unit 10 of the first embodiment, the skin region representative value calculation unit 10A detects a skin region included in the image data to be processed read from the storage device 4 (skin region detection circuit). 12), a representative value obtained from the detected skin area is calculated (process of the skin area representative value calculating circuit 13). Further, the skin area representative value calculation unit 10A in the image correction apparatus 1A sets a range in which the predetermined coefficient (α2, α1) should be taken based on the calculated skin area representative value and the skin black body locus.

  FIG. 16 is a flowchart showing the flow of processing of the search target coefficient range setting circuit 71. The processing of the search target coefficient range setting circuit 71 will be described with reference to the chromaticity diagrams shown in FIGS.

  The chromaticity value is calculated from the skin area representative values (tfR, tfG, tfB) calculated by the skin area representative value calculating circuit 13 (step 81). A skin region representative chromaticity value (r0, b0) is obtained.

  Next, a point (r value, b value) on the skin black body locus H closest to the calculated skin region representative chromaticity value (r0, b0) is calculated (step 82). A point on the skin black body locus H closest to the skin region representative chromaticity value (r0, b0) is defined as (rn, bn). In the chromaticity diagram shown in FIG. 17, the plot point A is the skin region representative chromaticity value (r0, b0), and the plot point C is the skin skin body locus closest to the skin region representative chromaticity value A (r0, b0). Points (rn, bn) on H are shown.

The R component and G component of the skin region representative values (tfR, tfG, tfB) are respectively multiplied by a number of predetermined coefficients (α2, α1) (see step 83 and equation 3). The predetermined coefficient (α2, α1) is, for example, a set of a number of numerical values obtained by changing the numerical values for α2 and α1 in the range of 0.5 to 2.0 in increments of 0.01. That is, a large number of coefficient multiplication skin region representative values ( tfR , tfG , tfB ) are calculated.

A large number of calculated coefficient multiplication skin region representative values ( tfR , tfG , tfB ) are converted into chromaticity values (step 84). A large number of coefficient-multiplied skin region representative chromaticity values (r1, b1) are calculated.

  In the chromaticity diagram shown in FIG. 18, a plot point D (indicated by “x”) indicates a number of coefficient-multiplied skin region representative chromaticity values (r1, b1).

  The distance between the point C (rn, bn) on the skin black body locus H closest to the skin region representative chromaticity value A (r0, b0) and a number of coefficient-multiplied skin region representative chromaticity values D (r1, b1) L is calculated according to the following equation 9 (step 85).

  The distance L between the point C (rn, bn) on the skin black body locus H closest to the skin area representative chromaticity value A (r0, b0) and the coefficient multiplication skin area representative chromaticity value D (r1, b1) is If it is 0.1 or less (YES in step 86), the predetermined coefficients (α2, α1) used to calculate the coefficient-multiplied skin region representative chromaticity value D (r1, b1) are within the search range. It is determined. That is, the predetermined coefficient (α2) used for calculating the coefficient multiplication skin region representative chromaticity value D (r1, b1) plotted in the range surrounded by the broken line in FIG. 18 (the range at the distance L from the plot point C). , Α1) is determined to be within the search range.

  On the other hand, when the distance L between the point C (rn, bn) on the skin black body locus H and the coefficient multiplication skin region representative chromaticity value D (r1, b1) is longer than 0.1 (NO in step 86), It is determined that the predetermined coefficients (α2, α1) used for calculating the coefficient multiplication skin region representative chromaticity value D (r1, b1) are not within the search range. In this way, it is determined for all the predetermined coefficients (α2, α1) whether they are the search range or the search range. Data (identification code) for identifying whether the predetermined coefficient (α2, α1) is a predetermined coefficient in the search range or a predetermined coefficient that is not the search range is assigned to each of the large number of predetermined coefficients (α2, α1) (steps 87 and 88).

  Since the skin representative pixel is an appropriate skin pixel obtained from the face image portion of the person included in the input image, if the white balance of the input image is appropriate, the skin region obtained from the representative skin pixel The representative chromaticity value A (r0, b0) falls on the skin black body locus H or in the vicinity thereof. If the white balance of the input image is not appropriate, the skin region representative chromaticity value A (r0, b0) obtained from the representative skin pixel does not enter the skin black body locus H or the vicinity thereof.

  Predetermined coefficients (α2, α1) belonging to the search range are the coefficient multiplication skin region representative chromaticity value D (r1, b1) on the skin black body locus H closest to the skin region representative chromaticity value A (r0, b0). Such a predetermined coefficient is entered within a predetermined distance L centered at the point C (rn, bn). That is, the predetermined coefficient (α2, α1) belonging to the search range converts the skin region representative chromaticity value A (r0, b0) onto or near the point C (rn, bn) on the skin black body locus H. Therefore, it can be said that this is a candidate for correcting the white balance of an input image whose white balance is not appropriate.

  Further, if the input image to be processed is image data obtained by photographing a light source type that is not suitable for black body radiation measurement, for example, under a fluorescent lamp, the skin black body locus H and the gray black body locus G are Because it is created assuming a light source type suitable for measuring blackbody radiation (for example, sunlight), it is darker than that obtained under a light source type suitable for measuring blackbody radiation. The gray pixel detection process using the body locus H and the gray black body locus G has low detection accuracy.

  However, the predetermined coefficient (α2, α1) belonging to the search range described above detects the face image portion of the person included in the input image, and the skin region representative chromaticity value obtained from the representative skin pixel of the detected face image portion. It is obtained using A (r0, b0). Even if the photographing light source is a light source type that is not suitable for measurement of black body radiation, the skin region representative chromaticity value A (r0, b0) is assumed to be black body radiation if the white balance of the input image is appropriate. It is considered that it enters on or near the skin black body locus H created assuming a light source type (for example, sunlight) suitable for measurement. The human face image portion included in the input image is detected, and the above-described search range obtained using the skin region representative chromaticity value A (r0, b0) obtained from the representative skin pixel of the detected face image portion is included. The predetermined coefficients (α2, α1) belong to the accuracy of the gray pixel detection process using the skin black body locus H and the gray black body locus G, even if the photographing light source is not suitable for the measurement of black body radiation. It is also what prescribes such a range that does not cause a relatively low decrease.

  As will be described later, the predetermined coefficient finally determined is determined from predetermined coefficients (α2, α1) in the search range. Based on the finally determined predetermined coefficient, gray pixels included in the input image are detected, and the color temperature of the photographing light source is estimated (determined) based on the detected gray pixels. It can be said that the determination of the search range of the predetermined coefficient is equivalent to determining the color temperature range of the photographing light source.

  19 and 20 are flowcharts showing the flow of processing in the gray pixel detection unit 20A. 4 is different from the flowchart showing the flow of processing in the gray pixel detection unit 20 of the first embodiment shown in FIG. 4 in that the processing (steps 91 and 92) using the search range of the predetermined coefficient is added. The same processes as those in the flowchart shown in FIG. 4 are denoted by the same reference numerals, and a detailed description thereof is avoided.

  The coefficient multiplying circuit 21A includes the skin area representative values (tfR, tfG, tfB) obtained by the above-described skin area representative value calculating circuit 13 and the reduced image data generated by the reduced image generating circuit 11 (which constitutes a reduced image). RGB values for each of a plurality of pixels) and a large number of predetermined coefficients (α2, α1) to which data for identifying whether it is a search range or not are input.

  The coefficient multiplication circuit 21A selects one of a number of predetermined coefficients (α2, α1) given from the search target coefficient range setting circuit 71 (step 91). As described above, each of a large number of predetermined coefficients (α1, α2) is given data (identification code) indicating whether it is in the search range or not. If the selected predetermined coefficient (α2, α1) is not in the search range (NO in step 92), another predetermined coefficient (α2, α1) is selected (step 91) (predetermined coefficient not in the search range ( α2, α1) means that the pixel is not suitable for gray candidate pixel detection).

When the selected predetermined coefficient (α2, α1) is within the search range (YES in step 92), the selected predetermined coefficient (α2, α1) is the R component of the skin region representative value (tfR, tfG, tfB). And the G component (step 51) and converted into chromaticity values (calculation of coefficient multiplied skin region representative chromaticity values (r ₁ , b ₁ )) (step 52). As in the first embodiment, coefficient multiplication skin region chromaticity values (r ₁ , b ₁ ), data representing the skin black body locus H stored in the memory 26, data representing the gray black body locus G, and the like were used. Gray pixel detection processing is performed (FIG. 20). In the process of calculating the predetermined coefficient (α2, α1) that maximizes the gray candidate pixel count N (FIG. 20, step 62), the predetermined coefficient (α2, α1) belonging to the search range is calculated (selected). Needless to say.

Modification In the second embodiment described above, the point C (rn, bn) on the skin black body locus H closest to the skin region representative chromaticity value A (r0, b0) and a number of predetermined coefficients (α2, α1) A number of predetermined coefficients (α 2, α 1) in the search range or not in the search range based on the distance L with the multiple coefficient multiplication skin region representative chromaticity values D (r 1, b 1) obtained using These are distinguished (see FIG. 18). In the modified example, the weight w is assigned to each of the large number of predetermined coefficients (α2, α1) instead of distinguishing the large number of predetermined coefficients (α2, α1) from those in the search range or those not in the search range. Is granted.

  The weight w for each of a large number of predetermined coefficients (α2, α1) is calculated by the following equation.

w (α2, α1) = 1-10 · L (when L ≦ 0.1)
w (α2, α1) = 0 (when L ≦ 0.1) Equation 10

  In Expression 10, L is obtained by using a point C (rn, bn) on the skin black body locus H closest to the skin region representative chromaticity value A (r0, b0) and a predetermined coefficient (α2, α1). This is a distance from a large number of coefficient-multiplied skin region representative chromaticity values D (r1, b1).

  When the weight w is used, an evaluation function value using the calculated gray candidate pixel number N is calculated. The evaluation function formula f (α2, α1) is expressed by the following formula.

  f (α2, α1) = − w (α2, α1) · N Equation 11

  The evaluation function may be any function as long as the weighting factor w increases and the evaluation function value decreases, and as the gray candidate pixel count N increases, the evaluation function value decreases. For example, the evaluation function expression may be:

  f (α2, α1) = − w (α1, α2) · log10 (1 + N) Expression 12

  When the weight w and the evaluation function formula f are used, instead of searching for the coefficient (α2, α1) that maximizes the gray candidate pixel number N (step 62 in FIG. 20), the evaluation function value f (α2, α1) is used. The coefficient (α2, α1) that minimizes) is retrieved.

  When the evaluation function value f (α2, α1) is large, there is a possibility that an erroneous gray pixel is detected. In this case, the reward coefficient c may be calculated according to the evaluation function value f (α2, α1), and the white balance correction coefficient may be corrected by the calculated reward coefficient c. FIG. 21 shows a graph (lookup table) representing the relationship between the evaluation function value f (α2, α1) and the lowward coefficient c.

  In the graph showing the relationship between the evaluation function value f (α2, α1) and the lowward coefficient c shown in FIG. 21, the evaluation function value f decreases as the weighting coefficient w increases, and the evaluation function value increases as the number of gray candidate pixels increases. This is a case where an evaluation function expression is defined so that f becomes small. When the evaluation function expression is defined so that the evaluation function value f is larger as the weighting coefficient w is larger and the evaluation function value f is larger as the gray candidate pixel number N is larger, the reward coefficient c is the evaluation function value f ( Needless to say, it should be a monotonically increasing function for (α2, α1).

  In the modification of the second embodiment described above, instead of calculating the weight w for each of a large number of predetermined coefficients (α2, α1), first, the skin area representative chromaticity value A (r0, b0) is the most. A distance L between a point C (rn, bn) on the near skin black body locus H and a number of coefficient-multiplied skin region representative chromaticity values D (r1, b1) obtained by using a number of predetermined coefficients (α2, α1). Based on the above, a large number of predetermined coefficients (α2, α1) are distinguished from those in the search range and those outside the search range, and the weight w is calculated only for the predetermined coefficient (α2, α1) in the search range. Good.

It is a chromaticity diagram showing a gray black body locus G. 1 is a block diagram showing an overall configuration of a digital print system. It is a block diagram which shows the electric constitution of the image correction apparatus in 1st Example. It is a flowchart which shows the flow of a process of a gray pixel detection part. On the chromaticity diagram, a skin black body locus H, a gray black body locus G, a plot point A of the skin region representative chromaticity value, and a plot point B of the coefficient multiplication skin region representative chromaticity value are shown. A state in which a provisional color temperature is obtained is shown. The plot points on the skin black body locus H corresponding to the provisional color temperature and the plot points on the gray black body locus G corresponding to the provisional color temperature are shown. A gray candidate pixel range is shown. The relationship between the distance L3 between the skin area representative chromaticity value subjected to white balance correction and the reference skin color chromaticity value and the lowward coefficient c is shown. The relationship between the number of gray candidate pixels N and the lowward coefficient c is shown. It is a block diagram which shows the electrical structure of the modification of an image correction apparatus. On the chromaticity diagram, the black body locus and the black body locus for white people, yellow people, and black people are shown. It is a block diagram which shows the electrical structure of the modification of an image correction apparatus. On the chromaticity diagram, the skin black body locus and gray black body locus for camera A, and the skin black body locus and gray black body locus for camera B are shown. It is a block diagram which shows the electric constitution of the image correction apparatus in 2nd Example. It is a flowchart which shows the flow of a search object coefficient range setting process. In the chromaticity diagram, the skin black body locus, the plot point A of the skin region representative chromaticity value, and the plot point C on the skin black body locus H closest to the plot point A are shown. A predetermined distance range from the plot point C and the plot point D of the coefficient multiplication skin region representative chromaticity value are shown. It is a flowchart which shows the flow of a process of a gray pixel detection part. It is a flowchart which shows the flow of a process of a gray pixel detection part. The relationship between the evaluation function value and the reward coefficient is shown.

Explanation of symbols

1, 1A Image correction device 2 Input device 3 Display device 4 Storage device 5 Printer
11, 41 Reduced image generation circuit
12 Skin area detection circuit
13 Skin area representative value calculation circuit
21, 21A Coefficient multiplication circuit
22 Chromaticity conversion circuit
23 Gray candidate pixel detection circuit
24 Gray candidate pixel count calculation circuit
25 Coefficient optimization circuit
26 memory
31 Color temperature calculation circuit
32 Correction coefficient calculation circuit
42 White balance correction circuit
43 Enlarged image generation circuit
71 Search target coefficient range setting circuit

Claims (12)

A skin image region detecting means for detecting a skin image region included in an image represented by given image data based on an image shape or an image structure;
Color temperature range narrowing means for narrowing down the color temperature range of the photographic light source that the gray pixel to be detected should have based on color information obtained from the pixels constituting the skin image area detected by the skin image area detecting means ,
Gray pixel detection means for detecting gray pixels included in the image represented by the image data based on color information obtained from the pixels constituting the skin image area detected by the skin image area detection means; and Color temperature estimation means for estimating the color temperature of the photographic light source based on the gray pixels detected by the gray pixel detection means ,
The gray pixel detection means is:
A pixel having a color temperature included in the color temperature range narrowed down by the color temperature range narrowing means is detected as a detection target candidate, and gray pixels are detected from the detection target candidates.
Image processing device. The gray pixel detection means is:
First coefficient multiplication means for multiplying a representative value of color information obtained from pixels constituting the skin image area by a predetermined coefficient;
First chromaticity value conversion means for converting the coefficient multiplication skin region representative value calculated by the first coefficient multiplication means to a chromaticity value;
Determining means for determining whether the coefficient-multiplied skin region representative chromaticity value obtained by the first chromaticity value converting means is present in the vicinity of the skin black body locus;
When it is determined by the determining means that the coefficient multiplication skin region representative chromaticity value is present in the vicinity range of the skin black body locus, on the skin black body locus closest to the coefficient multiplication skin region representative chromaticity value Provisional color temperature estimation means for estimating a color temperature corresponding to the point as a provisional color temperature;
A color information obtained from each of the pixels constituting the image represented by the image data is multiplied by the predetermined coefficient that causes the coefficient-multiplied skin region representative chromaticity value to exist in the vicinity of the skin black body locus. Coefficient multiplication means, and second chromaticity value conversion means for converting a plurality of coefficient multiplication pixel values obtained by the second coefficient multiplication means into chromaticity values,
Among the pixels constituting the image represented by the image data, the coefficient multiplication pixel chromaticity value obtained by the second chromaticity value conversion means is the provisional color estimated by the provisional color temperature estimation means. A pixel that falls in the vicinity of a point on the gray blackbody locus corresponding to the temperature is specified, and the specified pixel is detected as a gray pixel.
The image processing apparatus according to claim 1. An optimization coefficient search means for searching for the predetermined coefficient that causes the coefficient multiplication skin region representative chromaticity value to be present in the vicinity of the skin black body locus, wherein the number of gray pixels is maximized;
The first coefficient multiplication means and the second coefficient multiplication means are:
Using a predetermined coefficient searched by the optimization coefficient search means,
The image processing apparatus according to claim 2. The above color temperature range narrowing means is:
Third coefficient multiplying means for multiplying a representative value of color information obtained from the pixels constituting the skin image area detected by the skin image area detecting means by each of a plurality of predetermined coefficients;
A third chromaticity value converting means for converting a plurality of coefficient multiplied skin region representative values calculated by the third coefficient multiplying means to chromaticity values, and the third chromaticity value converting means obtained by the third chromaticity value converting means; Skin blackness closest to the skin region representative chromaticity value obtained by converting the representative value of color information obtained from the pixels constituting the skin image region to a chromaticity value among a plurality of coefficient multiplication skin region representative chromaticity values A coefficient range search means for searching for a range of the predetermined coefficient in which the coefficient-multiplied skin region representative chromaticity value is calculated such that the distance from the value on the body locus is within the predetermined distance;
The image processing apparatus according to claim 1 . The above color temperature range narrowing means is:
Third coefficient multiplying means for multiplying a representative value of color information obtained from the pixels constituting the skin image area detected by the skin image area detecting means by each of a plurality of predetermined coefficients;
A third chromaticity value converting means for converting a plurality of coefficient multiplied skin region representative values calculated by the third coefficient multiplying means to chromaticity values, and the third chromaticity value converting means obtained by the third chromaticity value converting means; A skin black body closest to a skin area representative chromaticity value obtained by converting a plurality of coefficient multiplication skin area representative chromaticity values and a representative value of color information obtained from pixels constituting the skin image area into a chromaticity value A weighting means for calculating a distance from a value on the trajectory and weighting each of the plurality of predetermined coefficients according to the calculated distance;
The gray pixel detection means is:
Fourth coefficient multiplying means for multiplying the representative value of color information obtained from the pixels constituting the skin image area by the predetermined coefficient;
Fourth chromaticity value conversion means for converting the coefficient multiplication skin region representative value calculated by the fourth coefficient multiplication means to a chromaticity value;
Determining means for determining whether the coefficient-multiplied skin region representative chromaticity value obtained by the fourth chromaticity value converting means is present in the vicinity of the skin black body locus;
When it is determined by the determining means that the coefficient multiplication skin region representative chromaticity value is present in the vicinity range of the skin black body locus, on the skin black body locus closest to the coefficient multiplication skin region representative chromaticity value Provisional color temperature estimation means for estimating the color temperature corresponding to the value of
A fifth coefficient multiplication that multiplies the color information obtained from the pixels constituting the image represented by the image data by a predetermined coefficient that causes the coefficient-multiplied skin region representative chromaticity value to exist in the vicinity of the skin body locus. And fifth chromaticity value conversion means for converting a plurality of coefficient multiplication pixel values obtained by the fifth coefficient multiplication means into chromaticity values,
Of the pixels constituting the image represented by the image data, the provisional color estimated by the provisional color temperature estimation means is the coefficient multiplication pixel chromaticity value calculated by the fifth chromaticity value conversion means. A pixel that falls in the vicinity of a value on a gray black body locus corresponding to temperature is identified, and the identified pixel is detected as a gray pixel.
Using such an evaluation function, the gray pixel detection means uses a value that decreases as the number of gray pixels detected by the gray pixel detection means increases, and decreases as the weight for the predetermined coefficient increases. Means for calculating an evaluation function value based on the number of gray pixels detected by the above and the weight of the predetermined coefficient used in the fourth and fifth coefficient multiplying means, and a predetermined coefficient that minimizes the evaluation function value Further comprising an optimization coefficient search means for searching for
The fourth and fifth coefficient multiplication means use predetermined coefficients searched by the optimization coefficient search means.
The image processing apparatus according to claim 1 . The above color temperature range narrowing means is:
Third coefficient multiplying means for multiplying a representative value of color information obtained from the pixels constituting the skin image area detected by the skin image area detecting means by each of a plurality of predetermined coefficients;
A third chromaticity value converting means for converting a plurality of coefficient multiplied skin region representative values calculated by the third coefficient multiplying means to chromaticity values, and the third chromaticity value converting means obtained by the third chromaticity value converting means; A skin black body closest to a skin area representative chromaticity value obtained by converting a plurality of coefficient multiplication skin area representative chromaticity values and a representative value of color information obtained from pixels constituting the skin image area into a chromaticity value A weighting means for calculating a distance from a value on the trajectory and weighting each of the plurality of predetermined coefficients according to the calculated distance;
The gray pixel detection means is:
Fourth coefficient multiplying means for multiplying the representative value of color information obtained from the pixels constituting the skin image area by the predetermined coefficient;
Fourth chromaticity value conversion means for converting the coefficient multiplication skin region representative value calculated by the fourth coefficient multiplication means to a chromaticity value;
Determining means for determining whether the coefficient-multiplied skin region representative chromaticity value obtained by the fourth chromaticity value converting means is present in the vicinity of the skin black body locus;
When it is determined by the determining means that the coefficient multiplication skin region representative chromaticity value is present in the vicinity range of the skin black body locus, on the skin black body locus closest to the coefficient multiplication skin region representative chromaticity value Provisional color temperature estimation means for estimating the color temperature corresponding to the value of
A fifth coefficient multiplication that multiplies the color information obtained from the pixels constituting the image represented by the image data by a predetermined coefficient that causes the coefficient-multiplied skin region representative chromaticity value to exist in the vicinity of the skin black body locus. And fifth chromaticity value conversion means for converting a plurality of coefficient multiplication pixel values obtained by the fifth coefficient multiplication means into chromaticity values,
Among the pixels constituting the image represented by the image data, the coefficient-multiplied pixel chromaticity value calculated by the fifth chromaticity value conversion means is the provisional color estimated by the provisional color temperature estimation means. A pixel that falls in the vicinity of the value on the gray blackbody locus corresponding to the temperature is identified, and the identified pixel is detected as a gray pixel.
Using such an evaluation function, the gray pixel detection means uses a value that increases as the number of gray pixels detected by the gray pixel detection means increases and increases as the weight for the predetermined coefficient increases. Means for calculating an evaluation function value on the basis of the number of gray pixels detected by the above and the weights of the predetermined coefficients used in the fourth and fifth coefficient multiplying means, and a predetermined coefficient that maximizes the evaluation function value Optimization coefficient search means for searching for
The fourth and fifth coefficient multiplication means use predetermined coefficients searched by the optimization coefficient search means.
The image processing apparatus according to claim 1 . The color temperature estimation means is
Calculating the color temperature of each gray pixel detected by the gray pixel detecting means;
The average value of the calculated color temperatures is used as the color temperature of the photographic light source.
The image processing apparatus according to claim 1. Based on the color temperature of the photographic light source estimated by the color temperature estimation means, a white balance correction value calculation means for calculating a white balance correction value, and a correction value calculated by the white balance correction value calculation means, White balance correction means for performing white balance correction on the image data;
The image processing apparatus according to claim 1, further comprising: When the difference between the representative value of the skin image area subjected to white balance correction by the white balance correction means and the skin image reference value is larger than a predetermined value, the white balance correction value is reduced so as to weaken the correction effect of the white balance correction. First reward means for correcting
The image processing apparatus according to claim 8 , further comprising: A second reward means for correcting the white balance correction value so as to weaken the correction effect of the white balance correction when the number of gray pixels detected by the gray pixel detection means is smaller than a predetermined value;
The image processing apparatus according to claim 8 , further comprising: Detecting a skin image area included in an image represented by given image data based on an image shape or an image structure;
Based on the color information obtained from the pixels constituting the detected skin image area, the range of the color temperature of the photographing light source that the gray pixel to be detected should have is narrowed down,
Based on the color information obtained from the pixels constituting the detected skin image area, the gray pixels included in the image represented by the image data are converted into pixels having a color temperature included in the narrowed color temperature range. Detect from inside ,
Estimate the color temperature of the photographic light source based on the detected gray pixels.
Image processing method. Skin image area detection processing for detecting a skin image area included in an image represented by given image data based on an image shape or an image structure;
A color temperature range narrowing-down process for narrowing down the color temperature range of the photographing light source that should be possessed by the gray pixel to be detected, based on the color information obtained from the pixels constituting the detected skin image area;
Based on the color information obtained from the pixels constituting the skin image area detected by the skin image area detection process, the gray pixels included in the image represented by the image data are set to the narrowed color temperature range. A gray pixel detection process for detecting a pixel having a color temperature included , and a color temperature estimation process for estimating a color temperature of a photographing light source based on the gray pixel detected by the gray pixel detection process;
An image processing program for causing a computer to execute.

Images