JP4052001B2 - Image processing apparatus and method, recording medium, and program - Google Patents

Description

[0001]
BACKGROUND OF THE INVENTION
The present invention relates to an image processing apparatus and method, a recording medium, and a program, and is particularly suitable for application to an apparatus that adjusts the color of an entire image so that a specific color becomes a desired color by adjusting white balance. The present invention relates to an image processing apparatus and method, a recording medium, and a program.
[0002]
[Prior art]
Cameras that handle color images, both digital and analog, such as video cameras and still cameras, are widespread, but the colors captured by these cameras differ from those seen by the human eye. There is a case. This is because, for example, there is a light source with a strong red color or a light source with a strong green color due to the spectral distribution of the light source, and the color balance of the reflected light from the subject that has received light from the light source is lost. However, in a digital camera or the like using a light receiving element, the cause of sensing the color of the light source is an example. As a result, there are cases where the image is reddish or greenish as a whole.
[0003]
To prevent this, the digital camera has a function to adjust the color balance so that the color of the subject image captured by the light receiving element is adjusted to the same color as the subject seen by the human eye. ing. Such color balance adjustment is generally called white balance adjustment in the sense that the color under a white light source with appropriate color balance is reproduced. This is performed using the intensity ratio of the three primary colors of R (red), G (green), and B (blue) under a light source.
[0004]
[Problems to be solved by the invention]
As described above, when adjusting the color tone of a subject captured by a digital camera so that the color tone is the same as that of a subject seen by the human eye, various situations, for example, shooting in sunlight, indoors There is a problem that it is difficult to always perform the optimum adjustment under different circumstances such as shooting with the camera.
[0005]
Even if the color tone is adjusted by adjusting the white balance or the like, there is a possibility that the wrong adjustment is made based on an erroneous determination, which may give an uncomfortable feeling to a person who views the image. In particular, humans are characterized in that judgments about memory colors such as skin color, green color, and sky blue color are more sensitive than other colors. For this reason, in an image in which these colors are captured, there is a problem that if the color tone of the subject recognized by the user is different from that of the other colors, it reacts more sensitively and feels uncomfortable. It was.
[0006]
Even if the color tone of the captured image faithfully reproduces the color tone of the subject, the reproduction may cause the user to feel uncomfortable. For example, as described above, although humans are sensitive to skin color, it may be preferable for the skin color that the image acquired by the camera is different from the color of the actual image. For example, even if a white person actually has a reddish skin color, when he sees an image that faithfully reproduces that color, he or she feels that the skin color should be white. Experience has shown that it can be uncomfortable. Therefore, there is a problem that it is not always desirable to faithfully reproduce actual colors.
[0007]
In addition, in a close-up image of a person's face, there is a problem that it is difficult to distinguish white under a light source with a low color temperature. For this reason, an erroneous determination is made, and based on the determination, color correction, etc. Since the process has been executed, there is a problem that the skin color is reproduced as pale.
[0008]
If the user does not like the color tone of the image acquired by the digital camera, the user can change (process) the color tone of the image using predetermined software. However, with respect to still images, there is a problem that even if such processing can be performed, it cannot be applied to moving images, or even if it can be performed.
[0009]
In addition, correction using such software strongly reflects the taste and sense of the user who made the correction, so it may be far from the actual color tone and reproduce the actual color tone. There was a problem that the situation was not so favorable in the sense of doing.
[0010]
The present invention has been made in view of such a situation, and an object thereof is to reproduce a desired color such as a skin color as a preferable color by adjusting white balance.
[0011]
[Means for Solving the Problems]
An image processing apparatus according to the present invention is based on an imaging unit that captures an image, a determination unit that divides an image captured by the imaging unit into a plurality of regions, and determines colors in the region, and a determination result by the determination unit. And a predetermined number of areas corresponding to the four corners of the image are set in advance as processing targets. Skin color A determination means for determining whether or not there is an area that is determined to be Skin color Target value calculating means for calculating a target value for white balance adjustment when it is determined that there is no area determined to be A difference value calculating means for calculating a difference value between a luminance value of a first area of the plurality of areas and a luminance value of a second area located around the first area, and a difference value calculating means The number of first areas where the calculated difference value is greater than or equal to a predetermined value is greater than the number of first areas where the calculated difference value is less than or equal to the predetermined value, and the average of the luminance values of areas located at predetermined positions in the image If the value is less than or equal to a predetermined value, or the number of first regions where the difference value calculated by the difference value calculation means is greater than or equal to the predetermined value is equal to the number of first regions that are less than or equal to the predetermined value And an adjustment means for adjusting white balance using the target value calculated by the target value calculation means when the number of skin color areas in the image is within a predetermined number. It is characterized by providing.
[0014]
The image processing method according to the present invention divides a captured image into a plurality of regions, determines colors in the regions, and determines in advance a predetermined number of regions corresponding to the four corners of the image based on the determination results. Is set as Skin color Determine whether there is an area that is determined to be Skin color If it is determined that there is no area determined to be, the target value for white balance adjustment is calculated. Then, a difference value between the luminance value of the first area of the plurality of areas and the luminance value of the second area located around the first area is calculated, and the calculated difference value is a predetermined value. When the number of the first areas is larger than the number of the first areas which are equal to or less than the predetermined value, and the average value of the luminance values of the areas located at the predetermined positions of the image is equal to or less than the predetermined value Alternatively, the number of first areas where the calculated difference value is equal to or greater than a predetermined value is equal to the number of first areas where the calculated difference value is equal to or less than the predetermined value, and the number of skin color areas within the predetermined number is within the predetermined number. If there is, adjust the white balance using the calculated target value. Including steps.
[0015]
The recording medium program of the present invention divides a captured image into a plurality of regions, determines colors in the regions, and processes in advance into a predetermined number of regions corresponding to the four corners of the image based on the determination result. It is set as target Skin color Determine whether there is an area that is determined to be Skin color If it is determined that there is no area determined to be, the target value for white balance adjustment is calculated. Then, a difference value between the luminance value of the first area of the plurality of areas and the luminance value of the second area located around the first area is calculated, and the calculated difference value is a predetermined value. When the number of the first areas is larger than the number of the first areas which are equal to or less than the predetermined value, and the average value of the luminance values of the areas located at the predetermined positions of the image is equal to or less than the predetermined value Alternatively, the number of first areas where the calculated difference value is equal to or greater than a predetermined value is equal to the number of first areas where the calculated difference value is equal to or less than the predetermined value, and the number of skin color areas within the predetermined number is within the predetermined number. If there is, adjust the white balance using the calculated target value. Including steps.
[0016]
The program of the present invention divides a captured image into a plurality of areas, determines colors in the areas, and sets in advance a predetermined number of areas corresponding to the four corners of the image as processing targets based on the determination results. Has been Skin color Determine whether there is an area that is determined to be Skin color If it is determined that there is no area determined to be, the target value for white balance adjustment is calculated. Then, a difference value between the luminance value of the first area of the plurality of areas and the luminance value of the second area located around the first area is calculated, and the calculated difference value is a predetermined value. When the number of the first areas is larger than the number of the first areas which are equal to or less than the predetermined value, and the average value of the luminance values of the areas located at the predetermined positions of the image is equal to or less than the predetermined value Alternatively, the number of first areas where the calculated difference value is equal to or greater than a predetermined value is equal to the number of first areas where the calculated difference value is equal to or less than the predetermined value, and the number of skin color areas within the predetermined number is within the predetermined number. If there is, adjust the white balance using the calculated target value. A program that causes a computer to execute steps.
[0017]
In the image processing apparatus, method, and program of the present invention, a captured image is divided into a plurality of regions, the colors in the regions are determined, and a predetermined number of regions corresponding to the four corners of the image are processed in advance. Is set as Skin color If it is determined that there is no area, the target value for white balance adjustment is calculated. The difference value between the luminance value of the first region of the plurality of regions and the luminance value of the second region located around the first region is calculated, and the calculated difference value is equal to or greater than a predetermined value. The number of the first areas is greater than the number of the first areas that are equal to or less than the predetermined value, and the average value of the luminance values of the areas located at the predetermined positions of the image is equal to or less than the predetermined value The number of first areas where the calculated difference value is equal to or greater than a predetermined value is equal to the number of first areas where the calculated difference value is equal to or less than the predetermined value, and the number of skin color areas within the predetermined number is within the predetermined number. If the calculated target value is used, the white balance is adjusted. Is done.
[0018]
DETAILED DESCRIPTION OF THE INVENTION
Embodiments of the present invention will be described below with reference to the drawings. FIG. 1 is a diagram showing the configuration of an embodiment of an image processing apparatus to which the present invention is applied. An image processing apparatus 10 illustrated in FIG. 1 is an apparatus that is provided in a digital video camera or a still camera and corrects a color of a captured image (video).
[0019]
A subject image (light) is received by a CCD (Charge Coupled Devices) 12 via a lens 11 of the image processing apparatus 10. The CCD 12 converts the intensity of the received light into an electric signal and supplies it to an S / H (Sampling / Hold) circuit 13. The S / H circuit 13 samples an input electric signal based on a predetermined sampling frequency, and temporarily holds the sampled value.
[0020]
The held value (signal) is supplied to an AGC (Automatic Gain Control) circuit 14. The AGC circuit 14 converts the input signal to an appropriate signal level for subsequent signal processing, and supplies the signal to an A / D (Analog / Digital) conversion circuit 15. The A / D conversion circuit 15 converts the input RGB color analog signal into a digital signal and outputs the digital signal to the RGB matrix circuit 16.
[0021]
The RGB matrix circuit 16 converts the input RGB color digital signals into digital signals of respective components of R (red), G (green), and B (blue), and a WB (white balance) amplifier 17 and WB. Output to the adjustment circuit 18. The WB adjustment circuit 18 is a circuit for adjusting the gain in the WB amplifier 17, and its detailed configuration is shown in FIG. 2 and will be described later.
[0022]
The WB amplifier 17 adjusts the white balance by multiplying the input R, G, B digital signals by a gain based on an instruction from the WB adjustment circuit 18 and supplies the white balance to the color difference calculation circuit 19. .
[0023]
Here, the white balance will be briefly described. When R (red), G (green), and B (blue) are added (R + G + B, where R = G = B), white is generated. For example, when papers that humans recognize as white are compared to images taken under clear sky and images taken under cloudy skies without white balance processing, the white paper in those images is , They are not recognized as the same whiteness. That is, white is not necessarily captured as a color that is necessarily recognized as white, depending on shooting conditions and the like.
[0024]
In view of this, white balance adjustment is basically performed on the signal of the imaged image so that white is recognized as white, regardless of the situation of the imaged image.
[0025]
The color difference calculation circuit 19 generates a color difference signal using the R signal, the G signal, and the B signal whose white balance has been adjusted. The generated color difference signals are an RG signal and a BG signal. In order to express R (red) and B (blue) with G (green) as a reference, such a color difference is calculated.
[0026]
Further, the color difference matrix circuit 20 converts the signal into an RY signal and a BY signal. Here, Y represents a luminance signal (luminance value). This Y signal is supplied from a circuit not shown.
[0027]
FIG. 2 is a diagram illustrating an internal configuration example of the WB adjustment circuit 18. The WB adjustment circuit 18 is supplied with the R signal, the G signal, and the B signal from the RGB matrix circuit 16, respectively, and these signals are supplied to the detection circuit 31. The detection circuit 31 calculates a luminance value, an integral value, and the like for each detection region described later, and outputs values necessary for the circuit to the WB amplifier 32 and the luminance peak value calculation circuit 33, respectively.
[0028]
The WB amplifier 32 adjusts the white balance with respect to the inputted R, G, and B signals for each region with a predetermined constant gain, and outputs the result to the color difference calculation circuit 34. The color difference calculation circuit 34 performs the same processing as the color difference calculation circuit 19 (FIG. 1). That is, an RG signal and a BG signal are generated from the input signal. The generated signal is supplied to the WB value calculation circuit 35.
[0029]
The WB value calculation circuit 35 determines whether or not a skin color is included in each area described later. If included, the included skin color is captured by a camera including the image processing apparatus 10. The white balance gain is calculated so that an ideal skin color is obtained in the obtained image.
[0030]
On the other hand, the output from the detection circuit 31 is also output to the luminance peak value calculation circuit 33. The luminance peak value calculation circuit 33 uses the luminance value, and the image in each region is an uneven image. It is determined whether or not there is, and the result is output to the WB value setting circuit 36.
[0031]
The WB value setting circuit 36 finally corrects the gain of the WB amplifier 17 (FIG. 1) based on the information supplied from the WB value calculation circuit 35 and the information supplied from the luminance peak value calculation circuit 33. The WB amplifier 17 is controlled based on the determination.
[0032]
The operation of the WB adjustment circuit 18 will be described with reference to the flowcharts of FIGS. FIG. 4 is a flowchart for explaining the details of the color discrimination processing in the region performed in step S11 of FIG. FIG. 5 is a flowchart for explaining details of the correction amount calculation processing performed in step S12 of FIG. FIG. 6 is a flowchart for explaining the details of the face portion discrimination process performed in step S13 of FIG. FIG. 7 is a flowchart for explaining the details of the correction process performed in step S14 of FIG.
[0033]
In step S11, the WB adjustment circuit 18 first performs color discrimination processing within the area. Here, the region (detection region) will be described. FIG. 8 is a diagram for explaining a region. One image picked up by the processing of the lens 11 and the CCD 12 is divided into a plurality of regions as shown in FIG. In the example shown in FIG. 8, one image is divided into 7 × 7 areas. Each area is assigned a number so that each area can be uniquely distinguished.
[0034]
In this case, one screen is divided into 7 × 7 areas, and one area is composed of a plurality of pixels. In step S21 (FIG. 4), the detection circuit 31 of the WB adjustment circuit 18 performs detection for each 7 × 7 region and outputs R, G, and B signals to the WB amplifier 32, respectively.
[0035]
In step S22, the WB amplifier 32 adjusts the white balance by multiplying a signal from each of the 7 × 7 areas by a certain gain. In this way, by adjusting the provisional white balance by multiplying by a constant gain, it is possible to accurately perform the detailed white balance adjustment by the subsequent processing.
[0036]
In step S 23, the output from the WB amplifier 32 is input to the color difference calculation circuit 34, and the RG signal and the BG signal are output to the WB value calculation circuit 35 by calculating the color difference. In step S24, the WB value calculation circuit 35 determines the color in each area using the input RG signal and BG signal, and sets color information based on the determination for each area. . The color discrimination will be described.
[0037]
FIG. 9 is a diagram for explaining data used when color determination is performed. The color is determined depending on where in the circle shown in FIG. 9 one coordinate determined by the RG signal and the BG signal exists. In FIG. 9, for example, a circle indicated as skin color indicates a range that is discriminated as skin color when a human sees a color represented by an RG signal and a BG signal located at coordinates in the circle. Represents.
[0038]
The size of the range represents a range in which, for example, a skin color is recognized as a skin color when a human sees the color represented by the RG signal and the BG signal. A small range indicates that a person is sensitive to that color, and a large range indicates that a person is relatively insensitive to that color.
[0039]
In step S24, the color information is set by first determining where the coordinates uniquely determined by the RG signal and the BG signal exist in the circle as shown in FIG. This is done by determining in which range the position is included. The color in the range where the coordinates exist is set as the color in the range that is the target of color information setting. Since the color information indicated by the circle as shown in FIG. 9 is used in this way, the WB adjustment circuit 18 holds the color information indicated by the circle as shown in FIG. 9 as data. By referring to the stored data, the color of the detection area to be processed is determined.
[0040]
As a result of the color discrimination, color information is set in each area. For example, the color information is “1” for an achromatic color, “2” for a skin color, “3” for a sky blue, “4” is set in the case of yellow, “5” is set in the case of yellow. Thus, only the characteristic color is set, and the other colors are set to “0”.
[0041]
Such setting of color information is an example, and the present invention is not limited to this. For example, in the embodiment to be described later, since the main purpose is to correct the skin color, only the skin color may be set. When only the skin color is set in this way, the color information can be set by simply 1 or 0. In other words, when the skin color is set, the flag information can be set and information can be set. It is possible to reduce the amount of data used.
[0042]
In the detection processing in step S21, if each RGB value is a processing result of “000” or “FFF”, it is determined that the region is an error region, and the subsequent processing may not be performed. .
[0043]
In step S11 (FIG. 3), when the color discrimination process in the region is completed, a correction amount calculation process is performed in the WB value calculation circuit 35 in step S12. The WB value calculation circuit 35 calculates the correction amount based on the processing of the flowchart as shown in FIG. Here, the correction amount is a value that determines a gain used for white balance adjustment in the WB amplifier 17 (FIG. 1), and is a value that indicates how much the gain used at that time is corrected. .
[0044]
The WB value calculation circuit 35 discriminates the color of each of the 7 × 7 areas before performing the correction amount calculation process. In step S31 (FIG. 5), it is determined whether or not there is an area determined to be a skin color in the four corner areas of the 7 × 7 area where the color is determined. The four corners are regions indicated by hatching in FIG. 8, and specifically, {[0] [0], [0] [1], [1] [0]} in the upper left and {[ 0] [6], [0] [5], [1] [6]}, lower right {[6] [6], [5] [6], [6] [5]}, and lower left {[6] [0], [5] [0], [6] [1]}.
[0045]
The reason for determining whether there is an area determined to be skin color in the four corner areas is as follows. That is, an image having skin color at the four corners of one screen is considered to be an image in which only the face is reflected on the entire screen and the face is completely cut off, but such an image is rare. . In addition, for an image having skin colors in the four corners, it is more accurate to determine that the image itself in the four corners is not a face but a skin-colored object other than the face, rather than a face. There are many cases.
[0046]
For this reason, it is determined whether there are areas determined to be skin color at the four corners of the captured image. If there are areas determined to be skin color at the four corners, the image is not a face. It is determined that the object is an object, and skin color correction, in this case, white balance correction is not performed. On the other hand, if it is determined in step S31 that there are no areas determined to be skin color at the four corners, the process proceeds to step S32, and the average value of the skin color area is calculated.
[0047]
The skin color area for which the average value is calculated is set as color information called skin color in the center 9 areas (area shown in the middle and hatched in FIG. 8) among the 7 × 7 areas. It is an area. As described above, the reason why only the central part of one image is processed is that the face usually exists near the center of the screen.
[0048]
Of course, a face may exist in a place other than the vicinity of the center of the image, but in such a case, there are many situations where the face is not mainly used. In addition, when there is a face other than near the center of the image, the landscape and the face (person) are often mainly captured at the same time. In such an image, the ratio of the face to the landscape is It seems to be relatively small.
[0049]
In consideration of such a situation, when it is determined that a face exists in the center portion of the image, the white balance is adjusted so that the skin color of the face becomes an ideal skin color. Therefore, here, the average value of the detection values of the skin color of the nine regions located in the center of the image is calculated in step S32. When the average value of the detection values of the skin color area is calculated in step S32, the target value of the WB value is set in step S33.
[0050]
The target value of the WB value is set as follows. Here, a case where G (green) is used as a reference will be described as an example. Here, among the nine regions located at the center of the image described above, the R signal of the region determined to be a skin color region is the Rr signal, the B signal is the Br signal, and the G signal is the Gr signal. These Rr signal, Br signal, and Gr signal are signals output from the detection circuit 31, respectively.
[0051]
The Rr signal, Br signal, and Gr signal are respectively added, and the value obtained by dividing by the added frame number, that is, the average value of each signal is defined as the Rr ′ value, Br ′ value, and Gr ′ value. Here, if Ggain of the white balance gain applied to the G signal is fixed, first, Gw = Gr × Ggain is established. When the average value of Gw is Gw ′, the following expression is established.
Rgain = ((RG) i + Gw ') / Rr'
Bgain = ((B−G) i + Gw ′) / Br ′
[0052]
In the above formula, (R−G) i and (B−G) i are values obtained empirically (experimental) when realizing a preferable skin color. These values can be set in accordance with the destination of the camera including the WB adjustment circuit 18, for example. This is because the preferred skin color may differ depending on the destination, so that it can be adapted to such a case.
[0053]
When the ideal (target) WB value is calculated by such an expression, the process of the flowchart shown in FIG. 5 (the process of step S12 in FIG. 3) is terminated, and the process proceeds to step S13, where the face portion is determined. The discrimination process is executed. With reference to the flowchart of FIG. 6, the face part discrimination process will be described. The determination processing of the face portion is performed by the luminance peak value calculation circuit 33.
[0054]
As shown in FIG. 8, this face portion discrimination process is performed by performing the process described later in each of the areas divided into 7 × 7 areas. As shown in FIG. 10, 25 areas of 5 × 5 located at the center of the image, in other words, areas excluding the outer periphery.
[0055]
In step S41, the peak value of the luminance value is calculated for each of the 25 areas. In step S42, it is determined whether or not one or more skin-colored areas exist in the eight areas around the predetermined area (the area to be processed). The color information indicating whether or not it is a skin color region may be supplied from the WB value calculation circuit 35.
[0056]
If it is determined in step S42 that one or more skin-colored areas are present among the eight areas located around the area to be processed, the process proceeds to step S43 and is determined not to exist. If YES, go to step S47. In step S43, the difference between the peak values of the luminance values is calculated. The areas for which the difference is calculated are the area to be processed and the area determined to be the skin color.
[0057]
In step S44, it is determined whether or not the difference value between the peak values of the luminance values calculated in step S43 is equal to or less than a predetermined value. When it is determined that the difference value between the peak values of the luminance values is equal to or less than the predetermined value, the process proceeds to step S45, and the area to be processed is set as a flat area (hereinafter referred to as F area). If it is determined that the difference value between the peak values of the luminance values is not equal to or smaller than the predetermined value, the process proceeds to step S46, and the area to be processed is a non-flat area (hereinafter referred to as D area). ) Is set.
[0058]
As described above, when the difference in the peak value of the luminance value between the region to be processed and one of the surrounding eight regions is large, the region in which an uneven object such as a face is imaged. It can be determined that there is. Therefore, as described above, it can be divided into a flat region (F region) and a non-flat region (D region) based on the difference value of the luminance values.
[0059]
In step S47, it is determined whether or not the setting of the F area or the D area has been completed for all areas to be processed, in this case, 25 areas. If it is determined in step S47 that the setting of the F area or the D area has been completed for the 25 areas, the face portion discrimination process shown in FIG. 6 is completed, and the process proceeds to step S14 in FIG. If it is determined that has not been completed, the process returns to step S42, and the subsequent processing is repeated.
[0060]
The correction process in step S14 (FIG. 3) will be described with reference to the flowchart of FIG. The correction process shown in FIG. 7 is performed in the WB value setting circuit 36. This correction processing mainly determines whether or not the face size is large. When the area determined to be a face is large, white balance adjustment is performed and the face is determined to be relatively small. In the process, white balance adjustment is not performed.
[0061]
In this embodiment, the determination of the face size and the determination of whether or not to adjust the white balance is performed in this embodiment by adjusting the white balance so that the skin color of the face becomes an ideal skin color. It is because it performs. If the white balance is adjusted according to the skin color of the face when the face area is small in the captured image, even if the color tone matches the skin color of the small face, Is considered to be an image in which an unfavorable white balance adjustment has been performed. In order to prevent such an image, the size of the face is determined.
[0062]
That is, in this embodiment, if the size of the face being imaged does not occupy a predetermined area of one screen, white balance adjustment that matches the skin color of the face is not executed. By doing so, it is possible to make the face color a preferable skin color and to prevent the color tone of the entire image from becoming strange.
[0063]
In step S51, the WB value setting circuit 36 determines whether or not the F area is smaller than the D area. If it is determined that the F area is small, the WB value setting circuit 36 proceeds to step S55. Proceed to step S52. In step S52, it is determined whether the F area and the D area are the same number. If it is determined that the number is the same, the process proceeds to step S54. If it is determined that the number is not the same, the process proceeds to step S53.
[0064]
When the process proceeds from step S51 to step S52 and then to step S53, it is determined that the F region is larger than the D region (F region> D region), in other words, it is determined to be a face portion. Since there are more areas determined not to be the face than the area to be processed, in step S53, it is determined not to execute the white balance adjustment process (stop the correction process). The cancellation of the correction processing means that the gain used in the WB amplifier 17 (FIG. 1) is continuously used as it is.
[0065]
On the other hand, if it is determined in step S52 that there are the same number of F regions and D regions, the process proceeds to step S54, and it is determined whether the skin color region is within nine regions. If it is determined that the number of areas determined to be faces and the number of areas determined to be non-faces is the same, when the number of skin color areas is within nine areas, the process proceeds to step S56 and white balance adjustment is performed. If the process is executed and there are nine areas or more, the process proceeds to step S53, and the white balance adjustment process is not executed.
[0066]
When the F area and the D area are the same number, the correction process is basically stopped. However, when the skin color area is within 9 areas, the correction process is executed exceptionally when the skin color area is within 9 areas. If this is the case, the accuracy of judgment will be low, and if the number of skin color areas decreases, the correction amount (the amount of change in white balance gain) will also be small, so the white balance will be adjusted based on a false judgment. This is because it is considered that there is little white balance deviation.
[0067]
When the number of skin color areas is small, the correction amount is small. When calculating gains (Rgain, Bgain) for correcting skin colors, the number of skin color areas (= area) in the screen is calculated according to WB. This is because the gain used in the amplifier 17 (FIG. 1) and the weighted average value of the gain for setting the ideal skin color described above are used as Rgain and Bgain. This is because when the skin color is small, the color balance of the entire screen will not be lost if correction is not made much.
[0068]
If it is determined in step S54 that the skin color area is within 9 areas, the process proceeds to step S56 so as to obtain a white balance gain for obtaining the ideal skin color already calculated by the above-described processing. An instruction is issued from the WB value setting circuit 36 to the WB amplifier 17. When this instruction is issued, white balance adjustment (correction) is performed.
[0069]
On the other hand, if it is determined in step S51 that the F area is smaller than the D area, in other words, the area determined not to be a face is determined to be less than the area determined to be a face. If YES, go to step S55. In step S55, it is determined whether or not the average value of the luminance values of nine regions located in the center of one image (the region shown in FIG. 8 that is hatched in the center) is equal to or less than a predetermined value. Is done.
[0070]
This determination serves as a luminance value limiter process. That is, as described above, the RG signal and the BG signal are used when calculating the white balance gains (Rgain, Bgain) for obtaining an ideal skin color. Since the error of the calculation result becomes large, this is performed to prevent such a situation where the error becomes large. Accordingly, when the white balance gain (Rgain, Bgain) for obtaining the ideal skin color is calculated using the values such as R / G and B / G, the process of step S55 is performed. This is a step that can be omitted.
[0071]
In this case, since the RG signal and the BG signal are used for gain calculation, the process of step S55 is performed. In general, it is easier to calculate RG and BG by subtraction than to calculate the values of R / G and B / G by dividing, and the circuit configuration is simple. In this case, the subtraction process is performed here.
[0072]
In step S55, when it is determined that the average value of the luminance values of the nine central regions is equal to or less than the predetermined value, the process proceeds to step S56, and when correction processing is performed and it is determined that the average value is equal to or greater than the predetermined value, In step S53, the correction process is stopped.
[0073]
In this way, the gain for white balance adjustment performed in the WB amplifier 17 (FIG. 1) is determined.
[0074]
As described above, first, an area that is a skin color is determined, and white balance adjustment is performed so that an image having the skin color area has an ideal skin color. Of these, the human skin color can be an ideal (preferred) color.
[0075]
Also, since the size of the face is determined and whether or not white balance adjustment is executed is determined based on the size, when the proportion of the face in the image is small, the white balance matched to the color of the face Thus, it is possible to prevent such a correction that an unfavorable color tone is obtained as a whole.
[0076]
That is, by applying the present invention, it is possible to realize a preferable skin color without distinguishing between a still image and a moving image while being simple, and also consider a skin color region (area) in the image. As a result, the background image can be processed within a range where there is no problem. In addition, since the correction amount is small when the skin color is small in a wide-angle image, the background color is also reproduced correctly. Thus, preferable images corresponding to various situations can be obtained.
[0077]
In the embodiment described above, a preferable skin color is reproduced. However, other colors such as sky blue and green may be reproduced as preferable colors. For example, the camera including the image processing apparatus 10 is provided with a mode called a sky blue mode, and when the mode is set, the white balance is adjusted so that the sky blue is the most preferable color. May be. Thus, it goes without saying that the present invention can be applied even when colors other than the skin color are reproduced as preferable colors. In addition, when it is set as the object which correct | amends colors other than skin color, it is preferable to set each setting mentioned above as the setting matched with the object color.
[0078]
Note that when the white balance is shifted to improve the appearance of the captured image, for example, when shooting at sunset or shooting in the shade, white can be used to effectively reproduce the situation at that time. When the balance is shifted, the color difference signal changes. For white and skin color, data used for color determination as shown in FIG. 9 is prepared separately and prepared separately. The color may be determined using existing data.
[0079]
In the above-described embodiment, the case where the complementary color CCD 12 is used has been described as an example. However, the color adjustment is performed by the same method regardless of the primary color CCD, three sheets, or a single plate. Is possible.
[0080]
In other words, the series of processes described above may be configured to be executed by hardware in the processes performed by each circuit of the WB adjustment circuit 18 as shown in FIG. You may make it run. When a series of processing is executed by software, various functions can be executed by installing a computer in which the programs that make up the software are installed in dedicated hardware, or by installing various programs. For example, it is installed from a recording medium in a general-purpose personal computer or the like.
[0081]
Before describing the recording medium, a general-purpose personal computer will be briefly described. FIG. 11 is a diagram illustrating an internal configuration example of a general-purpose personal computer. A CPU (Central Processing Unit) 111 of the personal computer executes various processes according to a program stored in a ROM (Read Only Memory) 112. A RAM (Random Access Memory) 113 appropriately stores data and programs necessary for the CPU 111 to execute various processes. The input / output interface 115 is connected to an input unit 116 including a keyboard and a mouse, and outputs a signal input to the input unit 116 to the CPU 111. The input / output interface 115 is also connected to an output unit 117 including a display and a speaker.
[0082]
In addition, a storage unit 118 constituted by a hard disk or the like and a communication unit 119 that exchanges data with other devices via a network such as the Internet are connected to the input / output interface 115. The drive 120 is used when data is read from or written to a recording medium such as the magnetic disk 131, the optical disk 132, the magneto-optical disk 133, and the semiconductor memory 134.
[0083]
As shown in FIG. 11, the recording medium is distributed to provide a program to the user separately from the personal computer, and a magnetic disk 131 (including a flexible disk) on which the program is recorded, an optical disk 132 (CD- Consists of package media including ROM (Compact Disc-Read Only Memory), DVD (including Digital Versatile Disc), magneto-optical disc 133 (including MD (Mini-Disc) (registered trademark)), or semiconductor memory 134 In addition, it is configured by a hard disk including a ROM 112 that stores programs and a storage unit 118 that are provided to the user in a state of being pre-installed in a computer.
[0084]
In this specification, the steps for describing the program provided by the medium are performed in parallel or individually in accordance with the described order, as well as the processing performed in time series, not necessarily in time series. The process to be executed is also included.
[0085]
As described above, according to the image processing apparatus and method, and the program of the present invention, , It becomes possible to easily adjust a predetermined color of the captured image, for example, a skin color to a color preferred by the user.
[Brief description of the drawings]
FIG. 1 is a diagram showing a configuration of an embodiment of an image processing apparatus to which the present invention is applied.
FIG. 2 is a diagram illustrating an internal configuration example of a WB adjustment circuit 18;
FIG. 3 is a flowchart for explaining the operation of the WB adjustment circuit 18;
FIG. 4 is a flowchart for describing color discrimination processing in an area in step S11.
FIG. 5 is a flowchart illustrating correction amount calculation processing in step S12.
FIG. 6 is a flowchart for explaining a face portion discrimination process in step S13;
FIG. 7 is a flowchart illustrating the correction process in step S14.
FIG. 8 is a diagram for explaining data used for color determination;
FIG. 9 is a diagram illustrating a region used for color determination.
FIG. 10 is a diagram illustrating a region used for discrimination of a face portion.
FIG. 11 is a diagram illustrating a medium.
[Explanation of symbols]
DESCRIPTION OF SYMBOLS 10 Image processing apparatus, 11 Lens, 12 CCD, 13 S / H circuit, 14 AGC circuit, 15 A / D conversion circuit, 16 RGB matrix circuit, 17 WB amplifier, 18 WB adjustment circuit, 19 Color difference calculation circuit, 20 Color difference matrix Circuit, 31 detection circuit, 32 WB amplifier, 33 luminance peak value calculation circuit, 34 color difference calculation circuit, 35 WB value calculation circuit, 36 WB value setting circuit

Claims (4)

An imaging means for capturing an image;
A determination unit that divides the image captured by the imaging unit into a plurality of regions, and determines colors in the regions;
Determining means for determining whether or not there is an area determined in advance as a skin color set as a processing target in a predetermined number of areas corresponding to the four corners of the image, based on a determination result by the determining means; ,
A target value calculation unit that calculates a target value for adjusting white balance when it is determined that there is no region determined to be the skin color by the determination unit ;
A difference value calculating means for calculating a difference value between a luminance value of a first area of the plurality of areas and a luminance value of a second area located around the first area;
The number of the first areas where the difference value calculated by the difference value calculating means is greater than or equal to a predetermined value is greater than the number of the first areas less than or equal to the predetermined value, and the predetermined value of the image When the average value of the luminance values of the area located at the position is equal to or less than a predetermined value, or the number of the first areas where the difference value calculated by the difference value calculating means is equal to or greater than a predetermined value, When the skin color area is equal to or less than the predetermined number in the image, the target value calculated by the target value calculating means is used. An image processing apparatus comprising: adjusting means for adjusting white balance . Dividing the captured image into a plurality of regions, determining colors in the regions,
Based on the determination result, it is determined whether or not the predetermined number of areas corresponding to the four corners of the image includes an area that is determined to be a skin color that is set in advance as a processing target;
When it is determined that there is no area determined to be the skin color , a target value for adjusting white balance is calculated ,
Calculating a difference value between a luminance value of a first region of the plurality of regions and a luminance value of a second region located around the first region;
An area located at a predetermined position of the image, wherein the number of the first areas where the calculated difference value is equal to or greater than a predetermined value is greater than the number of the first areas which are equal to or less than the predetermined value. The first area where the average value of the luminance values is less than or equal to a predetermined value, or the number of the first areas where the calculated difference value is greater than or equal to the predetermined value is less than or equal to the predetermined value And adjusting the white balance using the calculated target value when the skin color area is within a predetermined number in the image. Dividing the captured image into a plurality of regions, determining colors in the regions,
Based on the determination result, it is determined whether or not the predetermined number of areas corresponding to the four corners of the image includes an area that is determined to be a skin color that is set in advance as a processing target;
When it is determined that there is no area determined to be the skin color , a target value for adjusting white balance is calculated ,
Calculating a difference value between a luminance value of a first region of the plurality of regions and a luminance value of a second region located around the first region;
An area located at a predetermined position of the image, wherein the number of the first areas where the calculated difference value is equal to or greater than a predetermined value is greater than the number of the first areas which are equal to or less than the predetermined value. The first area where the average value of the luminance values is less than or equal to a predetermined value, or the number of the first areas where the calculated difference value is greater than or equal to the predetermined value is less than or equal to the predetermined value And when the skin color region is within a predetermined number in the image, the computer-readable method includes a step of adjusting white balance using the calculated target value. A recording medium on which the program is recorded. Dividing the captured image into a plurality of regions, determining colors in the regions,
Based on the determination result, it is determined whether or not the predetermined number of areas corresponding to the four corners of the image includes an area that is determined to be a skin color that is set in advance as a processing target;
When it is determined that there is no area determined to be the skin color , a target value for adjusting white balance is calculated ,
Calculating a difference value between a luminance value of a first region of the plurality of regions and a luminance value of a second region located around the first region;
An area located at a predetermined position of the image, wherein the number of the first areas where the calculated difference value is equal to or greater than a predetermined value is greater than the number of the first areas which are equal to or less than the predetermined value. The first area where the average value of the luminance values is less than or equal to a predetermined value, or the number of the first areas where the calculated difference value is greater than or equal to the predetermined value is less than or equal to the predetermined value A program for causing a computer to execute a step of adjusting white balance using the calculated target value when the skin color area is within a predetermined number in the image .

Images