JP4277315B2 - White balance correction apparatus and method - Google Patents

Description

  The present invention relates to a white balance correction apparatus and method, and more particularly to a white balance correction apparatus and method capable of performing appropriate white balance correction on a color image signal.

  When performing white balance correction on the image signal obtained by the color image sensor, paying attention to the high probability that the light source color is low in saturation, this low saturation is detected in the shooting screen. There is known a method in which white balance correction is performed using a correction value depending on the color temperature of the portion (for example, see Patent Document 1).

  By the way, when performing white balance correction, it is necessary to consider a so-called color feria phenomenon in which an object color is erroneously determined as a light source color and is overcorrected. Specifically, when an object color having a color similar to the light source color enters the screen, such an object color is erroneously determined as the light source color and overcorrected.

  Therefore, conventionally, for example, the white balance is not completely corrected in order to reduce the color feria phenomenon, and an appropriate white balance correction such as keeping the correction amount to 50% to 60%, for example, is sacrificed to some extent. Measures were taken to end up.

The inventor detects a distribution on a specific color coordinate for color information (integrated value) of each part of the screen, determines that the portion closest to gray on the color coordinate is the most likely light source, A method for correcting the white balance based on the distance on the color coordinate from the most likely light source has already been proposed (see Patent Document 2).
JP-A-6-98335 JP 2004-274344 A (particularly columns 0074 to 0085 of the specification)

  In a scene where orange and yellow object colors exist, such an object color is misidentified as a tungsten light source color. For example, orange and yellow are corrected to gray, and what was originally gray is overcorrected to be bluish. It will be.

  Also, since spot light is partial light and does not illuminate a wide area, calculating the white balance correction value based on such spot light may result in inappropriate white balance correction. become.

  In particular, in an indoor scene, there is a scene in which the lowest saturation portion (that is, the portion closest to gray) in the screen is not a light source. For example, a blue object color under a yellow light source is reduced in color because the light source color and the object color have a symmetric hue relationship with each other, and actually approaches a dark gray color. Therefore, assuming that the portion closest to gray on the screen is most likely the light source, the white balance is corrected based on the object color, so there may be cases where the optimal white balance correction is not performed. . In addition, even when the combination of the light source and the object color is daylight and yellow, the mercury lamp light source and red, the red light source and green cyan, the subtractive or additive color mixture is applied to the object color or the light source color. It becomes difficult to judge whether there is any.

  The present invention has been made in view of such circumstances, and a white balance correction device capable of preventing color feria phenomenon and performing optimum white balance correction according to the scene, thereby improving image quality, and It aims to provide a method.

In order to achieve the object, the invention according to claim 1 calculates a white balance correction value based on an image signal for each color, and determines the white balance of the image signal based on the white balance correction value. A white balance correction device for correcting, a means for detecting color information of each part obtained by dividing a screen based on an image signal in each part of the screen, and detecting brightness information of each part of the screen Color information removing means for removing color information that seems to be an object color that approximates a predetermined light source color among a plurality of color information in the screen based on brightness information of each part of the screen; and a white balance correction value calculating means for calculating a correction value of the white balance based on the plurality of color information as likely object color that approximates the predetermined light source color has been removed, the image signal Is composed of R, G, and B signals for each color of red, green, and blue, and the color information is a ratio of an integrated value of the B signal and an integrated value of the G signal for each portion of the screen. If the B / G value is equal to or less than a threshold value and the brightness information is equal to or greater than a value determined for each B / G value, the color information removing unit includes the B / G value. It is configured to remove information .

  According to this configuration, even in a scene in which, for example, an orange or yellow object color is easily misidentified as a tungsten light source color, color information that seems to be an object color that approximates such a predetermined light source color is brightness information. Since the white balance correction value is calculated after being removed based on the above, the color feria phenomenon is prevented and the optimum white balance correction according to the scene is performed.

The invention according to claim 2 is a white balance correction method for calculating a white balance correction value based on an image signal for each color and correcting the white balance of the image signal based on the white balance correction value. Detecting the color information of each part obtained by dividing the screen based on the image signal in each part of the screen; detecting the brightness information of each part of the screen; A color information removal step of removing color information that is likely to be an object color that approximates a predetermined light source color from among a plurality of color information based on brightness information of each part of the screen, and approximates the predetermined light source color comprises a step of calculating a correction value of the white balance based on the plurality of color information by what seems object color which has been removed, the image signal is red, green, each color of R signals of blue, G The color information includes a B / G value that is a ratio of the integrated value of the B signal and the integrated value of the G signal for each part of the screen, and the color information removing step When the B / G value is equal to or smaller than a threshold value and the brightness information is equal to or larger than a value determined for each B / G value, the color information is removed .

  According to the present invention, it is possible to prevent a color feria phenomenon and perform an optimal white balance correction according to the scene, thereby improving image quality.

  The best mode for carrying out the present invention will be described below in detail with reference to the accompanying drawings.

  FIG. 1 is a block diagram showing a schematic configuration of a camera 10 as an embodiment of a white balance correction apparatus according to the present invention.

  The camera 10 is a digital camera having a still image and moving image shooting function.

  The overall operation of the camera 10 is centrally controlled by a central processing unit (CPU) 12. The CPU 12 functions as a control unit that controls the camera 10 according to a predetermined program. For example, automatic exposure (AE) control processing, automatic focus adjustment (AF) control processing, and auto white balance (AWB) control processing are performed.

  A ROM 16 connected to the CPU 12 via the bus 14 stores a program executed by the CPU 12 and various fixed data necessary for the operation of the program. The memory (SDRAM) 18 is used as an area for various control processes performed by the CPU 12 and is used as a temporary storage area for image data. The VRAM 20 temporarily stores image data for image display, and has an A area 20A and a B area 20B.

  The camera 10 is provided with a mode selection switch 22 and a shooting button 24. A screen operation key 26 including a menu key, an OK key, a cross key, a cancel key, and the like is provided. Signals from these various operation means (22, 24, 26) are input to the CPU 12, and the CPU 12 controls each part of the camera 10 based on the input signals, and controls lens drive control, imaging control, image signal processing control, and image. Control such as recording control and image reproduction control is performed.

  The mode selection switch 22 is an operation for switching between a photographing mode for photographing a subject and recording image data on a predetermined recording medium 32 and a reproduction mode for reproducing image data recorded on the recording medium 32. Means. When the movable piece 22A is connected to the contact point a, the still image shooting mode is set. When the movable piece 22A is connected to the contact point b, the moving image shooting mode is set. When the movable piece 22A is connected to the contact point c, the reproduction mode is set. Is set. The shooting button 24 is an operation means for inputting a shooting preparation instruction and a shooting start instruction, and is composed of a two-stroke switch having an S1 switch that is turned on when half-pressed and an S2 switch that is turned on when fully pressed. ing. The menu key is an operation means for inputting an instruction to display a menu on the screen of the image display device 28. The OK key is an operation means for inputting an instruction to confirm and execute the selected content. The cross key is an operation means for inputting instructions in four directions, up, down, left, and right for selecting items in the menu. The cancel key is an operation means for inputting a cancellation instruction such as selection contents and instruction contents.

  The image display device 28 is composed of a liquid crystal display (LCD) capable of color display. The image display device 28 can be used as an electronic viewfinder for checking the angle of view at the time of shooting, and is used as a means for reproducing and displaying a recorded image. The image display device 28 is also used as a user interface display screen, and displays information such as menus, selection items, and setting contents as necessary.

  The media controller 34 performs predetermined signal conversion in order to exchange input / output signals suitable for the recording medium 32 mounted in the media socket 30.

  The USB interface unit 36 is a means for connecting a personal computer or other external device to the camera 10. By connecting the camera 10 and an external device using a USB cable (not shown), it is possible to exchange data with the external device. Of course, the communication method is not limited to USB, and IEE1394 or wireless or other communication methods may be applied.

  Next, the shooting function of the camera 10 will be described.

  When the shooting mode is selected by the mode selection switch 22, power is supplied to the shooting-related portion (shooting unit) including the CCD 38, and shooting is ready. As described above, the shooting modes include the still image shooting mode and the moving image shooting mode. However, in order to facilitate the understanding of the present invention, the case where the still image shooting mode is mainly set as the shooting mode will be described below. explain.

  The lens unit 40 is an optical unit that includes a photographic lens 42 including a focus lens and an iris combined mechanical shutter 44 using an iris. The lens unit 40 is electrically driven by a lens driving unit 46 and a diaphragm driving unit 48 controlled by the CPU 12, thereby performing zoom control, focus control, and iris control. The light that has passed through the lens unit 40 is imaged on the light receiving surface of the CCD 38. A large number of photodiodes (light receiving elements) are two-dimensionally arranged on the light receiving surface of the CCD 38, and red (R), green (G), and blue (B) primary color filters corresponding to the respective photodiodes. They are arranged in a predetermined arrangement structure (Bayer, G stripe, etc.). The CCD 38 has a so-called electronic shutter function for controlling the charge accumulation time (shutter speed). The CPU 12 controls the charge accumulation time in the CCD 38 via the timing generator 50. The subject image formed on the light receiving surface of the CCD 38 is converted into a signal charge of an amount corresponding to the amount of incident light by each photodiode. The signal charge accumulated in each photodiode is sequentially read out as a voltage signal (image signal for each color of R, G, B) corresponding to the signal charge based on the drive pulse given from the timing generator 50 according to the instruction of the CPU 12.

  The R, G, B image signals taken out from the CCD 38 are sent to an analog processing unit (CDS / AMP circuit) 52, where each pixel is sampled and held (correlated double sampling processing) and amplified. Thereafter, it is supplied to the A / D converter 54 and converted from analog to digital. The image signal output from the A / D converter 54 is temporarily stored in the memory 18 via the image input controller 56. The image signal processing circuit 58 processes the R, G, and B image signals stored in the memory 18 in accordance with instructions from the CPU 12. Specifically, the image signal processing circuit 58 is a synchronization circuit (a circuit that corrects the spatial deviation of the R, G, and B signals accompanying the color filter array of the CCD 38 and converts the image signal into a simultaneous type. ), Function as image signal processing means including a white balance correction circuit, a gamma correction circuit, a YC signal generation circuit, etc., and perform predetermined image signal processing using the memory 18 in accordance with commands from the CPU 12. Image data obtained by the processing of the image signal processing round 58 is stored in the VRAM 20.

  As for image output to the image display device 28, image data is read from the VRAM 20 and sent to the video encoder 60 via the bus 14. The video encoder 60 converts the input image data into a signal of a predetermined display system (for example, NTSC system) and outputs it to the image display device 28. Specifically, the image signal output from the CCD 38 is alternately rewritten into the A area 20A and the B area 20B of the VRAM 20 as image data for each frame. Of these A area 20A and B area 20B, image data is read from an area other than the area where the image data is rewritten. Then, by periodically rewriting the image data in the VRAM 20, the photographer can check the shooting angle of view by the video (through movie image) displayed on the image display device 28.

  When the shooting button 24 is half-pressed by the photographer who has confirmed the shooting angle of view and S1 is turned on, the CPU 12 starts AE and AF processing as shooting preparation processing. The R, G, and B image signals output from the CCD 38 are temporarily stored in the memory 18 via the CDS / AMP circuit 52, the A / D converter 54, and the image input controller 56, and also the AF detection circuit. 62 and the AE / AWB detection circuit 64.

  The AE / AWB detection circuit 64 includes a circuit that divides a screen into a plurality of areas (for example, 16 × 16, 8 × 8, etc.) and integrates image signals for each of the R, G, and B colors for each divided area. The integrated value is provided to the CPU 12. The CPU 12 detects the subject brightness based on the integrated value acquired from the AE / AWB detection circuit 64, and calculates an EV value corresponding to the subject brightness. This EV value is arranged in a memory 18 so that it can be referred to in the white balance correction process described later, such as the EV value for the entire screen and the EV value for each divided area on the screen. The CPU 12 determines an aperture value and a shutter speed according to a predetermined program diagram, and controls the electronic shutter of the CCD 38 and the iris 44 of the lens unit 40 according to these to obtain an appropriate exposure amount.

  The AF control in the camera 10 is, for example, a contrast AF that moves a focusing lens (a moving lens that contributes to focus adjustment among the lens optical systems constituting the photographing lens 42) so that the high-frequency component of the G signal of the video signal is maximized. Applies. That is, the AF detection circuit 62 cuts out a signal in a focus-to-image area preset in a high-pass filter that passes only a high-frequency component of the G signal, an absolute value processing unit, and a screen (for example, the center of the screen). An area extraction unit and an integration unit that integrates absolute value data in the AF area are configured. The integrated value data obtained by the AF detection circuit 62 is notified to the CPU 12. The CPU 12 calculates a focus evaluation value (AF evaluation value) at a plurality of AF detection points while moving the focusing lens by controlling the lens driving unit 46, and determines a lens position where the evaluation value is a maximum as a focus position. To do. Then, the lens driving unit 46 is controlled so as to move the focusing lens to the obtained in-focus position. The calculation of the AF evaluation value is not limited to a mode using the G signal, and a luminance signal (Y signal) may be used.

  After shooting preparation processing such as AE processing and AF processing is performed by half-pressing the shooting button 24 (S1 on), when the shooting button 24 is fully pressed (S2 on), a shooting operation for recording an image is performed. Start.

  The CPU 12 acquires an integrated value for each R, G, B signal for each divided area from the AE / AWB detection circuit 64. The integrated value for each divided area notified from the AE / AWB detection circuit 64 to the CPU 12 includes a mode in which values are simply integrated for each of the R, G, and B signals, and a mode in which average values are used. Then, the CPU 12 calculates a white balance correction value based on the integrated value for each R, G, B signal obtained from the AE / AWB detection circuit 64 for each divided area, and the image signal processing circuit 58 uses the R, G, White balance processing is applied to the B signal. This white balance correction process will be described in detail later.

  The image signals for each of the R, G, and B colors that have been subjected to white balance correction by the image processing circuit 58 are further subjected to image processing such as gamma correction by the image processing circuit 58 to obtain YC signals (luminance signal Y and color difference signal C). ) And compressed in a predetermined format by the compression / decompression circuit 66, and then recorded as image data on the recording medium 32 via the media controller 34 and the media socket 30.

  When the playback mode is selected by the mode selection switch 22, the compressed image data recorded on the recording medium 32 is read out via the media controller 34. The read image data is decompressed by the compression / decompression circuit 66 and displayed on the image display device 28 via the VRAM 20 and the video controller 60. The user can reproduce and enjoy an image with appropriate white balance correction.

  Next, an outline of a basic white balance correction function will be described with reference to FIG. FIG. 2 is a block diagram showing a main part related to the white balance correction function of the camera 10. 2, the same parts as those in FIG. 1 are denoted by the same reference numerals, and the description thereof is omitted.

  A digital signal processing circuit 100 in FIG. 2 is a part of the image signal processing circuit 58 in FIG. 1 and includes a synchronization circuit 102, a white balance correction circuit 104, a gamma correction circuit 106, and a YC signal generation circuit 108. In FIG. 2, a memory 110 is provided in the digital signal processing circuit 100 in order to facilitate understanding of the signal flow, but this memory 110 is actually included in another memory 18 in the figure. .

  The R, G, B signals (image signals) output from the CCD 38 are temporarily stored in the memory 18 via the CDS / AMP circuit 52, the A / D converter 54, and the image input controller 56. The synchronization processing circuit 102 converts the dot-sequential R, G, B signals read from the memory 18 into simultaneous equations, and outputs the R, G, B signals to the white balance correction circuit 104 simultaneously. The white balance correction circuit 104 includes multipliers 104R, 104G, and 104B for increasing and decreasing the digital values of the R, G, and B signals. The R, G, and B signals are respectively multiplied by the multipliers 104R, 104G, and 104B, respectively. Added to 104B.

  The multipliers 104R, 104G, and 104B receive R, G, and B signals, respectively, and the CPU 12 receives gain values Rg, Gg, and Bg (white balance correction values). 104R, 104G, and 104B multiply R, G, and B signals by gain values Rg, Gg, and Bg, respectively, and R, G, and B signals (R ′, G ′, and B ′) that are white balance corrected by this multiplication. Is output to the gamma correction circuit 106. The CPU 12 calculates the gain values Rg, Gg, and Bg.

  The R, G, B signals (image signals) generated by the CCD 38 are input to the integrating circuit 112 via the CDS / AMP 52, the A / D converter 54, and the image input controller 56. Is accumulated for each R, G, B signal. This integration circuit 112 is included in the AE / AWB detection circuit 64 of FIG. Multipliers 112R, 112G, and 112B are provided between the integration circuit 112 and the CPU 12, and an adjustment gain is added to the multipliers 112R, 112G, and 112B in order to adjust device variations. . The CPU 12 acquires an integrated value for each divided area from the integrating circuit 112, and calculates a white balance correction value based on the acquired integrated value.

  Assuming that the image signals before white balance correction are R, G, B and the gain values are Rg, Gg, Bg, the image signals R ′, G ′, B ′ after white balance correction are expressed by (Equation 1).

(Formula 1)
R ′ = Rg × R
G ′ = Gg × G
B ′ = Bg × B
The gamma correction circuit 106 changes the input / output characteristics so that the white balance corrected R, G, B signals (image signals composed of R ′, G ′, B ′ in Equation 1) have the desired gamma characteristics, This is output to the YC signal generation circuit 108. The YC signal generation circuit 108 generates a luminance signal Y and color difference signals Cr and Cb from the gamma-corrected R, G, and B signals. The luminance signal Y and the color difference signals Cr and Cb are stored in the memory 110. An image can be displayed by reading the YC signal in the memory 110 and outputting it to the image display device 28 via the VRAM 20 and the video encoder 60 of FIG.

  Hereinafter, the flow of the auto white balance (AWB) control process will be described in detail. Each step of the auto white balance control process is executed by the CPU 12 in accordance with a program stored in advance in the ROM 16 of FIG.

  FIG. 3 shows the flow of color information detection processing, brightness information detection processing, catamari calculation processing, and object color likelihood calculation processing in the auto white balance control processing.

  In FIG. 3, the auto white balance (AWB) control by the CPU 12 is started in response to the full depression (S2 on) of the shooting button 24 (step S100).

  Note that R (red), G (green), and B (blue) image signals (R, G, and B signals) generated by the CCD 38 are temporarily stored in the memory 18 as digital data for each shooting screen. The One shooting screen is divided into a plurality of areas (for example, 8 × 8, 16 × 16, etc.), and R, G, B is added to each divided area (divided area) by the integrating circuit 112 in FIG. Since the integrated value for each color of the signal (referred to as “RGB integrated value”) is calculated, the RGB integrated value of each divided area is acquired from the integrating circuit 112 (step S102), and the R signal is obtained for each divided area. The ratio R / G of the integrated value of G and the integrated value of the G signal and the ratio B / G of the integrated value of the B signal and the integrated value of the G signal are calculated (step S104). In other words, the RGB integrated value is converted from (R, G, B) coordinates to (R / G, B / G) coordinates for each divided area. The RGB integrated values converted into such (R / G, B / G) coordinates are referred to as “R / G, B / G values”. Information indicating the color of each part of the shooting screen such as the R / G and B / G values is also referred to as “color information” of each part.

  Further, brightness information for each divided area is calculated based on the image signals (R, G, B signals) for each color (step S106). Here, as the brightness information, in the present embodiment, an “EV value” indicating luminance is calculated.

  Next, color information (R / G, B / G values) that approximate each other on (R / G, B / G) coordinates are grouped together, and color information (coordinates) that represents each group is calculated. To do. A group in which color information that approximates each other on specific color coordinates is collected is called “catamari”, and color information (coordinates) that represents each group (catamari) is called “catamari coordinates”. To do. In other words, “catamari” is detected, and “catamari coordinates” are calculated (step S108).

  Here, there are various modes of color information grouping (in other words, division area grouping based on color information).

  First, as shown in FIG. 8, a plurality of detection frames are provided in advance on (R / G, B / G) coordinates 300, and color information (R / G, B / G values) of each divided area. There is a mode in which a detection frame in which is inserted is determined, and color information (R / G, B / G values) that are in the same detection frame are detected as a single catamaran. That is, there is an aspect in which divided areas whose colors are approximated on the color coordinate are detected as a single catamari based on the color information. Then, the catamary coordinates obtained by averaging the color information (R / G, B / G values) of the divided areas included in each detection frame are obtained.

  Second, paying attention to the color information (R / G, B / G value) of each divided area sequentially on the (R / G, B / G) coordinates, as shown in FIG. 9 (a) or (b). In addition, there is a mode in which color information that falls within a circle with a predetermined radius centering on the noticed color information is detected as one catalyzer. Then, for example, the coordinates of the center of the circle for each catamari are set as catamari coordinates. In this second grouping mode, it is possible to detect the catamarism without providing a detection frame as in the first grouping mode described above.

  In addition, since the present inventor has already filed a patent application (Japanese Patent Application No. 2003-378092) for the more preferable second grouping mode, the details thereof relate to the patent application as necessary. Refer to the description, drawings, etc. Here, the outline of the configuration of the more preferable aspect will be described. The color information of each divided area is paid attention sequentially, and the color information of the noticed divided area and other divisions that approximate the color information mutually. A group consisting of area color information is sequentially formed, and the first step for obtaining representative color information representative of each formed group is executed, and the representative color information of each group is sequentially noticed. A second group that sequentially forms a new group of representative color information and representative color information of another group that approximates the representative color information, and obtains representative color information that represents each newly formed group. Steps are executed, and the second step is repeated until the value of the representative color information converges.

  A comparison of the effects of the first grouping mode and the second grouping mode will be briefly shown. FIG. 9A shows a case where color information is concentrated mainly in one catamari in the color coordinate system composed of the R / G coordinate axis and the B / G coordinate axis, and FIG. In the color specification coordinates composed of the coordinate axes and the B / G coordinate axes, a case is shown in which color information is scattered in a plurality of catamaries. In any case, there is a catamaria that straddles a plurality of conventional detection frames, and such a conventional detection frame as shown in FIG. 9A or FIG. 9B is straddled. In the case of the distribution state, according to the second grouping aspect, the distribution state of the color information can be accurately grasped as compared with the first grouping aspect described with reference to FIG. There is an advantage that appropriate white balance correction can be performed. On the other hand, the first grouping mode described with reference to FIG. 8 has an advantage that the detection process of the distribution state of the color information is simplified in a system in which the number of detection frames is small.

  After grouping (step S108), the average value of the EV values (catalymic EV values) is calculated for each catamari (step S110). Here, it can be said that the catamari EV value is brightness information of each catamari.

  Next, object colors having orange and yellow (RY) hues that are close to the tungsten light source color, which is a low color temperature light source, are excluded from a population consisting of a plurality of catamarry coordinates based on the catamary EV value. .

  For example, when correcting the white balance with gray as an example, if an object color of orange or yellow hue is misidentified as a tungsten light source color, the orange and yellow will approach gray. In addition to being corrected, what was originally gray is corrected to bluish. The inventor of the present invention paid attention to the fact that the lower the color temperature, the lower the illuminance in general if the light source is used. In the present embodiment, even if the color is orange or yellow, if the brightness (luminance) is equal to or higher than a predetermined value determined for each hue, it is determined that the possibility of an object color is extremely high, and such color information is displayed as white. Exclude from the balance correction value calculation population.

  Specifically, first, based on the B / G value and the catamari EV value (which is the brightness information of each catamari) for each catamari, a membership function of the light source color is used and each catamaran coordinate (each catamari The color information of the light source is detected (step S112).

  FIG. 10 is an explanatory diagram showing an example of the membership function of the light source color likelihood, where the horizontal axis is the catalyma EV value, the vertical axis is the B / G value of each catamari, and the hatched portion is likely to be an object color It shows that.

The catamary coordinate of the i-th catamaly is (Ri / Gi, Bi / Gi), its catamary EV value is EVi, the lower limit threshold for the catamary EV value is X1, its B / G value is Y1, and the upper limit threshold the X2, is the B / G value and Y 2.

  Then, when EVi <X1, if Bi / Gi ≦ Y1, the membership function outputs “0” as the evaluation value of the light source color.

  When EVi> X2, if Bi / Gi ≦ Y2, the membership function outputs “0” as the evaluation value of the light source color.

  When X1 ≦ EVi ≦ X2, if Bi / Gi ≦ (Y2-Y1) / (X2-X1) × EVi− (X1 × Y2-X2 × Y1) / (X2-X1), the evaluation value of the light source color is evaluated. As a result, the membership function outputs “0”.

  Then, the catamary coordinates evaluated as having a light source color likelihood of “0”, that is, the catamary coordinates of the object color whose color approximates to the tungsten-based light source color, should not be incorporated in the calculation of the white balance correction value. Then, it is removed from the population of the catamari coordinates (steps S114 and S116). Then, it is determined whether there is any other catamamy (step S118), and steps S112 to S116 are repeated for all the catamaries.

  Here, the catamary coordinates that are likely to be orange and yellow object colors are excluded from the set of catamary coordinates (catalyst color information) used to determine the light source and calculate the white balance correction value. In the correction, white balance correction is performed with reference to a light source color.

  Note that if the light source color likelihood and the object color likelihood determination are performed using only brightness information, color information that should originally be incorporated in white balance correction other than color information that approximates the tungsten light source color may be removed. . Therefore, in the R / G and B / G coordinate systems, focusing on the B / G value, it is noted that determination to the extent that it approximates to the tungsten light source system is easy and accurate. Are used that output evaluation values of the light source color likelihood and the object color likelihood based on both the B / G value and the EV value parameter.

  FIG. 4 shows the flow of the spot lightness calculation process in the auto white balance control process. In this process, color information that seems to be spotlight is excluded from the white balance correction value calculation set.

  If the white balance is corrected based on the spot light, an appropriate color image may not be obtained. That is, spot light is so-called partial light and does not irradiate light over a wide area, so the white balance is corrected in a wide area where spot light is not irradiated based on the spot light. It is caused by that. When the white balance is corrected based on the spot light, an effect of remaining color is caused if the spot light is not dominant in the screen. The remaining color tone is positively taken as a high-class atmosphere in a strong light source such as tungsten, such as tungsten, but generally appears to be a fatal side effect under a light source such as a fluorescent lamp. Therefore, in this embodiment, light source selection based on area occupancy is introduced so as to prevent side effects caused by spot light, particularly under a white light source.

  In FIG. 4, first, the catamary coordinates (Ri / Gi, Bi / Gi) and the catamary EV value (EVi) of each catamaly are read from the memory 18 (step S120).

  Next, an average value (average EV value) of the EV values of the entire shooting screen is calculated (step S122).

  Next, the area (catamari area) of each catamari on the photographing screen is calculated (step S124). Here, in the present embodiment, the number of divided areas belonging to each catamari (that is, the number of R / G and B / G values belonging to each catamari) is detected as the catamari area.

  Next, the maximum value (maximum catalog area) among the plurality of catalog areas is detected (step S126). Here, in the present embodiment, as the maximum catamary area, the number of the divided areas of the catamaries to which the number of the divided areas to which they belong is the maximum (that is, the number of R / G and B / G values belonging to the largest catamaries). ) Is detected.

  Then, it is determined whether or not each spotlight is likely to be spot light, and processing that removes the spot light that is likely to be spotlight from the population for white balance correction value calculation is performed (steps S128 to S138).

  Specifically, first, as shown in (Equation 2), the difference between the catamari EV value (Evi) indicating the brightness of each catamari and the average EV value (EVav0) indicating the brightness of the entire screen is calculated. (Step S128), it is determined whether or not the difference is larger than a predetermined threshold value (SPT_EVTH) (Step S130).

(Formula 2)
EVi-EVav0> SPT_EVTH
That is, it is determined whether or not the image is bright with a difference equal to or greater than a predetermined threshold compared with the brightness of the entire screen.

  Further, as shown in (Equation 3), the difference between the number (m1) of the divided areas of the maximum catamari indicating the area of the maximum catamari and the number (mi) of the divided areas for each catamari indicating the area of each catalyzer. Is calculated (step S132), and it is determined whether the difference is larger than a predetermined threshold (SPT_MTH) (step S134).

(Formula 3)
m1-mi> SPT_MTH
That is, it is determined whether or not the area has a small area with a difference equal to or greater than a predetermined threshold value compared to the maximum area in the shooting screen.

  Further, as shown in (Equation 4), it is determined whether or not the catamari EV value (EVi) indicating the brightness of each catamari is larger than a predetermined threshold (EVTH2) (step S136).

(Formula 4)
EVi> SPT_EVTH2
That is, it is determined whether or not the spot has a certain brightness specific to the spot light.

  It is determined that the catamari satisfying all of the above (Equation 2), (Equation 3), and (Equation 4) is likely to be spot light, so that it is not incorporated into the calculation of the white balance correction value. The catamari coordinates are removed from the white balance correction value calculation population (step S138).

  Then, it is determined whether there is any other catalyzer (step S140), and the process of excluding color information that is likely to be spotlight for all catamaries (steps S128 to S138) is repeated.

  In other words, the color of the catamari that is sufficiently brighter than the average brightness of the entire screen, the area occupied by the entire screen is sufficiently small compared to other color parts, and has a certain level of brightness that is characteristic of a spot light source. Information (catamari coordinates) is excluded from the mother set because it seems to be spot light.

  It should be noted that such spot light source exclusion is particularly limited to a specific color temperature region that easily affects image quality. Therefore, a specific light source detection frame (such as a fluorescent light source frame) is preliminarily used as a spot light removal frame so that the spot light is removed only when it enters the fluorescent light or sunny detection frame on the color coordinates. It is more preferable to specify and configure such that a spot-like light is removed only when entering such a removal frame.

  Further, appropriate values for the threshold values SPT_EVTH, SPT_MTH, and SPT_EVTH2 are set in the ROM 16 in advance.

  In the brightness-dependent object color likelihood calculation process described with reference to the flowchart of FIG. 3, color information that is likely to be misidentified as a predetermined light source color, such as a tungsten light source or a white fluorescent lamp, is calculated for white balance correction value calculation. In the spot light likelihood calculation process described with reference to the flowchart of FIG. 4, the color information that seems to be a dominant spot light source in the screen is removed from the population for calculating the white balance correction value. In the saturation reference definition process described below using the flowchart of FIG. 5, what is conventionally defined as the reference light source with the closest reference to gray on the color coordinate is used as the reference light source. Each time the target value on the color coordinate is changed, control is performed to define the saturation reference based on the distance from the target value on the color coordinate.

  Here, in order to facilitate understanding of the present invention, an outline of calculation of a white balance correction value (gain value) will be described.

  In order to calculate the white balance correction value, it is necessary to calculate the color evaluation value HCLi, which is a weighting coefficient used for calculating the white balance correction value. This color evaluation value HCLi is calculated from the saturation evaluation value HSi and brightness elements. That is, the white balance correction value is expressed by (Formula 5).

(Formula 5)
F (color evaluation value HCLi) = F (saturation evaluation value HSi) × F (element of brightness)
Here, the element of brightness is determined by the EV value calculated based on the photographing exposure value and the brightness of the subject. The saturation evaluation value HSi is calculated based on the distance from the coordinates of the low saturation light source in the color coordinate space (saturation distance from the reference light source) in principle.

  According to (Equation 5), the saturation evaluation value HSi is calculated, the color evaluation value HCLi is calculated based on the saturation evaluation value HSi and the brightness element, and the white balance correction value ( Details of the process of calculating the (gain value) will be described below.

  First, the catamary coordinates (Ri / Gi, Bi / Gi) as the color information of each catamari obtained by the catamaran calculation process described with reference to FIG. 3 are read from the memory 18 (step S142).

  Next, an EV value (average EV value) indicating the overall brightness (luminance) of the shooting screen in the shooting scene is calculated (step S144). An EV value indicating the luminance when the S1 switch is on may be read.

  Next, it is determined whether or not the target coordinate setting of the saturation distance is on (step S146). That is, it is determined whether the gray coordinate (1, 1) is used as the reference point on the color coordinate or whether the coordinate fluctuating from the gray coordinate (1, 1) according to the brightness is used.

  If it is determined that the target coordinate setting is ON / OFF, the gray coordinate (1, 1) is set as the target coordinate (step S148). That is, by turning off the target coordinate setting, the gray coordinates (1, 1) can be fixed to the target coordinates as in the conventional case.

  On the other hand, when the target coordinate setting is ON, a process of setting the coordinate that varies from the gray coordinate according to the brightness as the target coordinate is executed (steps S150 to S156).

  Specifically, first, the EV threshold value (lower limit value E_CBT_EV1 and upper limit value E_CBT_EV2) and target coordinates (E_CBT_R, E_CBT_B) at the time of low luminance are read from the ROM 16 in which these are set in advance (step S152). Here, the EV threshold value indicates a range of brightness (average EV value) in which the target coordinates can be varied from the gray coordinates. Further, the target coordinates (E_CBT_R, E_CBT_B) at the time of low luminance are target coordinates set when the brightness (average EV value) is smaller than the lower limit value E_CBT_EV1 of the EV threshold value.

  Next, the target coordinates are obtained using a membership function that outputs the target coordinates (CBT_R, CBT_B) that vary according to the brightness (step S152). An example of the membership function is shown in FIG. Here, if the average EV value (brightness information) is within the range indicated by the EV threshold values (lower limit value E_CBT_EV1 and upper limit value E_CBT_EV2), as shown in FIG. 11, according to the average EV value (brightness information). Fluctuating target coordinates (CBT_R, CBT_B) are output. When the average EV value (brightness information) is smaller than the lower limit EV threshold value E_CBT_EV1, target setting coordinates (E_CBT_R, E_CBT_B) at the time of low luminance are output, and the average EV value (brightness information) is the upper limit EV threshold value E_CBT_EV2 When it is larger, the gray coordinate (1, 1) is output.

  However, it is determined whether or not the target coordinate is within the designated detection frame that allows the target coordinate change on the color coordinate (R / G, B / G) (step S154). The output of the membership function is invalidated and gray coordinates (1, 1) are set as the target coordinates (step S148).

  When the target coordinate setting is on and the target coordinate is within the specified detection frame, the coordinates (CBT_R, CBT_B) that vary from the gray coordinates according to the brightness are set as the target coordinates of the saturation distance. (Step S156).

  Next, the saturation distance ΔS from the target coordinate to each catalyzed coordinate is calculated on the color coordinate (R / G, B / G) (step S166).

  When the gray coordinate (1, 1) is set as the target coordinate, the saturation distance ΔS of each catalyzed coordinate (Ri / Gi, Bi / Gi) is obtained by (Formula 6).

(Formula 6)
ΔSi = {(1−Ri / Gi) ² + (1−Bi / Gi) ² } ^1/2
When coordinates (CBT_R, CBT_B) that change from gray according to the brightness are set as the target coordinates, the saturation distance ΔS of each catamari coordinate (Ri / Gi, Bi / Gi) is obtained by (Equation 7).

(Formula 7)
ΔSi = {(CBT_R−Ri / Gi) ² + (CBT_B−Bi / Gi) ² } ^1/2
It is determined whether or not there is a next catamaran (step S164), the saturation distance ΔS is calculated for all the catamaria (step S162), and the catamaran with the smallest saturation distance ΔS is detected (step S166). A catamari having the smallest degree distance ΔSi is defined as a reference light source (step S168). That is, the catamary coordinates (color information representing the catamaries) closest to the target coordinates on the color coordinates (R / G, B / G coordinates) are defined as the coordinates of the reference light source.

  FIG. 6 is a flowchart showing the flow of a process (white balance correction value calculation process) for calculating a white balance correction value based on the reference light source determined in the saturation reference definition process.

  First, on the color coordinate, a saturation distance ΔDi from the reference light source to each other catamari is obtained (step S170).

  Assuming that the representative coordinates (reference light source coordinates) of the catamari defined as the reference light source are (Xn, Yn) and the other catamari coordinates are (Xi, Yi), the saturation distance ΔDi from the reference light source is obtained by (Equation 8). It is done.

(Formula 8)
ΔDi = {(Xn−Xi) ² + (Yn−Yi) ² } ^1/2
Next, a saturation evaluation value HSi is obtained using a membership function for the saturation evaluation value as shown in FIG. 12 (step S172).

  The membership function for the saturation evaluation value shown in FIG. 12 increases the saturation evaluation value assuming that the light source color is dominant in the portion on the shooting screen that is close to the reference light source. In the portion on the screen, the saturation evaluation value is reduced because the object color is dominant.

  That is, when viewed on the R / G axis and the B / G axis, it is as shown in FIG. In FIG. 13, if the light source color is dominant when it is inside the first neighboring circle 400, the saturation evaluation value is large, and if it is outside the second neighboring circle 402, it is necessary to move away from this circle. The saturation evaluation value is lowered because the object color is dominant as the distance increases.

  After obtaining the saturation evaluation value HSi as described above, the brightness evaluation value is calculated (step S174), and the saturation evaluation value HSi and the brightness evaluation value (brightness) are calculated as in (Formula 5) described above. The color evaluation value HCLi, which is a weighting coefficient of the white balance correction value, is calculated based on the element (step S176).

  Next, a white balance correction value (gain value) is calculated based on the calculated color evaluation value HCLi (step S178).

  Specifically, R / G and B / G gains Gr ′ and Gb ′ are calculated as shown in (Equation 9) so as to bring the respective catamari coordinates to target values.

(Formula 9)
Gr ′ = Σ (Gri × HCLi) / ΣHCLi
Gb ′ = Σ (Gbi × HCLi) / ΣHCLi
The R / G and B / G gain values Gr ′ and Gb ′ obtained as described above are converted into gain values Rg, Gg, and Bg for each of R, G, and B, and then are used for white balance correction shown in FIG. R, G, B separate multipliers 140R, 140G, 140B are set (step S180).

  When the gain value as the white balance correction value is set in this way, the white balance of the image signal for each color of R, G, B is corrected.

  Especially in the room, the lower the color temperature, the more yellow and red will be added to the yellow, and even the gray one will become yellow-red. For human visual recognition, adaptation is intended to recognize that gray is originally gray, but because it is incomplete adaptation, the original gray color will remain to some extent as a reddish yellow color. Focusing on this point, in this embodiment as well, the target value serving as a reference for saturation is set to yellowish red depending on the scene. Since the light source has the property that the illuminance becomes lower as the color temperature becomes lower, it is possible to set the target value serving as the color saturation reference to be yellowish red as the luminance is lower.

  As a result, even if there is an object that has a complementary hue under a yellowish red light source, the target value is set to yellow-red in advance. Can be applied.

  Conversely, even in the high color temperature region, by changing the target value according to the brightness, the color feria phenomenon is prevented and the optimum white balance correction is performed.

  FIG. 7 shows a functional block diagram of the white balance correction apparatus that performs the auto white balance control process as described above.

  In FIG. 7, the white balance correction device 10 mainly includes an image signal acquisition unit 56a, a color information detection unit 12a, a brightness information detection unit 12b, a catamari calculation unit 12c, an object color likelihood calculation unit 12d, a spot Luminance calculation unit 12e, target value variation unit 12f, saturation reference calculation unit 12h including saturation distance calculation unit 12g from target value, light source determination unit 12i, saturation distance calculation unit 12j from light source, A white balance correction value calculation unit 12p including a saturation evaluation value calculation unit 12k, a brightness evaluation value calculation unit 12m, and a weighting calculation unit 12n, a white balance correction unit 100a, an image signal output unit 100b, and a memory 18 Consists of including.

  The image signal acquisition unit 56a acquires an image signal for each color.

  The color information detection unit 12a detects color information for each divided area of the shooting screen based on an image signal in each divided area.

  The brightness information calculation unit 12b detects brightness information of the entire shooting screen and brightness information for each divided area of the shooting screen based on the image signal.

  The catamari calculation unit 12c detects the color information catamari on the color coordinates, color information representing each catamari (catamari coordinates), brightness information representing each catamari (catamari EV value), and each catamari. The area on the photographing screen (catamari area) is detected.

  Note that the catamar is a set of divided areas whose colors are similar to each other, and thus is a portion of the shooting screen, the catamary coordinates are color information of each portion, and the catamary EV value is brightness information of each portion. Yes, the catamari area is the area of each part.

  Here, the catamari is a set of divided areas that are approximated to be a lump with coordinates (colors) on the color coordinate, and the divided areas that are always approximated to a position on the shooting screen are indicated as a lump. In some cases, a plurality of divided areas having different positions on the shooting screen form a single catamaly.

  The object color likelihood calculation unit 12d detects the object color likelihood and the light source color likelihood of the color information of each part of the shooting screen based on the color information and brightness information of each part of the shooting screen, and a plurality of colors in the shooting screen. Color information that seems to be an object color that approximates a predetermined light source color (for example, a tungsten light source color) (that is, color information that is determined not to be a light source color based on brightness information) is removed from the information.

  The spot light likelihood calculating unit 12e detects the spot light likelihood of the color information of each part of the shooting screen based on the image signal and color information in each part of the shooting screen, and the spot light from the plurality of color information in the shooting screen. This is to remove apparent color information. Specifically, on the basis of the catamari EV value and the catamari area, the area occupancy rate in the screen, the brightness difference (brightness relative value) with other catamaris, and the absolute brightness value are determined. Detects and removes catamari coordinates that appear to be spot light.

  The target value changing unit 12f changes the target value (target coordinates) of color information serving as a reference for light source determination from gray according to the brightness information of the entire photographing screen or a predetermined portion.

  The saturation distance calculation unit 12g from the target value calculates a distance (saturation distance) from the target value (target coordinates) on the color coordinates.

  The saturation reference definition unit 12h determines the reference light source based on the saturation distance from the target value (target coordinates).

  The light source discriminating unit 12i discriminates the light source part based on the target value (target coordinates) and the color information of each part of the photographing screen.

  The saturation distance calculation unit 12j from the reference light source calculates a distance (saturation distance) from the coordinates of the reference light source on the color coordinate.

  The saturation evaluation value calculation unit 12k calculates a saturation evaluation value based on the distance (saturation distance) from the coordinates of the reference light source.

  The brightness evaluation value calculation unit 12m calculates a brightness evaluation value.

  The weighting calculation unit 12n calculates a color evaluation value that is a weighting coefficient of the white balance correction value based on the saturation evaluation value and the brightness evaluation value.

  The white balance correction value calculation unit 12p calculates a white balance correction value given as a gain to the image signal so as to correct the white balance of the image signal. Specifically, the white balance correction value is calculated by weighting the color evaluation value calculated based on the saturation evaluation value and the brightness evaluation value.

  The white balance correction unit 100a corrects the white balance by performing a calculation using a white balance correction value on the image signal.

  The image signal output unit 100b outputs an image signal with the white balance corrected.

  The relationship between the constituent elements of the white balance correction apparatus 10 having the functional configuration shown in FIG. 7 and the main constituent elements of the white balance correction apparatus 10 having the specific configuration shown in FIG. The image signal acquisition unit 56a is configured by the image input controller 57 of FIG. 2, and includes a color information calculation unit 12a, a brightness information calculation unit 12b, a catamari calculation unit 12c, an object color likelihood calculation unit 12d, a spot light likelihood calculation unit 12e, The saturation reference definition unit 12h and the white balance correction value calculation unit 12p are configured by the CPU 12 of FIG. 2, and the white balance correction unit 100a and the image signal output unit 100b are configured by the digital signal processing circuit 100 of FIG. It is configured.

  In addition, the aspect which detects the subset which is the subset of the imaging | photography screen with the detection frame demonstrated using FIG. 8, and the catalogue which is the subset of the imaging | photography screen is detected without using the detection frame demonstrated using FIG. An aspect may be implemented as a separate aspect, or two aspects may be combined.

  In addition, the present invention is not limited to the case where catamari detection is performed, and it is possible to obtain white information using color information and brightness information for each divided area without calculating color information and brightness information for each catalog. The present invention can also be applied when performing balance correction.

  Further, the case where the color image signal is an image signal of each color of R (red), G (green), and B (blue) has been described as an example, but other color image signals (for example, Y (yellow) ), M (magenta), and C (cyan) image signals), the present invention may be applied to white balance correction.

  Further, although the case where the coordinates (color coordinates) representing the color information are (R / G, B / G) coordinates has been described as an example, the present invention is such that the color coordinates are such (R / G, (B / G) coordinates are not particularly limited, and other color coordinates that can determine saturation and brightness may be used. In that case, the same color coordinate as the color coordinate of the acquired image signal may be used as the color coordinate. For example, (R, G, B) coordinates may be used as they are.

  In addition, although the case where the white balance correction device is a digital camera has been described as an example, the present invention is not particularly limited to a digital camera, and color image signals such as an image display device, a video camera, a camera-equipped mobile phone, etc. Needless to say, the present invention may be applied to any white balance correction apparatus that performs white balance correction.

  In addition, this invention is not limited to the example demonstrated in embodiment, Of course, in the range which does not deviate from the summary of this invention, various design changes and improvements may be performed.

1 is a block diagram showing an example of a digital camera as an embodiment of a white balance correction apparatus according to the present invention. Block diagram showing the main parts of the white balance correction device 1st flowchart which shows the flow of one Embodiment of the white balance correction method which concerns on this invention. 2nd flowchart which shows the flow of one Embodiment of the white balance correction method which concerns on this invention. Third flowchart showing the flow of one embodiment of the white balance correction method according to the present invention. A fourth flowchart showing a flow of an embodiment of a white balance correction method according to the present invention. The block diagram which shows the functional structure of one Embodiment of the white balance correction apparatus which concerns on this invention. Explanatory drawing which shows the example of the detection frame provided on the color coordinate Explanatory drawing which shows the example of the catamari distribution on the color coordinate Explanatory drawing which shows the example of the function which outputs the evaluation value of object color likeness using brightness information as a parameter Explanatory drawing which shows the example of the function which outputs the target value which fluctuates using brightness information as a parameter Explanatory drawing showing an example of a function that outputs a saturation evaluation value Explanatory drawing used for explanation of processing for performing light source discrimination based on the distance from the reference light source on the color coordinate

Explanation of symbols

  DESCRIPTION OF SYMBOLS 10 ... White balance correction apparatus (camera), 12 ... CPU, 12a ... Color information detection part, 12b ... Brightness information detection part, 12c ... Catamari calculation part, 12d ... Object color-likeness calculation part, 12e ... Spot light-likeness calculation part , 12f: target value changing unit, 12g: saturation distance calculation unit from target value, 12h ... saturation reference definition unit, 12i ... light source determination unit, 12j ... saturation distance calculation unit from light source, 12k ... saturation evaluation Value calculation unit, 12m ... Brightness evaluation value calculation unit, 12n ... Weighting calculation unit, 12p ... White balance correction value calculation unit, 18 ... Memory, 38 ... CCD (Solid-state imaging device), 56 ... Image input controller, 100 ... Digital Signal processing circuit, 112 ... integrating circuit

Claims (2)

A white balance correction device that calculates a white balance correction value based on an image signal for each color and corrects the white balance of the image signal based on the white balance correction value,
Means for detecting color information of each part obtained by dividing the screen based on an image signal in each part of the screen;
Means for detecting brightness information of each part of the screen;
Color information removing means for removing color information that is likely to be an object color that approximates a predetermined light source color among a plurality of color information in the screen based on brightness information of each part of the screen;
White balance correction value calculating means for calculating the white balance correction value based on a plurality of pieces of color information from which an object color similar to the predetermined light source color is removed;
With
The image signal is composed of an R signal, a G signal, and a B signal for each color of red, green, and blue,
The color information includes a B / G value that is a ratio of an integrated value of the B signal and an integrated value of the G signal for each part of the screen,
The color information removing unit removes the color information when the B / G value is equal to or smaller than a threshold value and the brightness information is equal to or larger than a value determined for each B / G value. White balance correction device. A white balance correction method for calculating a white balance correction value based on an image signal for each color and correcting the white balance of the image signal based on the white balance correction value,
Detecting color information of each part obtained by dividing the screen based on an image signal in each part of the screen;
Detecting brightness information of each part of the screen;
A color information removal step of removing color information that is likely to be an object color that approximates a predetermined light source color among a plurality of color information in the screen based on brightness information of each part of the screen;
Calculating a correction value of the white balance based on a plurality of pieces of color information from which an object color similar to the predetermined light source color is removed;
Including
The image signal is composed of an R signal, a G signal, and a B signal for each color of red, green, and blue,
The color information includes a B / G value that is a ratio of an integrated value of the B signal and an integrated value of the G signal for each part of the screen,
In the color information removal step, the color information is removed when the B / G value is not more than a threshold value and the brightness information is not less than a value determined for each B / G value. White balance correction method.

Images