JP4067033B2 - Image processing apparatus and method - Google Patents

Description

  The present invention relates to an image processing apparatus and method, and particularly suitable for a digital camera or a video camera having an image recording function, and has a technique for performing white balance control by discriminating between a light source and an object color at the time of image recording. The present invention relates to an image processing apparatus and method.

  Conventionally, there has been a demand for auto white balance control used for digital cameras, video cameras, and the like to keep trackability over a wide range of color temperatures, and thereby, it has been attempted to obtain optimal white balance control for images. However, if an attempt is made to maintain the ability to follow a wide range of color temperatures, the object color may be mistakenly identified as a light source as a side effect, which may cause a color feria phenomenon. Increasing the followability in such auto white balance control has a so-called trade-off relationship in which the probability of erroneously determining the light source at the same time increases.

  It is important to prevent white balance failure in a low color temperature region, which is a red-yellow light source region, in a wide range of color temperature regions where such a trade-off relationship is established. Red-yellow object colors are very similar in RGB distribution to light sources such as tungsten with low color temperature, and it is difficult to distinguish between object color scenes and light source scenes, and correction is not performed with priority on object colors. Whether it is better (in this case, the tungsten light source cannot be corrected and sacrificed) or the color feria phenomenon of the object color with priority given to the light source (the object color is corrected to white because the object color is misidentified as the light source) The idea of camera design has diverged greatly as I closed my eyes.

  In order to solve such a problem, Patent Document 1 below uses a method of taking a white balance in a low saturation portion, focusing on the fact that there is a high probability that the light source has low saturation. The rate was low. In other words, object color scenes can be saved almost without overcorrection, but when shooting blue object colors under a tungsten light source in a blue object color scene, the blue object color is less saturated than the tungsten light source. It will be.

In addition, in the light source scene, in the case of a light source in which two or more light sources are mixed, there is a problem that the correction is insufficient because only the white balance of the low saturation light source is applied. For example, if a night view, neon sign, etc. are visible from a window at a tungsten light source restaurant, etc., the neon sign is considered to be less saturated, and the amount of white balance gain correction is insufficient, resulting in an image with too much tungsten light source remaining. End up. Such scenes are by no means the shooting frequency.
JP-A-6-98335

  In view of the above facts, an object of the present invention is to provide an image processing apparatus and method capable of performing appropriate white balance control according to a scene.

  The present invention of claim 1 is an image processing apparatus for processing image data of an image obtained by photographing a subject using an imaging means for converting an optical image into an electrical signal, and comprising a screen on which the subject is imaged. A color information acquisition unit that divides the image into a plurality of areas and acquires color information for each area and a detection frame that indicates at least a color distribution range corresponding to a light source type are set, and the acquired color information for each area Based on the number acquisition means for determining the number of areas that fall within the detection frame based on the number of areas, the ratio of the number of areas belonging to the low-saturation region is small from the determined number of areas, and the ratio of the main light sources on the screen is determined to be low A color evaluation value between the evaluation value that follows a wide range of color temperatures and the low saturation evaluation value that retains the color of the light source, depending on the ratio of the number of areas belonging to the low saturation region Follow the wide range of color temperature Color evaluation value calculation means for calculating a color evaluation value by changing the ratio between the evaluation value to be evaluated and the evaluation value for low saturation that leaves the color of the light source, and a white balance gain value is calculated based on the calculated color evaluation value And an image processing apparatus.

  According to the first aspect of the present invention, when the ratio of the number of areas belonging to the low saturation region is small from the number of areas obtained, the light source usually has a high probability of low saturation. It is determined that the ratio of the light source is low, and a color evaluation value is calculated that takes between an evaluation value that follows a wide range of color temperatures and an evaluation value for low saturation that retains the color of the light source. The color evaluation value is obtained by changing the ratio between the evaluation value that follows the wide range of color temperature and the evaluation value for low saturation that retains the color of the light source according to the ratio of the number of areas belonging to the low saturation region. Is. This ratio varies with the scene on the screen and depends on the ratio of the main light sources. That is, when the ratio of the number of areas belonging to the low saturation area is particularly small, it cannot be determined that the area of the low saturation area is really a light source, so the color evaluation value is for low saturation. The ratio of evaluation values is lowered to increase the ratio of evaluation values that follow a wide range of color temperatures. As a result, it is possible to prevent side effect elements such as color feria that occur when the white balance gain value is obtained based on the low saturation region. The ratio of the evaluation value that follows the wide range of color temperatures and the low saturation evaluation value that retains the color of the light source is automatically changed depending on the scene on the screen.

  According to the first and second aspects of the present invention, even when the light source cannot be specified, a side effect element such as a color feria can be prevented, and the scene pass rate of the screen can be further increased. Therefore, an appropriate white balance gain value can be calculated, and the image quality is improved.

  A preferred embodiment of a digital camera that functions as an image processing apparatus according to the present invention will be described below with reference to the accompanying drawings.

  FIG. 1 is a block diagram of a digital camera that functions as an image processing apparatus according to the first embodiment of the present invention.

  The camera 10 is a digital camera having image recording and playback functions, and the overall operation of the camera 10 is controlled by a central processing unit (CPU) 12. The CPU 12 functions as a control unit that controls the camera system according to a predetermined program, and performs various calculations such as automatic exposure (AE) calculation, automatic focus adjustment (AF) calculation, and automatic white balance adjustment (AWB) calculation. Functions as a calculation means.

  Note that automatic exposure, automatic focus adjustment, and automatic white balance adjustment may be described as AE, AF, and AWB, respectively.

  A ROM 16 connected to the CPU 12 via the bus 14 stores programs executed by the CPU 12, various data necessary for control, and the like. Note that the ROM 16 which is a nonvolatile storage means may be rewritable or may be rewritable like an EEPROM.

  The memory (SDRAM) 18 is used as a program development area and a calculation work area for the CPU 12, and is also used as a temporary storage area for image data and audio data. The VRAM 20 is a temporary storage memory dedicated to image data, and includes an A area 20A and a B area 20B. The memory 18 and the VRAM 20 can be shared.

  The camera 10 is provided with a mode selection switch 22, a shooting button 24, and other operation means 26 such as a menu / OK key, a cross key, and a cancel key. Signals from these various operation units (22 to 26) are input to the CPU 12, and the CPU 12 controls each circuit of the camera 10 based on the input signals. For example, lens driving control, photographing operation control, image processing control, image Data recording / reproduction control, display control of the image display device 28, and the like are performed.

  The mode selection switch 22 is an operation means for switching between a still image shooting mode, a moving image shooting mode, and a playback mode. When the mode selection switch 20 is operated to connect the movable contact piece 22A to the contact point a, the signal is input to the CPU 12, the camera 10 is set to the still image shooting mode, and the movable contact piece 22A is connected to the contact point b. The camera 10 is set to the moving image shooting mode. Further, when the movable piece 22A is connected to the contact c, the camera 10 is set to a “reproduction mode” for reproducing a recorded image.

  When the image data to be reproduced is a still image, the still image is displayed on the image display device 28. When the image to be reproduced is a moving image, first, a representative image of the file is displayed, and whether or not to display an index is selected. When the index is not displayed, playback is started from the beginning of the file. When the index is displayed, when a desired index image is selected and the playback start operation is performed, the selected index image is displayed on the image display device 28. A movie is played from an image corresponding to.

  The shooting button 24 is an operation button for inputting an instruction to start shooting, and functions as a recording start (start) / stop (stop) button in moving image shooting, and functions as a release button in still image shooting. The shooting button 24 is composed of a two-stroke switch having an S1 switch that is turned on when half-pressed and an S2 switch that is turned on when fully pressed.

  The menu / OK key is an operation having both a function as a menu button for instructing to display a menu on the screen of the image display device 28 and a function as an OK button for instructing confirmation and execution of selection contents. Key. The cross key is an operation unit for inputting instructions in four directions, up, down, left, and right, and functions as a button (cursor moving operation means) for selecting an item from the menu screen or instructing selection of various setting items from each menu. To do. The up / down key of the cross key functions as a zoom switch at the time of shooting or a playback zoom switch at the time of playback, and the left / right key functions as a frame advance (forward / reverse feed) button in the playback mode. The cancel key is used when deleting a desired item such as a selection item, canceling an instruction content, or returning to the previous operation state.

  The image display device 28 is composed of a liquid crystal display capable of color display. The image display device 28 can be used as an electronic viewfinder for checking the angle of view at the time of shooting, and is used as a means for reproducing and displaying a recorded image. The image display device 28 is also used as a user interface display screen, and displays information such as menu information, selection items, and setting contents as necessary. Instead of the liquid crystal display, other types of display devices (display means) such as an organic EL can be used.

  The digital camera 10 has a media socket (media mounting portion) 30, and a recording medium 32 can be mounted on the media socket 30. The form of the recording medium is not particularly limited, and various media such as a semiconductor memory card represented by XD-PictureCard (registered trademark) and smart media (registered trademark), a portable small hard disk, a magnetic disk, an optical disk, and a magneto-optical disk. Can be used.

  In carrying out the present invention, the number of media sockets is not particularly limited, and a plurality of media sockets may be used. Further, the means for recording the image data is not limited to the removable medium, but may be a recording medium (internal memory) built in the digital camera 10.

  The media controller 34 performs necessary signal conversion in order to deliver an input / output signal suitable for the recording medium 32 attached to the media socket 30.

  The digital camera 10 also includes a USB interface unit 36 as a communication means for connecting to a personal computer or other external devices. By connecting the digital camera 10 to an external device using a USB cable (not shown), data can be exchanged with the external device. Of course, the communication method is not limited to USB, and IEEE1394, Bluetooth, or other communication methods may be applied.

  Next, the photographing function of the digital camera 10 will be described.

  When the still image shooting mode or the moving image shooting mode is selected by the mode selection switch 22, power is supplied to an imaging unit including a CCD solid-state imaging device (hereinafter referred to as a CCD) 38 and the camera is ready for shooting.

  The lens unit 40 is an optical unit that includes a photographic lens 42 including a focus lens and an aperture / mechanical shutter 44. The lens unit 40 is electrically driven by a lens driving unit 46 and a diaphragm driving unit 48 that are controlled by the CPU 12 to perform zoom control, focus control, and iris control.

  The light that has passed through the lens unit 40 is imaged on the light receiving surface of the CCD 38. A large number of photodiodes (light receiving elements) are two-dimensionally arranged on the light receiving surface of the CCD 38, and primary color filters of red (R), green (G), and blue (B) are provided for each photodiode. They are arranged in a predetermined arrangement structure (Bayer, G stripe, etc.). The CCD 38 has an electronic shutter function for controlling the charge accumulation time (shutter speed) of each photodiode. The CPU 12 controls the charge accumulation time in the CCD 38 via the timing generator 50. Instead of the CCD 38, another type of image sensor such as a MOS type may be used.

  The subject image formed on the light receiving surface of the CCD 38 is converted into a signal charge of an amount corresponding to the amount of incident light by each photodiode. The signal charge accumulated in each photodiode is sequentially read out as a voltage signal (image signal) corresponding to the signal charge based on a drive pulse given from the timing generator 50 in accordance with a command from the CPU 12.

  The signal output from the CCD 38 is sent to an analog processing unit (CDS / AMP) 52, where the R, G, B signals for each pixel are sampled and held (correlated double sampling processing), amplified, and then A / Applied to the D converter 54. The A / D converter 54 converts R, G, and B signals that are sequentially input into digital signals and outputs them to the image input controller 56.

  The image signal processing circuit 58 processes a digital image signal input via the image input controller 56 in accordance with a command from the CPU 12. That is, the image signal processing circuit 58 is a synchronization circuit (a processing circuit that converts a color signal into a simultaneous expression by interpolating a spatial shift of the color signal associated with the color filter array of a single CCD), and generates a luminance / color difference signal. It functions as an image processing means including a circuit, a gamma correction circuit, a contour correction circuit, a white balance correction circuit, etc., and performs predetermined signal processing using the memory 18 in accordance with commands from the CPU 12.

  Note that the image input controller 56 includes a memory controller that performs data read control and write control of the memory 18 and the VRAM 20.

  The RGB image data input to the image signal processing circuit 58 is converted into a luminance signal (Y signal) and a color difference signal (Cr, Cb signal) by the image signal processing circuit 58, and predetermined processing such as gamma correction is performed. Applied. The image data processed by the image signal processing circuit 58 is stored in the VRAM 20.

  When the captured image is output to the image display device 28 on the monitor, the image data is read from the VRAM 20 and sent to the video encoder 60 via the bus 14. The video encoder 60 converts the input image data into a predetermined signal for display (for example, an NTSC color composite video signal) and outputs the signal to the image display device 28.

  Image data representing an image for one frame is rewritten alternately in the A region 20A and the B region 20B by the image signal output from the CCD 38. Of the A area 20A and B area 20B of the VRAM 20, the written image data is read from an area other than the area where the image data is rewritten. In this manner, the image data in the VRAM 20 is periodically rewritten, and a video signal generated from the image data is supplied to the image display device 28, whereby the image being captured is displayed on the image display device 28 in real time. Is done. The photographer can check the shooting angle of view from the video (through movie image) displayed on the image display device 28.

  When the shooting button 24 is pressed halfway and S1 is turned on, the digital camera 10 starts AE and AF processing. That is, the image signal output from the CCD 38 is input to the AF detection circuit 62 and the AE / AWB detection circuit 64 via the image input controller 56 after A / D conversion.

  The AE / AWB detection circuit 64 includes a circuit that divides one screen into a plurality of areas (for example, 8 × 8, 16 × 16) and integrates RGB signals for each divided area, and provides the integrated value to the CPU 12. . The CPU 12 detects the brightness of the subject (subject brightness) based on the integrated value obtained from the AE / AWB detection circuit 64, and calculates an exposure value (shooting EV value) suitable for shooting. In accordance with the obtained exposure value and a predetermined program diagram, the aperture value and the shutter speed are determined, and the CPU 12 controls the electronic shutter and iris of the CCD 38 to obtain an appropriate exposure amount.

  The AE / AWB detection circuit 64 calculates an average integrated value for each color of the RGB signals for each divided area and provides the calculation result to the CPU 12. The CPU 12 obtains the integrated value of R, the integrated value of B, and the integrated value of G, obtains the ratio of R / G and B / G, and uses the R / G and B / G values to determine the white balance by the method described later. An adjustment value is calculated, the amplifier gain of the white balance adjustment circuit is controlled according to the calculated white balance adjustment value, and the signal of each color channel is corrected. Details of the white balance control will be described later.

  The AF control in the camera 10 is, for example, a contrast AF that moves a focusing lens (a moving lens that contributes to focus adjustment among the lens optical systems constituting the photographing lens 42) so that the high-frequency component of the G signal of the video signal is maximized. Applies. That is, the AF detection circuit 62 cuts out a signal in a focus target area preset in a high-pass filter that passes only a high-frequency component of the G signal, an absolute value processing unit, and a screen (for example, the center of the screen). An area extraction unit and an integration unit that integrates absolute value data in the AF area are configured.

  The integrated value data obtained by the AF detection circuit 62 is notified to the CPU 12. The CPU 12 calculates a focus evaluation value (AF evaluation value) at a plurality of AF detection points while moving the focusing lens by controlling the lens driving unit 46, and determines a lens position where the evaluation value is a maximum as a focus position. To do. Then, the lens driving unit 46 is controlled so as to move the focusing lens to the obtained in-focus position. The calculation of the AF evaluation value is not limited to a mode using the G signal, and a luminance signal (Y signal) may be used.

  The shooting button 24 is half-pressed, AE / AF processing is performed when S1 is turned on, the shooting button 24 is fully pressed, and the shooting operation for recording starts when S2 is turned on. The image data acquired in response to S2 ON is converted into a luminance / color difference signal (Y / C signal) by the image signal processing circuit 58, subjected to predetermined processing such as gamma correction, and then stored in the memory 18. The

  The Y / C signal stored in the memory 18 is compressed according to a predetermined format by the compression / decompression circuit 66 and then recorded on the recording medium 32 via the media controller 34. For example, a still picture is recorded in the JPEG (Joint Photographic Experts Group) format, and a moving picture is recorded in the motion JPEG format.

  In the motion JPEG method, the image data acquired from the CCD 38 is temporarily stored in the internal memory 18, and then converted into luminance / color difference signals (Y / C signals) by the image signal processing circuit 58, and JPEG compression is performed within one image. However, still images are continuously stored in the external recording medium 32 at a constant frame rate (for example, 30 frames / second).

  The file format of the moving image is not limited to motion JPEG, and other file formats such as MPEG may be applied. Also, the recording format of the still image file is not limited to the JPEG format, and other formats such as a TIF format and a BMP format may be used. In the case of the MPEG system, recording is continuously performed on the external recording medium 32 while performing intra-frame compression and inter-frame compression.

  During moving image shooting, so-called “mountain climbing” continuous AF processing (continuous AF) is performed. That is, the focusing lens is slightly moved back and forth along the optical axis, and the focusing lens is gradually moved to the maximum point of the AF evaluation value while checking the increase / decrease direction of the focus evaluation value.

  The sound during video shooting is detected by a microphone (not shown in FIG. 1), and the detection signal (sound signal) is converted into a digital signal by an A / D converter (not shown in FIG. 1). It is sent to a processing circuit (not shown in FIG. 1). The audio signal processing circuit is connected to the bus 14 and converts the audio data into a predetermined signal format in accordance with an instruction from the CPU 12 while using the memory 18. The audio data generated in this way is stored in the memory 18, compressed by the compression / decompression circuit 66 together with the image data, and then recorded on the recording medium 32.

  After the start of recording, when the photographing button 24 is pressed again (S1 is turned on), the recording operation is stopped. There are a mode in which the recording operation is performed while the shooting button 24 is continuously pressed and the recording is stopped by releasing the pressing, and a mode in which the recording is automatically stopped at a certain time after the recording is started.

  When the playback mode is selected by the mode selection switch 22, the compressed data of the last image file (last recorded file) recorded on the recording medium 32 is read. When the file related to the last recording is a still image file, the read image compression data is expanded to an uncompressed YC signal via the compression / decompression circuit 66, and the image signal processing circuit 58 and the video encoder 60 are used. After being converted into a display signal, it is output to the image display device 28. Thereby, the image content of the file is displayed on the screen of the image display device 28.

  When the reproduction target file is a moving image file, an index image is displayed and a screen for accepting an instruction to start moving image reproduction is displayed. When a desired index image is selected from the displayed index images and a predetermined operation (for example, pressing the down key of the cross key) is performed from the operation unit, the playback process of the moving image file corresponding to the selected index image starts. Then, the moving image is displayed on the image display device 28, and the audio data reproduced by the audio signal processing circuit is output to a speaker (not shown in FIG. 1) via the D / A converter, together with the moving image. Recorded audio is also played.

  During single-frame playback of a still image (including playback of the first frame of a moving image), the file to be played is switched by operating the right or left key of the four-way controller (forward / reverse frame advance). Can do. The image file at the frame-advanced position is read from the recording medium 32, and still images and moving images are reproduced and displayed on the image display device 28 in the same manner as described above.

  Next, the auto white balance adjustment function of the digital camera 10 configured as described above will be described with reference to FIGS.

  FIG. 2 is a block diagram of a portion related to the auto white balance process of the digital camera 10 according to the present embodiment. 2, parts that are the same as or similar to those in FIG.

  A digital signal processing circuit 100 in FIG. 2 is a part of the image signal processing circuit 58 in FIG. 1, and includes a synchronization circuit 102, a white balance adjustment circuit 104, a gamma correction circuit 106, a YC signal generation circuit 108, and a memory 110. It is configured. Here, the memory 110 is the same as the memory 18 of FIG. 1, but in order to facilitate understanding of the present invention, the memory 18 and the memory 110 are shown separately in FIG. In FIG. 1, the memory 18 and the memory 110 are collectively referred to as the memory 18.

  The R, G, and B signals output from the A / D converter 54 are temporarily stored in the memory 18 via the image input controller 56. The synchronization circuit 102 converts the dot-sequential R, G, and B signals read from the memory 18 into simultaneous equations, and outputs the R, G, and B signals to the white balance adjustment circuit 104 at the same time. The white balance adjustment circuit 104 includes multipliers 104R, 104G, and 104B for increasing and decreasing the digital values of the R, G, and B signals. The R, G, and B signals are multiplied by the multipliers 104R, 104G, and 104B, respectively. Added to 104B.

  White balance correction values (gain values) Rg, Gg, and Bg for white balance control are added from the CPU 12 to the other inputs of the multipliers 104R, 104G, and 104B. The multipliers 104R, 104G, and 104B are respectively Two inputs are multiplied, and R ′, G ′, and B ′ signals that have undergone white balance adjustment by this multiplication are output to the gamma correction circuit 106.

  Assuming that the white balance correction values are Rg, Gg, Bg, the signals before correction are R, G, B, and the correction results in the white balance adjustment circuit 104 are R ′, G ′, B ′, the correction results R ′, G ′ and B ′ are expressed by the following equations.

[Equation 1]
R ′ = Rg × R
G ′ = Gg × G
B ′ = Bg × B
The gamma correction circuit 106 changes the input / output characteristics so that the white balance adjusted R ′, G ′, and B ′ signals have desired gamma characteristics, and outputs them to the YC signal generation circuit 108. The YC signal generation circuit 108 generates a luminance signal Y and chroma signals Cr and Cb which are color difference signals from the R, G and B signals subjected to gamma correction. These luminance signal Y and chroma signals Cr and Cb are stored in the memory 110. By reading the YC signal in the memory 110 and outputting it to the image display device 28, a moving image or a still image can be displayed on the image display device 28.

  Next, the flow of auto white balance (AWB) control will be described.

  FIG. 6 is a flowchart showing the flow of auto white balance control.

  The auto white balance control is started in response to full depression of the shooting button (S2 ON) (step 200), and RGB data for one screen acquired from the CCD 38 is stored in the memory 18. The imaging screen is divided into a plurality of areas (for example, 8 × 8, 16 × 16, etc.), and an average integrated value for each color of R, G, and B signals is calculated for each area (step 202).

  The ratio R / G between the R signal and the G signal and the ratio B / G between the B signal and the G signal are obtained from the obtained integrated value for each color (step 204).

  The average integrated value for each color of the R, G, and B signals for each area is calculated by the integrating circuit 112 shown in FIG. 2 and sent to the CPU 12. Further, multipliers 112R, 112G, and 112B are provided between the integrating circuit 112 and the CPU 12, and an adjustment gain is added to the multipliers 112R, 112G, and 112B in order to adjust device variations. Yes. The integrating circuit 112 shown in FIG. 2 is included in the AE / AWB detection circuit 64 shown in FIG.

  Here, in order to calculate the white balance gain correction amount, it is necessary to calculate the color evaluation value HCLi. The color evaluation value HCLi is a weighted coefficient for AWB calculation using a low-saturation light source as a light source, and is a coefficient calculated from a saturation evaluation value HSi and a brightness element for estimating a low-saturation light source. is there. That is,

[Equation 2]
F (color evaluation value HCLi) = F (saturation evaluation value HSi) × F (element of brightness)
It is. The brightness element is a function that is determined by the EV value calculated from the photographic exposure value and is weighted by the brightness of the subject. Here, the luminance (EV value Evi) of each area is obtained by the following equation.

[Equation 3]
Evi = Ev + log2 (Gi / 45)
However, Ev: Shooting EV value Gi: Calculated based on the average integrated value of G in each area. In addition, 45 in the said formula is an appropriate value in the value after A / D conversion.

  The saturation evaluation value HSi is calculated by the following procedure.

  The saturation evaluation value is used to separate the object color and the light source using a distance discrimination algorithm using a system using low saturation as the light source. That is, this algorithm obtains the distance between the distribution of the detection frame with the lowest saturation and the distribution of the other detection frames on the axis on which the frame is prepared for each color temperature described below, and the distribution of the other detection frames is the highest. This is an evaluation value conversion method in which the closer to the distribution of detection frames with lower saturation, the higher the evaluation value is determined by determining that the distribution of detection frames with the lowest saturation seems to be a light source.

  First, the color information (R / G, B / G) obtained for each area determines which detection frame is included in the detection frames represented on the color coordinate 300 of FIG. Used to do. The detection frames such as the shade detection frame and the daylight white detection frame in FIG. 3 are frames represented on the color coordinate 300 with the horizontal axis being R / G × 50 and the vertical axis being B / G × 100. Yes, a range of color distribution such as a light source type is defined for each detection frame.

  FIG. 3 shows a blue sky detection frame, a shade detection frame, a daylight color detection frame, a sunny detection frame, a green detection frame, a daylight white detection frame, a tungsten 1 detection frame, a tungsten 2 detection frame, and a tungsten 3 detection frame. However, the types and number of detection frames, the setting range, and the like are not limited to this example, and various designs are possible.

  Using the detection frames shown in FIG. 3, a distribution of 256 points corresponding to each area of the imaging screen is obtained, and the distribution of 256 points is averaged for each detection frame. That is, for only the points that are output to each detection frame, the number that has entered each detection frame and the average coordinates (Ri / Gi, Bi / Gi) in each detection frame are calculated (step 206 in FIG. 6).

  A distance ΔSi from the origin (gray) of the color coordinate 300 in FIG. 3 to the average coordinate (Ri / Gi, Bi / Gi) of each detection frame in which the distribution has occurred is obtained (step 208 in FIG. 6), and the most colored A degree close to the origin (degree on the R / G axis and B / G axis) is defined as a light source (steps 210 and 212), and a detection frame with the lowest saturation is used as the light source.

  That is, ΔSi is obtained by the following equation.

[Equation 4]
ΔSi = {(1−Ri / Gi) 2 + (1−Bi / Gi) 2} 1/2
Next, a saturation distance ΔDi (on the R / G axis and B / G axis) from the average coordinate of the detection frame defined as likely to be a light source to the average coordinate of another detection frame is obtained (step 214). Here, the average coordinates (Ri / Gi, Bi / Gi) of the light source are replaced with (Xn, Yn). Since (Xn, Yn) is the low saturation detection frame average value, the distance ΔDi of each frame coordinate average (Xi, Yi) from this low saturation detection frame average value (Xn, Yn) is obtained by the following equation. It is done.

[Equation 5]
ΔDi = {(Xn−Xi) 2 + (Yn−Yi) 2} 1/2
Next, in order to change the correction amount of the color information in each detection frame where the distribution exists according to the distance from the light source, the saturation evaluation value HSi is determined using the function of FIG. 4 (step 216). The function shown in FIG. 4 corrects the distribution of the position where the average coordinates are close to the low saturation detection frame as a light source by increasing the saturation evaluation value, and distributes the distribution as the average coordinates become farther from the low saturation detection frame. Is to gradually lower the saturation evaluation value as an object color that has a strong influence of the colors it has.

  That is, when viewed on the R / G axis and the B / G axis, it is as shown in FIG. In FIG. 5, the one in the first neighboring circle 400 is close to the low saturation detection frame, and can be determined to be very light source. Those outside the second neighboring circle 402 are determined to have a higher object color probability as they move away from the circle, and the saturation evaluation value becomes lower.

  Through the above processing, the saturation evaluation value HSi is output (step 218).

  Thereafter, the color evaluation value HCLi is calculated by the above equation (2) (step 220).

  A white balance gain correction amount is obtained using the calculated color evaluation value HCLi.

  First, R / G-axis and B / G-axis gain correction amounts Gri and Gbi are calculated so as to bring the coordinate average value of each detection frame to gray (step 222). Then, final R / G and B / G gains Gr ′ and Gb ′ are calculated by weighting processing with the color evaluation value HCLi according to the following equation (step 224).

[Equation 6]
Gr ′ = Σ (Gri × HCLi) / ΣHCLi
Gb ′ = Σ (Gbi × HCLi) / ΣHCLi
In this way, when the white balance gains Gr ′ and Gb ′ are calculated, the auto white balance control ends.

  According to the first embodiment, a low saturation region is estimated as a light source, and an evaluation value that is weighted according to the color distance on the R / G axis and B / G axis from the light source is calculated. By calculating the white balance gain correction value based on this, overcorrection of the object color can be prevented, and the image quality of the camera is improved.

  Next, a digital camera that functions as an image processing apparatus according to the second embodiment will be described with reference to FIGS.

  The second embodiment pays attention to the fact that there are cases in which a correction amount that makes use of the color of the light source is preferred depending on the scene, and cases in which, for example, gray ones are desired to be taken in gray, white balance gain that makes use of the light source characteristics It is characterized by having a function that can be corrected by switching the specific gravity between the case of correcting and the case of performing white balance gain correction that completely follows depending on the scene. As a result, the scene pass rate can be made higher.

  Since the configuration of the digital camera that functions as the image processing apparatus according to the second embodiment is the same as that of the first embodiment, the description thereof is omitted.

  Assuming that the white balance correction values are Rg, Gg, Bg, the signals before correction are R, G, B, and the correction results in the white balance adjustment circuit 104 are R ′, G ′, B ′, the correction results R ′, Similarly to the first embodiment, G ′ and B ′ are represented by the above formula (1). The calculation method of the white balance gain correction amount is different from that of the first embodiment and will be described.

  The flow of auto white balance (AWB) control according to the second embodiment will be described.

  FIG. 9 is a flowchart showing a flow of auto white balance control according to the second embodiment.

  The auto white balance control is started in response to full depression of the shooting button (S2 ON) (step 230), and the RGB data for one screen acquired from the CCD 38 is stored in the memory 18. The imaging screen is divided into a plurality of areas (for example, 8 × 8, 16 × 16, etc.), and an average integrated value for each color of R, G, and B signals is calculated for each area (step 232).

  A ratio R / G between the R signal and the G signal and a ratio B / G between the B signal and the G signal are obtained from the obtained integrated value for each color (step 234).

  The average integrated value for each color of the R, G, and B signals for each area is calculated by the integrating circuit 112 shown in FIG. 2 and sent to the CPU 12. Further, multipliers 112R, 112G, and 112B are provided between the integrating circuit 112 and the CPU 12, and an adjustment gain is added to the multipliers 112R, 112G, and 112B in order to adjust device variations. Yes. The integrating circuit 112 shown in FIG. 2 is included in the AE / AWB detection circuit 64 shown in FIG.

  Here, in order to calculate the white balance gain correction amount, it is necessary to calculate the weighted evaluation value HCLi ′. The weighted evaluation value HCLi ′ is a weighted coefficient for AWB calculation, and is a coefficient calculated from the color evaluation value HCLi and brightness elements. That is,

[Equation 7]
F (weighted evaluation value HCLi ′) = F (color evaluation value HCLi) × F (element of brightness)
It is. The brightness element is a function that is determined by the EV value calculated from the photographic exposure value and is weighted by the brightness of the subject. Here, the luminance (EV value Evi) of each area is obtained by the following equation.

[Equation 8]
Evi = Ev + log2 (Gi / 45)
Where Ev: EV value
Gi: Calculated based on the average integrated value of G in each area. In addition, 45 in the said formula is an appropriate value in the value after A / D conversion.

  The color evaluation value HCLi is a function that can calculate an evaluation value (following evaluation value HCi) that becomes completely gray depending on the scene, or switch to the saturation evaluation value HSi. The color evaluation value HCLi is expressed by the following equation using the follow-up evaluation value HCi and the saturation evaluation value HSi.

[Equation 9]
α = α '× β
HCLi = HSi × α + HCi × (1−α) (α <= 1)
Since α ′ is a value for each light source for each color temperature, it is possible to place importance on the saturation evaluation value depending on the light source. The correction amount β forming α is determined using the function of FIG.

  The function of FIG. 7 reduces the specific gravity of the saturation evaluation value because it cannot be determined that the light source is really a light source when the number of low saturation distributions (area ratio of the screen) is small when low saturation is used as the light source. It is. As a result, a color evaluation value for the entire scene is calculated.

  The follow-up evaluation value HCi is calculated by the following calculation (step 236).

[Equation 10]
F (follow-up evaluation value HCi) = F (number of distributions) × F (element of brightness)
As shown in FIG. 8, the follow-up evaluation value HCi detects a group of colors (lumps) in the color distribution on the screen on the R / G axis and B / G axis, and determines the number of the respective chunks. This is a calculated evaluation value, which is a function of the number of distributions. That is, the follow-up evaluation value is calculated from the screen area ratio and brightness. The follow-up evaluation value is determined by weighting a large number of detected numbers on the R / G axis and B / G axis distribution as shown in FIG.

  In the above equation (9), the saturation evaluation value HSi that looks like a light source is mixed with the follow-up evaluation value HCi that looks at the color dominance ratio of the screen. This has the effect of alleviating inconveniences when the light source discrimination is not successful and the correction amount is insufficient.

  Thereafter, the follow-up evaluation value HCi is output (step 238).

  The saturation evaluation value HSi is calculated in the same manner as in the first embodiment (step 240).

  Then, the color evaluation value HCLi is calculated by the equation [Equation 9] (step 242) and output (step 244).

  Thereafter, the weighted evaluation value HCLi ′ is calculated by the above method (step 246), and the final auto white balance gain is thereafter calculated by the same method as in the first embodiment (steps 248 and 250). ).

  According to the second embodiment, switching between the wide-range tracking algorithm and the low-saturation algorithm that keeps the color of the light source can be automatically performed according to the scene, and the image quality of the camera is improved. The ratio between the perfect color temperature tracking algorithm and the low saturation light source algorithm varies from scene to scene and depends on the main light source probability.

  In the first and second embodiments, the R / G and B / G coordinate systems are used for the color coordinate, but other coordinate systems such as the R, G, and B coordinate systems are used for the color coordinate system. May be.

1 is a block diagram of a digital camera according to an embodiment of the present invention. FIG. 3 is a block diagram of a portion related to white balance processing of the digital camera according to the present embodiment. The figure which showed the light source kind detection frame on a color coordinate. The figure which showed the function which calculates a saturation evaluation value based on the distance to a light source. The figure which showed the distance from the light source to the average coordinate value of another detection frame on the R / G axis and the B / G axis. 6 is a flowchart showing a flow of auto white balance control according to the first embodiment. The figure which showed the function which calculates correction amount (beta) based on the number of distribution of low saturation. The schematic diagram which detects the group of the color in the distribution of the color in a screen on a R / G axis | shaft and a B / G axis. The flowchart which showed the flow of the auto white balance control of 2nd Embodiment.

Explanation of symbols

  DESCRIPTION OF SYMBOLS 10 ... Digital camera, 12 ... CPU, 18 ... Memory, 24 ... Shooting button, 58 ... Image signal processing circuit, 64 ... AE / AWB detection circuit, 104 ... White balance adjustment circuit, 112 ... Integration circuit, 300 ... Color coordinate

Claims (2)

An image processing apparatus that processes image data of an image obtained by photographing a subject using an imaging unit that converts an optical image into an electrical signal,
A color information acquisition unit that divides a screen on which a subject is imaged into a plurality of areas and acquires color information for each area;
A number acquisition means for setting a detection frame indicating a range of color distribution corresponding to at least a light source type, and determining the number of areas entering the detection frame based on the acquired color information for each area;
When the ratio of the number of areas belonging to the low-saturation area is small from the obtained number of areas, it is determined that the ratio of the main light source on the screen is low, and the evaluation value that follows a wide range of color temperatures and the color of the light source are determined. A color evaluation value that falls between the remaining low saturation evaluation values, and retains the evaluation value that follows the wide range of color temperatures and the color of the light source according to the ratio of the number of areas belonging to the low saturation region A color evaluation value calculating means for calculating a color evaluation value in which the ratio to the evaluation value for low saturation is changed;
A computing means for computing a white balance gain value based on the calculated color evaluation value;
An image processing apparatus comprising: An image processing method for processing image data of an image obtained by photographing a subject using an imaging means for converting an optical image into an electrical signal,
Dividing the screen on which the subject is imaged into a plurality of areas and obtaining color information for each area;
Setting a detection frame indicating a range of color distribution corresponding to at least a light source type, and obtaining the number of areas entering the detection frame based on the acquired color information for each area;
When the ratio of the number of areas belonging to the low-saturation area is small from the obtained number of areas, it is determined that the ratio of the main light source on the screen is low, and the evaluation value that follows a wide range of color temperatures and the color of the light source are determined. A color evaluation value that falls between the remaining low saturation evaluation values, and retains the evaluation value that follows the wide range of color temperatures and the color of the light source according to the ratio of the number of areas belonging to the low saturation region Calculating a color evaluation value by changing a ratio with the evaluation value for low saturation;
Calculating a white balance gain value based on the calculated color evaluation value;
An image processing method comprising:

Images