JP2012119885A - Light source determination device and light source determination method - Google Patents

Abstract

PROBLEM TO BE SOLVED: To simply and accurately determine an LED light source without using a stochastic method.SOLUTION: It is possible to simply and accurately determine whether a light source is a white LED, on the basis of a signal amount of the imaging signal corresponding to the light of a wave length near 80 nm and a signal amount of the imaging signal corresponding to the light of a wave length near 450 nm which are obtained from a solid state imaging device.

Description

  The present invention relates to light source determination of an imaging scene, and more particularly to determination of a white LED light source.

  Patent Document 1 discloses a technique for calculating a white balance gain using a black body locus and a fluorescent lamp locus.

  Patent Document 2 discloses a technique for measuring spectral characteristics for each wavelength, calculating a color temperature from a spectral energy distribution, and calculating a white balance gain according to the color temperature.

  According to Patent Document 3, in the method of estimating the light source based on the mapped position in the evaluation space, many of daylight observation values overlap with those of the YAG (yttrium, aluminum, garnet) type of light emitting diode (LED). Therefore, it is very difficult to distinguish between these two light sources, and it is difficult to reproduce natural colors. For this reason, in the invention of Patent Document 3, the light source estimation circuit outputs an input image captured by an image sensor having four types of color filters in which emeralds are added to RGB as an RGBE signal via the A / D conversion circuit. The light source including the LED is estimated by projecting the RGBE signal onto the one-dimensional axis of the linear discrimination method.

  Patent Documents 4 and 5 show examples of membership functions. The membership function is a function that outputs an evaluation value representing the certainty of various light sources such as shade and daylight using the luminance level as a variable.

  In Patent Document 4, white balance gain values Rg, Gg, and Bg for performing good white balance against shade and daylight are prepared in advance, and these gain values Rg, Gg, and Bg are prepared as a white balance adjustment circuit. Thus, white balance control suitable for the light source determined from the evaluation value is performed.

  In Patent Document 5, the white balance correction values Rg, Gg, and Bg are control values for changing the color of the environmental light source to a predetermined color from the reference WB gain value so that the white balance is appropriate in shooting under each light source. A value between GR, GB, and GG and a value capable of performing white balance control suitable for the environmental light source specified by the evaluation value Hmax. Therefore, white balance control is appropriately performed for each environmental light source. And the occurrence of color feria can be suppressed.

  Patent Document 6 exemplifies a method for correcting color information distributed below the fluorescent lamp locus.

  Patent Document 7 is an example of a calculation formula for correlated color temperature.

JP 2003-259196 A JP 2006-54678 A JP 2007-300253 JP JP 2000-224608 A JP 2004-186876 A (particularly paragraph 0115) JP 2006-222672 A (particularly paragraph 0061) JP 2010-258703 (particularly paragraph 0032)

  FIG. 10 shows a distribution range of correlated color temperature and subject luminance corresponding to artificial light sources such as a light bulb, a light bulb color fluorescent lamp, a white fluorescent lamp, a daylight white fluorescent lamp, a daylight color fluorescent lamp, a sodium lamp, a mercury lamp, and a white LED. . FIG. 10 shows the distribution range of correlated color temperature and subject luminance corresponding to natural light sources such as sunrise / sunset, daylight light source, cloudy sky and shade.

  There is a technique for discriminating outdoor light sources / indoor light sources (fluorescent lamps, light bulbs) depending on whether the color temperature (correlated color temperature) and brightness (subject luminance) of the scene belong to the distribution range as shown in FIG. Generally known. Since white LEDs are brighter than conventional light sources, the distribution range overlaps with the distribution range of outdoor light sources. Therefore, it is difficult to distinguish between the LED light source and the outdoor light source based on the distribution range of the correlated color temperature and the subject luminance.

  An LED light source is much brighter than a conventional artificial light source, and rather close to the brightness of an outdoor light source, so that it is particularly difficult to distinguish it from an outdoor light source. In Patent Document 3, since the LED light source and sunlight are probabilistically discriminated by the linear discriminant method, the accuracy of discrimination depends on the sample. Further, in Patent Document 3, since information is projected onto a one-dimensional space and the light source is discriminated, the discrimination accuracy is not high.

  The present invention performs LED light source discrimination simply and accurately without using a probabilistic method. Further, according to the present invention, when the LED light source is determined, appropriate auto white balance correction is performed.

  The present invention is based on an imaging signal output from a solid-state imaging device, and acquires a signal amount of an imaging signal corresponding to light having a wavelength near 480 nm and a signal amount of an imaging signal corresponding to light having a wavelength near 450 nm. And a signal ratio calculation unit that calculates a ratio of a signal amount of an imaging signal corresponding to light having a wavelength of about 480 nm acquired by the signal amount acquisition unit and a signal amount of imaging signal corresponding to light having a wavelength of about 450 nm. And a white LED light source determination unit that determines whether or not the light source is a white LED according to the magnitude relationship between the ratio calculated by the signal ratio calculation unit and a predetermined threshold.

  Preferably, when the white LED light source determination unit determines that the light source is a white LED, the light source determination device sets a white balance correction amount suitable for the white LED, and the imaging signal according to the set white balance correction amount. An AWB correction unit for correcting the white balance of

  Preferably, the AWB correction unit, when the white LED light source determination unit determines that the light source is a white LED, each of the representative colors red (R), green (G), and blue (B) estimated from the imaging signal A normal white balance correction amount is set so that the signal levels are equal.

  Preferably, the light source determination device is based on a subject luminance calculation unit that calculates subject luminance based on an imaging signal output from the solid-state imaging device, and a membership function that uses the subject luminance calculated by the subject luminance calculation unit as a variable. And an evaluation value calculation unit for calculating an evaluation value indicating the certainty of the light source.

  Preferably, the light source determination device includes a conventional artificial light source determination unit that determines whether the light source is an artificial light source other than the white LED when the white LED light source determination unit determines that the light source is not a white LED, When the conventional artificial light source determination unit determines that the light source is an artificial light source other than the white LED, the AWB correction unit determines the white balance correction amount from the normal white balance correction amount according to the evaluation value calculated by the evaluation value calculation unit. Is also set to be small or comparable.

  Preferably, the light source determination device includes a natural light source determination unit that determines whether the light source is a natural light source when the white LED light source determination unit determines that the light source is not a white LED, and the AWB correction unit includes: When the natural light source determination unit determines that the light source is a natural light source, the white balance correction amount is set to be smaller than the normal white balance correction amount according to the evaluation value calculated by the evaluation value calculation unit.

  Preferably, the light source determination device includes a trajectory storage unit that stores a trajectory indicating the distribution of the color temperature of the white LED light source in a predetermined color space, and a signal division unit that divides an image made up of imaging signals into a plurality of areas. Based on the trajectory stored in the trajectory storage unit, the divided area color information acquisition unit that acquires color information of each area divided by the signal division unit and the color information of each area acquired by the divided area color information acquisition unit A color information correction unit that corrects the color information, and when the white LED light source determination unit determines that the light source is a white LED, the AWB correction unit generates an image signal after the color information correction unit corrects the color information. Based on this, a white balance gain suitable for the white LED is set, and the color information correction unit corrects the white balance of the imaging signal after the color information is corrected.

  Preferably, the color information correction unit corrects the color information of each area so as to approach the locus stored in the locus storage unit.

  The signal amount acquisition unit is configured to output an imaging signal corresponding to light having a wavelength near 480 nm from an imaging signal obtained from a specific pixel corresponding to a cyan filter in a pixel group of a solid-state imaging element corresponding to a predetermined color filter. In addition to acquiring the signal amount, the signal amount of the imaging signal corresponding to light having a wavelength near 450 nm is acquired from the imaging signal obtained from the specific pixel corresponding to the blue filter.

  According to the present invention, the light source determination device is based on the imaging signal output from the solid-state imaging device, and the signal amount of the imaging signal corresponding to light having a wavelength near 480 nm and the signal amount of imaging signal corresponding to light having a wavelength near 450 nm. A step of calculating the ratio of the signal amount of the imaging signal corresponding to the acquired light having a wavelength near 480 nm and the signal amount of the imaging signal corresponding to the light having a wavelength near 450 nm; Determining whether or not the light source is a white LED according to a magnitude relationship with a predetermined threshold value.

  According to the present invention, a light source can be simply and accurately based on a signal amount of an imaging signal corresponding to light having a wavelength near 80 nm obtained from a solid-state imaging device and a signal amount of imaging signal corresponding to light having a wavelength near 450 nm. It can be determined whether or not is a white LED. In addition, since the light source is discriminated from the two-dimensional information according to the present invention, the information amount of the light source discrimination is more accurate than the prior art.

Block diagram of the imaging apparatus of the first embodiment The figure which illustrated the arrangement of C / B pixel group X Flowchart of light source discrimination / AWB correction processing of the first embodiment The figure which illustrated the spectral characteristic of white LED The block diagram of the imaging device of 2nd Embodiment Flowchart of light source discrimination / AWB correction processing of the second embodiment The block diagram of the imaging device of 3rd Embodiment Flowchart of light source discrimination / AWB correction processing of the third embodiment The figure which illustrated locus L1-L3 Diagram illustrating the distribution range of correlated color temperature and subject brightness corresponding to various light sources The figure which illustrated the membership function of the AWB correction amount which makes a subject luminance value a variable

<First Embodiment>
FIG. 1 is a block diagram illustrating an imaging apparatus 100 to which a light source determination device according to an embodiment of the present invention is applied. An image pickup apparatus 100 shown in the figure is a digital camera having a recording and playback function for still images and moving images, and the overall operation of the image pickup apparatus 100 is centrally controlled by a central processing unit (CPU) 13. The CPU 13 functions as control means for controlling the camera system according to a predetermined program, and performs various calculations such as automatic exposure (AE) calculation, automatic focus adjustment (AF) calculation, and white balance (WB) adjustment calculation. Functions as a means.

  A ROM 18 connected to the CPU 13 via the bus 1 stores programs executed by the CPU 13, various data necessary for control, CCD pixel defect information, various constants / information relating to camera operation, and the like.

  The RAM 11 is used as a program development area and a calculation work area for the CPU 13 and as a temporary storage area for image data and audio data.

  The imaging apparatus 100 is provided with an operation unit 19 including a mode selection switch, a shooting button, a menu / OK key, a cross key, a cancel key, and the like. A signal from the operation unit 19 is input to the CPU 13, and the CPU 13 controls each circuit of the imaging apparatus 100 based on the input signal. For example, lens drive control, shooting operation control, image processing control, and image data recording / reproduction are performed. Control and display control of the display device 30 are performed.

  The mode selection switch is an operation means for switching the operation mode of the imaging apparatus 100 to the shooting mode or the playback mode.

  The shooting button is an operation button for inputting an instruction to start shooting, and includes a two-stroke switch having an S1 switch that is turned on when half-pressed and an S2 switch that is turned on when fully pressed.

  The menu / OK key is an operation key having both a function as a menu button for instructing to display a menu on the screen of the display device 30 and a function as an OK button for instructing confirmation and execution of selection contents. It is. The cross key is an operation unit for inputting instructions in four directions, up, down, left, and right, and functions as a button (cursor moving operation means) for selecting an item from the menu screen or instructing selection of various setting items from each menu. To do. The up / down key of the cross key functions as a zoom switch at the time of shooting or a playback zoom switch at the time of playback, and the left / right key functions as a frame advance (forward / reverse feed) button in the playback mode. The cancel key is used when deleting a desired item such as a selection item, canceling an instruction content, or returning to the previous operation state.

  The display device 30 is configured by a liquid crystal display capable of color display. The display device 30 can be used as an electronic viewfinder for checking the angle of view at the time of shooting, and is used as a means for reproducing and displaying a recorded image. The display device 30 is also used as a user interface display screen, and displays information such as menu information, selection items, and setting contents as necessary. Instead of the liquid crystal display, other types of display devices (display means) such as an organic EL can be used.

  The imaging apparatus 100 has an external memory I / F 16, and a memory card 17 can be attached to the external memory I / F 16. The form of the recording medium mounted on the external memory I / F 16 is not limited to the memory card 17, but a semiconductor memory card represented by xD-PictureCard (trademark) and smart media (trademark), a portable small hard disk, a magnetic disk, Various media such as an optical disk and a magneto-optical disk can be used.

  The compression / decompression processing circuit 12 performs necessary signal conversion in order to deliver an input / output signal suitable for the memory card 17 mounted in the external memory I / F 16.

  Next, the shooting function of the imaging apparatus 100 will be described. When the operation mode of the image pickup apparatus 100 is set to the shooting mode by the mode selection switch, power is supplied to the image pickup unit 2 including the color CCD solid-state image pickup element (hereinafter referred to as CCD) 21 and the camera is ready for shooting.

  The optical unit 20 includes a photographing lens including a focus lens, and a mechanical shutter combined with an aperture. The optical unit 20 is electrically driven by a motor drive unit 24 controlled by the CPU 13 to perform zoom control, focus control, and iris control.

  The light that has passed through the optical unit 20 is imaged on the light receiving surface of the CCD 21. A large number of photodiodes (light receiving elements) are two-dimensionally arranged on the light receiving surface of the CCD 21, and red (R), green (G), and blue (B) primary color filters corresponding to the respective photodiodes. They are arranged in a predetermined arrangement structure (Bayer, G stripe, etc.). Thereby, pixels corresponding to the respective colors are formed on the light receiving surface of the CCD 21.

  The light-receiving surface of the CCD 21 is a C / B pixel group in which pixels corresponding to the B filter and pixels corresponding to the primary color filter of cyan (C) are adjacent within a predetermined size (two or more pixels) adjacent region. Contains. Where the C / B pixel group is provided on the light receiving surface of the CCD 21 is arbitrary. Preferably, the C / B pixel group is arranged in a region that is not used for forming a two-dimensional image or does not interfere with the formation of a two-dimensional image. The minimum unit of the C / B pixel group is one C pixel and one B pixel. However, since C light and B light may accidentally enter regardless of the white LED light source, the unit of the C / B pixel group exceeds 2 pixels, and the C / B signal amount in a wide range of pixel groups. Are preferably averaged.

  For example, as shown in FIG. 2A, by arranging C instead of the G filter in the GB line at the periphery of the effective pixel region of the light receiving surface where the RGB filters are arranged in a Bayer shape, / B pixel group X can be formed.

  Alternatively, when the imaging apparatus 100 employs an optical system that can acquire an image having a phase difference, within the effective pixel area of the light receiving surface, the area within which the phase difference is acquired (for example, a phase difference AF area or a parallax image). In the acquisition area), C / B pixel groups can be formed by alternately arranging C and B instead of G and B. In FIG. 2B, in the region R for acquiring the phase difference, the GB line of either or both of the pixel group (main pixel) of the left viewpoint image and the pixel group (subpixel) of the right viewpoint image. The C / B pixel group X is formed by arranging C instead of the G filter. Although illustration is omitted, a pixel group that forms a two-dimensional image may be arranged outside the region R, and the color information of the C pixel in the region where the phase difference is acquired is the surrounding G It can also be complemented with pixels. An emerald (E) filter may be used instead of the cyan (C) filter. The image signal of the pixel corresponding to the C filter or E filter may or may not be used for image formation.

  The CCD 21 has an electronic shutter function for controlling the charge accumulation time (shutter speed) of each photodiode. The CPU 13 controls the charge accumulation time in the CCD 21 via the timing generator (TG) 31. Instead of the CCD 21, another type of image sensor such as a MOS type may be used.

  The subject image formed on the light receiving surface of the CCD 21 is converted into a signal charge of an amount corresponding to the amount of incident light by each photodiode. The signal charge accumulated in each photodiode is sequentially read out as a voltage signal (image signal) corresponding to the signal charge based on a drive pulse given from the TG 31 according to a command from the CPU 13.

  The signal output from the CCD 21 is sent to an analog signal processing unit 22 where the R, G, B signals for each pixel are sampled and held (correlated double sampling processing), amplified, and then A / D converter 23. Added to. The dot sequential R, G, B, and C signals converted into digital signals by the A / D converter 23 are stored in the RAM 11 via the memory I / F 10.

  The digital signal processing unit 3 processes the R, G, and B signals stored in the RAM 11 according to instructions from the CPU 13. That is, the digital signal processing unit 3 includes a synchronization circuit (a processing circuit that interpolates a spatial shift of the color signal associated with the color filter array of the single CCD and converts the color signal into a simultaneous expression), a white balance correction circuit, It functions as image processing means including a gamma correction circuit, contour correction circuit, luminance / color difference signal generation circuit, etc., and performs predetermined signal processing using the RAM 11 in accordance with commands from the CPU 13.

  The digital signal processing unit 3 converts the input RGB image data into a luminance signal (Y signal) and a color difference signal (Cr, Cb signal) and performs predetermined processing such as gamma correction. The image data processed by the digital signal processing unit 3 is stored in the RAM 11.

  When the captured image is output to the display device 30 on the monitor, the image data is read from the RAM 11 and sent to the video encoder 32 via the bus 1. The video encoder 32 converts the input image data into a predetermined signal for display (for example, an NTSC color composite video signal) and outputs the converted signal to the display device 30.

  Image data representing an image for one frame is rewritten alternately in the A area and the B area of the RAM 11 by the image signal output from the CCD 21. The written image data is read from the area other than the area where the image data is rewritten, among the A area and B area of the RAM 11. In this way, the image data in the RAM 11 is periodically rewritten, and a video signal generated from the image data is supplied to the display device 30, whereby the image being captured is displayed on the display device 30 in real time. . The photographer can check the shooting angle of view from the video (through movie image) displayed on the display device 30.

  When the shooting button is pressed halfway (S1 = ON), the imaging apparatus 100 starts AE and AF processing. That is, the image signal output from the CCD 21 is input to the AF detection circuit 5 and the AE / AWB detection circuit 4 via the image input controller 56 after A / D conversion.

  The AE / AWB detection circuit 4 includes a circuit that divides one screen into a plurality of areas (for example, 8 × 8 to 16 × 16) and integrates RGB signals for each divided area, and provides the integrated value to the CPU 13. . The CPU 13 detects the brightness of the subject (subject brightness) based on the integrated value obtained from the AE / AWB detection circuit 4, and calculates an exposure value (shooting EV value) suitable for shooting. The aperture value and the shutter speed are determined according to the obtained exposure value and a predetermined program diagram, and the CPU 13 controls the electronic shutter and iris of the CCD 21 according to this to obtain an appropriate exposure amount.

  The AE / AWB detection circuit 4 calculates an average integrated value for each color of the RGB signals for each divided area during automatic white balance adjustment. The AE / AWB detection circuit 4 obtains an integrated value of R, an integrated value of B, and an integrated value of G, obtains a ratio of R / G and B / G for each divided area, and R / G, B / G The light source type is determined based on the distribution of the R / G and B / G axis coordinates in the color space. The AE / AWB detection circuit 4 calculates an AWB correction value (AWB gain value) for the R, G, and B signals according to the determined light source type, and digitally corrects the signal of each color channel with this AWB correction value. The signal processing unit 3 is controlled. Details of the white balance adjustment will be described later.

  The AF control in the imaging apparatus 100 is, for example, a contrast that moves a focusing lens (a moving lens that contributes to focus adjustment among the lens optical systems constituting the photographing lens 42) so that the high-frequency component of the G signal of the video signal is maximized. AF is applied. That is, the AF detection circuit 5 is a high-pass filter that passes only a high-frequency component of the G signal, an absolute value processing unit, and an AF area that cuts out a signal in an AF area set in advance in the screen (for example, the center of the screen). An extraction unit and an integration unit that integrates absolute value data in the AF area are configured.

  The integrated value data obtained by the AF detection circuit 5 is notified to the CPU 13. The CPU 13 calculates a focus evaluation value (AF evaluation value) at a plurality of AF detection points while moving the focusing lens by controlling the motor driving unit 24, and determines a lens position where the evaluation value is a maximum as a focus position. To do. Then, the motor drive unit 24 is controlled so as to move the focusing lens to the obtained in-focus position. The calculation of the AF evaluation value is not limited to a mode using the G signal, and a luminance signal (Y signal) may be used. Further, phase difference AF or other AF control may be performed.

  As described above, when the AE / AF processing is performed by half-pressing the shooting button (S1 = ON), the shooting operation for recording is started by fully pressing the shooting button (S2 = ON). Image data acquired in response to S2 = ON is converted into a luminance / color difference signal (Y / C signal) in the digital signal processing unit 3, and subjected to predetermined processing such as gamma correction, and then stored in the RAM 11. The

  The Y / C signal stored in the RAM 11 is compressed according to a predetermined format by the compression / decompression processing circuit 12 and then recorded on the memory card 17 via the external memory I / F 16. For example, a still image is recorded in JPEG (Joint Photographic Experts Group) format.

  When the playback mode is selected by the mode selection switch, the compressed data of the last image file (last recorded file) recorded on the memory card 17 is read. When the last recorded file is a still image file, the read compressed image data is decompressed to an uncompressed YC signal via the compression / decompression circuit 12, and via the digital signal processing unit 3 and the video encoder 32. After being converted into a display signal, it is output to the display device 30. Thereby, the image content of the file is displayed on the screen of the display device 30.

  During single-frame playback of still images (including playback of the first frame of a movie), the file to be played can be switched (forward / reverse frame advance) by operating the right or left key of the four-way controller. it can. The image file at the frame-advanced position is read from the memory card 17, and still images and moving images are reproduced and displayed on the display device 30 in the same manner as described above.

  The AE / AWB detection circuit 4 includes a signal ratio calculation unit 40, an AWB correction gain calculation unit 41, an LED light source determination unit 42, a subject luminance value calculation unit 43, a conventional artificial light source / outdoor light source determination unit 44, and a correlated color temperature calculation unit 45. The conventional light source determination threshold value calculation unit 46 is included. These functions will be described later.

  FIG. 3 is a flowchart of the light source determination / AWB correction process executed by the imaging apparatus 100. This process starts in response to a half-press of the shutter button.

  In S <b> 1, the correlated color temperature calculation unit 45 calculates the correlated color temperature K <b> 0 based on the RGB signal output from the A / D converter 23. This can be performed in the same manner as in Patent Document 7, for example.

  In S <b> 2, the LED light source determination unit 42 acquires the integrated value of the signal voltage (pixel signal) of the C signal output from the A / D converter 23 and corresponding to the C / B pixel group.

  In S <b> 3, the LED light source determination unit 42 acquires the integrated value of the signal voltage (pixel signal) of the B signal output from the A / D converter 23 and corresponding to the C / B pixel group.

In S4, the signal ratio calculation unit 40 calculates a ratio r between the integrated value of the C signal and the integrated value of the B signal. That is,
r = integrated value of C signal / integrated value of B signal.

  Instead of the integrated value of the signal voltage of the C signal / B signal, the average value of the signal voltage of the C signal / B signal is calculated, and the average value of the signal voltage of the C signal and the average value of the signal voltage of the B signal are calculated. The ratio r may be calculated. Similarly, when the emerald (E) filter is used, either the ratio of the integrated values or the ratio of the average values may be calculated.

  In S5, the LED light source determination unit 42 determines whether or not the ratio r <predetermined threshold value Thresh_LED. If No, the process proceeds to S6, and if Yes, the process proceeds to S7. It is assumed that Thresh_LED is stored in the ROM 18.

  This determines whether or not the light source is an LED based on the spectral characteristics of the white LED as shown in FIG. As illustrated in FIG. 4, according to the spectral characteristics of the white LED, C (near 480 nm) is extremely low and B (near 450 nm) is extremely high. Therefore, in a scene under a white LED, r is considered to be an extremely low value, which is less than a certain threshold value Thresh_LED.

  In S6, the LED light source determination unit 42 determines that the light source is not a white LED.

  In S7, the LED light source determination unit 42 determines that the light source is a white LED.

  In S8, the AWB correction gain calculation unit 41 branches to the processing of S9 to S13 when it is determined that the light source is a white LED, and branches to the processing of S14 when it is determined that the light source is a white LED. .

  In S9, the conventional light source determination threshold value calculator 46 calculates the artificial light source determination threshold value Thresh_K0. Specifically, as shown in FIG. 10, the upper limit value of the subject luminance in the distribution range corresponding to each artificial light source other than the white LED such as a light bulb or a light bulb color fluorescent light is Thresh_K0.

  In S10, the subject luminance value calculation unit 43 calculates the subject luminance value Bv0.

  In S11, the conventional artificial light source / outdoor light source determination unit 44 determines whether or not Bv0 <Thresh_K0. If No, the process proceeds to S12. If Yes, the process proceeds to S13.

  In S12, the AWB correction gain calculation unit 41 calculates an AWB gain optimum for the outdoor light source (sunlight).

  In S13, the AWB correction gain calculation unit 41 calculates an AWB gain optimum for the artificial light source.

  In S14, the AWB correction gain calculation unit 41 calculates the optimum AWB gain for the white LED.

  The digital signal processing unit 3 corrects the white balance of the image by multiplying the RGB signals by the AWB gain calculated in S12, S13, or S14.

  As described above, whether or not the light source is a white LED simply and accurately based on the signal amount of the imaging signal corresponding to light having a wavelength near 480 nm and the signal amount of the imaging signal corresponding to light having a wavelength near 450 nm. Can be judged.

Second Embodiment
FIG. 5 is a block diagram illustrating an imaging apparatus 200 according to the second embodiment. In the figure, the same reference numerals are assigned to the same blocks as those in the first embodiment. The imaging apparatus 200 further includes an AWB correction amount calculation unit 47.

  FIG. 6 shows a flowchart of the light source discrimination / AWB correction process executed by the imaging apparatus 200.

  S1 to S11 are the same as in the first embodiment. However, when it determines with No in S11, it progresses to S21, and when it determines with Yes, it progresses to S23.

  In S21, the AWB correction amount calculation unit 47 sets the AWB correction amount LW_AWB to be smaller than the normal AWB correction amount according to the evaluation value obtained by the membership function using the subject luminance value Bv0 as a variable. The normal AWB correction amount is an AWB correction amount in which the red (R), green (G), and blue (B) color signal levels of input image data are equal (R = G = B: gray). Say.

  In S22, the AWB correction gain calculation unit 41 calculates an AWB gain according to the AWB correction amount LW_AWB set in S21. The digital signal processing unit 3 corrects the white balance of the image by applying the calculated AWB gain to the RGB signals.

  In S23, the AWB correction amount calculation unit 47 sets the AWB correction amount LW_AWB to be smaller than the normal AWB correction amount or the normal value according to the evaluation value obtained by the membership function using the subject luminance value Bv0 as a variable. Is set to the same level as the AWB correction amount.

  In S24, the AWB correction gain calculation unit 41 calculates an AWB gain according to the AWB correction amount LW_AWB set in S23. The digital signal processing unit 3 corrects the white balance of the image by applying the calculated AWB gain to the RGB signals.

  In S25, the AWB correction amount calculation unit 47 sets the AWB correction amount LW_AWB to the same level as the normal AWB correction amount. For example, the average value of R, G, B signals, Ri, Gi, Bi is obtained for each area obtained by dividing the image data by a predetermined unit, and the coordinates Ci of the R / GB-G plane are obtained for each divided area Xi. = (Ri / Gi, Bi / Gi) is calculated. The AWB correction amount calculation unit 47 estimates the representative color C (light source coordinates) of the image from the distribution of Ci in each divided area. The representative color is estimated from the average of the distribution of Ci and the center of gravity. Then, the AWB correction amount calculation unit 47 calculates an AWB correction amount (normal AWB correction amount) such that the color signal levels of red (R), green (G), and blue (B) of the representative color C of the image are equal. Let LW_AWB of the image.

  In S26, the AWB correction gain calculation unit 41 calculates an AWB gain according to the AWB correction amount LW_AWB set in S25.

  The digital signal processing unit 3 corrects the white balance of the image by multiplying the RGB signal by the AWB gain calculated in S22, S24, or S26.

  In the correlated color temperature-subject luminance value space (see FIG. 10), the distribution range of the conventional artificial light source or outdoor light source has a slightly overlapping portion. Therefore, it is necessary to vary the AWB correction amount according to the evaluation value obtained by the membership function (for example, see FIG. 11) of the AWB correction amount using the subject luminance value as a variable. On the other hand, for the white LED, the light source is discriminated from a viewpoint different from the correlated color temperature and the subject luminance value, so there is no need to change the correction amount according to the evaluation value, and the AWB correction amount (gain) is always set. Set to the normal correction amount.

  The white LED is an indoor light source, and the light source color residue is disliked. However, in the above processing, the AWB correction amount when it is determined as a white LED is set to a normal correction amount, and the AWB is higher than the conventional artificial light source or outdoor light source. Increase the correction. Thereby, the light source color residue of the white LED scene can be improved without causing any adverse effect on the AWB correction of the outdoor light source scene or the conventional artificial light source scene.

<Third Embodiment>
FIG. 7 is a block diagram showing an imaging apparatus 300 according to the third embodiment. In the figure, the same reference numerals are assigned to the same blocks as those in the first embodiment. The imaging apparatus 300 further includes a signal dividing unit 50, a color information calculating unit 51, a color information correcting unit 52, and a locus holding unit 53.

  FIG. 8 shows a flowchart of the light source discrimination / AWB correction process executed by the imaging apparatus 300.

  S1 to S11 are the same as in the first embodiment.

  In S31, the signal dividing unit 50 divides image data (R / G / B signal) for one screen stored in the RAM 11 into a plurality of areas. The color information calculation unit 51 obtains an average value of R, G, B signals, Ri, Gi, Bi for each area. The color information calculation unit 51 calculates the coordinates Ci = (Ri / Gi, Bi / Gi) of the R / GB / G plane for each divided area Xi.

  The color information correction unit 52 reads a trajectory (black body trajectory) L1 corresponding to the outdoor light source stored in the trajectory holding unit 53, and a normal line extending from the coordinate Ci to the black body trajectory L1 for each divided area Xi. An intersection point Pi with the black body locus L1 is calculated. The color information correction unit 52 performs correction for bringing the coordinates Ci closer to (matching) Pi for each divided area Xi. That is, the object color is excluded from the image data based on the light source color distribution characteristic of the outdoor light source.

  In S32, an AWB correction amount LW_AWB for the R, G, and B signals after the correction in S32 is calculated, and an AWB gain is calculated based on LW_AWB. This is the same as in Patent Document 6.

  In S33, as in S31, the color information calculation unit 51 calculates the coordinates Ci of the R / GB-G / G plane for each divided area Xi. The color information correction unit 52 reads a locus L2 that includes a color range that can be used as an artificial light source other than an LED and is stored in the locus holding unit 53, and a normal line extending from the coordinate Ci to the locus L2 for each divided area Xi. An intersection point Qi with the locus L2 is calculated. A possible range as an artificial light source other than the LED is a range sandwiched between the locus L1 and the locus L2. The color information correction unit 52 performs correction for bringing the coordinate Ci close to (matches) Qi for each divided area Xi. That is, the object color is excluded from the image data based on the light source color distribution characteristic of the artificial light source other than the LED.

  In S34, an AWB correction amount LW_AWB for the R, G, B signals after the correction in S33 is calculated, and an AWB gain is calculated based on the AWB correction amount LW_AWB. This is the same as in Patent Document 6.

  In S35, as in S31, the color information calculation unit 51 calculates the coordinates Ci of the R / GB-G / G plane for each divided area Xi.

  The color information correction unit 52 reads the trajectory L3 corresponding to the LED light source stored in the trajectory holding unit 53, and determines the intersection Ri between the normal line extending from the coordinate Ci to the trajectory L3 and the trajectory L3 for each divided area Xi. calculate. The calculation method of the intersection is performed in the same manner as in Patent Document 5. The color information correction unit 52 performs correction for bringing the coordinates Ci closer to (matching) Ri for each divided area Xi. That is, the object color is excluded from the image data based on the light source color distribution characteristic of the LED light source.

  In S36, an AWB correction amount LW_AWB for the R, G, and B signals after the correction in S35 is calculated, and an AWB gain is calculated based on the AWB correction amount LW_AWB. The calculation method of the AWB correction amount LW_AWB is the same as in the second embodiment. For example, the AWB correction amount that makes the red (R), green (G), and blue (B) color signal levels of the representative values of the coordinates Ci after approaching the locus L3 equal is LW_AWB.

  The digital signal processing unit 3 corrects the white balance of the image by multiplying the RGB signal by the AWB gain calculated in S32, S34, or S36.

  FIG. 9A shows an example of the intersection P'i corresponding to the black body locus L1 and the light source detection values Ci and Ci of the divided areas. FIG. 9B shows an example of the intersection point L ′ corresponding to the locus L2 and the light source detection values Ci and Ci of the divided areas. FIG. 9C shows an example of the intersection R′i corresponding to the locus L3 and the light source detection values Ci and Ci of the divided areas.

  The black body locus L1 is a locus when gray is photographed under a solar light source (high color temperature to low color temperature). The integrated value deviating from the black body locus under the solar light source is affected by the object color. In FIG. 9A, C3 distributed above the blackbody locus L1 and C2 and C4 distributed below are respectively set to coordinates P′3, P′2, and P′4 that approach the blackbody locus L1. It has been corrected.

  The artificial light source locus L2 other than the LED is a locus including a color range that can be an artificial light source other than the LED. A portion between the black body locus and the artificial light source locus L2 is a possible range of the light source color. On the R / GB / G plane, the integrated value distributed below the artificial light source locus L2 is affected by the object color. In FIG. 9B, C3 distributed above the locus L2 and C4 distributed below the locus L2 are corrected to coordinates Q′3 and Q′4 approaching the locus L2, respectively.

  The LED trajectory L3 is a trajectory when gray is photographed under an LED light source (color can be adjusted so that the color temperature is high to low). As shown in FIG. 9C, the black body locus L1 is different from the artificial light source locus L2 other than the LED. Under the LED light source, the integrated value deviating from this locus is influenced by the object color. In FIG. 9C, C1 and C3 distributed above the trajectory L3 and C4 distributed below the trajectory L3 are corrected to coordinates R′1, R′3 and R′4 approaching the trajectory L3, respectively. . In addition, the moving direction for approaching the locus | trajectories L1-L3 is not restricted to a normal line direction, Other directions may be sufficient.

  If it is known that the scene is under the white LED light source, the light source color is always on the white LED locus. If the light source color is not on the white LED locus, it is affected by the object color, so that the object color is corrected so as to approach the white LED locus. Thereby, the light source color residue of the white LED scene can be improved without causing any adverse effect on the AWB correction of the outdoor light source scene or the conventional artificial light source scene. Furthermore, it is possible to suppress AWB color feria due to the object color in a scene under a white LED light source.

2: imaging unit, 3: digital signal processing unit, 4: AE / AWB correction unit, 13: CPU, 40: signal ratio calculation unit, 41: AWB correction gain calculation unit, 42: LED determination unit, 43: subject luminance value Calculation unit 44: Conventional artificial light source / outdoor light source determination unit 45: Correlated color temperature calculation unit 46: Conventional light source determination threshold calculation unit 47: AWB correction amount calculation unit 50: Signal division unit 51: Color information calculation , 52: Color information correction unit, 53: Trajectory holding unit

Claims (12)

A signal amount acquisition unit that acquires a signal amount of an imaging signal corresponding to light having a wavelength near 480 nm and a signal amount of an imaging signal corresponding to light having a wavelength near 450 nm, based on an imaging signal output from the solid-state imaging device; ,
A signal ratio calculation unit that calculates a ratio of a signal amount of an imaging signal corresponding to light having a wavelength near 480 nm acquired by the signal amount acquisition unit and a signal amount of imaging signal corresponding to light having a wavelength near 450 nm;
A white LED light source determination unit that determines whether the light source is a white LED according to a magnitude relationship between the ratio calculated by the signal ratio calculation unit and a predetermined threshold;
A light source determination device comprising:   When the white LED light source determination unit determines that the light source is a white LED, it sets a white balance correction amount suitable for the white LED, and corrects the white balance of the imaging signal according to the set white balance correction amount. The light source determination device according to claim 1, further comprising an AWB correction unit that performs the operation.   The AWB correction unit, when the white LED light source determination unit determines that the light source is a white LED, representative colors of red (R), green (G), and blue (B) estimated from the imaging signal The light source determination device according to claim 2, wherein a normal white balance correction amount is set such that the signal levels are equal. A subject luminance calculation unit for calculating subject luminance based on an imaging signal output from the solid-state imaging device;
An evaluation value calculation unit that calculates an evaluation value indicating the probability of the light source based on a membership function with the subject luminance calculated by the subject luminance calculation unit as a variable;
The light source determination device according to claim 2 or 3. When the white LED light source determination unit determines that the light source is not a white LED, a conventional artificial light source determination unit that determines whether the light source is an artificial light source other than a white LED,
When the conventional artificial light source determination unit determines that the light source is an artificial light source other than a white LED, the AWB correction unit determines a white balance correction amount according to the evaluation value calculated by the evaluation value calculation unit. The light source determination device according to claim 4, wherein the light source determination device is set to be smaller than or equal to the balance correction amount. When the white LED light source determination unit determines that the light source is not a white LED, the white LED light source determination unit includes a natural light source determination unit that determines whether the light source is a natural light source,
When the natural light source determination unit determines that the light source is a natural light source, the AWB correction unit sets the white balance correction amount to be greater than the normal white balance correction amount according to the evaluation value calculated by the evaluation value calculation unit. The light source determination device according to claim 4 or 5, wherein the light source determination device is set to be small. A trajectory storage unit that stores a trajectory indicating the distribution of the color temperature of the white LED light source in a predetermined color space;
A signal dividing unit that divides an image formed of the imaging signals into a plurality of areas;
A divided area color information acquisition unit that acquires color information of each area divided by the signal division unit;
A color information correction unit that corrects the color information of each area acquired by the divided area color information acquisition unit based on the trajectory stored in the trajectory storage unit;
With
When the white LED light source determination unit determines that the light source is a white LED, the AWB correction unit determines a white color suitable for the white LED based on the imaging signal after the color information correction unit corrects the color information. The light source determination device according to claim 2, wherein a balance gain is set, and the color information correction unit corrects a white balance of the imaging signal after the color information is corrected.   The light source determination device according to claim 7, wherein the color information correction unit corrects the color information of each area so as to approach the locus stored in the locus storage unit.   The signal amount acquisition unit picks up an image corresponding to light having a wavelength near 480 nm from an image pickup signal obtained from a specific pixel corresponding to a cyan color filter in a pixel group of the solid-state image pickup element corresponding to a predetermined color filter. The signal amount of the signal is acquired, and the signal amount of the imaging signal corresponding to light having a wavelength near 450 nm is acquired from the imaging signal obtained from the specific pixel corresponding to the blue filter. The light source determination device according to item.   The signal ratio calculation unit calculates a ratio between an average signal amount of an imaging signal corresponding to light having a wavelength near 480 nm acquired by the signal amount acquisition unit and an average signal amount of imaging signal corresponding to light having a wavelength near 450 nm. The light source determination apparatus according to any one of claims 1 to 9.   The signal ratio calculation unit calculates a ratio between an integrated signal amount of an imaging signal corresponding to light having a wavelength near 480 nm acquired by the signal amount acquisition unit and an integrated signal amount of an imaging signal corresponding to light having a wavelength near 450 nm. The light source determination apparatus according to any one of claims 1 to 9. The light source determination device
Obtaining a signal amount of an imaging signal corresponding to light having a wavelength near 480 nm and a signal amount of an imaging signal corresponding to light having a wavelength near 450 nm based on an imaging signal output from the solid-state imaging device;
Calculating a ratio of a signal amount of an imaging signal corresponding to the acquired light having a wavelength near 480 nm and a signal amount of an imaging signal corresponding to the light having a wavelength near 450 nm;
Determining whether the light source is a white LED according to the magnitude relationship between the calculated ratio and a predetermined threshold;
A light source determination method for executing

Images