JP2006229425A - Imaging apparatus and portable telephone with it - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide an imaging apparatus correcting a color shading with a high accuracy even when an illumination is changed. <P>SOLUTION: For eliminate unnecessary infrared rays, the imaging apparatus has a correction-factor arithmetic means for computing a correction factor for correcting the approximate color shading by a nonlinear curve function towards an end from the center (an optical axis) of an image sensor as an imaging surface. For that, the image sensing device further has an integrating means for integrating at least a screen one-surface or more of the color signals of a red (R), a green (G) and a blue (B) obtained by the image sensor and obtaining integrated values ΣR, ΣG and ΣB, a correction-factor adjusting means for computing an adjusting factor for adjusting the correction factor from the ratios of the integrated values ΣR, ΣG and ΣB acquired from the integrating means, and a shading correction means having the correction-factor arithmetic means. The shading correction means corrects the color shading at a value changed in response to the adjusting factor acquired from the correction-factor adjusting means. <P>COPYRIGHT: (C)2006,JPO&NCIPI

Description

  The present invention relates to an image pickup apparatus that corrects color shading (color unevenness) and shading (color unevenness) that are different in color and brightness between the center and the periphery of a screen in the image pickup apparatus, and more particularly, to an image pickup apparatus having a small optical system. The present invention also relates to a mobile phone provided with an imaging device.

  In recent years, image sensors, which are solid-state imaging devices such as CCDs, are widely used in digital cameras, video cameras, and the like. The light incident on the central portion of the solid-state image sensor is mainly vertically incident light, whereas the light incident on the periphery of the solid-state image sensor at a shallow angle increases. For this reason, the light intensity received by the pixels differs between the central portion and the peripheral portion of the image sensor, and shading occurs. Furthermore, since the refractive index of the optical system has wavelength dependency, color shading also occurs at the same time. Therefore, in order to suppress shading and color shading, the arrangement of microlenses (hereinafter referred to as on-chip microlenses) formed for the purpose of collecting light for each pixel configured on the image sensor can be adjusted, There is known a method in which a different correction coefficient is set in advance for each position and color signal, and correction is performed using the correction coefficient.

  For example, in the conventional imaging device disclosed in Patent Document 1, as the distance from the center of the image sensor is increased, the pitch of the on-chip microlens is arranged so as to be gradually smaller than the pixel pitch. The incident light incident obliquely is collected efficiently and shading is suppressed.

  In another conventional imaging apparatus disclosed in Patent Document 2, a correction coefficient corresponding to the coordinates (X, Y) of the image sensor is expressed as an N-order surface function of coordinates (X, Y) (N is 2 or more). Therefore, shading and color shading correction are performed by approximating each color signal and multiplying this correction coefficient by each color signal of the pixel corresponding to the coordinates (X, Y).

Special No. 2600250 (2nd page, Fig. 1) JP-A-8-79773 (page 3, FIG. 2)

  In the conventional imaging apparatus, shading correction is performed by adjusting the arrangement of on-chip microlenses. However, in a small imaging device provided in a mobile device such as a mobile phone, it is necessary to dispose the optical system and the image sensor at a short distance, so the incident angle of the light beam is smaller than that of a conventional digital camera or video camera. And increases from 20 degrees to 25 degrees or more (the angle at normal incidence is 0 degree). For this reason, there is a problem that shading cannot be sufficiently suppressed only by adjusting the arrangement of the on-chip microlenses. Furthermore, since the refractive index of the on-chip microlens has wavelength dependency, color shading occurs simultaneously with shading, and there is a problem that color shading becomes remarkable especially in the small-sized imaging device described above.

In another conventional imaging apparatus described above, a function form (each coefficient of an Nth-order curved surface function with respect to coordinates on the image sensor) that gives a correction coefficient that enables optimum color shading correction under a specific illumination has been determined in advance. Thereby, although color shading can be corrected under this specific illumination, there is a problem that sufficient color shading correction cannot be performed under different illumination.

Further, in order to save space in a small imaging device, an evaporation type infrared cut filter having a multilayer structure in which thin film layers are stacked is often used. This filter is a multilayer film structure in which two or more kinds of thin film layers having different refractive indexes are laminated by electron beam vapor deposition or sputtering vapor deposition, and has transmittance only in the visible light region due to light interference. It is. However, since the interference effect differs depending on the incident angle of light, for example, the cutoff wavelength has an incident angle dependency. Since this dependency promotes color shading, it is particularly difficult to perform proper color shading correction in an imaging apparatus using an evaporation type infrared cut filter.

  An image pickup apparatus according to the present invention is arranged on an image pickup surface, and is arranged in front of an image pickup element composed of a plurality of pixels, and transmits red (R), green (G), and blue (B), respectively. An image pickup apparatus having a filter, the correction coefficient calculating means for calculating a correction coefficient for each signal corresponding to the R, G, and B from the pixel position, and R obtained from the plurality of pixels of the image pickup device, A signal corresponding to each of G and B is accumulated over at least one surface of the screen to obtain integrated values ΣR, ΣG and ΣB, and a ratio of at least two of the integrated values among the integrated values ΣR, ΣG and ΣB. Correction coefficient adjusting means for calculating an adjustment coefficient for adjusting the correction coefficient, and adjusting the correction coefficient according to the adjustment coefficient, and performing shading correction for each signal corresponding to R, G, and B. Those having a shading correction unit.

  The imaging apparatus according to the present invention can satisfactorily correct color shading that differs for each illumination light.

Embodiment 1 FIG.
FIG. 1 is a block diagram showing a configuration of an imaging apparatus according to Embodiment 1 of the present invention. The optical system 1 includes a lens 1a and an infrared removing filter (hereinafter referred to as IRCF (InfraRed Cut Filter)) 1b, and a diaphragm 1c shown in FIG. 2 is provided as necessary. Light incident from the optical system 1 forms an image on the image sensor 2 in which color filters having spectral sensitivity characteristics corresponding to the three primary colors of red (R), green (G), and blue (B) are arranged. The infrared cut filter 1b is an optical component for removing unnecessary infrared light from the light incident from the lens 1a, and transmits only light having a wavelength necessary for imaging. In the image sensor 2, a signal for R (R signal) is arranged from a pixel in which an R color filter is arranged, a signal (G signal) for G is arranged from a pixel in which a G color filter is arranged, and a B color filter is arranged. A pixel signal (B signal) is output from each pixel.

  The image sensor 2 photoelectrically converts incident light and outputs an analog signal at a level corresponding to the amount of incident light for each pixel. The analog signal is subjected to necessary analog signal processing in the analog signal processing means 3. Here, the analog signal processing means correlated double sampling for removing noise from the output value of the image sensor 2 and signal amplification by an automatic gain control circuit. The analog signal processing means 3 includes these signal processing means. It is equipped. Next, each pixel is converted into a digital signal by the A / D converter 4 and sent to the digital signal processing means 5.

  The digital signal processing means 5 performs a clamp process for adjusting the black level of the output signal of the image sensor 2 and a pixel interpolation process for generating a signal corresponding to a color different from the filter color of each pixel by interpolation. The R signal, G signal, and B signal, which are color signals output from the digital signal processing means 5, are input to the corresponding shading correction means 6a, 6b, and 6c, respectively, and the shading of the respective color signals is corrected. That is, the shading correction unit 6a corrects the shading of the R signal, the shading correction unit 6b corrects the shading of the G signal, and the shading correction unit 6c corrects the shading of the B signal. A G ′ signal and a B ′ signal are output. The operation related to this correction will be described in detail later.

  The R ′ signal, the G ′ signal, and the B ′ signal, which are signals subjected to shading correction, are respectively subjected to gain adjustment for white balance by the corresponding white balance means 7a, 7b, and 7c. Tone conversion is performed by the corresponding gamma means 8a, 8b and 8c, respectively. Finally, RGB / YUV conversion means 11 converts the signals into Y, U and V signals and outputs them.

  On the other hand, the integrating means 9 integrates at least one screen (frame) each of the R signal, G signal and B signal output from the digital signal processing means 5 and supplies the integrated values ΣR, ΣG and ΣB to the correction coefficient adjusting means 10. Output. The correction coefficient adjusting means 10 outputs three adjustment coefficients corresponding to the respective color signals obtained by calculation from the above three integrated values to the corresponding shading correction means 6a, 6b and 6c. Each shading correction means 6a, 6b, 6c performs appropriate shading correction using the adjustment coefficient or the like. The shading and color shading correction will be specifically described below.

  FIG. 2 shows light rays incident from the optical system 1 and imaged on the image sensor 2. In FIG. 2, the optical aperture position of the lens 1a is shown as an aperture 1c. The light beam incident through the stop 1c forms an image on the image sensor 2 by the lens 1a. The incident angle of the light beam on the image sensor 2 is shallower than vertical at the end of the image sensor 2. Therefore, the incident light quantity of the received light amount of the pixel at the end is smaller than that in the center of the image sensor 2. This is generally called shading. In particular, in the case of a small imaging device, the distance between the diaphragm 1c and the image sensor 2 is shortened, so that the incident angle at the end of the image sensor 2 is further shallow, and shading becomes remarkable. The image sensor 2 is provided with a photodiode for photoelectric conversion (not shown) for each pixel. Due to this shading, the output signal level varies depending on the position of the image sensor.

  In addition, since the refractive index of the lens has a wavelength dependency, a phenomenon in which the sensitivity differs for each wavelength occurs particularly near the end of the image sensor 2. This is generally called color shading. The positional relationship between the lens 1a and the diaphragm 1c shown in FIG. 2 is an example, and the diaphragm 1c may be located closer to the image sensor 2 than the lens 1a. In this case, the incident angle is further increased, and the color Shading becomes noticeable.

FIG. 3 shows the R signal, G signal, and B signal on the image sensor 2 with respect to the image height indicating the distance from the center (the center of the image sensor 2 is 0% and the diagonal end point is 100%). Indicates the signal level. As shown in FIG. 3, normally, as the image height increases, R light on the long wavelength side deviates from the photodiode, and the R signal is lowered (attenuated) compared to other color signals, resulting in color shading. . In the conventional technique, the on-chip microlens position for efficiently condensing the image sensor 2 on the photodiode for each pixel is shifted according to the image height (generally referred to as scaling). There is a limit to the incident angle, and there has been a problem that a small imaging device mounted on a mobile phone or the like cannot completely improve shading and color shading only by scaling a microlens.

In addition to the above factors, there are other factors that cause color shading. The image sensor 2 has the spectral sensitivity characteristics shown in FIG. 4, and is also sensitive to an infrared region (a wavelength region invisible to human eyes) that is not necessary for imaging. For this reason, the normal imaging apparatus includes an IRCF 1b for removing unnecessary infrared rays. In a small imaging device, the IRCF 1b is of a multi-layered film type (deposited layers of dozens of thin films having different refractive indexes are deposited on a thin glass plate by vapor deposition or the like in order to make the optical system small. In many cases, IRCF (having a spectral transmittance that removes infrared rays by using interference of light transmitted and reflected) is used.

  FIG. 5 shows the spectral characteristics of the deposited multilayer film type IRCF1b. The transmittance according to the wavelength is shown. In the figure, 0 degrees is the target spectral characteristic when IRCF1b is designed, the half value (transmittance is 50%) is approximately 670 nm, and the wavelength of light exceeding 700 nm that is not visible to human eyes is almost Unnecessary infrared light is removed by not transmitting. However, since the optical path length when passing through the thin film varies depending on the incident angle of light, the spectral characteristics differ depending on the incident angle. FIG. 5 shows spectral characteristics when the incident angles are 0 degrees, 15 degrees, 20 degrees, and 25 degrees. As the incident angle increases, the transmittance on the long wavelength side decreases due to the wavelength-dependent effect. For this reason, the signal level of the R signal further decreases as the image height increases.

  Thus, as the incident angle increases, color shading becomes apparent due to the wavelength dependency of the refractive index of the lens 1a and the wavelength dependency of the transmittance of the IRCF 1b. This color shading can be corrected by optimizing the correction coefficient in advance when the illumination is specified, but if the illumination is different from the optimized illumination, the correction will be inappropriate. Color shading occurs.

  FIG. 6 shows spectral characteristics of black body radiation of 3000 K (Kelvin) and 5500 K, which are almost the same color rendering as sunlight, and spectral characteristics of daylight white fluorescent lamps often used as indoor lighting. Each illumination has its spectral characteristics greatly different. The shading amount for each color signal in each illumination is shown in FIGS. 7 and 8 show color shading when black body radiation having a color temperature of 3000K and 5500K is used as illumination, respectively, and FIG. 9 shows color shading when a daylight fluorescent lamp is used as illumination. Yes. The optical system used here is a normal optical system in a camera for a mobile phone, and as the IRCF 1b, a deposited multilayer film type filter having the characteristics shown in FIG. 5 is used. 7 to 9, the horizontal axis represents the incident angle of the light beam. 0 degrees is the optical axis and corresponds to an image height of 0%. In a small imaging device provided in a mobile phone or the like, the incident angle at the diagonal end point of the image sensor 2 is about 25 degrees, and therefore, for example, an incident angle of 25 degrees corresponds to an image height of 100%. In each figure, normalization is performed so that R, G, and B have the same signal level when the incident angle is 0 degrees. Whichever illumination is used, the R signal decreases as the incident angle increases. However, it can be seen that when the daylight white fluorescent lamp in FIG. 9 is used for illumination, the decrease in the R signal is much slower than when other illuminations are used.

  As shown in FIGS. 7 to 9, since the degree of color shading varies greatly depending on illumination light, when color shading correction is performed based on a fixed correction amount as in the prior art, for example, under sunlight When color shading correction is performed on the basis of color shading, there is a problem that the correction amount of the R signal is overcorrected under a fluorescent lamp and the periphery of the image becomes red. Conversely, when the correction amount is set based on the color shading amount under fluorescent light, the correction amount is not sufficient under sunlight, and green and cyan are excessively emphasized in the surrounding area, or white balance is However, in order to whiten excessive green in the periphery, there was a problem that magenta became excessively strong in the center. The operation of the present invention for solving these problems will be described below.

  The shading correction means 6a, 6b, and 6c correct the R signal, the G signal, and the B signal that are color signals, respectively, and output the R ′ signal, the G ′ signal, and the B ′ signal that are corrected color signals. Since the configuration and operation of the shading correction means 6a, 6b, and 6c are all the same, here, the correction means 6a for the R signal will be described as a representative. FIG. 10 is a diagram showing a specific configuration of the shading correction means 6a. The coordinate value generating means 6g is a means for outputting a coordinate value indicating which position in the horizontal or vertical direction the input R signal corresponds to on the image sensor 2. This coordinate value generating means 6g generates a horizontal coordinate value and a vertical coordinate value from the pixel clock CLK, the horizontal synchronization signal HD, and the vertical synchronization signal VD. First, the horizontal coordinate position is given by counting the number of clocks of CLK. This count is reset by HD. The vertical coordinate position is given by counting the number of HDs. This count is reset by VD. Further, a value obtained by subtracting one-half pixel as an offset value from the horizontal coordinate position and the vertical coordinate position is output as coordinates (x, y) on the image sensor 2. In this case, the origin of coordinates is, for example, the upper left corner of the image sensor 2.

  The correction coefficient calculation means 6f is a calculation means for determining a correction coefficient fr (x, y) for correcting color shading based on the coordinates (x, y) obtained by the coordinate value generation means 6g. Specifically, the coordinate value (x, y) output from the coordinate value generating means 6g is input, and the value fr = fr (x, y) shown in Equation 1 is calculated.

In Equation 1, a01, a02, a10, a11, a12, a20, a21, and a22 are predetermined constants so that the R signal of the blackbody radiation 5500K shown in FIG. 8 can be corrected appropriately. It has been established. That is, each constant is optimized so that the relative value of the R signal in FIG. 8 is corrected to almost 1 regardless of the incident angle.

  In the above description, the processing is performed with the upper left corner of the image sensor 2 as the origin, but it is also effective to give the coordinate value (x, y) with the center of the image sensor 2 as the origin. In this case, the image height D is given by the length of the vector (x, y). Since the shading amount is determined by the image height D, the correction coefficient fr is given by a function fr (D) of the image height D.

In Expression 2, b01 and b02 are predetermined constants that are determined so that the R signal of the blackbody radiation 5500K shown in FIG. 8 can be corrected appropriately. Thus, the calculation can be simplified by giving the coordinate value (x, y) with the center of the image sensor 2 as the origin.

Although the R signal has been described so far, the correction of the G signal and the B signal is the same as described above, and the correction coefficients fg and fb are obtained using the shading correction means 6b and 6c, respectively. However, as each constant in Expression 1 or Expression 2, another constant optimized for each of the G signal and the B signal is used, and the relative value of each signal in FIG. Corrected to be almost 1

Here, the constants are determined so as to optimize the case where the black body radiation 5500K is used as illumination. The reason for this is that the black body radiation 5500K is almost equal to the daylight of sunlight most often used in the imaging apparatus. This is because this is used as the reference illumination in order to respond. However, as will be described later, in the invention according to the present embodiment, since the correction means for the case where illumination different from the reference illumination is used, it is possible to optimize each constant for other illumination such as a fluorescent lamp. A substantially similar effect can be obtained. Expressions 1 and 2 given as correction expressions are examples, and for example, correction with higher accuracy can be performed by adding higher-order terms. An appropriate order and function type may be selected according to required characteristics.

  As described above, the correction coefficient calculator 6f calculates a correction coefficient for correcting shading of each color signal on the image sensor 2 when, for example, the black body radiation 5500K is used as the reference illumination. That is, a correction coefficient for correcting the relative value of each color signal in FIG. 8 so as not to depend on the incident angle is given. However, when the illuminations are different, the constants given by Equation 1 or Equation 2 are not appropriate for the illumination, so that optimal correction cannot be performed. Therefore, the correction coefficient adjusting means 10 described later calculates adjustment coefficients Cr, Cg, Cb for each color signal when illumination different from the reference illumination is used, and a correction coefficient fr, Multiplication with fg and fb is performed, and the multiplication value fr × Cr and the like are output to the adder 6h. The constant a00 added to the multiplication value by the adder 6h is the gain of the entire signal. If a00 = 1, the input signal is output at the same signal level when the shading correction coefficient is 0. Now, FIG. 11 shows an example of the correction coefficient for each color signal when a00 = 1. As shown in FIG. 8, the G signal and the B signal do not depend greatly on the incident angle, and show a relative value of approximately 1. Therefore, the correction coefficients fg and fb may be zero without correction. It becomes. Further, since the R signal attenuates as the incident angle increases, the correction coefficient fr is provided so that the value increases from 0 according to the coordinate position so that the signal is amplified as the image height increases. It has been.

The correction coefficient adjusting means 10 adjusts the adjustment coefficients Cr, Cg, and Cb for adjusting the correction coefficients fr, fg, and fb calculated by the correction coefficient calculating means 6f, when illumination different from the reference illumination is used. calculate. As shown in FIG. 12, the correction coefficient adjusting means 10 includes a dividing means 10a and an adjustment coefficient calculating means 10b. The dividing unit 10a obtains two ratios from the three values of the integrated values ΣR, ΣG, and ΣB input from the integrating unit 9. Here, a case will be described where values ΣR / ΣG and ΣB / ΣG are calculated by respectively dividing ΣR and ΣG by ΣB, but other ratios may be used. Then, the adjustment coefficient calculation unit 10b calculates the adjustment coefficients Cr, Cg, and Cb from the two ratios of the integrated values.
A method for calculating the adjustment coefficients Cr, Cg, and Cb by the adjustment coefficient calculation unit 10b will be specifically described with reference to FIG. In FIG. 13, the ratio of two integrated values (ΣR / ΣG, ΣB / ΣG) obtained by the dividing means 10a is plotted as a black dot on a two-dimensional plane, and on the other hand, it is obtained under sunlight with different color temperatures ( The locus of ΣR / ΣG, ΣB / ΣG) is indicated by a solid line. The distance H between the curve indicated by the solid line and the black point is a two-dimensional difference between the ratio of the two integrated values obtained and the ratio of the two integrated values that would be obtained under sunlight illumination with the same correlated color temperature. The adjustment coefficients Cr, Cg and Cb are calculated according to the value of H. When the image is taken under illumination (such as a light bulb) according to black body radiation or sunlight, the two ratios (ΣR / ΣG, ΣB / ΣG) of the integrated value of the color signal of the achromatic subject are It becomes such a value. Also, if the entire image is integrated, an achromatic image is obtained (Evans' theorem). Normally, the auto white balance and the like adjust the gain of each color signal so that the integrated value of the entire image becomes white according to this theorem. Therefore, if the value of the integrated value of the entire image is calculated, it is possible to obtain a value equivalent to the integrated value of the color signal when imaging with an achromatic subject under the illumination.

  On the other hand, artificial light such as a fluorescent lamp has poor color rendering properties compared to sunlight. For example, two ratios (ΣR / ΣG, ΣB / ΣG) of integrated values of color signals obtained by subject imaging under a daylight fluorescent lamp are: As indicated by the black dots in FIG. 13, the value is at a position away from the curve. If the color rendering properties of the illumination are further deteriorated, in FIG. 13, the black spot is usually further away from the curve to the lower left side. From this distance H, the correction coefficient adjusting means 10 calculates adjustment coefficients Cr, Cg and Cb. An example of the relationship between the distance H and the adjustment coefficients Cr, Cg, and Cb is shown in FIG. When the value of the distance H is approximately 0, it is determined that the illumination is the same as the reference illumination, and the value of each adjustment coefficient is 1. As the distance H increases, the value of each adjustment coefficient decreases from 1. Note that the relationship between the adjustment coefficients Cr, Cg, and Cb and the distance H is not the same, and is different for each adjustment coefficient. For example, in FIG. 14, the value of each adjustment coefficient when H = 0 is 1, but the slope of each adjustment coefficient when H increases is different. For example, comparing FIG. 8 and FIG. 9, the G signal and the B signal do not show a large change in the incident angle dependency even if the illumination changes. Therefore, regardless of the distance H, even if Cg = Cb = 1, Generally a good approximation. On the other hand, with respect to the R signal, the incident angle dependency differs greatly between sunlight and a fluorescent lamp, so Cr needs to be determined so as to decrease as the distance H increases as shown in FIG.

  Another example of the adjustment coefficient calculation method in the adjustment coefficient calculation means 10b is shown in FIG. The adjustment coefficient values (Cr, Cg, Cb) for each color signal corresponding to the ratio of two integrated values (ΣR / ΣG, ΣB / ΣG) obtained in the dividing means 10a are determined in advance as shown in FIG. By providing an LUT (look-up table) in which it is stored in a two-dimensional memory, the value of (Cr, Cg, Cb) can be calculated with respect to the ratio of the two obtained integrated values. The color signal integration ratio ΣR / ΣG is input as address A and ΣB / ΣG is input as address B, and adjustment coefficients (Cr, Cg, Cb) provided according to the distance H are stored in the memory. , The adjustment coefficients (Cr, Cg, Cb) corresponding to the addresses A and B are output. The adjustment coefficient for the R signal correction coefficient fr is Cr, the adjustment coefficient for the G signal correction coefficient fg is Cg, and the adjustment coefficient for the B signal correction coefficient fb is Cb. By providing the LUT as described above, the adjustment coefficient can be determined without performing an operation such as calculating the distance H.

  The memory is largely divided into regions (in the example shown in FIG. 15, regions A to D), and these regions are determined according to the distance H from the sunlight curve. In each area, a correction adjustment coefficient for each color signal is recorded as (Cr, Cg, Cb). The area A is regarded as light under sunlight, and the adjustment coefficients Cr, Cg, and Cb for correcting each color signal are all 1. Since the distance H increases from the region B to the region C and the region D, the adjustment coefficient is recorded as a table value so that it gradually becomes smaller or equal. For example, in region B, (Cr, Cg, Cb) = (a, b, c), in region C (Cr, Cg, Cb) = (d, e, f), in region D (Cr, Cg, Cb) ) = (G, h, i), 1 ≧ a ≧ d ≧ g, 1 ≧ b ≧ e ≧ h, 1 ≧ c ≧ f ≧ i. Thus, the correction coefficient adjusting means 10 calculates the adjustment coefficients Cr, Cg, Cb for each color signal from the values obtained from the integrating means 9, and outputs them to the shading correction means 6a, 6b and 6c, respectively.

  In the shading correction means 6a, 6b and 6c, the adjustment coefficients Cr, Cg and Cb inputted to the shading correction means 6a, 6b and 6c are respectively multiplied by the correction coefficients fr, fg and fb outputted from the correction coefficient calculation means 6f by the multiplier 6e and added. After a00 (= 1) is added to each in the unit 6f, the multiplier 6d multiplies the R signal, the G signal, and the B signal, which are input signals, in the multiplier 6d for correction. Here, in the adjustment coefficient shown in FIG. 15, a = 0.5, d = 0.4, g = 0.2 for Cr, b = e = h = 1.0 for Cg, and Cb , C = f = i = 1.0. That is, only the R signal is subjected to shading correction. As described above, the correction coefficient calculation means 6f has its correction coefficients fr, fg, and fb determined as shown in FIG. 11 so as to eliminate color shading with sunlight 5500K. For example, in the case of a daylight fluorescent lamp, the correction adjustment coefficients Cr, Cg, and Cb are values of 1 or less, and thus the correction coefficient f is adjusted in a decreasing direction. When the lunch white fluorescent lamp is located in the region B, (Cr, Cg, Cb) = (0.5, 1.0, 1.0). Therefore, since the correction degree of the R signal is half that shown in FIG. 11, even when the daylight fluorescent lamp has a small attenuation of the R signal relative to the incident angle as shown in FIG. The correction can be performed appropriately. Further, since the G signal and the B signal are almost the same value as sunlight even in the case of a daylight fluorescent lamp, the adjustment coefficients Cg and Cb are set to 1. In this way, appropriate color shading correction can always be performed regardless of illumination by the adjustment coefficient output from the correction coefficient adjusting means 10.

  Still another example of the adjustment coefficient calculation means by the correction coefficient adjustment means 10 will be described. The correction coefficient adjusting unit 10 includes a calculating unit, and the calculating unit has an Nth-order near-order expression that approximates the curve of sunlight shown in FIG. 16, or a large amount of artificial light such as a fluorescent lamp. Having an approximate straight line B = −jA + k (where j and k are constants, B is ΣB / ΣG, and A is ΣR / ΣG) in the vicinity of the correlated color temperature (approximately 4000 K to 5500 K). It is also possible to calculate the distance H from the measured value (black dot), and calculate and output each of Cr, Cg, and Cb corresponding to the distance H as shown in FIG.

Embodiment 2. FIG.
FIG. 17 is a block diagram showing a configuration of an imaging apparatus according to Embodiment 2 of the present invention. The process until the signal output from the image sensor 2 is output from the digital signal processing means 5 is the same as in the first embodiment.

  The R signal, the G signal, and the B signal output from the digital signal processing means 5 are respectively integrated over at least one screen in the integrating means 9, and integrated values ΣR, ΣG, and ΣB are obtained. These are output to the correction coefficient adjusting means 20 and the exposure control means 12. The exposure control means 12 is a means for controlling the exposure value of the image pickup apparatus. When the brightness of the image obtained from the integrated value is darker than the target brightness, the timing is set so as to increase the exposure value. A control signal is output to a generator (hereinafter referred to as TG) 13. The TG 13 is a means for generating a drive pulse for driving the image sensor 2. The exposure time of the image sensor 2 is lengthened when the exposure value is increased, and the image sensor 2 is decreased when the exposure value is decreased. A drive pulse is output so that the exposure time is shortened. If the value obtained from the integrating means 9 is small even if a driving pulse that maximizes the exposure time is output by the TG 13, the exposure control means 12 is an automatic gain control means (not shown) provided in the analog signal processing means 3. Control signal to increase the signal gain. As described above, the exposure control means 12 controls the exposure by outputting control signals to the TG 13 and the analog signal processing means 3.

The exposure control means 12 outputs a control signal for performing the exposure control 12, and simultaneously records the accumulation time S of the image sensor 2 and the signal gain G of the analog signal processing means 3, and the accumulation time S and the signal gain G are output to the correction coefficient adjusting means 20.

  As shown in FIG. 18, the correction coefficient adjusting unit 20 includes a luminance signal generating unit 20a that generates an integrated value ΣY of the luminance signal from the integrated value obtained from the integrating unit 9, and a normalizing unit 20b that normalizes the luminance signal ΣY. And a comparator 20c and adjustment coefficient calculation means 20d. The luminance signal generation unit 20a calculates an integrated value ΣY of the luminance signal from the integrated value of the input color signal. For example, the luminance signal is calculated by ΣY = ΣR + 2 · ΣG + ΣB. The normalizing means 20b divides ΣY output from the luminance signal generating means 20a by the exposure time S output from the exposure control means 12 and the signal gain G to calculate ΣY / (S · G). Since the integrated value of the color signal obtained from the integrating means 9 is a signal value for which exposure control has already been performed, ΣY can be obtained as a substantially constant value for the approximate subject illuminance. Therefore, the brightness of the subject can be calculated by dividing the integrated value ΣY of the obtained luminance signal by the accumulation time S and the signal gain G, which are exposure conditions when the signal is obtained. Even when ΣY is constant, the brightness of the subject is getting darker as the accumulation time S and the signal gain G of the image sensor 2 are increased. ΣY / (S · G) output from the normalizing means 20b is input to the comparator 20c, compared with a predetermined value, and the result is output to the adjustment coefficient calculating means 20d.

In general, external light (sunlight) has a brightness of several thousand lx (lux) to several hundred thousand lx, and indoor light such as a fluorescent lamp has a brightness of about several tens of lx to 1000 lx, which is significantly higher than that of external light. Its brightness is low. Therefore, for example, a value corresponding to the brightness of the subject obtained when the brightness is approximately 1000 lx (for example, the brightness when the reflectance of the subject is assumed to be about 18%) is compared with the comparator 20c. As a result, when the value obtained from the normalizing means 20b is larger than the comparison value provided in the comparator 20c, it is outside light (sunlight), and when it is smaller, it is indoor light (fluorescent lamp). Judgment can be made.

  The adjustment coefficient calculation unit 20d outputs the adjustment coefficients Cr, Cg, and Cb as in the first embodiment. However, when the comparator 20c determines that the light is outside light (sunlight), all the values are 1. As a result, the correction coefficients fr, fg, and fb output from the correction coefficient calculation means 6f in the shading correction means 6a, 6b, and 6c are used as correction coefficients as they are. On the other hand, when the comparator 20c determines that the light is indoor light (fluorescent lamp), similarly to the first embodiment, Cr outputs a value smaller than 1 so that the correction degree of the R signal becomes small. Also in this case, Cr is smaller than 1, and the adjustment coefficients Cg and Cb are approximately the same value as 1 for B and G in which the amount of sunlight and shading do not change much. Thereby, appropriate color shading correction can be performed according to sunlight and a fluorescent lamp, and overcorrection or insufficient correction can be solved.

  Each color signal output from the shading correction means 6a, 6b and 6c is processed in the same manner as in the first embodiment.

Embodiment 3 FIG.
FIG. 19 is a block diagram showing a configuration of an imaging apparatus according to Embodiment 3 of the present invention. The process until the signal output from the image sensor 2 is output from the digital signal processing means 5 is the same as in the first and second embodiments.

  The accumulating unit 9 accumulates at least one R signal and B signal, which are color signals, and outputs the accumulated values ΣR and ΣB to the correction coefficient adjusting unit 30. As shown in FIG. 20, the correction coefficient adjusting means 30 includes a dividing means 30a for calculating ΣR / ΣB, and an adjustment coefficient calculating means 30b for calculating adjustment coefficients Cr, Cg and Cb from the output values ΣR / ΣB of the dividing means 30a. Has been. ΣR / ΣB obtained from the dividing means 30a is a value that changes in accordance with the color temperature of the illumination. ΣR / ΣB increases as the color temperature decreases, and ΣR / ΣB decreases as the color temperature increases. The correction coefficient adjusting means 30 makes the values of Cg and Cb smaller than 1 as shown in FIG. 21 as ΣR / ΣB increases, that is, as the color temperature decreases. Cr outputs a value of approximately 1. As a result, as shown in FIGS. 7 and 8, color shading can be appropriately corrected for different shading amounts depending on the color temperature.

In the above, the adjustment coefficient is obtained from the value of ΣR / ΣB, but the same effect can be obtained by using the value of ΣR / ΣG or ΣR / (ΣG + ΣB) instead of the value of ΣR / ΣB. Furthermore, by combining the means for obtaining the adjustment coefficient shown in the first or second embodiment and the means for obtaining the adjustment coefficient shown in the present embodiment, it is possible to perform better color shading correction.

It is a block diagram which shows the structure of the imaging device which concerns on Embodiment 1 of this invention. It is a figure which shows the incident angle of the light ray with respect to the image sensor which concerns on this invention. It is explanatory drawing of the color shading which showed the color signal level with respect to the image height based on this invention. It is a figure which shows the spectral sensitivity characteristic for every color of the image sensor which concerns on this invention. It is a figure which shows the spectral characteristic with respect to the incident angle of IRCF based on this invention. It is a figure which shows the spectral characteristic for every kind of illumination based on this invention. It is a figure which shows the relative value of each color signal level with respect to the incident angle in case the color temperature is 3000K based on this invention. It is a figure which shows the relative value of each color signal level with respect to an incident angle in case the color temperature which concerns on this invention is 5500K. It is a figure which shows the relative value of each color signal level with respect to the incident angle at the time of the daylight fluorescent lamp which concerns on this invention. It is a figure which shows the specific structure of the shading correction | amendment means based on this invention. It is a figure which shows the correction coefficient of the shading with respect to the coordinate position which concerns on this invention. It is a figure which shows the specific structure of the correction coefficient adjustment means which concerns on Embodiment 1 of this invention. It is a figure which shows the ratio of the integrated value of the color signal under sunlight and the ratio of the integrated value of the color signal under fluorescent lamp concerning Embodiment 1 of this invention. FIG. 5 is a relationship diagram between a distance H and an adjustment coefficient C according to Embodiment 1 of the present invention. It is a figure which shows the table structure in LUT which concerns on Embodiment 1 of this invention. It is a figure which shows a near-order type | formula in order to calculate how far the obtained value which concerns on Embodiment 1 of this invention is from the locus | trajectory of a black body radiation. It is a block diagram which shows the structure of the imaging device which concerns on Embodiment 2 of this invention. It is a figure which shows the specific structure of the correction coefficient adjustment means based on Embodiment 2 of this invention. It is a block diagram which shows the structure of the imaging device which concerns on Embodiment 3 of this invention. It is a figure which shows the specific structure of the correction coefficient adjustment means based on Embodiment 3 of this invention. It is a figure which shows the relationship of the adjustment coefficient C with respect to ratio (SIGMA) R / (SIGMA) B of the integrated value of the color signal which concerns on Embodiment 3 of this invention.

Explanation of symbols

1 Optical System 1a Lens 1b Infrared Rejection Filter (IRCF)
2 Image sensor 3 Analog signal processing means 4 A / D converter 5 Digital signal processing means 6a R shading correction means 6b G shading correction means 6c B shading correction means 6d Multiplier 6e Multiplier 6f Correction coefficient calculation means 6g Coordinate value Generating means 6h White balance gain circuit 7b of adder 7a R White balance gain circuit 7c G White balance gain circuit 8a R gamma correction means 8b G gamma correction means 8c B gamma correction means 9 Integration means 10. Correction coefficient adjustment unit 10a according to Embodiment 1 Division unit 10b Adjustment coefficient calculation unit 11 RGB / YUV conversion unit 20 Correction coefficient adjustment unit 20a according to Embodiment 2 Luminance signal generation unit 20b Normalization unit 20c Comparator 20d Adjustment Coefficient calculation means 30 according to the third embodiment Positive coefficient adjusting means 30a dividing means 30b adjustment coefficient calculating means

Claims (12)

An image sensor disposed on the imaging surface and including a plurality of pixels;
An image pickup apparatus that is arranged in front of the image pickup element and has color filters that transmit R, G, and B, respectively,
Correction coefficient calculation means for calculating a correction coefficient for each signal corresponding to R, G and B from the pixel position;
Integration means for integrating signals corresponding to R, G, and B obtained from the plurality of pixels of the image sensor over at least one screen, respectively, to obtain integrated values ΣR, ΣG, and ΣB;
Correction coefficient adjusting means for calculating an adjustment coefficient for adjusting the correction coefficient from a ratio of at least two of the integrated values ΣR, ΣG and ΣB;
An image pickup apparatus comprising: a shading correction unit that adjusts the correction coefficient according to the adjustment coefficient and performs shading correction for each of the signals corresponding to the R, G, and B. 2. An imaging apparatus according to claim 1, further comprising an infrared cut filter for removing unnecessary infrared rays. The imaging apparatus according to claim 2, wherein the infrared cut filter includes a thin film having a multilayer structure in which two or more kinds of thin film layers having different refractive indexes are stacked. The imaging apparatus according to claim 1, wherein the shading correction unit includes a multiplier that obtains the adjusted correction coefficient by multiplying the correction coefficient by the adjustment coefficient. The shading correction means includes an adder for adding the adjusted correction coefficient and a signal amplification factor, and a multiplier for multiplying the added value by a signal value for each pixel. The imaging device according to claim 1. The imaging apparatus according to claim 4, wherein the shading correction unit includes a calculation unit that calculates an image height that is a distance from a center of the imaging element for each pixel. The correction coefficient adjusting means includes a two-dimensional vector composed of two ratios obtained from the integrated values ΣR, ΣG and ΣB integrated by the integrating means, and an integrated value ΣR obtained under sunlight having the same correlated color temperature, It has an adjustment coefficient calculation means for calculating an absolute value H of a difference on a two-dimensional plane from a two-dimensional vector having two ratios obtained from ΣG and ΣB, and calculating the adjustment coefficient from the H. An imaging apparatus according to any one of claims 1 to 6. The imaging apparatus according to claim 7, wherein the adjustment coefficient calculation unit includes a table in which two ratios of the integrated values ΣR, ΣG, and ΣB correspond to the adjustment coefficient. The correction coefficient adjusting means includes a luminance signal generating means for calculating a luminance signal indicating the brightness of the entire image from the integrated values ΣR, ΣG and ΣB obtained from the integrating means, and the adjustment according to the magnitude of the luminance signal. The imaging apparatus according to claim 1, further comprising an adjustment coefficient calculation unit that calculates a coefficient. The correction coefficient adjusting unit includes a comparator that compares a value of the luminance signal obtained from the luminance signal generating unit with a predetermined constant, and the value of the luminance signal is compared with the constant by the comparator. The imaging apparatus according to claim 9, wherein when it is determined to be small, the value of the adjustment coefficient is decreased. The correction coefficient adjusting means has a divider for calculating a ratio ΣR / ΣB or ΣR / ΣG obtained from the integrated values ΣR, ΣG and ΣB obtained from the integrating means, and The imaging apparatus according to claim 1, wherein the adjustment coefficient is increased as a ratio of integrated values increases. A mobile phone comprising the imaging device according to claim 1.

Images