JP2006060462A - Imaging device - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide an imaging device capable of realizing white balance adjustment with reduced S/N degradation of the color of a low signal level. <P>SOLUTION: A charge storage period setting part 32a sets a point of time T2 as a charge storage period, and a correction amount calculation part 32b calculates the correction amount of the signal level of an R color on the basis of the average output signal level g2 of a G color and the average output signal level r2 of an R color at the point of time T2. Thus, as electric gain adjustment in an AWB part 8, the gain adjustment of g2/r2 times to the signal level of the R color is sufficient. In the case of correcting the signal level of the R color at a point of time T1, the gain adjustment of g1/r1 times for the signal level of the R color is needed. Since the difference of g2-r2 is smaller than the difference of g1-r1, g2/r2 becomes smaller than g1/r1. Thus, the electric gain adjustment amount in the AWB part 8 is reduced. <P>COPYRIGHT: (C)2006,JPO&NCIPI

Description

  The present invention relates to an image pickup apparatus, and more particularly to an image pickup apparatus that can improve the quality of image quality by solving the problem of deterioration of S / N of a color having a low signal level when white balance is adjusted.

  A conventional imaging apparatus that performs dynamic range expansion and prevention of S / N degradation on the low luminance side by weighting and adding a plurality of images having different exposure times and the like with a plurality of coefficients having a sum of 1 is provided. It is disclosed in the following Patent Document 1.

  Further, a conventional image pickup apparatus that individually controls the charge accumulation period of each photoelectric conversion element for each color by adjusting the charge read pulse application timing for each color and thereby adjusts the white balance is disclosed in Patent Document 2 below. Is disclosed.

JP-A-11-191860 (FIG. 7) Japanese Unexamined Patent Publication No. 2000-350222 (FIG. 1)

  However, in the conventional imaging device disclosed in Patent Document 1, the charge accumulation period is changed for each field, and the image is weighted for each color and then added (frame addition). There is a problem that the image is blurred.

  Further, since the conventional imaging device disclosed in Patent Document 2 controls the charge accumulation period by changing the application timing of the charge readout pulse, the application timing of the charge readout pulse is individually controlled for each color. There is a problem that must be done.

  Moreover, in the conventional imaging device disclosed in Patent Document 2, it is necessary to precisely control the end point of the charge accumulation period for each color, so that the processing for controlling the application timing of the charge readout pulse is complicated. There is also a problem of becoming.

  The present invention has been made to solve such a problem, and an imaging apparatus capable of realizing white balance adjustment with low S / N degradation of a low signal level color without adjusting the charge accumulation period for each color. The purpose is to obtain.

  An image pickup apparatus according to the present invention includes an image pickup element having a plurality of light receiving elements having different sensitivity characteristics and saturation characteristics, an integration unit that integrates the output signal level of the image pickup element for each color, and an integration result by the integration unit. A charge accumulation period setting means for setting a charge accumulation period of the image sensor, and white balance adjustment based on an integrated value for each color of the output signal level of the image sensor obtained during the charge accumulation period. White balance adjusting means to be implemented.

  According to the imaging apparatus according to the present invention, it is possible to obtain an imaging apparatus capable of realizing white balance adjustment with low S / N deterioration of a color with a low signal level without adjusting the charge accumulation period for each color.

Embodiment 1 FIG.
FIG. 1 is a block diagram showing the overall configuration of a single-plate imaging apparatus according to Embodiment 1 of the present invention. As shown in FIG. 1, the imaging apparatus according to the first embodiment includes a lens 1, a diaphragm 2, an imaging element 3, a CDS (Co-related Double Sampling) unit 4, an A / D conversion unit 5, The apparatus includes a linear correction unit 6, an AWB (Auto White Balance) unit 8, an interpolation unit 9, an image quality processing unit 10, an integration unit 13, and a CPU 15. The CPU 15 includes an integrated value determination unit 32, a charge accumulation period control unit 34, an AWB control unit 33, and a control signal generation unit 100. Although FIG. 1 shows an example in which the integrated value determination unit 32, the charge accumulation period control unit 34, and the AWB control unit 33 are all implemented by software in the CPU 15, they are configured as separate hardware. It is also possible.

  The lens 1 condenses the optical signal sent from the subject. The diaphragm 2 controls the amount of an optical signal incident from the lens 1 (hereinafter referred to as “incident light amount”). The image sensor 3 photoelectrically converts an optical signal incident through the diaphragm 2 and outputs an electrical signal corresponding to the optical signal. The CDS unit 4 performs noise reduction processing by the correlated double sampling method on the signal input from the image sensor 3, and outputs the processed signal. The A / D conversion unit 5 converts the analog signal input from the CDS unit 4 into a digital signal. The linear correction unit 6 corrects the non-linear characteristic of the image pickup device 3, corrects the output of the A / D conversion unit 5, and outputs it as a RAW data signal 12 proportional to the amount of incident light. Based on the control signal 24 input from the CPU 15, the AWB unit 8 balances the R (red), G (green), and B (blue) colors with respect to the RAW data signal 12 input from the linear correction unit 6. Is adjusted (hereinafter referred to as “white balance adjustment”), and the adjusted signal is output.

  The interpolation unit 9 performs synchronization processing on the signal input from the AWB unit 8 and outputs a signal after the processing. The image quality processing unit 10 performs image quality processing such as noise reduction, edge enhancement, and gamma adjustment on the signal input from the interpolation unit 9 and outputs the processed signal.

  The integration unit 13 performs integration for each integration area based on the integration area information 23 sent from the control signal generation unit 100, and outputs an integration result 14.

  The control signal generation unit 100 manages timing control of the single-plate imaging device according to the first embodiment. For example, the timing control of the charge accumulation period control unit 34, the integrated value determination unit 32, and the AWB control unit 33 is performed in synchronization with the timing of reading the charge of the image sensor 3.

  In the example illustrated in FIG. 1, the control signal generation unit 100 is configured in the CPU 15, but may be configured by hardware separate from the CPU 15. As a result, the CPU 15 can obtain advantages such as reduced task management and faster processing. The same applies to Embodiments 2 and 3 described later.

  FIG. 2 is a plan view showing an example of the configuration of the image sensor 3. A light receiving unit 41 is disposed in each of the plurality of color filters 40 arranged in the Bayer array. The arrangement of the color filters 40 may be a honeycomb arrangement in addition to the Bayer arrangement. Since the color filter 40 is regularly arranged, color signals of R, G, and B colors are output from the image sensor 3 in a time series according to a certain rule. Therefore, color signals of R, G, and B colors can be extracted from the RAW data signal 12 shown in FIG.

  FIG. 3 is a plan view illustrating a configuration example of the light receiving unit 41 included in the imaging element 3. The light receiving unit 41 has a configuration in which two types of light receiving elements 50 and 51 having different sensitivity characteristics and saturation characteristics are arranged as a set. The light receiving element 50 is a light receiving element with high sensitivity but a small amount of saturated light, and the light receiving element 51 is a light receiving element with low sensitivity but a large amount of saturated light. Here, in addition to the configuration in which the two types of light receiving elements 50 and 51 are arranged side by side in one pixel, the pixel in which the light receiving element 50 is disposed and the pixel in which the light receiving element 51 is disposed as separate pixels, It is also possible to adopt a configuration in which the same rows, staggered patterns, etc. are arranged regularly.

  FIG. 4 is a graph showing the sensitivity characteristics of the light receiving elements 50 and 51. The horizontal axis of the graph is the amount of incident light L, and the vertical axis is the output signal level. The high sensitivity light receiving element 50 increases the output signal level in proportion to the incident light amount until the incident light amount is L1, and when the incident light amount becomes L1 or more, the output signal level is clipped at S1 (L1). )have.

  Further, the low-sensitivity light receiving element 51 has an output signal level that increases in proportion to the incident light amount until the incident light amount becomes L2 (> L1), and when the incident light amount becomes L2 or more, the output signal level becomes S2 (L2). And has a sensitivity characteristic S2 (L) for clipping. The slope of the sensitivity characteristic S2 (L) of the light receiving element 51 is smaller than the slope of the sensitivity characteristic S1 (L) of the light receiving element 50. Regarding the sensitivity characteristic S2 (L) of the light receiving element 51, the output signal level when the incident light quantity is L1 is S2 (L1) (<S1 (L1)).

The sensitivity characteristic M (L) is obtained by adding the sensitivity characteristic S1 (L) of the light receiving element 50 and the sensitivity characteristic S2 (L) of the light receiving element 51. That means
M (L) = S1 (L) + S2 (L)
It is.

The output signal level M (L1) is a result of adding the output signal levels of the light receiving elements 50 and 51 when the amount of incident light is L1. When the incident light quantity is L1, only the light receiving element 50 reaches the saturation level,
M (L1) = S1 (L1) + S2 (L1)
It becomes.

The output signal level M (L2) is a result of adding the output signal levels of the light receiving elements 50 and 51 when the amount of incident light is L2. When the amount of incident light is L2, the light receiving elements 50 and 51 are both saturated.
M (L2) = S1 (L2) + S2 (L2) = S1 (L1) + S2 (L2)
It becomes.

The linear correction unit 6 shown in FIG. 1 performs correction so that the polygonal characteristic (nonlinear characteristic) of the sensitivity characteristic M (L) becomes the linear characteristic (linear characteristic) shown by the one-dot chain line in FIG. Correction (hereinafter referred to as “linear correction”) is performed using the coefficient k. That is, the sensitivity characteristic H (L) after linear correction is
H (L) = SI (L) + kS2 (L)
It is.

The value of the correction coefficient k can be calculated from the output signal levels S1 (L1) and S2 (L1) when the amount of incident light is L1. When the linear characteristic H (L) shown in FIG. 4 is obtained, k = S1 (L1) / S2 (L1). In this case, the output signal level H (L2) shown in FIG.
H (L2) = S1 (L1) + {S1 (L1) / S2 (L1)} × S2 (L2)
It becomes.

  In addition, when linear correction is not performed and the sensitivity characteristic S1 (L) and the sensitivity characteristic S2 (L) are simply added, k = 1. Since the value of the correction coefficient k varies depending on the amount of incident light, the imaging state, the state of the subject, etc., the dynamic range of the output signal level can be improved (expanded) optimally according to the imaging conditions. .

  FIG. 5 is a block diagram specifically showing the configuration of the integration unit 13, the integration value determination unit 32, the AWB control unit 33, and the charge accumulation period control unit 34 shown in FIG. Referring to FIGS. 1 and 5, integration unit 13 includes integration units 13 a and 13 b. Further, the integrated value determination unit 32 includes a charge accumulation period setting unit 32a and a correction amount calculation unit 32b.

  Referring to FIG. 5, integrating unit 13a includes color extracting unit 30a and a total of three signal level integrating units 31a provided for each of R, G, and B colors. The color extraction unit 30a separates the RAW data signal 12 input from the linear correction unit 6 shown in FIG. 1 for each of R, G, and B colors. The RAW data signal 12 separated for each color is input to the signal level integration unit 31a for each color, and each signal level integration unit 31a sets the signal level of the corresponding color for a certain period (for example, one frame period). Accumulate.

  Here, integration area information 23a is input to each signal level integration unit 31a from the CPU 15 shown in FIG. The integration area information 23a is information for designating a part of the area to be subjected to signal level integration processing in the entire screen. As an example, four areas at the center of a total of 16 areas obtained when one screen is divided into four vertically and horizontally are designated as integration areas. Each signal level integration unit 31a performs the above-described signal level integration processing for the integration area specified by the integration area information 23a.

  Data 14a relating to the integrated value of the signal level for each color is input to the charge accumulation period setting unit 32a. The charge storage period setting unit 32a compares the integrated values of the signal levels of the respective colors based on the data 14a, so that the dynamic range of the signal level of the subject in the imaging state can be secured. 3 charge accumulation periods (time point T2 shown in FIG. 7 described later) are set. A method for setting the charge accumulation period by the charge accumulation period setting unit 32a will be described in detail later.

  Data 16a related to the charge accumulation period set by the charge accumulation period setting unit 32a is input to the charge accumulation period control unit 34. The charge accumulation period control unit 34 generates and outputs a control signal 21 for controlling an electronic shutter (not shown) of the image sensor 3 based on the data 16a in order to control the charge accumulation period of the image sensor 3. . As shown in FIG. 1, the control signal 21 is input to the image sensor 3.

  1 and 5, the charge accumulation period control unit 34 may also generate a control signal 20 for controlling, for example, the opening / closing amount of the diaphragm 2 based on the data 16a.

  Referring to FIG. 5, integrating unit 13b includes color extracting unit 30b and a total of three signal level integrating units 31b provided for each of R, G, and B colors. The color extraction unit 30 b separates the RAW data signal 12 whose charge accumulation period is controlled by the charge accumulation period control unit 34 for each of R, G, and B colors. The RAW data signal 12 separated for each color is input to the corresponding color signal level integration unit 31b, and each signal level integration unit 31b receives the RAW obtained in the charge accumulation period T2 shown in FIG. The data signal 12 is integrated based on the integrated area information 23b output from the CPU 15, and an average output signal in that area, that is, an incident light amount is calculated.

  Data 14b regarding the integrated value of the signal level for each color is input to the correction amount calculation unit 32b. The correction amount calculation unit 32b calculates a signal level correction amount for appropriately adjusting the white balance by comparing the integrated values of the signal levels of the respective colors based on the data 14b. A method of calculating the correction amount of the signal level by the correction amount calculation unit 32b will be described in detail later.

  Data 16b relating to the correction amount of the signal level calculated by the correction amount calculation unit 32b is input to the AWB control unit 33. The AWB control unit 33 generates and outputs a control signal 24 for controlling the AWB unit 8 shown in FIG. 1 based on the data 16b. As shown in FIG. 1, the control signal 24 is input to the AWB unit 8.

  FIG. 6 is a block diagram illustrating another configuration example of the integrating unit 13 illustrated in FIG. Referring to FIGS. 5 and 6, integrating unit 13 c is configured by integrating (unifying into one) integrating unit 13 a and integrating unit 13 b having the same functions. The integrating unit 13c includes a color extracting unit 30c and a total of three signal level integrating units 31c provided for each of R, G, and B colors. The color extraction unit 30c is a unit that shares the color extraction unit 30a and the color extraction unit 30b, and the signal level integration unit 31c is a unit that uses the signal level integration unit 31a and the signal level integration unit 31b. . Integration area information 23c is input to each signal level integration unit 31c from the CPU 15 shown in FIG. The integration area information 23c corresponds to the integration area information 23a in the charge accumulation period setting process, and corresponds to the integration area information 23b in the signal level correction amount calculation process.

  Compared with the case where the integration unit 13a and the integration unit 13b are individually provided as shown in FIG. 5 by sharing the integration unit 13a and the integration unit 13b having the same functions as shown in FIG. The apparatus configuration can be simplified and the cost can be reduced.

  Next, a method for setting the charge accumulation period by the charge accumulation period setting unit 32a shown in FIG. 5 and a method for calculating the signal level correction amount by the correction amount calculation unit 32b will be described. In the following description, it is assumed that the color temperature is high, that is, the output signal level of the image sensor 40 corresponding to the R color is lower than the output signal level of the image sensor 40 corresponding to the colors G and B. To do. However, this assumption is only an example, and the magnitude relationship of the output signal level of the image sensor 40 of each color may differ from this assumption depending on the imaging state of the subject, but even in that case, the present invention can be applied. Is possible.

  FIG. 7 shows the relationship between the charge accumulation period and the output signal level in the case of using the imaging device 41 (see FIG. 3) having two light receiving elements 50 and 51 whose sensitivity characteristics are given by the graph of FIG. It is a graph. The horizontal axis of the graph is the charge accumulation period, and the vertical axis is the integrated value of the output signal level from the image sensor 41. When the average output signal level of each color in the charge accumulation period T1 is R color: r1, G color: g1, and B color: b1, the average output signal level has a high color temperature (blue sky, cloudy sky, fluorescent lamp, etc.) ), G1, b1 >> r1, and r1 >> b1 when the color temperature is low (incandescent lamp, halogen, dusk, etc.). Thus, the average output signal level of each color varies depending on the color temperature.

  In the conventional system, the incident light amount is adjusted to L1 by setting the charge accumulation period based on the G color signal having a high average output signal level (g1 in FIG. 7). For this reason, since the amount of incident light is small for the R color with a low average output signal level (r1 in FIG. 7), when white balance adjustment is performed, the quantization error is higher than the signals g1 and b1 with a high average output signal level. As a result, an adverse effect such as deterioration of color S / N or deterioration of graininess occurs at a low gradation.

  Therefore, the charge accumulation period is adjusted to T2 so that the incident light amount of the R color signal having a low average output signal level satisfies L1. In this case, for the color signals g2 and b2 having a high average output signal level, the dynamic range is improved by the sensitivity characteristic of the light receiving element 51, and the output signal can be taken out without being saturated. Therefore, in the linear correction unit 6, the output signal level is corrected in consideration of the difference in sensitivity characteristics between the light receiving element 50 and the light receiving element 51, so that the color S / N deterioration, graininess deterioration, etc. at a low gradation level. Can be avoided.

  Further, since the charge accumulation period of the high-sensitivity light receiving element 50 and the charge accumulation period of the low-sensitivity light receiving element 51 are set to be the same, there is a feature that the simultaneity of the imaging screen is maintained. As an effect of simultaneity, it is not necessary to correct and synthesize the imaging data of the high sensitivity light receiving element 50 and the low sensitivity light receiving element 51, and the cost of the imaging apparatus system can be reduced.

  The configuration using the average output signal level has been described so far, but the light receiving element 51 having the output signal level M (L2) of the incident light amount L2 shown in FIG. The saturation level can be detected, and fine adjustment (the charge accumulation period T2 is shortened in the direction of T1) can be performed in the charge accumulation period in which the light receiving element 51 is not clipped at the saturation level. As a result, it is possible to suppress the white balance shift and to correct the coloring in the saturation region due to the saturation of the single color while securing the dynamic range.

  FIG. 8 is a timing chart showing the relationship between the operation of the electronic shutter and the charge accumulation period. Referring to FIGS. 1 and 8, during the charge discharge period T1sh, an electronic shutter drive pulse is intermittently applied from the CPU 15 to the image sensor 3 at regular intervals. As a result, the electronic shutter is intermittently driven, and the accumulated charges of the light receiving elements 50 and 51 included in the imaging element 3 are discharged to the substrate. That is, the charge accumulation operation by the light receiving elements 50 and 51 is not performed in the charge discharging period T1sh.

  On the other hand, in the charge accumulation period T1exp, the electronic shutter driving pulse is not applied to the image sensor 3, and the electronic shutter is not driven. As a result, the light receiving elements 50 and 51 perform a charge accumulation operation.

  Since the frame period FR1 is constant, it is necessary to adjust the length of the charge discharge period T1sh in order to ensure the charge accumulation period set by the charge accumulation period setting unit 32a. Therefore, the number of electronic shutter drive pulses applied from the CPU 15 to the image sensor 3 is increased or decreased according to the set charge accumulation period.

  Referring to FIG. 7, next, the correction amount calculation unit 32b starts the charge accumulation operation (time zero), and then the G average output signal level g2 at the time when the charge accumulation period has elapsed (time T2). And an R color average output signal level r2, a correction amount of the R color signal level is calculated. Specifically, a correction amount of g2 / r2 is calculated in order to correct the difference of g2-r2. As a result, in the AWB unit 8 shown in FIG. 1, the gain is adjusted by g2 / r2 times the R signal level, and the white balance is adjusted appropriately.

  In the example shown in FIG. 7, the G color average output signal level g2 and the B color average output signal level b2 at the time point T2 are substantially equal, so correction for the B color is not necessary. However, when the average output signal level b2 is so small that it cannot be ignored than the average output signal level g2, the output signal level for B color can be corrected by the same method as the correction for the R color. .

  Thus, according to the imaging apparatus according to the first embodiment, the charge accumulation period setting unit 32a sets the time point T2 shown in FIG. 7 as the charge accumulation period, and the correction amount calculation unit 32b is Based on the average output signal level g2 for G color and the average output signal level r2 for R color, a correction amount for the R signal level is calculated. Accordingly, as the electrical gain adjustment in the AWB unit 8 shown in FIG. 1, it is sufficient to perform gain adjustment of g2 / r2 times the R signal level. By the way, referring to FIG. 7, when the R signal level is to be corrected at the time point T1 as in the conventional imaging apparatus, it is necessary to perform gain adjustment of g1 / r1 times the R signal level. There is. As is apparent from FIG. 7, the difference between g2 and r2 is smaller than the difference between g1 and r1, so that g2 / r2 is smaller than g1 / r1. Therefore, according to the imaging apparatus according to the first embodiment, it is possible to reduce the electrical gain adjustment amount in the AWB unit 8 as compared with the case where the R signal level is corrected at the time point T1, and as a result. S / N degradation of the R color due to the gain adjustment can be suppressed.

Embodiment 2. FIG.
FIG. 9 is a schematic diagram illustrating a configuration of the imaging element 3 provided in the three-plate imaging device according to Embodiment 2 of the present invention. The image pickup device 3 includes an R color image pickup device 43 provided with an R color filter 43R, a G color image pickup device 44 provided with a G color filter 44G, and a B color filter 45B. A B-color image sensor 45 is provided. An output signal SR corresponding to R color is output from the image sensor 43, an output signal SG corresponding to G color is output from the image sensor 44, and an output signal SB corresponding to B color is output from the image sensor 45. The Each of the imaging elements 43 to 45 includes the high sensitivity light receiving element 50 and the low sensitivity light receiving element 51 illustrated in FIG. 3.

  FIG. 10 is a block diagram showing an overall configuration of a three-plate imaging device according to Embodiment 2 of the present invention. As shown in FIG. 10, the imaging apparatus according to the second embodiment includes a lens 1, an aperture 2, R, G, and B three imaging elements 3a to 3c, a CDS unit 4a, an A / D conversion unit 5a, and a linear correction unit. 6, an AWB unit 8, an image quality processing unit 10, an integration unit 13, and a CPU 15. The CPU 15 includes an integrated value determination unit 32, a charge accumulation period control unit 34, an AWB control unit 33, and a control signal generation unit 100. In FIG. 10, an example in which the integrated value determination unit 32, the charge accumulation period control unit 34, and the AWB control unit 33 are all realized by software in the CPU 15 is shown. It is also possible.

  The lens 1 condenses the optical signal sent from the subject. The diaphragm 2 controls the amount of incident light of the optical signal incident from the lens 1. The image sensors 3a to 3c photoelectrically convert R, G, and B color optical signals incident through the aperture 2 and output electrical signals corresponding to the optical signals. The CDS unit 4a performs noise reduction processing by the correlated double sampling method on the signals input from the image sensors 3a to 3c, and outputs the processed signal. The A / D converter 5a converts the analog signal input from the CDS unit 4a into a digital signal. The linear correction unit 6 corrects the non-linear characteristics of the image sensors 3a to 3c, corrects the output of the A / D conversion unit 5a, and outputs it as a RAW data signal 12 proportional to the amount of incident light. Based on the control signal 24 input from the CPU 15, the AWB unit 8 performs white balance adjustment on the RAW data signal 12 input from the linear correction unit 6 and outputs the adjusted signal.

  The image quality processing unit 10 performs image quality processing such as noise reduction, edge enhancement, and gamma adjustment on the signal input from the AWB unit 8 and outputs the processed signal.

  Referring to FIG. 7, paying attention to the output signal level of G color, the average output signal level of G color rises rapidly from zero to T1 during the charge accumulation period, and when the charge accumulation period is T1, G color The amount of charge accumulated in the light receiving element 50 corresponding to is saturated. After the time T1, the average output signal level of G color rises gradually.

  Paying attention to the output signal level of the R color, since the output signal level of the R color is low, the charge accumulation amount of the light receiving element 50 corresponding to the R color is not saturated even at the time point T1. The average output signal level of R color at time T1 is r1 (<g1). The charge accumulation amount of the light receiving element 50 corresponding to the R color is saturated at the time point T2. The average output signal level of R color at time T2 is r2 (= g1).

  FIG. 11 is a block diagram specifically showing the configuration of integrating unit 13, integrated value determining unit 32, AWB control unit 33, and charge accumulation period control unit 34 according to the second embodiment, corresponding to FIG. is there. Output signals SR, SG, and SB output from the image sensors 43, 44, and 45 shown in FIG. 9 are processed by the CDS unit 4a, the A / D conversion unit 5a, and the linear correction unit 6 shown in FIG. Then, the signal is input to the corresponding color signal level integration unit 31a shown in FIG.

  Each signal level integration unit 31a integrates each signal level of the output signals SR, SG, and SB for a certain period (for example, one frame period), targeting the integration area specified by the integration area information 23a. The integrated value of the signal level related to the R color is significantly larger than the integrated value of the signal level related to the G color due to the imaging in a state where the color temperature is high and the influence of the spectral sensitivity characteristic on the R color filter 43R. Get smaller. Data 14a relating to the integrated value of the signal level for each color is input to the charge accumulation period setting unit 32c.

  The charge accumulation period setting unit 32c individually sets the charge accumulation period of the image sensor 3 for each color by comparing the integrated values of the signal levels of the respective colors based on the data 14a. Specifically, referring to FIG. 7, the period from time zero to time T1 is set as the length of the charge accumulation period of each color G and B, and the period from time zero to time T2 is set to R color. Is set as the length of the charge accumulation period. The time point T2 can be obtained based on the time point T1 and the average output signal levels r1 and g1 at the time point T1. Expressed by the equation, T2 = (g1 / r1) × T1. Here, the charge accumulation period setting unit 32c sets the end point of the charge accumulation period to the same point for all the colors of R, G, and B. Therefore, the start time of the R color charge accumulation period is set to a time earlier than the start time of the G and B color charge accumulation periods by a time corresponding to T2-T1. In other words, the charge accumulation period setting unit 32c determines the start time of the R color charge accumulation period according to the difference between the G average output signal level g1 and the R average output signal level r1 at the time T1. It is set at a time earlier than the start time of the charge accumulation period of each color of G and B.

  FIG. 12 is a timing chart showing the relationship between the operation of the electronic shutters for each color of R, G, and B and the charge accumulation period for each color of R, G, and B. For example, focusing on the frame period FR3 (N), the R color charge accumulation period T3expR is started earlier than the G and B charge accumulation periods T3expG and T3expB. The end points of the charge accumulation periods T3expR, T3expG, and T3expB are all the same, and the accumulated charge amounts of all colors are read out by one charge read clock applied at the end of the frame period FR3 (N).

  Referring to FIG. 11, each signal level integration unit 31b targets the integration area specified by the integration area information 23b, and outputs signal SR, SG, SB whose charge storage period is controlled by the charge storage period control unit 34. And the signal levels of the corresponding colors are integrated. Data 14b regarding the integrated value of the signal level for each color is input to the correction amount calculation unit 32d. The correction amount calculation unit 32d calculates a signal level correction amount for appropriately adjusting the white balance by comparing the integrated values of the signal levels of the respective colors based on the data 14b.

  As described above, according to the imaging apparatus according to the second embodiment, the charge accumulation period setting unit 32c determines the G average output signal level g1 and the R average output signal level r1 at the time T1 illustrated in FIG. 12, the R color charge accumulation period T3expR is set longer than the G color charge accumulation period T3expG, and T3expR = T2. Therefore, in the imaging with the extended charge accumulation period T3expR, the average output signal level of R color increases as compared with the case where the charge accumulation period T3expR is not extended. As a result, the gain correction amount of the R average output signal level in the correction amount calculation unit 32d can be set to r1 / g1 = 1. Alternatively, the gain correction amount of the average output signal level of the R color in the gain correction amount calculation unit 32d can be reduced as compared with the case where the extension control of the charge accumulation period T3expR is not performed.

  Therefore, according to the imaging apparatus according to the second embodiment, the electrical gain adjustment amount in the AWB unit 8 can be reduced, and as a result, the S / N degradation of the R color due to the gain adjustment is suppressed. can do. In addition, it is not necessary to perform an interpolation process by adopting the three-plate method, and it is possible to construct a system with the configuration shown in FIG. 10, and to improve the color purity and prevent the generation of false colors due to the interpolation process. Is possible.

  In the above description, for simplification of description, an example in which only the R charge storage period T3expR is extended has been described. However, the time T1 illustrated in FIG. 7 and the average output signal level b1 at the time T1 are illustrated. , G1 and the B color charge accumulation period T3expR can be extended.

Embodiment 3 FIG.
FIG. 13 is a block diagram showing an overall configuration of a single-plate imaging apparatus according to Embodiment 3 of the present invention. As shown in FIG. 13, the imaging apparatus according to the third embodiment includes a lens 1, an aperture 2, an imaging device 3, a CDS unit 4, an A / D conversion unit 5, a linear correction unit 6, a color extraction / synthesis processing unit 70, An interpolation unit 9, an image quality processing unit 10, an integration unit 13, and a CPU 15 are provided. The configuration of the image sensor 3 is the same as the configuration shown in FIG. In the third embodiment, each image sensor 41 included in the image sensor 3 includes the high sensitivity light receiving element 50 and the low sensitivity light receiving element 51 shown in FIG. The color extraction / synthesis processing unit 70 is controlled by a control signal 82 from the CPU 15.

  The CPU 15 includes a charge accumulation period control unit 34, an integrated value determination unit 32, a color extraction / synthesis control unit 101, and a control signal generation unit 100. FIG. 13 shows an example in which the charge accumulation period control unit 34, the integrated value determination unit 32, and the color extraction / synthesis control unit 101 are all implemented by software in the CPU 15, but separately as hardware. It is also possible to configure. The color extraction / synthesis control unit 101 controls the color extraction / synthesis processing unit 70 with a control signal 82 based on the determination result of the integrated value determination unit 32.

  FIG. 14 is a block diagram specifically showing the configuration of the color extraction / synthesis processing unit 70 shown in FIG. The color extraction / synthesis processing unit 70 has an image memory (field memory) 71 capable of storing image data for several frames, an image data extraction unit 72, and an image data synthesis unit 73. When the average output signal levels of the G color and the B color are substantially the same and the average output signal level of the R color is low, the image data extraction unit 72 stores the G color and B color image data stored in the image memory 71. Extract. In the subsequent frame, the image data extraction unit 72 adjusts the average output signal level of R color to be the same as the average output signal level of G color and B color extracted in the previous frame, and stores them in the image memory 71. The R color image data is extracted. The image data synthesizing unit 73 synthesizes the extracted R, G, B color image data.

  7 and 13, a period from time zero to time T1 is set as a first charge accumulation period T2exp1, and a period from time zero to time T2 is set as a second charge accumulation period T2exp2. Data 16a relating to the set charge accumulation periods T2exp1 and T2exp2 is input to the charge accumulation period control unit 34. The charge accumulation period control unit 34 generates and outputs a control signal 21 for controlling the electronic shutter of the image sensor 3 based on the data 16 a in order to control the charge accumulation period of the image sensor 3. The control signal 21 is input to the image sensor 3.

  FIG. 15 is a timing chart showing the relationship between the operation of the electronic shutter and the charge accumulation period. With reference to FIGS. 14 and 15, first, in the frame period FR2 (N), imaging is performed in the first charge accumulation period T2exp1. An image acquired by this imaging (hereinafter referred to as “first image”) is stored in the image memory 71. In the subsequent frame period FR2 (N + 1), imaging is performed in the second charge accumulation period T2exp2. An image acquired by this imaging (hereinafter referred to as “second image”) is also stored in the image memory 71 in the same manner as the first image.

  Note that the first charge accumulation period T2exp1 in the frame period FR2 (N) and the second charge accumulation period T2exp2 in the frame period FR2 (N + 1) are detected in the frame period before the frame period FR2 (N−1). It is set based on the result.

  The image data extraction unit 72 extracts and extracts image data corresponding to G and B colors (hereinafter referred to as “first image data”) from the first image stored in the image memory 71. The first image data is sent to the image data composition unit 73. The image data extraction unit 72 extracts and extracts image data corresponding to the R color (hereinafter referred to as “second image data”) from the second image stored in the image memory 71. The second image data is sent to the image data composition unit 73. The image data synthesis unit 73 generates image data corresponding to one image by synthesizing the first image data and the second image data.

  As described above, according to the imaging device according to the third embodiment, the charge accumulation period setting unit 80 includes the first charge accumulation period T2exp1 (= T1) and the second charge accumulation period T2exp2 (= T2) is set. Then, the first image is acquired by imaging in the first charge accumulation period T2exp1, the second image is acquired by imaging in the second charge accumulation period T2exp2, and the image data synthesis unit 73 By combining the first image data corresponding to each color of G and B extracted from the inside and the second image data corresponding to the R color extracted from the second image, one sheet Image data corresponding to the image is generated. Accordingly, since the image data obtained by the image data synthesizing unit 73 has substantially the same signal level for each color of R, G, and B, the AWB unit 8 shown in FIG. 1 can be omitted. As a result, it is possible to avoid the deterioration of the S / N of the R color due to the gain adjustment accompanying the AWB adjustment.

  In the above description, for simplification of description, an example in which the charge accumulation periods T2exp1 of the colors G and B are made the same has been described. However, the time T1 shown in FIG. 7 and the average output at the time T1 are shown. A B color charge accumulation period may be set based on the signal levels b1 and g1. In this case, since an individual charge accumulation period is set for each color of R, G, and B, an image is acquired for each color using three consecutive frame periods, and the three images stored in the image memory 71 are stored. The image data extracted from the image is synthesized by the image data synthesis unit 73.

1 is a block diagram illustrating an overall configuration of a single-plate imaging device according to Embodiment 1 of the present invention. It is a top view which shows an example of a structure of an image pick-up element. It is a top view which shows the structural example of the light-receiving part which an image pick-up element has. It is a graph which shows the sensitivity characteristic of a light receiving element. FIG. 2 is a block diagram specifically illustrating configurations of an integration unit, an integration value determination unit, an AWB control unit, and a charge accumulation period control unit illustrated in FIG. 1. It is a block diagram which shows the other structural example of the integrating | accumulating part shown in FIG. 5 is a graph showing a relationship between a charge accumulation period and an output signal level when an imaging device having two light receiving elements whose sensitivity characteristics are given by the graph of FIG. 4 is used. 6 is a timing chart showing the relationship between the operation of the electronic shutter and the charge accumulation period. It is a schematic diagram which shows the structure of the image pick-up element with which the 3 plate type imaging device which concerns on Embodiment 2 of this invention is provided. It is a block diagram which shows the whole structure of the 3 plate type imaging device which concerns on Embodiment 2 of this invention. FIG. 6 is a block diagram specifically showing the configuration of an integration unit, an integration value determination unit, an AWB control unit, and a charge accumulation period control unit according to Embodiment 2 of the present invention, corresponding to FIG. 5. 6 is a timing chart showing the relationship between the operation of electronic shutters for each color of R, G, and B and the charge accumulation period for each color of R, G, and B. It is a block diagram which shows the whole structure of the single plate-type imaging device which concerns on Embodiment 3 of this invention. FIG. 14 is a block diagram specifically illustrating a configuration of a color extraction / synthesis processing unit illustrated in FIG. 13. 6 is a timing chart showing the relationship between the operation of the electronic shutter and the charge accumulation period.

Explanation of symbols

3, 41, 43 to 45 Image sensor, 8 AWB unit, 13 integrating unit, 31a to 31c signal level integrating unit, 32 integrated value determining unit, 32a, 32c charge accumulation period setting unit, 32b correction amount calculating unit, 33 AWB control 34, charge storage period control unit, 50, 51 light receiving element, 70 color extraction / synthesis processing unit, 71 image memory, 72 image data extraction unit, 73 image data synthesis unit.

Claims (6)

An image sensor having a plurality of light receiving elements having different sensitivity characteristics and saturation characteristics;
Integrating means for integrating the output signal level of the image sensor for each color;
Charge accumulation period setting means for setting a charge accumulation period of the image sensor based on the accumulation result by the accumulation means;
An image pickup apparatus comprising: a white balance adjustment unit that performs white balance adjustment based on an integrated value for each color of the output signal level of the image pickup element obtained during the charge accumulation period. The plurality of light receiving elements are:
A first light receiving element having high sensitivity and a small saturated light amount;
The imaging apparatus according to claim 1, further comprising a second light receiving element having low sensitivity and a large saturation light amount.   The imaging apparatus according to claim 1, further comprising a correction unit that corrects a nonlinear characteristic of incident light amount versus output signal level related to an output of the imaging element into a linear characteristic.   The imaging apparatus according to claim 1, wherein the charge accumulation period setting unit sets the charge accumulation period based on a color signal having the lowest output signal level accumulated by the accumulation unit. .   The imaging apparatus according to claim 1, wherein the charge accumulation period setting unit sets the different charge accumulation period for each color. The charge accumulation period setting means sets a first charge accumulation period for the first color, sets a second charge accumulation period for the second color,
An image storage unit that stores a first image acquired by imaging in the first charge accumulation period and a second image acquired by imaging in the second charge accumulation period;
First image data corresponding to the first color and second image data corresponding to the second color are respectively selected from the first and second images stored in the image storage unit. An image data extraction unit to be extracted;
The imaging apparatus according to claim 1, further comprising: an image data synthesis unit that synthesizes the first and second image data extracted by the image data extraction unit.

Images