JP2006121151A - Signal processing method, signal processing apparatus and physical information acquiring apparatus - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To perform signal processing with signal level amplification, while preventing the problem of deterioration in the S/N ratio, in an imaging apparatus. <P>SOLUTION: A gain analyzing unit 230 sets changeover switches 312, 314 on output terminals 312b, 314b, to acquire an RGB signal from a solid-state imaging element 10 (sensor). The unit 230 analyzes total gain values per pixel of the sensor, and computes an electric charge accumulation time for each RGB pixel so that the electric charge accumulated amount for the gain, equal to the result of analysis can be controlled by using electronic shutter control. In this process, main photographing is executed at the exposure time decided as above. For a signal acquired by a sensor, the level of a signal to be outputted from the sensor can be previously increased, by prolonging the exposure time, even if the processing for increasing the signal level is not performed, and the processing for increasing the signal level can be performed; while avoiding the problem of deterioration in S/N ratio generated, when amplified signal processing is performed for the output signal. <P>COPYRIGHT: (C)2006,JPO&NCIPI

Description

  The present invention relates to a signal processing method, a signal processing device, and a physical information acquisition device. More specifically, for example, a plurality of unit components that are sensitive to electromagnetic waves input from outside such as light and radiation are arranged, and the physical quantity distribution converted into an electric signal by the unit components is addressed. The present invention relates to a signal processing technique for reading out a signal and acquiring information for a predetermined purpose, which is suitable for use in a physical quantity distribution detection device such as a solid-state imaging device that can be arbitrarily selected by control and read out as an electrical signal. For example, the present invention relates to signal processing techniques for AGC, WB, and γ correction.

  Physical quantity distribution formed by arranging multiple unit components (for example, pixels) that are sensitive to changes in physical quantity such as electromagnetic waves or pressure (contact, etc.) input from outside such as light and radiation, in a line or matrix form. Sensing semiconductor devices are used in various fields.

  For example, in the field of video equipment, CCD (Charge Coupled Device) type or MOS (Metal Oxide Semiconductor) type or CMOS (Complementary Metal-oxide Semiconductor) type imaging that detects changes in light (an example of electromagnetic waves) which is an example of physical quantity. A solid-state imaging device using an element (imaging device) is used. Here, “solid” means made of semiconductor.

  For example, by reducing the size and price of solid-state image sensors represented by CCDs and CMOS image sensors, various video devices using them, such as digital still cameras for taking still pictures, mobile phones with cameras, or videos for taking moving pictures Cameras etc. are spreading rapidly. In particular, CMOS image sensors are attracting attention as a replacement for CCDs in the future because they can be manufactured with lower power consumption and lower cost than CCDs.

  In recent years, with the advancement of semiconductor technology, the number of pixels of solid-state image sensors has been rapidly increasing. For example, solid-state image sensors having several million pixels have been developed and used for digital still cameras and movies for which high resolution is required. Used for video cameras. Among them, the CMOS sensor is a solid-state imaging device in which each pixel is provided with a photoelectric conversion element and a readout circuit. Since each pixel can be accessed at random or read out at high speed, it is a promising sensor for the future. is there.

  In the field of computer equipment, fingerprint authentication devices that detect fingerprint images based on changes in electrical characteristics and optical characteristics based on pressure are used. These read out, as an electrical signal, a physical quantity distribution converted into an electrical signal by a unit component (a pixel in a solid-state imaging device).

  On the other hand, some solid-state imaging devices have an amplification driving transistor such as an electrostatic induction transistor or a MOS transistor in a pixel signal generation unit that generates a pixel signal corresponding to the signal charge generated in the charge generation unit. There is an amplifying solid-state imaging device including a pixel having a solid-state imaging device (APS: Active Pixel Sensor / gain cell) configuration. For example, many CMOS solid-state imaging devices have such a configuration.

  In such an amplification type solid-state imaging device, in order to read out pixel signals to the outside, address control is performed on an imaging unit (also referred to as a pixel unit) in which a plurality of unit pixels are arranged, and signals from individual unit pixels are transmitted. Is arbitrarily selected and read out. That is, the amplification type solid-state imaging device is an example of an address control type solid-state imaging device.

  For example, an amplification type solid-state imaging device which is a kind of XY address type solid-state imaging device in which unit pixels are arranged in a matrix form an active element (MOS transistor) such as a MOS structure in order to give the pixel itself an amplification function. ) To form a pixel. That is, signal charges (photoelectrons) accumulated in a photodiode which is a photoelectric conversion element are amplified by the active element and read out as image information.

  In this type of XY address type solid-state imaging device, for example, a large number of pixel transistors are arranged in a two-dimensional matrix to form an imaging unit, and a signal charge corresponding to incident light for each line (row) or each pixel. Accumulation is started, and a current or voltage signal based on the accumulated signal charge is read from each pixel in a predetermined order by addressing.

  Here, in the MOS (including CMOS) type, as an example of address control, one row is accessed simultaneously, pixel signals are read out from the imaging unit in units of rows, and predetermined signal processing is performed as necessary. In addition, a method of sequentially reading out pixel signals for one row of each row to the output side is often used. For example, the signal output of the pixels arranged in a matrix is sequentially sent to the vertical signal line for each row, and is further connected from the vertical signal line to the horizontal readout line in the horizontal direction.

  The analog pixel signal read from the imaging unit is converted into digital data by an analog-digital converter (AD converter; Analog Digital Converter) as necessary.

  In an imaging apparatus that captures an image using an imaging element (imaging device) such as a CCD (Charge Coupled Device) imaging element or a CMOS (Complementary Metal-Oxide Semiconductor) imaging element, a digital image signal processing LSI (Large Scale Integrated) is used. The function of the camera can be configured around electronic components by electronic processing on image data using a combination with a circuit (large-scale integrated circuit).

  For example, primary color separation processing that separates and synchronizes primary color signals of red (R), green (G), and blue (B) from complementary color imaging data, white balance adjustment processing that takes white balance (WB), Luminance data (Y) and color data (C) (summary) are obtained by performing pixel interpolation processing that supplements pixels using peripheral pixel data, gamma (γ) correction processing that adjusts the degree of gradation, or color difference matrix processing and encoding processing. YC signal generation processing for generating digital video data) (see, for example, Non-Patent Document 1).

Kazuya Yonemoto, “Basics and Applications of CCD / CMOS Image Sensors”, CQ Publisher, August 10, 2003, first edition, p166-167

  FIG. 12 is a diagram illustrating a configuration example of a conventional imaging apparatus having a mechanism for realizing functions such as white balance adjustment processing, pixel interpolation processing, and gamma correction processing.

  As shown in FIG. 12, the solid-state imaging device 910 operates according to the drive pulse from the timing control unit 991. Further, the entire signal signal processing system includes a CDS (Correlated Double Sampling) circuit 962a that reduces noise by sampling an analog imaging signal from the solid-state imaging device 910, and a CDS-processed pixel signal. A preamplifier unit 962 having an amplification amplifier 962b having an AGC (Automatic Gain Control) function to be amplified, and an AD (Analog / Digital) conversion unit 964 that converts an analog signal output from the preamplifier unit 962 into a digital signal. The former stage signal processing system 960a is provided. Further, the digital signal input from the AD conversion unit 964 is configured to include a subsequent signal processing system 960b that performs predetermined image processing.

  In the signal processing system 960a in the previous stage, in an imaging apparatus using a color solid-state imaging device, the signal output from the CCD solid-state imaging device 910 obtained by imaging all the pixels with the same exposure time is usually the CDS circuit 962a → Amplification amplifier 962b with an AGC processing function is transmitted in the order of AD conversion unit 964, and is input to a signal processing system (image signal processing unit 966) at the subsequent stage.

  In the subsequent signal processing system 960b, the digital video is output through the white balance amplifier unit 982 → pixel interpolation processing unit 984 → gamma correction processing unit 986 → YC signal generation processing unit 988.

<About WB>
In the white balance adjustment processing in the white balance amplifier unit 982, the color balance of the captured video is adjusted with respect to an image captured under external light having a biased color temperature. That is, even under a light source having a different color temperature, a human being unconsciously corrects in the brain, so that a white object can be recognized as white. However, when a subject image is captured using an image sensor (image sensor) such as a CCD image sensor or a CMOS image sensor, white is not output as white due to the influence of the color temperature of the light source.

  For example, when white is included in a subject, a signal of R (red) pixels is sufficiently obtained when imaged in an environment such as indoors where the color temperature is low, but signals of G (green) and B (blue) pixels are small. Therefore, it becomes a reddish video. On the other hand, when the image is taken in an environment where the color temperature is high, such as outdoors, a sufficient B (blue) pixel signal is obtained, but the G (green) and R (red) pixel signals are small. Become. This is generally said that the white balance is lost. The color temperature is the temperature (K) of a black body having the same chromaticity as the test light source.

  In order to eliminate the phenomenon of white balance loss, conventionally, a white subject is photographed white regardless of the color temperature of the illumination light in order to match the image signal acquired by the image sensor with the human eye. As described above, the image pickup apparatus is equipped with an auto white balance adjustment function. For example, in an auto white balance processing unit of camera signal processing, a predetermined gain is individually applied to each of a plurality of color signals that are color-separated so that white appears white, and correction is performed.

<About pixel interpolation processing>
In addition, the pixel interpolation processing in the pixel interpolation processing unit 984 can be said to be a function unique to a digital camera. In order to generate an image file having the same number of pixels as the number of pixels in the solid-state image sensor 910, a color that does not exist at a certain pixel position. The component is estimated from the surrounding pixels and the pixel signal is supplemented.

<About gamma correction processing>
Further, in the gamma correction processing in the gamma correction processing unit 986, the gradation level is adjusted so that the pixel signal level with respect to the light amount matches the human visual characteristic (luminosity) of identifying brightness in proportion to the logarithm of the light amount. Adjust the change characteristics to give the knee characteristics. Generally, this gradation change characteristic is called a gamma (γ) curve. In color image processing, the gamma (γ) curve of each color is adjusted so that faithful color reproduction is performed.

  However, in the AGC process, the white balance adjustment process, and the gamma correction process, the pixel signal is amplified or reduced by analog or digital processing, and particularly during amplification involving the process of increasing the signal level to be processed. Since noise is also amplified, it is difficult to avoid the problem of S / N degradation.

  For example, in the AGC processing in the amplification amplifier 962b, the signal level is amplified so that the maximum value of the signal output level matches the input range of the AD conversion unit 964 in the next stage, which causes a deterioration in the S / N of the image. .

  Further, in the white balance adjustment processing in the white balance amplifier unit 982, the level of each RGB pixel is set so that the output video becomes white (R: G: B = 1: 1: 1) when white is photographed with the light source at the time of photographing. Amplify. When shooting with light having a biased color temperature, it is necessary to amplify a color signal with a low signal level to obtain a color image with a good white balance, which causes a deterioration in the S / N of the image.

  For example, when an image is taken in an environment where the color temperature is low, such as indoors, a signal of R (red) pixel is sufficiently obtained, but a signal of G (green) pixel and B (blue) pixel becomes small and white balance is obtained. Therefore, it is necessary to amplify the signal levels of the G pixel and the B pixel. In particular, the B pixel has a smaller signal level and needs to be amplified with a larger amplification factor. Further, when an image is taken in an environment where the color temperature is high, such as outdoors, a signal of B (blue) pixel is sufficiently obtained, but a signal of G (green) pixel and R (red) pixel becomes small and white balance is obtained. Therefore, it is necessary to amplify the signal levels of the G pixel and the R pixel. In particular, the R pixel has a smaller signal level and needs to be amplified with a larger amplification factor.

  In the gamma correction processing in the gamma correction processing unit 986, the change characteristic of the pixel signal, that is, the gradation characteristic is adjusted so that the amplification factor for the portion with a small amount of light is larger than the portion with a large amount of light. Therefore, as a whole, the signal level is amplified for a low-brightness subject in which the amount of imaged light is insufficient, and the signal level is suppressed for a high-brightness subject. In particular, a low-brightness subject amplifies the signal level with a larger amplification factor than a high-brightness subject, which is a factor that degrades the S / N of an image. Originally, the low-brightness subject portion has a low signal level and is disadvantageous to noise, and since this is amplified with a large amplification factor, the problem of S / N degradation is greater than that of the high-brightness subject portion.

  The present invention has been made in view of the above circumstances, and substantially has a function accompanied by processing for increasing a signal level, such as AGC processing, white balance control processing, or gamma correction processing, while avoiding the problem of S / N degradation. It aims to provide a new mechanism that can be executed in a practical manner.

  The signal processing method according to the present invention includes a detection unit that detects change information according to a change in incident physical quantity, and a unit signal generation unit that generates a unit signal based on the change information detected by the detection unit. A signal processing method for performing predetermined signal processing based on a unit signal output from an apparatus for physical quantity distribution detection that is included in a component and the unit components are arranged in a predetermined order. Instead of amplified signal processing for increasing the level of the output unit signal, a unit signal of a level corresponding to the amplification factor in the amplified signal processing is output from the unit component while suppressing S / N degradation associated with the amplified signal processing. As described above, the method includes a step of adjusting the detection time in the detection unit.

  Here, “adjusting the detection time in the detection unit individually” is not essential to adjust the detection time independently for all unit components, but the attribute of the unit component in relation to signal processing The detection time may be adjusted according to each attribute.

  In short, the level of the unit signal output from the unit component is adjusted by adjusting the detection time in advance so that the unit signal of the level corresponding to the amplification factor in the amplified signal processing is output from the unit component. Thus, S / N degradation that can occur when amplified signal processing is performed on a unit signal output from a unit component is suppressed.

  The signal processing apparatus according to the present invention is a suitable apparatus for carrying out the signal processing method according to the present invention, and is replaced with amplified signal processing for increasing the level of the unit signal output from the unit component. A control unit that adjusts the detection time in the detection unit so that a unit signal having a level corresponding to the amplification factor in the amplification signal processing is output from the unit component while suppressing S / N degradation associated with the amplification signal processing. It was supposed to be prepared.

  An apparatus for physical quantity distribution detection according to the present invention is an apparatus to which the signal processing apparatus according to the present invention is applied.

  The inventions described in the dependent claims define further advantageous specific examples of the signal processing method, the signal processing device, and the physical quantity distribution detection device according to the present invention.

  For example, as amplified signal processing that increases the level of the unit signal output from the unit component, white balance processing is performed so that the entire imaging signal obtained by the color imaging device approaches white (white balance adjustment control) Also, the so-called AGC process for correcting the sensitivity variation of each detection unit, and the change characteristic of the unit signal with respect to the change in the physical quantity is adjusted so that the amplification factor for the part with the small physical quantity change is larger than the part with the large physical quantity change. There is a so-called gamma correction process.

  Here, in the case of “adjusting the detection time in the detection unit individually”, in the case of AGC processing and gamma correction processing, processing with different amplification factors is required for each unit component, so all unit configurations The detection time should be adjusted independently for the elements. On the other hand, in the case of the white balance adjustment system, it is not necessary to make the amplification factor different for every unit component, and the amplification factor may be made different for each color component. On the other hand, it is preferable to adjust the detection time to be independent for each color.

  In addition, when “adjusting the detection time in the detection unit individually”, the size of the unit signal corresponding to the change in the physical quantity for each detection unit is detected in advance in the preparation process, the detection unit performs detection for a predetermined detection time. In accordance with the magnitude of the unit signal corresponding to the change in the physical quantity for each detection unit detected here, the level according to the amplification factor in the amplification signal processing while suppressing the S / N deterioration accompanying the amplification signal processing It is preferable to individually calculate the detection time in the detection unit such that the unit signal is output from the unit component. And in response to this result, in this step, it is preferable to individually control the detection time of each detection unit based on the calculated detection time.

  In other words, during the preparatory process prior to the main acquisition of the signal (advance acquisition mode), image signal processing accompanied with processing for increasing the signal level, such as AGC processing, white balance control processing, or gamma correction processing, is executed. In order to execute the function by controlling the detection time in the detection unit, the detection time for each detection unit is calculated. Thereafter, at the time of the main process (main acquisition mode) in which the main acquisition of the signal is performed, the detection time of each detection unit is individually controlled based on the calculated individual detection time.

  Also, “adjusting the detection time in the detection unit individually” is preferably performed using an electronic shutter function. Alternatively, this electronic shutter function may be used in combination with a mechanical shutter function that opens and closes the entrance of a physical quantity to the detection unit.

  Here, after the change information detected by the detection unit is read out, the next detection starts, and when the change information is cleared before the read-out, the detection time is from the clear to the next read-out change information. Therefore, when “adjusting the detection time in the detection unit individually using the electronic shutter function”, a method of individually adjusting the timing for clearing the change information detected in the detection unit, and the change detected in the detection unit Either of the methods for individually adjusting the timing of reading information, or a combination of both can be used.

  According to the present invention, the detection time in the detection unit is adjusted in advance so that a unit signal of a level corresponding to the amplification factor in the amplified signal processing is output from the unit component. Accordingly, the level of the signal output from the detection unit can be increased in advance by extending the detection time without performing processing for increasing the signal level for the signal acquired by the detection unit. . Therefore, it is possible to execute signal processing accompanied by processing for increasing the signal level while avoiding the problem of S / N degradation that may occur when the amplified signal processing is performed on the unit signal output from the unit component. Become.

  By using a physical quantity distribution detection device that can adjust the detection time for each attribute according to the attribute of the unit component in relation to signal processing, the unit signal output from the unit component Even if the amplified signal processing is not performed, signal processing that increases the signal level, such as AGC processing, white balance adjustment, or gamma correction, can be realized by adjusting the detection time and output from the unit component. Since the amplified signal processing is not actually performed on the unit signal, it is possible to avoid the problem of S / N degradation that may occur when realistic amplified signal processing is performed.

  Hereinafter, embodiments of the present invention will be described in detail with reference to the drawings.

<Overall configuration of digital still camera; CMOS type>
FIG. 1 is a schematic configuration diagram showing a first embodiment of an imaging apparatus (camera system) according to the present invention. The image pickup apparatus according to the first embodiment includes, for example, a CMOS image pickup device 12 that is an example of the solid-state image pickup device 10, a camera module 3 having an image pickup lens 50 that captures an optical image of a subject Z, and an image pickup signal obtained by the camera module 3. The digital still camera 1 includes a main body unit 4 that generates a video signal based on the image and outputs the video signal to a monitor or stores an image in a predetermined storage medium. In addition, you may make it comprise as the imaging device module 3a with the form which integrated the camera module 3 and the main body unit 4. FIG.

  The CMOS image pickup device 12 has an image pickup unit 12a including a large number of sensor units. As a color image capturing application, the sensor unit is provided with a color filter of any one of color separation filters composed of a combination of color filters on a light receiving surface such as a photodiode on which light is incident. As an example, sensor units (unit pixels) arranged in a square lattice using three colors of red (R), green (G), and blue (B) using a basic color filter of a so-called Bayer array. They are arranged so as to correspond to color filters (primary color filters). Or it is good also as a thing of the complementary color filter structure which combined four colors, cyan (C), magenta (M), yellow (Y), and green (G).

  When the primary color signal processing is performed as the signal processing, if the primary color filter is used, red (R), green (from the image signal (combination of pixel signals of a plurality of colors) obtained by the CMOS image sensor 12 is obtained. The primary color separation unit that separates the primary color signals of G) and blue (B) can be omitted.

  Further, the CMOS image pickup device 12 includes an analog front end unit 12b including a preamplifier unit having a CDS processing unit and an AGC processing unit and an AD conversion unit around the imaging unit 12a including a large number of sensor units, and a CMOS imaging unit. A drive control unit 12 c that drives and controls the element 12 is taken into the CMOS image sensor 12. The analog front end unit 12b also includes a reference signal generation unit that supplies a reference voltage for AD conversion, an output circuit, and the like.

  In the following, a case where a CMOS image sensor, which is an example of an XY address type solid-state imaging device, is used as a device will be described as an example. The CMOS image sensor will be described on the assumption that all pixels are made of NMOS. However, this is merely an example, and the target device is not limited to a MOS imaging device. Embodiments described later are applied to all semiconductor device for physical quantity distribution detection in which a plurality of unit components that are sensitive to electromagnetic waves input from the outside such as light and radiation are arranged in a line or matrix. The same applies.

  Specifically, the digital still camera 1 is applied as a camera capable of capturing a color image in the still image capturing mode. In addition to the still image capturing mode, a moving image capturing mode is also prepared at a frame rate close to 30 frames / second (for example, 10 frames / second or more) using thinning-out reading.

  The solid-state imaging device 2 is configured by the solid-state imaging device 10 (in this example, the CMOS imaging device 12) in the camera module 3 itself. The solid-state imaging device 2 is provided as an imaging unit 12a, an analog front end unit 12b, and a drive control unit 12c formed on one semiconductor substrate.

  The drive control unit 12c has a control circuit function for reading out signals from the imaging unit 12a in a predetermined order. For example, the control circuit functions include a horizontal scanning unit (column scanning circuit) that controls column addresses and column scanning, a vertical scanning unit (row scanning circuit) that controls row addresses and row scanning, and an internal clock. And a drive signal operation unit (also referred to as a communication / timing control unit or a timing signal generation unit) having a function of generating a drive pulse for controlling each scanning unit and other functional units. The drive control unit 12c also includes a shutter control unit that generates a control pulse for an electronic shutter and controls a mechanical shutter as a functional unit unique to the present embodiment.

  Each element of these drive control units is integrally formed in a semiconductor region such as single crystal silicon using the same technology as the semiconductor integrated circuit manufacturing technology together with the imaging unit 12a, and is a solid-state imaging device which is an example of a semiconductor system (Imaging device).

  When the scanning unit and the drive signal operation unit, which are strongly related to the CMOS imaging device 12 (imaging unit 12a) to be used, are integrated on the same substrate as the imaging unit 12a, the handling and management of the members can be performed. It becomes simple. Moreover, since these are integrated with the optical system 5 as a module, the digital still camera 1 (completed product) can be easily manufactured.

  A CMOS sensor is an example of an XY address type solid-state imaging device. Signals can be extracted from a pixel at an arbitrary position by addressing, and signal charges obtained from the pixels are selected by a shift register in order. Unlike the CCD (Charge Coupled Device) type image sensor that reads out the pixel signals, the order in which the pixel signals are read out can be set relatively freely.

  In the digital still camera 1 including the CMOS image sensor 12 having such a configuration, pixel signals output from unit pixels are supplied to the analog front end unit 12b via vertical signal lines in units of rows. In the analog front end unit 12b, CDS processing and AGC processing are performed, and further converted into digital data by the AD conversion processing unit. The pixel data digitized by the AD conversion processing unit is transmitted to a horizontal signal line via a horizontal selection switch (not shown) driven by a horizontal selection signal from the horizontal scanning unit, and further input to an output circuit.

  With such a configuration, the imaging unit 12a in which light receiving elements (photoelectric conversion elements) as charge generation units (detection units) are arranged in a matrix form a vertical scanning unit (row scanning circuit) that controls row addresses and row scanning. ), The pixel signal is sequentially output for each vertical column for each row. An image corresponding to the imaging unit 12a in which light receiving elements (photoelectric conversion elements such as photodiodes) are arranged in a matrix, that is, a frame image, is shown as a set of pixel signals of the entire imaging unit 12a.

  The processing system of the digital still camera 1 is roughly composed of an optical system 5, a signal processing system 6, a recording system 7, a display system 8, and a control system 9. Needless to say, the camera module 3 and the main unit 4 are accommodated in an exterior case (not shown), and an actual product (finished product) is finished.

  The optical system 5 photoelectrically converts the collected light image and an imaging lens 50 having a mechanical shutter 52, a lens 54 for condensing a light image of the subject, and a diaphragm 56 for adjusting the light amount of the light image. It is comprised from the CMOS image pick-up element 12 converted into an electrical signal. The light L1 from the subject Z passes through the shutter 52 and the lens 54, is adjusted by the diaphragm 56, and enters the CMOS image sensor 12 with an appropriate brightness. At this time, the lens 54 adjusts the focal position so that an image composed of the light L <b> 1 from the subject Z is formed on the CMOS image sensor 12.

  When the mechanical shutter 52 is opened, external light is irradiated to a photoelectric conversion element (not shown) (for example, a photodiode) that constitutes the imaging unit 12a of the CMOS image sensor 12, and when the mechanical shutter 52 is closed, the external light irradiated to the photoelectric conversion element is blocked. It is done. Therefore, the exposure time (charge accumulation time) of the photoelectric conversion element can be started by opening the mechanical shutter 52, and the exposure time to the photoelectric conversion element can be ended by closing the shutter 52.

  Further, by applying an electronic shutter signal from the drive control unit 12c to the CMOS image sensor 12, the charge in the photoelectric conversion element can be thrown away to the substrate. That is, the exposure time of the photoelectric conversion element can be controlled also by the electronic shutter signal. If the mechanical shutter 52 and the electronic shutter are used in combination, the degree of freedom in controlling the exposure time is increased.

  The signal processing system 6 includes an amplification amplifier that amplifies an analog imaging signal from the CMOS image sensor 12, a preamplifier unit that includes a CDS processing unit and an AGC amplifier unit that reduce noise by sampling the amplified imaging signal, a preamplifier, and the like. A digital signal processor (Large Scale Integrated) comprising a digital signal processor (DSP) that performs predetermined image processing on an AD conversion unit that converts an analog signal output from the unit into a digital signal, and a digital signal input from the AD conversion unit An image signal processing unit 66 as a circuit (large-scale integrated circuit).

  The image signal processing unit 66 uses, for example, primary color separation processing that separates and synchronizes primary color signals of red (R), green (G), and blue (B) from complementary color imaging data, and pixel data using peripheral pixel data. Pixel interpolation processing to compensate for, vertical stripe noise correction processing to correct vertical stripe noise components caused by smear phenomenon and blooming phenomenon, white balance adjustment processing to take white balance (WB), gamma correction processing to adjust the gradation degree, Rather than using an optical zoom lens, resolution conversion processing (that is, electronic zoom processing) that enlarges or reduces an image electronically, that is, image data processing, or matrix calculation or encoding processing is performed to perform luminance data (Y) or color A YC signal generation process for generating data (C) is performed.

  Note that components unique to the present embodiment relating to image signal processing that involves processing for increasing the signal level, such as AGC processing, white balance control processing, or gamma correction processing, will be described in detail later.

  The image signal processing unit 66 configured by the DSP can be configured such that all processing of each functional part is performed by a digital processing circuit using dedicated hardware, and part of these functional parts is performed by software processing. It can also be set as the structure to perform.

  Although the mechanism for performing predetermined processing by software can flexibly cope with parallel processing and continuous processing, the processing time becomes longer as the processing becomes complicated, so that a reduction in processing speed becomes a problem. On the other hand, it is possible to construct an accelerator system with a higher speed by using a hardware processing circuit. Even if the processing is complicated, the accelerator system can prevent a reduction in processing speed and can obtain a high throughput.

  The recording system 7 includes a memory (recording medium) 72 that can be attached to and detached from a device such as a flash memory that stores image data, and the image data processed by the image signal processing unit 66 is encoded (compressed) into the memory 72. A CODEC (Compression / Decompression) 74 is recorded, read out, decoded (expanded), and supplied to the image signal processing unit 66.

  The display system 8 includes a DA (Digital / Analog) conversion unit 82 that converts the image signal processed by the image signal processing unit 66 into analog, and a liquid crystal (LCD) that functions as a finder by displaying an image corresponding to the input video signal. A video monitor 84 composed of a liquid crystal display (LCD), an organic EL (Electro Luminescence), and the like, and a video encoder 86 that encodes an analog image signal into a video signal suitable for the video monitor 84 in the subsequent stage. . Note that the DA conversion unit 82 and the video encoder 86 may be reversely arranged so that the encoding process is performed digitally. In this case, the video encoder 86 can be taken into the image signal processing unit 66.

  First, the control system 9 includes a synchronization signal generation unit 41 that generates a reference signal such as a horizontal synchronization signal SHD and a vertical synchronization signal SVD that defines a reference for driving a CMOS image sensor 12 that is an example of the solid-state image sensor 10, and a digital still. And a central control unit 92 including a CPU (Central Processing Unit) for controlling the entire camera 1. The synchronization signal generation unit 41 generates a horizontal synchronization signal SHD and a vertical synchronization signal SVD having a predetermined cycle based on an instruction from the central control unit 92 and supplies the generated signals to the solid-state imaging device 10 side 12c. The synchronization signal generator 41 and the central controller 92 constitute a reference signal supply unit and a timing control device.

  The control system 9 includes a ROM (Read Only Memory) 93a that is a read-only storage unit, a RAM (Random Access Memory) 93b that is an example of a volatile storage unit that can be written and read at any time, and a nonvolatile memory. An RAM (described as NVRAM) 93c, which is an example of a storage unit, and an EEPROM (Electrically Erasable and Programmable ROM) 93d, which is an example of a non-volatile storage unit that stores individual device data such as white spot position information and various adjustment data A storage unit (memory unit) 93 is provided. Various memories in the storage unit 93 excluding the central control unit 92 such as a CPU and the EEPROM 93d can be taken into the image signal processing unit 66 constituted by a DSP.

  In the above description, the “volatile storage unit” means a storage unit in which the stored contents are lost when the power of the digital still camera 1 is turned off. On the other hand, the “non-volatile storage unit” means a storage unit that maintains the stored contents even when the main power of the digital still camera 1 is turned off. Any memory device can be used as long as it can retain the stored contents. The semiconductor memory device itself is not limited to a nonvolatile memory device, and a backup power supply is provided to make a volatile memory device “nonvolatile”. You may comprise as follows. Note that the special application is not limited to a semiconductor memory element, but may be configured using an external medium such as a magnetic disk or an optical disk by using an external drive device.

  In the digital still camera 1 configured as such an electronic computer, when a series of processing is executed by software, a program configuring the software is installed from a recording medium (in this example, the ROM 93a). . This software includes an OS (operating system; basic software) running on the computer.

  The program for causing the central control unit 92 to execute a predetermined process may be distributed (acquired or updated) through any portable storage medium such as a non-volatile semiconductor memory card such as a CD-ROM or a flash memory. Alternatively, the program may be downloaded and acquired from a server or the like via a network such as the Internet or updated.

  The central control unit 92 reads the control program stored in the ROM 93a of the storage unit 93 constituted by a semiconductor memory or the like, and the entire digital still camera 1 based on the read control program or a command from the user. Control the operation and signal processing. A configuration is realized in which the digital still camera 1 is configured in software using a CPU or memory, that is, the digital still camera 1 is functioned in software using the function of a computer (electronic computer) such as a personal computer.

  In such a configuration, the central control unit 92 controls the entire system via the system bus 99. The ROM 93a stores data common to the apparatus such as a control program of the central control unit 92. The RAM 93b is configured by an SRAM (Static Random Access Memory) or the like, and stores program control variables, data for various processes, and the like. The RAM 93b includes an area for temporarily storing image data read by the solid-state imaging device 10, image data edited by a predetermined application program, image data read from the memory 72, and the like.

  The control system 9 also includes an exposure controller 94 that controls the shutter 52 and the diaphragm 56 so that the brightness of the image sent to the image signal processing unit 66 is maintained at an appropriate level, and the image signal processing unit 66 from the CMOS image sensor 12. The drive control unit 12c having a drive signal operation unit (timing generator; TG) for controlling the operation timing of each function unit up to and including keys and switches for the user to input shutter timing, zoom operation or other commands A portion 98 is provided.

  The central control unit 92 controls the image signal processing unit 66, the CODEC 74, the memory 72, the exposure controller 94, and the drive signal operation unit connected to the system bus 99 of the digital still camera 1.

  The digital still camera 1 includes automatic control mechanisms such as auto focus (AF), automatic gain adjustment, auto white balance (AWB), automatic exposure (AE), and gamma adjustment. These controls are processed using an output signal obtained from the CMOS image sensor 12. For example, the exposure controller 94 has its control value set by the central control unit 92 so that the brightness of the image sent to the image signal processing unit 66 is kept at an appropriate level, and controls the diaphragm 56 according to the control value. . Specifically, the central control unit 92 acquires an appropriate number of luminance value samples from the image held in the image signal processing unit 66, and the average value thereof falls within a predetermined appropriate luminance range. The control value of the diaphragm 56 is set so as to be within the range.

  In addition, as a function unique to the present embodiment, when executing image signal processing that involves processing for increasing the signal level, such as AGC processing, white balance control processing, or gamma correction processing, the preceding acquisition prior to the main acquisition of the signal is first performed. In the mode, in order to execute these processing functions by controlling the detection time (exposure / accumulation time), the detection time corresponding to the gain value for increasing the signal level is calculated for each pixel. Thereafter, in the main acquisition mode in which the main acquisition of the signal is performed, the detection time of each pixel is controlled based on the calculated individual detection time. This point will be described in detail later.

  The drive signal operation unit of the drive control unit 12c is controlled by the central control unit 92, and the operation of the image pickup unit 12a, the analog front end unit 12b (preamplifier unit or AD conversion unit) of the CMOS image pickup device 12, or the image signal processing unit 66 is controlled. Necessary timing pulses are generated and supplied to each unit. The operation unit 98 is operated when the user operates the digital still camera 1.

  An outline of a series of operations of the digital still camera 1 provided with such a CMOS image sensor 12 is as follows. First, the drive signal operation unit of the drive control unit 12c generates a drive pulse of a predetermined voltage level and inputs it to a predetermined terminal such as the imaging unit 12a or the analog front end unit 12b.

  When the subject Z is imaged, the optical image of the subject Z formed on the light receiving surface of the CMOS image sensor 12 via the imaging lens 50 (the shutter 52 and the lens 54) is received by each sensor unit including a photodiode. Is converted into a signal charge in an amount corresponding to the amount of incident light.

  The imaging unit 12a accumulates signal charges in a floating diffusion (not shown) on the output side of the sensor unit, converts the accumulated signal charges into a signal voltage, and is driven via an output circuit having a source follower configuration (not shown), for example. An image pickup signal (CCD output signal) Vout is output under the control of the drive pulse emitted from the signal operation unit.

  The voltage signals sequentially read from the CMOS image sensor 12, that is, R, G, and B pixel signals corresponding to the pixels are preamplified in the analog front end unit 12b based on the sample pulses from the drive signal operation unit. The unit is subjected to CDS processing, etc., converted into digital R, G, and B pixel data by the AD conversion unit, and then temporarily stored in the RAM 93b of the storage unit 93.

  The R, G, and B pixel data stored in the RAM 93b are subjected to a synchronization process, a gamma correction process, and the like by the image signal processing unit 66, and then the luminance data Y and the color (chroma) data U, V. (Or Cr, Cb) (collectively referred to as YC data) and temporarily stored in the RAM 93b of the storage unit 93.

  The display system 8 can display a through image, a captured still image, and the like by reading the YC data stored in the RAM 93b and outputting the YC data to a video monitor 84 made of liquid crystal or the like.

  The YC data after photographing is compressed into a predetermined format such as JPEG (Joint Photographic Experts Group) by a CODEC 74 having a compression / decompression function, and then recorded on a recording medium such as the memory 72. Further, in the playback mode, the image data recorded in the memory 72 or the like is decompressed by the CODEC 74 and then output to the video monitor 84 to display the playback image.

<Configuration example of CMOS image sensor>
FIG. 2 is a schematic configuration diagram of a CMOS image sensor 12 which is an example of the solid-state image sensor 10. In the following description, it is assumed that the CMOS image pickup device is composed of all NMOS pixels. However, this is an example, and the target device is not limited to the NMOS type imaging device. Embodiments described later are applied to all semiconductor device for physical quantity distribution detection in which a plurality of unit components that are sensitive to electromagnetic waves input from the outside such as light and radiation are arranged in a line or matrix. The same applies.

  In the CMOS image sensor 12, unit pixels including a light receiving element (photoelectric conversion element such as a photodiode) as a detection unit (not shown) that outputs a signal corresponding to the amount of incident light are arranged in a square lattice of rows and columns (ie, It has an imaging unit (in a two-dimensional matrix), the signal output from each unit pixel is a voltage signal, and a CDS (Correlated Double Sampling) processing function unit and other functional units are provided for each vertical column. It is of the column type provided.

  That is, as shown in FIG. 2, the CMOS imaging device 12 includes an imaging unit (pixel unit) 410 in which a plurality of unit pixels 403 (an example of unit constituent elements) are arranged in rows and columns (in a two-dimensional matrix). (Corresponding to the imaging unit 12a in FIG. 1), a so-called area sensor, a drive control unit 407 (corresponding to the drive control unit 12c in FIG. 1) provided outside the imaging unit 410, and columns arranged in each vertical column A column processing unit 420 (corresponding to the analog front end unit 12b in FIG. 1) having a signal processing unit (denoted as column AD in the figure) 422 is provided.

  As the drive control unit 407, for example, a horizontal scanning unit 412 and a vertical scanning unit 414 are provided. Further, as another component of the drive control unit 407, operation of a drive signal for supplying a control pulse at a predetermined timing to each functional unit of the CMOS image sensor 12 such as the horizontal scanning unit 412, the vertical scanning unit 414, or the column processing unit 420. 416 (an example of a read address controller) is provided.

  Each element of the drive control unit 407 is integrally formed in a semiconductor region such as single crystal silicon using a technique similar to the semiconductor integrated circuit manufacturing technique together with the imaging unit 410, and is an example of a semiconductor system. It is configured as an element (imaging device).

  In FIG. 2, some of the rows and columns are omitted for the sake of simplicity, but in reality, tens to thousands of unit pixels 403 are arranged in each row and each column of the imaging unit 410. . Although illustration is omitted, the image pickup unit 410 is formed with a color separation filter having a predetermined color coding for each pixel. Although not shown, each pixel of the imaging unit 410 includes a photoelectric conversion element such as a photodiode and a transistor circuit (see FIG. 4 described later).

  On the signal path between the column processing unit 420 and the horizontal scanning unit 412, a load transistor unit including a load MOS transistor (not shown) having a drain terminal connected to each vertical signal line 418 is arranged (a diagram to be described later). 4), a load control unit (load MOS controller) for driving and controlling each load MOS transistor is provided.

  The unit pixel 403 includes a vertical scanning unit 414 via a vertical control line 415 for vertical column selection, a shutter control unit 414a via a shutter control line 415s for electronic shutter control, and a vertical signal line 418. Each is connected to a column processing unit 420. The horizontal scanning unit 412 and the vertical scanning unit 414 are configured to include a shift register, for example, and start a shift operation (scanning) in response to a drive pulse supplied from the drive signal operation unit 416. The vertical control line 415 includes various pulse signals for driving the unit pixels 403 in units of rows.

  The horizontal scanning unit 412 has a function of a reading scanning unit that reads out pixel data from the column signal processing unit 422. For example, the horizontal scanning unit 412 sequentially selects the column signal processing unit 422 of the column processing unit 420 in synchronization with the low-speed clock, and guides the signal to a horizontal signal line (horizontal output line). For example, a horizontal address setting unit 412a that defines a horizontal readout column (horizontal address) and each pixel data of the column signal processing unit 422 in accordance with a horizontal address line in accordance with the readout address defined by the horizontal address setting unit 412a. And a horizontal driving portion 412b led to 428.

  Although not shown, the horizontal address setting unit 412a includes a shift register or a decoder. The horizontal address setting unit 412a sequentially selects pixel data from the column signal processing unit 422, and selects the selected pixel data as a horizontal signal line 428. It has a function as a selection means to output to The horizontal signal line 428 corresponds to the number of bits n (n is a positive integer) handled by the AD conversion function of the column signal processing unit 422, for example, 10 (= n). Ten are arranged.

  The vertical scanning unit 414 selects a row of the imaging unit 410 and supplies a control pulse (line shift pulse) necessary for the row. For example, a vertical address setting unit 414a that generates a drive pulse that defines a vertical readout row in a normal readout operation, and row control for a unit pixel on the readout address (in the row direction) defined by the vertical address setting unit 414a. And a vertical driving unit 414b for driving the unit pixel 403 by supplying a driving pulse to the line. The vertical address setting unit 414a and the vertical drive unit 414b constitute a normal read control unit that supplies a drive signal for performing a normal read operation to the unit pixel 403.

  Although not shown, the vertical address setting unit 414a includes a vertical shift register or decoder that performs basic control of a row from which a signal is read. The vertical shift register or decoder is for selecting each pixel in units of rows when reading out pixel information from the imaging unit 410, and constitutes a signal output row selection means together with the vertical drive unit 414b of each row.

  Generally, the vertical address setting unit 414a can be configured to perform electronic shutter control in units of rows in addition to the row from which a signal is read. Unlike a normal CMOS sensor, control is performed in units of pixels instead of in units of rows. For this reason, in the present embodiment, the vertical scanning unit 414 includes a dedicated shutter timing control unit 414c separately from the vertical address setting unit 414a in order to perform electronic shutter control in units of pixels. The shutter timing control unit 414c and the vertical drive unit 414b constitute a shutter control unit that supplies a drive pulse for realizing the electronic shutter function to the unit pixel 403 for each pixel.

  The shutter timing control unit 414c is for selecting shutter pixels in units of pixels when performing an electronic shutter operation, and constitutes an electronic shutter pixel selection unit together with the vertical drive unit 414b of each row. When driving for the electronic shutter, the shutter timing control unit 414c sets the shutter pixel position by performing row selection of unit pixels and column selection as in the normal operation of the vertical address setting unit 414a. By adjusting the time interval of the shutter pixels with the readout row selected by the vertical address setting unit 414a, the exposure time (accumulation time) to the photoelectric conversion element (detection unit) of the imaging unit 410 is set in units of pixels. Adjust (details will be described later).

  Note that each pulse signal supplied from the vertical scanning unit 414 to the imaging unit 410 is arranged in either the horizontal direction or the vertical column direction with respect to the unit pixel 403 as long as the driving for each vertical column or horizontal column is possible. In other words, the physical wiring method of the drive clock line for applying the pulse signal is arbitrary.

  Although not shown, the drive signal operation unit 416 includes a functional block of a timing generator TG (an example of a read address control device) that supplies a clock necessary for the operation of each unit and a pulse signal at a predetermined timing, and an input clock via a terminal 401a. A communication interface functional block that receives data instructing CLK0, an operation mode, and the like, and outputs data DATA including information of the CMOS image sensor 12 via a terminal 401b. In addition, the horizontal address signal is output to the horizontal address setting unit 412a and the vertical address signal is output to the vertical address setting unit 414a, and the address setting units 412a and 414a select the corresponding row or column in response thereto.

  The drive signal operation unit 416 may be provided as another semiconductor integrated circuit independently of other functional elements such as the imaging unit 410 and the horizontal scanning unit 412. In this case, an imaging device which is an example of a semiconductor system is constructed by the imaging device including the imaging unit 410 and the horizontal scanning unit 412 and the drive signal operation unit 416. This imaging device may be provided as an imaging module in which peripheral signal processing circuits, power supply circuits, and the like are also incorporated.

  The column processing unit 420 as the analog front end unit 12b is configured to include a column signal processing unit 422 for each vertical column, and receives a signal of pixels for one row and processes the signal. As an example, each column signal processing unit 422 is provided with a signal transfer switch and a storage capacitor. Further, the column processing unit 420 has a function of noise removal means using a CDS (Correlated Double Sampling) process, and two samples such as a sample pulse SHP and a sample pulse SHD given from the drive signal operation unit 416. Based on the pulse, processing is performed on the voltage mode pixel information input via the vertical signal line 418 to obtain a difference between the signal level immediately after the pixel reset (noise level: 0 level) and the true signal level. Thus, noise signal components called fixed pattern noise (FPN) and reset noise due to fixed variation for each pixel are removed.

  The column processing unit 420 includes an AGC (Auto Gain Control) processing unit having a signal amplification function, an ADC (Analog Digital Converter) processing unit, etc. for each column, that is, a column signal processing unit, following the CDS processing function unit. It is provided for every 422.

  The details of the configuration of the AD conversion processing unit will be omitted, but as an example, a ramp-like reference signal (reference voltage) is supplied to the comparator (voltage comparator) and at the same time counting (counting) with the clock signal is performed. The analog conversion is started until the pulse signal is obtained by comparing the analog pixel signal input via the vertical signal line with the reference signal, thereby performing AD conversion.

  At this time, it is possible to perform CDS processing together with AD conversion by devising the circuit configuration, and the signal level (noise) immediately after the pixel reset is applied to the voltage mode pixel signal inputted through the vertical signal line. Level) and the true signal level (depending on the amount of received light). Thereby, noise signal components called fixed pattern noise (FPN) and reset noise can be removed.

  A voltage signal indicating pixel information processed by the column processing unit 420 is read out at a predetermined timing via a horizontal selection switch (not shown) driven by a horizontal selection signal from the horizontal scanning unit 412 and applied to the horizontal signal line 428. Then, the signal is transmitted to the output circuit 429 connected to the rear end of the horizontal signal line 428.

  The output circuit 429 supplies each pixel data output from the imaging unit 410 through the horizontal signal line 428 to an external circuit (not shown) (corresponding to the image signal processing unit 66 in FIG. 1) via the output terminal 401c. For example, the output circuit 429 may perform only buffering, or may perform black level adjustment, column variation correction, color-related processing, or the like before that.

  That is, in the column type CMOS image sensor 12 of the present embodiment, the output signal (voltage signal) from the unit pixel 403 is in the order of the vertical signal line 418 → column processing unit 420 → horizontal signal line 428 → output circuit 429. Is output. In the drive, pixel output signals for one row are sent in parallel to the column processing unit 420 via the vertical signal line 418, and the pixel data (imaging data D0) after CDS processing, AGC processing, and AD conversion is the horizontal signal line 428. To be output serially via

  In the above description, an example in which the column signal processing unit 422 performs CDS processing, AGC processing, and AD conversion has been described. For example, the column signal processing unit 422 performs only CDS processing and performs horizontal signal lines as analog pixel signals. As shown by the dotted line in the figure, functional elements such as an AGC unit 502, an AD conversion unit 504, and a digital amplifier unit (not shown) are configured to be built in the subsequent stage of the output circuit 429 and in the chip, The imaging data D0 may be output from the output terminal 401d.

<About electronic shutter function>
FIG. 3 is a diagram for explaining the electronic shutter function used in the present embodiment. As described above, in this embodiment, the exposure time (accumulation time) for the photoelectric conversion element (detection unit) of the imaging unit 410 is adjusted in units of pixels.

  The vertical scanning unit 414 is a functional unit that generates address information (specifically, transfer gate pulses TGv and TGh as drive pulses) for designating the shutter pixel position, and is a shutter target unit separately from the vertical address setting unit 414a. A shutter timing control unit 414c having a vertical shutter timing control unit 414cv for designating a row address of a pixel (shutter pixel) and a horizontal shutter timing control unit 414ch for designating a column address of a unit pixel to be shuttered is provided.

  The vertical shutter timing control unit 414cv supplies a drive pulse φTGv designating a row address to all unit pixels 403 in the same row, and the horizontal shutter timing control unit 414ch receives a drive pulse φTGh designating a column address in the same column. A two-dimensional matrix wiring configuration is adopted so that all unit pixels 403 are supplied. As a result, the row and column unit pixels 403 designated by the drive pulses φTGv and φTGh are designated as shutter pixels.

  When a CMOS image sensor 12 is used as the solid-state image sensor 10, generally, from the basic operation method, a pixel that has output a signal starts accumulation of signal charges obtained by photoelectric conversion again from that point. . For this reason, the accumulation period is shifted in accordance with the scanning timing of the imaging surface, that is, the accumulation period is shifted by the scanning time for each scanning line, which is so-called line exposure. Unlike the CCD type, it is a global exposure (Global Exposure) that accumulates the light incident on the photoelectric conversion element during the same period as signal charge and reads it from all the pixels to the vertical CCD at the same time to satisfy the simultaneous storage. Absent.

  Here, for example, as shown in FIG. 3, in the imaging region (imaging units 12a and 410), the readout row n and the shutter rows ns (ns1, ns2,..., Nsv) are expressed as Δs (Δs1, Δs2,. ) Consider a case where only lines are separated.

  Since the accumulation of signal charges starts again after the pixels in the target column of the plurality of rows ns1, ns2,..., Nsv that have received an instruction from the electronic shutter are reset, for example, the scanning direction of the imaging surface is from top to bottom. In this case, the time difference between the row n and the row n + Δs has a predetermined relationship between the frame rate and the number of scanning lines. By adjusting the interval between the readout row n and the shutter row ns, The accumulation time can be changed using the line period (one horizontal scanning period) as an adjustment unit.

  Here, in the conventional CMOS sensor, electronic shutter control is performed in units of rows by setting one shutter row ns. In this embodiment, the shutter timing control unit 414c sets the shutter row ns to ns1, n2. ns2,..., nsv are set as necessary. At this time, in each column, one unit pixel 403 is a shutter pixel, but unit pixels 403 having the same accumulation time have the same shutter row position.

  For the readout row n at a certain point set by the vertical address setting unit 414a, the shutter timing control unit 414c applies any one of the pixels in all columns (H1, H2,..., Hh) except the readout row n. Pixels are reset by setting a shutter row ns at a row position, that is, a position (time point) separated by Δs (Δs1, Δs2,..., Δsv) rows. This reset operation can be realized by sweeping away the charge accumulated in the photoelectric conversion element before the shutter timing, and in the case of the CMOS image sensor 12, it can be realized by turning on the transfer gate (details will be described later).

  The time until the pixels of the shutter rows ns1, ns2,..., Nsv are next set to the readout row n by the vertical address setting unit 414a is the accumulation time, that is, the readout row n and the shutter rows ns1, ns2,. Is the accumulation time. As a result, the accumulation time can be controlled for each pixel.

  By utilizing the characteristics of line exposure that the CMOS image sensor 12 has, the electronic shutter drive pulses φTGv and φTGh are individually supplied to each unit pixel 403, whereby the time difference between the read row n and the shutter row n + Δs can be calculated for each column. Each unit pixel 403 can be set, and the accumulation time can be easily controlled for each pixel. The drive pulse φTGv can be supplied in common to the same row and the drive pulse φTGh can be supplied in common to the same column, and the number of wires for the drive pulses φTGv and φTGh for each unit pixel 403 can be small. This is very different from the CCD type for global exposure.

  Although FIG. 3 shows an example in which the exposure time is adjusted within one field, the exposure time can be adjusted using a plurality of fields. In this case, since each row is normally read once in one field (about 1/60 seconds), for example, the exposure time corresponding to the white balance gain among the R, G, B color pixels is the shortest. For the color pixel, the exposure time is not adjusted (equivalent to setting the white balance gain to 1), and only the remaining color pixels are adjusted to a longer time so that the exposure time matches the white balance gain ( Equivalent to making the white balance gain larger than 1).

<Relationship between unit pixel circuit configuration example and drive circuit>
FIG. 4 is a diagram for explaining a relationship between a configuration example of the unit pixel 403 and a driving circuit (vertical scanning unit 414 in FIG. 2, particularly the shutter timing control unit 414c) related to the electronic shutter function for driving the unit pixel 403. is there. The unit pixel 403 has a configuration in which a floating diffusion (FDA) 438 having a diffusion layer having a parasitic capacitance as a main part is used as a charge storage unit, and the unit pixel 403 includes four transistors (TRansistor) in the unit pixel. It has a transistor type pixel configuration (hereinafter referred to as a 4TR configuration).

  As shown in the figure, the unit pixel 403 has a photoelectric conversion function for converting light into charges, and a charge generation unit 432 that has each function of a charge storage function for storing the charge. Changes in potentials of a read selection transistor 434 as an example of a transfer unit (charge reading unit / transfer gate unit / read gate unit), a reset transistor 436 as an example of a reset gate unit, a vertical selection transistor 440, and a floating diffusion 438 It has an amplifying transistor 442 having a source follower configuration, which is an example of a detection element to detect.

  The amplifying transistor 442 constituting the unit pixel 403 is connected to each vertical signal line 453 (corresponding to the vertical signal line 418 in FIG. 2), and the vertical signal line 453 is a load that forms a constant current source In for each vertical column. A load control signal Load from a load control unit (not shown) is commonly input as a constant bias to the gate terminal of each load MOS transistor 427 and connected to the drain of the MOS transistor 427. The load MOS transistor 427 connected to the transistor 442 continues to pass a predetermined constant current. In other words, the load MOS transistor 427 has its gate biased at a constant potential, and forms a signal output to the vertical signal line 453 by assembling the source follower with the amplification transistor 442 in the selected row.

  The horizontal wiring is generally common to pixels in the same row, and during normal driving without electronic shutter control, all unit pixels 403 in the same row are simultaneously driven and controlled by the vertical driving unit 414b of the vertical scanning unit 414. For example, a transfer drive buffer 452, a reset drive buffer 454, and a selection drive buffer 456 are accommodated in the vertical drive unit 414b.

  The read selection transistor 434 is driven by a transfer drive buffer 452 via a transfer wiring (read selection line) 455. The reset transistor 436 is driven by a reset driving buffer 454 via a reset wiring 456. The vertical selection transistor 440 is driven by the selection drive buffer 456 via the vertical selection line 452.

  Further, as a component unique to this embodiment, at least the transfer gate pulse for adjusting the exposure time is supplied from the vertical scanning unit 414 to the transfer drive buffer 452 for each pixel instead of for each row. If necessary, the reset gate pulse RG for adjusting the exposure time can be supplied to the reset transistor 436 for each pixel.

  For example, in addition to the vertical address setting unit (normal scanning) 414a for setting a readout row related to normal scanning, the vertical scanning unit 414 has a vertical address as a dedicated functional unit for controlling the accumulation time (exposure time) for each pixel. Similarly to the normal operation in the setting unit 414a, a vertical shutter timing control unit 414cv that outputs a transfer gate pulse TGv for selecting a row of a shutter target unit pixel (shutter pixel), and a horizontal direction ( A horizontal shutter timing control unit 414ch that outputs a transfer gate pulse TGh for selecting the position of the column), and designates the row interval between the normal readout row and the shutter pixel (that is, the row position of the shutter pixel). In addition, a shutter timing control unit 414c for specifying the column position of the shutter pixel is provided.

  The horizontal shutter timing control unit 414ch activates the transfer gate pulse TGh so that all the unit pixels 403 for one row operate simultaneously when the vertical address setting unit 414a instructs a normal readout row.

  Further, in accordance with the configuration of the shutter timing control unit 414c, the vertical drive unit 414b transfers the transfer selection buffer 434 for driving the normal selection row and the selected row of the shutter pixels at least for the transfer drive buffer 452. 452v and a transfer drive buffer 452h for driving the read selection transistor 434 with respect to the column position of the shutter pixel.

  The transfer drive buffer 452v uses a read pulse (transfer gate pulse) TG for designating a normal read row from the vertical address setting unit 414a and a transfer gate pulse TGv for designating a row of shutter pixels from the vertical shutter timing control unit 414cv. On the other hand, a logical sum circuit is configured to operate.

  The unit pixel 403 receives the drive pulses TGv and TGh from the transfer drive buffers 452v and 452h, and supplies a drive pulse to the gate of the read selection transistor 434 of the shutter pixel in each of the vertical and horizontal directions. For each pixel.

  The switch unit 439 includes a horizontal readout pulse (transfer gate pulse) TG for designating a normal readout row from the vertical address setting unit 414a, a transfer gate pulse TGv for designating a row of shutter pixels from the vertical shutter timing control unit 414cv, and horizontal. An AND circuit is configured to operate on the transfer gate pulse TGh that designates the column position (particularly for the shutter pixel) from the shutter timing control unit 414ch. Here, the phrase “especially for shutter pixels” is because the effect of the substantial logical product is to designate the row position and the column position of the shutter pixels.

  The unit pixel 403 includes a pixel signal generation unit 405 having an FDA (Floating Diffusion Amp) configuration including a floating diffusion 438 which is an example of a charge injection unit having a function of an amplification transistor 442 and a charge storage unit. ing. The pixel signal generation unit 405 is an example of a unit signal generation unit that generates a pixel signal as a unit signal. The floating diffusion 438 has a diffusion layer having parasitic capacitance in the main part.

  In the reset transistor 436 in the pixel signal generation unit 405, the source is connected to the floating diffusion 438, the drain is connected to the power supply VDD, and the reset pulse RST is input from the reset drive buffer 454 to the gate (reset gate RG).

  Here, the unit pixel 403 is a pixel with a 4TR configuration that includes a selection transistor inserted in series with the amplification transistor 442 and selects a pixel. Of the amplification transistor 442 and the vertical selection transistor 440, The vertical selection transistor 440 is a type on the vertical signal line 453 side.

  That is, the amplifying transistor 442 has a drain connected to the power supply VDD (for example, 2.5 V), a source connected to the drain of the vertical selection transistor 440, and a gate connected to the floating diffusion 438. The vertical selection transistor 440 has a gate (particularly referred to as a vertical selection gate SELV) connected to the vertical selection line 452 and a source connected to the vertical signal line 453 via the pixel line 451. A vertical selection signal is applied to the vertical selection line 452 from the selection drive buffer 456.

  In such a configuration, since the floating diffusion 438 is connected to the gate of the amplifying transistor 442, the amplifying transistor 442 sends a signal corresponding to the potential of the floating diffusion 438 (hereinafter referred to as FD potential) via the pixel line 451. To the vertical signal line 453.

  The reset transistor 436 resets the floating diffusion 438. The read selection transistor (transfer transistor) sv transfers the signal charge generated by the charge generation unit 432 to the floating diffusion 438. Many pixels are connected to the vertical signal line 453. To select a pixel, the vertical selection transistor 440 is turned on only for the selected pixel. Then, only the selected pixels in the same row are connected to the vertical signal line 453, and the signals of the selected pixels in the same row are simultaneously output to the vertical signal line 453.

  Although not shown in the drawing, the amplifying transistor 442 may be of the type in which the amplifying transistor 442 is on the vertical signal line 453 side of the amplifying transistor 442 and the vertical selecting transistor 440.

  Here, in the present embodiment, the shutter timing control unit 414c that controls the charge accumulation time for each pixel has a predetermined column position on a predetermined row via the transfer drive buffer 452 and the transfer wiring (read selection line) 455. The readout selection transistor 434 of the unit pixel 403 is controlled. Although the readout row is controlled in units of rows, the shutter pixels can be controlled not only in the row position but also in the horizontal (column; column) position, so that the exposure time can be controlled for each pixel.

  The storage timing is controlled by dividing the assigned line by the shutter timing control unit 414c for controlling the accumulation time and the vertical address setting unit 414a for controlling the address position (row) for normal reading. The vertical column direction can be used for setting the exposure time, and in a plurality of pixels on the readout row, one pixel can be assigned for long-time accumulation and the other pixel can be assigned for short-time accumulation.

  By doing so, the accumulation time can be freely set for each pixel, so that the short time accumulation side also has a degree of freedom in the accumulation time. As a result, the degree of freedom in setting the charge accumulation time is greatly expanded, and the usability is improved. However, as described with reference to FIG. 3, the control of the exposure / accumulation period (shutter period) is performed with the line cycle as one adjustment unit.

  Further, such accumulation time setting is not performed only by the vertical address setting unit 414a, but a shutter timing control unit 414c dedicated to the electronic shutter is provided to control the accumulation time, so that the control is easy. Become.

  In addition, instead of performing exposure time control in units of rows, for example, after reading out the pixel signal on the long-time accumulation side within the same horizontal period, the short-time accumulation is performed, and the pixel on the short-time accumulation side is immediately It is also possible to read out the signal. However, in this case, since the short-time accumulation side has an accumulation time of one horizontal period (for example, 64 microseconds) or less, there is no flexibility in setting the accumulation time on the short-time side.

<Driving method of unit pixel>
FIG. 5 and FIG. 6 are timing charts for explaining a method of acquiring the pixel signal (unit signal output from the unit pixel 403) by driving the unit pixel 403 shown in FIG. In particular, regarding the electronic shutter, a case where exposure time control by the electronic shutter is performed with a line period as one adjustment unit will be described. FIG. 5 shows a case where the mechanical shutter 52 is not used together in controlling the accumulation time, and FIG. 6 shows a case where the mechanical shutter 52 is used together.

  In the 4TR configuration shown in FIG. 4, the reset transistor 436 resets the floating diffusion 438. Specifically, the floating diffusion 438 is reset by discarding the signal charges (here, electrons) of the floating diffusion to the power supply wiring.

  The read selection transistor (transfer transistor) sv transfers the signal charge generated by the charge generation unit 432 to a floating diffusion 438 which is an example of a charge storage unit.

  Since the floating diffusion 438 is connected to the gate of the amplifying transistor 442, which is an example of the unit signal generator, the amplifying transistor 442 is a signal corresponding to the potential of the floating diffusion 438 (hereinafter also referred to as FD potential) (in this example). Voltage signal) is output to the vertical signal line 453 which is an example of the output signal line via the pixel line 451 when the vertical selection transistor 440 is on. That is, a large number of pixels are connected to the vertical signal line 453. To select a pixel, the vertical selection transistor 440 is turned on only for the selected pixel. Then, only the selected pixel is connected to the vertical signal line 453, and the signal of the selected pixel is output to the vertical signal line 453.

  Specifically, as shown in the timing chart of FIG. 5, the read pulse (transfer gate pulse) TG becomes active (high level in this example), drives the read selection transistor 434, and enters the charge generation unit 432 Signal charges generated by photoelectric conversion of light are transferred to a floating diffusion 438 that functions as an accumulation node and read out.

  Here, the signal charge generated by photoelectric conversion of the light incident on the charge generation unit 432 is accumulated in the charge generation unit 432 until the read selection transistor 434 is turned on. Therefore, the exposure time can be controlled by the electronic shutter operation by adjusting the timing (ts) for turning on the read selection transistor 434 before reading the pixel signal. The exposure time for the unit pixel 403 of the imaging unit 12a is adjusted by adjusting the time interval of the shutter pixel between the shutter timing (for example, ts1, ts2) and the readout row selected as usual (corresponding to t10 to t20). (Accumulation time) can be adjusted.

  At the shutter timing (for example, ts1, ts2), a transfer gate pulse TG for adjusting the exposure time is supplied from the shutter timing control unit 414c to the transfer drive buffer 452, and is generated by the charge generation unit 432 before that. The signal charges thus transferred are transferred to the floating diffusion 438. By supplying a reset gate pulse RG for adjusting the exposure time to the reset transistor 436 as necessary, shutter timing (for example, ts1, ts2) prior to pixel signal reading processing (corresponding to t10 to t20). Sometimes the signal charge transferred to the floating diffusion 438 is swept away.

  When reading out pixel signals in the readout row (corresponding to t10 to t20), the first thing to be done in the horizontal scanning line blanking period is to activate the vertical selection pulse SEL (high level in this example) and turn on the vertical selection transistor 440. (T10), the output of the amplifying transistor 442 in the readout row is connected to the vertical signal line 453 so that the charge of the floating diffusion 438 can be detected by the amplifying transistor 442, and the vertical signal line 453, the current source In ( The load MOS transistor 427) and the amplifying transistor 442 constitute a source follower circuit. The potential of the vertical signal line 453 follows the potential fluctuation of the floating diffusion 438. As a result, only the potential determined by the gate potential of the amplifying transistor 442 corresponding to the charge amount of the floating diffusion 438 is transmitted to the vertical signal line 453.

  Also, with the start of the horizontal scanning line blanking period, the pixel signal generation unit 5 is first reset to the reference voltage in a state where the signal charge Qsig is accumulated in the charge generation unit 432 after the shutter timing (for example, ts1, ts2). That is, the reset gate pulse RG is activated (high level in this example) (t11), and the reset transistor 436 is turned on to discharge the dark current integrated value accumulated in the floating diffusion 438. As a result, the floating diffusion 438 is set to the power supply voltage value (Vdd).

  During the electronic shutter operation, the signal charge swept out at the shutter timing (for example, ts1, ts2) is present in the floating diffusion 438, but that amount is also swept out. Therefore, it is not essential to sweep away the signal charge transferred to the floating diffusion 438 at the shutter timing (for example, ts1, ts2) immediately after the shutter timing (for example, ts1, ts2). When the reset gate pulse RG is inactive (low level in this example) (t12), the potential of the floating diffusion 438 slightly drops due to coupling.

  At this time, a sample pulse SHP is output from the drive control unit 12c and is supplied to the gate of the shift transistor forming the CDS function unit in the analog front end unit 12b, and each shift transistor is turned on. That is, the clamp pulse SHD is supplied from the drive control unit 12c and supplied to the gate of the clamp transistor forming the CDS function unit, each clamp transistor is turned on, and the reset level Srst is detected (t14).

  Next, the read selection transistor 434 for the charge generation unit 432 is driven to read the signal component So corresponding to the signal charge Qsig from the charge generation unit 432. That is, the transfer gate pulse TG is set to the high level (t16), the reading selection transistor 434 is turned on, and the signal charge Qsig stored in the charge generation unit 432 is transferred to the floating diffusion 438. The charge amount of the signal charge Qsig transferred to the floating diffusion 438 is detected by the amplifying transistor 442, and a potential corresponding to the charge amount is generated and transmitted to the vertical signal line 453.

  Thereafter, the clamp pulse SHD is supplied from the drive control unit 12c (t18), the clamp transistor is turned on, and the pixel signal level Ssig corresponding to the signal charge Qsig detected by the charge generation unit 432 is detected.

  Here, the column processing unit of the analog front end unit 12b can detect the true signal component So by removing the offset component by taking the difference between the reset level Srst and the pixel signal level Ssig. It is possible to remove fixed pattern noise for each pixel.

  After the signal charge transfer is completed and a sufficient time has passed, the vertical selection pulse SEL is made inactive (low level in this example) (t20).

  As shown in FIG. 6, when the mechanical shutter 52 is used in combination, the shutter 52 can be opened within one horizontal period (HS; line cycle). Therefore, when the exposure time control is performed only with the electronic shutter, the line cycle must be set as one adjustment unit. On the other hand, when the mechanical shutter 52 is used together, the degree of freedom in controlling the exposure time is increased. The exposure time can be finely adjusted.

  However, it is necessary to consider that the effect of the mechanical shutter 52 contributes to all pixels. For example, with respect to AGC adjustment and gamma correction, only the white balance adjustment can be performed using signal processing with amplification processing for increasing the signal level of the pixel signal output from the solid-state imaging device 10 as in the past. Of controlling exposure time so that white balance can be achieved while avoiding (suppressing) the problem of causing S / N degradation due to amplification of noise accompanied by processing involving increasing the signal level of Consider the case of adopting. In this case, among the R, G, and B color pixels, when the exposure time is the same, the color pixel having the smallest signal level is used in combination with the mechanical shutter 52 to finely adjust the exposure time within the line period. do it.

<< Signal processing function >>
FIG. 7 is a block diagram that focuses on an image signal processing function that involves a process of increasing the signal level in the digital still camera 1.

  As shown in the figure, the signal processing system 6 includes a signal processing system 6a in the preceding stage including a preamplifier unit 62 having a CDS processing unit 62a and an AGC processing unit 62b, and an AD conversion unit 64, and an image processing unit (DSP) 66. And a rear-stage signal processing system 6b.

  In the AGC processing unit 62b, the sensitivity variation (sensitivity variation) of each photoelectric conversion element (charge detection unit) of the CMOS image sensor 12 is corrected, and the gain is adjusted so that the AD conversion unit 64 has an appropriate input level. To do.

  As a signal processing function in the subsequent signal processing system 6b, that is, the image processing unit (DSP) 66, a digital clamping unit 200 that clamps a black reference of imaging data acquired by the solid-state imaging device 10 and digitized by the AD conversion unit 64. Extracting R, G, and B primary color data from the image data clamped by the digital clamp unit 200 and synchronizing the primary color separation / synchronization processing unit 202, and pixel interpolation that supplements pixels using peripheral pixel data The processing unit 204 includes a white balance amplifier unit 210 that is an example of a color signal amplification unit that can adjust the gain of pixel data. Note that the arrangement order of the pixel interpolation processing unit 204 and the white balance amplifier unit 210 may be reversed.

  The primary color separation / synchronization processing unit 202 converts digital image signals supplied from the digital clamp unit 200 to R (red), G (green), and B (blue) when the solid-state imaging device 10 uses a complementary color filter. ) Are R signals Sr1, G signals Sg1, and B signals Sb1, which are primary color signals, and these are supplied to the pixel interpolation processing unit 204.

  In order to generate an image file having the same number of pixels as the number of pixels in the solid-state imaging device 10 (CMOS imaging device 12), the pixel interpolation processing unit 204 estimates a color component that does not exist at a certain pixel position from surrounding pixels and outputs a pixel signal. compensate. For example, since R signal components can be acquired from R (red) pixels, but G (green) and B (blue) signal components cannot be acquired, each pixel signal of the interpolated pixels G and B is obtained from surrounding pixels. Get guessed. In this case, for example, the signal component of the interpolated pixel can be obtained by a simple average of the signals of the peripheral pixels, but it is more preferable to perform adaptive processing in consideration of the resolution and the false signal.

  As the white balance amplifier 210, white balance amplifiers 210R, 210G, and 210B are individually provided for each color. When a general subject is imaged, various color components are present randomly on the entire screen. Considering that if all the color components of the entire screen are integrated, each element of the color signal (for example, R, G, B) will be extracted approximately equally, or the integrated value of the color difference component will be zero. The amplifier gain is adjusted to approach the control target, that is, the achromatic color. That is, the level of each RGB pixel is adjusted so that the output video is white (R: G: B = 1: 1: 1) when white is photographed with the light source at the time of photographing. At this time, at least one color is accompanied by a process of amplifying the signal level.

  The image signal processing unit 66 performs luminance signal processing and color signal processing on the primary color data R, G, and B acquired by the white balance amplifier unit 210 to obtain luminance data Y (or lightness data L) and two kinds of data. A signal processing unit 220 that converts to color data U and V and outputs the data is provided. Each data Y, U, V generated by the signal processing unit 220 is sent to the recording system 7 for image recording, or sent to the display system 8 for display output.

  For example, the signal processing unit 220 includes a gamma correction unit 222 and a color difference matrix unit 224. Of course, this configuration example is an example, and includes components other than these processing functions.

  The gamma correction unit 222 performs gamma (γ) correction for faithful color reproduction based on the R signal Sr3, the G signal Sg3, and the B signal Sb3, and outputs the output signal R, Gamma (γ) corrected for each color. G and B are input to the color difference matrix unit 224. For example, the gamma correction unit 222 has an initial value of the gamma correction value in a memory table (not shown), and can change the gamma correction value from the outside.

  The color difference matrix unit 224 inputs the color difference signals RY and BY obtained by performing the color difference matrix processing to the video encoder 86.

  The video encoder 86 digitally modulates the color difference signals RY and BY with a digital signal corresponding to the color signal subcarrier, and then synthesizes the digital signal with a luminance signal Y generated by a luminance signal generation unit (not shown). After being converted into a video signal VD (= Y + S + C; S is a synchronization signal and C is a chroma signal), it is input to the DA converter 82. The DA converter 82 converts the digital video signal VD into an analog video signal V.

  Further, the image signal processing unit 66 is characterized by the AGC processing unit 62b, the white balance amplifier unit 210, and the gamma acquired at the time of preceding photographing (provisional photographing) as a preparation stage performed prior to the main photographing as characteristic portions of the present embodiment. A gain analysis unit 230 that analyzes the amplification amount (gain) of the video signal output through the signal processing unit 220 in units of pixels with reference to data from the correction unit 222 is provided.

  The gain analysis unit 230 may obtain the total gain GAtotal at once. For example, the gain analysis unit 230 individually obtains the AGC gain GAagc related to AGC processing, the white balance gain GAwb related to white balance adjustment, and the gamma gain GAgamma related to gamma correction, and their product. The total gain GAtotal may be obtained by (GAagc * GAwb * GAgamma). Based on the analysis result, the gain analysis unit 230 sets an exposure time for each pixel so that the pixel signal output from the imaging unit 12a has an output signal level corresponding to the AGC, white balance adjustment, or gamma correction amount. The obtained exposure time information is stored in a predetermined storage unit (not shown).

  In addition, on the camera module 3 side, based on the analysis result of the gain analysis unit 230, the pixel signal output from the imaging unit 12a is set to an output signal level corresponding to the AGC, WB, or gamma correction amount. A timing control unit 91 (corresponding to the drive control units 12c and 407) for controlling the timing of the drive pulse on the module 3 side is provided.

  The gain analysis unit 230 and the timing control unit 91 suppress S / N degradation accompanying amplification signal processing instead of amplification signal processing that increases the level of the pixel signal output from the unit pixel 403 as a unit component. However, the detection time (exposure time) in the photoelectric conversion element (detection unit) so that a pixel signal of a level corresponding to the amplification factor in the amplification signal processing is output from the unit pixel 403 of each color (R, G, B). ) Is configured.

  Further, the present invention includes the control unit including the gain analysis unit 230 and the timing control unit 91, and excludes the solid-state imaging device 10 and the members of the optical system 5 (the imaging lens 50 and the shutter 52) in the camera module 3. Such a signal processing apparatus is configured. The apparatus for detecting a physical quantity distribution according to the present invention includes the control unit including the gain analysis unit 230 and the timing control unit 91, and further includes the solid-state imaging device 10 in the camera module 3.

  Here, for example, when paying attention to the white balance control function, the gain analysis unit 230 obtains the integrated value of the color signal (primary color signal or color difference signal) for each field and generates white balance control information.

  Based on the analysis result, the gain analysis unit 230 is configured so that the pixel signal output from the imaging unit 12a has an output signal level corresponding to the AGC, WB, or gamma correction amount, and the timing control unit 91 on the camera module 3 side. By cooperating with (corresponding to the drive control units 12c and 407), electronic shutter speed control is performed to determine the exposure time by adjusting the interval between the reset gate pulse RG and the transfer gate pulse TG (readout pulse) for each pixel. Controls the amount of charge accumulated during actual shooting.

  For example, by increasing the exposure time, it is possible to perform processing to increase the level of the signal output from the photoelectric conversion element of the imaging unit 12a, so that substantially avoiding the problem of S / N degradation, Image signal processing involving processing for increasing the signal level can be executed. By separately controlling the exposure times of R, G, and B pixels, AGC processing, white balance adjustment, and gamma correction can be performed for each pixel, and AGC processing, white balance adjustment, and gamma correction are performed. In fact, since the process for increasing the signal level is not performed on the pixel signal output from the CMOS image sensor 12, the S / N deterioration caused by the circuit gain due to the AGC process, white balance adjustment, or gamma correction is suppressed. be able to.

  Further, the signal processing system 6 switches between a 1-input (312a) -2 output (312b, c) type between the CDS processing unit 62a and the AGC processing unit 62b in order to switch the signal transmission route between the preceding shooting and the main shooting. A switch 312 is provided. One output terminal 312 b of the changeover switch 312 is connected to the AGC processing unit 62 b, and the other output terminal 312 c is connected to the AD conversion unit 64.

  Similarly, a 1-input (314a) -2 output (314b, c) type changeover switch is provided between the pixel interpolation processing unit 204 and the white balance amplifier unit 210 in order to switch the signal transmission route between the preceding shooting and the main shooting. 314 is provided for each of the R, G, and B systems. One output terminal 314 b of the changeover switch 314 is connected to the white balance amplifier unit 210, and the other output terminal 314 c is connected to the color difference matrix unit 224.

  Although not shown, a 1-input-2-output type changeover switch 316 (not shown) is provided between the white balance amplifier unit 210 and the gamma correction unit 222 in order to switch the signal transmission route between the preceding shooting and the main shooting. May be provided in each of the R, G, and B systems. In this case, the other output terminal 314 c of the changeover switch 314 is connected to the input terminal 316 a of the changeover switch 316 in the same manner as the output of the white balance amplifier unit 210. One output terminal 316 b of the changeover switch 316 is connected to the gamma correction unit 222, and the other output terminal 316 c is connected to the color difference matrix unit 224.

  Each change-over switch 312, 314, 316 (not shown) may be controlled by a common control signal so as to switch the signal transmission route between the preceding shooting and the main shooting in conjunction with each other, or each preceding switch individually. You may control by a separate control signal so that the signal transmission route | route between imaging | photography and this imaging | photography may be switched.

  If each of the switches 312, 314, and 316 is controlled by an individual control signal, the pixel output from the solid-state imaging device 10 as usual in regard to any one (or more) of AGC, white balance adjustment control, and gamma correction. While performing a process that increases the signal level for the signal, the exposure time can be controlled independently for each pixel so that the exposure time corresponding to the adjustment gain is obtained for the remaining processes. it can.

<Description of operation>
Next, details of the preparation step (provisional imaging) and the main step will be described. First, as a preparatory process (temporary imaging) stage, the gain analysis unit 230 sets the changeover switch 312 to the output terminal 312b side so that the output signal of the CDS processing unit 62a passes through the AGC processing unit 62b, and performs image signal processing. The changeover switch 314 is set on the output terminal 314b side so that the signal of the unit 66 passes through the white balance amplifier unit 210 and the gamma correction unit 222.

  In the preparation step, the mechanical shutter 52 is opened and RGB signals are acquired. Note that the timing of this preparation step may be determined, for example, when the user half-presses the shutter button.

  At this time, the electric signal from the CMOS image sensor 12 passes through the CDS processing unit 62a and the AGC processing unit 62b, is amplified by the AGC gain GAagc for each pixel, and then enters the image signal processing unit 66 from the AD conversion unit 64. The image signal processing unit 66 passes through the digital clamp unit 200, the primary color separation / synchronization processing unit 202, the pixel interpolation processing unit 204, and the white balance amplifier unit 210, and the white balance gain GAwb for each color of RGB is set so that white balance can be obtained. Each is decided. Further, the gamma correction unit 222 applies gamma correction with a gamma gain GAgamma for each pixel corresponding to the signal level (subject luminance).

  Next, as a calculation process step, the gain analysis unit 230 analyzes the total gain value in the AGC processing unit 62b, the white balance amplifier unit 210, and the gamma correction unit 222 in units of pixels of the CMOS image sensor 12. In addition, the charge accumulation time is calculated for each RGB pixel so that the amount of gain equivalent to the analysis result can be controlled on a pixel-by-pixel basis using the electronic shutter speed control to control the charge accumulation amount of the solid-state image sensor 10 (CMOS image sensor 12). To do. Then, the control timing of the reset pulse and the readout pulse that determine the exposure time (charge accumulation time) is determined and stored.

  Next, for example, when the user fully presses the shutter button, this step (main photographing) is executed with the exposure time determined above. In this process, the gain analysis unit 230 first sets the changeover switch 312 to the output terminal 312c side so that the output signal of the CDS processing unit 62a skips the AGC processing unit 62b and enters the AD conversion unit 64, and the image The changeover switch 314 is set on the output terminal 314c side so that the signal of the signal processing unit 66 skips the white balance amplifier unit 210 and the gamma correction unit 222 and enters the color difference matrix unit 224.

  Next, the gain analysis unit 230 and the timing control unit 91 work together so that a pixel signal is output from the CMOS image sensor 12 in consideration of a gain equivalent to the stored analysis result in the preparation process. The charge accumulation time is controlled in units of pixels. Thereby, a pixel signal giving a gain equivalent to the analysis result is output from the CMOS image sensor 12.

  The pixel signal to which a gain equivalent to these analysis results is given passes through the CDS processing unit 62 a, skips the AGC processing unit 62 b via the changeover switch 312, and enters the AD conversion unit 64. Thereafter, the pixel switch is passed through the pixel interpolation processing unit 204, the white balance amplifier unit 210 and the gamma correction unit 222 are skipped by the changeover switch 314, and the color difference matrix unit 224 is entered and output as a digital video signal.

  That is, normally, the signal level output from the CMOS image sensor 12 is directly manipulated to increase the signal level (that is, handled by the electrical signal level), AGC processing, white balance adjustment, or signal level adjustment in gamma correction. Is not performed on the signal itself output from the CMOS image sensor 12, but is performed by increasing or decreasing the amount of accumulated charge by controlling the exposure time.

  By increasing the exposure time, it is possible to perform processing for increasing the level of the signal output from the photoelectric conversion element of the imaging unit 12a. Therefore, processing for increasing the signal level while avoiding the problem of S / N degradation. The accompanying image signal processing can be executed.

  AGC processing, white balance adjustment, and gamma correction are performed by individually controlling the exposure time of R, G, and B pixels so that the exposure time corresponding to the gain acquired by the analysis in the gain analysis unit 230 is obtained. In addition, in the AGC process, the white balance adjustment, and the gamma correction, the pixel signal output from the CMOS image sensor 12 is virtually not subjected to the process for increasing the signal level. It is possible to suppress S / N degradation that occurs due to circuit gain due to white balance adjustment or gamma correction. It is possible to suppress the S / N deterioration caused by the circuit gain due to AGC processing, white balance adjustment, or gamma correction, and to obtain a video with good reproducibility.

  For example, when attention is focused on white balance adjustment control, the entire image pickup signal representing the subject obtained by the image pickup device (solid-state image pickup device 10) approaches white in a normal signal path route during conventional white balance adjustment and preparation steps. Further, the white balance amplifiers 210R, 210G, and 210B of the white balance amplifier unit 210 perform gain adjustment (referred to as white balance gain adjustment) for the primary color R signal Sr1, G signal Sg1, and B signal Sb1. Take white balance.

  On the other hand, at the time of the main imaging of the present embodiment, instead of the white balance gain adjustment in the white balance amplifier unit 210, in order to correspond to the gain adjustment value of each color for white balance analyzed in the preparation process, The detection time (exposure time) of the unit pixel 403 corresponding to each color pixel is adjusted individually (here, “for each color pixel”). Since the pixel signal level output from the photoelectric conversion element of the imaging unit 12a is adjusted for each of R, G, and B color pixels by adjusting the exposure time, the pixel signal output from the unit pixel 403 is white with amplification processing. Even if the gain adjustment for the balance is not performed, the gain adjustment for the white balance can be substantially performed. Therefore, the white balance can be adjusted while avoiding the problem of the S / N deterioration.

  The same applies to the AGC processing. In the conventional signal path route in the conventional AGC processing and the preparation process, the individual photoelectric conversion elements (in the solid-state imaging device 10) of the imaging device (solid-state imaging device 10) are adjusted by adjusting the gain for each pixel. The sensitivity variation of the charge detection unit) is corrected. Further, the gain is adjusted so that the AD converter 64 has an appropriate input level. For example, when the exposure time is insufficient and the output voltage from the pixel is small, an appropriate gain is set and the amplified signal is input to the AD converter 64.

  On the other hand, at the time of the main imaging in the present embodiment, instead of the AGC processing in the AGC processing unit 62b, the unit corresponding to each color pixel so as to correspond to the gain adjustment value for AGC processing analyzed in the preparation process. The detection time (exposure time) of the pixel 403 is individually adjusted. By adjusting the exposure time for each pixel, the reading gain is made variable for each pixel.

  Since the pixel signal level output from the photoelectric conversion element of the imaging unit 12a is adjusted for each of R, G, and B color pixels by adjusting the exposure time, the AGC accompanied by amplification processing is performed on the pixel signal output from the unit pixel 403. Since the gain adjustment for the AGC process can be substantially performed without adjusting the gain of the process, the AGC process can be performed while avoiding the problem of the S / N deterioration.

  The same applies to the gamma correction processing. In the conventional signal path route during the conventional gamma correction processing and the preparation process, the gradation change characteristic with respect to the light intensity is adjusted, and faithful color reproduction is performed during color image processing. Adjust the gamma curve for each color so that

  On the other hand, at the time of the main imaging in the present embodiment, instead of the gamma correction processing in the gamma correction unit 222, it corresponds to each color pixel so as to correspond to the gain adjustment value for the gamma correction processing analyzed in the preparation process. The detection time (exposure time) of the unit pixel 403 is individually adjusted. Since the pixel signal level output from the photoelectric conversion element of the imaging unit 12a is adjusted for each of R, G, and B color pixels by adjusting the exposure time, the pixel signal output from the unit pixel 403 is a gamma that accompanies amplification processing. Since the gain adjustment for the gamma correction process can be substantially performed without performing the gain adjustment for the correction process, the gamma correction process can be performed while avoiding the problem of S / N degradation.

  In the conventional gamma correction processing, since the signal level is amplified with a larger amplification factor in a low-luminance subject than in a high-luminance subject, the problem of S / N degradation is larger than that in a high-luminance subject. In the method of the embodiment, the gain adjustment for the gamma correction process is substantially performed by adjusting the exposure amount without performing the amplification process for increasing the pixel signal level to be processed. The problem of S / N degradation does not occur even in the subject. It can be said that the effect of improving the S / N degradation is higher for a low-luminance subject.

  The gamma correction process has a larger gain adjustment range than the AGC process and white balance adjustment, and the gain adjustment range becomes larger as the subject has a lower luminance. In this respect, it is considered that the gamma correction process is more effective than the white balance adjustment control and the AGC process in the method of the present embodiment that adjusts the exposure amount.

<Overall configuration of digital still camera; CCD type>
FIG. 8 is a schematic configuration diagram showing a second embodiment of the imaging apparatus according to the present invention. In the image pickup apparatus of the second embodiment, the CMOS image pickup device 12 in the first embodiment is changed to a CCD image pickup device 11 capable of reading out all pixels by, for example, an interline transfer (IT) method. A circuit unit having functions equivalent to those of the analog front end unit 12b having the preamplifier unit and the AD conversion unit and the drive control unit 12c is changed to be provided separately from the CCD image pickup device 11. Other points are almost the same as those in the first embodiment.

  A drive control unit 96 in the camera module 3 receives a timing signal generation unit 40 that generates various pulse signals for driving the CCD image pickup device 11 and a pulse signal from the timing signal generation unit 40, A driver (drive unit) 42 for converting the drive pulse for driving the image pickup device 11 and a drive power supply 46 for supplying power to the CCD image pickup device 11 and the driver 42 are provided.

  In addition, the image signal processing unit 66 constituted by a DSP generates various memories of the storage unit 93 excluding the central control unit 92 and the EEPROM 93d composed of a CPU and the like, and further synchronization signals for defining the operation timing of the CCD image pickup device 11. The image pickup device module 3a is configured to include the camera module 3 including the CCD image pickup device 11 as a main part, the image signal processing unit 66, and the EEPROM 93d.

  In the illustrated example, the preamplifier unit 62 and the AD conversion unit 64 of the signal processing system 6 are built in the camera module 3. However, the configuration is not limited to this, and the preamplifier unit 62 and the AD conversion unit 64 are included in the main unit 4. The structure provided in can also be taken. In addition, a configuration in which the DA conversion unit is provided in the image signal processing unit 66 may be employed.

  Further, although the timing signal generation unit 40 is built in the camera module 3, the configuration is not limited to such a configuration, and a configuration in which the timing signal generation unit 40 is provided in the main unit 4 can also be adopted. In addition, the timing signal generation unit 40 and the driver 42 are separate components, but the configuration is not limited to this, and the timing signal generation unit 40 and the driver 42 may be integrated (timing generator with built-in driver). By doing so, a more compact (small) digital still camera 1 can be configured.

  Further, the timing signal generator 40 and the driver 42 may each be configured by a circuit with individual discrete members, but are provided as an IC (Integrated Circuit) formed on a single semiconductor substrate. Is good. By doing so, not only can it be made compact, but the handling of the members becomes easy, and both can be realized at low cost. In addition, the digital still camera 1 can be easily manufactured. Further, the timing signal generation unit 40 and the driver 42 which are strongly related to the CCD image pickup device 11 to be used are integrated by being mounted on the common substrate with the CCD image pickup device 11 or mounted in the camera module 3. If integrated, the handling and management of the members become simple. In addition, since these are integrated as a module, the digital still camera 1 (completed product) can be easily manufactured. The camera module 3 may be composed only of the CCD image pickup device 11 and the optical system 5.

<Configuration example of CCD image sensor>
FIG. 9 is a schematic configuration diagram of a CCD image sensor 11 which is an example of the solid-state image sensor 10. For example, as shown in FIG. 9, on a semiconductor substrate 521, a number of sensor units (photosensitive units; photocells) 511 each including a photodiode as an example of a light receiving element corresponding to a pixel (unit cell) are horizontally ( They are arranged in a two-dimensional matrix in the (row) direction and the vertical (column) direction. These sensor units 511 convert incident light incident from the light receiving surface into signal charges having a charge amount corresponding to the amount of light, and accumulate the signal charges. The sensor unit 511 is provided with a color separation filter for capturing a color image, as in the case of the CMOS image sensor 12.

  In the CCD image pickup device 11, a vertical CCD (V register unit, vertical transfer unit) 513 provided with a plurality of vertical transfer electrodes corresponding to the driving phase is arranged for each vertical column of the sensor unit 511. The transfer direction of the vertical CCD 513 is the vertical direction in the figure, and a plurality of vertical CCDs 513 are arranged in this direction. Further, a read gate (ROG) unit 512 is interposed between the vertical CCD 513 and each sensor unit 511, and a channel stop CS is provided at a boundary portion of each unit cell. An imaging area 514 is configured by a plurality of vertical CCDs 513 that are provided for each vertical row of the sensor units 511 and vertically transfer signal charges read from the sensor units 511 by the read gate unit 512.

  The signal charge accumulated in the sensor unit 511 is read out to the vertical CCD 513 by applying a drive pulse corresponding to the read pulse (field shift pulse) XSG to the read gate unit 512 in the vertical blanking period. The vertical CCD 513 is driven to transfer by a drive pulse φVv based on the vertical transfer clock Vv, and the read signal charges are sequentially arranged in the vertical direction in portions corresponding to one scanning line (one line) in a part of the horizontal blanking period. Forward. This vertical transfer for each line is called a line shift.

  The CCD image pickup device 11 includes a horizontal CCD (H register unit) extending in a predetermined (for example, left and right) direction adjacent to each transfer destination side end of the plurality of vertical CCDs 513, that is, the vertical CCD 513 in the last row. Horizontal transfer unit) 515 is provided for one line. The horizontal CCD 515 is driven to transfer by drive pulses φH1 and φH2 based on, for example, two-phase horizontal transfer clocks H1 and H2, and the signal charges for one line transferred from the plurality of vertical CCDs 513 are transferred after the horizontal blanking period. The images are sequentially transferred in the horizontal direction during the horizontal scanning period. For this reason, a plurality of (two) horizontal transfer electrodes 529 corresponding to the two-phase driving are provided.

  At the end of the transfer destination of the horizontal CCD 515, as an example of a unit signal generator, for example, a charge voltage converter 516 having a floating diffusion amplifier (FDA) configuration is provided. The charge / voltage converter 516 sequentially converts the signal charges transferred horizontally by the horizontal CCD 515 into voltage signals and outputs the voltage signals. This voltage signal is derived as a CCD output (Vout) corresponding to the amount of incident light from the subject. Thus, the interline transfer type CCD image pickup device 11 is configured.

  With reference to FIG. 8 and FIG. 9, a series of operations of the digital still camera 1 provided with the CCD image pickup device 11 will be outlined as follows. First, the timing signal generator 40 generates various pulse signals such as a transfer clock Vv for vertical transfer and a read pulse XSG. These pulse signals are converted into drive pulses of a predetermined voltage level by the driver 42 and then input to predetermined terminals of the CCD image sensor 11.

  When the subject Z is imaged, an optical image of the subject Z formed on the light receiving surface of the CCD image sensor 11 via the imaging lens 50 (shutter 52 and lens 54) is received by each sensor unit 511 including a photodiode or the like. The signal charge is converted into an amount corresponding to the amount of incident light.

  The signal charge accumulated in each of the sensor units 511 is applied to the transfer channel terminal electrode of the read gate unit 512 by the read pulse XSG emitted from the timing signal generation unit 40, and the potential below the transfer channel terminal electrode is deepened. Thus, the data is read out to the vertical CCD 513 through the readout gate portion 512. Then, the vertical CCD 513 is driven based on the six-phase (eight-phase) vertical drive pulse φVv, so that it is sequentially transferred to the horizontal CCD 515.

  The horizontal CCD 515 is sent to one line vertically transferred from each of the plurality of vertical CCDs 513 based on the two-phase horizontal drive pulses φH1 and φH2 emitted from the timing signal generation unit 40 and converted to a predetermined voltage level by the driver 42. Corresponding signal charges are sequentially horizontally transferred to the charge-voltage converter 516 side.

  The charge-voltage converter 516 accumulates signal charges sequentially injected from the horizontal CCD 515 in a floating diffusion (not shown), converts the accumulated signal charges into a signal voltage, for example, via an output circuit having a source follower configuration (not shown), An image pickup signal (CCD output signal) Vout is output under the control of the reset pulse RG generated from the timing signal generator 40.

  That is, in the CCD image pickup device 11, the signal charges detected in the image pickup area in which the sensor units 511 are arranged two-dimensionally in the vertical and horizontal directions are horizontally converted by the vertical CCDs 513 provided corresponding to the vertical columns of the sensor units 511. Vertical transfer is performed up to the CCD 515, and thereafter, signal charges are transferred in the horizontal direction by the horizontal CCD 515 based on the two-phase horizontal transfer pulses H1 and H2. Then, the operation of converting the potential to a potential corresponding to the signal charge from the horizontal CCD 515 by the charge voltage conversion unit 516 and outputting the same is repeated.

  The voltage signal sequentially read out from the CCD image sensor 11, that is, each R, G, B pixel signal corresponding to the pixel is subjected to CDS processing by the preamplifier 62 based on each sample pulse from the timing signal generator 40. And AGC processing, etc. are performed, converted into digital R, G, and B pixel data by the AD conversion unit 64, and then temporarily stored in the RAM 93 b of the storage unit 93. Hereinafter, it is the same as that of 1st Embodiment using the CMOS image pick-up element 12. FIG.

<Electronic shutter function in CCD type>
Here, the CCD image pickup device 11 is provided with an overflow drain in parallel with the photoelectric conversion device (sensor unit 511) or as a vertical overflow drain structure, and the photoelectric conversion device (electronic shutter pulse SUB) is used before that by the shutter gate pulse (electronic shutter pulse SUB). By making it possible to sweep the signal charge accumulated in the detection unit) into the substrate, a so-called electronic shutter function for arbitrarily controlling the charge accumulation time (shutter speed) can be realized. The charge accumulation time is defined by the period from when the shutter is opened to when the read pulse XSG is applied to the read gate portion 512, regardless of the mechanical type or electronic type using the electronic shutter pulse. The electronic shutter function can also be realized by adjusting XSG for each pixel.

  In this case, in the mechanism for electronic shutter in the normal CCD image pickup device 11, the electronic shutter operation is performed in which all pixels are exposed for the same time, whereas in this embodiment, the exposure time is set for each pixel. Configure to be adjustable. The exposure time is a period from when the electronic shutter pulse SUB is supplied until the read gate unit 512 is supplied with the read pulse XSG and a signal charge is read from the vertical CCD 513. Therefore, the electronic shutter pulse SUB and the read pulse XSG are used. The output timing of at least one of these, preferably any one of them may be set for each pixel. Note that when only the white balance adjustment function is configured to control the accumulation time, at least one of the electronic shutter pulse SUB and the readout pulse XSG, preferably any one of the outputs, is output for each color pixel of R, G, and B. What is necessary is just to make it the structure which can adjust timing.

  For example, in consideration of a general CCD type characteristic in which signal charges are simultaneously read out from all pixels to a vertical CCD to perform global exposure, the readout pulse XSG is supplied to all pixels in common, and the electronic shutter pulse SUB is applied to each pixel. The wiring form that can be supplied to the power supply is sufficient. In this case, in order to individually supply the electronic shutter pulse SUB to each unit cell, the signal charge is simultaneously read from each pixel in the same row and then the light incident on the photoelectric conversion element is accumulated as the signal charge. Unlike the MOS type, the number of wirings increases because a mechanism for individually setting the rows is required.

  In this respect, in order to reduce the number of wires for the electronic shutter control pulse (for example, the electronic shutter pulse SUB), the signal level with respect to the pixel signal output from the CCD image sensor 11 is conventionally used for AGC processing and gamma correction. The control pulse for the electronic shutter is independently supplied to the unit cell for each of the R, G, and B color pixels so that the exposure time commensurate with the white balance gain is obtained while executing the process involving the process of increasing This configuration is a realistic aspect.

  As a specific configuration, for example, a two-dimensional matrix wiring configuration as shown in FIG. 3 can be adopted. For example, a shutter timing having a vertical shutter timing control unit 544cv and a horizontal shutter timing control unit 544ch. The control unit 544 c can be provided in the timing signal generation unit 40. In FIG. 9, the shutter timing control unit 544 c is shown as being removed from the timing signal generation unit 40.

  In this case, the vertical shutter timing control unit 544cv generates vertical electronic shutter pulses SUBv_R, G, and B for the R, G, and B color pixels that specify the row address of the unit cell (shutter pixel) to be shuttered. The shutter timing control unit 544ch may generate the vertical electronic shutter pulses SUBv_R, G, and B for the R, G, and B color pixels that specify the column address of the unit pixel to be shuttered. Note that each electronic shutter pulse SUBv_R, G, B, SUBh_R, G, B is supplied to all unit cells in the same row or column via the driver 42. In FIG. 9, this point is saved. Show.

  In addition, a switch unit 519 that receives each electronic shutter pulse SUBv_R, G, B, SUBh_R, G, B and supplies the electronic shutter pulse to the shutter pixels (unit cells) in the vertical and horizontal directions is provided. It may be provided in the vicinity of the sensor portion 511. The switch unit 519 is for the own color among the vertical electronic shutter pulses SUBv_R, G, and B specifying the row of the shutter pixels from the vertical shutter timing control unit 544cv, and the shutter from the horizontal shutter timing control unit 544ch. A logical product circuit may be configured to operate with respect to the one for the own color among the horizontal electronic shutter pulses SUBh_R, G, and B that specify the column position of the pixel.

  In FIG. 9, a configuration example in which the output timing of the electronic shutter pulse SUB is adjusted for each color pixel of R, G, and B so that the exposure time corresponding to the white balance gain acquired by the analysis of the gain analysis unit 230 is obtained. However, the electronic shutter pulse SUB can be replaced with the readout pulse XSG. In this case, although not shown in the drawing, the electronic shutter pulse SUB may be supplied to all the pixels, and the readout pulse XSG may be configured to be supplied for each color pixel.

  Further, it is possible to adopt a configuration in which the output timings of both the electronic shutter pulse SUB and the readout pulse XSG are adjusted for each of the R, G, and B color pixels so that the exposure time corresponds to the white balance gain.

<Driving method of unit cell>
10 and 11 are timing charts for explaining a method of driving the unit cell shown in FIG. 9 to obtain a pixel signal. Here, FIG. 10 shows an example in which the output timing of the electronic shutter pulse SUB is adjusted for each of R, G, and B color pixels. In this case, among the electronic shutter pulses SUB_R, G, and B, the color pixel (R pixel in the example in the figure) having the longest exposure time corresponding to the white balance gain is first activated (H level in this example), Before the signal charges accumulated in the sensor unit 511 are swept out into the substrate, accumulation of new signal charges is started (t31).

  After that, among the electronic shutter pulses SUB_R, G, and B, the color pixel (G pixel in the example in the figure) having the next long exposure time corresponding to the white balance gain is activated (H level in this example), Accumulation of new signal charges is started after sweeping out signal charges previously accumulated in the sensor unit 511 into the substrate (t33).

  Thereafter, among the electronic shutter pulses SUB_R, G, and B, the color pixel (B pixel in the example in the figure) having the shortest exposure time corresponding to the white balance gain is activated (H level in this example), and before that After the signal charges accumulated in the sensor unit 511 are swept out into the substrate, accumulation of new signal charges is started (t35).

  Finally, in the vertical blanking period (active L), the readout pulse XSG is made active (H level in this example), thereby reading the signal charges of all the R, G, B color pixels to the vertical CCD 513 (t37). ). By doing so, the exposure times Tr, Tg, Tb corresponding to the white balance gain can be adjusted for each color pixel of R, G, B.

  As shown in the figure, when the mechanical shutter 52 is used together, among the electronic shutter pulses SUB_R, G, B, the color pixel (R in the example shown in the figure) has the longest exposure time corresponding to the white balance gain. Pixel).

  FIG. 11 shows an example in which the output timing of the readout pulse XSG is adjusted for each of R, G, and B color pixels. In this case, among the readout pulses XSG_R, G, B, among the color pixels of R, G, B, the color pixel (R pixel in the example in the figure) that has the longest exposure time commensurate with the white balance gain is used. By making it active (H level in this example) within the ranking period, the signal charge of the color pixel is read out to the vertical CCD 513. The electronic shutter pulse SUB is activated (H level in this example) in accordance with the exposure time of the color pixel having the longest exposure time commensurate with the white balance gain among the R, G, B color pixels (t41). ). The electronic shutter pulse SUB is supplied in common to all the R, G, and B color pixels.

  Thereafter, among the readout pulses XSG_R, G, and B, the color pixel (G pixel in the example) whose exposure time corresponding to the white balance gain becomes the next long is made active (H level in this example), and the color The signal charge of the pixel is read out to the vertical CCD 513 (t43). Thereafter, among the readout pulses XSG_R, G, and B, the color pixel (B pixel in the example in the figure) having the shortest exposure time corresponding to the white balance gain is activated (H level in this example), and the color pixel Is read out to the vertical CCD 513 (t45).

  Finally, in the vertical blanking period, by making active (H level in this example) a color pixel (R pixel in the example) having the longest exposure time corresponding to the white balance gain, The signal charge is read out to the vertical CCD 513 (t37). By doing so, the exposure times Tr, Tg, Tb corresponding to the white balance gain can be adjusted for each color pixel of R, G, B.

  As shown in the figure, when the mechanical shutter 52 is used together, the electronic shutter pulse SUB can be substituted.

  It should be noted that the exposure time for one of the R, G, and B color pixels can be achieved regardless of whether the electronic shutter pulse SUB or the readout pulse XSG is used for adjusting the exposure time. Can be left unadjusted. Normally, since the readout pulse XSG is output in one field (about 1/60 seconds), the white balance among the R, G, B color pixels is determined based on the adjustment direction of the exposure time shown in FIGS. For the color pixel (R pixel in the example shown in the figure) with the longest exposure time commensurate with the gain, the exposure time is not adjusted (equivalent to setting the white balance gain to 1), and white balance is applied only to the remaining color pixels. The exposure time may be adjusted to a shorter time (equivalent to making the white balance gain less than 1) so that the exposure time matches the gain.

  10 and 11 show an example in which the exposure time is adjusted to match the white balance gain within one field, but the exposure time is adjusted to match the white balance gain within a plurality of fields. You can also do it. In this case, for example, since the readout pulse XSG is normally output in one field (about 1/60 seconds), among the R, G, B color pixels, the color pixel having the shortest exposure time commensurate with the white balance gain. With regard to, the exposure time is not adjusted (equivalent to setting the white balance gain to 1), and only the remaining color pixels are adjusted to a longer time so that the exposure time matches the white balance gain (white balance gain) Is equivalent to making the value greater than 1).

  Therefore, also in the configuration of the second embodiment, the first embodiment using the CMOS image sensor 12 as the solid-state image sensor 10 regardless of which one of the electronic shutter pulse SUB and the read pulse XSG is used for adjusting the exposure time. In the same manner as in the embodiment, using the electronic shutter function of the CCD image pickup device 11, exposure corresponding to the white balance gain of each color in pixel units (in the example of FIG. 4, for each of R, G, and B color pixels). By adjusting the exposure time (accumulation time) to the photoelectric conversion element (sensor unit 511) individually (in this case, “for each color pixel”) so as to be time, white balance adjustment can be performed. it can. In the white balance adjustment, since the process for increasing the signal level is not actually performed on the pixel signal output from the CMOS image sensor 12, the S / N deterioration caused by the circuit gain by the conventional white balance adjustment is suppressed. can do.

  Note that FIG. 9 shows a configuration in which the exposure time to the photoelectric conversion element (sensor unit 511) is adjusted for each of R, G, and B color pixels in order to reduce the number of wirings. A configuration in which the electronic shutter pulse SUB is independently supplied to each pixel may be employed. In this case, not only white balance adjustment, but also AGC processing and gamma correction, AGC processing is performed by supplying an electronic shutter pulse SUB to each pixel independently so that an exposure time corresponding to each gain is obtained. In the white balance adjustment and the gamma correction, the pixel signal output from the CMOS image sensor 12 is virtually not subjected to a process for increasing the signal level. Therefore, the circuit gain by the AGC process, the white balance adjustment or the gamma correction is not used. S / N degradation that occurs in the case can be suppressed.

<Relationship with similar technologies>
Note that, as a mechanism similar to the first and second embodiments, the charge accumulation times of the photoelectric conversion elements of the respective colors are individually determined so as to obtain white balance before this step, and based on the individual charge accumulation times. For example, Japanese Patent Laid-Open No. 2000-3504222 (hereinafter also referred to as Reference 1) proposes a mechanism for obtaining a color image signal with good color reproducibility by controlling the charge accumulation time of each color photoelectric conversion element. .

  However, even though the mechanism described in this reference 1 can obtain an image with good color reproducibility by individually controlling the charge accumulation time (exposure time) of the photoelectric conversion elements of each color so as to achieve white balance. The purpose is to increase the dynamic range of color images. On the other hand, in the present invention, as in the above-described embodiment, while avoiding the problem that causes the S / N deterioration due to the amplification of noise due to the amplification accompanied by the process of increasing the signal level to be processed ( The exposure time is controlled so that the white balance can be achieved (with suppression), and the concept of the purpose of controlling the exposure time is completely different.

  Therefore, the mechanism of Reference 1 also controls the exposure time for the purpose of white balance adjustment as in this embodiment, but the adjustment range of the exposure time and the obtained effect are completely different, and the technical idea is completely different. Is clear.

  In other words, the mechanism described in Reference 1 requires the combined use of a mechanical shutter and an electronic shutter, and the description of “Effects of the Invention” uses both a mechanical shutter and an electronic shutter. Thus, it is said that a color image signal with good color reproducibility can be obtained and a video with a large dynamic range can be obtained. The combined use of a mechanical shutter is an essential requirement for obtaining an image with a large dynamic range. In addition, there is no explanation of why a video with a large dynamic range can be obtained by using a mechanical shutter and an electronic shutter together.

  On the other hand, in this embodiment, a mechanical shutter and an electronic shutter can be used in combination, but this increases the degree of freedom in controlling the exposure time compared to the case where the exposure time is controlled only by the electronic shutter. It can be used to reduce the number of control pulses for the electronic shutter, and the combined use of the mechanical shutter is not essential, and it is not used for the purpose of obtaining an image with a large dynamic range.

  In addition, the mechanism described in Reference Document 1 is presumed to use a charge transfer type imaging device such as a CCD. This is because the embodiment of Reference 1 discloses only a case of a solid-state imaging device having a vertical charge transfer path for transferring charges in the vertical direction. Even in the claims describing the device, a plurality of photoelectric conversion elements and A vertical charge transfer path for transferring charges in the vertical direction, a read gate for reading charges from the photoelectric conversion element to the vertical charge transfer path, and a horizontal charge transfer path for transferring charges transferred by the vertical charge transfer path in the horizontal direction Is described.

  Therefore, in an XY address type solid-state imaging device such as a CMOS sensor, while avoiding a problem that causes S / N degradation due to amplification of noise due to amplification accompanied by processing for increasing a signal level to be processed. There is no disclosure of a mechanism for controlling the exposure time so that white balance can be achieved (while suppressing), and “a person skilled in the art can make various modifications and applications based on the disclosure of the present specification”. In other words, based on the above description, in an XY address type solid-state imaging device such as a CMOS sensor, noise is also amplified by amplification accompanied by processing for increasing a signal level to be processed. Neither can it be conceived to control the exposure time so as to achieve white balance while avoiding (suppressing) the problem that causes N degradation.

  Further, the mechanism described in Reference 1 only discloses that white balance is performed by controlling the exposure time for each color pixel on the assumption that a charge transfer type imaging device such as a CCD is used. There is no disclosure regarding the problem of S / N degradation in other processes involving increasing the signal level to be processed, such as AGC processing and gamma correction processing.

  Therefore, regardless of what the imaging device is, based on the description in Reference Document 1, the noise is also amplified by the amplification accompanied by the process for increasing the signal level of the processing target, resulting in S / N degradation. While avoiding (suppressing) the problem, it is impossible to conceive of controlling the exposure time to perform the AGC process and the gamma correction process. It is impossible to improve S / N degradation due to gamma correction in an AGC or low-luminance subject.

  As mentioned above, although this invention was demonstrated using embodiment, the technical scope of this invention is not limited to the range as described in the said embodiment. Various changes or improvements can be added to the above-described embodiment without departing from the gist of the invention, and embodiments to which such changes or improvements are added are also included in the technical scope of the present invention.

  Further, the above embodiments do not limit the invention according to the claims (claims), and all combinations of features described in the embodiments are not necessarily essential to the solution means of the invention. Absent. The embodiments described above include inventions at various stages, and various inventions can be extracted by appropriately combining a plurality of disclosed constituent elements. Even if some constituent requirements are deleted from all the constituent requirements shown in the embodiment, as long as an effect is obtained, a configuration from which these some constituent requirements are deleted can be extracted as an invention.

  For example, in the above-described embodiment, as an example of a solid-state imaging device capable of reading signals from individual unit pixels, a CMOS sensor or a CCD sensor including a pixel unit that generates signal charges by receiving light is shown as an example. However, the generation of signal charges is not limited to light, and can be applied to electromagnetic waves such as infrared rays, ultraviolet rays, or X-rays in general, and a large number of elements that receive these electromagnetic waves and output analog signals according to the amount The matters described in the above embodiments can be applied to a semiconductor device including the unit component.

  In addition, the imaging unit 10 is not limited to a plurality of unit signal generation units arranged in a two-dimensional manner, and may be arranged in a long shape. For example, even if the imaging unit 10 is replaced with a line sensor, the technique described in the above embodiment can be applied in the same manner, and the same effect can be obtained.

  Note that “arranged in a long shape” means a form in which the short side and the long side are sufficiently recognized, and is not limited to a typical line sensor arranged long in one row. For example, it is meant to include a long arrangement in a plurality of rows or a staggered arrangement.

  In the above-described embodiment, image signal processing involving processing for increasing the signal level to be processed, such as AGC processing, white balance adjustment processing, or gamma correction processing, is performed on the signal output from the unit component. Instead, the example of executing by increasing the signal output from the unit component by adjusting the exposure time has been described, but the mechanism described in the above embodiment is not limited to the solid-state imaging device, but the signal level to be processed The present invention can be applied to any electronic device that uses a process for increasing the size.

  For example, in the above-described embodiment, an application example of an imaging apparatus using a CMOS type or CCD type solid-state imaging device that is sensitive to electromagnetic waves input from the outside such as light and radiation has been described. The mechanism described in the above embodiment can be applied to anything that detects a change, and is not limited to light or the like. For example, fingerprint information is based on a change in electrical characteristics or a change in optical characteristics based on pressure. The present invention can be similarly applied to signal processing in a mechanism for detecting other physical changes, such as a fingerprint authentication device for detecting the image (see Japanese Patent Application Laid-Open Nos. 2002-7984 and 2001-125734).

1 is a schematic configuration diagram illustrating a first embodiment of an imaging apparatus (camera system) according to the present invention. It is a schematic block diagram of a CMOS image sensor. It is a figure explaining the electronic shutter function used by this embodiment. It is a figure explaining the relationship between the structural example of a unit pixel, and the drive circuit in connection with the electronic shutter function which drives a unit pixel. FIG. 5 is a timing chart for explaining a method for acquiring a pixel signal by driving the unit pixel shown in FIG. 4 (part 1); FIG. 5 is a timing chart for explaining a method for acquiring a pixel signal by driving the unit pixel shown in FIG. 4 (part 2); It is a block diagram which paid its attention to the image signal processing function accompanying the process which enlarges a signal level in a digital still camera. It is a schematic block diagram which shows 2nd Embodiment of the imaging device which concerns on this invention. It is a schematic block diagram of a CCD image sensor. FIG. 10 is a timing chart for explaining a method of acquiring a pixel signal by driving the unit cell shown in FIG. 9 (No. 1). FIG. 10 is a timing chart illustrating a method for acquiring a pixel signal by driving the unit cell illustrated in FIG. 9 (part 2); FIG. 11 is a diagram illustrating a configuration example of a conventional imaging apparatus having a mechanism for realizing functions such as white balance adjustment processing, pixel interpolation processing, and gamma correction processing.

Explanation of symbols

  DESCRIPTION OF SYMBOLS 1 ... Digital still camera, 3 ... Camera module, 4 ... Main body unit, 5 ... Optical system, 6 ... Signal processing system, 7 ... Recording system, 8 ... Display system, 9 ... Control system, 10 ... Solid-state image sensor, 11 ... CCD image sensor, 12 ... CMOS image sensor, 62 ... Preamplifier unit, 62a ... CDS processor, 62b ... AGC processor, 64 ... AD converter, 66 ... Image signal processor, 91 ... Timing controller, 92 ... Central control 96: Drive control unit, 200: Digital clamp unit, 202 ... Primary color separation / simulation processing unit, 204 ... Pixel interpolation processing unit, 210 ... White balance amplifier unit, 220 ... Signal processing unit, 222 ... Gamma correction unit, 224 ... Color difference matrix unit, 230 ... Gain analysis unit, 312, 314, 316 ... Changeover switch

Claims (32)

The unit configuration element includes a detection unit that detects change information according to a change in the incident physical quantity, and a unit signal generation unit that generates a unit signal based on the change information detected by the detection unit. A signal processing method for performing predetermined signal processing based on the unit signal output from an apparatus for physical quantity distribution detection in which elements are arranged in a predetermined order,
Instead of amplified signal processing for increasing the level of the unit signal output from the unit component, the level corresponding to the amplification factor in the amplified signal processing while suppressing S / N degradation associated with the amplified signal processing The signal processing method characterized by including the process of adjusting individually the detection time in the said detection part so that a unit signal may be output from the said unit component. The amplification signal processing is processing for white balance so that an entire imaging signal representing a subject obtained by a color imaging device as an apparatus for detecting the physical quantity distribution approaches white,
The signal processing according to claim 1, wherein the step of individually adjusting the detection time in the detection unit individually adjusts the detection time of the detection unit for each color so as to achieve the white balance. Method. The amplified signal processing is processing for correcting the sensitivity variation of each of the detection units,
The step of individually adjusting the detection time in the detection unit is to individually adjust the detection time of each of the detection units so that the variation in sensitivity is offset. Signal processing method. The amplified signal processing is a process of adjusting a change characteristic of the unit signal with respect to a change in the physical quantity so that an amplification factor for a part with a small change in the physical quantity is larger than a part with a large change in the physical quantity,
The step of individually adjusting the detection time in the detection unit is such that the detection time of the detection unit is longer in the portion where the change in the physical quantity is smaller than in the portion where the change in the physical quantity is large. The signal processing method according to claim 1, wherein the detection time is individually adjusted. The step of individually adjusting the detection time in the detection unit,
A preparatory step of performing the detection in the detection unit for a predetermined detection time, and detecting the magnitude of the unit signal corresponding to the change in the physical quantity for each detection unit;
According to the magnitude of the unit signal corresponding to the change in the physical quantity for each of the detection units detected in the preparation step, the amplification factor in the amplified signal processing is suppressed while suppressing S / N degradation associated with the amplified signal processing. A calculation step of individually calculating a detection time in the detection unit so that the unit signal of a corresponding level is output from the unit component;
The signal processing method according to claim 1, further comprising: a main step of individually controlling the detection time of each detection unit based on the detection time calculated in the calculation step. The signal processing method according to claim 1, wherein the step of individually adjusting the detection time in the detection unit is executed using an electronic shutter function that clears the change information detected by the detection unit. The step of individually adjusting the detection time in the detection unit is executed by using a mechanical shutter function that opens and closes the incidence of the physical quantity to the detection unit in combination with the electronic shutter function. The signal processing method as described. The signal according to claim 1, wherein the step of individually adjusting the detection time in the detection unit is executed using an electronic shutter function that reads the change information detected by the detection unit at different times. Processing method. 9. The step of individually adjusting the detection time in the detection unit is executed by using a mechanical shutter function for opening and closing incidence of the physical quantity to the detection unit in combination with the electronic shutter function. The signal processing method as described. The unit configuration element includes a detection unit that detects change information according to a change in the incident physical quantity, and a unit signal generation unit that generates a unit signal based on the change information detected by the detection unit. A signal processing device that performs predetermined signal processing based on the unit signal output from an apparatus for physical quantity distribution detection in which elements are arranged in a predetermined order,
Instead of amplified signal processing for increasing the level of the unit signal output from the unit component, the level corresponding to the amplification factor in the amplified signal processing while suppressing S / N degradation associated with the amplified signal processing A signal processing apparatus comprising: a control unit that individually adjusts a detection time in the detection unit so that a unit signal is output from the unit component. The amplification signal processing is processing for white balance so that an entire imaging signal representing a subject obtained by a color imaging device as an apparatus for detecting the physical quantity distribution approaches white,
The said control part adjusts the detection time of the said detection part of each color separately as a method of adjusting the detection time in the said detection part separately so that the said white balance can be taken. Signal processing device. The amplified signal processing is processing for correcting the sensitivity variation of each of the detection units,
The said control part adjusts the detection time of each said detection part separately as a method of adjusting the detection time in the said detection part separately so that the said sensitivity variation may be offset. A signal processing device according to 1. The amplified signal processing is a process of adjusting a change characteristic of the unit signal with respect to a change in the physical quantity so that an amplification factor for a part with a small change in the physical quantity is larger than a part with a large change in the physical quantity,
The control unit, as a method of individually adjusting the detection time in the detection unit, so that the detection time of the detection unit is longer in the portion where the change in the physical quantity is smaller than the portion where the change in the physical quantity is large. The signal processing device according to claim 10, wherein the detection time of each of the detection units is individually adjusted. The controller is
The detection is performed by the detection unit for a predetermined detection time, the magnitude of the unit signal corresponding to the change in the physical quantity for each detection unit is detected, and the change in the physical quantity for each detected detection unit is detected. According to the magnitude of the corresponding unit signal, the unit signal having a level corresponding to the amplification factor in the amplified signal processing is output from the unit component while suppressing S / N degradation associated with the amplified signal processing. As described above, an analysis unit that individually calculates the detection time in the detection unit,
The signal processing apparatus according to claim 10, further comprising: a timing control unit that individually controls the detection time of each detection unit based on the detection time calculated in the analysis unit. The said control part performs the method of adjusting individually the detection time in the said detection part using the electronic shutter function which clears the said change information detected by the said detection part. Signal processing device. The control unit executes a technique of individually adjusting a detection time in the detection unit by using a mechanical shutter function for opening and closing incidence of the physical quantity to the detection unit in combination with the electronic shutter function. The signal processing apparatus according to claim 15. The said control part performs the method of adjusting individually the detection time in the said detection part using the electronic shutter function which reads the said change information detected by the said detection part at different time, respectively. A signal processing device according to 1. The control unit executes a technique of individually adjusting a detection time in the detection unit by using a mechanical shutter function for opening and closing incidence of the physical quantity to the detection unit in combination with the electronic shutter function. The signal processing device according to claim 17. The unit configuration element includes a detection unit that detects change information according to a change in the incident physical quantity, and a unit signal generation unit that generates a unit signal based on the change information detected by the detection unit. A device for detecting a physical quantity distribution in which elements are arranged in a predetermined order,
Instead of amplified signal processing for increasing the level of the unit signal output from the unit component, the level corresponding to the amplification factor in the amplified signal processing while suppressing S / N degradation associated with the amplified signal processing An apparatus for detecting a physical quantity distribution, comprising: a control unit that individually adjusts a detection time in the detection unit so that a unit signal is output from the unit component. The amplification signal processing is processing for white balance so that an entire imaging signal representing a subject obtained by a color imaging device as an apparatus for detecting the physical quantity distribution approaches white,
20. The control unit according to claim 19, wherein the control unit individually adjusts the detection time of the detection unit for each color so as to achieve the white balance as a method of individually adjusting the detection time in the detection unit. Device for physical quantity distribution detection. The amplified signal processing is processing for correcting the sensitivity variation of each of the detection units,
The said control part adjusts the detection time of each said detection part separately as a method of adjusting the detection time in the said detection part separately so that the said sensitivity variation may be offset. The device for physical quantity distribution detection described in 1. The amplified signal processing is a process of adjusting a change characteristic of the unit signal with respect to a change in the physical quantity so that an amplification factor for a part with a small change in the physical quantity is larger than a part with a large change in the physical quantity,
The control unit, as a method of individually adjusting the detection time in the detection unit, so that the detection time of the detection unit is longer in the portion where the change in the physical quantity is smaller than the portion where the change in the physical quantity is large. The apparatus for detecting a physical quantity distribution according to claim 19, wherein the detection time of each of the detection units is individually adjusted. The controller is
The detection is performed by the detection unit for a predetermined detection time, the magnitude of the unit signal corresponding to the change in the physical quantity for each detection unit is detected, and the change in the physical quantity for each detected detection unit is detected. According to the magnitude of the corresponding unit signal, the unit signal having a level corresponding to the amplification factor in the amplified signal processing is output from the unit component while suppressing S / N degradation associated with the amplified signal processing. As described above, an analysis unit that individually calculates the detection time in the detection unit,
The apparatus for physical quantity distribution detection according to claim 19, further comprising: a timing control unit that individually controls the detection time of each detection unit based on the detection time calculated in the analysis unit. The said control part performs the method of adjusting individually the detection time in the said detection part using the electronic shutter function which clears the said change information detected by the said detection part. Device for physical quantity distribution detection. The control unit executes a technique of individually adjusting a detection time in the detection unit by using a mechanical shutter function for opening and closing incidence of the physical quantity to the detection unit in combination with the electronic shutter function. The apparatus for physical quantity distribution detection according to claim 24. The control unit individually supplies a drive signal for performing a normal read operation to the unit component and a drive signal for realizing the electronic shutter function to the unit component individually. The apparatus for physical quantity distribution detection according to claim 24, further comprising: a shutter control unit. The said control part performs the method of adjusting the detection time in the said detection part separately using the electronic shutter function which reads the said change information detected by the said detection part at different time, respectively. The device for physical quantity distribution detection described in 1. The control unit executes a technique of individually adjusting a detection time in the detection unit by using a mechanical shutter function for opening and closing incidence of the physical quantity to the detection unit in combination with the electronic shutter function. The apparatus for physical quantity distribution detection according to claim 27. The control unit individually supplies a drive signal for performing a normal read operation to the unit component and a drive signal for realizing the electronic shutter function to the unit component individually. The apparatus for physical quantity distribution detection according to claim 27, further comprising: a shutter control unit. The apparatus for physical quantity distribution detection according to claim 19, wherein reading of the unit signal from each of the unit components is controlled by address control. The physical quantity distribution detecting unit according to claim 19, further comprising a charge transfer path for transferring a signal charge representing the change information detected by the detecting unit to the unit signal generating unit. apparatus. The detection unit is arranged in a vertical direction and a horizontal direction on a two-dimensional plane,
A vertical charge transfer path for transferring the signal charge detected by the detection unit in the vertical direction;
A read gate for reading the signal charge from the detection unit to the vertical charge transfer path;
32. The apparatus for detecting physical quantity distribution according to claim 31, further comprising: a horizontal charge transfer path that transfers the signal charge transferred by the vertical charge transfer path to the unit signal generator in the horizontal direction. .

Images