JP2013153381A - Imaging apparatus and imaging method - Google Patents

Abstract

PROBLEM TO BE SOLVED: To reduce signal variation of a black image of an effective pixel in a CMOS imaging element.SOLUTION: In an imaging apparatus using a CMOS imaging element having an integration multiplication A/D section, signal values of respective pixels at each number of integration times of a maximum use temperature and at each bit shift are acquired and stored in a frame memory. In a plurality of reference temperatures, the signal values of the respective pixels of a black image in the CMOS imaging element in a maximum bit shift amount and the maximum number of integration times are acquired and are stored in the frame memory. One of an average value and unevenness of a black image signal of the effective pixel in the CMOS imaging element and respective effective pixel black image signal values are corrected by adapting for each temperature, each number of integration times and each bit shift by using the signal value of each pixel for each number of integration times and each bit shift of the maximum use temperature, which is stored in the frame memory, the signal value of each pixel of the black image of the CMOS imaging element in the maximum bit shift amount and the maximum number of integration times of the plurality of reference temperatures, which are stored in the frame memory, and information on the temperature of the CMOS imaging element.

Description

  The present invention relates to an imaging apparatus having a solid-state imaging device, and relates to a method for reducing noise contained in an image signal output from the imaging device.

A CMOS (Complimentary Metal Oxide Semiconductor) image sensor is generally less sensitive than a CCD image sensor. Therefore, a CMOS image sensor having an integral multiplication A / D section and aiming at higher sensitivity than the CCD image sensor is under development (see Patent Document 1). However, since the dark portion unevenness due to uneven signal output of the black image of the photo diode and the video signal (white scratch) of the photo diode having abnormally large black signal output are also integrated and multiplied, the white scratch and dark portion unevenness increase.
Therefore, conventionally, in order to reduce the influence of white scratches in the optical black pixel portion, the 12 pixel signals of the horizontal optical black pixel (Horizontal-Optical Black) (H-OB) portion of the CMOS image sensor are averaged. This is stored as a black signal reference signal for a line, and the stored black image reference signal is subtracted from the output signal of the effective pixel portion of the solid-state imaging device.

  However, the nonuniformity of the signal output of the black image is greatly correlated with the integral multiplication of the CMOS image pickup device having the integral multiplication A / D portion, and the white scratch and the dark portion unevenness are also integral multiplied. The level of is likely to increase. Further, the leakage current of the integral multiplication becomes a fluctuation of the reference dark part level when the integral multiplication is changed. Further, the fluctuation of the power supply and the GND becomes a fluctuation of the integral multiplication, and the dark portion is uneven when the integral multiplication is increased.

For this reason, even if the output signals of the H-OB portion are averaged, a white flaw component remains. Further, when this average signal is subtracted from the output signal of the effective pixel portion, a horizontal stripe is generated.
Since the dark current of the photodiode generally increases by 6 dB when the temperature of the image sensor rises by 6 ° C., the dark level, white scratches, and dark portion shading fluctuate due to temperature fluctuation. Further, the leakage current of integral multiplication increases as the temperature rises, so that the dark part level due to integral multiplication fluctuates due to temperature fluctuation.

For this reason, when the signal level change characteristic and the noise change characteristic of the fan amplifier integration frequency gain-up method and the bit shift gain-up method are verified, the dark part level and the dark part shading fluctuate due to the gain increase. Further, there is a problem that the dark part level fluctuates even with respect to temperature fluctuation.
For example, since the dark part level and the dark part shading increase due to the gain increase, the signal processing based on the video level 0 of the black image cannot be performed correctly. As for the output image, the black portion becomes whiter due to the increase in the video level of the black image, and the dark portion shading causes the image to become darker.

  Also, with the progress of integration of digital signal processing circuits, it is easy to store and arithmetically process output signals of multiple lines not only with video-only memory integrated DSPs but also with inexpensive general-purpose FPGAs (Field Programmable Gate Arrays). It became possible to realize.

JP, 2004-304413, A Integral multiplication A / D

  An object of the present invention is to reduce fluctuations in a dark part of an output of a CMOS image sensor having an integral multiplication A / D part.

  In order to solve the above-described problems, the present invention provides an image pickup apparatus using a CMOS image pickup device having an integral multiplication A / D section, and an average value of black image signals (effective black image signals) of effective pixels of the CMOS image pickup device. The bit shift amount and the integration count are either the average value of the image signal, that is, the dark portion level) and the unevenness (dark portion shading), or the value of the black image signal of each effective pixel (the image signal value of the black image, that is, the dark portion value). And an imaging method in which correction is performed according to the temperature of the CMOS image sensor.

  Further, in an image pickup apparatus using a CMOS image pickup device having an integral multiplication A / D unit, the number of integrations of the highest use temperature and the signal value of each pixel for each bit shift are acquired and stored in a frame memory, At the reference temperature, the signal value of each pixel of the black image of the CMOS image sensor at the maximum bit shift amount and the maximum integration count is acquired and stored in the frame memory, and each integration count of the maximum use temperature stored in the frame memory and The signal value of each pixel for each bit shift, the signal value of each pixel of the black image of the CMOS image sensor at the maximum bit shift amount and the maximum number of integrations of a plurality of reference temperatures stored in the frame memory, and the CMOS Using the temperature information of the image sensor, the effective pixels of the CMOS image sensor are adapted to each temperature, each integration count, and each bit shift. It is an imaging method for correcting the average value of the image signal of the black image (the dark area level) uneven (the dark portion shading), or one of the values of the image signal of the black image of each active pixel (dark region value).

  An image pickup apparatus using a CMOS image pickup device having an integral multiplication A / D unit includes a temperature sensor for detecting the temperature of the CMOS image pickup device, a signal processing unit, and a frame memory, and each integration of the maximum use temperature. The image signal value of each pixel for each number of times and each bit shift is acquired and stored in the frame memory, and at a plurality of reference temperatures, each pixel of the black image of the CMOS image sensor at the maximum bit shift amount and the maximum number of integrations is obtained. The image signal value is acquired and stored in the frame memory, the number of integrations of the maximum use temperature stored in the frame memory and the image signal value of each pixel for each bit shift, and a plurality of reference temperatures stored in the frame memory The image signal value of each pixel of the black image of the CMOS image sensor at the maximum bit shift amount and the maximum number of integrations, and the temperature sensor Using the information on the temperature of the CMOS image sensor, the average value (dark part level) of the image signal of the black image of the effective pixel of the CMOS image sensor is adapted to each temperature, each integration count, and each bit shift. This is an imaging apparatus that corrects one of unevenness (dark part shading) or the value (dark part value) of the image signal of a black image of each effective pixel.

  As described above, according to the present invention, the average value of the image signal of the black image of each pixel of the CMOS image sensor output having the integral multiplication A / D portion, that is, the dark portion level is constant, and an image signal in which the dark portion is stable is obtained. It is done.

Schematic diagram showing the characteristics of the image level of the number of integrations and the standard incident light quantity of a CMOS image sensor having an integral multiplication A / D section Schematic diagram showing ideal black image and video level Schematic diagram showing black image and video level when CMOS imaging device having integral multiplication A / D section is integrated 3 times. Schematic diagram showing black image and video level when CMOS imaging element having integral multiplication A / D section is integrated 16 times. Schematic diagram showing black image and video level when CMOS imaging element having integral multiplication A / D section is integrated 32 times. Schematic diagram showing the number of integrations of a CMOS image sensor having an integral multiplication A / D unit and the video level of a black image The value when the video level is set to 1 IRE with the bit shift [16: 9] is used as a reference value, and the characteristics of the bit shift and the video level according to the change amount of the video level when the bit shift from the reference to [7: 0] one by one Schematic diagram showing The value at bit shift [7: 0] and video level 100IRE is used as a reference value, and the bit shift and video level are changed by the amount of change in video level when the bit shift from the reference value to [16: 9] one by one. Schematic diagram showing characteristics Schematic diagram showing black image and video level of black image during bit shift [16: 9] Schematic diagram showing the black image and the video level of the black image during bit shift [9: 2] Schematic diagram showing black image and video level of black image at bit shift [7: 0] Schematic diagram showing the characteristics of bit shift and video level of black image 1 is a block diagram of an image pickup apparatus having a CMOS image pickup element having an integral multiplication A / D unit including a video level correction of a black image according to an embodiment of the present invention. The schematic diagram which shows the video level correction | amendment of the black image with respect to the gain-up and temperature of one Example of this invention

  A CMOS image sensor having an integral multiplication A / D section using FIGS. 1 to 11 which are schematic diagrams showing the characteristics of the image level of the number of integrations and the standard incident light quantity of the CMOS image sensor having an integral multiplication A / D section. The following describes the sensitivity improvement characteristics of the video level, the characteristics of the dark part level and the dark part shading, and the dark part changing operation by the gain-up method of the integration number.

  IRE is a unit that expresses the amplitude of a video signal defined by the Institute of Radio Engineers, one of the predecessors of IEEE (The Institute of Electrical and Electronics Engineers, Inc.). It became a name. 0.714V becomes 100IRE.

  Fig. 1 shows the characteristics of the number of integrations and the video level. The value at the time of integration of 32 times and the video level of 100 IRE was used as a reference value, and the amount of change in the video level when the number of integrations was reduced by 1 from the reference was measured. There is no data for the number of integrations 0 to 2 because it is out of the current specifications. In addition, the calibration is retaken every integration number.

  The amount of change in the video level of the black image due to the number of integrations is shown in FIG. 2 as an example of an ideal black image and video level. As an example of actual measurement values, FIG. FIG. 5 shows the black image and video signal level at 32 times.

  It can be seen that the video level increases linearly with the number of integrations for the integration frequency gain-up method. It can be seen that the black image video level also increases linearly with the number of integrations, as the black image video level varies with the number of integrations. FIG. 6 shows a characteristic diagram of the video level of a black image corresponding to the number of integrations while ignoring the white scratch / noise level.

  Next, the video level sensitivity enhancement characteristic and the video level characteristic of the black image by the bit shift gain increase method will be described.

  FIGS. 7A and 7B show the bit shift and video level characteristics of the display output ranges [16: 9] to [7: 0]. FIG. 7-1 shows the amount of change in the video level when the bit shift [16: 9] and the video level 1IRE are set as the reference value, and the bit level is shifted to [7: 0] one by one from the reference. . Fig. 7-2 shows the amount of change in the video level when the bit shift is [7: 0] and the video level is 100IRE, and the value is changed to [16: 9] one by one from the reference value. did.

  As an example of actual measurement values, the amount of change in the video level of the black image due to the bit shift is shown in FIG. 8 as the black image and video level during the bit shift [16: 9], and as shown in FIG. FIG. 10 shows the black image and the video signal level when the bit shift is [7: 0] times. Calibration was performed once in the initial stage. Ignore white scratch noise.

  It can be seen that the video level increases linearly with the bit shift in the bit shift gain up system. It can be seen that the video level of the black image increases linearly with the bit shift with respect to the change in the video level of the black image due to the bit shift. FIG. 11 is a characteristic diagram of the video level of the black image corresponding to the bit shift while ignoring the white scratch / noise level.

  FIG. 12 is a block diagram of an image pickup apparatus having a CMOS image pickup device having an integral multiplication A / D unit including a video level correction of a black image according to an embodiment of the present invention, and gain increase and temperature according to an embodiment of the present invention. The image level correction operation of the black image of the CMOS image sensor having the integral multiplication A / D unit according to the embodiment of the present invention will be described with reference to FIG. To do.

  In FIG. 12, 1 is an imaging device, 2 is an optical system, 3 is a CMOS image sensor having an integral multiplication A / D unit, 4 is a temperature sensor, 5 is a video level correction unit for black images, and 6 is a video image for black images. A level correction signal detector, 7 is a video signal processor, and 9 is a CPU. In the black image level correction unit 5 shown in FIG. 12, reference numeral 11 denotes a subtractor. In the black image video level correction signal detection unit 6 of FIG. 12, reference numerals 21 to 22 denote memory units, and reference numeral 23 denotes a black image video level correction signal calculation unit.

  The black image video level correction unit 5 and the black image video level correction signal detection unit 6 may be integrated into the video signal processing unit 7. Further, the memory unit 21 and the memory unit 22 may be integrated.

  In the black image video level correction signal detection unit 6 in FIG. 12, the video level of each black image up to 32 integrations is recorded in the memory 21 as a calibration value, and bit shifts [11: 4] to [7: 0] ] Are recorded in the memory 22 as calibration values. The calibration value recorded in the memory 21 and the memory 22 is extracted as a correction coefficient according to the number of integrations used and the bit shift, and the black image level correction signal calculator 23 calculates the black level video level correction signal from the correction coefficient. To do. The temperature sensor 4 detects the temperature of the CMOS image sensor having an integral multiplication A / D unit, and the CPU 9 detects the temperature of the CMOS image sensor having the integral multiplication A / D unit in the video level correction signal calculation unit 23 for a black image. The temperature information is sent, and the video level correction signal of the black image corresponding to the temperature is calculated from the correction coefficient.

  In the first embodiment, in the black image level correction signal detector 6 shown in FIG. 12, the black image level correction signal calculator 23 calculates the average value (dark portion level) and unevenness of the black image dark current according to the temperature. (Dark part shading) is calculated and output as a video level correction signal of a black image.

Then, the black image level correction signal calculated from the output Vi of the CMOS image sensor having the integral multiplication A / D unit is subtracted by the subtracting unit 11 of the black image level correcting unit 5 of FIG. Is sent to the video signal processing unit 7 as the post-video level corrected signal Vm.
The video signal processing unit 7 in FIG. 12 performs signal processing such as gamma, knee, contour correction, and color signal processing on the video level-corrected signal Vm of the black image, and outputs it from the imaging device 1 as an output video signal Vo.

  As an example of the temperature coefficient of the video level correction signal of the black image, the black image is assumed to have 0 to 50 ° C. in increments of 10 ° C. (or 5 ° C.), and the information calculated from the correction coefficient is multiplied by the coefficient corresponding to the temperature. Of the image level of the black image is calculated and subtracted from the output Vi of the CMOS image sensor having the integral multiplication A / D unit to set the video level of the black image after the video level correction of the black image to 0IRE. Is shown in FIG.

  As described above, according to the present invention, by correcting the average value of dark current (dark part level) and unevenness (dark part shading) by the bit shift amount, the number of integrations, and the temperature of the CMOS image sensor, the video level of the black image is corrected. No effect on signal processing based on. Furthermore, the dark portion does not increase whiteness due to gain increase, and an image signal in which the dark portion is stable can be obtained.

In the second embodiment, description of operations similar to those of the first embodiment will be omitted, and configurations and operations different from those of the first embodiment will be described.
The first embodiment calculates and corrects the dark current average value (dark portion level) and unevenness (dark portion shading), whereas the second embodiment calculates and corrects the dark current of each pixel of the CMOS image sensor.
In the video level correction signal detection unit 6 of the black image in FIG. 12, the calibration value of each pixel recorded in the memory 21 and the memory 22 is extracted as a correction coefficient according to the number of integrations used and the bit shift, and the video level of the black image The correction signal calculation unit 23 calculates a video level correction signal for each pixel of the black image from the correction coefficient for each pixel. The temperature sensor 4 detects the temperature of the CMOS image sensor having an integral multiplication A / D unit, and the CPU 9 detects the temperature of the CMOS image sensor having the integral multiplication A / D unit in the video level correction signal calculation unit 23 for a black image. The temperature information is sent, and the video level correction signal of each pixel of the black image corresponding to the temperature is calculated from the correction coefficient of each pixel.

In the second embodiment, the frame memory capacity is likely to be larger than in the first embodiment. However, a RAM (Random Access Memory) of about Gbit has a small size and low power consumption, and the price is about 100 yen or less. So no problem.
In the first embodiment, a so-called white defect of an abnormally large dark current cannot be corrected. However, in Example 2, since the dark current of each pixel is calculated and corrected, white defects can be corrected simultaneously.

  As described above, according to the second embodiment of the present invention, by correcting the dark current of each pixel of the CMOS image sensor including white flaws based on the bit shift amount, the number of integrations, and the temperature of the CMOS image sensor, The signal processing based on the video level is not affected. Furthermore, the dark portion does not increase whiteness due to gain increase, and an image signal in which the dark portion is stable can be obtained.

The present invention also provides a dark portion close to a black image by avoiding the influence of fluctuations due to the integral multiplication amount and temperature of the black image signal output of the photo diode of the CMOS image sensor having the integral multiplication A / D portion. The image signal can be effectively used over a wide temperature range. Since the image signal of the dark part can be effectively used, the imaging device is effectively made highly sensitive. By effectively increasing the sensitivity over this wide temperature range, imaging that requires high image quality even at low illuminances such as telemedicine for sleeping sick people, such as information cameras at broadcast stations (so-called weather cameras) and facial color checks of sleeping faces, etc. It can be used for applications. Furthermore, the present invention can be used for imaging applications such as long-distance railway monitoring and strait monitoring where lighting cannot be secured. It can be used for imaging applications such as monitoring of long-distance railways in Russia, Mongolia, Northeast China, and desert areas, which require a wide temperature range, and border monitoring in dry areas such as Mexico.
Furthermore, since the signal output of the high luminance image is maintained, the dynamic range of the image pickup apparatus is effectively expanded. By realizing both high sensitivity and dynamic range expansion over a wide temperature range, wide dynamic range imaging for high-definition monitoring becomes possible. Therefore, it should be used for monitoring applications to prevent the fall of railway platforms in backlit conditions where high definition and wide dynamic range are required over a wide temperature range, and for monitoring the simultaneous identification of the vehicle model, driver's face and license plate. Can do.

1: imaging device, 2: optical system,
3: CMOS image sensor having integral multiplication A / D section, 4: Temperature sensor,
5: Black image video level correction unit, 6: Black image video level correction signal detection unit,
7: Video signal processing unit, 9: CPU, 11: Subtraction unit,
21 to 22: memory unit, 23: black image video level correction signal calculation unit,

Claims (3)

In an imaging apparatus using a CMOS image sensor having an integral multiplication A / D section, the average value and unevenness of the black image image signal of the effective pixel of the CMOS image sensor, or the image signal of the black image of each effective pixel An imaging method in which one of the values is corrected according to the bit shift amount, the number of integrations, and the temperature of the CMOS image sensor.
In an imaging apparatus using a CMOS imaging device having an integral multiplication A / D unit, the number of integrations of the maximum operating temperature and the signal value of each pixel for each bit shift are acquired and stored in a frame memory, and a plurality of reference temperatures , The signal value of each pixel of the black image of the CMOS image sensor at the maximum bit shift amount and the maximum number of integrations is acquired and stored in the frame memory, and each integration number and each bit of the maximum operating temperature stored in the frame memory The signal value of each pixel for each shift, the signal value of each pixel of the black image of the CMOS image sensor at the maximum bit shift amount and the maximum number of integrations of a plurality of reference temperatures stored in the frame memory, and the CMOS image sensor The black image of the effective pixel of the CMOS image sensor is adapted to each temperature, each integration count and each bit shift using the temperature information of Imaging method for correcting the average value and irregularity of the image signal, or one of the values of the image signal of the black image of each active pixel. In an imaging device using a CMOS imaging device having an integral multiplication A / D section,
Acquire the signal value of each pixel for each integration count of each maximum use temperature and each bit shift and store it in the frame memory,
At a plurality of reference temperatures, the signal value of each pixel of the black image of the CMOS image sensor at the maximum bit shift amount and the maximum number of integrations is acquired and stored in a frame memory;
The CMOS imaging at each integration number of the maximum use temperature stored in the frame memory and the signal value of each pixel for each bit shift, and the maximum bit shift amount and the maximum integration number of the plurality of reference temperatures stored in the frame memory Using the signal value of each pixel of the black image of the element and the temperature information of the CMOS image sensor, the black image of the effective pixel of the CMOS image sensor is adapted to each temperature, each number of integrations, and each bit shift. The imaging method which correct | amends one of the average value and nonuniformity of an image signal, or the value of the image signal of the black image of each effective pixel. In an imaging device using a CMOS imaging device having an integral multiplication A / D section,
A temperature sensor for detecting the temperature of the CMOS image sensor, a signal processing unit, and a frame memory;
The number of integrations of the maximum operating temperature and the image signal value of each pixel for each bit shift are acquired and stored in the frame memory, and at a plurality of reference temperatures, the CMOS image sensor at the maximum bit shift amount and the maximum number of integrations is obtained. The image signal value of each pixel of the black image is acquired and stored in the frame memory,
The CMOS in each integration number of the maximum use temperature stored in the frame memory and the image signal value of each pixel for each bit shift, and the maximum bit shift amount and the maximum integration number of the plurality of reference temperatures stored in the frame memory Using the image signal value of each pixel of the black image of the image sensor and the temperature information of the CMOS image sensor detected by the temperature sensor, the CMOS is adapted to each temperature, each number of integrations, and each bit shift. An imaging apparatus that corrects one of an average value and unevenness of a black image image signal of an effective pixel of an image sensor, or a value of a black image image signal of each effective pixel.

Images