JP4057814B2 - Image information processing device - Google Patents

Description

[0001]
BACKGROUND OF THE INVENTION
The present invention relates to an image information processing apparatus, and more particularly to an image information processing apparatus capable of expanding a dynamic range of an image sensor unit.
[0002]
[Prior art]
In general, in an image information processing apparatus, an image sensor is used for an image pickup unit, and such an image sensor is usually created by a CMOS circuit technology.
Currently, CMOS image sensors are mainly used for image information processing apparatuses. However, since these image sensors capture images with the same exposure time for all pixels, the dynamic range is about 60 to 70 dB. And small. This is because the dynamic range of the photodetector itself in the image sensor cannot be fully utilized due to noise in the readout circuit and a decrease in the signal saturation level caused by lowering the power supply voltage to reduce power consumption.
For this reason, when shooting a subject in backlight or a subject with a large amount of light difference such as a window side seen from the room, the highlight incident part is saturated due to the limit of the dynamic range of the pixel circuit, and the subject in the entire image area is not enough. There is a problem that it is not possible to image with gradation.
[0003]
In order to solve these problems, Yang et al. Disclosed a method for changing the exposure time for each pixel according to the illuminance and displaying the illuminance of the light incident on the image sensor as a floating point. is doing.
D. Yang, A. E1 Gamal, B Fowler, H Tian, "A 640 × 512 CMOS Image Sensor with Ultrawide Dynamic Range Floating-Point Pixel-Level ADC", IEEE J. Solid-State Circuits, vol. 34, pp. 1821-1834, Dec 1999.
A feature of the Young et al. Method is that a pixel circuit capable of variably setting the exposure time for each pixel in accordance with the illuminance irradiated to the pixel is provided.
[0004]
That is, the exposure time is variably set for each pixel so that the exposure time is lengthened when the illuminance is small and the exposure time is shortened when the illuminance is large.
Here, the exponent part is the exposure time (T), the mantissa part is the output voltage (V) of the pixel, and the exposure time (T) and the illuminance (E) are expressed as in equation (1).
T = T ₀ × 2 ^{n
E = V × 2 - n (} 1)
Here, E represents illuminance, V represents output voltage at time T, and T represents exposure time.
That is, in Expression (1), the range of illuminance values that can be expressed by floating-point expression of the output voltage V using the exposure time T of the pixel that has output the output voltage V is expanded. Therefore, the dynamic range can be expanded.
[0005]
[Problems to be solved by the invention]
Thus, Young's method expands the dynamic range by changing the exposure time in accordance with the illuminance.
However, in the method of variably setting only the exposure time for each pixel, it is necessary to lengthen the exposure time in order to capture a dark part. In this case, when it is necessary to capture an image with a relatively short exposure time, such as a moving object, there is a problem that it is difficult to clearly image the object.
The present invention has been made in order to solve the above-described problem. When there is a large difference in brightness between the subjects included in the imaging region, the subject can be captured with sufficient gradation even if it is captured with a relatively short exposure time. An object of the present invention is to provide an image information processing apparatus that can be used.
[0006]
[Means for Solving the Problems]
The image information processing apparatus according to the present invention includes a pixel circuit capable of variably setting an exposure time for each pixel in accordance with the illuminance applied to the pixel, and a first pixel value output from the selected pixel. And a readout circuit that calculates a second pixel value by multiplying by a predetermined gain corresponding to the pixel value.
According to the present invention, in the image information processing apparatus, the illuminance E applied to the pixel is calculated by the following equation.
Exposure time T,
T = T ₀ × 2 ⁿ (n = 0, 1, 2,...)
Gain G
G = 2 ^m (m = 0, 1, 2,...)
When changing the illuminance E,
E = V × 2− ^{(n + m)}
Here, T ₀ is the minimum exposure time, and V is the second pixel value.
[0007]
Further, in the image information processing apparatus of the present invention, the gain is obtained by converting the first pixel value into a PWM signal by changing the slope of the ramp waveform signal in accordance with the first pixel value. To do.
In the image information processing apparatus of the present invention, a plurality of ramp waveform signals having different inclinations are prepared in advance, and any one of the ramp waveform signals is selected according to the first pixel value.
In the image information processing apparatus of the present invention, a memory circuit that stores only the exposure time, the gain and the second pixel value or the exposure time for each pixel is provided.
The image information processing apparatus of the present invention is characterized in that the first pixel value is obtained from an output value from a selected pixel through a noise removal circuit that removes reset noise.
Furthermore, in the image information processing apparatus of the present invention, the noise removal circuit is a CDS circuit.
[0008]
DETAILED DESCRIPTION OF THE INVENTION
Hereinafter, embodiments of the present invention will be described in detail with reference to the drawings.
The feature of the present invention is that the conventional image sensor sets the illuminance with the same exposure time for all the pixels, whereas the exposure time is variably set for each pixel according to the illuminance irradiated to the pixels. Further, when reading out each pixel value, the pixel value is read out by changing the gain of the readout circuit.
Here, when the gain is increased by n times, there is an effect similar to that when the exposure time is increased by n times. That is, by changing both the exposure time and the gain for each pixel, the dynamic range of the image sensor can be expanded while suppressing an increase in the exposure time.
[0009]
FIG. 1 is a diagram showing an overall block configuration of an image sensor unit of an image information processing apparatus according to the present invention.
The image sensor unit includes a pixel circuit 21, a readout circuit 22, and a memory circuit 23.
The pixel circuit 21 includes a reset circuit 11, a photodetector (PD) 12, an IV conversion circuit 13, an exposure time determination circuit 14, and an output amplifier 15.
The reset circuit 11 initializes the pixel value held in each pixel circuit. The PD 12 generates a photocurrent according to the illuminance applied to the pixel, and the photocurrent is converted into a voltage value by the IV conversion circuit 13 and output to the exposure time determination circuit 14. The voltage value and a preset threshold value are compared by the exposure time determination circuit 14, and the exposure time is determined for each pixel circuit. The exposure time determined for each pixel is written and stored in the memory circuit 23 via the output amplifier 15.
[0010]
The read circuit 22 includes a CDS circuit 16, an AGC circuit 17, and an AD conversion circuit 18. When reading each pixel value, first, a target pixel is selected, and the output amplifier 15 is operated to read the pixel value. Since pixels are usually arranged in an array, reset noise for each pixel is removed by a CDS (Correlated Double Sampling) circuit 16 at the end of a column or row of the pixel array. Based on the output value after noise removal, an AGC (Auto Gain Control) circuit 17 multiplies a predetermined gain according to the pixel value to adjust the gain. The gain set here is written and stored in the memory circuit 23.
Next, the output value of the AGC circuit 17 is A / D converted by the A / D conversion circuit 18 and written to the memory circuit 23.
Note that the CDS circuit 16 is used as necessary to remove noise between the pixel circuits, and is not necessarily required.
[0011]
In this way, three types of information of pixel value, exposure time, and gain can be obtained for each pixel. By obtaining the illuminance of incident light using the following equation based on these three types of information, the dynamic range of the image sensor can be expanded as compared with the case where only the pixel output voltage is used.
Exposure time T,
T = T ₀ × 2 ⁿ (n = 0, 1, 2,...)
Gain G
G = 2 ^m (m = 0, 1, 2,...)
When changing the illuminance E,
E = V × 2− ^{(n + m)}
Here, T ₀ is the minimum exposure time, and V is the second pixel value.
[0012]
FIG. 2 is a layout diagram of the first embodiment of the image information processing apparatus of the present invention. A pixel circuit 21 is arranged on a two-dimensional matrix, and a readout circuit 22 is arranged for each row or column. An X-direction address generation circuit 24 and a Y-direction address generation circuit 25 are arranged around the pixel array, and a predetermined pixel is selected by applying a predetermined address signal, and a pixel value and an exposure time are selected for each selected pixel. The gain is written into the memory circuit 23. Data reading to the outside is performed via the memory circuit 23.
[0013]
FIG. 3 is a circuit diagram illustrating a configuration example of the pixel circuit 21, and FIG. 4 is a circuit diagram illustrating a configuration example of the readout circuit 22.
FIG. 5 is a circuit diagram showing a configuration example of the CDS circuit 16, and FIG. 6 is a circuit diagram showing a configuration example of the AGC circuit 17.
Further, FIG. 7 shows an operation timing chart of FIGS. The operation of the first embodiment will be described with reference to FIGS.
[0014]
The operation can be roughly divided into two phases. The first phase is a phase in which exposure time is set for each pixel while light is incident on all pixels and exposure is performed, and the value (time stamp) is written in the memory circuit 23.
The second phase is a phase in which after amplification with a gain set for each pixel, the output is A / D converted and written to the memory circuit 23 together with the gain set as a pixel value.
The phase in which the exposure time is determined and the time stamp is written in the memory circuit is shown as t1 to t9 in FIG. In this first phase, the shutter signal is activated, and the mode change-over switches 31 and 41 in each pixel circuit 21 and readout circuit 22 are set to the exposure time readout mode as shown in FIGS.
[0015]
In addition, the reset (RESET) signal is also activated, the reset switch 32 is turned on, and the value held in the sample capacitor 33 in each pixel is simultaneously measured for all pixels. Initialize to RESET. Also, the value of the register 42 in the reading circuit is initialized. Subsequently, in a period from t2 to t3 when the reset signal is active and Sampling Clk is also active, the switch 37 is turned on, and the PD12 (Vpd35) portion is similarly initialized.
FIG. 8 shows another example of the configuration of the readout circuit 22. In this example of configuration, the values of the register 42 and the counter 83 are initialized at the time of initialization. The register 42 in FIGS. 4 and 8 may be provided for all of the reading circuits 22 in FIG.
[0016]
From the timing t3 when the initialization ends, a photocurrent flows according to the amount of light incident on the PD 12, and the voltage Vsample34 between the terminals of the sample capacitor 33 decreases. The comparator 36 is used to compare the Vsample 34 with the threshold value (Vth) at each sampling clock (Sampling Clk) timing (t4 to t7) at intervals of 2 ⁿ . If the voltage Vsample is lower than the threshold value Vth, the output of the comparator 36 is inverted at that time, the switch 37 is turned OFF, and the exposure of the pixel is completed.
At the same time, the switch 38 is turned ON, and the terminal voltage Vpd35 of the PD 12 is set to the initial voltage BIAS. Fix to RESET. After the maximum exposure time (t8) elapses, the threshold Vth is set to the reset voltage (BIAS) in order to end the exposure of all pixels. The above comparison operation is performed with a value higher than (RESET).
[0017]
Thereafter, a read (READ) signal for selecting the pixel is activated using the X-direction address generation circuit 24 and the Y-direction address generation circuit 25, and if the exposure of the selected pixel is completed, the readout circuit 22 is set inside. The value of the register 42 counting a certain exposure time is written in the memory circuit 23. Thus, the comparison between the voltage Vsample and the threshold value Vth is repeated until the maximum exposure time has elapsed (t3 to t8). In order to perform the comparison only immediately after the write comparator 36 to the memory circuit 23 is inverted, the time stamp is checked while confirming whether the content of the memory circuit 23 of the selected pixel is an initial value or other value. Write.
[0018]
In the above-described embodiment, the sampling clock of 2 ⁿ intervals is used. However, when the exposure time interval is set to 2 ⁿ , 2 ⁻ⁿ is used to correct an A / D conversion result described later using the exposure time. Just double it and get better. This is set in this way because it can be easily realized by shifting the digital value by 1 bit.
When the maximum exposure time (t8) elapses, the shutter signal is made inactive, and the exposure ends (t9). This completes the first phase. Next, the mode change-over switches 31 and 41 in each pixel circuit 21 and readout circuit 22 are set to the pixel value readout mode (the reverse of the exposure time readout mode). Thereby, the second phase is started. First, a pixel to be read is selected, and the reed switch 38 is turned on. Next, the pixel value of the selected pixel is read out. At this time, by taking the difference between the pixel value read using the CDS circuit 16 and the pixel value at the time of resetting, it becomes possible to remove the reset noise between the pixels.
[0019]
FIG. 5 is a circuit diagram showing a configuration example of the CDS circuit 16. At time t10, the CDS1 switch 51 is turned on, the CDS2 switch 52 is turned off, and the pixel value after exposure is read from Vin. Next, the CDS1 switch 51 is turned off at the timing of t11, the CDS2 switch 52 is turned on, and the pixel value at the time of reset is similarly read out from Vin (t12 to t13), and the pixel value after exposure is removed. And the difference between the reset pixel value and Vout is read out from Vout.
The CDS circuit 16 can be variously modified in addition to the configuration example shown in FIG.
[0020]
Next, the signal Vin is input to the AGC circuit 17 shown in FIG. The AGC circuit includes a voltage determination unit that determines the magnitude of an input signal and an amplification unit that can set a plurality of gains.
The voltage determination unit compares the magnitude of the input signal Vin with several types of threshold values Vth (1) and Vth (2) using the comparators 61 and 62, and determines the gain of the amplifier circuit at the next stage to determine the memory circuit 23. Output to. In FIG. 6, when the gain is 1, 2, 4 times, values of 11, 01, 00 are written in the memory. The switches 63, 64, and 65 are switched according to the set gain to amplify the input signal (t13 to t14). The amplified signal Vout is A / D converted into a digital value and written to the memory circuit 23. .
[0021]
In this manner, three pieces of information such as exposure time, gain, and pixel value necessary for expanding the dynamic range of the image sensor can be obtained. Here, it is assumed that the exposure time is expressed in 5 steps (3 bits), the gain is expressed in 3 steps (2 bits), and the pixel value is expressed in 256 steps (8 bits). As the exposure time increases, the time stamp representing the exposure time changes from 100 to 000. The gain also changes from 11 to 00 as the gain increases. The pixel value changes from 0000000 to 11111111 depending on the illuminance. Using the time stamp obtained in this way, the 8-bit pixel value is multiplied by ²ⁿ , and the gain is set to 00 → 2 ⁰ , 01 → 2 ¹ , 11 → 2 ² times to obtain the pixel value It is possible to represent information more than 8-bit grayscale information using only the. That is, an image with an expanded dynamic range can be obtained.
[0022]
Next, a second embodiment of the present invention will be described.
In the second embodiment, the configurations of the AGC circuit 17 and the readout circuit 22 are different from those in the first embodiment.
FIG. 8 is a circuit diagram showing a configuration example of the readout circuit used in the second embodiment.
FIG. 9 is a circuit diagram showing a configuration example of the AGC circuit.
[0023]
The second embodiment is characterized in that the pixel value output from the selected pixel is converted into a PWM signal by selecting the slope of the ramp waveform signal according to the value, and gain is obtained. That is, the input voltage is converted into a PWM (Palse Width Modulation) signal proportional to the magnitude using the ramp waveform signal, and the time is counted by the clock signal. Then, the counted value (digital value) becomes a value proportional to the input voltage value (analog value), and AD conversion can be performed.
[0024]
FIG. 10 is a diagram showing a configuration example of the ramp waveform generating circuit, and FIG. 11 is a diagram showing the relationship between the slope of the ramp waveform and the input pulse to the counter unit. In the readout circuit, the analog signal is converted into a PWM signal by comparing the magnitude of the signal (input signal) after the CDS processing and the ramp waveform signal by the comparator 81. Thereafter, a logical product with the clock is obtained by the logical product circuit 82 and counted by the counter 83 to be converted into a digital value. The result is written in the memory circuit 23. The ramp waveform generating circuit shown in FIG. 10 includes a capacitor 101, a reset switch 102, and a constant current source 103. First, the reset voltage (BIAS RAMP RESET) is charged in the capacitor 101 by turning on the reset switch 102.
[0025]
Next, the reset switch 102 is turned off to charge the capacitor according to the current amount of the constant current source 103. As a result, ascending waveform (ramp waveform) signal having an inclination corresponding to the amount of current is output as shown in FIG. In the AGC circuit shown in FIG. 9, the switches 91, 92 and 93 are sequentially switched according to the magnitude of the input signal, and the slope of the ramp waveform is selected from a, b and c as shown in FIG. The gain can be adjusted. In this configuration, a ramp waveform that rises to the right is used, but depending on the configuration of the CDS circuit, a ramp waveform that descends to the right can also be used.
[0026]
As described above, in the second embodiment, the pixel value output from the selected pixel is converted into a PWM signal, and the pulse width is counted by the clock signal to perform A / D conversion. In the first and second embodiments (FIGS. 3, 4, and 8), one output line from the pixel is used by switching with a switch. However, one output line is used for each output. A configuration may be adopted in which the output lines are used one by one (the total is two) and the switching by the switch is not performed.
[0027]
【The invention's effect】
As described in detail based on the above embodiments, in the present invention, a pixel circuit and a readout circuit that can variably set the exposure time and gain for each pixel are provided, and these values are used for the image sensor. The dynamic range can be expanded. Therefore, it is possible to provide an image information processing apparatus that can capture each subject with sufficient gradation even when the difference in brightness between subjects included in the imaging region is large or even a moving object.
[Brief description of the drawings]
FIG. 1 is a block diagram showing the overall configuration of an image sensor unit used in an image information processing apparatus of the present invention.
FIG. 2 is a diagram illustrating an arrangement of image sensor units according to the first embodiment of the present invention.
FIG. 3 is a circuit diagram showing a configuration example of each pixel circuit used in the first embodiment.
FIG. 4 is a circuit diagram illustrating a configuration example of a reading circuit used in the first embodiment;
FIG. 5 is a circuit diagram showing a configuration example of a CDS circuit used in the first embodiment.
FIG. 6 is a circuit diagram showing a configuration example of an AGC circuit used in the first embodiment.
FIG. 7 is an operation timing chart showing the circuit operation of FIGS.
FIG. 8 is a circuit diagram showing a configuration example of a readout circuit used in the second embodiment of the present invention.
FIG. 9 is a circuit diagram showing a configuration example of an AGC circuit used in the second embodiment of the present invention.
FIG. 10 is a circuit diagram showing a configuration example of a ramp waveform generating circuit used in the second embodiment of the present invention.
FIG. 11 is a diagram showing the relationship between the slope of the ramp waveform and the number of input pulses to the counter unit for explaining the operating principle of the second embodiment of the present invention.
[Explanation of symbols]
11 Reset circuit 12 Photo detector 13 IV conversion circuit 14 Exposure time determination circuit 15 Output amplifier 16 CDS circuit 17 AGC circuit 18 A / D conversion circuit 21 Pixel circuit 22 Reading circuit 23 Memory

Claims (7)

A pixel circuit that can variably set the exposure time for each pixel according to the illuminance irradiated to the pixel,
In the first images information processing apparatus it and a read circuit for calculating a second pixel value is multiplied by a predetermined gain in accordance with the first pixel value to the pixel value output from the selected pixel There,
The exposure time T and the gain G are determined when the parameters m and n of the relational expressions of the illuminance E applied to the pixel and the exposure time T and the gain G established based on the illuminance E simultaneously satisfy the following expressions. Image processing apparatus characterized by
Exposure time T,
T = T0 × 2n (n = 0, 1, 2,...)
Gain G
G = 2m (m = 0, 1, 2,...)
When changing the illuminance E,
E = V × 2- (n + m)
Here, T0 is the minimum exposure time, and V is the second pixel value. The image information processing apparatus according to claim 1,
An image information processing apparatus, wherein the gain is obtained by converting the first pixel value into a PWM signal by changing a slope of a ramp waveform signal in accordance with the first pixel value. The image information processing apparatus according to claim 2 ,
An image information processing apparatus, wherein a plurality of ramp waveform signals having different inclinations are prepared in advance, and any one of the ramp waveform signals is selected according to the first pixel value. The image information processing apparatus according to claim 1,
An image information processing apparatus comprising a memory circuit for storing the exposure time, the gain, and the second pixel value for each pixel. The image information processing apparatus according to claim 1,
An image information processing apparatus comprising a memory circuit for storing the exposure time for each pixel. The image information processing apparatus according to claim 1,
An image information processing apparatus characterized in that the first pixel value is obtained from an output value from a selected pixel through a noise removal circuit for removing reset noise. The image information processing apparatus according to claim 6 .
An image information processing apparatus, wherein the noise removal circuit is a CDS circuit.

Images