JP2004080189A - Signal processor and signal processing method for solid state imaging device - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To expand the dynamic range of a photoelectric conversion element by effectively utilizing a range over a knee characteristic range of the charge accumulation quantity in the element, without using a combination of a usual exposure with a short time exposure. <P>SOLUTION: A signal processor circuit 4 previously measures the input/output characteristics (relation between the input and output levels of an incident light ) in nonlinear ranges around and above a knee point of a solid state imaging device 1 every pixel and stores the result in a memory 5. In the actual imaging time, it counts back the level of the incident light from an output value of the imaging device 1, based on the stored input/output characteristics, and calculates its imaging signal, using the level of the incident light outputted by the count back, so that the signal is utilizable as an output signal even in ranges around and over the knee point being a saturation range of the photoelectric conversion element, thus expanding the dynamic range in the practical use. <P>COPYRIGHT: (C)2004,JPO

Description

[0001]
TECHNICAL FIELD OF THE INVENTION
The present invention relates to a signal processing device and a signal processing method for a solid-state imaging device used in various camera systems and the like.
[0002]
[Prior art]
2. Description of the Related Art Conventionally, the charge accumulation amount (saturation signal level) of a photoelectric conversion element constituting a pixel array of a solid-state image sensor has variation for each pixel, and appears as fixed pattern noise at the time of high luminance.
Due to this limitation, in the conventional solid-state imaging device, since the light amount detection is performed using only the portion where the charge storage characteristics of each photoelectric conversion element can ensure linearity as a whole, the smallest saturation signal of each photoelectric conversion element is used. The overall dynamic range was regulated by the level.
FIG. 4 is an explanatory diagram illustrating an example of a use characteristic region in which two (Knee) characteristics of two pixel signals, a variation in a saturation signal level, and the like are used.
As shown in the figure, since the saturation signal levels Qs1 and Qs2 are different due to the difference between the two characteristics S1 and S2 of the two pixel signals, the area between the point a and the point where the relationship between the light quantity and the output signal is linear is used. It is necessary to do.
[0003]
On the other hand, when imaging a scene in which an extremely bright (high luminance) object and a relatively dark (low luminance) object are mixed, such as when photographing an indoor area to an outdoor area, a high luminance object or a low Adjusting the exposure time at an appropriate level corresponding to any one of the luminance subjects has a problem that overexposure or underexposure occurs because the dynamic range of the solid-state imaging device is narrow.
[0004]
Therefore, in recent years, in order to solve such a problem, a signal VH obtained by charge accumulation by a normal exposure time (hereinafter, referred to as a normal exposure signal) and a signal VH obtained by charge accumulation by a short time exposure (hereinafter, referred to as a short signal). A method has been proposed in which the dynamic range is expanded by signal processing in which VL and a time exposure signal are each adjusted to an appropriate level and added (for example, Japanese Patent Application No. 11-271238). reference).
FIG. 5 is an explanatory diagram showing signal processing when a high dynamic range image is obtained by adding such a normal exposure signal and a short exposure signal.
For example, normal exposure and short-time exposure are separately performed using the electronic shutter function of the solid-state imaging device, and the normal exposure signal VH shown in FIG. 5A and the short exposure shown in FIG. By adjusting the levels of the time exposure signals VL and adding them by the adder 10, a high dynamic range image as shown in FIG. 5C can be obtained.
[0005]
[Problems to be solved by the invention]
However, in the method of combining the normal exposure and the short-time exposure as in the above-described related art, there is a problem that the exposure operation needs to be performed a plurality of times, and the exposure control is complicated.
In addition, VH and VL signals are added at an appropriate level in the signal processing. However, the signal processing becomes complicated, and a shift occurs in a synthesized image due to different timings taken by multiple exposures. There was such a problem.
[0006]
Therefore, an object of the present invention is to provide a solid-state imaging device capable of expanding a dynamic range by effectively utilizing a region equal to or more than a knee characteristic region of a charge storage amount by a photoelectric conversion element without using a combination of normal exposure and short-time exposure. And a signal processing method.
[0007]
[Means for Solving the Problems]
In order to achieve the above object, the present invention measures correction data calculation means for calculating correction data for a saturation region of a photoelectric conversion element by measuring input / output characteristics of each pixel of a solid-state imaging device in advance; A storage unit for storing the correction data calculated by the data calculation unit, and a saturation curve of the photoelectric conversion element by correcting the imaging data from the solid-state imaging device using the correction data during actual imaging. And a correction operation means for approximating the equation to an ideal curve.
The present invention also provides a correction data calculation step of calculating correction data for a saturation region of a photoelectric conversion element by measuring input / output characteristics of each pixel of a solid-state imaging device in advance, and calculating the correction data by the correction data calculation step. Storing the corrected data in the storage means, and correcting the image data from the solid-state image sensor using the correction data at the time of actual imaging, so that the saturation curve of the photoelectric conversion element is ideal. And a correction operation step of approximating the curve.
[0008]
In the present invention, the input / output characteristics of each pixel of the solid-state imaging device are measured in advance to calculate and store correction data for the saturation region of the photoelectric conversion device. By correcting the data using the correction data, since the saturation curve of the photoelectric conversion element is approximated to an ideal curve, without using a method of combining normal exposure and short-time exposure as in the above-described prior art. In addition, an output signal can be obtained by effectively using a region where the amount of charge stored by the photoelectric conversion element is saturated, that is, a region equal to or larger than the knee characteristic region, and the dynamic range of the solid-state imaging device can be expanded.
[0009]
BEST MODE FOR CARRYING OUT THE INVENTION
Hereinafter, embodiments of the present invention will be described in detail with reference to the drawings.
The signal processing device and the signal processing method according to the present embodiment are designed to measure the input / output characteristics (the relationship between the input level and the output level of the input light) of the solid-state imaging device before and after the knee point and in a region having no linearity above the knee point. Measured and stored for each pixel, and based on the stored input / output characteristics, inversely calculates the level of the input light from the output value at the time of actual imaging, and uses the level of the input light obtained by the inverse calculation to obtain an image signal. By calculating, the region before and after the knee point, which is the saturation region of the photoelectric conversion element, and the region above the knee point can be used as an output signal, thereby expanding the practical dynamic range.
That is, in the present embodiment, the dynamic range can be expanded by effectively utilizing the area before and after the knee point having no linearity, which has not been used conventionally, and the area above the knee point.
[0010]
FIG. 1 is a block diagram illustrating a configuration example of a camera system including a solid-state imaging device and a signal processing device thereof according to an embodiment of the present invention.
As illustrated, the camera system according to the present embodiment includes a solid-state imaging device 1, a CDS / AGC circuit 2, an A / D circuit 3, a signal processing unit 4, and a storage device 5, and a subject (light source A). Is performed.
[0011]
The solid-state imaging device 1 is, for example, a CCD image sensor, a CMOS image sensor, or the like, and has an imaging region in which a plurality of unit pixels each including a photoelectric conversion element such as a photodiode are arranged in a two-dimensional array on a semiconductor chip. The reflected light obtained from the subject is converted into a pixel signal by each unit pixel and output in the form of an analog image signal.
The CDS / AGC circuit 2 performs processing such as noise removal by CDS (correlated double sampling) and AGC (automatic gain control) on the image signal from the solid-state imaging device 1, and the A / D circuit 3 This converts the analog image signal processed by the CDS / AGC circuit 2 into a digital image signal.
[0012]
The signal processing unit 4 includes, for example, a DSP (Digital Signal Processor) and a microprocessor. The signal processing unit 4 performs various operations such as color separation, white balance, and gamma correction on the digital image signal output from the A / D circuit 3. It performs signal processing. In particular, in the present example, the input / output characteristics (the relationship between the input level and the output level of the input light) of the region having no linearity before and after the knee point of the solid-state imaging device 1 and beyond the knee point are measured and stored for each pixel. The level of the input light is inversely calculated from the output value of the solid-state imaging device 1 at the time of actual imaging based on the arithmetic processing for storing in the device 5 and the stored input / output characteristics, and the level of the input light obtained by the inverse calculation Has an arithmetic function of performing arithmetic processing for calculating an image pickup signal by using.
The storage device 5 stores various operation data calculated by the signal processing unit 4.
[0013]
The light source A is a measurement reference for measuring input / output characteristics (relationship between input level and output level of input light) for each pixel before and after the knee point of the solid-state imaging device 1 and in a region having no linearity beyond the knee point. It is assumed that the light source is a light source and has a function of controlling the light amount level with high accuracy.
The configuration shown in FIG. 1 is an example of a camera system to which the present invention can be applied, and does not particularly limit the present invention. For example, a configuration in which CDS or AGC processing is performed after A / D conversion or a case where the solid-state imaging device 1 itself has a digital output function is applicable.
[0014]
Next, a signal processing method for expanding the dynamic range in the camera system having such a configuration will be described.
First, as a first method, for example, at the time of manufacturing a camera system, light from a light source A is input to the solid-state imaging device 1 from low-level light to high-level light, and the input / output characteristics of each pixel of the solid-state imaging device 1 are changed. It is stored in the storage device 5.
FIG. 2 is an explanatory diagram showing an example of input / output characteristics of a certain pixel. The vertical axis indicates the amount of charge stored in the photoelectric conversion element (output level), and the horizontal axis indicates the amount of light from the light source A. For convenience, the ordinate represents points b, d, and f, and the abscissa represents points a, c, e, and g. And the point f is the saturation signal level.
The knee characteristic at this time can be approximately approximated by the saturation signal level f and the gradient {(dc) / (ba)}.
Here, as a specific storage method of the knee characteristic in the storage device 5, 1 byte is assigned to the saturation signal level f and 2 bytes are assigned as the gradient {(dc) / (ba)}, and the solid-state imaging device 1 Assuming that the number of effective pixels is N, it can be realized with a storage capacity of about 3N bytes.
[0015]
Next, a description will be given of a method of calculating an output signal in a region where there is no linearity before and after the knee point and at or above the knee point for expanding the dynamic range.
First, the light amount of the light source A is changed and input stepwise from low-level light to high-level light so that output signals corresponding to the light amounts a, c, and g shown in FIG. 2 are obtained. Then, the signal processing unit 4 measures the output signal for each pixel, and temporarily stores the value in the storage device 5 as raw image data.
Next, based on the raw image data, arithmetic processing is performed by the signal processing unit 4, and the saturation signal amount f and the slope corresponding to the pixel are calculated and stored in the storage device 5. The saturation signal amount f can be obtained from the output signal corresponding to the light amount g, and the slope can be calculated from the output signals b and d corresponding to the light amounts a and c as {(dc) / (ba)}.
[0016]
Next, processing at the time of actual imaging will be described.
FIG. 3 is an explanatory diagram showing the variation of the two characteristics and the saturation signal level of the two pixel signals at the time of actual imaging, in which the vertical axis represents the charge accumulation amount (output level) of the photoelectric conversion element, and the horizontal axis represents the distance from the subject. Are shown. The contents shown in FIG. 3 basically correspond to the contents shown in FIG.
Here, at the time of actual imaging, the raw imaging data obtained from the solid-state imaging device 1 is corrected using the saturation signal amount and the inclination stored in the storage device 5.
First, when the level of the raw imaging data obtained at the time of actual imaging is smaller than the saturation signal level (that is, within the range of the arrow a shown in FIG. 3), the level of the raw imaging data becomes Since the image data substantially exists on the inclination line of {(dc) / (ba)}, the level of the raw image data becomes the output level.
[0017]
However, when the level of the raw imaging data obtained at the time of actual imaging is close to the saturation signal level, the linearity of the characteristic is impaired, and a saturation curve as shown in FIG. 2 is obtained. The level of the imaging data is approximated by a gradient line of {(dc) / (ba)}.
That is, as shown in FIG. 2, on the inclination line of {(dc) / (ba)}, the light amount corresponding to the saturation signal level is at point e, and therefore, based on the level of the raw imaging data. , The corresponding light quantity from point c to point e is calculated back. Then, the output level on the slope line corresponding to this light amount is output as the output level of the pixel.
In this manner, for a pixel having a low saturation signal level, the output signal is linearly expressed by the slope line as the ideal curve in the saturation region where the linearity of the input / output characteristics cannot be ensured (the region before and after the knee point and the region above the knee point). By approximation, the dynamic range can be expanded.
For example, as shown in FIG. 3, the dynamic range of a pixel having a small saturation signal level can be expanded from arrow a to arrow b, which can contribute to the expansion of the dynamic range of the entire solid-state imaging device.
[0018]
In the above-described signal processing method, correction data is stored at the time of manufacturing the camera system. However, the correction data can be rewritten according to the situation, such as when the color temperature of the light source is different.
In this case, by using the electronic shutter of the solid-state imaging device to adjust the light amount stepwise from a low level to a high light for a light source having a constant light amount, it is possible to input the same correction data as described above.
Further, in the above-described example, the saturation curve is approximated from the saturation signal level f and the slope line. However, as a measurement method for approximating a more accurate ideal curve, a polygonal line approximation between two points is performed a plurality of times to approximate the curve. It is also possible to store the graph table data for each pixel and read out the value to perform signal processing.
In the above-described example, the correction is performed for each pixel. However, in order to reduce the capacity of the storage device 5 and improve the processing speed, an area including a plurality of pixels is set as a unit area, and the correction process is performed for each unit area. Such a configuration may be used.
In other words, as a specific approximation or correction method, various methods can be adopted, from a simple one to a high-precision one, depending on, for example, the grade and application of the camera system.
[0019]
Further, in the above embodiments, an example has been described in which the present invention is configured as a signal processing device of a solid-state imaging device. However, the present invention provides a solid-state imaging device having a solid-state imaging device and a signal processing circuit having similar functions. It is also possible to regard it as an imaging device.
FIG. 6 is an explanatory diagram showing one configuration example of such a solid-state imaging device, and shows an example in which a CCD image sensor and a signal processing circuit are provided on one semiconductor chip 11.
In this solid-state imaging device, unit pixels 120 each including a photoelectric conversion element such as a photodiode are provided in a two-dimensional array in a two-dimensional pixel array 110, and a plurality of vertical pixels are arranged along each column of the unit pixels 120. A register 130 is provided. Then, each vertical register 130 reads out the signal charges generated and accumulated by each unit pixel 120, transfers the signal charges in the vertical direction, and outputs the signal charges to a horizontal register 140 provided on the side of the two-dimensional pixel array 110.
[0020]
The horizontal register 140 transfers the signal charges from each of the vertical registers 130 in the horizontal direction for each row, and outputs the signal charges to the output unit 150.
The output unit 150 converts the signal charge transferred from the horizontal register 140 into a voltage signal via the floating diffusion unit, further amplifies the voltage signal, and outputs the voltage signal to the signal processing circuit 160 at the subsequent stage.
The signal processing circuit 160 includes, for example, the CDS / AGC circuit 2, the A / D circuit 3, the signal processing unit 4, the storage device 5, and the like shown in FIG.
Although not shown in the figure, a gate insulating film, transfer electrodes for each of the registers 130 and 140, a light-shielding film, an interlayer insulating film, and various other components are provided on the semiconductor chip 11 provided with the two-dimensional pixel array 110 and the like. A wiring layer and the like are stacked, and a color filter and a microlens are mounted thereon in an on-chip structure.
The details of the processing in the signal processing circuit 160 are the same as those described in the example of FIG.
Further, in the example shown in FIG. 6, an example is shown in which the signal processing circuit is mounted on the same semiconductor chip provided with the solid-state imaging device. The same applies to the case where a CMOS image sensor is used as a solid-state imaging device.
[0021]
【The invention's effect】
As described above, in the present invention, the input / output characteristics of each pixel of the solid-state imaging device are measured in advance to calculate and store correction data for the saturation region of the photoelectric conversion device. Since the saturation curve of the photoelectric conversion element is approximated to the ideal curve by correcting the imaging data from the imaging element using the correction data, the saturation region of the charge accumulation amount by the photoelectric conversion element, that is, the knee is obtained. An output signal can be obtained by effectively using a region equal to or larger than the characteristic region.
Therefore, the dynamic range of the solid-state imaging device can be expanded without using a method of combining the normal exposure and the short-time exposure as in the above-described related art.
[Brief description of the drawings]
FIG. 1 is a block diagram illustrating a configuration example of a camera system including a solid-state imaging device and a signal processing device thereof according to an embodiment of the present invention.
FIG. 2 is an explanatory diagram showing an example of input / output characteristics of a certain pixel of a solid-state imaging device used in the camera system shown in FIG.
FIG. 3 is an explanatory diagram showing two characteristics of two pixel signals and a variation in a saturation signal level during actual imaging of the camera system shown in FIG. 1;
FIG. 4 is an explanatory diagram illustrating an example of a use (Knee) characteristic of two pixel signals, a variation in a saturation signal level, and a use characteristic region in a solid-state imaging device.
FIG. 5 is an explanatory diagram showing signal processing in the case where a conventional exposure signal and a short-time exposure signal are added to obtain a high dynamic range image.
FIG. 6 is an explanatory diagram illustrating a configuration example of a solid-state imaging device according to a second embodiment of the present invention;
[Explanation of symbols]
A: light source, 1: solid-state image sensor, 2: CDS / AGC circuit, 3: A / D circuit, 4: signal processing unit, 5: storage device.

Claims (11)

By measuring the input / output characteristics of each pixel of the solid-state imaging device in advance, correction data calculation means for calculating correction data for the saturation region of the photoelectric conversion element,
Storage means for storing the correction data calculated by the correction data calculation means,
At the time of actual imaging, by correcting imaging data from the solid-state imaging device using the correction data, a correction operation unit that approximates a saturation curve of the photoelectric conversion element to an ideal curve,
A signal processing device for a solid-state imaging device, comprising: 2. The solid-state imaging device according to claim 1, wherein the correction data calculation unit calculates the correction data based on a slope of a portion where the input / output characteristics of the photoelectric conversion element has a linear characteristic and a saturation signal level. Element signal processing device. 3. The correction calculation unit according to claim 2, wherein the input / output characteristics of the photoelectric conversion element approximate a saturation curve of the photoelectric conversion element to an ideal curve using a slope line of a portion having a linear characteristic. Signal processing device for solid-state imaging device. 2. The signal processing device for a solid-state imaging device according to claim 1, wherein the correction data calculation unit calculates the correction data for every pixel of the solid-state imaging device. 2. The signal processing device for a solid-state imaging device according to claim 1, wherein the correction data calculation unit calculates the correction data for each of a plurality of pixels of the solid-state imaging device. By measuring the input / output characteristics of each pixel of the solid-state imaging device in advance, a correction data calculation step of calculating correction data for the saturation region of the photoelectric conversion element,
A storage step of storing the correction data calculated in the correction data calculation step in storage means,
At the time of actual imaging, by correcting imaging data from the solid-state imaging device using the correction data, a correction operation step of approximating a saturation curve of the photoelectric conversion element to an ideal curve,
A signal processing method for a solid-state imaging device, comprising: 7. The solid-state imaging device according to claim 6, wherein in the correction data calculating step, the correction data is calculated based on a slope and a saturation signal level of a portion where the input / output characteristics of the photoelectric conversion element have linear characteristics. Element signal processing method. 8. The correction operation step according to claim 7, wherein the saturation curve of the photoelectric conversion element is approximated to an ideal curve by using a slope line of a portion where the input / output characteristic of the photoelectric conversion element has a linear characteristic. A signal processing method for a solid-state imaging device. 7. The signal processing method for a solid-state imaging device according to claim 6, wherein in the correction data calculation step, the correction data is calculated for every pixel of the solid-state imaging device. 7. The signal processing method for a solid-state imaging device according to claim 6, wherein in the correction data calculating step, the correction data is calculated for each of a plurality of pixels of the solid-state imaging device. A solid-state imaging device that images a subject, and a signal processing circuit that processes a signal output from the solid-state imaging device,
The signal processing circuit,
By measuring the input / output characteristics of each pixel of the solid-state imaging device in advance, correction data calculation means for calculating correction data for the saturation region of the photoelectric conversion element,
Storage means for storing the correction data calculated by the correction data calculation means,
At the time of actual imaging, by correcting imaging data from the solid-state imaging device using the correction data, a correction operation unit that approximates a saturation curve of the photoelectric conversion element to an ideal curve,
A solid-state imaging device comprising:

Images