JP2830482B2 - High dynamic range imaging device - Google Patents

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a high dynamic range image pickup apparatus, and more particularly to an apparatus capable of accurately picking up a still image or a quasi-still image having a large luminance difference over an extremely wide dynamic range.

[0002]

2. Description of the Related Art FIG. 5 shows a schematic configuration of an image pickup apparatus including a conventional image sensor. In the device shown in FIG. 1, after the output signals from the pixels constituting the image sensor 1, that is, the light receiving elements, are sequentially read out and amplified through the amplifier circuit 2 in accordance with the image light guided to the image sensor 1. , A / D converter 3. This input signal is A /
The data is converted into digital data in the D converter 3 and stored in the memory 4.
Is stored. The control circuit 5 includes, for example, the image sensor 1
Sequentially read signals from the respective light receiving elements of the A / D converter 3
Are sequentially written to each address of the memory 4. As described above, for example, the image data stored in the memory 4 is read out by a computer (not shown) and subjected to image processing, and then displayed on a display device.

[0003]

However, in such a conventional image pickup apparatus, an image signal obtained by one exposure is applied to all light receiving elements substantially simultaneously.
Since reading is performed only once, there is a disadvantage that the dynamic range of the imaging device cannot be widened.

[0004] This point will be described in detail with reference to FIG. 6, which shows the relationship between the output level of a light receiving element of a general image sensor and time. That is, in FIG. 6, the light receiving elements A, B, C,
The relationship between the output level of D and time is shown. As shown in the figure, when the image sensor is read at time T, the outputs V _{B, 1} of the pixels _{VA, 1} and B are normally read, but the outputs V _{C, 1} of the pixel _C and the pixel D are read normally. the output V _{D of 1} can not obtain the correct output level because it is less noise level V _0. Also, when is read at time 2T, but the output V _{B, 2} of pixels B are read successfully, the output V _{A, 1} of the pixel A is not obtained saturation level V ₂ becomes correct output level. As for the pixels C and D, an accurate output level cannot be obtained because they are lower than the noise level as in the previous case.

[0005] In contrast, when the reads at time NT sufficiently long storage time, the output V _C of the pixel _C, the output V _{D N} and pixel _D, but _N can be read, the pixel A Output V
_A, the output V _N and pixel V _{B, N} can not be read correct output level value is saturated in the middle.

Therefore, in the readout method of the conventional imaging apparatus, the intensity of light, that is, the bright direction is limited by the saturation level, and the output of the weak light, that is, the direction of dark is limited by the noise level. The range is limited by the dynamic range of the light receiving element of the image sensor used, and has a disadvantage that it is at most several thousand times.

An object of the present invention is to provide an imaging apparatus capable of obtaining an extremely wide dynamic range without being limited by the dynamic range of a light receiving element of an image sensor to be used, in view of the above-mentioned problems of the conventional apparatus. Is to provide.

[0008]

In order to achieve the above object, the present invention provides a non-destructive image sensor, read control means for repeating a read operation a plurality of times during exposure of the image sensor, and Level comparing means for sequentially comparing the read output level from each light receiving element of the image sensor with a predetermined reference level, and corresponding to the read output level when the read output level of the light receiving element first exceeds the reference level Data memory means for storing a value to be read, and number memory means for storing a value corresponding to the number of repetitions of the read operation up to that time when the read output level of the light receiving element exceeds the reference level for the first time. ,
An image output is obtained by extrapolating a value corresponding to the output level stored in the data memory means based on a value corresponding to the number of repetitions of the read operation stored in the number-of-times memory means and a predetermined number of times of reading. And a calculating means.

[0009]

In the above arrangement, the reading control means performs the reading operation of each light receiving element a predetermined number of times during the exposure of the image sensor. The level comparing means compares the read output level of each light receiving element with a reference level, and stores the value corresponding to the read output level in the data memory means when the read output level exceeds the reference level for the first time. And a value corresponding to the number of repetitions of the read operation up to that time is stored in the number memory means. When the reading of the predetermined number of times is completed, the calculating means extrapolates the read output level stored in the data memory means by the number of repetitions stored in the number memory means and the predetermined number of readings. The image output of each light receiving element is obtained.

In such an operation, it is possible to obtain an output higher than the noise level by setting a sufficiently long accumulation time for a signal having a very low luminance. In addition, for a pixel in which the luminance of incident light is high and is saturated during accumulation, a value corresponding to a true output level can be obtained by extrapolation using data before saturation stored in the data memory. . Therefore, the dynamic range can be greatly expanded without being restricted by the characteristics of the light receiving element.

[0011]

Embodiments of the present invention will be described below with reference to the drawings. FIG. 1 shows a schematic configuration of a high dynamic range imaging apparatus according to one embodiment of the present invention. The device shown in the figure includes an image sensor 11, an amplifier circuit 12, an A / D converter 13,
It comprises a data memory 14, a count memory 15, a counter 16, a comparison circuit 17, a multiplication / division circuit 18, a control circuit 19 and the like.

The image sensor 11 is a non-destructive type image sensor having many light receiving elements.
The amplification circuit 12 is for buffering and amplifying the output of the image sensor 11. The A / D converter 13 sequentially converts the read output of the image sensor 11 amplified by the amplifier circuit 12 into a digital value. The data memory 14 stores a value corresponding to the digitized read output level given from the A / D converter 13 for each pixel based on the output of the comparison circuit 17. The number-of-times memory 15 is controlled based on the output of the comparison circuit 17, and stores, for each pixel, the number of times of reading until the read output level of the light receiving element exceeds a predetermined value. The counter 16 counts the number of times of reading. The comparison circuit 17 is a circuit for comparing the digitized read output level value of the light receiving element with a predetermined value. The multiplication / division circuit 18 is an operation circuit that performs multiplication / division for calculating a final output. The control circuit 19 supplies a clock pulse, a control signal, and the like to the above-described circuit blocks to control their operations.

The operation of the above-described imaging apparatus will be described with reference to the flowchart of FIG. Prior to reading from the image sensor 11, a reset signal is sent from the control circuit 19 to reset the counter 16 to zero, and the count memory 15 stores, for example, the maximum number of repetitions N in an area corresponding to all light receiving elements. .

Next, an increment signal is sent from the control circuit 19, and the counter 16 is incremented by one. That is,
The counter 16 is set to 1 indicating the first reading. In this state, the output of each pixel of the image sensor 11 is sequentially read, and the read signal is amplified by the amplifier circuit 12 and then converted to a digital value by the A / D converter 13.

The output digital value of the A / D converter 13 is V
_j, expressed as _k, compares the digital values _{V j,} the reference value _{V 1} in the comparison circuit 17 _k. Here, j indicates the pixel number of the image sensor 11, and k indicates the number of times of reading. That is, V _{j, k} is the k-th output of the j-th pixel. As a result of comparison by the comparison circuit 12, the output V
_{j, if k} exceeds the reference value V ₁ as the read output V _{A, 1} 1 th pixel A shown is in FIG. 6 for example,
The number of times of reading k, in this case 1, is stored in an area of the number of times memory corresponding to the read light receiving element. That is, N _j =
k. Here, _Nj is data of the number memory of the j-th pixel. Further, the read output value _{VA, 1} is stored in an area of the data memory 14 corresponding to the pixel A.
That is, M _j = V _{j, k} is set. Here, _Mj is data in the data memory 14 of the j-th pixel.

[0016] In contrast, the pixel B in FIG. 6, C, each pixel B for which the output of the first does not exceed the reference value _{V 1} as D, C, the output value of the D _{V B,} 1, V _{C, 1} , V
_{D and 1} are stored in corresponding areas of the data memory 14, respectively. In this case, no writing is performed in the corresponding area of the count memory 15, and the maximum read count N is left inserted. As described above, all the pixels are read, and when the reading is completed, 1 is added to the counter 16 and the counted value is set to 2, and the second reading is performed.

In the second reading, reading of the output of the light receiving element of each pixel of the image sensor 11 and reading of the count memory 15 are sequentially performed. Then, for example, the output V _{A, 2} of the pixel A in FIG. 6 but above the reference value V _1, the value N _A of the corresponding region of the time memory 15 is set to "1" at the first reading Therefore, the data is not stored in the data memory 14 and the count memory 15 because it is not N.

On the other hand, the readout output VB _{, 2 of the} pixel B
Exceeds the reference value V ₁ and the area of the number-of-times memory 15 corresponding to the pixel B is N.
= 2 is stored in the corresponding area of the count memory 15. That is, the storage data N _B in the area corresponding to the pixel B of the time memory 15 is "2".

[0019] In contrast, the pixel C in FIG. 6, the read output value of the D _V C, _{2, V D,} since ₂ does not exceed the reference value _{V 1,} these output values _{V C,} 2, _{V D, 2} is stored in the corresponding area of the data memory 14. However, the value stored in the count memory 15 is not changed.

The above operation is repeated N times while sequentially incrementing the value of the counter 16. As a result, finally, the pixels A, B, C, and D in FIG.
, V _{A, 1} , V _{B, 2} , V
_{C, N} , VD _{, N} are stored, and 1, 2, N, N are stored in the count memory 15, respectively.

Next, the output of each pixel is calculated by the multiplication / division circuit 18. That is, assuming that the output of the pixel j is V _j , it can be calculated by V _j = (N / N _j ) · M _j . This is what determined during N-th reads N _j-th read output M _j stored in the data memory extrapolation to 3, i.e. after the exposure time NT, the output V _j .

[0022] In the example of FIG. 6, the final output _{V A} of the pixel A
The _{V A, 1} · N becomes, the final output _{V B} of the pixel B V
_{B, 2} · N / 2, and the final output VC of the pixel _C is VC _{, N}
Next, and final output _{V D} of the pixel D becomes _{V D, N.}

According to the above procedure, the output of the pixel saturated during the accumulation can be obtained by extrapolation. Therefore, the original dynamic range of the image sensor can be greatly expanded, that is, N times. For example, N = 1
Assuming that the dynamic range is 000, the dynamic range that is several thousand times the original value of the image sensor is further expanded by a factor of 1,000 to obtain a dynamic range that is several million times.

[0023] Note that the reference value V ₁ of the purpose of comparison in the comparison circuit 17 is a convenient to set the order of about half for example the saturated output level V _2, which is a reference value V ₁ is the output level This is because, if it exceeds the threshold, a level that is expected to exceed the saturation point in the next reading is set as the reference level. Of course, it is also possible to use a reference level other than approximately half the saturation level V _2.

FIG. 4 shows a schematic configuration of a high dynamic range imaging apparatus according to another embodiment of the present invention. In the apparatus shown in the figure, the output of the image sensor 11 is input to a data processing device such as a microcomputer 20 via an amplifier circuit 12 and an A / D converter 13. The microcomputer 20 performs the processing operation shown in FIG. 2 by a built-in program.

The apparatus shown in FIG. 4 is advantageous when the total accumulation time of the image sensor 11 is long and the reading speed in one exposure may be low, for example, for astronomical observation. The circuit configuration can be simplified in comparison.

[0026]

As described above, according to the present invention, an output higher than the noise level can be obtained by increasing the accumulation time for a weak signal, and a light receiving element for a strong signal. Since the final output can be obtained by extrapolation based on the value in a short accumulation time before the output of the image sensor is saturated, the original dynamic range of the image sensor can be greatly expanded, and the subject having a large luminance difference can be obtained. Can be accurately imaged.

[Brief description of the drawings]

FIG. 1 is a block diagram showing a schematic configuration of a high dynamic range imaging apparatus according to one embodiment of the present invention.

FIG. 2 is a flowchart for explaining the operation of the imaging apparatus of FIG. 1;

FIG. 3 is an explanatory diagram showing a method of calculating a final output performed by a multiplication / division circuit in the apparatus of FIG. 1;

FIG. 4 is a block diagram showing a schematic configuration of a high dynamic range imaging apparatus according to another embodiment of the present invention.

FIG. 5 is a block diagram illustrating a schematic configuration of a conventional imaging device.

FIG. 6 is a graph showing a relationship between an exposure time of a light receiving element of an image sensor and an output level.

[Explanation of symbols]

 DESCRIPTION OF SYMBOLS 11 Image sensor 12 Amplification circuit 13 A / D converter 14 Data memory 15 Number memory 16 Counter 17 Comparison circuit 18 Multiplication / division circuit 19 Control circuit 20 Microcomputer

Claims (3)

(57) [Claims] A non-destructive image sensor having a plurality of light-receiving elements; read-out control means for repeating a predetermined number of read-out operations during exposure of the image sensor; Level comparing means for sequentially comparing the read output level with a predetermined reference level; and data for storing a value corresponding to the read output level when the read output level of the light receiving element exceeds the reference level for the first time. Memory means, a number memory means for storing a value corresponding to the number of repetitions of the read operation up to that time when the read output level of the light receiving element exceeds the reference level for the first time, and stored in the data memory means. The value corresponding to the output level and the value corresponding to the number of repetitions of the read operation stored in the number-of-times memory means. And a calculating means for obtaining an image output by performing an extrapolation calculation based on the predetermined number of times of reading. 2. The high dynamic range imaging device according to claim 1, wherein the predetermined reference level is a value equal to or more than 倍 of a saturation level of the light receiving element. 3. The output level of the light receiving element exceeds the reference level for the first time by comparing the number of repetitions of the read operation during each read operation with a value corresponding to the number of repetitions stored in the number memory means. The high dynamic range imaging device according to claim 1, wherein it is determined whether or not the image has been captured.