JPH0250584A - Dynamic range enlarging system - Google Patents

Abstract

PURPOSE:To enlarge the dynamic range of an imager by adding and/or averaging the read signals for plural times outputted from the non-destructive read type imager. CONSTITUTION:A non-destructive read type imager 21 is driven by a driver 29, the driver 29 is controlled by a timing control circuit 30, and an exposure start signal and read signals at different timing are supplied to the imager 21 by a driver 29. The photoelectric converting output of the imager 21 is compressed to a logarithmic value by guiding to a logarithm converting circuit 22, its output is guided to a clipping circuit 23, and the output of the clipping circuit 23 is converted into a digital value by guiding to an A/D converter 24. The signal, which is converted into the digital value, is inputted to an addition/averaging circuit 25, the output of the A/D converter 24 and the memory value of a buffer memory 26 are added, and their mean value is calculated. Thus, the dynamic range of the imager can be effectively enlarged.

Description

[Detailed Description of the Invention] [Industrial Application Field] The present invention is a dynamic range expansion system, more specifically, an imaging apparatus using a non-destructive readout type imager, in which the dynamic range of the imager is expanded. Regarding a dynamic range expansion system.

[Prior Art] When the exposure amount is P and the output intensity of the imager is I, the photoelectric conversion characteristics of the imager can generally be expressed by a characteristic line 11 as shown in FIG.

However, in Fig. 6, logarithmic values are taken on the horizontal and vertical axes, and the exposure amount is logP and the output intensity of the imager is #o.
It is set as GL. As is clear from this characteristic line 11,
The output intensity of the imager does not maintain an unlimited proportional relationship with the exposure amount; it becomes a noise level for low exposure amounts, a saturation level for high exposure amounts, and the linear portion 11a in between. Maintains a proportional relationship only for the range of exposure. In other words, the linear portion 11a of this characteristic line 11 corresponds to the dynamic range used for photoelectric conversion for imaging.

Here, normally, the characteristic line 11 has a transient region 11 d between the linear portion 11 a and the saturation level region 11 b and between the linear portion 11 a and the noise level region 11 C, respectively.
11 e exists, but now the transient region 1 of this characteristic line
Further explanation will be made using a characteristic line 12 obtained by a broken line approximation as shown in FIG. 7, ignoring 1d and 1e. In Figure 7, set NogI-y and set the lower limit of the linear part (proportional part) to g
og P L and the upper limit is II) og P n, the following formula holds true for the entire range of exposure. However, C is a constant.

y-C-flOgPL ・・・・・・・・・(1
)(P<PL) y=c-gogP ・・・・・・・・・(2
) (P≦P≦P11) y = C-, l! og Pu...
...(3) (P>PH) Also, the exposure W P (lux-see) is the product of the imager plate surface illuminance E (lux) and the exposure time t (see), so Exposure time (shaft seconds) t
The exposure amount P at k is -E-tk, and the output intensity y of the imager at this time is yk-C-gogPL from (1) to (3) above. (4
) (E<PL/lk) yk-C-gog(E-tk) (5
) CP /l ≦E≦PH/lk) kyk-C-gogPH-...4B) (E>PH/
lk) becomes. However, k-1, 2,...n.

That is, in the range of P/l≦E≦Pn/tk, the output yk of the imager is not distorted and is proportional to the product of the plate surface illuminance E and the exposure time tk only for k.

Therefore, as shown in Figure 8, if we take the logarithm of the imager's plate surface illuminance E, l)ogE, on the horizontal axis, and llogl, the logarithmic value of the imager's output intensity, on the vertical axis, we can calculate the exposure time. For example, in the case of a certain exposure time (relatively slow shutter speed) tl, it will follow the characteristic line 13-■ and a shorter exposure time (relatively fast shutter speed) t. In the case of t2, the characteristic line 13-2
Follow. In other words, these characteristic lines 1B-1 and 13-2 are translated along the illuminance and QogE axes according to the exposure time t, and when the subject is dark, the exposure time t1 is selected and NogCP /l) ~iJog(P, ■
/l ■), and when the subject is bright, the exposure time t is selected and the
Linear photoelectric conversion is performed only in the illuminance range of og (PH/12), for example, gog (P
/t) log(P, I/t2), it was not possible to photoelectrically convert the subject in the illuminance range with one photographing.

[Problem to be Solved by the Invention] As described above, in the conventional exposure control system, the characteristic line 13-1 to the characteristic line 13-2 changes depending on the exposure time.
Although it is possible to translate each characteristic line in parallel, j? og(P
/l ) ~ fIog (P, , /lk) The k illuminance range, that is, the dynamic range could not be expanded. To further explain this using FIG. 9, when performing backlight photography, the difference in illumination between the sun 15 and the person 16 within the same image frame 14 is large;
If the exposure time is adjusted to the illuminance of , the image of person 16 will be blackened,
If the exposure time is matched to the illuminance of the person 16, the image of the sun 15 will be washed out, and with normal exposure control one of the parts has to be sacrificed.

On the other hand, considering the amount of charge accumulated over time in imagers, imagers such as CCDs and MOS type image sensors, which are generally widely used, have the following characteristics as shown in Fig. 10.
Assuming that the plate surface illuminance is constant, the amount of accumulated charge increases in proportion to time, and when a readout pulse αa is given at time t after exposure time t has elapsed, the amount of accumulated charge is read out at this time t. At the same time, the accumulated charge amount of the imager is reset to zero, the first exposure is completed, and at the same time, the second exposure is started. Similarly, during the second exposure, the accumulated charge amount is discharged at time tb when the readout pulse is applied, the accumulated charge amount in the imager is reset to zero, and the exposure ends. Such conventional imagers include 5IT (Static Injection Transistor), CMD (Charge Modification φ Device), AMI (Unblended Moss Intelligent Imager), and CID (Charge Modification Device), which have been developed in recent years. A non-destructive readout type imager such as a fist injection device is called a destructive readout type imager. Regarding non-destructive read-out imagers, see "Optical Technology" Contact Vol. 1.25 No. 11 '87 11J
(Published by Japan Optomechatronics Association, November 20, 1986) PP, 20-2g (652
~860) r-amplification type solid-state imaging probe" (Atsushi Yusa, Fumihiko Ando), but here, the effects related to the present invention will be briefly described.

In a non-destructive readout imager, as shown in FIG.
At this time point t, the level of the accumulated charge amount is read out, but the accumulated charge amount in the imager is not reset, and the accumulation of charges continues as it is, and the accumulated charge amount continues to increase. After this, even if a second read pulse is applied, the level of the accumulated charge is read out in the same way at the time tal when this read pulse is applied, without affecting the amount of accumulated charge in the imager. .

As is clear from a comparison of Fig. 10.11, the second exposure time tβ is from the previous (first) charge amount readout time t to the current charge amount readout time t in the destructive readout type imager. However, in the case of a non-destructive readout type imager, it is the time from the start of charge accumulation to the current charge amount readout time tat, including the previous exposure time t, and after that from time t to time ta1. until a
It is the time that has passed. The same holds true for the third, fourth and subsequent charge readings. That is, in the case of a non-destructive readout type imager, the exposure start point is always the same for multiple readings of the exposure amount (accumulated charge amount), and the timing of charge readout differs depending on each exposure time. In other words, with a non-destructive readout imager, as long as the charge readout is performed in the order of short exposure times (high speed shutter time) to long exposure times (low speed shutter speed), the time required for non-destructive readout imagers is significantly shorter than that of conventional destructive readout imagers. Multiple exposure outputs can be obtained within a single period.

The present invention provides a dynamic range expansion system that uses such a non-destructive read-out imager to photograph a subject with a wide illuminance distribution in a single photographing opportunity.

[Means and effects for solving the problem] The dynamic range expansion system of the present invention uses a first means to vary the timing of a non-destructive read-out imager in response to the same exposure start operation of the imager. The video signals are read out a plurality of times, and the video signals read out a plurality of times are added and/or averaged by a second means provided in operational relation with the first means. By obtaining the video signal output of the imager through the second means, the dynamic range of the imager is effectively expanded.

[Example] FIG. 3 shows the substantial photoelectric conversion characteristics of a non-destructive read-out imager. This characteristic line 20 has an exposure time t of 11 degrees t2, as explained in FIG. 8 above. ta (
tl>t2>ta) and three different characteristic lines 13
-1.13-2.13-3 is the additive average (logarithmic average), and its slope is the characteristic line 13-1.13-2 above.
．． That's 1/3 of 13-3. This is also clear from the formula below.

Now, the values of y of the characteristic lines 13-1.13-2.13-3 are yl, y2. Assuming y3, the formula of characteristic line 20 which is the average of this is y-1/3 (y, +y2+ya) town...(7
).

Here, P o / t k - P L / i k + t
The lower limit of the proportional part of characteristic line 13-2 is defined as characteristic line 13-1.
If we match the upper limit of the proportional part of characteristic line 13-2 with the lower limit of the proportional part of characteristic line 13-3,
The above formula (7) is y = C-j! og P L
・・・・・・■+1/3C(24!ogP+D
og(E・11)) ・・・・・・■+t/3c
lfIogP + fl ogP + D og(
E・+2)l−・・■L H +1/3C12j! ogP + Il og(E・
13)) ・・・・・・■+ C-+7 ogP
H...■ becomes, and the first term ■ to the fifth term ■ of this equation
Each illuminance range is: ■(E≦pL/11) ■(P Il ≦E≦pH/l,) ■ (P Il
≦E≦PH/12)■(PL/13≦E≦P n
/ta)■(E≧P,,/13). However, here, P n / it - P L
/t2.

Since PH/12-PL/13, the equations of the second term (■) and the fourth term (■) above are equal to the equation of the third term (■), and as a result, the illumination L of P Il ≦E≦P u / t s In the degree range Dw, the formulas for the second term ■ to the fourth term ■ are expressed as follows:
It can be summarized in one formula.

y-1/3ClΩog P t, + i) og P
rlflog(E・+2))・・・・・・・・・(8
) In other words, this equation (8) represents the proportional part of the characteristic line 20 in FIG. 3 that shows the substantial photoelectric conversion characteristics by the non-destructive readout type imager. This means that in a photographing opportunity of - degrees, the photoelectric conversion output of the subject is different from the exposure time (shaft seconds) tt, +2. When ta is read out three times, the three read out imager outputs are averaged, and finally, in the illuminance range DIW of P Il ≦E≦PH/13, 1 is obtained according to the proportional part of the characteristic line 20. This means that two photoelectric conversion outputs can be obtained, and in this case, the dynamic range Dw on the input side of the imager is substantially expanded three times.

Furthermore, to explain this in relation to the case of backlight photography shown in FIG. 9, in FIG. 3, the illuminance of the sun 15 is f
If the illuminance of iogE and the person 16 is gOgEl,l, the difference in illuminance between the two is large, so as mentioned above, 1
With the imager output using one exposure time, it is not possible to obtain the images of the sun 15 and the person 16 as two images without being crushed, but three imager outputs read out with different exposure times are averaged. Then, as is clear from FIG. 3, the image of the sun 15 is not completely whitened out like the imager output with only exposure time t1 or +2, but the imager output with only exposure time t3. The image of Big Game 16 is larger (brighter) than
Unlike the imager output with only exposure time t or +3, the imager output does not turn completely black, but is smaller (darker) than the imager output with only exposure time t1. Therefore, the dynamic range Dw at the input end of the imager is expanded. Note that since the imager output range is constant when viewed from the output side of the imager, it can also be regarded as compression of the dynamic range.

Next, the configuration and operation of an embodiment of the dynamic range expansion system of the present invention will be explained with reference to the block diagram shown in FIG. 1 and the time chart shown in FIG. 2.

In FIG. 1, the imager 21 is a non-destructive readout type imager such as SIT. This imager 21 is driven by a driver 29, and the driver 29 is controlled by a timing control circuit 30. That is, in response to the timing signal from the timing control circuit 30, the driver 29 provides the imager 21 with an exposure start signal and a readout signal at different timings in order to obtain different exposure times during one shooting as described above. . First, when a timing signal for an exposure start command is issued from the timing control circuit 30, a reset pulse shown in FIG. 2 is sent from the driver 29 to the imager 21, which causes the imager 21 to discharge unnecessary charges and reset After that, charge is accumulated according to the amount of exposure. Further, since the timing signal of the exposure start command is sent from the timing control circuit 30 to the photometry circuit 31, the photometry integral output of the photometry circuit 31 changes as shown in FIG. 2 at the same time as the imager starts exposure. I will do it.

The photoelectric conversion output of the imager 21 is led to a logarithmic conversion circuit 22 and compressed into a logarithmic value. The logarithmically compressed output of the imager 21 is led to a clipping circuit 23, where it is sent to a transient region near the saturation level and near the noise level (lld shown in FIG. 6).

11e) is clipped. That is, by passing through the clip circuit 23, the photoelectric conversion output characteristics of the imager 21 change from those shown in FIG. 6 to those shown in FIG. The effects of using this clip circuit 23 will be described later. Clip circuit 23
The output is led to an A/D converter 24 and converted into a digital value. This signal converted into a digital value is input to the averaging circuit 25.

Here, as mentioned above, when photographing at -degrees, three types of exposure times t1°t2. ts
Let us consider the case where the outputs of the imager 21 are respectively read out at the time when (tl>t2>ts) has elapsed. Δ of the AL optical circuit 31 that measures the exposure amount of the imager 21
-1 When the light integrated output reaches level g3, the imager 21
A first read pulse is applied to the imager 21, and the video signal of the exposure time t3 is first read out from the imager 21, and its digital value x1 is input to the averaging circuit 25. The averaging circuit 25 adds the output of the A/D converter 24 and the value stored in the buffer memory 26 and calculates the average value, but in this case, since nothing is stored in the buffer memory 26, The signal from the A/D converter 24 is sent as is to the memory 27. Both this memory 27 and the buffer memory 26 have a storage capacity for storing only one image. The memory 27 is controlled by the timing signal of the timing control circuit 30 and stores the signal value X sent from the averaging circuit 25, and also sends this stored value x1 to the buffer memory 26. After this, the photometric integral output becomes level ρ2,
When the second read pulse is applied to the imager 21 and the value x2 of the video signal at the exposure time t2 read from the imager 21 is input to the averaging circuit 25, the signal value stored in the buffer memory 26 is Xt is sent to the averaging circuit 25 in response to a timing signal from the timing control circuit 30, and is averaged to the value x2 of the video signal. The value of this averaged signal, that is, the signal value of X2 (x l + X 2 ) / 2 71"
is sent to the memory 27 and stored after clearing the stored value. This stored value entry in the memory 27 is again sent to the buffer memory 26, where the previous stored value is cleared and a stored value 77 is recorded. Then, when the photometric integral output reaches level 111 and the third read pulse is applied and the value x3 of the video signal at exposure time t3 read from the imager 21 is input to the averaging circuit 25,
Since this value x3 and the value stored in the buffer memory 26 are averaged by the averaging circuit 25, the average value becomes -(2+xa)/3. is sent to the memory 27 and stored after clearing the stored value.

Thereafter, when a command signal for ending exposure is issued from the timing control circuit 30, the final addition average value Xa stored in the memory 27 is converted into an analog value by the D/A converter 28 and output. The average added voltage value 71 outputted from this D/A converter 28 is along the characteristic line 20 shown in FIG. 3, and the dynamic range Dw
is expanding.

Here, regarding the clip circuit 23, as mentioned above, the output characteristics of the imager 21 are actually
There are transient regions lid and lie near the upper limit saturation level and lower limit noise level of 1a (see FIG. 6), but by using the clip circuit 23, the output characteristics of the imager 21 can be made transient. Therefore, each characteristic line 13-1°13 of the exposure time tt, tz, and ta shown in FIG. 3 does not exist (see FIG. 7).
-2.13-3 also immediately follows a straight line parallel to the axis of 17ogE above and below the proportional part. Furthermore, clipping the top and bottom of the output of the imager 21 including the transient region seems to narrow the viewing range; Since the above characteristic line 13-1.13-2゜13-3 is cut off by What is obtained is shown by the characteristic line 20 in FIG. In the embodiment shown in FIG. 1, the clipping circuit 23 is replaced by the logarithmic conversion circuit 22.
and the A/D converter 24; however, it may be placed before the logarithmic conversion circuit 22 as long as it is between the imager 21 and the averaging circuit 25. It is also possible to arrange it at the subsequent stage and clip it with a digital value.

Furthermore, the A/D conversion characteristic of the A/D converter 24 may be provided with a clipping function.

Note that in the present invention, the number of times the charge is read out from the imager 21 within one photographing time is not limited to three as in the above embodiment, so if expressed in a general formula, the imager output is read out at different exposure times. The average value data when read n times is x - ((n-1)x, -, +x,)/n ---
−<9).

As can be seen from this equation (9), the buffer memory 26
The memory 27 is configured to store a numerical value k in addition to the arithmetic average value, and when the next (k+1)th signal value xk+1 is input to the arithmetic average circuit 25, at this time, The above average value is multiplied by the value k (that is, the sum of all signals up to the kth time) is (k+1
) signal value xk+1 is added, and the above value k
By dividing by the value (k+1) which is obtained by adding 1 to , the additive average value 〒 is obtained. The reason why such an averaging method is adopted is that the buffer memory 26
This is because the memory 27 has a storage capacity only for each image, and thus the memories 26 and 27 can be made inexpensive. Of course, if a memory with a large storage capacity is used, it is possible to adopt a normal averaging method in which n signal values are sequentially added, and after all signal values have been added, the number is divided by the numerical value n.

Incidentally, in the above embodiment, a case has been described in which the dynamic range is expanded three times on the logarithmic axis as shown by the characteristic line 20 in FIG. 3, but various other examples can be considered. Of course, the exposure times used for -degree photography opportunities may be reduced to two types, or increased to 4FIi or higher, but as in the above example, three types of exposure times were used. Explain other properties of the case.

The characteristic line 20 represents an expanded dynamic range DW.
γ (the slope of the straight line) is uniformly 1/3 in the range of This increases the range but reduces the contrast.

For example, the characteristic line 40 shown in FIG. 4 has different exposure times tt, t2 . Each characteristic line of ta 13-1゜13-2. 1
3-3 does not deviate much, these characteristic lines are averaged, and the inflection point of this characteristic line 31 is from the lower limit of characteristic line 13-1 to the lower limit of characteristic line 13-2 from the side of low illuminance logH. lower limit of characteristic line 13-3, characteristic line 13
-1 upper limit, characteristic line 13-2 upper limit, and characteristic line 13-
3, and the dynamic range Dw is 5/
It has been expanded three times. Therefore, although the effect of expanding the dynamic range Dw of this characteristic line 40 is small, the γ characteristic matches the single characteristic line 13-2 so that the contrast is high in the middle range of brightness of many main subjects. Contrast is kept low in brighter and darker illuminance areas, which is a practical characteristic.

It is also possible to obtain a characteristic line 50 as shown in FIG. This characteristic line 50 shows different exposure times t t
, t 2 , ta characteristic lines 13-1° 13-2.
In the case where 13-1 is shifted to an intermediate value between the amount of shift in the case of FIG. 3 and the amount of shift in the case of FIG. 4,
These characteristic lines are averaged, and the inflection point of this characteristic line 50 is from the side with low illuminance NogE to the characteristic line 13-
1, the lower limit of characteristic line 13-2, the upper limit of characteristic line 13-1, the lower limit of characteristic line 13-3, the upper limit of characteristic line 13-2, and the upper limit of characteristic line 13-3. Dw is expanded 7/3 times.

Also, when averaging the output of the imager 21,
Weighting may be performed using an additive method. for example,
In FIG. 3, three different exposure times tt, t2. t
Each characteristic line 13-1゜13-2.13-3 of a is 1:2:
When weighting is added at a ratio of 1, a characteristic line 60 as shown by the two-dot chain line in the same figure is obtained. Being ignored. In this case, as in the case of characteristic line 20, the dynamic range Dw is expanded three times, and the γ characteristic is brought closer to characteristic line 13-2 in order to emphasize contrast in the intermediate brightness range of most of the main subjects. Accordingly, γ is made smaller (contrast is kept low) in brighter and darker illuminance areas. Note that weighting changes the addition ratio by, for example, 2
It goes without saying that if the ratio is set to 1:1, a characteristic line with emphasis on contrast in dark areas can be obtained, and with a ratio of 1:1:2, a characteristic line with emphasis on contrast in bright areas can be obtained.

In the above embodiments, the memory 27 is connected to the A/D converter 24. Although a digital memory is used in combination with the D/A converter 28, it is of course possible to obtain the same effect by using an analog memory. Furthermore, it goes without saying that various modifications are possible, such as integrating the memory section and the imager into one chip as a more effective mode.

[Effects of the Invention] As described above, according to the present invention, the dynamic range of the imager can be expanded by adding and/or averaging multiple readout signals output from the nondestructive readout type imager. Can be done.

[Brief explanation of the drawing]

FIG. 1 is a block diagram of a dynamic range expansion system showing an embodiment of the present invention. FIG. 2 is a time chart showing the imager signal readout shown in FIG. 1 above. FIG. Imager's photoelectric conversion characteristic diagram for explaining the operating principle of dynamic range expansion in the embodiment shown in FIG. 1, and FIG. Fig. 6 is a photoelectric conversion characteristic diagram of Imager for explaining the dynamic range; Fig. 7 is a photoelectric conversion characteristic diagram of Imager used in the dynamic range expansion system of the present invention. An ideal imager's photoelectric conversion characteristic diagram. Figure 8 is an imager's photoelectric conversion characteristic diagram showing two characteristic lines when exposure times are different. Figure 9 is an image of a subject during backlight photography. FIG. 10 is a diagram showing the amount of accumulated charge versus time in the destructive readout type imager, and FIG. 11 is a diagram showing the amount of accumulated charge versus time in the nondestructive readout type imager used in the present invention. It is a line diagram. 21...Non-destructive readout type imager (first means) 23... Clip circuit (clip means) 25.
...Additional averaging circuit (second means) 29...
Driver (first means) 30... Timing control circuit (first means)

Claims (2)

[Claims] (1) A first means for reading out video signals a plurality of times at different timings corresponding to the same exposure start operation of the imager in a non-destructive readout type imager; a second means provided in association for adding and/or averaging the video signals read out multiple times; and obtaining the video signal output of the imager through the second means. A dynamic range expansion system characterized by effectively expanding the dynamic range of the imager. (2) The dynamic range expansion system according to claim (1), characterized in that a clip means is inserted in the video signal transmission path from the first means to the second means.