JPS63306779A - Image pickup device - Google Patents

Abstract

PURPOSE:To obtain an image pickup device whose dynamic range is virtually wide and by which a moving image can be obtained by consecutively outputting images different in exposure amount under the control of a control means, and providing a synthesis means that consecutively synthesizes the images to obtain a moving image. CONSTITUTION:The gradation characteristic, in case of an ordinary video camera, is linear for input/output up to 100%, but for 100-400%, it comes to be KNEE characteristic. In one field where the time resolution is about the same as that of a field accumulation CCD image pickup element, two sheets of pictures are fetched in. Signals are read out from the image pickup element 103 faster than normal video rate, and they are converted on time-base to return to the normal rate. Field memories 90, 91 respectively have a memory capacity equal to one field length, and in the memory 90, a 1/1000sec accumulating signal is delayed to come synchronous with 1/120sec read. In the memory 91, the signals are elongated on time-base two times in order to modify video signals in the unit of 1/120sec to NTSC signals in the unit of 1/60sec.

Description

DETAILED DESCRIPTION OF THE INVENTION [Field of Industrial Application] The present invention relates to an imaging device having a substantially wide dynamic range.

[Conventional technology]

Imaging devices are widely used as video camera units such as camera-integrated VTRs and still video cameras. Video cameras that use image pickup tubes or solid-state image sensors have a narrower dynamic range than traditional silver-halide photographic systems, and as a result, when backlit, etc., whites and darks are crushed (a common term for areas with extremely high or low brightness levels). ) etc. occur.

With conventional video cameras, in such cases, the aperture can be opened by about two stops either manually or by operating the backlight compensation button.
The amount of light was being adjusted.

However, even when such backlight compensation is performed appropriately,
Even if the main subject has the proper exposure, overexposure will occur in the background, resulting in a screen with only a white background.
In other words, the narrow dynamic range of the imaging device cannot be solved by simply adjusting the light amount so that the exposure of the main subject is appropriate, as is the case with conventional devices. In conventional imaging devices that convert into electrical signals, a configuration has been considered in which a single screen is synthesized from a plurality of screens with different exposure amounts obtained from the same subject.

[Problem that the invention seeks to solve]

However, this conventional imaging device targets still images, and cannot obtain moving images with a wide dynamic range.

In view of these problems, an object of the present invention is to provide an imaging device that has a wide substantial dynamic range and can obtain moving images.

[Means for solving problems]

An imaging device according to the present invention includes an imaging means, a control means for successively outputting images with different exposure amounts from the imaging means, and a control means for successively outputting images with different exposure amounts among the images output from the imaging means. and compositing means for compositing and obtaining a moving image.

[Effect]

The control means allows a plurality of images with different exposure amounts to be obtained in succession, so the composition by the composition means can be performed by video or video.
By performing the process at a speed that follows the rate, a synthesized moving image with an appropriate exposure amount can be obtained. The synthesis speed of the synthesis means is sufficiently fast (it can be done), so there is no problem.

〔Example〕

Hereinafter, one embodiment of the present invention will be described with reference to the drawings. FIG. 1 shows a block diagram of the overall configuration when the present invention is applied to a camera-integrated VTR.

In FIG. 1, 100 is a camera section, 200 is a processing section,
300 is a recording section. In the camera unit 100, the light beam entering from the optical system 101 is limited in light amount by the aperture 102, and is imaged on the image sensor 103. The image sensor 103 is composed of an image sensor, a semiconductor image sensor such as a MOS, or a COD. The focus drive circuit 107, the aperture drive circuit 106, and the image sensor drive circuit 105 respectively control the optical system 101, the aperture 102, and the image sensor 1 under the control of the camera control circuit 108.
Drive 03. The camera signal processing circuit 104 is a well-known circuit that performs T-correction and other processing similar to the signal processing circuit of a normal video camera.

The video signal output from the camera section 100 is processed by the processing section 20.
0 is converted into a digital signal by the A/D converter 201,
The arithmetic circuit 202 performs pixel data conversion, which will be described later.
The D/A converter 203 converts the signal back to an analog signal, and the recording unit 3
00.

204 is an image memory for calculation in the calculation circuit 202, and 205 is its addressing circuit. The addressing circuit 205 outputs write and read address control signals for the image memory 204 in response to timing signals from the control circuit 108 of the camera unit 100.

In the recording section 300, the analog signal from the D/A converter 203 is recorded on the VTR recorder 3.1 using a known method.

Next, the operation of the image sensor 103 will be explained. FIG. 2 is a more detailed block diagram of the camera section 100, and FIG. 3 is a timing chart of the camera section 100, taking the NTSC signal as an example. The field index (Fl) signal is: The *VILK signal, which is a signal for distinguishing between the odd (ODD) field and the even (f! VEN) field that constitute one frame, is a vertical blanking signal, and the H (high) period is the valid screen, and the L ( The low) part corresponds to the vertical planking period. T□0. is a signal for controlling the charge accumulation time of the image sensor 103; for example, in the case of a CCD image sensor, the pixel output is sent to C for vertical transfer.
This is a pulse for reading out to OD. The iris gate signal is a signal that specifies whether to use a 1/1000 second accumulation signal or a 1760 second accumulation signal as a reference video signal for automatic exposure, which will be described later.

In the illustrated example, 171,000 during the vertical blanking period.
1/1000 of that in the next valid screen period.
Outputs second accumulation signal. Immediately after the 1/1000 second accumulation period, charge is accumulated for substantially 1760 seconds, and the 1760 second accumulation signal is output during the effective screen period of the next field. In this way, for each field, two types (1/
1000 seconds and 1760 seconds) are output alternately.

In addition, in FIG. 2, 20 is the camera signal processing circuit 10.
A known AEwi control circuit 22 receives a signal (for example, a video signal) from 4 and calculates a control signal for exposure control.
is a known AFil that outputs a control signal for focus control.
The li circuit 24 is a 1/2 frequency dividing circuit that divides the frequency of the vertical blanking signal V, □ by two. 26 and 27 are sample and hold circuits, 28 are inverters, and 29 and 30 are switches for selecting whether to use the output of the 1/2 frequency divider circuit 24 or its inverted signal from the inverter 28 to determine the sampling timing. Sample and hold circuit 26
．． The outputs of 27 are applied to an aperture drive circuit 106 and a focus drive circuit 107, respectively, to perform automatic exposure control and automatic focus adjustment.

In the above example, it is a combination of 1/1000 seconds and 1760 seconds, and the light amount changes by about 4 steps (24 times).
For example, in the case of a camera using a CCD image sensor, EVEN
If you adjust the exposure to the main subject based on the accumulation time of 1760 seconds in the field, in that BVEN field, overexposure tends to occur in the evening, but with OD that reduces the light intensity by 4 steps.
In the D field, blackout often occurs on the main subject. Note that this example assumes that the exposure is adjusted to the background side during backlight correction, and of course, it may be set to a value other than 1/1000 seconds depending on the situation.

In the present invention, such overexposure and/or underexposure in one field is actively utilized to improve the screen. In other words, areas where blown-out highlights or blown-out shadows occur are replaced with corresponding areas from other fields (no blown-out highlights or blown-out highlights occur because the exposure is different), and the signals from both fields are combined to create the final image. Signal. The basic idea will be explained with reference to FIG. In FIG. 4, the main subject is schematically shown as a vertically long rectangle.

In FIG. 4, the through (T) image refers to the direct output of the image sensor 103, and the memory (M) image or memory output refers to the signal of the previous field once stored in the image memory 204.
In the through-the-lens image, the backlit main subject is blown out in shadows in each ODD field, and the background is blown out in highlights in each EVEN field. Furthermore, since the memory image consists of a signal delayed by one field period, blown-out highlights and blown-out shadows occur in a different field from the through-the-lens image.

Therefore, by appropriately combining the through-the-lens image and the memory image, a good image without blown-out highlights or blown-out shadows can be obtained. That is, for each field, the signals of the live view image and the memory image are compared with a predetermined threshold value, and if the signal is larger than the threshold value, it is set as 1, and if it is smaller than the threshold value, it is set as 0, and blown-out highlights or blown-out shadows are determined for each pixel.

FIG. 6 shows the relationship between the threshold value, the brightness value of the pixel, and the field. In Figure 6(a), the horizontal axis is the brightness level, and the vertical axis is 1.
Shows the frequency of appearance of each brightness level in the 100 pages.

As shown in FIG. 6(a), the threshold Thl is set so as to be able to determine blown-up shadows, and the threshold Th2 is set so as to be able to determine blown-out highlights. In other words, an image below Thl is determined to be blown-up shadows, and an image above the threshold Th2 is determined to be blown-out highlights. FIG. 6(b) shows the relationship between each field and the darkness value. As described above, since overexposure and underexposure occur alternately in the ODD field and the EVEN field, the threshold for determining this is also changed for each field.

In this way, it is possible to determine which pixel part of which field has a blocked-up shadow or a blown-out highlight, and by using the judgment result, it is possible to select a pixel signal with an appropriate exposure amount for the through image and the memory image. Take the AND of A and judgment B, and in the ODD field, select the through image signal for pixels where the AND is 1, and select the memory image signal for pixels where the AND is 0. S-I! Vl! By creating the opposite relationship in the N field, a selection flag as shown in FIG. 4 can be obtained. The picture at the bottom of FIG. 4 shows a composite image based on the selection flag. In this figure, we assumed that the main subject was moving at a constant speed, and confirmed the effect of time axis shift on the image, but it can be seen that the resulting video is sufficient for practical use.

FIG. 5 shows that the threshold value Th1. A detailed configuration block diagram of a circuit portion forming a comparison with Th2 and a selection flag is shown.

The Th switching control signal is a signal in which IIHII and "L" are inverted for each field, like the Fl signal, and is applied to the second threshold generation circuit 52 via the threshold generation circuit 53 and the inverter 51. The threshold value generation circuits 52 and 53 generate the threshold value Thl in the relationship shown in FIG. 6(b) in response to the switching signal.
Or generate the same Th2. Comparison circuits 54 and 55 respectively compare the memory image and the through image with the thresholds from the threshold generation circuits 52 and 53, and output signals A and B as determination results. AND gate 56 ANDs the A and B signals and outputs a selection flag signal. switch 57
is switched in accordance with the selection flag signal to select the memory image or through image signal.

FIG. 7 shows a gradation characteristic diagram. The solid line in (a) is the characteristic diagram of a normal video camera, and the input/output is linear up to 100χ, and for inputs beyond that (100 to 400
χ), there is a relationship with a gentle slope called the KNEE characteristic. Assuming that this point of change is Pl, this point of change shifts to position P2 during high-speed shutter operation. However, Pl
is 1760 seconds, and P2 is 1/250 of the 2-step exposure change.
Suppose it is seconds. As mentioned above, in the case of a difference of 4 steps,
The relationship is as shown in (1) and (5) in FIG. 7(d). By the way,
(1) in Figure 7(d) is l/60 seconds, (2) is 1/12
5 seconds, (3) is 1/250 seconds, (4) is 11500 seconds,
(5) is a characteristic diagram when the time is 171000 seconds. Synthesize a characteristic with a desired curve from two characteristics with different slopes.

Figure 7('b) shows an example of the composite characteristics in each case where the thresholds for determining overexposure and underexposure are different, and Figure 7(C) shows the average of the signals of two corresponding pixels and output. (1), selecting one (2), and averaging using appropriate coefficients (3).

In the above example, it is realized in 1 second! The time resolution is 30 images, which is comparable to a frame storage CCD image sensor. Therefore, an embodiment will be described in which two screens are captured in one field in order to achieve a time resolution comparable to that of a field accumulation CCD image sensor.

FIG. 9 shows a modified part of the configuration example, and FIG. 8 shows a timing chart. In principle, the signal of the image sensor 103 is read out at a speed faster than the normal video rate, and the signal is converted back to the normal rate by time axis conversion.The field memories 90 and 91 each store one field worth of images. It has a storage capacity equivalent to the information, and the memory 90 has a storage capacity of 1/1
The accumulation signal is delayed by 1/1000 seconds to synchronize with the 20 second readout, and the memory 91 takes twice the time to change the video signal of 1/120 second to the NTSC signal of 1/60 second. (a) to (d) in FIG. 9, which perform the decompression process.
) correspond to signals (a) to (d) in FIG. As the image sensor 103 that has such an effect, an MO3 solid-state image sensor using an XY address method can be considered.

Next, another detailed example of the control circuit 108 is shown in FIG.

Master clock generator 40 generates a master clock for internal use in control circuit 108 in accordance with an external reference signal. Clock generator 41 for 1/1000 shutter
generates a high-speed clock according to its master clock, and the clock generator 42 for 1/60 shutter generates a low-speed clock according to its master clock. The switch 45 is switched for each field and alternately applies the outputs of the clock generators 41 and 42 to the drive circuit 105. AE! The rlJ5fB signal generator 43 generates an AE control signal for aperture control based on the video signal from the camera signal processing circuit 104. The control signal holding circuit 44 holds the control signal for one field.

The switch 46 is switched for each field, and alternately applies the output of the AE control signal generator 43 and the holding signal from the control signal holding circuit 44 to the aperture control circuit 106. A switching signal generator 47 controls switching of switches 45,46. Switches 45 and 46 are switched synchronously.

In this embodiment, a clock generator is provided for low speed and high speed, and the clocks are switched by the output signal of a switching signal generator that generates a signal for each field, which simplifies the circuit configuration and operation. This effect is particularly suitable for videos.

In the above embodiments, screens with different exposure amounts were generated by changing the shutter speed, but if a high-speed aperture device is available, the aperture may be changed at high speed. It may also be realized by a method of electrically controlling the neutral density filter, as in the example shown in FIG.

〔Effect of the invention〕

As can be easily understood from the above explanation, according to the present invention, it is possible to substantially widen the dynamic range. For example, even in backlighting, not only the main subject but also the background can be An image with a certain amount of exposure can be obtained as a moving image.

[Brief explanation of the drawing]

Figure 1 shows a camera-integrated VTR using an embodiment of the present invention.
2 is a specific configuration block diagram of the control circuit 108 of the camera unit shown in FIG. 1, FIG. 3 is an operation timing chart of the image sensor, and FIG. 4 is a conceptual diagram of image processing according to the present invention. , FIG. 5 is a concrete block diagram of the arithmetic circuit 202 shown in FIG. 1, FIG. 6 is a diagram explaining a method for determining threshold values for determining overexposure and underexposure, FIG. 7 is a gradation characteristic diagram, and FIG.
9 is a timing chart of another embodiment, FIG. 9 is a block diagram of its configuration, and FIG. 10 is an example of the control circuit 108 of FIG. 1. 100・-Camera section 200・-Processing section 300-...
・Recording Department I! 5 Figure 6 (a) Jinmuginshu (b) Threshold ■ Direct support also (d) 5 Ittas and -F Moatshi 95 7th figure ← l Shito + - no V - le)'--←-- −−−−Sword−
? Meto'-1 Figure 8 Figure 9 Procedure Nefu official document (method/spontaneous) 1. Display of the case 1986 Patent Application No. 143886 2 Name of the invention Imaging device 3 Person making the amendment Relationship to the case Patent applicant 3-30-2 Shimomaruko, Ota-ku, Tokyo Canon Co., Ltd. Representative Ryuzo Kaku 4. Agent 8170 Higashiikebukuro, Toshima-ku, Tokyo - 31-13-
402 No. 6, subject of amendment Drawing 7 attached to the application,
Contents of correction The engraving of the drawing (no changes to the content) will be corrected as shown in the attached sheet.

Claims (2)

[Claims] (1) An imaging means; a control means for continuously outputting images with different exposure amounts from the imaging means; 1. An imaging device comprising: compositing means for obtaining . (2) The apparatus according to claim (1), wherein the control means is means for cyclically and continuously outputting images having different exposure amounts.