JPS63306780A - Image pickup device - Google Patents

Abstract

PURPOSE:To substitute a signal having a part of inappropriate exposure amounts with a signal of a picture element part corresponding to a part obtained by picking up an image with different exposure amount, and to obtain moving pictures with virtually wide dynamic range by accumulating plural video signals of different exposure amounts temporarily. CONSTITUTION:A direct output from an image pickup element 103 is herein called a through image, and signals in the preceding field that are temporarily stored in an image memory 204 are called a memory image or a memory output. Of the through image, the image of a main object at the time of halation is painted black in every odd numbered field, while the background is partial void in every even-numbered field. Since a memory image is made of signals that are delayed for one-field length, a partial void and a black painting are generated in fields other than those of a through image. Hence by suitably combining a through image and a memory image, a partial video that includes no void or black painting can be obtained. That is, in every field, the signals of a through image and a memory image are compared with a prescribed thresh old, making them '1' when larger than the threshold, but '0' when smaller in order to decide on a partial void or black-painting for every picture element.

Description

DETAILED DESCRIPTION OF THE INVENTION [Field of Industrial Application] The present invention relates to an imaging device having a substantially wide dynamic range.

[Conventional technology]

Imaging devices are widely used as video camera parts such as camera-integrated VTRs and still video cameras. - The range is narrow, so when there is backlight, etc., whites and shadows are crushed (common name for areas where the brightness level is extremely high or low).

In conventional video cameras, in such cases, the aperture was opened by about two stops to adjust the amount of light either manually or by operating a backlight correction button.

However, even if such backlight compensation is performed properly,
Even if the main subject has the proper exposure, overexposure will occur in the background, leaving the screen with only a white background.

In other words, the narrow dynamic range of the imaging device cannot be solved by simply adjusting the light amount so that the exposure amount of the main subject is appropriate, as in conventional devices. For example, in a conventional imaging device that converts a still image into an electrical signal using a line scanner or the like, a configuration is considered in which a single screen is synthesized from a plurality of screens obtained from the same subject and each having a different amount of exposure.

[Problem that the invention seeks to solve]

However, this conventional imaging device targets still images, and cannot obtain moving images with a wide dynamic range.

In view of these problems, an object of the present invention is to provide an imaging device that can obtain moving images with a substantially wide dynamic range.

[Means for solving problems]

An imaging apparatus according to the present invention includes an imaging means, a control means for outputting video signals of a plurality of screens of the same subject with different exposure amounts from the imaging means, and a control means for outputting video signals of the plurality of screens output from the imaging means. Image memory that can store images freely,
It consists of an arithmetic processing means that selects or processes corresponding signals of a plurality of screens corresponding to the pixel of interest as necessary to generate an output pixel signal when the signal of the pixel of interest is determined to be inappropriate.

[Effect]

By outputting video signals from multiple screens with different exposure amounts from the imaging means and temporarily storing the video signals in the image memory, it is possible to detect which pixels have an appropriate exposure amount or which have an inappropriate exposure amount. Not only can the pixel portion be determined, but also a portion with an inappropriate exposure amount can be replaced with a signal from a corresponding pixel portion of a video signal photographed with a different exposure amount. Therefore, an appropriate image can be obtained over a wide exposure range, and the dynamic range of the imaging means is substantially expanded.

〔Example〕

Hereinafter, one embodiment of the present invention will be described with reference to the drawings. FIG. 1 shows a block diagram of the overall configuration when the present invention is applied to a camera-integrated VTR.

In FIG. 1, 100 is a camera section, 200 is a processing section,
300 is a recording section. In the camera unit 100, the light beam entering from the optical system 101 is limited in light amount by the aperture 102, and is imaged on the image sensor 103. The image sensor 103 consists of an image sensor, a semiconductor image sensor such as a MOS, or a CCD. The focus drive circuit 107, the aperture drive circuit 106, and the image sensor drive circuit 105 respectively control the optical system 101, the aperture 102, and the image sensor 1 under the control of the camera control circuit 108.
Drive 03. The camera signal processing circuit 104 is a well-known circuit that performs γ correction and other processing similar to the signal processing circuit of a normal video camera.

The video signal output from the camera section 100 is processed by the processing section 20.
0 is converted into a digital signal by the A/D converter 201,
The arithmetic circuit 202 performs pixel data conversion, which will be described later.
The D/A converter 203 converts the signal back to an analog signal, and the recording unit 3
00.

204 is an image memory for calculation in the calculation circuit 202, and 205 is its addressing circuit. The addressing circuit 205 writes data into the image memory 204 in response to a timing signal from the control circuit 108 of the camera unit 100.
Outputs read address control signal.

In the recording section 300, the analog signal from the D/A converter 203 is recorded on the VTR recorder 301 using a known method.

Next, the operation of the image sensor 103 will be explained. FIG. 2 is a more detailed block diagram of the configuration of the camera section 100, and FIG. 3 shows a timing chart of the camera section 100, taking an NTSC signal as an example. The field index (FI) signal is a signal for distinguishing between odd (ODD) and even (EVEN) fields that make up one frame. *VILJ signal is a vertical blanking signal and is a high (H) signal. The period corresponds to the effective screen, and the L (low) part corresponds to the vertical blanking period. C for vertical transfer
This is a pulse for reading out to OD. The iris gate signal is a signal that specifies whether to use a 1/1000 second accumulation signal or a 1760 second accumulation signal as a reference video signal for automatic exposure, which will be described later.

In the illustrated example, 171,000 during the vertical blanking period.
1/10OO of the second is accumulated in the next valid screen period.
Outputs second accumulation signal. Then, charge is accumulated for substantially 1/60 second during the effective screen period immediately after the 1/1000 second accumulation period, and the 1760 second accumulation signal is output during the effective screen period of the next field. In this way, two types of light intensity signals (1/1000 seconds and 1760 seconds) are alternately output for each field.

In addition, in FIG. 2, 20 is the camera signal processing circuit 10.
A known AE control circuit receives a signal (for example, a video signal) from 4 and calculates a control signal for exposure control, and 22 is a known AE control circuit that outputs a control signal for focus control.
1113 in FIG. 31 is a 1/2 frequency divider circuit that divides the frequency of the vertical blanking signal VIILJ by two. 26.27 is a sample and hold circuit, 28 is an inverter, 29.30 is 1
This is a switch for selecting which of the output of the /2 frequency divider circuit 24 or its inverted signal by the inverter 28 determines the sampling timing. The outputs of the sample and hold circuits 26 and 27 are applied to the aperture drive circuit 106 and the focus drive circuit 107, respectively, to perform automatic exposure control and automatic focus adjustment.

In the above example, it is a combination of 1/1000 seconds and 1760 seconds, and the light amount changes by about 4 steps (24 times).
For example, in the case of a camera using a CCD image sensor, EVEN
If you adjust the exposure to the main subject based on the accumulation time of 1760 seconds in the field, overexposure tends to occur in the background with that EVEN field, but with OD that reduces the light amount by 4 steps.
In the D field, blackout often occurs on the main subject. Note that this example assumes that the exposure is adjusted to the background side during backlight correction, and of course, it may be set to a value other than 1/1000 seconds depending on the situation.

In the present invention, such overexposure and/or underexposure in one field is actively utilized to improve the screen. In other words, areas where blown-out highlights or blown-out shadows occur are replaced with corresponding areas from other fields (no blown-out highlights or blown-out highlights occur because the exposure is different), and the signals from both fields are combined to create the final image. Signal. The basic idea will be explained with reference to FIG. In FIG. 4, the main subject is schematically shown as a vertically long rectangle.

In FIG. 4, the through (T) image refers to the direct output of the image sensor 103, and the memory (M) image or memory output refers to the signal of the previous field once stored in the image memory 204.
In the through-the-lens image, the backlit main subject is blown out in shadows in each ODD field, and the background is blown out in highlights in each EVEN field. Furthermore, since the memory image consists of a signal delayed by one field period, blown-out highlights and blown-out shadows occur in a different field from the through-the-lens image.

Therefore, by appropriately combining the through-the-lens image and the memory image, a good image without blown-out highlights or blown-out shadows can be obtained. That is, for each field, the signals of the through image and the memory image are compared with a predetermined darkness value, and if it is larger than the darkness value, it is set as 1, and if it is smaller, it is set as O, and it is determined whether there is overexposure or underexposure for each pixel.

FIG. 6 shows the relationship between the threshold value, the brightness value of the pixel, and the field. In Figure 6(a), the horizontal axis is the brightness level, and the vertical axis is 1.
Indicates the frequency of appearance of each brightness level on the screen.

As shown in FIG. 6(a), the threshold value Thl is set to allow determination of underexposure, and the darkness value Th2 is set so as to determine overexposure. In other words, an image below Thl is determined to be blown-up shadows, and an image above the threshold Th2 is determined to be blown-out highlights. FIG. 6(b) shows the relationship between each field and the darkness value. As described above, since overexposure and underexposure occur alternately in the ODD field and the EVEN field, the threshold for determining this is also changed for each field.

In this way, it is possible to determine which pixel portion of which field has a blocked-up shadow or a blown-out highlight, and by using the decision result, it is possible to select a pixel signal with an appropriate exposure amount for the through-the-lens image and the memory image. For example, take the logical product of judgment A and judgment B, and in the ODD field, select the signal of the through image for the pixel whose logical product is 1, and select the signal of the memory image for the pixel whose logical product is 0. By selecting a signal and having the opposite relationship in the EVEN field, a selection flag as shown in FIG. 4 is obtained. The picture at the bottom of FIG. 4 shows a composite image based on the selection flag. In this figure, we assumed that the main subject was moving at a constant speed, and confirmed the effect of time axis shift on the image, but it can be seen that the resulting video is sufficient for practical use.

FIG. 5 shows that the threshold value Th1. A detailed configuration block diagram of a circuit portion forming a comparison with Th2 and a selection flag is shown.

The Th switching control signal is a signal in which "H" and "L" are inverted for each field, like the Fl signal, and is applied to the second dark value generation circuit 52 via the threshold generation circuit 53 and the inverter 51. be done. The threshold value generation circuits 52 and 53 generate the threshold value Thl or Th2 in the relationship shown in FIG. 6(1)) in accordance with the switching signal. Comparison circuits 54 and 55 respectively compare the memory image and the through image with the thresholds from the threshold generation circuits 52 and 53, and output signals A and B. AND gate 56 ANDs the A and B signals and outputs a selection flag signal. The switch 57 is switched according to the current selection flag signal to select the memory image or live image signal.

FIG. 7 shows a gradation characteristic diagram. The solid line in (a) is the characteristic diagram of a normal video camera, and the input/output is linear up to 100χ, and for inputs beyond that (100 to 400
χ), there is a relationship with a gentle slope called the KNEE characteristic. Assuming that this point of change is Pl, this point of change shifts to position P2 during high-speed shuffling. However, Pi
is 1/60 second, and P2 is 1/250 of the 2-step exposure change.
Suppose it is seconds. As mentioned above, in the case of a difference of 4 steps,
The relationship is as shown in (1) and (5) in FIG. 7(d). By the way,
(1) in Figure 7(d) is 1760 seconds, (2) is 1/12
5 seconds, (3) is 1/250 seconds, (4) is 11500 seconds,
(5) is a characteristic diagram when the time is 171000 seconds. Synthesize a characteristic with a desired curve from two characteristics with different slopes.

FIGS. 7(b) and (C) are examples of the synthesis.

A method of synthesizing gradation characteristics will be specifically explained. Let us take as an example the case where either the change point PI (in the case of low shutter speed) or P2 (in the case of high shutter speed) from the linear part to the KNEE characteristic part is controlled to be the 100χ point. . As shown in FIG. 6, a threshold value is provided to determine overexposure and underexposure on the screen, and the gradation characteristics change depending on the setting of the threshold value. Figure 7 (b) shows the state of the change.
). The characteristics of (1) to (3) are that when the threshold Th2 for overexposure determination is sequentially changed from a low value to a high value, the position where the high-speed shutter side is selected by pixel switching by the switch 57 is high brightness. Shows how it changes to the side.

In FIG. 5, one pixel signal is selected by the switch 57, but the data of two corresponding pixels may be processed to obtain the desired signal. For example, the characteristic (2
) is shown in FIG. 7(C) when various calculation methods are selected. Characteristic (2) in FIG. 7(C) is the same as characteristic (2) in FIG. 7(b). Figure 7 (C)
The characteristic +1) indicates the case where average value processing is performed using the data of the pixel for which the "overexposure" determination has been made and the data of the corresponding pixel on another screen. Characteristic (3) shows the case where subtraction processing is performed. For example, the corresponding 2
Let the pixel data be Dl and Dl, respectively, and the processing result is D
Then, in average value processing, D= CDI +Dz)
/2, and in the subtraction process D”Dl k (Dl
Di). However, k is changed according to the set threshold value. In characteristic (3) of FIG. 7(C), k is approximately 1.88. In addition to such average value processing and subtraction processing, so-called offset processing such as constant addition and constant subtraction may be used, or these may be used in combination.

Next, another detailed example of the control circuit 108 is shown in FIG. 8. A master clock generator 40 generates a master clock for internal use in the control circuit 108 in accordance with an external reference signal. Clock generator 41 for 1/1000 shutter
generates a high-speed clock according to its master clock, and the clock generator 42 for the 1760 shifter generates a low-speed clock according to its master clock. The switch 45 is switched for each field and alternately applies the outputs of the clock generators 41 and 42 to the drive circuit 105. The AE control signal generator 43 generates AE? for aperture control based on the video signal from the camera signal processing circuit 104.
#I? Generates a B signal. The control signal holding circuit 44 holds the control signal for one field.

The switch 46 is switched for each field, and alternately applies the output of the AE control signal generator 43 and the holding signal from the control signal holding circuit 44 to the aperture control circuit 106. A switching signal generator 47 controls switching of switches 45,46. Switches 45, 46 switch synchronously.

In this embodiment, a clock generator is provided for low speed and high speed, and the clocks are switched by the output signal of a switching signal generator that generates a signal for each field, which simplifies the circuit configuration and operation. This effect is particularly suitable for videos.

〔Effect of the invention〕

As can be easily understood from the above description, according to the present invention, the dynamic range of the image sensor can be substantially widened. Specifically, even when backlit, it is possible to take pictures in which not only the main subject but also the background is properly exposed. Furthermore, if the main subject is properly exposed,
The quality of the background (level of overexposure, etc.) can be adjusted to suit your preference, allowing for more natural depictions and intentional drawings. Of course, conversely, it is also possible to set the background to proper exposure and adjust the degree of blackout of the main subject. In addition to the pixel data calculation method, a countless wide range of characteristics can be obtained by combining the shutter speed.

[Brief explanation of the drawing]

Figure 1 shows a camera-integrated VTR using an embodiment of the present invention.
2 is a specific configuration block diagram of the control circuit 108 of the camera unit shown in FIG. 1, FIG. 3 is an operation timing chart of the image sensor, and FIG. 4 is a conceptual diagram of image processing according to the present invention. , FIG. 5 is a concrete block diagram of the arithmetic circuit 202 shown in FIG. 8
The figure is another configuration block diagram of the control circuit 108. 100- Camera section 200-... Processing section 300-...
・Recording department

Claims (2)

[Claims] (1) An imaging means, a control means for causing the imaging means to output video signals of a plurality of screens with different exposure amounts of the same subject, and an image capable of storing some of the video signals of the plurality of screens output from the imaging means. memory and, if the signal of the pixel of interest is determined to be inappropriate, select or process the corresponding signals of a plurality of screens corresponding to the pixel of interest as necessary;
An imaging device comprising: arithmetic processing means that generates an output pixel signal. (2) The control means sequentially outputs the video signals of the plurality of screens from the imaging means every predetermined period, and the arithmetic processing means performs the arithmetic processing within the predetermined period and outputs the output image signal. An imaging device according to claim (1).