JPH0817455B2 - Imaging device - Google Patents

Description

The present invention relates to an image pickup device having a substantially wide dynamic range.

[Conventional technology]

The image pickup device is widely used as a video camera unit such as a VTR with a built-in camera and a still video camera. A video camera that uses an image pickup tube or a solid-state image sensor has a narrower dynamic range than a conventional silver halide photography system, and therefore, underexposure or underexposure (a common name for parts with extremely high or low brightness levels). Occurs.
In a conventional video camera, in such a case, the aperture is opened by about 2 apertures manually or by operating the backlight compensation button,
I was adjusting the amount of light.

However, even when such backlight correction is appropriately performed, overexposure occurs in the background even if the main subject has an appropriate exposure amount, and the screen has only a white background. In other words, the narrow dynamic range of the image pickup device cannot be solved only by adjusting the light amount so that the exposure amount of the main subject becomes appropriate as in the conventional device. For example line
In a conventional image pickup apparatus that converts a still image into an electric signal by using a scanner, it is considered that one screen is composed from a plurality of screens obtained from the same subject and having different exposure amounts.

[Problems to be solved by the invention]

However, such an image pickup device is intended for a still image and cannot obtain a moving image with a wide dynamic range.

In view of such a situation, the present applicant has presented an image pickup apparatus that can obtain a moving image signal having a substantially wide dynamic range. However, the present invention provides an image pickup apparatus that can appropriately perform exposure control and focusing control. It is intended to be presented.

[Means for solving problems]

An image pickup apparatus according to the present invention stores an image pickup unit that picks up the same subject a plurality of times with different exposure amounts, and a memory that stores the immediately preceding screen among a plurality of screens with different exposure amounts obtained by the image pickup unit. Means and the signal level in the screen stored in the storage means and the signal level in the next screen are respectively compared with predetermined reference values to select an appropriate level signal, and the selected appropriate level signal is selected. The control means for optimizing the signal level of the entire screen by composing one screen and the signal of a predetermined one screen among a plurality of screens with different exposure amounts obtained by the imaging means are displayed. The exposure adjustment signal forming unit that periodically holds and forms the exposure adjustment signal and the output of the exposure adjustment signal forming unit are used in common to perform the exposure amount adjustment of the entire imaging performed a plurality of times by the imaging unit. Dew And having a quantity adjustment means.

The image pickup apparatus according to the present invention also stores an image pickup unit that picks up the same subject a plurality of times with different exposure amounts, and a previous screen among a plurality of screens with different exposure amounts obtained by the image pickup unit. A signal of the proper level is selected by comparing the signal level in the screen and the signal level in the next screen stored in the storage unit with a predetermined reference value, and the signal of the selected proper level is selected. And a control means for optimizing the signal level of the entire screen by synthesizing one screen and a plurality of screens with different exposure amounts obtained by the imaging means, and only the signal of one predetermined screen Is periodically held to form an image pickup adjustment signal, and the output of the adjustment signal formation unit is commonly used to perform automatic focus adjustment and whitening for the entire plurality of image pickups by the image pickup unit. And having at least one of them performs adjustment means of the lance adjustment.

[Action]

By the above means, it is possible to obtain an image signal having an appropriate signal level in any part of the screen. For example, it is possible to obtain an image signal without blown-out highlights in the screen even in the backlit state. As a result, the dynamic range is substantially expanded.

Furthermore, the exposure amount adjustment, the automatic focus adjustment, and / or the white balance adjustment can be properly performed between a plurality of images, and these adjustment operations are stabilized.

〔Example〕

An embodiment of the present invention will be described below with reference to the drawings.
FIG. 1 is a block diagram of the entire configuration when the present invention is applied to a camera-integrated VTR.

In FIG. 1, 100 is a camera unit, 200 is a processing unit, and 300.
Is a recording unit. In the camera unit 100, the amount of light rays incident from the optical system 101 is limited by the diaphragm 102, and the image pickup device 1
Image on 03. The image pickup device 103 is composed of an image pickup tube and a semiconductor image pickup device such as a MOS or CCD. The focus drive circuit 107, the diaphragm drive circuit 106, and the image pickup device drive circuit 105 are camera control circuits.
Under the control of 108, the optical system 101, the diaphragm 102, and the image sensor 103 are driven, respectively. The camera signal processing circuit 104 is a well-known circuit that performs γ correction and other processing similar to the signal processing circuit of a normal video camera.

The video signal output from the camera unit 100 is output by the processing unit 200.
Connected to the digital signal by A / D converter 201,
The pixel data conversion described later in 2 is performed, and the D / A converter 203
Is converted back into an analog signal and supplied to the recording unit 300. 204
Is an image memory for calculation in the arithmetic circuit 202, and 205 is its addressing circuit. Addressing circuit 20
Reference numeral 5 outputs a write / read address control signal for the image memory 204 in response to a timing signal from the control circuit 108 of the camera section 100.

In the recording unit 300, the analog signal from the D / A converter 203 is recorded in the VTR recorder 301 by a known method.

Next, the operation of the image sensor 103 will be described. FIG. 3 shows a timing chart of the camera unit 100 when the NTSC signal is taken as an example. Field index (FI) signals are odd (ODD) fields and even (E)
This is a signal to distinguish it from the VEN) field. V _BLK
The signal is a vertical blanking signal, the H (high) period corresponds to the effective screen, and the L (low) portion corresponds to the vertical blanking period. T _pulse is a signal for controlling the charge accumulation time of the image sensor 103, and is a pulse for reading pixel output to the vertical transfer CCD in the case of a CCD image sensor, for example. The iris gate signal is a signal that specifies whether to use a 1/1000 second accumulated signal or a 1/60 second accumulated signal as a video signal that serves as a reference for automatic exposure, which will be described later.

In the illustrated example, 1/1000 second accumulation is performed during the vertical blanking period, and the 1/1000 second accumulation signal is output during the next effective screen period. Then, the charge is accumulated for substantially 1/60 seconds in the effective screen period immediately after the 1/1000 second accumulation period, and the 1/60 second accumulation signal is output in the effective screen period of the next field. In this way, there are two types (1/1000 seconds and 1/60 seconds) for each field.
Seconds) light intensity signals are output alternately.

In the above embodiment, a combination of 1/1000 sec and 1/60 second, since the light quantity change of about 4 stages (2 ^4x), for example, a CCD
In the case of a camera using an image sensor, 1/60 in the EVEN field
When the exposure is adjusted to the main subject based on the accumulation time of 2 seconds, overexposure is likely to occur in the background in the EVEN field, whereas black shadow often occurs in the main subject in the ODD field where the four-step light amount is reduced. . It should be noted that this example assumes the case where the exposure is adjusted to the background side at the time of backlight correction, and of course, it may be set to a value other than 1/1000 seconds depending on the situation at the spot.

According to the present invention, such overexposure and / or underexposure in one field is positively utilized to improve the screen. In other words, the portion where whiteout or blackout occurs is replaced by the corresponding portion of another field (blackout or whiteout does not occur because the exposure is different), and the signals of both fields are combined to produce the final image. Signal. The basic idea will be described with reference to FIG. In FIG. 4, the main subject is schematically shown by a vertically long rectangle. In FIG. 4, a through (T) image means a direct output of the image sensor 103, and a memory (M) image or a memory output means an image memory 204.
The signal of the immediately preceding field once stored in. In the through image, the main subject at the time of backlighting is blacked out in each ODD field, and the background is overexposed in each EVEN field. Also, since the memory image is composed of signals delayed by one field period, overexposure and underexposure occur in a field different from that of the through image.

Therefore, if the through image and the memory image are properly combined, a good image without overexposure and underexposure can be obtained. In other words, the through image signal and the memory image signal are compared with a predetermined threshold value for each field, and if the value is larger than the threshold value, the value is set to 1 and if the signal is smaller than the threshold value, whiteout or blackout is determined for each pixel. FIG. 6 shows the threshold value and the brightness value of the pixel,
Indicates the relationship with the field. In FIG. 6A, the horizontal axis represents the brightness level and the vertical axis represents the appearance frequency of each brightness level in one screen. As shown in FIG. 6 (a), the threshold value Th1 is set so as to be able to determine blackout, and the threshold value Th2 is set so as to be able to determine whiteout. That is, it is determined that Th1 or less is blackout, and threshold Th2 or more is overexposure. FIG. 6B shows the relationship between each field and the threshold value. As described above, overexposure and underexposure alternate between the ODD field and the EVEN field, so the threshold value for the determination is also changed for each field.

In this way, it is possible to determine which pixel portion of which field is blackout or overexposure, and thus the determination result can be used to select a pixel signal with an appropriate exposure amount for a through image and a memory image. For example, the logical product of the determination A and the determination B is taken, and in the ODD field, the through image signal is selected for the pixel having the logical product 1 and the memory image is selected for the pixel having the logical product 0. By selecting a signal and setting it in the opposite relationship in the EVEN field, a selection flag as shown in FIG. 4 is obtained. The picture at the bottom of FIG. 4 shows a composite image according to the selection flag. In this figure, the effect of the time axis shift on the image was confirmed on the assumption that the main subject moves at a constant velocity, but it can be seen that the moving image can be practically sufficient.

FIG. 5 shows the above threshold value in the arithmetic circuit 202 of the processing unit 200.
FIG. 3 is a detailed block diagram showing the configuration of a circuit part for comparing with Th1 and Th2 and forming a selection flag. The Th switching control signal is a signal in which “H” and “L” are inverted for each field like the FI signal, and is applied to the second threshold generation circuit 52 via the threshold generation circuit 53 and the inverter 51. It Threshold generator circuit 52,5
3 is the threshold value T of the relationship in FIG. 6 (b) according to the switching signal.
Generates h1 or Th2. The comparison circuits 54 and 55 compare the memory image and the through image with the threshold values from the threshold value generation circuits 52 and 53, respectively, and output the A signal and the B signal. The AND gate 56 calculates the logical product of the A signal and the B signal and outputs a selection flag signal. The switch 57 switches according to the selection flag signal and selects a signal of a memory image or a through image.

FIG. 7 shows a gradation characteristic diagram. The solid line in (a) is the characteristic diagram of a normal video camera, the input and output are linear up to 100%, and the slope called the KNEE characteristic for inputs higher than that (100 to 400%). Has a loose relationship. If this change point is P1, then this change point moves to the position of P2 during high-speed shuttering. However, it is assumed that P1 is 1/60 second and P2 is 1/250 second of the change of the exposure amount in two steps. In the case of the difference of 4 steps as described above, (1) and (5) in FIG.
It becomes a relationship. By the way, (1) in Fig. 7 (d) is 1/60
Seconds, (2) 1/125 seconds, (3) 1/250 seconds, (4) 1/50
0 seconds and (5) are characteristic diagrams when 1/1000 second is set. A characteristic having a desired curve is synthesized from two characteristics having different inclinations. FIG. 7B shows an example of the combined characteristics when the threshold values for overexposed and underexposed areas are changed. The characteristics (1) to (3) are the overexposed threshold values Th2 in order from the lowest value. When the value is changed to a high value, a state in which the position for selecting the high-speed shutter side is changed to the high-luminance side by the pixel switching by the switch 57 is shown. FIG. 7 (c) shows the combined characteristic when various calculation methods are selected with reference to the characteristic (2) of FIG. 7 (b). (2) is the same as (b) and (2) in FIG. 7, (1) is a composite characteristic when an average value is obtained, and (3) is a composite characteristic when subtraction processing is performed with an appropriate coefficient.

FIG. 2 shows a more detailed block diagram of the camera section 100. 20 is a known AE control circuit that receives a signal (for example, a luminance signal) from the camera signal processing circuit 104 and calculates a control signal for exposure control, and 22 is a known AE control circuit that outputs a control signal for focus control. The AF control circuit 24 is a 1/2 divider circuit that _{divides the} vertical blanking signal V _BLX by two. 26 and 27 are sample and hold circuits, 28 is an inverter, and 29 and 30 are switches for selecting which of the output of the 1/2 frequency divider circuit 24 and its inverted signal from the inverter 28 determines the sampling timing. The outputs of the sample and hold circuits 26 and 27 are applied to the diaphragm drive circuit 106 and the focus drive circuit 107, respectively, and automatic exposure control and automatic focus adjustment are executed.

As described above, when signals of a plurality of screens with different exposure amounts can be obtained in one field, it is necessary to change part of the camera signal processing. That is, AE (automatic aperture adjustment),
This is a case where an internal image pickup signal is used in each control system such as AF (automatic focus adjustment) and AWB (automatic white balance adjustment). Therefore, as a main example, the AE process will be described with reference to FIG. The signal obtained from the image sensor 103 is sent from the camera signal processing circuit 104 to the AE control circuit 20 for AE control, and the AE control circuit 20 operates the diaphragm drive circuit 106 so that the serve loop of the AE control operates. Supply a control signal to the. This control signal is a dynamic
Aperture 10 so that it is within the range, that is, when it is bright
2 is narrowed down, and the aperture 102 is changed to open when it is dark.

However, when the light quantity is converted in a cycle such as every field, an appropriate AE operation cannot be expected. I mean,
This is because the response speed of the AE servo is much longer than the change in the field or frame period and settles at a halfway value. In order to avoid this, the control signal for the AE operation should be limited to only one of the plurality of exposure amount images. In FIG. 2, a sample and hold circuit 26 is provided, and the switch 29 defines the sample timing so that only a control signal corresponding to any one (for example, 1/60 seconds) screen is selected.

In FIG. 2, the sampling pulse is formed by the vertical blanking (V _BLK ) signal, and the 1/2 frequency divider 24 forms a signal that is inverted for each V _BLK signal. The signal obtained by inverting the output of the frequency divider 24 by the inverter 28 is connected to the b contact of the switch 29.
If the contact a side is selected with switch 22, ODD field, 1
Sampling pulse, which is the reference for AE loop control, is applied to the sample and hold circuit 26.
On the other hand, if the contact point b side is selected, the sampling pulse whose 1/60 second accumulation signal is the reference for the AE loop control is applied to the sample hold circuit 26. The same applies to the AF control circuit 22, sample / hold circuit 27, and switch 30 for AF control. Illustration of the AWB control is omitted.

The motion loop of the video camera is set to have a response of a few seconds, avoiding sudden changes in the screen. Therefore, for example, when the exposure time of 1/1000 second and 1/60 second is switched for each field, the AE loop shows the same response as when the exposure time of approximately 1/250 second is continuously performed. Since this can be regarded as a steady error, it can be solved by adding an offset bias according to the difference in the exposure amount to the AE loop. FIG. 8 shows an example of the modified configuration. Reference numeral 80 is a bias generation circuit that generates the bias, 81 is an adder that adds the bias to the AE loop, and 82 is an adder that adds the same offset bias to the AF loop. The same applies to AWB.

Next, another detailed example of the control circuit 108 is shown in FIG. The master clock generator 40 generates a master clock for the control circuit 108 according to an external reference signal. The 1/1000 shutter clock generator 41 generates a high speed clock according to its master clock, and the 1/60 shutter clock generator 42 generates a low speed clock according to its master clock. The switch 45 switches for each field, and alternately applies the outputs of the clock generators 41 and 42 to the drive circuit 105. AE control signal generator 43
Generates an AE control signal for aperture control based on the video signal from the camera signal processing circuit 104. The control signal holding circuit 44 holds the control signal for one field. The switch 46 switches for each field, and the AE control signal generator
The output of 43 and the holding signal from the control signal holding circuit 44 are alternately applied to the aperture control circuit 106. The switching signal generator 47 is
It controls the switching of the switches 45 and 46. The switches 45 and 46 switch in synchronization.

In this embodiment, a clock generator is provided for each of low speed and high speed, and the clock is switched by the output signal of the switching signal generator that generates a signal for each field, so that the circuit configuration and operation are simplified. The effect is that it is especially suitable for movies.

〔The invention's effect〕

As can be easily understood from the above description, according to the present invention, it is possible to take an image so that not only the main subject but also the background can be properly exposed even in the case of backlight, and a substantial dynamic range is expanded. sell. In addition, the basic operation of the camera can be operated accurately.

[Brief description of drawings]

FIG. 1 is a block diagram of a camera-integrated VTR using an embodiment of the present invention, and FIG. 2 is a control circuit of the camera section shown in FIG.
108 is a specific block diagram of the configuration, FIG. 3 is an operation timing chart of the image pickup device, FIG. 4 is a conceptual diagram of image processing according to the present invention, and FIG. 5 is a specific configuration block diagram of the arithmetic circuit 202 of FIG. 6, FIG. 6 is a diagram for explaining a method of determining a threshold value for determining overexposed areas and underexposed areas, FIG. 7 is a gradation characteristic diagram, FIG. 8 is a modified configuration example of FIG. 2, and FIG. It is a block diagram of other examples of composition. 100 …… Camera section, 200 …… Processing section, 300 …… Recording section

Front page continued (72) Inventor Toshiyuki Masui, 770 Shimonoge, Takatsu-ku, Kawasaki-shi, Kanagawa Canon Inc., Tamagawa Plant (72) Takashi Kobayashi, 770, Shimonoge, Takatsu-ku, Kawasaki-shi, Kanagawa Canon Inc.

Claims (2)

[Claims] 1. An image pickup means for picking up the same subject a plurality of times with different exposure amounts, and a storage means for storing the immediately previous screen among a plurality of screens with different exposure amounts obtained by the image pickup means. An appropriate level signal is selected by comparing the in-screen signal level stored in the storage means with the next in-screen signal level, respectively, and a selected appropriate level signal is synthesized. Control means for optimizing the signal level of the entire screen by configuring one screen, and cyclically only the signal of one predetermined screen among a plurality of screens with different exposure amounts obtained by the imaging means. An exposure adjustment signal forming means for holding and forming an exposure adjustment signal, and an exposure adjustment for performing exposure adjustment for the entire plurality of imaging operations by the imaging means by commonly using the output of the exposure adjustment signal format means. hand An imaging device having a step. 2. An image pickup means for picking up the same subject a plurality of times with different exposure amounts, and a storage means for storing the immediately preceding screen among a plurality of screens with different exposure amounts obtained by the image pickup means. An appropriate level signal is selected by comparing the in-screen signal level stored in the storage means with the next in-screen signal level, respectively, and a selected appropriate level signal is synthesized. Control means for optimizing the signal level of the entire screen by configuring one screen, and cyclically only the signal of one predetermined screen among a plurality of screens with different exposure amounts obtained by the imaging means. An adjustment signal forming unit that holds and forms an image adjustment signal and an output of the adjustment signal forming unit are commonly used to perform automatic focus adjustment and white balance adjustment of the entire image pickup performed by the image pickup unit. An image pickup apparatus comprising: an adjusting unit that performs at least one of the above.