JP2751126B2 - Imaging device having gradation control function - Google Patents

Description

Description: TECHNICAL FIELD The present invention relates to, for example, an image pickup device for picking up an image signal, and particularly to an image pickup device having a gradation control function suitable for backlight correction and the like.

[Prior Art] When capturing an image of a subject with a video camera or the like, the gradation in one screen becomes extremely wide, such as in the case of backlit shooting with high illuminance.

An imaging device using a solid-state imaging device or the like used in a video camera or the like does not have a dynamic range width enough to convert all of such wide gradations into a video signal, and thus the brightness level may be increased or decreased. Causes clipping. This is a phenomenon called “overexposure” or “underexposure”, in which the gradation of extremely bright or dark portions cannot be expressed in one screen.

Especially in backlit shooting, the main subject is shaded and extremely dark compared to the brightness of the background,
"Black crush" will occur.

As a countermeasure against this, conventionally, there has been used a method called BLC (Backlight Control) for performing backlight correction by increasing the exposure amount. Hereinafter, with reference to the drawings, backlight correction by a conventional method will be described in detail.

First, the BLC in the exposure adjustment mechanism using iris
The operation will be described with reference to the configuration diagram of the backlight correction circuit shown in FIG.

The light rays from the object incident from the optical system 10 are iris 11
As a result, the light amount is limited, and an image is formed on the image sensor 12. A photoelectric conversion signal corresponding to the imaging state is output from the image sensor 12, converted into a video signal by the signal processing circuit 13, and output as a video signal.

On the other hand, the above-described video signal is also supplied to the AE circuit 14, and the AE circuit 14 generates an iris control signal according to the video signal. The driver 16 receives an iris control signal from the AE circuit 14 via the adder 15, generates a drive signal for driving the ig meter 17, and realizes automatic adjustment of the light amount by the iris.

In the concept of the automatic adjustment of the light amount described above, the light amount is generally adjusted in accordance with the integral value of one screen of the luminance level of the video signal, so that attention is paid only to the average brightness of the entire screen. You are doing. for that reason,
In the case of backlight photography, for example, the amount of light incident on the image sensor 12 is adjusted in accordance with a very bright background, so that the main subject becomes extremely dark and “blackout” occurs.

In such a case, the photographer needs to turn on the BLC and issue a backlight correction instruction. In the circuit diagram of FIG. 17,
By turning on the BLC switch 19, the correction signal generator 1
From (8), the signal for turning the iris to the “open” side is
Is supplied to the driver 16. Driver 16
The meter 17 is driven in the opening direction, the amount of light incident on the image sensor 12 is increased, and backlight correction is performed.

By turning on the BLC switch 19 and performing the backlight correction according to the conventional method, the image becomes as shown in FIG. 18 (a).
The state changes from BLC-OFF (before backlight correction) shown in (b) to BLC-ON (after backlight correction) shown in (b). As can be seen from the figure, with BLC-OFF, the background is well captured, but the main subject is "black out" and not well captured. On the other hand, in the BLC-ON, the gradation of the main subject is expressed and well captured.

[Problem to be Solved by the Invention] However, in the above-described conventional imaging apparatus,
As can be seen from FIG. 18 (b), by performing the backlight correction, the amount of light received by the image sensor is increased, so that the main subject is well photographed, but the background is overexposed. I can't copy it.

That is, in the backlight correction by the conventional method, clipping occurs at one of the upper and lower levels due to the restriction of the dynamic range of the image sensor, and both the main subject and the background cannot be imaged well. There was.

Therefore, the imaging apparatus of the present invention has been made in view of the above-described problems, and has as its object to realize good exposure to both the main subject and the background, and to have an apparently wide gradation. An object of the present invention is to provide an imaging device having a tone control function capable of expression.

[Means for Solving the Problems] In order to solve the above problems and achieve the object, an imaging device having a gradation control function of the present invention receives an optical signal, performs photoelectric conversion, and accumulates information. Sensing means, and control means for changing the storable charge amount of the sensing means. The control means controls the storable charge amount in a plurality of steps during a unit light accumulation period for forming one screen. And the levels of the plurality of levels are controlled so as to be set in accordance with the luminance distribution characteristics of the image information.

[Operation] In the above configuration, the imaging apparatus having the gradation control function according to the present invention sets the amount of storable electric charge to be small during the unit light accumulation period to perform exposure suitable for a low-luminance subject. An operation of setting a large amount of storable charge and performing exposure suitable for a high-brightness subject is performed, and the level of the storable charge in each operation is determined according to the luminance distribution characteristics in the image information. The setting works so as to realize good exposure for both the low-luminance subject and the high-luminance subject in one screen.

Hereinafter, a preferred embodiment of the present invention will be described in detail with reference to the drawings.

FIG. 1 is a block diagram showing the configuration of one embodiment. In the figure, the lower half from the center is the same as the circuit configuration of the conventional example shown in FIG. The concept of the operation of the present embodiment will be described with reference to FIG.

Light rays from the subject incident from the optical system 10 are
The light amount is adjusted by 11 to form a subject image on the image sensor 1. The signal photoelectrically converted and output by the image sensor 1 is converted into a video signal by the signal processing circuit 13 and output to the outside as a video signal, while being supplied to the AE circuit 14 and the image analysis circuit 2. You.

The image analysis circuit 2 analyzes the luminance information and the like of the video signal, and supplies, for example, a luminance histogram and the like to the controller 3. By switching the mode switching switch 5 in accordance with the condition of the subject, one of the shooting modes, NORMAL-MODE (normal shooting mode) and BLC-ON-MODE (backlight correction mode), is selected. The controller 3 receives a photographing mode selection signal from the mode switching switch 5 and information such as a luminance histogram from the image analysis circuit 2,
The drive control of the image sensor 1 is performed according to the information.

On the other hand, an iris control signal corresponding to the video signal is generated from the AE circuit 14 and is applied to the driver 16 via the adder 15. The driver 16 receives the iris control signal, generates a drive signal for driving the ig meter 17, and performs automatic light amount adjustment by the iris.

A correction signal used in a state such as backlight is generated by the correction signal generator 4. The BLC switch 19 is interlocked with the mode switching switch 5 described above. When the subject is in a backlight state, that is, when the mode switching switch 5
ON when C-ON-MODE (backlight compensation mode) is selected
State. When the BLC switch 19 is turned on, the correction signal from the correction signal generator 4 is applied to the adder 15 as in the case of the conventional backlight correction, and the AE circuit is opened in the direction in which the iris 11 opens.
Shift the output of 14.

Therefore, in the present embodiment, in the backlight state, the iris control similar to the conventional backlight correction is performed, and the amount of light incident on the image sensor 1 is increased (the BLC switch 19 is ON).
The controller 3 applies a predetermined drive control to the image pickup device 1 so as to obtain a good exposure state for both the main subject and the background as shown in FIG. 2B.

By the way, in the case of BLC-OFF, it was explained in the prior art.
As in the case of the BLC-OFF, the “blackout” shown in FIG. 2A has occurred.

Next, the imaging device 1 controlled by the controller 3 will be described below.

There are many types of imaging devices based on the principle of photoelectric conversion, such as CCD, MOS, and BASIS. The explanation is given with the example of a CCD that is currently most commonly used for video cameras.

Among CCDs, differences in charge readout method such as FT type (frame transfer type), IT type (interline transfer type), FIT type (frame interline transfer type), and difference in unnecessary charge processing such as VOD type According to the semiconductor device structure. Here, VOD stands for Vertical Overflow Drain
Drain).

First, a basic structure of an interline transfer type (IT type) CCD will be described, and a high-speed shutter operation which is one of the basic operations of the CCD will be described.

Fig. 4 (a) shows an interline transfer type (IT
(Type) Conceptual diagram of CCD, 41 is a sensor unit (sensing means) for performing photoelectric conversion, 42 is a vertical transfer register, 44 is a horizontal transfer register, and 45 is an output amplifier. FIG. 4B is a cross-sectional view and a potential diagram along the line AA ′ in the figure.

In FIG. 4 (b), reference numeral 46 denotes a channel stop (CS) for separating pixels, 47 denotes a readout gate (ROG) for transferring the electric charge accumulated in the sensor section 41 to the vertical transfer register 42, 48
Denotes a substrate, and 49 denotes an oxide film.

The operation at the time of the high-speed shutter will be described with reference to FIGS. FIG. 5 is a diagram of one field (for example, about 1/60 second in NTSC) of a standard television signal,
φROG is a pulse applied to the readout gate 47,
When the logic level is “H”, the potential of the readout gate 47 decreases, and the electric charge of the sensor unit 41 is transferred to the vertical transfer register 42. The removal pulse φSUB is a pulse applied to the substrate 48, and sweeps out (removes) the electric charge accumulated in the sensor unit 41 at the time of “H” through the φSUB terminal.

In this example, in FIG. 5, φROG is during the vertical retrace interval, and φSUB is during the horizontal retrace interval. After reading the charge of the sensor unit 41 at time t0, the next period starts,
During the horizontal flyback period at time t1, φSUB = “H”, so that the charge from t0 to t1 does not remain in the sensor unit 41. Since φSUB = “L” during the period from time t1 to t2, the electric charge during this period is accumulated in the sensor unit 41, and φROG = at time t2.
The signal is transferred to the vertical transfer register 2 by the “H” pulse. After all,
The exposure time in this case is (t2−t1).

In this way, a high-speed shutter operation of the IT type CCD is realized.

Next, the frame interline transfer type (FIT
Type) The operation of the CCD will be described.

FIG. 6 is a conceptual diagram of a frame interline transfer type (FIT type) CCD. The difference from the interline transfer type CCD shown in FIG.
There is that. The number of storage cells in the storage unit 63 is the same as the number of cells in the sensor unit (sensing means) 61. After the charge from the sensor unit 61 is transferred to the vertical transfer register 62,
The data is transferred to the storage unit 63 during the vertical retrace period, and then transferred to the horizontal transfer register 64 at a predetermined timing.
It is read out through 65.

The cross-sectional view and the potential diagram along the line AA 'in FIG. 6 are the same as those in FIG. 4B, and the mechanism for discharging the electric charge and reading out the electric charge from the sensor unit 61 to the vertical transfer register 62. Is the same.

 The following is the operation of the FIT type CCD.

Next, the operation of the VOD type CCD image sensor shown in FIG. 3 will be described. The subject light indicated by the upper arrow is received by the photodiode (PD) of the sensor unit 33 via the oxide layer (SiO ₂ ) 32 and the hole accumulation layer for dark current reduction. The aluminum layer (A
1) Light is shielded at 36. Due to this shading, the light receiving area as the CCD is not effectively utilized, and the aperture ratio is reduced. In this embodiment, a condenser lens 31 for compensating for a decrease in the aperture ratio is provided for each pixel.

The charge generated by the photoelectric conversion is V-CCD for vertical transfer.
(Vertical transfer register) 38, sequentially transferred on a two-dimensional plane, and output as a voltage value from the read amplifier.

A channel stop portion 39 is provided for each pixel, and is separated so that charges generated in each pixel are not mixed. A P layer 34 and an N-SUB layer 35 are provided below the sensor unit 33, and a substrate bias potential Vs applied to both layers.
Unnecessary charges are processed by the ub 30.

FIG. 7 is a conceptual diagram showing a charge discharging operation of a VOD type CCD by a potential potential. The line indicated by “CHARGE” is a potential potential diagram in the state of accumulation of photocharge, and “DISCHA
A line indicated by “RGE” is a potential potential diagram in an unnecessary charge discharging state. In the "CHARGE" state, charges are accumulated in the concave portion above the electrogram. In the “DISCHARGE” state, ΔVsub is further applied to the substrate voltage Vsub to make Vsub + ΔVsub, so that the above-mentioned concave portion disappears and the accumulated electric charge is discharged to the lower part.

Incidentally, the P layer is disposed below the N-type V-CCD in order to prevent unnecessary charges having a long recombination time from escaping from below and entering the vertical transfer register 38 as noise when unnecessary charges are discharged.

FIG. 8 is a timing chart showing changes in the Vsub potential during the high-speed shutter operation described above. First,
Within one field period of the standard television signal,
As shown in (c), the Vsub potential is changed in a pulsed manner at predetermined time intervals, and the discharge of the unnecessary charges accumulated in the sensor unit 61 is repeated until a predetermined time. Thereafter, in the latter half of one field period (one screen), photocharge accumulation is performed for the time t, and the accumulated charge indicated as “HIGH” is read out as image information by the read pulse shown in FIG. In this way, a high-speed shutter effect for accumulating the time t is obtained. The difference in the state of charge accumulation is shown in (a), with "NORMAL" during normal accumulation and "HIGH" during high-speed shutter operation.

The characteristic operation of this embodiment will be described below based on the basic operation of the CCD described above.

It has already been described that the shape of the concave portion of the potential potential can be changed by changing the substrate voltage Vsub in the VOD type CCD. Thus, the amount of photocharges that can be accumulated in the concave portions can be controlled. Tenth
The figure shows how a plurality of levels of the Vsub potential are set and selectively controlled. The concave portion of the potential potential is set deeper in LEVEL2 than in LEVEL1, so that more charges can be stored than in LEVEL1.

Conceptually, the ease of charge overflow is controlled by controlling the Vsub potential, and the black circle in the figure becomes the difference between LEVEL1 and LEVEL2, indicating that overflow is occurring. The storable amount is LE
In VEL1, the charge is indicated by a white circle, and in LEVEL2, the sum of the charge indicated by a white circle and the charge indicated by a black circle is used. Thus, Vsub
By controlling the potential, the saturation charge level (the amount of storable charge) can be intentionally made variable.

An example in which the above-described Vsub potential is controlled and the state of LEVEL1 and the state of LEVEL2 are switched at a predetermined timing within one screen is shown in a timing chart of FIG.

After discharging unnecessary charges, in the LEVEL1 state, photocharge accumulation is performed for the time t1 up to the saturation charge level of MAX, and then in the LEVEL2 state, photocharge accumulation is performed for the time t2 up to the saturation charge level of the MAX level raised state. ,read out. here,
Is the saturated charge level (the amount of storable charge) when the Vsub potential is in the LEVEL1 state, and is the saturated charge level when the Vsub potential is in the LEVEL2 state.

According to the above-described operation, the charge exceeding the saturation charge level overflows during the first time t1 in one screen, and the charge exceeds the saturation charge level.
11 Like the characteristics of the flat part indicated by the solid line in FIG.
Clipping occurs in the high luminance area. For example, assuming that the saturation charge level is 100%, all object light having a luminance higher than this is clipped. Next, the control is switched in the latter half of one screen, the Vsub potential is set to the LEVEL2 state, the saturation charge level is also raised, and the accumulation of the photocharge is continued for the time t2 with the accumulation margin.

Here, for example, as an NTSC television signal,
If t1 + t2 = 1/60 second, t1 = 1/6 if t2 = 1/500 second
It can be regarded as 0−1 / 500 ≒ 1/60 seconds.

This is usually 1/60 second, 1/125 second, 1
Considering that the steps of / 250 sec. And 1/500 sec. Correspond to one step (one stop) in terms of aperture, respectively, the subject with the proper exposure at t1 and the subject with high brightness by 3 stops Indicates that the proper exposure is given by t2.

Here, it is assumed that there is a luminance difference of three stops between the main subject and the background in the backlight state. During time t1, a screen corresponding to the BLC-ON state (backlight correction state) of the conventional example is captured, the main subject is properly exposed, and the background portion is clipped due to high luminance. In the exposure during t2 hours, an image corresponding to the BLC-OFF state (state in which backlight is not corrected) of the conventional example is imaged, and the background portion brighter by three stops than the main subject is properly exposed, and almost all main subjects are not exposed. No image is captured. The charge stored during the time t1 and the charge stored during the time t2 are added in the photocharge storage unit of each pixel, and as a result, a screen as shown in FIG. 2B is obtained.

Next, FIG. 9 shows a configuration example of the controller 3 described in FIG. As described above, the controller 3
Is supplied with, for example, luminance histogram information from the image analysis circuit 2, and in addition, controls the driving of the image sensor 1 in accordance with the mode state selected by the mode switching switch 5.

In response to the selection signal of the mode switching switch 5, the control mode switch 301 changes the drive mode of the controller 3 to N.
Set to one of ORMAL-MODE (normal shooting mode) and BLC-ON-MODE (backlight correction mode).

In the case of NORMAL-MODE (normal shooting mode), the LE potential in FIG. 10 is set so that the Vsub potential (substrate voltage value) becomes the maximum value of the saturation charge level determined by the performance of the image sensor 1 itself.
Set to the state shown in VEL2.

In the case of BLC-ON-MODE (backlight compensation mode), Vsub
Is switched between LEVEL1 and LEVEL2 in FIG. 10 during shooting of one screen. Here, the setting of the switching timing (t1 and t2) and the setting of the saturated charge level of LEVEL1 are performed by the timing control circuit 303 and the level control circuit 302, respectively, based on the image analysis result such as the above-mentioned luminance histogram. Will be As shown in FIG. 12, the overall photoelectric conversion characteristic that changes according to the setting of the level and the timing is a non-linear characteristic. The state changed by the level setting is indicated by a white arrow, and the state changed by the timing setting is indicated by a black arrow.

Next, the relationship between the backlight correction effect and the image analysis according to the present embodiment will be described using FIG. 13 showing an example of a luminance histogram.

An automatic exposure adjusting mechanism of a video camera or the like usually controls an iris or the like so that an integrated value of a luminance signal for one screen of a video signal approaches a predetermined value. Therefore, as shown in curves a and b of the luminance histogram shown in FIG.
Even when the brightness distribution in the screen is different, if the integrated value of the luminance signal for one screen is the same, the iris 11 is controlled in the same manner. Curve A is a luminance occurrence frequency distribution pattern that tends to occur during backlit shooting, and curve B is a luminance occurrence frequency distribution pattern that tends to occur during forward light shooting.

The BLC is switched by the mode switch 5 for backlight compensation.
When -ON-MODE (backlight correction mode) is selected, the BLC switch 19 is turned on, and the curve A changes to a curve C as shown in (b). Since the imaging signal that has been “blackened” has been raised to a value close to the above-mentioned predetermined value (indicated by a dashed line), which is a guideline for proper exposure, the imaging signal in the bright part causes high-brightness clipping. The frequency is a pattern biased toward the bright part.

Next, the overexposed portion of the curve C is converted to an appropriate exposure state by the non-linear processing of FIG. As a result, the shaded portion in (b) is converted into a shaded portion of the bright portion in the curve (b) as in the dead zone, and as a result, an image close to a normal light state is obtained.

Therefore, utilizing the features of image analysis as described above,
By appropriately performing predetermined arithmetic processing in the timing control circuit 303 and the level control circuit 302, it is possible to set the timing of switching between LEVEL1 and LEVEL2 and the saturation charge level of LEVEL1.

After the backlight correction, discontinuity of the luminance level or level inversion may possibly occur partially, but this does not pose a problem at the time of backlight because the probability of occurrence is extremely low.

Next, how to set the potential level of LEVEL1 (setting of the saturation level of the electric charge) will be described.

Fig. 14 shows the accumulation time for each intensity of incident light (25%, 50%, 100%) when the potential potential is changed and the saturation level of the charge, which indicates the likelihood of photocharge overflow, is set in three different ways. The relationship between t (horizontal axis) and the accumulated charge amount (vertical axis) is shown.

When the potential is set so that the saturation level is the middle clip point of the output signal, the incident light intensity is 25%
Light reaches the saturation level at t = t1,
In the period t2, the light with the incident light intensity of 800% is just t = t
It was assumed that the maximum saturation level was reached at 2. Similarly, at level ', 50% and 400% light combinations saturate at t1 and t2, respectively, and at level', 100% and 200%
Are saturated at t1 and t2, respectively, and the photoelectric conversion characteristics at this time are shown in FIG.

In FIG. 15, the one-dot chain line -a shows the characteristics when the incident light intensity is 25%, and the dotted line -b shows the characteristics when the incident light intensity is 800%. The solid line shows the characteristic obtained by combining the two.

Thus, by changing the potential,
It can be seen that various photoelectric conversion characteristics can be obtained.

Appropriate photoelectric conversion can be performed by controlling the above-described setting of the potential potential using a captured video signal.

FIGS. 16 (a) and 16 (b) are conceptual diagrams of an example in which a video signal is subjected to histogram analysis by the image analysis circuit 2. FIG. (A) shows the appearance frequency distribution of the luminance level Y, and (b) shows the cumulative frequency graph. Various methods can be conceived for associating the above-described potential potential setting with the imaging screen, and it is thought that each method has a certain advantage, and an example thereof will be described below.

The dotted lines (a) in (a) and (b) of FIG. 16 show that the appearance frequency is concentrated on the dark part, and is concentrated on the bright part, especially the MAX level which seems to be clipping.
In such a case, it is appropriate to set the potential potential to the level shown in FIG. 14 and to exert a large effect of compressing the luminance difference in one screen.

The solid line (b) in FIG. 16 indicates the frequency of appearance of bright portions that concentrate on the intermediate luminance level and do not cause large clipping. In such a case, on the contrary, it is appropriate to set the potential potential to the level 'in FIG. 14 so that the compression of the luminance difference is made relatively low and the halftone is easily reproduced.

Such a determination is made by setting a threshold (solid line c) for the cumulative frequency shown in FIG. 16 (b), for example, by classifying the luminance level when the cumulative frequency reaches 50% (shown by a dash-dot line, ' ,
In other words, when the point where the luminance level reaches the threshold value is within the range, the potential potential is set to the level, and when the point is within the range, the level is set to the level. In this case, it is sufficient to set the potential as "level". Of course, computer determination using a more complicated determination rule is also possible.

As explained above, the VOD type C that is currently becoming mainstream
By adding a simple control processing circuit to a CD, it is possible to effectively perform backlight correction, which is a major problem when shooting a video camera.

The background portion, which flew white during conventional correction and could not be reproduced, can be imaged simultaneously with the main subject, and this effect is enormous.

Also, if a finer gradation expression is desired in a high luminance area in a wide range of gradations, the potential level of LEVEL1 can be set to be more easily overflown by automatic or manual operation. It is possible.

Optimal photoelectric conversion characteristics are automatically set according to image information, so there is no need for the photographer to judge when backlighting is required, and it is not necessary to press the backlight button or set backlight compensation while looking at the small electronic viewfinder EVF. Was. Therefore, there is an effect that a video camera which has a wide gradation expression, is easy to operate, has few shooting errors, and can be realized.

In addition, the present invention can be applied to a modification or a modification of the above embodiment without departing from the gist thereof.

[Effects of the Invention] As described above, in the imaging apparatus having the gradation control function of the present invention, during the unit light accumulation period, the amount of storable charge is set to a small value to perform exposure suitable for a low-luminance subject. The operation to be performed and the operation to set a large amount of storable charge to perform exposure suitable for a high-brightness subject are respectively performed. By setting according to the characteristics,
There is an effect that good exposure can be realized for both a low-luminance subject and a high-luminance subject in the screen.

[Brief description of the drawings]

FIG. 1 is a block diagram showing a configuration of an embodiment of the present invention, FIGS. 2 (a) and 2 (b) are diagrams showing an example of imaging in a backlight state, and FIG. 3 is a configuration example of a VOD type CCD. Fig. 4 is a conceptual diagram of an interline transfer type CCD, Fig. 5 is a diagram showing a high-speed shutter operation, Fig. 6 is a conceptual diagram of a frame interline transfer type CCD, and Fig. 7 is a VOD. FIG. 8 is a timing chart of the vertical reset operation of the VOD type CCD, FIG. 9 is a detailed configuration diagram of the controller, FIG. 10 is a conceptual diagram of the potential level control, FIG. The figure shows the timing chart of the potential level control, FIG. 12 shows the non-linear characteristics of the photoelectric conversion, FIGS. 13 (a) and (b) show examples of the luminance histogram, and FIG. 14 shows the accumulation. Figure 15 shows the relationship between time and the amount of accumulated charge. 16 (a) and 16 (b) are conceptual diagrams of an example of a histogram analysis of a video signal, FIG. 17 is a configuration diagram of a conventional backlight correction circuit, and FIG. It is a conceptual diagram showing the effect of the conventional backlight correction. In the figure, 5: mode switching switch, 10: optical system,
11 ... Iris, 15 ... Adder, 17 ... ig meter, 19 ...
… It is a BLC switch.

Claims (2)

(57) [Claims] 1. A unit for forming one screen, comprising: sensing means for receiving a light signal, performing photoelectric conversion and accumulating information, and control means for changing a storable charge amount of the sensing means. The control means changes the storable charge amount into a plurality of stages during a light accumulation period, and controls the levels of the plurality of stages to be set in accordance with the distribution characteristics of luminance in image information. An imaging device having a gradation control function characterized by the following. 2. The gradation control function according to claim 1, wherein the storable charge amount is changed so as to be smaller at the front part and larger at the rear part during the unit light accumulation period. Imaging device having the same.