JPH11298801A - Image pickup device - Google Patents

Abstract

PROBLEM TO BE SOLVED: To prevent light receiving elements of CCD image sensor having a function of an electronic shutter from being saturated and to avoid disturbance in a moving image. SOLUTION: A CCD image sensor 3 has a function of an electronic shutter, a saturation detection section 7 detects saturation of each of light receiving elements of the CCD image sensor 3 and a noise reducer section 11 averages image pickup signals in each field or frame. When the saturation detection section 7 detects saturation of the CCD image sensor 3, a microcomputer a accelerates a shutter speed of an electronic shutter and increases the cyclic amount of the noise reducer section 11. Thus, the saturation of the light receiving elements of the CCD image sensor 3 is prevented and is disturbance in a moving image is avoided.

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to an image pickup apparatus suitable for use in a video camera apparatus, a still camera apparatus, and the like for picking up an image of a subject using a solid-state image pickup device. The present invention relates to an imaging apparatus that controls dynamic resolution while preventing saturation of elements to eliminate unnaturalness of a moving image.

[0002]

2. Description of the Related Art In today's video camera devices, a solid-state image pickup unit (a so-called CCD image sensor) formed by a plurality of light receiving elements as an image pickup means can be made smaller, lighter and lower in cost than an image pickup tube. Is provided. The CCD image sensor includes a frame transfer type CCD image sensor (FTCCD) and a frame interline type CCD image sensor (FITCC).
D), interline CCD image sensor (ILC
CD), all pixel readout type CCD image sensor and the like are known.

[0003] Among them, for example, an interline type CCD image sensor (ILCCD) is configured by alternately arranging light receiving elements and vertical transfer paths two-dimensionally.
The interline CCD image sensor is provided with two operation modes, a frame accumulation mode and a field accumulation mode. In the frame accumulation mode, only the charges of the light receiving elements in the odd lines are read out in the odd field, and only the charges of the light receiving elements in the even line are read out in the even field. In the field accumulation mode, the charges of all the light receiving elements are read out at both the odd field and the even field. In the odd field, each of the nth and n + 1th lines, the n + 2th line and the n + 3th line,. The electric charges are added and output in the vertical transfer path, and in the case of an even field, the combination of the electric charges to be added is changed. Each of them is added in the transfer path and output.

In the frame accumulation mode, the charge is read out from each light receiving element for each frame, so that the dynamic resolution is reduced. On the other hand, the charge from the light receiving element of each line is output without addition processing. Therefore, a high vertical resolution can be obtained. Conversely, in the field accumulation mode, the vertical resolution is reduced because the charges from the light receiving elements of each line are added and output, but the charges are read out from all the light receiving elements during one field. Therefore, a high dynamic resolution can be obtained.

[0005] In each of these imaging modes, for example, an image is captured in a frame accumulation mode when an image of a subject with a small movement is taken, and an image is captured in a field accumulation mode when an object with a lot of movement is taken. They are used depending on the case.

On the other hand, in the all-pixel readout CCD image sensor, the light receiving elements and the vertical transfer paths are two-dimensionally arranged alternately, and the charges vertically transferred from the light receiving elements on the odd lines via the vertical transfer paths. And a second horizontal transfer path for horizontally transferring the charges vertically transferred from the light receiving elements of the even lines via the vertical transfer paths (so-called dual transfer paths). Horizontal transfer path structure).

This all-pixel readout type CCD image sensor reads out electric charges from all the light receiving elements for each field, and charges read out from the light receiving elements on the odd numbered lines via the first horizontal transfer path. The electric charge read from the light receiving element of the line is output via the second horizontal transfer path. As a result, the reading of the charges of all the light receiving elements is completed for each field, and the charges read from the light receiving elements of each line are horizontally transferred without being added to the charges of the odd lines and the charges of the even lines. Therefore, both high vertical resolution and dynamic resolution can be obtained.

[0008] Here, the dynamic range of the imaging signal output from the CCD image sensor depends on the amount of imaging light that can be received by the CCD image sensor. That is, if the charge that can be accumulated in each light receiving element of the CCD image sensor is infinite, a large amount of imaging light can be taken in, and the dynamic range of the imaging signal can be expanded. However, in practice, the charge storage capacity of each light receiving element of the CCD image sensor is finite, and when receiving an amount of imaging light exceeding this finite capacity, each light receiving element is saturated within the charge storage period, and This causes a problem that blooming, smear and the like occur.

On the other hand, by controlling the shutter speed of the electronic shutter, which is a speed corresponding to the charge accumulation time of each light receiving element in the CCD image sensor, an arbitrary shutter speed can be realized in the same manner as a mechanical shutter. An imaging device has conventionally existed. In a CCD image sensor, light is usually converted into an electric signal to accumulate electric charges at a period of 1/60 second, and the electric charge accumulation time, that is, the shutter speed is reduced by, for example, 1/60 using this electronic shutter.
By setting arbitrarily to 1000 seconds, 1/2000 seconds, or the like, it is possible to improve the dynamic resolution of a captured image.

FIG. 2 shows a schematic configuration example of a conventional imaging apparatus. The conventional image pickup apparatus shown in FIG. 2 has a CCD image sensor 53 having an electronic shutter function, and has a configuration in which a mechanical shutter is not required by controlling the shutter speed of the electronic shutter. I have.

More specifically, the lens system 51 is such that the focus and the like are driven and controlled by a mechanical drive unit 60, and the iris (aperture) unit 52 is similarly controlled by the mechanical drive unit 60 to control the aperture amount. Is what is done. Light from the subject through the lens system 51 and the iris unit 52 is transmitted to a CC having a plurality of light receiving elements constituting a plurality of pixels.
The light enters the D image sensor 53.

The CCD image sensor 53 has a function as an electronic shutter. By setting a CCD drive pulse from the CCD drive unit 59 arbitrarily,
The shutter speed of the electronic shutter can be arbitrarily set. The imaging signal output from each light receiving element of the CCD image sensor 53 is correlated double sampling (CD
S) and an automatic gain control (AGC) unit 54.

The correlated double sampling and automatic gain control section 54 performs so-called correlated double sampling and automatic gain control on the image pickup signal from each light receiving element of the CCD image sensor 53. The image signal subjected to the correlated double sampling and the automatic gain control is sent to an analog / digital conversion (ADC) unit 55.

The analog / digital converter 55 converts an image signal into a digital signal. The digitally converted image signal is sent to the signal processing unit 56.

The signal processing section 56 performs a predetermined video processing process or the like on the image pickup signal from the analog / digital conversion section 55 to generate an image signal. This signal processing unit 5
6 is output from an output terminal 62.

When the signal level of the CCD image sensor 53 is controlled by an electronic shutter with the iris unit 52 fixed, the image signal supplied to the signal processing unit 56 is sent to a saturation detecting unit 57. The saturation detector 57 detects the saturation of the image signal from each light receiving element of the CCD image sensor 53 by comparing a predetermined threshold value set in advance with the level of the image signal. For example, when the level of the imaging signal of a certain specific pixel or more becomes larger than the threshold value, it is detected that the CCD image sensor 53 is saturated. The saturation detection signal from the saturation detection section 57 is sent to a microcomputer 58.

When a saturation detection signal is supplied from the saturation detection section 57, the microcomputer 58 controls the lens system 5
1 and the iris unit 52 to drive the CCD image sensor 5
The CCD drive unit 59 is controlled so that the shutter speed of the electronic shutter of the CCD image sensor 53 is increased in preference to controlling the amount of light incident on the CCD 3. Thus, saturation of each light receiving element of the CCD image sensor 53 is prevented.

[0018]

As described above, when the shutter speed of the electronic shutter is increased for the purpose of preventing the saturation of the CCD image sensor, the dynamic resolution of the image signal output from the CCD image sensor is increased. Become. That is, in the above-described conventional imaging apparatus, when the CCD image sensor is saturated, the shutter speed of the electronic shutter increases, contrary to the photographer's intention, and as a result, the dynamic resolution increases, and the captured moving image is disturbed ( Moving images become discontinuous and lose smoothness).

SUMMARY OF THE INVENTION The present invention has been made in view of the above-mentioned problems, and has been made in consideration of the above circumstances. The purpose is to provide the device.

[0020]

An image pickup apparatus according to the present invention comprises a plurality of light receiving elements corresponding to a plurality of pixels, wherein each of the light receiving elements photoelectrically converts the image pickup light. And outputting the accumulated signal charges as an imaging signal, a solid-state imaging means capable of arbitrarily setting a charge accumulation time, a saturation detection means for detecting saturation of each light receiving element of the solid-state imaging means, An averaging unit for averaging the imaging signal; and a control unit for controlling a charge accumulation time of the solid-state imaging unit and the averaging parameter based on a saturation detection output of the saturation detection unit.

The averaging means comprises a feedback filter for averaging the image signal by feeding back the image signal for each field or frame, and the averaging parameter is a value representing a feedback amount of the feedback filter.

Further, the control means controls the charge storage time of the solid-state imaging means to change for each field or frame, and the solid-state imaging means reads the electric charges read out a plurality of times in one field period or one frame period. Add and output.

[0023]

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS Preferred embodiments of the present invention will be described below in detail with reference to the drawings.

The imaging apparatus according to the present invention can be applied to, for example, a video camera apparatus or an electronic still camera apparatus. FIG. 1 shows the function of an electronic shutter as one embodiment of the imaging apparatus according to the present invention. A video camera apparatus having a CCD image sensor 3 provided and controlling the shutter speed of the electronic shutter to prevent each light receiving element of the CCD image sensor 3 from being saturated will be described as an example.

In FIG. 1, an image signal recording / reproducing unit usually provided in a video camera device, an image picked up by the CCD image sensor 3 and an image reproduced by the recording / reproducing unit are displayed. For example, a display unit such as an electronic viewfinder and a small liquid crystal monitor is not shown.

In the video camera apparatus according to the embodiment of the present invention shown in FIG. 1, the lens system 1 is driven and controlled by a mechanical system drive unit 10, and the iris (aperture) unit 2 is similarly controlled. The aperture amount is driven and controlled by the mechanical drive unit 10. Lens system 1
The light from the subject or the like whose light amount is adjusted by the iris unit 2 (including adjustment of the depth of field) by the iris unit 2 is a CCD image sensor having a plurality of light receiving elements corresponding to the pixels constituting the image. No. 3 is incident on the light receiving surface.

Although not shown in FIG. 1, the arrangement positions of magenta (Mg) and green (G) are arranged in front of the CCD image sensor 3, for example, for each odd line of the CCD image sensor 3. Inverted so-called complementary color M
A g-G inversion type color filter array is provided. Therefore, magenta (Mg) and green (G) imaging light is incident on each light receiving element of an odd line of the CCD image sensor 3, and the CCD image sensor The imaging light of cyan (Cy) and yellow (Ye) is incident on each light receiving element of the even-numbered line of No. 3. However, this color filter array is an example, and other than this, for example, a complementary color / Mg
A -G sequential color filter array, an RGB color filter array called a primary color / Bayer array type, a white (W), G, Cy, Ye color filter array, or the like may be provided.

The CCD image sensor 3 has a function as an electronic shutter, and the shutter speed of the electronic shutter can be arbitrarily set by arbitrarily setting the CCD drive pulse from the CCD drive unit 9. Things. The imaging signal output from each light receiving element of the CCD image sensor 3 is sent to a correlated double sampling (CDS) and automatic gain control (AGC) unit 4.

The correlated double sampling and automatic gain control section 4 performs so-called correlated double sampling and automatic gain control on the image pickup signal from each light receiving element of the CCD image sensor 3. The image signal subjected to the correlated double sampling and the automatic gain control is sent to an analog / digital conversion (ADC) unit 5.

The analog / digital converter 5 converts an image signal into a digital signal. The digitally converted image signal is sent to the signal processing unit 6.

The signal processing unit 6 is composed of, for example, a digital signal processor (DSP), and performs a predetermined video process process or the like on the image pickup signal from the analog / digital conversion unit 5 to generate an image signal. That is, the signal processing unit 6 performs color separation (RGB) based on the image signal obtained by the complementary color filter array, and performs a predetermined video process process or the like on the image signal. The image signal from the signal processing unit 6 is a noise reducer (N
R) sent to the unit 11.

The noise reducer 11 is a noise reducer having a recursive filter (feedback filter) configuration as shown in FIG. 1, and has a main configuration of multiplying the image signal stored in the memory 24 by a multiplication coefficient K. Multiplier 23
And a multiplication coefficient (1-K) with the image signal from the signal processing unit 6
And an adder 2 that adds the image signal multiplied by the multiplication coefficient (1-K) by the multiplier 21 and the image signal multiplied by the multiplication coefficient K by the multiplier 23.
The memory 24 stores the image signal output from the adder 22. Note that the multiplication coefficient K
Is a parameter that takes a value of 0 <K <1 and represents a recursive amount of the recursive filter (a feedback amount of the feedback filter).
The image signal from the noise reducer 11 is output from the output terminal 12 and sent to a recording system (not shown) for recording, or sent to a display system (not shown) for display.

The image signal supplied to the signal processing unit 6 is also sent to the saturation detection unit 7 via the signal processing unit 6. The saturation detector 7 includes a comparison circuit (not shown) that compares a predetermined threshold value set in advance with the level of the image signal, and compares the predetermined threshold value with the level of the image signal. By doing so, it is detected whether or not each light receiving element of the CCD image sensor 3 is saturated. That is, the saturation detector 7 detects that the CCD image sensor 3 is saturated, for example, when the level of the imaging signal from each light receiving element corresponding to a certain pixel or more exceeds a threshold. The saturation detection signal from the saturation detector 7 is sent to a microcomputer 8.

The microcomputer 8 includes a saturation detector 7
When a saturation detection signal is supplied from the CPU, the CCD driving unit 9 is controlled in preference to controlling the amount of light incident on the CCD image sensor 3 by driving the lens system 1 and the iris unit 2 to control the CCD image sensor. Control is performed to increase the shutter speed of the third electronic shutter. Thereby, in the video camera device of the present embodiment, the CCD image sensor 3
Of each light receiving element can be prevented. The lens system 1 and the iris unit 2 are driven by the microcomputer 8 controlling the mechanical system driving unit 10 after controlling the shutter speed of the electronic shutter.

However, when the CCD image sensor 3 is saturated, if the shutter speed of the electronic shutter of the CCD image sensor 3 is controlled to be faster than the control of the amount of incident light by the lens system 1 and the iris unit 2, As described above, the dynamic resolution of the imaging signal output from the CCD image sensor 3 is increased, and as a result, the captured moving image is disturbed (the moving image becomes discontinuous and loses smoothness).

Therefore, in the video camera apparatus of the present embodiment, when the CCD image sensor 3 is saturated, the microcomputer 8 controls the electronic shutter of the CCD image sensor 3 to increase the shutter speed. At the same time, control is performed so that the value of the multiplication coefficient K in the coefficient multipliers 21 and 23 of the noise reducer unit 11 is increased (the amount of repetition is increased).

That is, the noise reducer 11 is a noise reducer having a recursive filter configuration as shown in FIG. 1 and normally removes a noise component contained in an image signal. When the noise reduction unit 11 is saturated, the microcomputer 8 controls the microcomputer 8 to increase the value of the multiplication coefficient K (increase the amount of repetition), thereby increasing the amount of repetition of the recursive filter and averaging the image. And, as a result,
The shutter speed of the electronic shutter is increased to reduce the disturbance of the image whose dynamic resolution has been increased.
It is possible to reduce the change in the image and reduce the unnaturalness).

Thus, in the video camera device of the present embodiment, it is possible to prevent saturation of each light receiving element of the CCD image sensor 3 and at the same time to eliminate disturbance of a moving image.

The CCD image sensor 3 used in the video camera apparatus of the present embodiment is an all-pixel readout CCD image sensor that reads out the electric charges accumulated in all the light receiving elements for each field at a time. This C
The CD image sensor 3 is capable of performing imaging in at least two imaging modes of a “normal imaging mode” and a “dynamic range expansion mode (D range expansion mode)” described below. In addition, the CCD image sensor 3
Can variably control how to read out the charges stored in the light receiving element in accordance with the imaging mode, and can change the shutter speed of the electronic shutter for each field or for each frame.

Each of these imaging modes can be selected, for example, by the user operating an imaging mode selection key provided on an operation panel (not shown). The imaging mode selected by the operation is controlled to be executed, and at the time of reproduction, reproduction control according to the imaging mode at the time of recording is performed.

Here, the NTSC system, which is one of the standard television systems, will be described as an example. In the normal imaging mode, the charges of the light receiving elements on the odd and even lines are divided at 1/60 second intervals. This is the reading mode. In this normal imaging mode, the reading of the charges of all the light receiving elements is completed for each field, and the charges read from the light receiving elements of each line are divided into odd-line charge and even-line charge without adding processing. Therefore, high vertical resolution and high dynamic resolution can be obtained.

On the other hand, in the dynamic range expansion mode, the charge of the odd-numbered lines and the total charge of the even-numbered lines of each light receiving element of the CCD image sensor 3 are successively read out, for example, twice in 1/60 second and added. In other words, this is an imaging mode in which the charges of the light receiving elements constituting the odd field and the even field are read out and added at, for example, every 1/120 second.

In the dynamic range expansion mode, the charge of the odd lines and the total charge of the even lines of each light receiving element of the CCD image sensor 3 are successively read out, for example, twice and added. An image signal with a wide dynamic range can be obtained.

In this dynamic range expansion mode, charges are read from each light receiving element of the CCD image sensor 3 twice during one field, so that even when the amount of imaging light becomes large, each charge is read out. The light receiving element can be hardly saturated.

That is, in this case, 2
Since the readout control is performed twice, even if the charge for the two times of readout is added, each light receiving element is set to two times more than when one readout control is performed in one field.
It can be hardly saturated. For this reason, it is possible to prevent the inconvenience of blooming, smear, and the like occurring in the captured image due to saturation of the light receiving element.

In other words, the imaging signals for two fields read from the CCD image sensor 3 during one field in the dynamic range expansion mode have the same imaging time (accumulation time). In addition, the dynamic range of the imaging signal can be expanded by adding the imaging signals, and even if the addition processing of the imaging signals for two fields is performed, the two have the same dynamic resolution. A good image can be obtained without causing any discomfort in the image of the even field.

In this example, the charge is read from each light receiving element twice during one field in the dynamic range expansion mode. However, this is performed three or four times. The read control may be performed an arbitrary number of times. This makes it possible to make each light receiving element of the CCD image sensor 3 less likely to be saturated three or four times than in the case of performing one read control in one field as in the normal imaging mode.
It can be more prominent.

In the video camera device according to the present embodiment, the above-described dynamic range expansion mode is used.
Further, in the dynamic range expansion mode, C
By changing the shutter speed of the electronic shutter of the CD image sensor 3 for each field or for each frame, the effect of preventing the saturation of the CCD image sensor 3 and the disturbance of the moving image by the noise reducer unit 11 can be improved as compared with the normal imaging mode. It is possible to make the suppression effect more remarkable.

That is, in the case of the dynamic range expansion mode, for example, as a first example, for example, at the time of the first reading of the odd field, the accumulated charge is read from each odd line in 1/60 second. When reading the even field, the stored charge is read from each even line in 1/500 second, and in the second reading of the odd field, the stored charge is read from each odd line in 1/60 second. At the time of reading, the accumulated charge is read from each even line in 1/500 second, and the reading operation is repeated in the same manner as described above. By changing the shutter speed of the electronic shutter of the CCD image sensor 3 for each field, the CCD is read. Higher effect of preventing saturation of image sensor 3 and suppressing disturbance of moving image Rukoto is possible.

In the case of the dynamic range expansion mode, for example, as a second example, at the time of the first reading of the odd field, the accumulated charge is read from each odd line in 1/60 seconds, and the first even field is read. When reading the field, the accumulated charge is read from each even line in 1/60 second, and when reading the second odd field, the accumulated charge is read from each odd line in 1/500 second. At the time of reading, the accumulated charge is read from each even field in 1/500 second, and the reading operation is repeated in the same manner as described above. By changing the shutter speed of the electronic shutter of the CCD image sensor 3 for each frame, C
It is possible to enhance the effects of preventing saturation of the CD image sensor 3 and suppressing disturbance of a moving image.

When the shutter speed of the electronic shutter is changed for each field as in the first example, a field memory is used as the memory 24 in FIG. When the shutter speed of the electronic shutter is changed for each frame as in the second example, a frame memory is used as the memory 24 in FIG. Further, in the case of the dynamic range expansion mode as in the first and second examples, for example, a value of 1/2 is used as the multiplication coefficient K.

As is apparent from the above description, the video camera device according to the present embodiment can prevent the saturation of the CCD image sensor 3 without newly adding or changing the signal processing of the video camera device. In addition, it is possible to suppress an unnatural image due to an unnecessary increase in the dynamic resolution, and it is also possible to output an image enlarged to a dynamic range several times or tens of times the conventional dynamic range.

Further, in the video camera apparatus according to the present embodiment, the provision of the noise reducer 11 can improve the signal-to-noise ratio (S / N) of the image signal.

In the above-described embodiment, the image pickup apparatus according to the present invention is used for the NTS, which is a Japanese standard television system.
Although the present invention has been described as conforming to the C system, the present invention can also be applied to different television systems such as the PAL system and the SECAM system.

The CCD image sensor 3 is an all pixel readout type CCD image sensor.
This may be another CCD image sensor such as an interline CCD image sensor (ILCCD).

The present invention is not limited to the above-described embodiment described as an example, and various changes can be made in accordance with the design and the like without departing from the technical idea of the present invention. Of course, it is.

[0057]

According to the first aspect of the present invention, there is provided an imaging apparatus comprising: a solid-state imaging unit capable of arbitrarily setting a charge accumulation time;
Saturation detection means for detecting the saturation of each light receiving element of the solid-state imaging means; averaging means for averaging the imaging signal for each field or frame; and charge accumulation time and averaging based on the saturation detection output of the saturation detection means. By having the control means for controlling the parameters, it is possible to prevent saturation of the light receiving element and to eliminate disturbance of the moving image.

According to a second aspect of the present invention, there is provided an image pickup apparatus comprising a feedback filter for averaging by returning an image signal for each field or frame as averaging means.
By setting the averaging parameter to a value representing the feedback amount of the feedback filter, it is possible to eliminate the disturbance of the moving image and to improve the signal-to-noise ratio of the image signal.

According to the third aspect of the present invention, the solid-state imaging means controls the charge storage time to change for each field or frame, and reads out a plurality of times in one field period or one frame period. By adding and outputting the charges, the dynamic range can be expanded, the saturation of the light receiving element can be prevented, and the disturbance of the moving image can be eliminated.

[Brief description of the drawings]

FIG. 1 is a block diagram of a main part of a video camera device according to an embodiment to which an imaging device according to the present invention is applied.

FIG. 2 is a block diagram of a main part of a conventional video camera device.

[Explanation of symbols]

DESCRIPTION OF SYMBOLS 1 ... Lens system, 2 ... Iris, 3 ... CCD image sensor, 4 ... Correlated double sampling and automatic gain control unit (C
DS / AGC unit), 5 ... Digital / analog converter, 6
... Signal processing unit, 7 ... Saturation detection unit, 8 ... Microcomputer, 9 ... CCD drive unit, 10 ... Mechanical system drive unit, 11 ...
Noise reducer part, 21, 23 multiplier, 22 adder, 24 memory

Claims (3)

[Claims] 1. A plurality of light receiving elements corresponding to a plurality of pixels. Each light receiving element photoelectrically converts imaging light and outputs a signal charge stored as an imaging signal, and a charge storage time can be set arbitrarily. Solid-state imaging means, saturation detection means for detecting saturation of each light-receiving element of the solid-state imaging means, averaging means for averaging the imaging signal for each field or frame, and a saturation detection output of the saturation detection means. And a control means for controlling the charge accumulation time of the solid-state imaging means and the averaging parameter based on the control signal. 2. The averaging means comprises a feedback filter for averaging the image signal by feeding back the image signal for each field or frame, and the averaging parameter is a value representing a feedback amount of the feedback filter. The imaging device according to claim 1, wherein: 3. The control unit performs control to change the charge accumulation time of the solid-state imaging unit for each field or frame, and the solid-state imaging unit reads the electric charge read multiple times in one field period or one frame period. The imaging device according to claim 1, wherein the image is added and output.