JP2001352486A - Image pickup device - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide an image pickup device which obtains video signal with a wide dynamic range, less noise component and superior S/N performance, by applying frequency emphasis processing to a signal with a small charge storage amount, because the image with a small charge storage amount is composited with an image with large charge storage amount so that it does not deteriorate in the S/N, after the composited image due to the noise component much included in the image whose charge storage amount is small. SOLUTION: Compositing a 'long' signal which is am image signal with much charge storage quantity with a 'short' signal which is an image signal with less charge storage quantity extends the dynamic range. Through the provision of a 'Long' signal frequency emphasis means 8 and a 'Short' signal frequency emphasis means 9 or a Mix signal frequency emphasis means 16, noise is suppressed in the frequency emphasis processing with respect to the 'Short' signal to obtain the image with a wide dynamic range and excellent S/N performance having less noise component, before or after the compositing.

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to an image pickup apparatus configured to expand a dynamic range by synthesizing an image signal having a small charge accumulation amount and an image signal having a large charge accumulation amount. The present invention relates to a technique for improving the image quality by eliminating as much as possible the effects of noise components contained in an image signal having a small charge accumulation amount.

[0002]

2. Description of the Related Art Conventionally, as a solid-state imaging device configured to obtain a video signal having a wide dynamic range by synthesizing two image signals having different exposure amounts, for example, Japanese Patent Application Laid-Open No. 9-214829 and Kaihei 9-275
The one disclosed in Japanese Patent Publication No. 527 is known.

[0003] In Japanese Patent Application Laid-Open No. 9-214829, an image having a wide dynamic range is obtained by shifting a level of two continuous field images photographed with different exposure times and then synthesizing them into one frame image. A digital still camera that can be obtained is disclosed.

In Japanese Patent Application Laid-Open No. 9-275527, a plurality of frame images having different exposure times obtained from a plurality of CCDs (charge-coupled devices) are level-shifted, and then combined into one frame image. Accordingly, a digital still camera capable of obtaining an image with a wide dynamic range has been disclosed.

Another example of a video camera in which the dynamic range is expanded by using a special CCD capable of reading out a long-time exposure signal and a short-time exposure signal within one field period is known (Technology of the Image Media Society of Japan). Report Vol.22, No.
3, pp1-6 (1998) "Development of single-chip Hyper-D color camera signal processing method").

[0006]

However, in a conventional imaging apparatus for obtaining a video signal having a wide dynamic range, an image signal having a small charge accumulation amount and an image signal having a large charge accumulation amount photographed by changing the exposure time are used. However, there is a problem that the S / N of the synthesized image signal is deteriorated and the image quality is deteriorated due to noise components included in the image signal having a small amount of accumulated electric charge due to a large amount of noise components.

SUMMARY OF THE INVENTION The present invention has been made to solve the above-mentioned problems. In an image pickup apparatus for obtaining a video signal having a wide dynamic range, an image having a wide dynamic range with a small noise component and excellent S / N performance is obtained. The purpose is to gain.

[0008]

According to the present invention, there is provided an imaging apparatus configured to expand a dynamic range by synthesizing an image signal having a small charge accumulation amount and an image signal having a large charge accumulation amount. The object is achieved by making the frequency enhancement processing for an image signal with a small accumulation amount different from the frequency enhancement processing for the image signal with a large charge accumulation amount.

That is, in the frequency emphasizing process, an image signal with a small charge accumulation amount is set to suppress a noise component, so that an image having a wide dynamic range and excellent S / N performance with a small noise component is provided. Can be obtained.

In the above description, two types of image signals are described as image signals having different charge accumulation amounts. However, this is for convenience of explanation, and it is not necessary to be limited to the three types. The above-described arbitrary number of image signals may be targeted.

[0011]

BEST MODE FOR CARRYING OUT THE INVENTION Hereinafter, embodiments of the present invention will be generally described.

An image pickup apparatus according to a first aspect of the present invention is an image pickup apparatus for expanding a dynamic range by combining an image signal having a small charge accumulation amount and an image signal having a large charge accumulation amount. It is characterized in that the frequency emphasis processing for the small image signal and the image signal with the large charge accumulation amount are different from each other.

The operation of the first invention is as follows. In the frequency enhancement processing for an image signal with a small charge accumulation amount and the frequency enhancement processing for an image signal with a large charge accumulation amount, noise is reduced by setting a noise component for an image signal with a small charge accumulation amount. It is possible to obtain an image with a wide dynamic range while generating a composite image signal having a small component and excellent S / N performance.

The imaging apparatus according to the second aspect of the present invention includes the above-described first aspect.
In the invention, the emphasis of the set frequency component of the image signal having the small charge accumulation amount is weakened, and the emphasis of the set frequency component of the image signal having the large charge accumulation amount is increased.

This second invention corresponds to the above-described first invention described on a more specific level, and is used as a frequency emphasis process for an image signal which tends to include a large amount of noise components due to a small charge accumulation amount. For example, by weakening the emphasis by lowering the frequency band or lowering the multiplication gain, it is possible to obtain an image having a small noise component and a wide dynamic range excellent in S / N performance as in the first aspect. Becomes possible.

The imaging apparatus according to the third invention of the present application is the imaging apparatus according to the first aspect.
The second invention is characterized in that the image signal having the small charge accumulation amount and the image signal having the large charge accumulation amount are synthesized after the frequency emphasizing process. this is,
This describes the temporal sequence of the synthesis of a plurality of types of image signals having different charge accumulation amounts and the frequency enhancement processing. In the third invention, the frequency enhancement processing is executed first, and the frequency enhancement processing is performed. Is to be synthesized with respect to the image signal that has been subjected to.

The imaging apparatus according to a fourth aspect of the present invention is the imaging apparatus according to the first aspect.
In the second invention, the image signal having a small charge accumulation amount and the image signal having the large charge accumulation amount are combined before the frequency emphasizing process. this is,
This describes the temporal sequence of the synthesis of a plurality of types of image signals having different charge accumulation amounts and the frequency enhancement process. In the fourth invention, the synthesis of the plurality of types of image signals is performed first, The frequency enhancement processing is performed on the synthesized image signal.

A preferred method of synthesizing is to multiply an image signal with a small charge accumulation by a coefficient exceeding 1, and to combine the result with an image signal with a large charge accumulation. Since the image signal having a smaller charge accumulation amount by multiplying the coefficient contains more noise components,
Performing a frequency emphasis process for suppressing noise components before the synthesis can improve the S / N performance of the synthesized image signal. The third invention is
This is insisted, and the fourth invention expresses that this is not the case.

The imaging apparatus according to a fifth aspect of the present invention is the imaging apparatus according to the first aspect.
In the fourth to fourth aspects, a driving method for acquiring the image signal with a small charge accumulation amount and a driving method for acquiring an image signal with a large charge accumulation amount are different from each other.

The operation of the fifth invention is as follows. For example, it is conceivable to generate an image signal with a small amount of charge accumulation by the field readout driving method and generate an image signal with a large amount of charge accumulation by the frame readout driving method. It is possible to obtain an image with a small dynamic component and excellent in S / N performance and a wide dynamic range.

The imaging apparatus according to the sixth aspect of the present invention includes the fifth aspect described above.
The invention is characterized in that, for the image signals having the different charge accumulation amounts different from each other, the signal formats are unified before synthesizing the image signals.

The operation of the sixth invention is as follows. For example, the signal format is different between the image signal having a small charge accumulation amount by the field readout driving method and the image signal having a large charge accumulation amount by the frame readout driving method in the above case. Therefore, before combining, for example, the signal format is unified with respect to the image signal having a large charge accumulation amount of the frame readout driving method by interpolating the image signal having a small charge accumulation amount in the field readout driving method. In other words, it is possible to satisfactorily combine the two image signals without any trouble.

According to a seventh aspect of the present invention, there is provided an image pickup device, an image pickup device, charge storage amount control means for controlling the charge storage amount of the image pickup device, and a plurality of image signals having different charge storage amounts by the charge storage amount control means. And an image synthesizing unit for synthesizing an output image signal of the frequency emphasizing unit to generate a synthesized image signal. In addition, the frequency emphasizing means is configured such that the frequency emphasizing process for an image signal having a small charge accumulation amount and the frequency emphasizing process for an image signal having a large charge accumulation amount are different from each other.

The seventh invention corresponds to the first invention described at a more specific level. This seventh
The operation according to the invention is as follows. In the frequency emphasis processing for the image signal with a small charge accumulation amount and the frequency emphasis processing for the image signal with a small charge accumulation amount, the frequency emphasis means sets the noise component to be suppressed for the image signal with the small charge accumulation amount. By doing so, it is possible to obtain an image with a wide dynamic range while generating a composite image signal with a small noise component and excellent S / N performance. In addition, the setting for suppressing the noise component for an image signal having a small charge accumulation amount is performed not before the synthesis of the two image signals but before the synthesis of the two image signals. At S /
It is possible to improve N performance.

According to an eighth aspect of the present invention, there is provided an image pickup device, an image pickup device, charge storage amount control means for controlling the charge storage amount of the image pickup device, and a plurality of image signals having different charge storage amounts by the charge storage amount control means. And an image synthesizing unit that synthesizes an output image signal of the imaging unit to generate a synthesized image signal; and a frequency emphasis unit that emphasizes a set frequency component of the synthesized image signal. The emphasizing means is configured such that the frequency emphasizing process for the image signal portion with a small charge accumulation amount and the frequency emphasizing process for the image signal portion with a large charge accumulation amount in the composite image signal are different from each other. is there.

The eighth invention also corresponds to the first invention described at a more specific level. This 8th
The operation according to the invention is as follows. In the frequency enhancement processing for an image signal with a small charge accumulation amount and the frequency enhancement processing for an image signal with a large charge accumulation amount, noise is reduced by setting a noise component for an image signal with a small charge accumulation amount. S / N with few components
It is possible to obtain an image with a wide dynamic range while generating a composite image signal with excellent performance. In addition, in order to perform the setting for suppressing the noise component for an image signal having a small charge accumulation amount, the setting for suppressing the noise component is performed after the two image signals are combined. However, the present invention is not limited to this. Has expressed.

According to a ninth aspect of the present invention, there is provided an image pickup device, an electric charge accumulation amount control means for controlling an electric charge accumulation amount of the image pickup device, and an image pickup device driving of the image pickup device having at least two different driving systems. A control unit, an imaging unit that obtains a plurality of image signals having different charge storage amounts by the charge storage amount control unit and different driving methods by the imaging device drive control unit, and an output image signal of the imaging unit. A frequency emphasizing unit for emphasizing a set frequency component; and an image synthesizing unit for synthesizing an output image signal of the frequency emphasizing unit to generate a synthesized image signal. The frequency emphasis processing for the signal and the frequency emphasis processing for the image signal with a large charge accumulation amount are configured to be different from each other. That.

The ninth invention corresponds to the fifth invention described at a more specific level. This ninth
The operation according to the invention is as follows. For example, it is conceivable to generate an image signal with a small amount of charge accumulation by the field readout driving method and generate an image signal with a large amount of charge accumulation by the frame readout driving method. S /
It is possible to obtain an image with excellent N performance and a wide dynamic range.

According to a tenth aspect of the present invention, there is provided an image pickup device, an electric charge storage amount control means for controlling the amount of electric charge stored in the image pickup device, and an image pickup device driving of the image pickup device having at least two different driving systems. A control unit, an imaging unit that obtains a plurality of image signals of different driving modes by the imaging device drive control unit, and an output image signal of the imaging unit. Image synthesizing means for synthesizing to generate a synthesized image signal;
Frequency emphasizing means for emphasizing a set frequency component of the synthesized image signal, wherein the frequency emphasizing means performs a frequency emphasizing process on an image signal portion having a small charge accumulation amount in the synthesized image signal and a charge accumulating amount. The frequency emphasizing process for many image signal portions is different from each other.

The tenth invention also corresponds to a description of the fifth invention at a more specific level. The operation according to the tenth aspect is as follows. For example, it is conceivable to generate an image signal with a small amount of charge accumulation by the field readout driving method and generate an image signal with a large amount of charge accumulation by the frame readout driving method. It is possible to obtain an image with a small dynamic component and excellent in S / N performance and a wide dynamic range. In addition, the setting for suppressing the noise component for an image signal having a small charge accumulation amount is performed not before the synthesis of the two image signals but before the synthesis of the two image signals. , It is possible to improve the S / N performance.

[0031] In the imaging apparatus according to an eleventh aspect of the present invention, in the seventh and ninth aspects, the frequency emphasizing means weakens emphasis of a set frequency component of an image signal having a small charge accumulation amount, and reduces the charge accumulation amount. The configuration is such that the emphasis on the set frequency components of many image signals is enhanced.

The operation of the eleventh invention is as follows. As a frequency emphasis process for an image signal that tends to include more noise components due to a small charge accumulation amount,
For example, by weakening the emphasis by lowering the frequency band or lowering the multiplication gain, it is possible to obtain an image with a small dynamic component and excellent S / N performance and a wide dynamic range.

[0033] In the imaging apparatus according to a twelfth aspect of the present invention, in the eighth and tenth aspects, the frequency emphasizing means enhances a set frequency component of an image signal portion having a small charge accumulation amount in the composite image signal. , And the emphasis on the set frequency component of the image signal portion having a large charge accumulation amount is enhanced.

The operation of the twelfth invention is as follows. As a frequency emphasis process for an image signal that tends to include more noise components due to a small charge accumulation amount,
For example, by weakening the emphasis by lowering the frequency band or lowering the multiplication gain, it is possible to obtain an image with a small noise component and an excellent S / N performance and a wide dynamic range.

According to a thirteenth aspect of the present invention, in the imaging apparatus according to the eighth and tenth aspects, the frequency emphasizing means includes image synthesizing ratio information of a plurality of image signals having different charge accumulation amounts in the image synthesizing means. It is configured to weaken the set frequency component of the image signal portion having a small charge accumulation amount in the composite image signal and increase the emphasis of the set frequency component of the image signal portion having a large charge accumulation amount. is there.

The operation of the thirteenth aspect is as follows. The image synthesis ratio information enables image synthesis in a state where the signal range of an image signal having a small amount of charge storage and the signal range of an image signal having a large amount of charge storage are smoothly continued, and the optimal frequency Enhancement processing can be performed, and a sufficiently high-quality image with a small noise component and excellent S / N performance and a wide dynamic range can be obtained.

According to a fourteenth aspect of the present invention, in the ninth and tenth aspects of the present invention, the imaging device drive control means includes a driving method for generating a field signal and a frame signal as the different driving methods. And a driving method for generating the data.

The operation of the fourteenth invention is as follows. For example, in a case where an image signal with a small amount of charge storage is generated by the field readout driving method and an image signal with a large amount of charge storage is generated by the frame readout driving method, a dynamic range with small noise components and excellent S / N performance Image can be obtained.

According to a fifteenth aspect of the present invention, in the ninth and tenth aspects of the present invention, the image pickup device drive control means includes a frame readout having an interlace drive method and a light shielding means as the different drive methods. And a control drive system.

According to a sixteenth aspect of the present invention, in the ninth and tenth aspects of the present invention, the imaging means includes an image signal by an interlace driving method having a small charge accumulation amount and a light shielding means having a large charge accumulation amount. It is configured to obtain an image signal by the frame readout control driving method.

The functions of the fifteenth and sixteenth aspects are as follows. By reading the image signal of an image having a large charge accumulation amount due to a long exposure time, it is possible to avoid extra exposure during reading by reading out the light, thereby improving the reliability of the signal and the high quality of the image. Can be ensured.

According to a seventeenth aspect of the present invention, in the ninth aspect of the present invention, the imaging means includes: a frame image signal by one of two different driving methods, a frame readout control driving method; It is configured to include an interpolating unit that generates a field image signal by a driving method and converts the field image signal into an interpolated frame image signal.

The operation of the seventeenth invention is as follows. The signal formats of the frame image signal and the field image signal are different from each other. Therefore, before combining, by interpolation means,
By interpolating the field image signal and unifying the signal format with the frame image signal, it is possible to satisfactorily combine the two image signals without any trouble.

In the imaging apparatus according to an eighteenth aspect of the present invention, in the seventeenth aspect, the frequency emphasizing means weakens the emphasis on the set frequency component of the interpolated frame image signal, and reduces the emphasis on the set frequency component of the frame image signal. It is designed to enhance the emphasis.

The operation of the eighteenth aspect is as follows. In the interpolated frame image signal, it is more likely that a false signal accompanying the interpolation is mixed in a high frequency band, as compared with a pure frame image signal in which no interpolation is performed. Therefore, in the frequency emphasizing process for an image signal with a small charge accumulation amount, by setting such that a false signal is suppressed, an image having a small dynamic signal component as well as a noise component and a wide dynamic range excellent in S / N performance is provided. Can be obtained.

In the imaging apparatus according to a nineteenth aspect of the present invention, in the tenth aspect described above, the imaging means includes a frame image signal based on one of two different driving modes, a frame readout control driving mode, and another interlaced signal. It is configured to include an interpolating unit that generates a field image signal by a driving method and converts the field image signal into an interpolated frame image signal.

The operation of the nineteenth aspect is as follows. The signal formats of the frame image signal and the field image signal are different from each other. Therefore, before combining, by interpolation means,
By interpolating the field image signal and unifying the signal format with the frame image signal, it is possible to satisfactorily combine the two image signals without any trouble.

In the imaging apparatus according to a twentieth aspect of the present invention, in the nineteenth aspect, the frequency emphasizing means weakens emphasis on a set frequency component of an interpolated frame image signal portion in the composite image signal, and The configuration is such that the emphasis of the set frequency component of the signal portion is enhanced.

The operation of the twentieth aspect is as follows. In the interpolated frame image signal, it is more likely that a false signal accompanying the interpolation is mixed in a high frequency band, as compared with a pure frame image signal in which no interpolation is performed. Therefore, in the frequency emphasizing process for an image signal with a small charge accumulation amount, by setting such that a false signal is suppressed, an image having a small dynamic signal component as well as a noise component and a wide dynamic range excellent in S / N performance is provided. Can be obtained.

(Specific Embodiment) Hereinafter, a specific embodiment of a dynamic range expansion type imaging apparatus according to the present invention will be described in detail with reference to the drawings.

(Embodiment 1) [Equivalent to claim 7] FIG. 1 is a block diagram showing a configuration of an imaging apparatus according to Embodiment 1 of the present invention. In FIG. 1, reference numeral 1 denotes an optical lens, 2 denotes a mechanical shutter for an optical diaphragm, and 3 denotes a solid-state imaging device using a CCD (charge-coupled device). In the first embodiment, the solid-state imaging device is generally used in a consumer solid-state imaging device. It is assumed that the interline transfer CCD (IT-CCD) is used. 4 is an analog signal processing means comprising a correlated double sampling circuit and an automatic gain control (AGC) circuit, 5 is an A / D conversion means, and 6 is an image signal converted by the A / D conversion means 5 into a digital signal. This is an image memory for storing. Reference numeral 7 denotes first digital signal processing means (I) for performing processing such as separation of a luminance signal and a chrominance signal on two systems of image signals (a long signal and a short signal in the figure) read from the image memory 6; Long signal frequency emphasizing means 9 for emphasizing a predetermined frequency component with respect to the long signal from the first digital signal processing means (I) 7, and a long signal frequency emphasizing means 9 for the short signal from the first digital signal processing means (I) 7. Short signal frequency emphasizing means for emphasizing a predetermined frequency component by
This is an image synthesizing unit that synthesizes two types of image signals, that is, a long signal from the ong signal frequency emphasizing unit 8 and a short signal from the short signal frequency emphasizing unit 9.

The mix signal obtained by the image synthesizing means 10 is subjected to processing such as matrix operation and encoding to a specific format in a second digital signal processing means (II) 11. The aperture drive control unit 12 controls opening and closing of the mechanical shutter 2. The solid-state imaging device drive control unit 13 controls the exposure of the solid-state imaging device 3 and controls the mode and timing of signal reading. It is a means to do. The charge storage amount control means 14 is composed of the solid-state imaging device drive control unit 13 and controls the charge storage amount of the solid-state imaging device 3. It is assumed that the operation modes and operation timings of all the constituent elements including the above are integrally controlled by the system control means 15.

[First Case of Configuration and Operation of Solid-State Imaging Device 3] Next, a first case of configuration and operation of the solid-state imaging device 3 will be described with reference to FIGS.

FIG. 2 is a schematic diagram for explaining the operation and configuration of the solid-state imaging device 3. In FIG. 2, the solid-state imaging device 3 is an interline transfer CCD (IT-CCD) capable of reading signals in an all-pixel reading mode, in other words, a mode in which accumulated charges of a photodiode in a vertical direction can be read independently. For the sake of convenience, the description will be made with a so-called 4 × 2 pixel configuration of 4 vertical pixels and 2 horizontal pixels. In FIG. 2, a photodiode is a portion where signal charges corresponding to light intensity are accumulated by photoelectric conversion.
The accumulated charges move to the vertical transfer CCD. At this time, the charges of the two upper and lower photodiodes adjacent to each other are mixed on the vertical transfer CCD. After that, the horizontal transfer CCD
(The difference between the short signal and the long signal will be described later).

FIG. 3 is an operation timing chart relating to exposure of a subject image and reading of an exposed signal in the solid-state imaging device 3 shown in FIG. 3A shows a vertical synchronization signal, FIG. 3B shows a read control pulse for controlling reading of signal charges from the photodiode of the solid-state imaging device 3, FIG. 3C shows an exposure signal on the photodiode of the solid-state imaging device 3, (D) shows a signal output from the solid-state imaging device 3.

In the solid-state image pickup device 3 having such a configuration, first, as shown in FIG. 2A, a signal having a small charge accumulation amount accumulated in the photodiode is generated as shown in FIG. 3B.
The read control pulse moves the CCD to the vertical transfer CCD (corresponding to S in the figure: hereinafter referred to as a short signal). Next, first, as shown in FIG. 2B, a signal having a large charge accumulation amount accumulated in the photodiode is a long signal shown in FIG. 3B.
The read control pulse moves to the vertical transfer CCD (corresponding to L in the figure: hereinafter, referred to as a long signal). here,
The charge storage amount of the photodiode is determined by the time interval of the read control pulse. For example, the short signal is determined by the time interval between the immediately preceding long read control pulse and the current read short read control pulse, and the long signal is determined by the time interval between the immediately preceding short read control pulse and the currently read long read control pulse. Thus short signal and long
The signal is alternately moved onto the vertical transfer CCD, and then is transferred to the long transfer within the field period via the horizontal transfer CCD.
The signal and the short signal are output for each line.

During this field period, a long signal and a short signal
The output signal of the solid-state imaging device that outputs a signal for each line is:
In the image memory 6 shown in FIG. 1, two signals of a long signal and a short signal are read out from the image memory 6 after being separated into a long signal and a short signal. In other words, a long signal and a short signal are obtained in each field, and the number of lines of each signal is の of the number of pixels in the vertical direction of the solid-state imaging device 3.

[Second Case of Configuration and Operation of Solid-State Imaging Device 3] Next, a second case of configuration and operation of the solid-state imaging device 3 will be described with reference to FIGS.

FIG. 4 is a schematic diagram for explaining the operation and configuration of the solid-state imaging device 3, and portions corresponding to the configuration in FIG. 2 are denoted by the same reference numerals. In FIG. 4, the solid-state imaging device 3
Is an interline transfer CCD (IT-CCD) capable of reading signals in the all-pixel reading mode. In FIG. 4, a photodiode is a portion in which signal charges corresponding to light intensity are accumulated by photoelectric conversion. After a predetermined time, the accumulated charges are moved to a vertical transfer CCD by a control pulse applied. . At this time, the charges of the adjacent photodiodes are independently moved to the vertical transfer CCD. Then, it is output to the outside via the horizontal transfer CCD.

FIG. 5 is an operation timing chart relating to exposure of a subject image and reading of an exposed signal in the solid-state image pickup device 3 shown in FIG. 4, and portions corresponding to those in FIG. Attach.

In the solid-state image pickup device 3 having such a configuration, first, as shown in FIG. 4A, a signal having a small charge accumulation amount accumulated in the photodiode is generated as shown in FIG.
The read control pulse moves the CCD to the vertical transfer CCD (corresponding to S in the figure: hereinafter referred to as a short signal). Next, first, as shown in FIG. 4B, a signal having a large charge accumulation amount accumulated in the photodiode is a long signal shown in FIG. 5B.
The read control pulse moves to the vertical transfer CCD (corresponding to L in the figure: hereinafter, referred to as a long signal). here,
The charge storage amount of the photodiode is determined by the charge reset period by the electronic shutter and the time interval between the read control pulses. For example, the short signal is determined by the time interval from the end of the charge reset by the electronic shutter to the short read control pulse, and the long signal is determined by the time interval between the immediately preceding short read control pulse and the current read long read control pulse. Thus, the short signal and long signal are
It is moved in time series on the vertical transfer CCD, and then
A long signal and a short signal are alternately output in field units via the horizontal transfer CCD.

The output signal of the solid-state imaging device for alternately outputting the long signal and the short signal in field units is obtained by synchronizing the long signal and the short signal in the image memory 6 of FIG. Two signals of the signal are read. In other words, every field
A long signal and a short signal are obtained, and the number of lines of each signal is equal to the number of pixels of the solid-state imaging device 3 in the vertical direction.

Next, the signal processing in the first digital signal processing means (I) 7 for the two signals of the long signal and the short signal from the image memory 6 in FIG. 1 will be described below. FIG. 6 is an example of a color filter of the solid-state imaging device 3, in which a color filter of a color difference line sequential method is formed.
Ye represents yellow, Mg represents magenta, Cy represents cyan, and G represents green, and one photodiode corresponds to one color filter. FIG. 6 also shows an example of signal processing of the first digital signal processing means (I) 7 in FIG. 1 for this color filter.

In the processing of the Sig1 line, a luminance signal (Mg + Ye + G + Cy) is obtained by adding two horizontal pixels, and a color difference signal (Mg + Cy) is obtained by subtracting two horizontal pixels.
-G-Ye = 2BG-Cb). Similarly, Si
In the processing of the g2 line, a luminance signal (Mg + Ye + G + Cy) is obtained by adding two horizontal pixels, and a color difference signal (Mg + Ye-G-Cy = 2R-G) is obtained by subtracting two horizontal pixels.
= Cr). Here, regarding the spectral sensitivity characteristics of each color filter, Mg = R + B, Cy = G + B, Ye
= R + G.

As described above, the first digital signal processing means (I) 7 separates the long signal separated into the luminance signal and the color difference signal.
Next, the two signals of the signal and the short signal are Lo
ng signal frequency emphasizing means 8 and Short signal frequency emphasizing means 9. The Long signal frequency emphasizing means 8 and the Short signal frequency emphasizing means 9 extract, for example, a predetermined frequency component of the input signal and perform a multiplication process on the extracted signal in order to enhance the edge and improve the response.
A process of adding to the original signal is performed. The frequency components extracted by the long signal frequency emphasizing means 8 and the short signal frequency emphasizing means 9 and the gain of the multiplication process are independently set by the system control means 15.

Here, since the short signal has a smaller amount of charge stored in the photodiode than the long signal, the short signal is often a signal having a large noise component and degraded S / N. Therefore, the setting of the short signal frequency emphasizing means 9 for the short signal is set such that noise is not emphasized as compared with the setting of the long signal frequency emphasizing means 8 for the long signal. As a specific setting example, the frequency band of the signal extracted from the short signal is set lower than the frequency band of the signal extracted from the long signal. Alternatively, the multiplication gain value for the extracted signal from the short signal is set lower than the multiplication gain value for the extracted signal from the long signal. This makes it possible to suppress noise in the short signal.

Next, the long signal (corresponding to the signal of long-time exposure) subjected to the optimum frequency emphasizing processing in this way and sh
The ort signal (corresponding to the signal of the short-time exposure) is combined by the image combining means 10 into one image signal. The basic principle of this combination is to take two images, a short signal and a long signal, as described above, and take an image with an increased dynamic range by combining the two images. The principle of such expansion of the dynamic range will be described with reference to FIG.

FIGS. 7A and 7B show the relationship between the brightness of the subject (the amount of light incident on the solid-state image sensor) and the amount of signal output from the solid-state image sensor during exposure. FIG. 7 (a)
As shown in (1), the amount of charge generated on the photodiode of the solid-state imaging device due to incident light during long-time exposure is large, and the amount of output signal is naturally large. However, there is an upper limit on the amount of electric charge stored in the photodiode. If the upper limit is exceeded, saturation, that is, a phenomenon in which a signal is broken occurs, and an object image cannot be accurately reproduced. Conversely, if the exposure time is set short as shown in FIG. 7B, it is possible to avoid saturation. However, this time, the low-luminance portion in the subject is lost in black and the S / N deteriorates.

Therefore, by using a signal obtained by long-time exposure (long signal) and a signal obtained by short-time exposure (short signal), an image composed of a long signal is adopted as a low-luminance part, and a high-luminance part is used. If an image composed of short signals is adopted and the two are combined, it is possible to reproduce the low-luminance part to the high-luminance part of the subject, and it is possible to expand the dynamic range of the imaging device. At this time, as shown in FIG.
If the synthesis is performed after multiplying the t signal by a gain corresponding to the ratio of the exposure amount to the long signal (ratio of the exposure time), as shown in FIG.
As shown in (1), the dynamic range can be expanded in accordance with the exposure ratio. For example, when the exposure amount ratio (exposure time ratio) between the short signal and the long signal is 1: D, the short signal is
By doubling and combining, the dynamic range can be expanded to D times.

Hereinafter, the composite image signal (mix signal) having the expanded dynamic range is subjected to processing such as matrix calculation and encoding to a specific format in the second digital signal processing means (II) 11.

As described above, in the image pickup apparatus that combines the long signal (long exposure signal) and the short signal (short exposure signal) to create a combined image signal with an expanded dynamic range, the Long signal frequency enhancement is performed. Means and a short signal frequency emphasizing means, and by setting the short signal to suppress noise, it is possible to obtain an image having a small dynamic component and excellent in S / N performance and having a wide dynamic range. .

(Embodiment 2) [Equivalent to claim 8] FIG. 8 is a block diagram showing a configuration of an imaging apparatus according to Embodiment 2 of the present invention. Portions corresponding to the configuration of the first embodiment shown in FIG. 1 are denoted by the same reference numerals.

FIG. 8 differs from FIG. 1 in that Mix signal frequency emphasizing means 16 is provided in place of Long signal frequency emphasizing means 8 and Short signal frequency emphasizing means 9, and the other configuration is the same as that of the embodiment. Same as 1 That is, in the figure, 1 is an optical lens, 2 is a mechanical shutter for an optical diaphragm, 3 is an interline transfer type solid-state imaging device (IT-CCD), and 4 is a correlated double sampling circuit and an automatic gain control circuit. Analog signal processing means, 5 is A / D conversion means, 6 is an image memory,
Reference numeral 7 denotes first digital signal processing means (I) for performing processing such as separation of a luminance signal and a color difference signal, and reference numeral 10 denotes two systems of a short signal and a long signal from the first digital signal processing means (I) 7. An image synthesizing unit 16 for synthesizing the image signal is a signal M for enhancing a predetermined frequency component of the mix signal from the image synthesizing unit 10.
The ix signal frequency emphasizing means 11 is a second digital signal processing means (II) for performing processing such as matrix operation and encoding to a specific format on the output signal of the Mix signal frequency emphasizing means 16. 12 is an aperture drive control means, 1
Reference numeral 3 denotes a solid-state imaging device drive control unit, 14 denotes a charge storage amount control unit, and 15 denotes a system control unit.

With respect to the imaging apparatus thus configured,
The following description focuses on differences from the first embodiment.

The solid-state imaging device 3 may have the configuration shown in FIGS. 2 and 3 in the first embodiment, or the configuration shown in FIGS. 4 and 5 by performing the same operation. The long signal and the short signal are read from the image memory 6 as two signals. This read long signal and
The short signal is subjected to signal processing such as color separation as shown in FIG. 6 in the first digital signal processing means (I) 7. Long separated into this luminance signal and color difference signal
The signal of the two systems of the signal and the short signal is
As a result, a mix signal having an expanded dynamic range is obtained in the same manner as described with reference to FIG. The combined signal is subjected to, for example, a predetermined frequency component of the input signal, which is subjected to a multiplication process in the Mix signal frequency emphasizing means 16 in order to enhance the edge and improve the response, and to perform a multiplication process on the extracted signal. Is added to the signal. The frequency components extracted by the Mix signal frequency emphasizing means 16 and the gain of the multiplication processing are set by the system control means 15.

Next, a first mode of the Mix signal frequency emphasizing means 16 will be described with reference to FIG.

As shown in FIG. 9, the combined mix signal is such that a long signal and a short signal (strictly speaking, a signal of the short signal × exposure ratio D) are replaced at the boundary of the saturation level of the photodiode. Similar to the first embodiment, compared to the long signal
Since the short signal has a small charge accumulation amount of the photodiode, it is often a signal having a large S / N ratio with a large amount of noise component, and the gain is increased by the exposure amount ratio.
In the mix signal frequency emphasizing means 16, the short in the mix signal
The frequency emphasis setting for the signal range is set such that noise is not emphasized compared to the frequency emphasis setting for the long signal range. As a specific setting example, lo
The frequency band of the extracted signal from the short signal range is set lower than the frequency band of the extracted signal from the ng signal range. Alternatively, the multiplication gain value for the extraction signal from the short signal range is set lower than the multiplication gain value for the extraction signal from the long signal range. This makes it possible to suppress noise in the short signal range.

As described above, the long signal range and the short signal range are detected at the level of the mix signal with respect to the mix signal, and the long signal range and the short signal range are detected.
Optimal frequency emphasis processing that matches the signal range and the short signal range can be performed.

Next, the second signal of the mix signal
Will be described with reference to FIG. in this case,
The operation of the image combining means 10 is also devised.

Two threshold values Th for the long signal level min
And Th When max is set and the long signal level is the equation (1),
That is, the signal level of the long signal is Th If there is no possibility of saturation below min, the synthesis coefficient k is set to 0, and when the long level is given by the equation (2), that is, when the long signal level is Th If the output of the solid-state image sensor is close to the saturation level at max or more,
The combining coefficient k is set to 1. The threshold Th max, Th min is appropriately determined according to the saturation characteristics and S / N of the solid-state imaging device to be used.

0 ≦ long signal level ≦ Th min ………………………… (1) Th max ≦ long signal level ………………………………………………………………………………………………………………………………………………………………………………………………………………………… (2) Is determined by the linear expression of Expression (4).

Th min <long signal level <Th max .................. (3) k = ｛1 / (Th max-Th min)｝ × (long signal level) − ｛Th min / (Th max-Th min)｝ (4) The long signal and the short signal are synthesized for each pixel by the equation (5) using the synthesis coefficient k obtained as described above. lon
The signal obtained by combining the g signal and the short signal is combined with the combined image signal (mix
Signal).

Synthesized image signal = (1−k) × long signal + k × short signal × D (5) Further, for example, the color filters of the solid-state imaging device 3 are arranged in a color difference line-sequential arrangement shown in FIG. In the case where the driving method of the solid-state imaging device 3 is such that the charges of the two upper and lower adjacent photodiodes are mixed on the vertical transfer CCD shown in FIG.
As shown in, for example, a long signal (Ye + Mg) L11 and this (Y
e + Mg) Short signal (Ye + Mg) S1 whose spatial position is the same as L11
The composite image signal (Ye + Mg) M11 is obtained from 1 and l
Assuming that the synthesis coefficient determined from the ong signal is k11, (6)
Synthesis is performed by the formula.

(Ye + Mg) M11 = (1−k11) × (Ye + Mg) L11 + k11 × (Ye + Mg) S11 × D (6) Other pixels of the composite image signal Similarly to the expression (6), it is obtained from the long signal and the short signal existing at the same spatial position.

Note that in equations (5) and (6), sh
The constant D multiplied by the ort signal is the ratio of the exposure amount (exposure time) of the long signal and the short signal. For example, the exposure amount (exposure time) of the long signal is TL, and the exposure amount of the short signal (exposure time). ) Is TS, D is obtained by equation (7).

D = TL / TS (7) As described above, by using the long signal and the short signal, the signal level of the long signal becomes the threshold value. The part below Thmin is a long signal, and the signal level is the threshold Th A portion larger than max, that is, a portion where the output of the solid-state imaging device 3 is close to saturation (a portion where the brightness of the captured image is high and the signal is normally broken) is a short signal, and a portion having intermediate brightness is a long signal and a short signal. By generating a composite image signal composed of signals obtained by weighting and adding, it is possible to expand the dynamic range of the captured image signal.

Next, the operation of the mix signal frequency emphasizing means 16 for the synthesized image signal (mix signal) thus obtained will be described. The signal of the set frequency suitable for the short signal range is Sig S, the gain for the extracted signal is G S and a signal of the set frequency suitable for the long signal range L,
The gain for the extracted signal is G Let L be the frequency emphasis signal Sig for the composite image signal (mix signal). M is in Figure 1
It is determined by the linear expression of Expression (8) using the combination coefficient k shown in 0.

Sig M = (1-k) × (G L × Sig L) + k × (G S × Sig S)... (8) As described above, the long signal range and the short signal range are smoothly and continuously replaced by the synthesis coefficient k. Optimal frequency emphasis processing can be performed according to the combination ratio of short signals.

As described above, in an image pickup apparatus that synthesizes a long signal (a signal for a long-time exposure) and a short signal (a signal for a short-time exposure) to create a synthesized image signal with an expanded dynamic range, the frequency of the Mix signal is emphasized. By providing the means and setting to suppress noise in the short signal range, it is possible to obtain an image with a small dynamic component and excellent S / N performance and a wide dynamic range.

(Embodiment 3) [Equivalent to Claims 9 and 17] FIG. 12 is a block diagram showing a configuration of an image pickup apparatus according to Embodiment 3 of the present invention. Portions corresponding to the configuration of the first embodiment shown in FIG. 1 are denoted by the same reference numerals.

FIG. 12 differs from FIG. 1 in that a mechanical shutter 17 also serving as an optical diaphragm, an interpolating means 18 and a shutter drive control means 19 for controlling opening and closing of the mechanical shutter 17 are provided. The charge storage amount control means 20 is composed of the solid-state image pickup device drive control means 13 and the shutter drive control means 19, and controls the charge storage amount of the solid-state image pickup device 3. Same as 1

In the figure, 1 is an optical lens, 17 is a mechanical shutter also used as an optical diaphragm, 3 is an interline transfer type solid-state image pickup device (IT-CCD), 4 is a correlated double sampling circuit and an automatic gain control circuit. 5, an A / D converter, 6 an image memory, and 7 a first unit for performing processing such as separation of a luminance signal and a color difference signal.
Digital signal processing means (I), 18 are interpolation means for performing an interpolation process on a short field signal which is one of the two output signals of the first digital signal processing means (I) 7; 1 digital signal processing means (I)
Emphasize predetermined frequency components for long frame signals
Long signal frequency emphasizing means 9 is short from interpolation means 18
Short signal frequency emphasizing means 10 for emphasizing a predetermined frequency component with respect to the t-interpolated signal, and 10 are two-system images of a long frame signal from Long signal frequency emphasizing means 8 and a short interpolated signal from Short signal frequency emphasizing means 9. An image synthesizing unit 11 for synthesizing the signal is a second digital signal processing unit (I) for performing processing such as matrix operation and encoding to a specific format on the signal obtained by the image synthesizing unit 10.
I) and 19 are shutter drive control means for controlling the opening and closing of the mechanical shutter 17; 13 is a solid-state image sensor drive control means for controlling the exposure of the solid-state image sensor 3 and controlling the mode and timing of signal reading; Reference numeral 20 denotes a solid-state imaging device drive control unit 13 and a shutter drive control unit 19
, A charge storage amount control means for controlling the charge storage amount of the solid-state imaging device 3, and 15 a system control means.

[0093] Regarding the imaging apparatus thus configured,
The following description focuses on differences from the first embodiment.

FIGS. 13 (a), (b), (c) and (d)
3 is a schematic diagram for explaining the configuration and operation of the solid-state imaging device 3. FIG. In the third embodiment, the solid-state imaging device 3
This is an interline transfer CCD (IT-CCD) capable of reading out signals in two readout modes, a field readout mode and a frame readout mode. For convenience of description, a so-called 4 pixel vertical and 2 horizontal pixel as shown in FIG. A description will be given of a configuration of 4 × 2 pixels.

FIGS. 13A and 13B are diagrams for explaining the field read mode in the IT-CCD. In FIG. 13A, a photodiode is a portion in which signal charges corresponding to light intensity are accumulated by photoelectric conversion. After a certain period of time, charges accumulated by a read control pulse applied are transferred to a vertical transfer CCD. Moving. At this time, the charges of the two upper and lower photodiodes adjacent to each other are mixed on the vertical transfer CCD and output to the outside via the horizontal transfer CCD. The above is the reading operation of the first field.

In the second field, as shown in FIG. 13B, the pair of photodiodes mixed on the vertical transfer CCD is shifted by one pixel in the vertical direction as compared with the case of the first field. As a result, signal reading for two fields is performed, and an image signal corresponding to one frame of the interlaced system can be read as a whole.

Next, the frame read mode will be described with reference to FIGS. 13 (c) and 13 (d). In the frame read mode, first, as shown in FIG. 13C, in the first field, the electric charge accumulated in the photodiode is transferred to the vertical transfer CCD by skipping one pixel in the vertical direction.
This is output to the outside via the horizontal transfer CCD. Then, as shown in FIG. 13D, in the second field, the charges of the photodiodes not transferred to the vertical transfer CCD in the first field are transferred to the vertical transfer CCD,
This is output to the outside via the horizontal transfer CCD. As described above, in the frame read mode, the charges on the photodiodes are output to the outside without being mixed by the vertical transfer CCD. As a result, signal reading for two fields is performed, and an image signal corresponding to one frame of the interlaced system can be read as a whole.

Next, a method of photographing a short signal and a long signal using the solid-state imaging device 3 shown in FIG. 13 will be described with reference to FIG. FIG. 14 is a timing chart relating to exposure of a subject image in the solid-state imaging device 3 and reading of an exposed signal. In the figure, (a) is a vertical synchronization signal, (b) is a read control pulse for controlling reading of signal charges from the photodiode of the solid-state imaging device 3, (c) is the open / close state of the mechanical shutter 17, and (d) is Exposure signal on the photodiode of the solid-state imaging device 3, (e)
Indicates a signal output from the solid-state imaging device 3.

At the time of the exposure of the short signal, the mechanical shutter 1
7 is opened, and exposure is performed using an electronic shutter function for a necessary exposure time, for example, 1/1000 second. 1/1
After the completion of the exposure for 2,000 seconds, the charge accumulated on the photodiode is moved to the vertical transfer CCD by the read control pulse. At this time, the solid-state imaging device 3 is driven in the field readout mode, and the charges accumulated on the photodiodes are mixed on the vertical transfer CCD and read out to the outside as described with reference to FIG. At this time, the image signal to be read is only the signal for the first field. However, only the signal for the second field may be used. FIG. 15 shows a short signal read in the field read mode. Note that the number of photodiodes in the vertical direction of the solid-state imaging device 3 is N (for convenience of explanation, N is an even number, but is not limited to this).

The color filter of the solid-state imaging device 3 is shown in FIG.
As shown in FIG. 15, the short signals read out as shown in FIG. 15 are Ye + Mg, Cy + G, and Ye, in which the four color signals of Ye, Cy, G, and Mg are added, respectively.
There are four types of signals, e + G and Cy + Mg. The number of lines in the vertical direction is 1/2 of the number N of photodiodes in the vertical direction.

Next, while reading the short signal,
Perform exposure of ong signal. The exposure period of the long signal is, for example, 1
/ 100 seconds. The exposure time of the long signal is controlled by opening and closing the mechanical shutter 17, and the mechanical shutter 17 is closed 1/100 second after the start of the exposure of the long signal to complete the exposure. By closing the mechanical shutter 17 in this manner, a signal that has been exposed for a long time is not exposed extra during the reading.

When the exposure of the long signal is completed, the accumulated charge on the photodiode is transferred to the vertical transfer C by the read control pulse.
Transferred to CD. At this time, the solid-state imaging device 3 is driven in the frame read mode, and as described with reference to FIG. 13C, the charge of the photodiode corresponding to the odd-numbered line in the vertical direction is read for the first field.
After the reading of the signal of the first field is completed, the charge of the photodiode corresponding to the even-numbered line in the vertical direction is read (second field), whereby a signal corresponding to one frame of a long signal is read from the solid-state imaging device 3. The period of the vertical synchronization signal shown in FIG. 14A is, for example, 1/100 second, and the reading of the signal for one field from the solid-state imaging device 3 is completed within one period of the vertical synchronization signal. .

FIG. 16 shows a long signal read in the frame read mode. As shown in FIG. 16, in the read long signal, the first field is a signal of two colors of Ye and Cy, and the second field is a signal of two colors of G and Mg. The number of lines in the vertical direction is 1/2 of the number N of photodiodes in the vertical direction in each field, and a signal of N lines corresponding to one frame is obtained by combining two fields.

By performing the above-described exposure and signal reading, two signals having different exposure times, that is, a short signal (short field signal) as one field image and a long signal (long frame signal) as one frame image are obtained. Can be obtained. The short signal has the number of horizontal lines.
Since the signal is half of the long signal, the short signal has a smaller number of pixels than the long signal.

Next, the two signals having different exposure times obtained by the solid-state imaging device 3 are converted into digital signals by the A / D converter 5 through the analog signal processor 4 and temporarily stored in the image memory 6. You.

From the image memory 6, the long frame signal and the sh
The ort field signal is read. When the long frame signal is read from the image memory 6, the first line of the first field, the first line of the second field, the second line of the first field, the second line of the second field, etc. It is assumed that the data is sequentially read from the top line when viewed as a frame image.

The long frame signal read from the image memory 6 is subjected to signal processing such as color separation in the first digital signal processing means (I) 7. As an example of this signal processing, when viewed as a frame signal, adjacent long signals of two upper and lower lines are added and mixed. This is because when the long signal and the short signal are combined, if the signal formats of the two signals are different, the combination cannot be performed.
The signal is subjected to the same processing as pixel mixing on the vertical transfer CCD of the solid-state imaging device 3 by the two horizontal line addition means in the first digital signal processing means (I) 7, and after the addition processing, FIG. Are separated into a luminance signal and a color difference signal by signal processing such as color separation shown in FIG. On the other hand, the short signal is also separated into a luminance signal and a color difference signal by signal processing such as color separation shown in FIG.

Thus, the first digital signal processing means (I) 7 separates the luminance signal and the color difference signal from each other.
A long frame signal and a short field signal are output. Here, the short field signal is interpolated into one frame image by the interpolation means 18. FIG. 17 shows the relationship between the long frame signal and the short interpolation signal at this point. FIG.
7A shows a long signal after color separation, FIG. 17B shows a short signal before interpolation processing, and FIG. 17C shows a sho after interpolation processing.
The rt signal is shown. As shown in FIGS. 17A and 17C, two horizontal line addition processing for a long signal and sh
By the interpolation processing for the ort signal, the signal formats of the long signal and the short signal match.

As described above, in the interpolation means 18, FIG.
The field image shown in (b) is converted into the frame image shown in (c) by interpolation processing, and the method will be described below.

For example, when a horizontal line signal between the second line and the third line in FIG. 17B is obtained by interpolation processing, it is necessary to create a horizontal line signal composed of Y + Cb signals. At this time, since the nearest Y (luminance) signal is the second line and the third line, a line between the second line and the third line is obtained from both of them by interpolation processing. On the other hand, the neighboring Cb (color difference) signal is
Since the line is a line, a line between the second line and the third line is obtained from the two by interpolation processing. However, since the spatial distance between the position where the horizontal line signal is obtained by the interpolation processing and the second and fourth lines is not equidistant, weighting is necessary according to the distance. Therefore, in the interpolation means 18, the Y (luminance) signal is added to the second line and the third line with weighting of 1/2 each,
The Cb (color difference) signal is added to the second line and the fourth line with weighting of 3/4 and 1/4, respectively.

Similarly, when a horizontal line signal between the third line and the fourth line is obtained by interpolation processing, it is necessary to create a horizontal line signal composed of Y + Cr signals. At this time, since the nearest Y (luminance) signal is the third line and the fourth line, a line between the third line and the fourth line is obtained from the both by interpolation processing. On the other hand, the neighboring Cr
Since the (color difference) signal is the third line and the fifth line, a line between the third line and the fifth line is obtained from both of them by interpolation processing.

By the above processing, a short interpolated signal corresponding to one frame obtained through interpolation processing from one frame long frame signal and one field short signal is generated.

Next, the long frame signal and the short interpolated signal are supplied to the long signal frequency emphasizing means 8 and the short signal, respectively.
The signal is input to the t-frequency emphasis means 9. The Long signal frequency emphasizing means 8 and the Short signal frequency emphasizing means 9 extract, for example, a predetermined frequency component of the input signal and perform a multiplication process on the extracted signal in order to enhance the edge and improve the response. Is added to the signal. The frequency components extracted by the long signal frequency emphasizing means 8 and the short signal frequency emphasizing means 9 and the gain of the multiplication process are independently set by the system control means 15.

Here, since the short interpolation signal has a smaller amount of charge stored in the photodiode than the long frame signal, the short interpolation signal is often a signal with a large noise component and a deteriorated S / N. Further, since the short interpolation signal has been subjected to the interpolation processing, it often has a false signal accompanying the interpolation processing in a high frequency band as compared with a pure frame signal. So, short
The setting of the Short signal frequency emphasizing unit 9 for the t-interpolation signal is such that noise and false signals are not emphasized as compared with the setting of the Long signal frequency emphasizing unit 8 for the long frame signal. As a specific setting example, the frequency band of the signal extracted from the short interpolation signal is set lower than the frequency band of the signal extracted from the long frame signal. Alternatively, the multiplication gain value for the extraction signal from the short interpolation signal is set lower than the multiplication gain value for the extraction signal from the long frame signal. This allows sh
It is possible to suppress the noise and the false signal level of the ort interpolation signal.

Next, the long frame signal (corresponding to the signal of long exposure) and the short interpolation signal (corresponding to the signal of short exposure) which have been subjected to the optimum frequency emphasizing processing as described above are
Synthesized into one image signal by the image synthesizing means 10,
It becomes a composite image signal (mix signal) having an expanded dynamic range, and is subjected to processing such as matrix calculation and encoding into a specific format in the second digital signal processing means (II) 11.

As described above, in the image pickup apparatus which combines the long signal (long exposure signal) and the short signal (short exposure signal) to create a combined image signal having an expanded dynamic range, the Long signal frequency enhancement is performed. Means and a short signal frequency emphasizing means, and by setting the short signal to suppress noise and false signal levels, an image having a small noise component and false signal component and excellent in S / N performance and having a wide dynamic range is obtained. It becomes possible.

(Embodiment 4) {[Claims 10 and 19]
Equivalent] FIG. 18 is a block diagram illustrating a configuration of an imaging device according to Embodiment 4 of the present invention. Portions corresponding to the configuration of the third embodiment shown in FIG. 12 are denoted by the same reference numerals. FIG. 18 differs from FIG. 12 in that a Mix signal frequency emphasizing unit 16 is provided instead of the Long signal frequency emphasizing unit 8 and the Short signal frequency emphasizing unit 9.
This is the same as in the third embodiment.

In the figure, 1 is an optical lens, 17 is a mechanical shutter that also serves as an optical diaphragm, 3 is an interline transfer type solid-state imaging device (IT-CCD), 4 is a correlated double sampling circuit and an automatic gain control circuit. 5, an A / D converter, 6 an image memory, and 7 a first unit for performing processing such as separation of a luminance signal and a color difference signal.
Are digital signal processing means (I), 18 for performing interpolation processing on a short field signal which is one of the two output signals of the first digital signal processing means (I) 7; Image synthesizing means for synthesizing two types of image signals of a long frame signal from the digital signal processing means (I) 7 and a short interpolation signal from the interpolation means 18;
6 is a Mix signal frequency emphasizing means for emphasizing a predetermined frequency component of the mix signal from the image synthesizing means 10, and 11 is a process such as matrix operation, encoding to a specific format, etc. on the signal obtained by the image synthesizing means 10. Second digital signal processing means (II), 19 is shutter drive control means, 13 is solid-state image sensor drive control means, 20 is solid-state image sensor drive control means 13 and shutter drive control means 19
, A charge storage amount control means for controlling the charge storage amount of the solid-state imaging device 3, and 15 a system control means.

With respect to the image pickup apparatus thus configured,
The following description focuses on the differences from the third embodiment.

By performing the same exposure and signal reading as in the third embodiment, it is possible to obtain two signals having different exposure times, that is, a short signal as one field image and a long signal as one frame image. . In addition, short
Since the number of horizontal lines is 1/2 of the long signal,
The ort signal has a smaller number of pixels than the long signal.

Next, two signals having different exposure times obtained by the solid-state imaging device 3 are converted into digital signals by an A / D converter 5 through an analog signal processor 4.
The image is temporarily stored in the image memory 6. Lo from the image memory 6
ng frame signal and short field signal are read out,
The first digital signal processing means (I) 7 outputs a long frame signal and a short field signal separated into a luminance signal and a color difference signal. Here, the short field signal is interpolated into one frame image by the interpolation means 18. The long frame signal and the short interpolation signal are mixed by the image synthesizing unit 10 with the dynamic range expanded.
Signal. The combined signal is subjected to, for example, a predetermined frequency component of the input signal, which is subjected to a multiplication process in the Mix signal frequency emphasizing means 16 in order to enhance the edge and improve the response. Is added to the signal. This Mix signal frequency emphasis means 1
The frequency components to be extracted in step 6 and the gain of the multiplication process are set by the system control means 15.

Here, since the short interpolation signal has a smaller amount of charge stored in the photodiode than the long frame signal, the short interpolation signal is often a signal having a large noise component and a deteriorated S / N. Further, since the short interpolation signal has been subjected to the interpolation processing, it often has a false signal accompanying the interpolation processing in a high frequency band as compared with a pure frame signal. So, Mix
In the signal frequency emphasizing means 16, s in the mix signal
The frequency emphasis setting for the hort signal range is set such that noise and false signals are not emphasized compared to the frequency emphasis setting for the long signal range.

As described above, the long signal range and the short signal range for the mix signal are detected at the level of the mix signal, and the long signal range and the short signal range are detected.
Optimal frequency emphasis processing that matches the signal range and the short signal range can be performed.

As described above, in an image pickup apparatus that synthesizes a long signal (a signal for a long-time exposure) and a short signal (a signal for a short-time exposure) to create a synthesized image signal with an expanded dynamic range, the frequency of the Mix signal is emphasized. By providing means for setting the level of noise and spurious signals in the short signal range, noise and spurious signal components are reduced and S
It is possible to obtain an image with a wide dynamic range and excellent / N performance.

Note that in Embodiments 3 and 4,
The long signal (long exposure signal) is the frame signal and is short
The t signal (short exposure signal) is the field signal,
The case where the frame signal is formed by the interpolation processing has been described. In this case, both a noise component due to the short-time exposure and a false signal obtained by converting the field signal into a frame signal by interpolation processing are present in the short signal. On the other hand, it is also possible to adopt a configuration in which the short signal (short-time exposure signal) is a frame signal, and the long signal (long-time exposure signal) is a field signal, and a frame signal is obtained by interpolation. In this case, the frequency emphasis processing for suppressing the noise component for the short signal and the frequency emphasis processing for suppressing the false signal component for the long signal which has been converted into the frame signal by the interpolation processing from the field signal are performed independently.

In the third and fourth embodiments, the case has been described where the frame signal is created from the field signal by the interpolation means after being separated into the luminance signal and the color difference signal by the first digital signal processing means. A frame signal may be created from the field signal by the interpolation means in the state of the color filter signal of the element, and thereafter, processing such as color separation may be performed.

Further, linear interpolation has been described as the interpolation means. However, the present invention is not limited to this. Higher-order interpolation may be performed.

Further, in each of the above embodiments, the case where the color filters are arranged in a color difference line-sequential manner as the color filter array of the image sensor has been described. However, the present invention is not limited to this. It suffices if color separation can be performed by readout drive and frame readout drive.

Further, only the basic operation has been described for the image synthesizing method in each of the above embodiments, since the detailed operation is not directly related to the present invention.

[0130]

According to the present invention, there is provided an imaging apparatus configured to expand a dynamic range by combining an image signal having a small charge accumulation amount and an image signal having a large charge accumulation amount. By differentiating the frequency enhancement process for the image signal with a small amount and the frequency enhancement process for the image signal with a large amount of charge accumulation,
That is, in the frequency emphasizing process, by setting such that the noise component is suppressed for an image signal with a small charge accumulation amount, it is possible to obtain an image with a small noise component and excellent in S / N performance and a wide dynamic range. It becomes possible. Further, by performing interpolation on an image signal having a small charge accumulation amount, the noise component and the false signal component are reduced and the S /
It is possible to obtain an image with excellent N performance and a wide dynamic range.

[Brief description of the drawings]

FIG. 1 is a block diagram illustrating a configuration of an imaging device according to Embodiment 1 of the present invention.

FIG. 2 is a schematic diagram illustrating the configuration and operation of a solid-state imaging device according to Embodiment 1 of the present invention.

FIG. 3 is an operation timing chart relating to exposure of a subject image and reading of an exposed signal of the solid-state imaging device according to the first embodiment of the present invention;

FIG. 4 is a schematic diagram illustrating another configuration and operation of the solid-state imaging device according to Embodiment 1 of the present invention.

FIG. 5 is another operation timing chart relating to exposure of a subject image of the solid-state imaging device and reading of an exposed signal according to the first embodiment of the present invention;

FIG. 6 is a schematic diagram illustrating an example of a color filter of the solid-state imaging device according to Embodiment 1 of the present invention.

FIG. 7 is an explanatory diagram of a principle of expanding a dynamic range according to the first embodiment of the present invention.

FIG. 8 is a block diagram illustrating a configuration of an imaging device according to a second embodiment of the present invention.

FIG. 9 is an explanatory diagram of a Mix signal frequency emphasis unit according to the second embodiment of the present invention.

FIG. 10 is another explanatory diagram of the Mix signal frequency emphasizing means according to the second embodiment of the present invention.

FIG. 11 is a diagram illustrating image synthesis according to the second embodiment of the present invention.

FIG. 12 is a block diagram illustrating a configuration of an imaging device according to a third embodiment of the present invention.

FIG. 13 is a schematic diagram illustrating a configuration and operation of a solid-state imaging device according to Embodiment 3 of the present invention.

FIG. 14 is an operation timing chart relating to exposure of a subject image of a solid-state imaging device and reading of an exposed signal according to Embodiment 3 of the present invention;

FIG. 15 is an explanatory diagram of a short signal read in the field read mode according to the third embodiment of the present invention.

FIG. 16 is an explanatory diagram of a long signal read in a frame read mode in Embodiment 3 of the present invention.

FIG. 17 is an explanatory diagram of an interpolation unit according to the third embodiment of the present invention.

FIG. 18 is a block diagram illustrating a configuration of an imaging device according to a fourth embodiment of the present invention.

[Explanation of symbols]

 DESCRIPTION OF SYMBOLS 1 ... Optical lens 2 ... Mechanical shutter for optical diaphragms 3 ... Solid-state image sensor 4 ... Analog signal processing means 5 ... A / D conversion means 6 ... Image memory 7 ... First digital signal processing means (I) 8 Long signal frequency emphasizing means 9 Short signal frequency emphasizing means 10 Image synthesizing means 11 Second digital signal processing means (II) 12 Aperture drive control means 13 Solid-state imaging Element drive control means 14 ... Charge accumulation amount control means 15 ... System control means 17 ... Mechanical shutter that also serves as an optical diaphragm 18 ... Interpolation means 19 ... Shutter drive control means 20 ... Charge accumulation amount control means

──────────────────────────────────────────────────続 き Continued on the front page (51) Int.Cl. ⁷ Identification symbol FI theme coat ゛ (Reference) H04N 5/335 H04N 5/335 PF term (Reference) 5B047 BB01 CA06 DA10 DC20 5C021 PA79 PA85 RB07 XA14 XB03 YA01 5C022 AB12 AB17 AB37 AC42 AC52 AC69 5C023 AA01 AA11 AA37 BA13 CA03 DA04 EA03 EA10 5C024 CX11 CX47 EX31 EX34 GY01 GY04 HX28

Claims (20)

[Claims] An image pickup apparatus for expanding a dynamic range by synthesizing an image signal having a small charge accumulation amount and an image signal having a large charge accumulation amount, wherein the image signal having a small charge accumulation amount and the charge accumulation amount are provided. An image pickup apparatus characterized in that the frequency emphasis processing for an image signal with a large number of signals is different from each other. 2. The image pickup apparatus according to claim 1, wherein the emphasis of the set frequency component of the image signal having the small charge accumulation amount is weakened, and the emphasis of the set frequency component of the image signal having the large charge accumulation amount is increased. apparatus. 3. The image signal having a small charge accumulation amount and the image signal having a large charge accumulation amount are synthesized after the frequency emphasizing process.
An imaging device according to claim 1. 4. The image signal according to claim 1, wherein the image signal having a small charge storage amount and the image signal having the large charge storage amount are combined before the frequency emphasizing process.
An imaging device according to claim 1. 5. A driving method for acquiring an image signal having a small charge accumulation amount and a driving method for acquiring an image signal having a large charge accumulation amount are different from each other. The imaging device according to any one of claims 1 to 4. 6. The image pickup apparatus according to claim 5, wherein the signal formats of the image signals having the different charge accumulation amounts different from each other in the driving method are unified before the image signals are combined. 7. An image pickup device, charge storage amount control means for controlling the charge storage amount of the image pickup device, image pickup means for obtaining a plurality of image signals having different charge storage amounts by the charge storage amount control means, A frequency emphasizing means for emphasizing a set frequency component of an output image signal of the means, and an image synthesizing means for synthesizing an output image signal of the frequency emphasizing means to generate a synthesized image signal, wherein the frequency emphasizing means has a charge storage amount. An image pickup apparatus characterized in that the frequency enhancement processing for an image signal with a small amount of charge and the frequency enhancement processing for an image signal with a large charge accumulation amount are different from each other. 8. An image pickup device, charge storage amount control means for controlling the charge storage amount of the image pickup device, imaging means for obtaining a plurality of image signals having different charge storage amounts by the charge storage amount control means, Image synthesizing means for synthesizing an output image signal of the means to generate a synthetic image signal; and frequency emphasizing means for emphasizing a set frequency component of the synthetic image signal, wherein the frequency emphasizing means comprises a charge in the synthetic image signal. An image pickup apparatus characterized in that a frequency enhancement process for an image signal portion with a small accumulation amount and a frequency enhancement process for an image signal portion with a large charge accumulation amount are different from each other. 9. An image pickup device, charge storage amount control means for controlling the charge storage amount of the image pickup device, image pickup device drive control means of the image pickup device having at least two different driving systems, and the charge storage amount Imaging means for obtaining a plurality of image signals having different charge accumulation amounts by the control means and different driving methods by the imaging element drive control means; Means for synthesizing an output image signal of the frequency emphasizing means to generate a synthesized image signal, wherein the frequency emphasizing means performs a frequency emphasizing process on an image signal with a small charge accumulation amount and a large charge accumulation amount. An imaging apparatus characterized in that frequency enhancement processing for an image signal is different from each other. 10. An image pickup device, charge accumulation amount control means for controlling the charge accumulation amount of the image pickup device, image pickup device drive control means for the image pickup device having at least two different driving systems, and the charge accumulation amount An imaging unit that obtains a plurality of image signals having different charge accumulation amounts by a control unit and different driving methods by the imaging device drive control unit; and a composite image signal generated by combining output image signals of the imaging unit. An image synthesizing unit; and a frequency emphasizing unit for emphasizing a set frequency component of the synthesized image signal, wherein the frequency emphasizing unit performs a frequency emphasizing process and an electric charge accumulation on an image signal portion having a small charge accumulation amount in the synthesized image signal. An image pickup apparatus characterized in that frequency emphasis processing for an image signal portion having a large amount is different from each other. 11. The frequency emphasizing means is configured to weaken emphasis on a set frequency component of an image signal having a small charge accumulation amount and to enhance emphasis on a set frequency component of an image signal having a large charge accumulation amount. The imaging device according to claim 7 or 9, wherein 12. The frequency emphasizing means weakens emphasis on a set frequency component of an image signal portion having a small charge accumulation amount in the composite image signal and increases emphasis on a set frequency component of an image signal portion having a large charge accumulation amount. The imaging device according to claim 8, wherein the imaging device is configured as described above. 13. The set frequency of an image signal portion having a small charge accumulation amount in the synthesized image signal using image synthesis ratio information of a plurality of image signals having different charge accumulation amounts in the image synthesizing means. Weaken the emphasis on the ingredients,
The imaging device according to claim 8, wherein the imaging device is configured to enhance emphasis on a set frequency component of an image signal portion having a large charge accumulation amount. 14. The image pickup device drive control means is configured to include a drive system for generating a field signal and a drive system for generating a frame signal as the different drive systems. The imaging device according to claim 9. 15. The image pickup device drive control means is configured to include an interlace drive method and a frame readout control drive method having a light blocking means as the different drive methods. Item 9
Alternatively, the imaging device according to claim 10. 16. The image pickup device according to claim 1, wherein said image pickup device is configured to obtain an image signal by an interlaced driving method having a small charge accumulation amount and an image signal by a frame readout control driving method having a light shielding portion having a large charge accumulation amount. The imaging device according to claim 9 or 10, wherein 17. The image pickup means generates a frame image signal based on one of two different drive modes, a frame read control drive mode, and a field image signal based on another interlace drive mode, and interpolates the field image signal. The imaging apparatus according to claim 9, further comprising an interpolation unit configured to convert the image into a frame image signal. 18. The apparatus according to claim 1, wherein said frequency emphasis means is configured to weaken emphasis on a set frequency component of said interpolated frame image signal and to enhance emphasis on a set frequency component of said frame image signal. 18. The imaging device according to 17. 19. The image pickup means generates a frame image signal by one of two different driving methods, a frame read control driving method, and a field image signal by another interlacing driving method, and interpolates the field image signal. The imaging apparatus according to claim 10, further comprising an interpolation unit that converts the image into a frame image signal. 20. The frequency emphasizing means is configured to weaken the emphasis on the set frequency component of the interpolated frame image signal portion in the synthesized image signal and increase the emphasis on the set frequency component of the frame image signal portion. The imaging device according to claim 19, wherein: