JP2001339633A - Solid image pickup device for still image - Google Patents

Abstract

PROBLEM TO BE SOLVED: To realize enlargement of dynamic range of an image which has good quality with stable and continuous gray scale characteristic regardless of change of condition and status of image capturing. SOLUTION: A mechanical shutter is used both as a shutter and an optical iris. Image signal captured by a solid image pickup element 2 is separated into short and long signal and outputted individually. An adder for two horizontal lines 3 adds two lines adjacent vertically and stores short and long signal in an image memory 4 temporally. Short and long signal is read from the image memory 4 and sent to a means for detecting exposure quantity ratio 6. A long signal integral means 62 and a short signal integral means 63 integrate long and short signal and obtain integrated values LSUM, SSUM. A means for calculating exposure quantity ratio 64 obtains exposure quantity ratio D. Exposure quantity ratio D is sent to a gain adjusting means 7. The means 7 multiplies short signal from the image memory 4 by D. An image synthesizing means 8 synthesizes long signal from the image memory 4 and short' signal adjusted gain.

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a still image constructed so as to expand a dynamic range of a photographed image by synthesizing a plurality of image signals having different exposure amounts, that is, a long exposure signal and a short exposure signal. The present invention relates to a solid-state imaging device.

[0002]

2. Description of the Related Art Two images of a short-time exposure signal and a long-time exposure signal are photographed using a solid-state image sensor capable of outputting a plurality of image signals having different exposure amounts, and these images are synthesized. A method for capturing an image with an expanded dynamic range is known.

The principle of such a dynamic range expansion will be described with reference to FIG. FIGS. 13A and 13B show the relationship between the brightness of the subject (the amount of light incident on the solid-state imaging device) at the time of exposure and the signal amount output from the solid-state imaging device. As shown in FIG. 13A, during long-time exposure, the amount of charge generated on the photodiode of the solid-state imaging device due to incident light is large, and naturally the amount of output signal is also large. However, there is an upper limit on the amount of electric charge stored in the photodiode. If the upper limit is exceeded, saturation, that is, a phenomenon in which a signal is broken occurs, and an object image cannot be accurately reproduced. Conversely, if the exposure time is set short as shown in FIG. 13B, it is possible to avoid saturation, but this time, the low-luminance portion in the subject is lost in black, and S
/ N deteriorates.

Therefore, using a signal obtained by long-time exposure (long signal) and a signal obtained by short-time exposure (short signal), an image composed of a long signal is adopted for a low-luminance part, and a high-luminance part is used. If an image composed of short signals is adopted and the two are combined, it is possible to reproduce the low-luminance part to the high-luminance part of the subject, and it is possible to expand the dynamic range of the imaging device. At this time, as shown in FIG.
For the ort signal, the ratio of the exposure amount to the long signal (ratio of the exposure time)
If the synthesis is performed after multiplying the gain corresponding to
As shown in (d), the dynamic range can be expanded in accordance with the exposure amount ratio. For example, when the exposure amount ratio (exposure time ratio) between the short signal and the long signal is 1: D, the dynamic range can be expanded to D times by synthesizing the short signal by D times.

[0005]

However, when an optical diaphragm is used also as a mechanical shutter as means for controlling the shutter time of a long signal or a short signal, the closing operation of the mechanical shutter is performed as shown in FIG. The time required from the start of the closing operation to the full closing differs depending on the degree of opening of the optical diaphragm immediately before the operation is performed, and the change in the incident exposure during the closing operation is complicated, and may vary depending on the situation. . That is, the exposure amount a of the long signal when the aperture of the optical diaphragm shown in FIG.
The exposure ratio a / b multiplied by the exposure b of the signal and FIG.
The exposure ratio a '/ b' obtained by multiplying the exposure amount a 'of the long signal by the exposure amount b' of the short signal when the aperture of the optical diaphragm shown in FIG. Therefore, it is difficult to accurately estimate the exposure amount incident on the solid-state imaging device during the closing operation period. Further, the ratio between the exposure amounts of the long signal and the short signal cannot be simply obtained as the ratio of the exposure time.

Further, when an object is photographed under illumination such as a fluorescent lamp whose luminance level changes periodically, FIG.
As shown in (2), the ratio of the exposure amount of the long signal and the exposure amount of the short signal varies depending on the timing of photographing. Similarly, the ratio of the exposure amount cannot be simply obtained as the ratio of the exposure time.

If an accurate gain corresponding to the ratio of the exposure amount cannot be given to the short signal due to the above-described factors, the combined image signal after the image combining process is performed as shown in FIG.
As shown in (1), there is a problem that the gradation characteristic becomes discontinuous near the incident exposure amount L1 at which the long signal is saturated, and as a result, there is a problem that the image quality is deteriorated.

SUMMARY OF THE INVENTION The present invention has been made to solve the above-mentioned problem, and has excellent image quality with a stable and continuous gradation characteristic at all times irrespective of fluctuations in imaging conditions and conditions. The purpose is to realize a wider range.

[0009]

SUMMARY OF THE INVENTION The present invention considers that the cause of the above-mentioned problem lies in the use of a fixed exposure amount ratio, and it is possible to deal with fluctuations in imaging conditions and conditions. This is related to the above problem by devising adaptive gain adjustment. That is, the present invention for a still image solid-state imaging device configured to expand a dynamic range by synthesizing a long-time exposure signal and a short-time exposure signal obtained by imaging takes the following means. Thus, the above problem is solved.

The basis of the synthesis of the long-time exposure signal and the short-time exposure signal is the exposure amount ratio of the two signals, but the exposure amount ratio varies depending on the conditions and conditions such as the degree of opening of the optical diaphragm and the type of light source. there's a possibility that. That is, if the exposure ratio is fixedly determined in advance, it is impossible to cope with the actual fluctuation of the exposure ratio. Therefore, in the present invention, the exposure amount ratio between the long-time exposure signal and the short-time exposure signal obtained by imaging is actually detected. That is, it is actually measured. In this way, the actual exposure ratio according to the actual conditions such as conditions and situations is obtained, and based on such actual exposure ratio, the long exposure signal and the short exposure signal for dynamically expanding the dynamic range are adaptively determined. After the gain of the exposure signal is adjusted, the long-time exposure signal and the short-time exposure signal are combined.

As described above, in the present invention, the adaptive gain adjustment between the long-time exposure signal and the short-time exposure signal is performed based on the actual exposure amount ratio according to the actual conditions such as conditions and conditions. Since the image synthesis of the two signals is performed, the tone characteristics of the synthesized image can be stably continued, and the quality of the image can be improved.

For example, when the exposure ratio fluctuates due to a time difference until the shutter is fully closed due to a difference in opening degree of the optical diaphragm, such as when the optical diaphragm is also used as a mechanical shutter, or when a fluorescent lamp is used. Even when the exposure amount ratio fluctuates due to the difference in shooting timing, such as when shooting under illumination where the luminance level changes periodically,
It is possible to realize a dynamic range expansion with excellent image quality having stable and continuous gradation characteristics.

[0013]

BEST MODE FOR CARRYING OUT THE INVENTION Hereinafter, embodiments of the present invention will be generally described.

[0014] The still image solid-state imaging device of the first invention of the present application comprises:
A still image solid-state imaging device that expands a dynamic range by synthesizing a long-time exposure signal and a short-time exposure signal obtained by imaging, wherein an exposure amount ratio between the long-time exposure signal and the short-time exposure signal is determined. It is characterized in that the gain of the long-time exposure signal and the short-time exposure signal in the dynamic range expansion is adjusted based on the detected exposure amount ratio.

The operation of the first invention is as follows. Instead of using a fixed exposure ratio fixed in advance, the exposure ratio of both signals was detected in real time and dynamically from the actual long exposure signal and the short exposure signal obtained by imaging, and the actual measurement was performed. Since the gain adjustment between the long-time exposure signal and the short-time exposure signal is performed based on the exposure amount ratio, the gain adjustment becomes accurate in accordance with the actual conditions and conditions. As a result, the brightness level may be changed due to the time difference until the shutter is fully closed due to a difference in opening degree of the optical aperture, such as when the optical aperture is also used as a mechanical shutter, or periodically, such as a fluorescent lamp. Even when the exposure ratio fluctuates due to the difference in the imaging timing as in the case of imaging under changing illumination, the exposure amount in accordance with the actual conditions such as the actually measured imaging conditions and conditions described above in real time and dynamically. The synthesized image obtained by synthesizing the long-time exposure signal and the short-time exposure signal in a state where the gain is adjusted based on the ratio is an image with an extended dynamic range of excellent image quality having stable continuous gradation characteristics. It is possible to do.

A still image solid-state imaging device according to a second aspect of the present invention is:
In the above first invention, the detection of the exposure ratio of the long-time exposure signal and the short-time exposure signal is performed for each block divided and formed on the imaging screen, and the gain adjustment is performed for each block. I have.

The operation of the second invention is as follows. In the same imaging screen, when there are a plurality of light sources such as a light source whose luminance level changes periodically such as a fluorescent lamp and a light source such as sunlight having no such change, each light source is illuminated. In the plurality of subject portions described above, there is a difference in the exposure ratio between the long exposure signal and the short exposure signal. In such a case, if the gain is adjusted based on the exposure amount ratio of both signals by actual measurement in the entire screen area, that is, the average or total exposure amount ratio, and the long exposure signal and the short exposure signal are combined. The accuracy of the exposure ratio is degraded for any one or any one of the plurality of types of subject portions by the plurality of types of light sources. Therefore, when the imaging screen is divided into blocks, the type of light source irradiation light for each block can be divided for each block. Then, the exposure ratio of both signals is actually measured in block units, and gain adjustment is performed based on the highly accurate exposure ratio measured in block units, and then the long exposure signal and short exposure signal are combined. The image can be an image having a dynamic range expanded with excellent image quality and stable continuous tone characteristics in a state corresponding to the light source type in block units.

A still image solid-state imaging device according to a third aspect of the present invention includes:
In the first aspect, detection of an exposure amount ratio between the long exposure signal and the short exposure signal is performed for each color component,
The gain adjustment is performed for each color component.

The operation of the third invention is as follows. The fluorescent lamp has a different brightness phase for each color component, and the exposure ratio between the long exposure signal and the short exposure signal differs for each color component depending on the timing of exposure. In such a case, when the gain is adjusted based on the exposure ratio of both signals obtained by simply measuring the luminance component and the long exposure signal and the short exposure signal are combined, the exposure ratio becomes The accuracy will be degraded. Therefore, the exposure ratio between the long-time exposure signal and the short-time exposure signal is actually measured for each color component, and the gain adjustment is performed based on the highly accurate exposure ratio actually measured for each color component. A composite image of the exposure signal and the short-time exposure signal can be an image having a dynamic range expanded with excellent image quality and a stable continuous gradation characteristic regardless of exposure timing.

A still image solid-state imaging device according to a fourth aspect of the present invention is:
In the first aspect, the detection of the exposure amount ratio between the long-time exposure signal and the short-time exposure signal is performed for each block divided and formed on the imaging screen and for each color component, and the gain adjustment is performed for each block. In addition, it is performed for each color component. This corresponds to a hybrid of the second and third aspects.

The operation of the fourth invention is as follows. The exposure ratio between the long-time exposure signal and the short-time exposure signal is actually measured for each block and each color component divided on the imaging screen, and the gain is determined based on the highly accurate exposure ratio for each block and each color component. The synthesized image of the long-time exposure signal and the short-time exposure signal after adjustment has stable and continuous gradation characteristics regardless of the type of light source and the timing of exposure. This makes it possible to form an image with a high dynamic range and excellent image quality.

According to a fifth aspect of the present invention, there is provided a still image solid-state imaging device comprising:
An imaging unit for independently outputting a plurality of image signals having different exposure amounts, an image memory storing a plurality of image signals having different exposure amounts from the imaging unit, and an exposure amount ratio of the plurality of image signals having different exposure amounts Exposure-ratio detecting means for detecting the exposure-ratio, gain-adjusting means for performing gain adjustment on a plurality of image signals having different exposure amounts read from the image memory based on the detected exposure-ratio, and after the gain adjustment Image synthesizing means for synthesizing a plurality of image signals having different exposure amounts. This corresponds to a more specific description of the first invention.

The operation of the fifth invention is as follows. The short-time exposure signal acquired during a certain period of time by the imaging means is transferred and stored in the image memory. In addition, the long-time exposure signal acquired by the imaging means during another period is also transferred and stored in the image memory. The order of acquiring the short-time exposure signal and the period of acquiring the long-time exposure signal may be either earlier or later. Both the short-time exposure signal and the long-time exposure signal temporarily stored in the image memory are sent to the exposure amount ratio detecting means. The exposure ratio detecting means detects an exposure ratio between the actually obtained long exposure signal and short exposure signal. The exposure amount ratio is not an exposure amount ratio fixedly determined in advance as in the case of the related art, but an exposure amount ratio based on real-time and dynamic conditions such as imaging conditions and conditions. In synthesizing the long-time exposure signal and the short-time exposure signal in the image synthesizing means, gain adjustment for both signals is performed based on the adaptive exposure amount ratio, but gain adjustment based on the exposure amount ratio is also performed. , Real-time dynamics according to the actual conditions such as imaging conditions and conditions.

Therefore, the luminance level is periodically changed due to the difference in the opening degree of the optical aperture, such as when the optical aperture is also used as a mechanical shutter, or until the shutter is fully closed, or periodically as in a fluorescent lamp. Even when the exposure ratio fluctuates due to the difference in the shooting timing as in the case of shooting under changing lighting, the gain is adjusted based on the exposure ratio adaptively measured as described above. A composite image obtained by combining the long-time exposure signal and the short-time exposure signal by the image synthesizing means can be an image having an extended dynamic range with excellent image quality and stable continuous gradation characteristics. Become.

A still image solid-state imaging device according to a sixth aspect of the present invention includes:
In the fifth aspect of the present invention, the exposure amount ratio detecting means includes:
Signal level determining means for determining signal levels of the plurality of image signals having different exposure amounts, integration means for obtaining an integral value for each of the plurality of image signals having different exposure amounts within a predetermined range by the level determination; And an exposure amount ratio calculating means for obtaining a ratio of a plurality of integral values.

The operation of the sixth invention is as follows. A plurality of image signals having different exposure amounts sent from the image memory to the exposure ratio detecting means are first input to the signal level determining means. In the signal level determination unit, for example, a high-luminance portion near the saturation level of the imaging unit is excluded for the long-time exposure signal, and a low-luminance portion where the influence of noise is large is excluded for the short-time exposure signal. In this way, after removing factors that degrade the accuracy because the exposure ratio is at a level where it is difficult to obtain the exposure amount ratio, the exposure ratio is sent to each integrating means. In this case, it is preferable that the excluded level is common to the long-time exposure signal and the short-time exposure signal. When the integration value of each image signal is obtained by each integration means, it corresponds to the exposure amount of each image signal. Then, the integrated value of each integrating means is sent to the exposure ratio calculating means, and the exposure ratio is obtained by division or the like. This makes it possible to actually measure a high-accuracy exposure amount ratio based on real-time dynamics based on actual conditions such as imaging conditions and conditions. Therefore, the gain adjustment is also performed with high precision, and in obtaining a synthesized image with an expanded dynamic range, stable continuity of the gradation characteristics can be achieved with high precision.

A still image solid-state imaging device according to a seventh aspect of the present invention comprises:
In the sixth aspect, the integration means is configured to calculate an integration value for each of the plurality of image signals having different exposure amounts in block units obtained by dividing a screen into a plurality of blocks.

The operation of the seventh aspect is as follows. As described above, when irradiating with a plurality of types of light sources having different aspects such as a fluorescent lamp and sunlight,
There is a difference in the exposure ratio between the long-time exposure signal and the short-time exposure signal for each part in the same imaging screen. Therefore, individual integrating means corresponding to each of the blocks obtained by dividing the imaging screen into blocks are provided, and the integrating means for each block calculates an integrated value for each of a plurality of image signals having different exposure amounts. The exposure ratio calculating means calculates the exposure ratio for each block. Gain adjustment is also performed in block units. As a result, the following advantages are obtained. That is, when the gain adjustment is performed based on the exposure ratio of both signals by the actual measurement in the entire screen area, that is, the average or overall exposure ratio, and the long exposure signal and the short exposure signal are combined, Although the exposure ratio is degraded in accuracy for any one or any of the plurality of types of subject portions by the type light source, the gain adjustment is performed based on the highly accurate exposure ratio actually measured in block units. Since the long-time exposure signal and the short-time exposure signal are synthesized, the synthesized image has an image quality with stable and continuous gradation characteristics in a state corresponding to the light source type in block units. An image with an excellent dynamic range can be obtained.

The still image solid-state imaging device according to the eighth aspect of the present invention comprises:
In the sixth aspect, the image pickup means includes a color separation means using a color separation filter or a color separation prism having different spectral characteristics, and the integration means includes a plurality of light sources having different exposure amounts for each color component. Is configured to obtain an integral value for each image signal.

The operation of the eighth aspect is as follows. As described above, the fluorescent lamp has a different brightness phase for each color component, and the exposure ratio between the long-time exposure signal and the short-time exposure signal differs for each color component depending on the timing of exposure. Therefore, an individual integrating means is provided for each color component, and the integrating means for each color component calculates an integrated value for each of a plurality of image signals having different exposure amounts. The exposure ratio calculating means calculates an exposure ratio for each color component. Gain adjustment is also performed for each color component. As a result, the following advantages are obtained. That is, if the gain adjustment is performed based on the exposure ratio of the two signals based on the mere measurement of the luminance component to synthesize the long exposure signal and the short exposure signal, the exposure ratio deteriorates in accuracy due to the timing of the exposure. However, since the long-time exposure signal and the short-time exposure signal after performing gain adjustment based on the highly accurate exposure ratio actually measured for each color component are combined, the combined image is In addition, it is possible to form an image having an extended dynamic range with excellent image quality and having stable and continuous gradation characteristics in a state corresponding to the light source type for each color component.

A still image solid-state imaging device according to a ninth invention of the present application is:
In the sixth aspect, the imaging unit has a color separation unit using a color separation filter or a color separation prism having different spectral characteristics, and the integration unit is a block unit that divides a screen into a plurality of blocks. In addition, an integrated value is obtained for each of a plurality of image signals having different exposure amounts for each color component.

According to the ninth aspect, a combined operation of the seventh and eighth aspects can be obtained. That is, the combined image of the long-time exposure signal and the short-time exposure signal after gain adjustment is performed based on the high-accuracy exposure amount ratio for each block and each color component, regardless of the type of light source, Irrespective of the timing of exposure, it is possible to form an image having a stable and continuous tone characteristic and an extended dynamic range with excellent image quality.

A still image solid-state imaging device according to a tenth aspect of the present invention is the still image solid-state imaging device according to the fifth to ninth aspects, wherein any one of the plurality of image signals having different exposure amounts transmitted from the imaging means to the image memory is provided. An image signal switching means for switching one of the image signals between the image signal from the image pickup means and the image signal from the image memory and sending it to the exposure amount ratio detecting means is provided.

The operation of the tenth aspect is as follows. When transmitting both the short-time exposure signal and the long-time exposure signal to the exposure ratio detecting means, if both the short-time exposure signal and the long-time exposure signal are read out from the image memory and transmitted, the short-time exposure from the imaging means is performed. Both the signal and the long-time exposure signal need to be temporarily stored in the image memory. That is, the transfer is performed after both signals have been stored in the image memory. In this case, it is necessary to wait for the completion of storage of both signals before transferring to the exposure amount ratio detecting means, and the time required for processing tends to be long. Therefore, one of the exposure signals is stored in the image memory, and the other is sent to the exposure ratio detecting means in parallel with the storage in the image memory. At the same time, the one exposure signal is read from the image memory and sent to the exposure ratio detecting means. As a result, the timing of transferring both signals to the exposure ratio detecting means is advanced, and the time required for the process of detecting the exposure ratio can be shortened.

In a still image solid-state imaging device according to an eleventh aspect of the present invention, in the fifth to tenth aspects, the gain adjusting means for the plurality of image signals having different exposure amounts is detected by the exposure amount ratio detecting means. The gain based on the calculated exposure amount ratio is provided to at least one of the plurality of image signals having different exposure amounts.
As a gain to be calculated based on the exposure ratio, as a result, continuity of gradation characteristics in expanding the dynamic range may be ensured, and it may be a gain for multiplying only the short-time exposure signal. Or, conversely, a gain for multiplying only the long-time exposure signal may be used, or a gain for multiplying the short-time exposure signal and a gain for multiplying the long-time exposure signal may be calculated separately. It is. This eleventh invention makes this point clear.

Generally, the exposure amount of the long exposure signal is larger than the exposure amount of the short exposure signal. Therefore, assuming that a value obtained by dividing the exposure amount of the long exposure signal by the exposure amount of the short exposure signal is an exposure ratio D, and that the exposure ratio D itself is a gain, the 1 becomes the same for the long exposure signal. After multiplying the short-time exposure signal by a D-fold gain, the two signals are combined. In the second method, the short-time exposure signal is left as it is, the long-time exposure signal is multiplied by a gain of 1 / D, and then the two signals are combined. Part 3
Is expressed as D = n / m, where m and n are arbitrary real numbers, and after multiplying the long-time exposure signal by an m-fold gain and multiplying the short-time exposure signal by an n-fold gain, Combine the signals. Note that 1 corresponds to the case where m = 1 and n = D, and 2 corresponds to the case where n = 1 and m = 1 / D.

(Specific Embodiment) Hereinafter, a specific embodiment of the still image solid-state imaging device according to the present invention will be described in detail with reference to the drawings.

(Embodiment 1) This embodiment 1 relates to a method of using a plurality of image signals having different exposure amounts and capturing an image having an expanded dynamic range by synthesizing the plurality of image signals. In the case where control is performed by using a mechanical shutter also serving as an optical diaphragm, or as illumination of the subject, the subject is photographed under illumination conditions of a light source such as a fluorescent lamp whose luminance level changes periodically. In such a case, even when the ratio of the exposure amounts of a plurality of signals having different exposure amounts cannot be simply obtained as the ratio of the exposure time, the exposure of the entire exposure area in each pixel of the long exposure signal and the short exposure signal is performed. By actually obtaining the quantitative ratio, the synthesized image after the image synthesizing process has stable and continuous gradation characteristics.

FIG. 1 is a block diagram showing a configuration of a still image solid-state imaging device according to Embodiment 1 of the present invention. FIG.
In the figure, reference numeral 1 denotes a mechanical shutter that also serves as an optical diaphragm. Reference numeral 2 denotes a solid-state imaging device, which is assumed to be an all-pixel readout type CCD (charge coupled device) in the first embodiment. Reference numeral 3 denotes two horizontal line adding means for adding two horizontal lines to the output from the solid-state imaging device 2, reference numeral 4 denotes an image memory for storing two frames of signals after addition of the horizontal lines from the solid-state imaging device 3, and reference numeral 5 denotes Shutter drive control means for controlling the opening and closing of the mechanical shutter 1; exposure amount ratio detection means 6 for obtaining an exposure ratio of a plurality of image signals having different exposure amounts; and gain adjustment for a plurality of image signals having different exposure amounts. And 8 is an image synthesizing means for synthesizing images to expand the dynamic range.

The reference numeral 61 in the block of the exposure amount ratio detecting means 6 denotes a signal level determining means, 62 denotes a long signal integrating means, 63 denotes a short signal integrating means, and 64 denotes a long screen integrating means by a long signal integrating means 62. LSUM and short at
Exposure ratio calculating means for calculating the ratio (LSUM / SSUM) to the integrated value SSUM in the entire screen area by the t signal integrating means 63.

FIG. 2 is a block diagram showing a configuration example of the two horizontal line addition means 3 in the block diagram of FIG. In the figure, reference numeral 31 denotes a one-line memory, which is means for delaying one line of an image signal output from the solid-state imaging device 2 by one horizontal synchronization period. Reference numeral 32 denotes an adder. The adder 32 adds the horizontal line signal delayed in the one-line memory 31 and the horizontal line signal input to the two-horizontal line adder 3, and
Addition of two lines above and below is performed.

FIG. 3 shows a configuration example of the image memory 4 in the block diagram of FIG. The image memory 4 needs a memory capacity capable of storing two frames.
As shown in FIG.
It has two frame memories, that is, a t-signal frame memory 41b, and can be controlled so that the long signal and the short signal can be stored independently in each frame memory.

The operation of the still image solid-state imaging device according to the first embodiment configured as described above will be described below.

In the first embodiment, a specific example will be described in which two images of a short-time exposure signal (short signal) and a long-time exposure signal (long signal) are taken and synthesized.

First, a method of capturing a short signal and a long signal will be described with reference to FIG. FIG. 4 shows the solid-state imaging device 2
4 is a timing chart for exposure of a subject image, reading of an exposed signal, and reading and writing operations of the image memory 4 in FIG. 4, reference numeral 401 denotes the timing of a vertical synchronization signal; 402, the timing of a read control pulse for controlling the reading of signal charges from the photodiode of the solid-state imaging device 2; 403, the timing indicating the open / close state of the mechanical shutter 1; Timing 405 indicating the amount of exposure signal on the photodiode of the solid-state imaging device 2
Are the timings of the signals output from the solid-state imaging device 2;
06 is the timing of the input (write) signal of the image memory 4, 407 is the timing of the output (read) signal from the short signal storage unit (short signal frame memory 41b) of the image memory 4, and 408 is the long signal of the image memory 4. This is the timing of the output (read) signal from the storage section (long signal frame memory 41a).

At the time of the exposure of the short signal, the mechanical shutter 1 is opened, and a necessary exposure time, for example, 1/1000 second exposure is performed by using the electronic shutter function. 1/100
After the 0 second exposure is completed, the charge stored on the photodiode is moved to the vertical transfer CCD by the read control pulse. At this time, the solid-state imaging device 2 is driven in the all-pixels reading mode.

Next, after the short signal is moved to the vertical transfer CCD, exposure of the long signal is performed. The exposure time of the long signal is, for example, 1/100 second. The exposure time of the long signal is controlled by opening and closing the mechanical shutter 1. In parallel with this, a short signal is output from the solid-state imaging device 2 for one frame.
For the ng signal, the accumulated charge on the photodiode is transferred to the vertical transfer CCD. At this time, the solid-state imaging device 2 is driven in the all-pixels reading mode. Note that the period of the vertical synchronization signal is, for example, 1/30 second, and the signal readout for one frame is performed by one of the vertical synchronization signal.
It shall be completed within a cycle.

The two signals of the short signal and the long signal having different exposure times obtained by the solid-state image pickup device 2 are individually individually processed by the two horizontal line adding means 3.
After the two horizontal line addition processing of adding and mixing the signals of the upper and lower adjacent two lines, as shown at timing 406, the short signal is stored in the image memory 4 during the period
The long signal is temporarily stored in the signal frame memory 41b, and the long signal is temporarily stored in the long signal frame memory 41a of the image memory 4 during the period.

Next, from the image memory 4, 40 in FIG.
At the timings shown in timings 7 and 408,
Is read out during the period.

During the period, the exposure amount ratio detecting means 6 causes the short signal (short frame signal) and the long signal (long
By integrating the exposure of each pixel in the entire screen area of each frame signal), the total exposure (SSUM, LS
UM).

During the period, the image memory 4
As for the timing for reading the t signal and the long signal, the lines at the corresponding positions on the solid-state imaging device 2 are sequentially read from the first line so as to be output at the same timing. Here, sh read from the image memory 4 during the period
The ort signal and the long signal are input to the exposure ratio detecting means 6. In the exposure amount ratio detecting means 6, the short signal and the long signal
In order to detect the exposure ratio on the whole screen of the signal, short
The integral value of the exposure amount of each pixel in the entire screen area of the signal and the long signal is calculated, and the ratio of the integrated value (LSUM / SS
UM).

Each of the long signal and the short signal input to the exposure ratio detecting means 6 in line synchronization is subjected to level judgment by the signal level judging means 61, and the signals judged to be out of a predetermined range are determined. , Long signal integrator 62 and short signal integrator 63 negate so as not to perform integration. Here, the level outside the predetermined range is a level at which it is difficult to determine the ratio of the exposure amount. Specifically, for the long signal, the signal of the high luminance portion near the saturation level of the solid-state imaging device 2 is used. The short signal is a signal in a low-luminance part where the influence of noise acts greatly.

FIG. 5 is a block diagram showing the structure of the signal level determining means 61. Here, reference numeral 6101 in FIG. 5 denotes means for determining the level of the long signal (comparator A), reference numeral 6102 denotes means for determining the level of the short signal (comparator B), reference numeral 6103 denotes an OR gate, and reference numeral 610.
4 is a gate means for gating a long signal, 6105 is
This is a gate means for applying a gate to a short signal. In the means 6101 for determining the level of the long signal, the high-brightness signal near the saturation level of the solid-state imaging device 2 is gated by the gate means 6104 so that the long signal integrator 62 does not integrate the signal, and The signal is also gated by the gate means 6105 and sh
o The rt signal integrating means 63 does not integrate.

Means 610 for determining short signal level
In 2, the gate means 6105 gates a low-luminance signal that is greatly affected by noise, and
The signal integration means 63 does not integrate, and the long signal of the corresponding pixel is gated by the gate means 6104 so that the long signal integration means 62 does not integrate.

The integrated value LSUM resulting from integration by the long signal integrator 62 and the integrated value SSUM resulting from integration by the short signal integrator 63 are input to the exposure ratio calculator 64 over the entire area of one screen. Then, by performing the calculation shown in the following equation (1), the exposure amount ratio D in the entire screen area is obtained.

D = LSUM / SSUM …………………………… (1) By the above method, the exposure amount of the short signal whose exposure amount is controlled by the electronic shutter and In addition, the exposure amount ratio D in the average of the entire screen with the long signal whose exposure amount is controlled by the mechanical shutter 1 can be obtained very accurately.

Even when the illumination of the subject is a fluorescent light,
Since the exposure ratio of the short signal and the long signal can be obtained by the same method and timing as described above, the description in this case is omitted.

Next, in the period shown in FIG.
The long signal and the short signal are read out from the image memory 4 in the same manner as in the period of (1), and the gain adjusting means 7 multiplies the gain by D times corresponding to the exposure amount ratio D calculated in the period of the short signal.

The signal after the D-fold gain is given to the short signal is hereinafter referred to as a short 'signal. Based on the long signal read from the image memory 4 and the gain 'adjusted short' signal, the image is synthesized by the subsequent image synthesizing means 8 by the method shown in the principle of dynamic range expansion described with reference to FIG. .

As described above, the ratio between the exposure amounts of the long signal and the short signal can be obtained with high accuracy.
The synthesized image by the image synthesizing means 8 using the signal and the gain 'adjusted short' signal can obtain a stable and continuous gradation characteristic from a low luminance portion to a high luminance portion.

The signal synthesized by the image synthesizing means 8 is subjected to processing such as separation of a luminance signal and a chrominance signal, noise removal, edge enhancement, gamma correction, matrix calculation, encoding to a specific format, and the like in the subsequent processing. Will be applied. Since the processing in the image synthesizing means 8 and the processing thereafter are not directly related to the object of the present invention, detailed description will be omitted.

In the first embodiment, the solid-state imaging device 2 has been described using an all-pixel readout type CCD (charge coupled device). However, the present invention is not limited to this. , Or a device that performs two-line addition reading on a solid-state image sensor, and can be applied to an imaging unit capable of outputting a plurality of image signals having various exposure amounts.

In the first embodiment, a signal obtained by adding two lines of the output from the solid-state image pickup device 2 by the two horizontal line addition means 3 is used by the exposure amount ratio detection means 6 to obtain a long signal.
Although the exposure amount ratio between the signal and the short signal is detected, the present invention is not limited to this, and the exposure amount ratio may be detected for a signal that does not perform two horizontal line additions on the output of the solid-state imaging device 2. There is no particular problem.

In the first embodiment, the image memory 4 can store two frames. However, the present invention is not limited to this. Any capacity is sufficient as long as the signal can be stored.

In the first embodiment, both the long signal and the short signal having different exposure amounts output from the solid-state imaging device 2 are frame signals. However, the present invention is not limited to this. The present invention can also be applied to the case where both are field signals, or the case where one is a field and the other is a frame or another signal having a different exposure amount and the number of lines is different. When the number of lines is different for each signal having such different exposure amount, in the exposure amount ratio detection period,
By controlling the image memory 4, for the signal with the larger number of lines, the pixel at the corresponding position is adjusted to the signal with the smaller number of lines at the same timing.
, It is possible to detect the exposure ratio.

(Embodiment 2) In the case of Embodiment 1 described above, a long-exposure signal and a short-time exposure signal having different exposure amounts are used, and these are combined to form an image having an expanded dynamic range. In order to obtain stable and continuous gradation characteristics as a synthesized image after image synthesis processing, the exposure ratio of the long-time exposure signal and the short-time exposure signal in the entire screen area is actually calculated when shooting. Then, an image synthesizing process was performed using a signal obtained by giving a gain corresponding to the exposure amount ratio to the short-time exposure signal and a long-time exposure signal.

At this time, it takes a time corresponding to one frame time to detect the exposure amount ratio between the long-time exposure signal and the short-time exposure signal, and further to synthesize the long-time exposure signal and the short-time exposure signal. Since a time corresponding to one frame time is required, the total processing time tends to be long.

The second embodiment is different from the first embodiment in that the image signal from the image pickup means and the image signal from the image memory are switched and sent to the exposure ratio detecting means. An object of the present invention is to reduce the time required from the start of photographing by the imaging means described in the first embodiment to the end of the image combining process.

FIG. 6 is a block diagram showing a configuration of a still image solid-state imaging device according to the second embodiment of the present invention. FIG.
, Except for the element denoted by reference numeral 9, is exactly the same as the configuration in the first embodiment, and the description is omitted here. Reference numeral 9 in FIG. 6 denotes a signal obtained by adding the output of the solid-state imaging device 2 by the two horizontal line addition means 3 (referred to as A side) and an output signal from the long signal storage unit from the image memory 4. (Referred to as B side). That is, the image signal switching means 9 switches to the A side when sending the long signal to the exposure ratio detecting means 6 and directly transmits the long signal from the two horizontal line adding means 3 to the exposure ratio detecting means 6.
When sending to the image synthesizing means 8, the signal is switched to the side B and the long signal read from the image memory 4 is sent to the image synthesizing means 8.

The operation of the still image solid-state imaging device according to the second embodiment configured as described above will be described below.

In the second embodiment, a specific example will be described in which two images of a short-time exposure signal (short signal) and a long-time exposure signal (long signal) are taken and synthesized.

First, a method of photographing a short signal and a long signal will be described with reference to FIG. FIG. 7 shows the solid-state imaging device 2
4 is a timing chart for exposure of a subject image, reading of an exposed signal, and reading and writing operations of the image memory 4 in FIG. 7, reference numeral 1101 denotes a timing of a vertical synchronization signal; 1102, a timing of a read control pulse for controlling reading of signal charges from the photodiode of the solid-state imaging device 2; 1103, a timing indicating the open / close state of the mechanical shutter 1; The timing indicating the amount of exposure signal on the photodiode of the solid-state imaging device 2, 1105 is the timing of the signal output from the solid-state imaging device 2, 1106 is the timing of the input (write) signal of the image memory 4, 1107 is the short of the image memory 4 The timing of an output (read) signal from the signal storage unit (short signal frame memory 41b), 1108 is a long signal storage unit (long signal frame memory 41) of the image memory 4.
This is the timing of the output (read) signal from a).

At the time of the exposure of the short signal, the mechanical shutter 1 is opened, and a necessary exposure time, for example, 1/1000 second exposure is performed by using the electronic shutter function. 1/100
After the 0 second exposure is completed, the charge stored on the photodiode is moved to the vertical transfer CCD by the read control pulse. At this time, the solid-state imaging device 2 is driven in the all-pixels reading mode.

Next, after moving the short signal to the vertical transfer CCD, exposure of the long signal is performed. The exposure time of the long signal is, for example, 1/100 second. The exposure time of the long signal is controlled by opening and closing the mechanical shutter 1. In parallel with this, a short signal is output from the solid-state imaging device 2 for one frame.
For the ng signal, the accumulated charge on the photodiode is transferred to the vertical transfer CCD. At this time, the solid-state imaging device 2 is driven in the all-pixels reading mode. Note that the period of the vertical synchronization signal is, for example, 1/30 second, and the signal readout for one frame is performed by one of the vertical synchronization signal.
It shall be completed within a cycle.

The two signals of the short signal and the long signal having different exposure times obtained by the solid-state image sensing device 2 are individually separated by two horizontal line adding means 3 into two adjacent upper and lower signals on the solid-state image sensing device 2. After the two horizontal line addition processing for adding and mixing the line signals, as shown at the timing of 1106, the short signal is stored in the sho
The long signal is temporarily stored in the rt signal frame memory 41b, and the long signal is temporarily stored in the long signal frame memory 41a of the image memory 4 during the period.

In this period, by switching the image signal switching means 9 to the A side, the long signal to be stored in the long signal storage section of the image memory 4 is converted to the signal level determination means in the exposure amount ratio detecting means 6. 6
At the same time as inputting to 1, the short signal stored in the short signal storage section of the image memory 4 is read out and input to the signal level determination means 61 of the exposure amount ratio detection means 6 as shown at the timing of 1107. In this case, the long input through the image signal switching means 9
Regarding the signal and the short signal output from the image memory 4, the lo
The ng signal and the short signal are controlled so as to be input to the signal level determination unit 61 of the exposure ratio detecting unit 6 at the same timing, and the process of detecting the exposure ratio is performed during the period. The process of detecting the amount of exposure during the period is similar to the process of detecting the amount of exposure during the period in the first embodiment, and a description thereof will be omitted. The feature of the second embodiment is that the processing in the period in the first embodiment is one frame period ahead.

Next, in the period shown in FIG.
The long signal and the short signal are read from the image memory 4. During this period, the image signal switching means 9 is switched to the B side, and the long signal from the long signal storage section of the image memory 4 is input to the image synthesis means 8 and the short signal from the short signal storage section of the image memory 4 is input. The signal is multiplied by a D-fold gain corresponding to the exposure amount ratio D calculated during the period in the gain adjusting unit 7, and then the image synthesizing unit 8
Is input to

The signal after the D times gain is given to the short signal is referred to as a short 'signal. Similarly to the first embodiment, based on the long signal read from the image memory 4 and the gain-adjusted short 'signal, a method as shown in the principle of dynamic range expansion described in FIG. The image is synthesized by the image synthesizing means 8 at the subsequent stage.

As described above, by adding the image signal switching means for switching between the image signal from the image pickup means and the image signal from the image memory, image synthesis from the start of image pickup by the image pickup means described in the first embodiment is performed. The time required to complete the processing can be reduced by one frame time.

(Embodiment 3) A method in which a short-time exposure signal and a long-time exposure signal having different exposure amounts are combined and used to capture an image having an expanded dynamic range, such as an outdoor or indoor image. When capturing an image of a subject with a different light source, the brightness level of the subject changes periodically due to the fluorescent light indoors, whereas the brightness level does not change periodically due to sunlight outdoors. There is a difference in the light source mode. Therefore, in a subject having a plurality of light sources, the ratio of the exposure amount of the long-time exposure signal to the exposure amount of the short-time exposure signal is different in each part of the screen, as described in the first embodiment. Therefore, a gain corresponding to a uniform exposure amount ratio cannot be given to the short-time exposure signal.

According to the third embodiment, the screen is divided into a plurality of blocks, and the exposure ratio between the short-time exposure signal (short signal) and the long-time exposure signal (long signal) is determined for each block. Even if such a light source exists, the synthesized image after the image synthesis processing for expanding the dynamic range has a stable and continuous gradation characteristic over the entire screen.

FIG. 8 is a block diagram showing a configuration of a still image solid-state imaging device according to Embodiment 3 of the present invention. FIG.
Since the configuration other than the internal configuration of the exposure amount ratio detecting means 6 is exactly the same as the configuration in the first embodiment,
Here, the description is omitted. In FIG. 8, reference numeral 61 in the exposure amount ratio detecting means 6 denotes a signal level determining means, 65a,
65b is a multiplexer, 66a and 66b are selectors,
67 divides the screen into n divided blocks, and for each block
long signal integration means for obtaining an integrated value LSUM of a long signal, 68
Divides the screen into n divided blocks and shor each block
This is a short signal integrating means for obtaining an integrated value SSUM of the t signal.
In each of the long signal integration means 67 and the short signal integration means 68 in block units, ΣBL1, ΣBL2
.. ΣBLn is an integrating means in units of n blocks. 6
4 is an integrated value of each block by the long signal integrating means 67
The corresponding integral value SSUM of each block by the LSUM and the short signal integrating means 68 is switched and input by the selectors 66a and 66b, and the integral value LSUM and the integral value SSUM are inputted in block units.
Exposure ratio calculating means for calculating the ratio of

The operation of the still image solid-state imaging device according to Embodiment 3 configured as described above will be described below.

First, regarding a specific example of the third embodiment, regarding the method of photographing the short signal and the long signal, the exposure time, and the method of controlling the solid-state imaging device 2 and the image memory 4,
Since it is completely the same as the specific example described in the first embodiment, the description is omitted here. The timing of the output signal from the solid-state imaging device 2 and the timing of the input / output signal of the image memory 4 at this time are exactly the same as the timing chart of FIG. 4 used in the description of the first embodiment. Will be described with reference to FIG.

In a specific example of the third embodiment, a case will be described in which an exposure amount ratio is obtained for each block obtained by dividing a screen into 8 × 6 = 48 as shown in FIG. As an example of the difference between the light source modes, block numbers 6, 7, 8, 14, and
The areas 15, 16, 22, 23, and 24 are the object areas where outdoor sunlight is the light source, and the other areas are the object areas where the fluorescent light is the light source.

In the third embodiment, as shown in FIG.
In the period, the signal output from the solid-state imaging device 2 is added to the two horizontal lines by the two horizontal line addition means 3, and then temporarily stored in the image memory 4.

During the period, the long signal integration means 67 and the short signal integration means 68 for each block are used for the short signal (short frame signal) read from the image memory 4,
In each long signal (long frame signal), the exposure amount of each pixel is integrated for each block divided into blocks, so that the total exposure amount (SSUM, LS
UM). During the period of
As for the timing of reading the ort signal and the long signal, the lines at the corresponding positions on the solid-state imaging device 2 are sequentially read from the first line so as to be output at the same timing.

The long signal and the short signal input to the exposure amount ratio detecting means 6 in line synchronization are subjected to level determination by the signal level determining means 61, and signals determined to have a level outside a predetermined range are determined. , Long signal integration means 67 and short signal integration means 68 negate so as not to perform integration for each block. Here, the level outside the predetermined range is a level at which it is difficult to determine the ratio of the exposure amount. Specifically, for the long signal, the signal of the high luminance portion near the saturation level of the solid-state imaging device 2 is used. And sh
The ort signal is a signal in a low-luminance area where the influence of noise is significant.

The signal determined by the signal level determination means 61 to have a level within a predetermined range is supplied to the multiplexer 65 so that each pixel signal is integrated by a corresponding block.
The paths are switched by a and 65b, and the integration is performed by the integration means $ BL1, $ BL2 ... $ BLn for each block.

During the period, when the long signal and the short signal of the pixels in the n-th (1-48) -th block divided from the image memory 4 are output, the selector 66 is controlled to control the n-th block. A signal obtained by calculating the integral value of the exposure amount of the block is controlled so as to be inputted to the exposure amount ratio calculating means 64, and the long signal in the n-th block and the short signal
The signal ratio is calculated, and a gain corresponding to the exposure amount ratio is given to the short signal from the image memory 4 by the gain adjusting means 7. Hereinafter, a signal obtained by giving a gain to the short signal by the gain adjusting means 7 is referred to as a short 'signal.

During the period, the image memory 4
Based on the long signal read out of the first and the gain-adjusted short 'signal, the image synthesizing means 8 at the subsequent stage is obtained by a method as shown in the principle of dynamic range expansion shown in FIG.
The synthesis is performed by

According to the above-described method, even in the case of a subject having a plurality of light sources, the ratio of the exposure amount of the long signal and the short signal in the block unit can be obtained with high accuracy over the entire screen. Therefore, the synthesized image by the image synthesizing means 8 using the long signal and the short 'signal whose gain has been adjusted can obtain a stable and continuous gradation characteristic from a low luminance portion to a high luminance portion.

In the subsequent processing, the signal synthesized by the image synthesizing means 8 is separated from the luminance signal and the chrominance signal, noise removal, edge enhancement, gamma correction, matrix operation, and the like as in the first embodiment. Processing such as encoding to a specific format is performed.

Note that, in the third embodiment, the screen is
Although the case where the exposure amount ratio is obtained for each of the eight divided blocks has been described, the present invention is not limited to this, and the number of screen divisions may be any number. The number of integrators for each block in the long signal integrator 67 and the short signal integrator 68 may be determined according to the number of divisions. Further, in the third embodiment, the block division method may be divided into equal intervals in the horizontal and vertical directions, or the intervals in the horizontal and vertical directions may be freely changed according to the subject. Good.

(Embodiment 4) The brightness of the fluorescent lamp is shown in FIG.
As shown in the variation diagram of the brightness of each color component of the fluorescent lamp, since the phase differs for each color component, the ratio of the amount of exposure differs for each color component depending on the timing of exposure. Therefore, in a method of photographing an image with an expanded dynamic range by using a short exposure signal and a long exposure signal having different exposure amounts, and combining them, a long exposure signal and a short exposure signal only with a luminance component are used. The combined image obtained by performing image synthesis for expanding the dynamic range by using a long-time exposure signal and a signal obtained by giving a gain corresponding to the exposure amount ratio to the short-time exposure signal is used for a long time. Near the saturation of the exposure signal, the characteristics of the color components become discontinuous.

In the fourth embodiment, the exposure amount ratio between the short-time exposure signal (short signal) and the long-time exposure signal (long signal) is obtained for each color component, so that the phase differs for each color component. The purpose of the present invention is to make a synthesized image after image synthesis processing for expanding a dynamic range stably obtain continuous tone characteristics and color characteristics.

FIG. 10 is a block diagram showing a configuration of a still image solid-state imaging device according to Embodiment 4 of the present invention. In FIG. 10, since the configuration other than the internal configuration of the exposure amount ratio detector 6 is exactly the same as the configuration in the first embodiment, the description is omitted here. In FIG. 10, reference numeral 61 in the exposure amount ratio detecting means 6 denotes a signal level judging means.
5a and 65b are multiplexers, 66a and 66b are selectors, and 69 is an integrated value LSUM of a long signal for each color component.
A long signal integration means 610 is a short signal integration means for obtaining an integrated value SSUM of a short signal for each color component. The long signal integrating means 69 and the short signal integrating means 61 for each color component
0, ΣMY, ΣMC, ΣGY, ΣG
C is [Magenta (Mg) + Yellow (Y
e)], [magenta (Mg) + cyan (Cy)], [green (G) + yellow (Ye)], [green (G)
+ Cyan (Cy)]. A selector 64a switches and inputs the integrated value LSUM for each color component by the long signal integrating means 69 and the integrated value SSUM for each color component by the short signal integrating means 610, and the integrated value LSUM and the integrated value for each color component. Exposure ratio calculation means for calculating the ratio with SSUM.

The operation of the still image solid-state imaging device according to Embodiment 4 configured as described above will be described below.

Here, the color filter on the photodiode of the solid-state imaging device 2 is a magenta (M
g), green (G), yellow (Ye), cyan (C
It is assumed that color filters having four different spectral characteristics y) are arranged for each pixel.

First, regarding a specific example of the fourth embodiment, regarding the method of photographing the short signal and the long signal, the exposure time, and the method of controlling the solid-state image sensor 2 and the image memory 4,
Since it is completely the same as the case of the specific example described in the first embodiment, the description is omitted here. Further, at this time, the output signal timing from the solid-state imaging device 2 and the input / output signal timing of the image memory 4 are exactly the same as the timing chart of FIG. 4 used in the description of the first embodiment. Will be described with reference to FIG.

In the fourth embodiment, as shown in FIG.
In the period, the signal output from the solid-state imaging device 2 is added to the two horizontal lines by the two horizontal line addition means 3, and then temporarily stored in the image memory 4.

In the period of, in each of the short signal and the long signal read from the image memory 4, the exposure amount of each pixel is integrated for each color component, so that the total exposure amount (SSUM, LSUM) of each color component is obtained. Ask.

In the period, the image memory 4
As for the timing for reading the t signal and the long signal, the lines at the corresponding positions on the solid-state imaging device 2 are sequentially read from the first line so as to be output at the same timing.

The long level signal and the short signal input to the exposure amount ratio detecting means 6 in line synchronization are subjected to level determination by the signal level determining means 61, and signals determined to have a level outside a predetermined range are determined. , Long signal integrator 69 and short signal integrator 610 negate so as not to perform integration for each color component. Here, the level outside the predetermined range is a level at which it is difficult to determine the ratio of the exposure amount. Specifically, for the long signal, the signal of the high luminance portion near the saturation level of the solid-state imaging device 2 is used. And sh
The ort signal is a signal in a low-luminance area where the influence of noise is significant.

The signal determined by the signal level determination means 61 to be within a predetermined range is a multiplexer 65 such that each pixel signal is integrated for each corresponding color component.
The paths are switched by a and 65b, and are integrated by integrating means 積分 MY, ΣMC, ΣGY, ΣGC for each color component.

Since the signal output from the image memory 4 is mixed by the two horizontal line adding means 3, two color components are represented by [magenta (Mg) + yellow (Y
e)], [magenta (Mg) + cyan (Cy)], [green (G) + yellow (Ye)], [green (G)
+ Cyan (Cy)] are output,
Integration is performed for each of these four types of color components.

During the period, the image memory 4
Lo of the color component of [magenta (Mg) + yellow (Ye)]
When the ng signal and the short signal are being output, the selector 66a, 66b is controlled to obtain the integrated value of the exposure amount of [magenta (Mg) + yellow (Ye)]. Control so as to be input, and a long signal and short in [magenta (Mg) + yellow (Ye)].
The ratio of the t signal is calculated, and the gain corresponding to the exposure ratio is given to the short signal of the color component [magenta (Mg) + yellow (Ye)] from the image memory 4 by the gain adjusting means 7.

In the same period, [Magenta (Mg) + Cyan (Cy)], [Green (G) + Yellow (Ye)] and [Green (G) + Cyan (Cy)] Long signal of color component and short
Similar processing is performed when a signal is being output.
Hereinafter, a signal obtained by giving a gain to the short signal by the gain adjusting means 7 is referred to as a short 'signal.

Based on the long signal and the short signal whose gain has been adjusted, the image is synthesized by the image synthesizing means 8 at the subsequent stage by a method as shown in the principle of dynamic range expansion shown in FIG.

According to the above method, even when the phase of each color component such as a fluorescent lamp is different, the ratio of the exposure amount of the long signal and the exposure amount of the short signal can be obtained with high accuracy over the entire screen. Signal and gain adjusted sh
The combined image by the image combining means 8 using the ort 'signal is
It is possible to obtain stable and continuous gradation characteristics and color characteristics from a low luminance portion to a high luminance portion.

In the subsequent processing, the signal synthesized by the image synthesizing means 8 is separated from the luminance signal and the chrominance signal, noise removal, edge enhancement, gamma correction, matrix operation, and the like, as in the first embodiment. Processing such as encoding to a specific format is performed.

In the fourth embodiment, magenta, cyan, green, and yellow color filter arrays on the photodiodes of the solid-state imaging device 2 have been described. However, the present invention is not limited to this. The present invention can be applied to filters having various color separation filters having different spectral characteristics. In addition, the present invention can be applied to a device having a colorization unit using a color separation prism instead of the color separation filter.

Although not shown, Embodiment 3
There may be an embodiment in which the third embodiment is mixed with the fourth embodiment. That is, the integration means $ BL1, $ BL2,.
積分 Integrating means for each color component for each of BLnΣM
The configuration may be such that Y, $ MC, $ GY, $ GC are added.

[0114]

According to the present invention, an exposure amount ratio between a long-time exposure signal and a short-time exposure signal obtained by imaging, that is, an actual exposure amount ratio according to the actual conditions such as imaging conditions and conditions is obtained. Based on the actual exposure amount ratio, after adaptively adjusting the gain between the long exposure signal and the short exposure signal when expanding the dynamic range, the long exposure signal and the short exposure signal are adjusted. Is composed so that the gradation characteristics of the synthesized image can be made to be stably continuous as compared with the conventional technology in which the exposure amount ratio is fixedly determined in advance, and the image quality is improved. Can be improved. Therefore, for example, when the exposure amount ratio fluctuates due to a time difference until the shutter is fully closed due to a difference in opening degree of the optical aperture, such as when the optical aperture is also used as a mechanical shutter, or as in a fluorescent lamp. Excellent image quality with stable and continuous gradation characteristics even when the exposure ratio fluctuates due to differences in shooting timing, such as when shooting under illumination where the luminance level changes periodically. Dynamic range expansion can be realized.

Further, the exposure ratio between the long exposure signal and the short exposure signal is actually measured for each block on the imaging screen, and the gain is adjusted based on the highly accurate exposure ratio actually measured for each block. In addition, if both signals are combined, even if there are a light source whose luminance level changes periodically like a fluorescent lamp and a light source like sunlight without such change, there are multiple types of light sources In addition, the synthesized image can be made into a high-quality image having a stable and continuous gradation characteristic in a state adapted to the type of light source in block units for expanding the dynamic range.

The exposure ratio of the long-time exposure signal and the short-time exposure signal is actually measured for each color component, and the gain is adjusted based on the highly accurate exposure ratio actually measured for each color component. If both signals are combined, even when photographed under a light source such as a fluorescent lamp, the brightness phase of which differs for each color component,
For the purpose of expanding the dynamic range, the synthesized image can be made into a high-quality image having stable and continuous gradation characteristics regardless of the type of light source.

In transferring the long-time exposure signal and the short-time exposure signal to the exposure ratio detecting means, one of the exposure signals is stored in the image memory, while the other is stored in the image memory. In parallel with the exposure amount detection means, and at the same time, the one exposure signal is read out from the image memory. The time required for the ratio detection processing can be reduced.

[Brief description of the drawings]

FIG. 1 is a block diagram illustrating a configuration of a still image solid-state imaging device according to Embodiment 1 of the present invention.

FIG. 2 is a block diagram illustrating a configuration example of a two-horizontal line addition unit in the still image solid-state imaging device according to the embodiment of the present invention;

FIG. 3 is a block diagram illustrating a configuration example of an image memory in the still image solid-state imaging device according to the embodiment of the present invention.

FIG. 4 is a timing chart for exposing a subject image, reading an exposed signal, and reading and writing an image memory in a solid-state imaging device in the still-image solid-state imaging device according to Embodiment 1 of the present invention;

FIG. 5 is a block diagram showing a configuration example of a signal level determination unit in the still image solid-state imaging device according to the embodiment of the present invention;

FIG. 6 is a block diagram illustrating a configuration of a still image solid-state imaging device according to Embodiment 2 of the present invention;

FIG. 7 is a timing chart for exposing a subject image in a solid-state imaging device, reading an exposed signal, and reading and writing an image memory in a solid-state imaging device in a solid-state imaging device according to Embodiment 2 of the present invention;

FIG. 8 is a block diagram illustrating a configuration of a still image solid-state imaging device according to Embodiment 3 of the present invention.

FIG. 9 is a schematic diagram showing a state of screen block division according to Embodiment 3 of the present invention.

FIG. 10 is a block diagram illustrating a configuration of a still image solid-state imaging device according to a fourth embodiment of the present invention.

FIG. 11 is a configuration diagram showing a color filter array of a solid-state imaging device in a still-image solid-state imaging device according to a fourth embodiment of the present invention;

FIG. 12 is a diagram illustrating a change in brightness for each color component of a fluorescent lamp.

FIG. 13 is an explanatory diagram of a principle of expanding a dynamic range according to the related art and the present invention.

FIG. 14 is an explanatory diagram of a closing operation of a mechanical shutter in a still image solid-state imaging device according to a conventional technique.

FIG. 15 is an explanatory view of an exposure operation when photographing a subject under a fluorescent lamp in a still image solid-state imaging device according to the related art.

FIG. 16 is an explanatory diagram of a defect after image synthesis processing when a gain corresponding to a ratio of an exposure amount in a still image solid-state imaging device according to the related art is not given to a short signal.

[Explanation of symbols]

 DESCRIPTION OF SYMBOLS 1 ... Mechanical shutter which also serves as an optical diaphragm 2 ... Solid-state imaging device 3 ... 2 Horizontal addition processing means 4 ... Image memory 5 ... Shutter drive control means 6 ... Exposure amount ratio detection means 7 ... Gain adjustment means 8 image synthesizing means 9 image signal switching means 41a frame memory for long signal 42b frame memory for short signal 61 signal level determining means 62 long signal integrating means for entire screen area 63 full screen Area short signal integrating means 64 Exposure ratio calculating means 65a Multiplexer 65b Multiplexer 66a Selector 66b Selector 67 Long signal integrating means in block units 68 Short signal integrating means in block units 69 Color Long signal integration means for each component 610 ... short signal integration means for each color component

──────────────────────────────────────────────────続 き Continued on the front page (51) Int.Cl. ⁷ Identification symbol FI Theme coat ゛ (Reference) H04N 5/335 H04N 5/335 Q 9/04 9/04 B // H04N 101: 00 101: 00 (72 ) Inventor Hiroya Kusaka 1006 Kadoma Kadoma, Kadoma, Osaka Prefecture Inside Matsushita Electric Industrial Co., Ltd. (72) Inventor Yasutoshi Yamamoto 1006 Kadoma Odaka, Kadoma City, Osaka Pref. AB04 BB01 BC06 CA19 CB15 CB16 DA01 DC20 5C022 AA13 AB12 AB17 AB20 AC42 AC52 AC54 AC56 AC69 AC74 5C024 BX01 CX47 CX56 DX07 EX34 GZ32 GZ42 HX23 HX28 HX30 HX31 HX57 HX58 5C065 AA03 BB48 CC08 CC09 DD09 DD08

Claims (11)

[Claims] 1. A still image solid-state imaging device for expanding a dynamic range by combining a long-time exposure signal and a short-time exposure signal obtained by imaging, wherein the long-time exposure signal and the short-time exposure signal are A solid-state imaging device for a still image, wherein a gain between a long-time exposure signal and a short-time exposure signal in the dynamic range expansion is adjusted based on the detected exposure ratio. 2. The method according to claim 1, wherein an exposure ratio between the long-time exposure signal and the short-time exposure signal is detected for each of blocks divided and formed on an imaging screen, and the gain adjustment is performed for each block. 2. The still image solid-state imaging device according to 1. 3. The apparatus according to claim 1, wherein an exposure amount ratio between the long exposure signal and the short exposure signal is detected for each color component, and the gain adjustment is performed for each color component. Still image solid-state imaging device. 4. A detection of an exposure ratio between the long-time exposure signal and the short-time exposure signal is performed for each block divided and formed on the imaging screen and for each color component, and the gain adjustment is performed for each block and each color component. 2. The still image solid-state imaging device according to claim 1, wherein the imaging is performed every time. 5. An image pickup means for independently outputting a plurality of image signals having different exposure amounts, an image memory storing a plurality of image signals having different exposure amounts from the image pickup means, and a plurality of images having different exposure amounts. An exposure ratio detection unit that detects an exposure ratio of a signal; and a gain adjustment unit that performs gain adjustment for a plurality of image signals having different exposure amounts read from the image memory based on the detected exposure ratio. An image synthesizing means for synthesizing a plurality of image signals having different exposure amounts after the gain adjustment. 6. An exposure amount ratio detection unit, comprising: a signal level determination unit for determining signal levels of a plurality of image signals having different exposure amounts; and a plurality of exposure amounts having different exposure amounts within a predetermined range based on the level determination. 6. An image forming apparatus according to claim 5, further comprising: integrating means for calculating an integral value for each image signal; and exposure amount calculating means for calculating a ratio of the plurality of integrated values.
3. The still image solid-state imaging device according to item 1. 7. The image processing apparatus according to claim 6, wherein said integrating means calculates an integrated value for each of the plurality of image signals having different exposure amounts in a unit of a block obtained by dividing a screen into a plurality of blocks. 3. The still image solid-state imaging device according to item 1. 8. The image pickup means has a color separation filter or a color separation prism having different spectral characteristics, and the integrating means further comprises a plurality of image signals having different exposure amounts for each color component. 7. The still-image solid-state imaging device according to claim 6, wherein an integrated value is calculated for each image. 9. The image pickup means has a color separation filter or a color separation prism having different spectral characteristics, and the integration means further comprises: a block unit which divides a screen into a plurality of blocks; 7. The still image solid-state imaging device according to claim 6, wherein an integrated value is obtained for each of the plurality of image signals having different exposure amounts for each component. 10. An image signal from the image pickup means and an image signal from the image memory for any one of the plurality of image signals having different exposure amounts sent from the image pickup means to the image memory. 10. The still-image solid-state imaging device according to claim 5, further comprising: an image signal switching unit that switches and sends the image signal to the exposure amount ratio detection unit. 11. A gain adjusting unit for a plurality of image signals having different exposure amounts, wherein a gain based on the exposure ratio detected by the exposure ratio detecting unit is set to at least one of the image signals having different exposure amounts. The still image solid-state imaging device according to any one of claims 5 to 10, wherein the still image solid-state imaging device is configured to be provided to each of the plurality of still images.