JPH0759103A - Method and device for solid-state color image pickup - Google Patents

Abstract

PURPOSE:To widen the dynamic range of a color signal component in a direction where a final video signal output level exceeds a standard value 100%. CONSTITUTION:A 1st image pickup signal with high photo sensitivity, a 2nd image pickup signal with intermediate photo sensitivity, and a 3rd image pickup signal with low photo sensitivity are obtained from a CCD image pickup element 12. Normally the 1st image pickup signal is fed to a frame memory 18 and the 2nd image pickup signal is fed to a frame memory 19. A data changeover circuit 23 monitors the 1st image pickup signal and switches the drive method of the CCD image pickup element 12 so as to obtain the 3rd image pickup signal when a color signal reaches a saturated level. Then a dynamic range is expanded by obtaining the color component by allowing a color signal matrix circuit 24 to use always the image pickup signal whose color signal is not saturated. A frame memory 20 keeps a level just before the color signal reaches a saturated level. Thus, a luminance component in a luminance signal synthesis circuit 31 is reproduced without loss of the dynamic range.

Description

Detailed Description of the Invention

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a solid-state color image pickup method and apparatus using a solid-state image pickup element such as a charge coupled image pickup element (hereinafter referred to as a CCD image pickup element).

[0002]

2. Description of the Related Art The actual subject to be imaged is from a location near 100,000 lux (lx) illuminated by sunlight in the same scene to a location below a few lx in shadow. The dynamic range of intensity is very wide. For this reason, the F value of the optical system cannot always be reduced. Therefore, in the conventional color camera, an unnatural reproduced image such as "whiteout" may occur in a portion having high illuminance, depending on the subject. Especially for surveillance cameras installed outdoors, cameras mounted on-board or mounted on satellites, the video signal output level is standard (1
A wider dynamic range was demanded in the direction of exceeding 100%).

As a method of expanding the dynamic range of a television camera, there are the following conventional techniques. As shown in FIG. 4, with respect to the means for separating the optical path such as the optical resolution prism 101 into two directions, two Cs that can obtain output signals with different optical sensitivities at respective image forming positions.
The CD image pickup devices 102 and 103 are provided. As a method of obtaining different photosensitivities, for example, the CCD image pickup devices 102 and 103 are driven by the CCD image pickup device drive circuits 104 and 105 so that the respective desired photosensitivities can be obtained. Then, each imaging output is output to the amplifier 1
After gain control in 06 and 107, addition is performed in the adder circuit 108 to obtain a signal with a wide dynamic range. Fig. 5 shows how the dynamic range of the video signal is expanded at this time.
Shown in. That is, (a) in the figure shows C for normal light sensitivity setting.
According to the characteristics of the image pickup signal output from the CD image pickup element 102, the output is saturated when the incident light level becomes A or higher. On the other hand, (b) shows the characteristics of the output image pickup signal from the CCD image pickup device 103 whose light sensitivity is set lower (for example, half) than that of (a). When the incident light level becomes B (> A) or more, the output is It is saturated. Further, (c) is an addition signal of (b) and (a), and has a characteristic having both the wider dynamic range characteristic of (b) and the higher photosensitivity characteristic of (a). . As a practical matter, (c) may be electrically adjusted to an appropriate level after addition due to the limitation of the electric dynamic range of the circuit itself.

However, this camera has the following problems. That is, the CCD image pickup devices 102, 1 of FIG.
Consider the case where 03 is a color CCD image pickup device having a color filter on the photoelectric conversion device. For example, as shown in FIG. 6, this color filter has four colors of magenta (hereinafter referred to as Mg), green (hereinafter referred to as G), cyan (hereinafter referred to as Cy), and yellow (hereinafter referred to as Ye). As shown in the figure, there are 4 rows and 2 columns, which are configured as a basic repeating unit, which is known as a so-called complementary color difference line-sequential color filter array. Since the driving method of the CCD image pickup device is the field storage method,
In the first field, the n line and the (n + 1) line, and the (n + 2) line and the (n + 3) line in the figure are added to form one imaging output. In the second field, the combination of addition is deviated by one line, and the addition is made as (n + 1) line and (n + 2) line, and as (n + 3) line and (n + 4) line, respectively, resulting in one imaging output. In this case, regarding the color signal component of the CCD output signal, two kinds of color difference signals, C _B = (E _Mg + E _Cy ) − (E _G + E _Ye ), and C _R = (E _G + E _Cy ) − (E _Mg + E _Ye ) are obtained line-sequentially alternately as a signal spatially modulated at the horizontal repetition frequency of the color filter. The problem at this time is that the amount of light that saturates the pixel changes depending on the color filter, so the light amount range in which the color signal component of the CCD output signal shows a normal value, that is, the dynamic range of the color signal component It means that it will become narrower. FIG. 7 shows a diagram for explaining the situation.

FIG. 7A is a diagram for explaining the example. For example, when the color temperature of the illumination light is 3100K and the output of each pixel is saturated when an achromatic subject is imaged, the amount of incident light is Ye. , Cy, G, Mg in this order. In this case, as can be seen from FIGS. 7B and 7C, both of the two color difference signals C _B and C _R do not hold regular color information when the light amount is equal to or greater than the Ye pixel saturation. That is, although FIG. 8 illustrates the E _G + E _Cy and E _Mg + E _Ye scan outputs, as shown in (8c) of the figure, the luminance signal component has the 21 characteristic but is not saturated up to the light amount B. However, as shown in (8b) of the figure, the color signal component is limited to the normal light amount A (<B) of the Ye pixel of the image signal having relatively low sensitivity. There was a problem that information could not be obtained. This means that when the amount of incident light becomes larger than A when viewed on a reproduced image, the hue and saturation of the color signal deteriorate.

[0006]

As described above, in a conventional solid-state color image pickup device, particularly in a surveillance camera installed outdoors or a camera mounted on a vehicle or mounted on a satellite, the dynamic range of the light quantity of the subject is Since it is wide, an unnatural reproduced image such as “whiteout” may occur depending on the subject. Especially, the final video signal output level is standard (100
%), A wider dynamic range was required. Further, in order to obtain a solid-state color image pickup device having a wide dynamic range that can be designed by using the conventional technique, particularly when the CCD image pickup device is a color CCD image pickup device having a color filter on its photoelectric conversion element, Since the amount of light that saturates the pixel changes depending on the color filter, the light amount range in which the color signal component of the CCD output signal shows a normal value, that is, the dynamic range of the color signal component is narrowed, and the sensitivity is the highest. The normal information can be obtained only up to the amount of light at which high pixels are saturated,
There was a problem.

Therefore, according to the present invention, the dynamic range of the color signal components and the final video signal output level are standard (10
It is an object of the present invention to provide an even wider solid-state color image pickup device in the direction exceeding 0%).

[0008]

The present invention has at least two features.
In a solid-state color imaging method that relatively widens the dynamic range of a video signal that is output according to the light intensity of an input optical image by using an imaging signal that has different types of light sensitivity, When the first image pickup signal having a high light intensity and the second image pickup signal having a relatively low light sensitivity are obtained in a time division manner, and it is detected that the color signal component is saturated by using the first image pickup signal. In addition, the third image pickup signal having a relatively lower photosensitivity than the second image pickup signal is switched so as to be obtained instead of the second video signal, and the first and second image pickup signals are used. The color component reproduction process is switched to the color component reproduction process using the second and third image pickup signals, and the luminance component reproduction process is performed by the first image pickup until saturation of the color signal component is detected. Hold the signal for a specified period It is intended to continue the reproduction processing using the second image pickup signal and the delayed first image signals.

[0009]

By the above means, when the color signal component of the first image pickup signal having a relatively higher photosensitivity (for example, a normal sensitivity setting) is saturated, the second image pickup signal having a relatively lower light sensitivity is saturated. Since the third image pickup signal is obtained by changing the setting to an even lower light sensitivity, the color signal component thereof is not saturated. Since the color signal component is used as the total color signal component, it is possible to extend the dynamic range of the color signal to the amount of light that is saturated by itself and cannot obtain regular color information. Further, when switching from the first image pickup signal to the third image pickup signal, the continuity of the level of the luminance signal is lost as it is, so the luminance signal component obtained from the first image pickup signal is replaced with a certain value ( For example, the brightness level value at the time of switching is held), and a total brightness signal component is obtained from that value and the brightness signal component of the second image pickup signal as in the conventional case. Therefore, the wide dynamic range property of the luminance signal can be maintained.

[0010]

Embodiments of the present invention will be described below with reference to the drawings. 1 is an embodiment of the present invention, FIG. 2 is a diagram for explaining the signal level of an image pickup signal obtained from the CCD image pickup device 12 of FIG. 1, and FIG. 3 is an image according to an embodiment of the present invention. It is a figure for demonstrating the operation | movement which the dynamic range of the brightness | luminance signal component and chrominance signal component of an output signal widens.

An optical image is incident on the light receiving surface of the CCD image pickup device 12 of FIG. Further, on the light receiving surface, color filters having the same arrangement as the color filters shown in FIG. 6 are provided so as to correspond to the light receiving elements one to one. A part of the pulse from the pulse generation circuit 14 becomes a CCD drive pulse in the CCD drive circuit 13, which drives the CCD image pickup element 12,
The incident optical image is converted into an electric signal and becomes an imaging output.

At this time, by utilizing so-called shutter drive as a CCD drive method, the CCD image pickup device 12
It is possible to obtain two types of image pickup signals having different light sensitivities. That is, there is a television broadcasting system 1
In one frame, the light accumulation time of each field is set to a normal one field period (for example, 1/6 in the NTSC system).
The first imaging output signal is obtained as 0 second). In the subsequent frame, the light accumulation time is set shorter than one field period to obtain the second image pickup output signal. FIG. 2 is a diagram showing an example of this image pickup output signal, and the signal level is shown to be large in the direction extending downward from the black level in FIG. First imaging output with relatively high light sensitivity (A in the figure)
It can be seen that, by comparison with this, the second image pickup output with low photosensitivity (B portion in the same figure) is obtained alternately every other frame. As a shutter driving method, a mechanical shutter is used to directly control the exposure time of the CCD, or a FIT-CCD having a light-shielding storage section and an image pickup section is used to horizontally or horizontally store the accumulated unnecessary charges. There are some known methods such as a method of controlling exposure by sweeping out during the vertical blanking period, but any of these methods may be used.

As described above, the CCD image pickup device 1 shown in FIG.
Two kinds of image pickup signals having different optical sensitivities sequentially read from No. 2 are amplified to a predetermined level by the preamplifier 16 in FIG. 1 and then analog / digital (hereinafter referred to as A / D)
It is converted into digital data by the converter 17 and written in the frame memories 18, 19 and 20. Here, the write control to each frame memory 18, 19 and 20 is as follows. That is, the first switch circuit 21 is switched every frame period by the control signal from the control circuit 15, and the above-mentioned first image pickup output signal (A part in FIG. 2) is stored in the frame memory 18 again. Second to 19
The image pickup output signal (section B in FIG. 2) is written. Both frame memories 18 and 19 retain the previous data until the next new data is written.
Further, although the first image pickup output signal is written in the frame memory 20, the opening / closing of the second switch circuit 22 is switched by the control of the data switching determination circuit 23.
The previous data may be retained as it is without writing new data.

Upon receiving the output of the frame memory 18, the data switching judgment circuit 23 opens the second switch circuit 22 when it judges that a part of the first image pickup output signal is about to be saturated, and opens the frame memory. The new data from the A / D converter 17 is not written in the data 20, and the previous data is held as it is. This judgment is made, for example, by using the data of the frame memory 18 as the CCD recorded in advance.
Alternatively, it may be determined that there is even one piece of data immediately before saturation as compared with the saturation level of the output of the image sensor.
By this control, a signal in which no data is saturated is always obtained from the frame memory 20.

When it is determined that the saturation is about to occur, the control circuit 15 controls the pulse generation circuit 14 again at the same time to further shorten the exposure time for shutter drive, and the portion A (first image pickup output) in FIG. The sensitivity in section B of the figure (second
(Corresponding to the imaging output of 1) is set lower than the sensitivity. This state is shown in FIG. Although the same figure is a graph of the signal level with respect to the light amount, the curve of (3a) corresponds to the first imaging output, and part of the data including the Ye pixels of the CCD image sensor is saturated with the light amount D, Immediately before that, the sensitivity setting by the shutter drive is set to, for example, 1/3 of (3a), and the third image pickup output of the photoelectric conversion characteristic of (3c) in the figure is obtained. Therefore, the data written in the frame memories 18, 19 and 20 are as follows. First, up to the light amount D in FIG.
a) The imaging output of (corresponding to the first imaging output) is written. Further, the image output of (3b) in the figure (corresponding to the second image output) is written in the frame memory 19. Next, when the light quantity exceeds D, the image output of (3b) is continuously written in the frame memory 19 as it is, but the image output of (3c) (corresponding to the third image output) is written in the frame memory 18. Is newly written, and new data is not written in the frame memory 20, and the data immediately before that is held as it is.

Of the data thus obtained, the other part of the output from the frame memory 18 is input to one input terminal 27 of the third switch circuit 28. The output from the frame memory 20 is the third switch circuit 28.
Is input to the other input terminal 26. Here, the switching of the third switch circuit 28 is controlled by the output of the data switching determination circuit 23. Specifically, up to the light amount D in FIG. 3 in which none of the pixels of the CCD image pickup device is saturated, that is, the second switch circuit 22 is closed and the same data as the frame memory 18 is written in the frame memory 20. The third switch circuit 28 is connected to the input terminal 27 when the switch is on. Further, when the light amount increases more than D and the image pickup output of (3a) starts to be saturated, that is, the second switch circuit 22 opens and new data is not written in the frame memory 20 and the previous data is held. At this time, the third switch circuit 28 is connected to the input terminal 26. As a result of the above operation, the output of the third switch circuit 28 is at the same level as the imaging output of (3a) up to the light amount D, as shown in FIG.
The value of (3a) at the time of is held. This will maintain the continuity of the luminance signal level.

The output of the third switch circuit 28 and the output of the frame memory 19 are input to the luminance signal synthesis circuit 31. The luminance synthesizing circuit 31 first generates respective luminance signal components from the above two input signals. This is the sum of two pixels in the horizontal direction of each imaging output, that is, in the case of having the filter of FIG. 6, first, in the addition scanning output of the n line and the (n + 1) line, it is the addition of (Mg + Cy) and (G + Ye). In the addition scanning output of the (n + 2) line and the (n + 3) line of (G + Cy) and (Mg + Ye),
And so on. The respective luminance signal components obtained by this operation are added and synthesized in the luminance signal synthesis circuit 31. The result is as shown by the curve (3d) in FIG. 3, and the dynamic range is expanded to B, which is the light amount at which all the data of the imaging output in (3b) is saturated. If necessary, the output of the luminance signal synthesizing circuit 31 is level-adjusted by the level adjusting circuit 32 so as to be within the dynamic range of the video signal processing circuit. Then, the gamma correction circuit 33 performs a gamma correction process to obtain a broadband luminance signal E _YH signal.

On the other hand, another part of the output of the frame memory 18 and a part of the output of the frame memory 19 are input to the color signal matrix circuit 24. In this color signal matrix circuit 24, for example, the three primary color signals E _R ′, E
Generate _G ′ and E _B ′. Here k ₁₁ to k ₃₄ are constants.

[0019]

[Equation 1] However, the used E _Mg + E _Cy , E _G + E _Ye , E _G + E _Cy ,
For E _Mg + E _{Ye, the} level-controlled one described below is used.
That is, the output data of the frame memory 18 is used. However, the output value thereof does not reach the light amount D in FIG. 3 in which none of the pixels of the CCD image pickup device is saturated due to the control of the shutter drive of the CCD image pickup device described above. The data obtained from (3a) of FIG. 3 is used. Further, when the light amount increases more than D and the image pickup output of (3a) starts to be saturated, (3c) of FIG.
The data obtained from is used. At this time, since the levels of the color difference signal components C _R and C _B of the image pickup output are discontinuous at the time of switching the light amount D, the level of the data obtained from (3c) of FIG. 3 is adjusted if necessary. is there. By this processing, the dynamic range of the color signal component can be expanded up to the light amount C, whereas the conventional value could be obtained only up to the light amount D in FIG.

Then, these three primary color signals E _R ′, E
_G ′ and E _B ′ are subjected to white balance adjustment by the white balance circuit 25, and then subjected to gamma correction processing by the gamma correction circuit 29 to become the three primary color signals E _R , E _G , and E _B. Further, the three primary color signals E _R , E _G , and E _B are input to the matrix circuit 30, and the low-frequency component E _YL of the luminance signal and the two color difference signals E are input.
_R-YL and _{EB -YL} . In the case of the NTSC system, for example, the following calculation is performed.

E _YL = 0.30E _R + 0.59E _G + 0.11E _B E _R-YL = 0.70E _R -0.59E _G -0.11E _B E _B-YL = -0.30E _R -0.59E _G + 0.89E _B The E _YH and E _YL signals obtained by the above processing are the first
Is input to the adder 34. Here, since the E _YH signal is input after being made negative, the operation of the adder 34 is substantially a subtractor. The output of the adder 34 is (E _YL −
E _YH ) LPF (low-pass filter) 36 band-limits the frequency range that affects the color reproducibility, and then is input to the second adder 37. The other input of the second adder 37, is input E _YH signal passing through (E _YL -E _YH) LPF37 and delay delay time correction circuit 35 which time the same. Then, the E _Y signal which is the final luminance signal is obtained. In this E _Y signal, the low frequency component that affects the color reproducibility of the E _YH signal is E _YL, which is the mixing ratio of the NTSC system.
It is replaced with a signal, and the color reproducibility is also good.

Regarding the final output signal, E _Y , E _R-YL
And the _EB-YL signal is passed through a D / A converter as necessary to be converted into an analog signal. As described above, in this embodiment, by using the image pickup signals having at least two kinds of photosensitivity, the dynamic range of the video signal output according to the light intensity of the input optical image is relatively widened. In the solid-state imaging device having a function, when the color signal component of the first image pickup signal having a relatively higher photosensitivity is saturated, the light sensitivity thereof is further increased than that of the second image pickup signal having a relatively lower photosensitivity. The third image pickup signal is obtained by changing the setting to a low light sensitivity so that the color signal component therein is not saturated, and the luminance signal component obtained from the first image pickup signal is replaced with a certain value. Further, the color signal component obtained from the third image pickup signal is used as the total color signal component. As a result, not only the luminance signal component,
With respect to the color signal components as well, it is characterized in that the dynamic range of the solid-state imaging device can be further expanded in the direction of exceeding 100% which is the standard output value of the final video signal.

Although the frame memories 18 to 20 have been described in FIG. 1 in the above description, they may be field memories. In that case, the capacity of the memory can be reduced to half, so that the circuit scale can be reduced, and the advantages of lower power consumption and lower cost can be obtained.

Further, in the above description, it was explained that only the output signal from the frame memory 18 is used as the signal used in the color signal matrix circuit 24, but the present invention is not limited to this. That is, the frame memory 1 is used up to the light amount D in FIG.
The signal of (3a) of FIG. 3 which is the output from 8 or the signal of (3b) of FIG. 3 which is the output of the frame memory 19 is used. Then, until the light amount exceeds D and reaches A (the light amount in the signal of (3b) of FIG. 3 including the Ye pixel starts to be saturated), the output from the frame memory 18 of (3c) of FIG. Either the signal or the signal (3b) in FIG. 3 which is the output of the frame memory 19 is used. Further, in the range where the light amount exceeds A, the output from the frame memory 18 is shown in FIG.
The configuration of using the signal of (3c) may be used.

[0025]

As described above, according to the present invention,
Since the dynamic range of the solid-state color image pickup device can be further expanded in the direction of exceeding the standard output value 100% of the final video signal not only in the luminance signal component but also in the color signal component, the conventional solid-state color image pickup device can be used. Unnatural reproduced images such as "whiteout" that have occurred depending on the subject are also greatly reduced. Especially, the effect is great for the color signal component. Further, even when the output of the solid-state image pickup device is subjected to image processing by a computer or the like, since the dynamic range of the original video signal is wide, unnecessary pseudo signals are not generated and the image quality is not deteriorated. In addition, when compared to the method of setting the solid-state color image pickup device to low sensitivity from the beginning and simply amplifying to the standard output in the signal processing circuit, the S
Higher image quality with less deterioration of N ratio.

[Brief description of drawings]

FIG. 1 is a schematic configuration diagram showing an embodiment of the present invention.

FIG. 2 is a diagram for explaining a signal level of an image pickup signal obtained from the CCD image pickup device 12 of FIG.

FIG. 3 is a diagram for explaining an operation of expanding a dynamic range of a luminance signal component and a color signal component of a video output signal according to an embodiment of the present invention.

FIG. 4 is a schematic configuration diagram of a conventional solid-state color imaging device.

FIG. 5 is a diagram for explaining an operation of expanding the dynamic range of the solid-state color imaging device in FIG.

FIG. 6 is a diagram showing an arrangement example of color filters on a CCD image sensor.

FIG. 7 is a diagram for explaining that a dynamic range of a color signal component in a CCD image pickup output signal is narrowed in a conventional solid-state color image pickup device.

FIG. 8 is a diagram for explaining that a dynamic range of a color signal component in a CCD image pickup output signal is narrowed in a conventional solid-state color image pickup device.

[Explanation of symbols]

 11 ... Imaging lens 12 ... Solid-state imaging device 13 ... CCD drive circuit 14 ... Pulse generation circuit 16 ... Preamplifier 17 ... A / D converter 18, 19, 20 ... Frame memory 23 ... Data switching determination circuit 21, 22, 28 ... Switch Circuit 24 ... Color signal matrix circuit 31 ... Luminance signal synthesizing circuit

Claims (2)

[Claims] 1. A solid-state color imaging method for relatively widening a dynamic range of a video signal output according to the light intensity of an input optical image by using an imaging signal having at least two kinds of photosensitivity. , A first image pickup signal having a relatively high light intensity and a second image pickup signal having a relatively low light sensitivity are obtained in a time division manner from the solid-state image pickup element, and a color signal component is obtained using the first image pickup signal. When saturation is detected, switching is performed so as to obtain a third image pickup signal having a photosensitivity lower than that of the second image pickup signal instead of the first video signal. The color component reproduction process using the image pickup signal of 10 is switched to the color component reproduction process using the second and third image pickup signals, and the saturation of the color signal component is detected in the luminance component reproduction process. The first at the previous time The imaging signal may be held a predetermined period, the solid-state color imaging method characterized by continuing the reproduction processing using the second image pickup signal and the delayed first image signals. 2. An image pickup section capable of arbitrarily obtaining first, intermediate, second and low third image pickup signals having relatively high photosensitivity, and a first image pickup section for delaying the first image pickup signal. Memory, the second
A second memory for delaying the image pickup signal, a third memory capable of capturing and non-capturing the first image pickup signal, and an output signal of the first memory to detect a saturation level of a color signal. Determination means, a brightness reproduction processing unit that reproduces a brightness component using the output signal from the second memory and the output signal from the first or third memory, the first memory and the first memory. And a color reproduction processing unit that reproduces a color component using an output signal from the second memory, and when the determination unit does not detect a saturation level of the color signal, the first and second image pickup units are provided to the image pickup unit. A state in which an image pickup signal is obtained, a state in which the brightness reproduction processing unit uses the first and second image pickup signals are set, and the first image pickup signal is loaded into the third memory to store a color signal. When a level that saturates is detected, The image pickup unit is switched to a state in which the second image pickup signal and the third image pickup signal are obtained, the dynamic range of color reproducibility in the color reproduction processing unit is expanded, and the input of the third memory is turned off. However, the luminance reproduction processing section is provided with a switching means for switching so that the output signals of the third memory and the second memory are introduced and maintaining the continuity of the dynamic range of the luminance signal. Solid-state color image pickup device.