JPH10294892A - Solid-state image pickup device - Google Patents

Abstract

PROBLEM TO BE SOLVED: To attain the suppression of smear while expanding a dynamic range by performing synthesizing processing to image signals while extracting and removing smear components from the plural image signals with different exposure amounts outputted from an image pickup means. SOLUTION: A smear extracting means 3A is equipped with a 1st subtraction part 32 for preparing a pseudo smear signal SM' by subtracting an attenuated signal Slong/N as an attenuated long-time exposure signal from a short-time exposure time Sshort. A smear removing means 4 is equipped with a 2nd subtraction part 13 for preparing a pseudo original long-time exposure signal SlongO' by subtracting the pseudo smear signal SM' from the long-time exposure signal Slong and passing the short-time exposure signal Sshort with no processing on the other hand. A comparator 15A judges whether the signal level change of long-time exposure signal Slong has linearity or not and based on the result, a synthesizing signal Smix is outputted to a selector 16.

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a solid-state imaging device used for a video camera or the like.

[0002]

2. Description of the Related Art Conventionally, as a solid-state imaging device for synthesizing images having different exposure amounts to obtain a video signal having a high dynamic range, one described in JP-A-7-322147 has been proposed.

[0003] A conventional solid-state imaging device of this kind includes an imaging means 50 and an image synthesizing means 51 as shown in FIG. The imaging means 50 includes an all-pixel readout solid-state imaging device (not shown) capable of outputting a signal having twice as many scanning lines as a normal solid-state imaging device. The electronic shutter operation is performed so that the pixel corresponding to one of the scanning lines has a normal exposure amount, and the electronic shutter operation is performed so that the pixel corresponding to the other scanning line has a smaller exposure amount. And a short-time exposure signal Sshort (small exposure amount) which is an output signal having different exposure amounts from each other.

The image synthesizing means 51 obtains a synthesized signal Smix by simply adding the output signals Slong and Sshort output from the imaging means 50.

FIG. 13 shows a solid-state imaging device (imaging means) on the horizontal axis.
The long-time exposure signal S in the case where the incident light amount with respect to 50 and the signal level of the output signal of the imaging means 50 are taken on the vertical axis
The relationship between the long, the short exposure signal Sshort, and the composite signal Smix is shown.

Here, the long-time exposure signal Slong is saturated with the amount of incident light indicated by the symbol A because the exposure amount is large.
Since the change in the signal level is large when the incident light amount is less than that, the S / N ratio is good. On the other hand, the short exposure signal Sshort
Although the signal amount is small and the S / N ratio is poor, the reproducibility of the signal at the incident light amount equal to or more than the saturated light amount A is good because the dynamic range is wide.

In the conventional solid-state imaging device, the image synthesizing means 5
1, by combining these two signals Slong and Sshort, a combined signal Smix having a good S / N ratio and a wide dynamic range is obtained.

[0008]

However, in the above-mentioned conventional solid-state imaging device, there is a problem that the smear component increases and the screen becomes difficult to see. This will be described below.

The conventional solid-state imaging device is often used for photographing a brighter scene because the dynamic range is expanded. However, the solid-state imaging device has a feature that when a high-luminance subject is photographed, smear tends to occur. Smear is a phenomenon in which a white line appears in the vertical direction on a display screen due to the influence of a signal component (hereinafter, referred to as a smear component SM) generated by light leaking into a vertical transfer unit included in the solid-state imaging unit 50. The higher the brightness of the subject, the more this occurs. In the above-described conventional solid-state imaging device, a smear component is generated along with adjacent scanning lines, and the generated smear component SM is increased by being added to each other by the image synthesizing unit 51, thereby increasing the smear on the display screen. The screen became difficult to see.

Accordingly, an object of the present invention is to suppress smear while expanding the dynamic range.

[0011]

SUMMARY OF THE INVENTION In order to solve this problem, the present invention relates to a method for extracting a smear component from a plurality of image signals output from an image pickup device with different amounts of exposure by a smear extractor, and then removing the smear component by a smear remover. The image processing apparatus is configured to remove the extracted smear component from the image signals having different exposure amounts and to perform a synthesis process.

[0012]

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS A solid-state image pickup device according to a first aspect of the present invention includes an image pickup means for outputting a plurality of image signals having different exposure amounts within a period for picking up an image for one screen. A signal level adjusting means for adjusting a signal level of an image signal having a different exposure amount, a smear extracting means for extracting a smear component from an image signal having a different level of exposure light, and extracting the smear component with the exposure amount. It is characterized in that it has a smear removing means for removing from different image signals and an image combining means for combining image signals with different exposure amounts from which smear has been removed, thereby having the following effects. That is, since the smear removal processing is performed when the smear component exists, the smear can be suppressed while expanding the dynamic range.

According to a second aspect of the present invention, in the solid-state imaging device according to the first aspect, the signal level adjusting means converts a signal level of an image signal having a large exposure amount into a signal level of an image signal having a small exposure amount. It is characterized in that it has an adjusting unit that adjusts to substantially match the signal level, and thus has the following operation. That is, the signal level of the image signal having a large exposure amount can be accurately adjusted.

According to a third aspect of the present invention, in the solid-state imaging device according to the first or second aspect, the adjustment unit includes an attenuator for attenuating a signal level of an image signal having a large exposure amount. This has the following effects. That is, it is possible to easily adjust the signal level.

According to a fourth aspect of the present invention, in the solid-state imaging device according to any one of the first to third aspects, the smear extracting means converts the image signal having a small exposure amount into the signal level adjusting means. It is characterized in that it has a first subtraction unit for subtracting an image signal having a large exposure amount whose level has been adjusted, and has the following operation. That is, the smear can be accurately and easily extracted.

According to a fifth aspect of the present invention, in the solid-state imaging device according to any one of the first to fourth aspects, the smear removing means extracts the image signal having a large exposure amount by the smear extracting means. It is characterized in that it has a second subtraction unit that performs subtraction processing on the obtained smear component, thereby having the following operation. That is, the removal of the smear component can be performed accurately and easily.

According to a sixth aspect of the present invention, in the solid-state imaging device according to any one of the first to fifth aspects, the image synthesizing means has a linearity in a signal level change of an image signal having a large exposure amount. A first judgment unit for judging whether or not the signal level is present, and when the first judgment unit judges that the signal level has linearity, an image signal with a large exposure amount from which the smear component has been removed is output as a composite signal. On the other hand, when the first determination unit determines that the signal level does not have linearity, the continuity of the gradation with respect to the image signal having a small exposure amount and the image signal having a large exposure amount from which the smear component has been removed is added. And a signal selection unit that outputs the resultant signal as a composite signal after performing signal processing to maintain the following. That is, it is possible to combine images accurately and easily.

According to a seventh aspect of the present invention, in the solid-state imaging device according to any one of the first to sixth aspects, the smear extracting means includes a level detector for detecting a signal level of the extracted smear component. A first smear suppression unit that suppresses the smear component when the signal level of the smear component detected by the level detection unit does not reach a predetermined level. It has the following effects. That is, since the smear component is suppressed based on the detection result by the level detection unit, it is possible to suppress a noise component that is conspicuous when the signal level of the smear component does not reach a predetermined level, and the image synthesis accuracy is accordingly reduced. Will increase.

According to an eighth aspect of the present invention, in the solid-state image pickup device according to any one of the first to seventh aspects, the smear extracting means has a characteristic that a signal level change of an image signal having a large exposure amount has a linearity. A second determination unit that determines whether or not the signal has a signal level change; and a second smear suppression unit that suppresses and outputs the extracted smear component when the second determination unit determines that the signal level change does not have linearity. And has the following effects. That is, the signal level of an image signal having a large exposure amount becomes unstable near the point where the signal level change becomes non-linear. For this reason, the extraction accuracy of the smear component extracted near the point where the signal level change of the image signal having a large exposure amount does not have linearity becomes poor. According to the present invention, in the vicinity of a point where the signal level change of the image signal having a large exposure does not have linearity, the output is performed in a state where the smear component is suppressed, whereby the extraction accuracy of the smear component in this region is improved. An adverse effect caused by the defect can be eliminated.

According to a ninth aspect of the present invention, in the invention according to any one of the first to eighth aspects, the imaging means has a solid-state imaging device in which color filters having different spectral characteristics are arranged for each pixel. The smear extraction performed by the smear extraction means and the smear removal performed by the smear removal means are performed for each pixel, thereby having the following operation. That is, the smear component can be removed even in a solid-state imaging device that performs color display.

Hereinafter, embodiments of the present invention will be described with reference to the drawings.

Embodiment 1 FIG. 1 is a configuration diagram showing a configuration of a solid-state imaging device according to Embodiment 1 of the present invention. The solid-state imaging device includes a solid-state imaging device 1 serving as an imaging unit, a signal level adjustment unit 2 that adjusts a signal level of an output signal of the solid-state imaging device 1, and a smear component extracted from the output signal of the solid-state imaging device 1 whose level has been adjusted. The solid-state imaging device 1 includes a smear extraction unit 3A, a smear removal unit 4 that removes a smear component from an output signal of the solid-state imaging device 1, and an image combination unit 5A that performs a synthesis process of the solid-state imaging device 1 from which smear has been removed.

The solid-state image pickup device 1 uses an all-pixel readout type solid-state image pickup device capable of outputting a signal having twice as many scanning lines as a normal solid-state image pickup device. FIG. 2 shows a configuration diagram of an example of such an all-pixel readout solid-state imaging device 1. The solid-state imaging device 1 includes a photoelectric conversion unit 6 arranged two-dimensionally, a vertical transfer unit 7 capable of transferring charges read by the photoelectric conversion unit 6 in the vertical direction, and a read-out operation to the vertical transfer unit 7. A pair of horizontal transfer units 8a and 8b for serially outputting the charged electric charges in the next field period,
and output amplifiers 9a and 9b for amplifying the outputs of a and 8b. In FIG. 2, the number of pixels is 4 × 6 pixels, but actually, for example, VGA (Video
Graphics Array) image format, etc.
A configuration of 0 × 480 pixels vertically is adopted. In the solid-state imaging device 1 configured as described above, an electronic shutter operation is performed so that a pixel corresponding to one of a pair of adjacent scanning lines has a normal exposure amount, and a pixel corresponding to the other scanning line. By performing an electronic shutter operation so that the pixel to be exposed is smaller than the normal exposure amount, a long exposure signal Slong (an exposure amount is large), which is an output signal having a different exposure amount,
A short-time exposure signal Sshort (a small exposure amount) is obtained.

The signal level adjusting means 2 includes a solid-state image sensor 1
While adjusting the signal level of the long-time exposure signal Slong output from the solid-state imaging device 1 (specifically, attenuating the signal at a constant attenuation rate (1 / N)). An adjustment unit 11 that passes through the processing and adjusts the level of the long exposure signal Slong is provided. The adjusting unit 11 is constituted by, for example, an attenuator.

The smear extracting means 3A outputs the short-time exposure signal S
From the short, the attenuated signal Slong / N, which is an attenuated long-time exposure signal, is subtracted [Sshort− (Slong /
N)] to generate a pseudo smear signal SM ′, and includes a first subtraction unit 12. The first subtraction unit 12 includes, for example, a subtractor.

The smear removing means 4 outputs the long exposure signal Slo
ng from the pseudo smear signal SM ′ (Slong−S
M ') to generate the pseudo original long exposure signal SlongO', while the short exposure signal Sshort is passed through unprocessed, and the second subtraction unit 1 performs the above subtraction processing.
3 is provided. The second subtraction unit 13 is composed of, for example, a subtractor.

The image synthesizing means 5A outputs the short-time exposure signal Ssh
The adder 14 adds the offset to the ort, and the signal level of the long-time exposure signal Slong is compared by comparing the signal level of the offset short-time exposure signal Sshort ′, which is the short-time exposure signal after the offset addition, with the threshold value Ssat ′. A comparator 15A that determines whether the change has linearity, and a pseudo original long exposure signal SlongO ′ and an offset short exposure signal Sshor based on the determination result of the comparator 15A.
and a selector 16 that selects one signal from t ′ and outputs the selected signal as a composite signal Smix. In the present embodiment, the comparator 15A constitutes a first determination unit, and the adder 14 and the selector 16 constitute a signal selection unit.

Next, the operation of the solid-state imaging device will be described. First, the operation of the solid-state imaging device 1 will be described.

A long exposure signal Slong (indicated by ● in FIG. 2) obtained by photoelectric conversion during a vertical effective period in one field period in the television system by the photoelectric conversion unit 6 is vertically transferred during a vertical blanking period. It is read by the unit 7. In addition, a short-time exposure signal Sshort obtained by photoelectric conversion during the vertical blanking period (in FIG.
Is also read out to the vertical transfer unit 7 independently of the long exposure signal Slong during the vertical blanking period. The long exposure signal Slong and the short exposure Sshort obtained in this way are sequentially and independently transferred from the vertical transfer unit 7 to the horizontal transfer units 8a and 8b, respectively, and the output amplifier 9a is output during the valid period of the next field. And 9b.

FIG. 3 shows the relationship between the long-time exposure signal Slong and the short-time exposure signal Sshort when the horizontal axis indicates the amount of incident light and the vertical axis indicates the output signal level of the solid-state image sensor 1. Here, it is known that the smear component SM maintains a constant level regardless of the output signal level of the solid-state imaging device 1. There is a correlation between the S / N ratio between the output signal level and the noise component, and when the signal level of the output signal is low, the noise component becomes conspicuous. Therefore, although the long exposure signal Slong is saturated with the saturation light amount A, a signal with a good S / N ratio can be obtained below the saturation light amount A. Conversely, the short-time exposure signal Ssh
The ort has a characteristic that the signal amount is small and the S / N ratio is poor below the saturation light amount A, but the signal can be reproduced even at the saturation light amount A or more.

Based on such characteristics, in the present embodiment, the long exposure signal Slong and the short exposure signal Sshort are synthesized after adjusting the signal level, whereby the S / N ratio is good and the dynamic I get a signal with a high range. Hereinafter, this will be described in detail based on specific operations.

The long-time exposure signal Slong and the short-time exposure signal Sshort output from the solid-state imaging device 1 include a constant smear component SM regardless of the magnitude of the signal level. Therefore, the long exposure signal Slong and the short exposure signal Sshort are obtained by adding the smear component SM to the original long exposure signal Slong0 and the original short exposure signal Sshort0 as shown in the following equation.

Slong = Slong0 + SM Sshort = Sshort0 + SM In this way, the long-time exposure signal Slong and the short-time exposure signal Sshort containing a constant smear component SM irrespective of the magnitude of the signal level are output from the solid-state image pickup device 1. It is inputted to the adjusting means 2 and the smear removing means 4.

In the signal level adjusting means 2, the adjusting section 11 attenuates the level of the long-time exposure signal Slong to 1 / N,
While outputting the attenuation signal Slong / N, the short-time exposure signal Sshort is output in an unprocessed state. The attenuation rate (1 / N) of the adjustment unit 11 is set as follows. That is, when the exposure amount of the short-time exposure signal Sshort is set to 1, the attenuation factor (1/1) is based on the ratio 1: N between the exposure amount of the short-time exposure signal Sshort and the exposure amount of the long-time exposure signal Slong.
N) is set.

The adjustment signal Slong / N and the unprocessed short-time exposure signal Sshort are input to the smear extracting means 3A.
In the smear extraction means 3A, the first subtraction unit 12 performs a process of subtracting the attenuation signal Slong / N from the short-time exposure signal Sshort (Sshort-Slong / N) to generate a pseudo smear signal SM 'and generate a smear. Output to the removing means 4.

The pseudo smear signal SM 'thus created is expressed by the following equation.

SM ′ = Sshort−Slong / N By substituting the above equations into the above equations, the following equations are obtained.

SM ′ = Sshort0 + SM−Slong0 / N−SM / N Since the ratio of the exposure amount between the original long-time exposure signal Slong0 and the original short-time exposure signal Sshort0 is N: 1, the following expression is used. Is rewritten as

SM ′ = SM × (N−1) / N Here, if N is sufficiently large, the pseudo smear signal SM ′
Can be regarded as equivalent to the smear component SM. For example, if N, which is an approximate ratio between the vertical effective period and the vertical blanking period, is 16, SM ′ = SM × 15/16, and SM ′ ≒ SM.

In the smear removing means 4, the second subtraction section 13 converts the long exposure signal Slong into the pseudo smear signal S
A process of subtracting (removing) M '(Slong-SM') is performed to generate and output the pseudo original long exposure signal Slong0 ', while passing the short exposure signal Sshort unprocessed.

The pseudo original long exposure signal SlongO '
The short exposure signal Sshort is input from the smear removing unit 4 to the image synthesizing unit 5A. In the image combining means 5A,
The adder 14 adds a specific offset Soff to the input short-time exposure signal Sshort (Ss
hort + Soff), the offset short-time exposure signal Ss
hort 'is created and output to the selector 16 and the comparator 15A. On the other hand, the input pseudo original long exposure signal SlongO 'is output to the selector 16 without processing.

The offset Soff added by the adder 14
Is set as follows: That is, as shown in FIG. 3, the signal level obtained by subtracting the smear component SM from the saturation signal level Ssat, which is the signal level of the long-time exposure signal Slong at the saturation light amount A, is set to a threshold level Ssat ′ (= Ssat−S
M). Then, at the saturation light amount A, the offset short-time exposure signal Sshort 'is set to the threshold level Ssat'.
Is added to the short-time exposure signal Sshort. Therefore, Soff = Ssat'-Ss
hort.

In the comparator 15A, the input offset short-time exposure signal Sshort 'and the above-described threshold level Ssat'
And outputs the result of the determination to the selector 16.
In the selector 16, when the comparator 15A determines that the offset short-time exposure signal Sshort 'is larger than the threshold level Ssat' (Sshort '>Ssat'), the signal level change of the long-time exposure signal Slong has linearity. If not, the selector 16 selects the short-time offset exposure signal Sshort 'as the composite signal Smix and outputs it. On the other hand, in the comparator 15A, the offset short-time exposure signal Sshort 'is smaller than the threshold value Ssat' (Sshort '<Ssa
If it is determined to be t '), it is determined that the signal level change of the long exposure signal Slong has linearity, and the pseudo original long exposure signal Slong0' is selected and output as the composite signal Smix.

FIG. 3 shows photoelectric characteristics showing the relationship between the signal level of each part and the amount of incident light at this time. In FIG. 3, the synthesized signal Smix is a long-time exposure signal Slong and a short-time exposure signal Ss
It can be seen that the smear SM included in hort is suppressed. That is, it is possible to perform the same signal processing as that in which the smear component SM is removed from the output signals (Slong, Sshort) having different exposure amounts output from the solid-state imaging device 1 and then the synthesis processing is performed. Can be suppressed and the smear component SM can be suppressed.

In the above description, the smear extracting means 3
A is configured only by the first subtraction unit 12, but the first
An amplifier is provided on the output side of the subtraction unit 12, and the first subtraction unit 12
By multiplying the output (pseudo smear signal SM ′) by N / (N−1), the smear component SM can be completely suppressed.

In the above description, the signal level obtained by subtracting the smear component SM from the saturation signal level Ssat, which is the signal level of the long exposure signal Slong at the saturation light amount A, is set as the threshold level Ssat '(= Ssat-SM). Thus, it was determined whether or not the change in the signal level of the long-time exposure signal Slong had reached the saturation light amount A, and the presence or absence of linearity was determined. However, the long exposure signal Slong has the saturation light amount A
It is known that the linearity of the signal level is lost as described above. Therefore, a value slightly smaller than the above-described threshold value Ssat ′ may be set as the threshold value.
It can be determined that the signal level change becomes non-linear due to the long-time exposure signal Slong becoming unstable at the position before this.

In the present embodiment, the solid-state image pickup device 1 of the all-pixel readout type is used as the image pickup means. However, by using a normal solid-state image pickup device, the solid-state image pickup device is driven at twice the frequency as compared with the conventional one. Can also obtain signals with different exposure amounts, and the same effects as in the present embodiment can be obtained.
However, in this case, there is a time difference in the time required for the signal components corresponding to the same pixel to be transferred in the image sensor, so that even for the same pixel, the smear component differs due to a change in the subject. The effect of smear suppression may be weakened.

Further, in the present embodiment, the monochrome solid-state image pickup device 1 is used as the image pickup means, but a multi-plate image pickup device using an optical prism and a color filter having different spectral characteristics for each pixel are provided. The same effect can be obtained by using a single-plate image sensor.

Further, in this embodiment, the solid-state imaging device 1 is used as the imaging means, and the output signal of the solid-state imaging device 1 is subjected to the smear removal processing. The same effect can be obtained by performing smear removal processing after performing the signal processing described above.

Further, in the present embodiment, the case of 16: 1 has been described as an example of the ratio (N: 1) of the exposure amount between the long exposure signal Slong and the short exposure signal Sshort. If the attenuator 11 in the adjusting means 2 is configured by a digital circuit and N is adjusted to be a power of 2, the attenuator 11 can be easily configured by a bit shift circuit.

Embodiment 2 In Embodiment 1 described above, the second subtractor 13 constituting the smear removing means 4 forcibly subtracts the pseudo smear signal SM 'from the long exposure signal Slong (Slong- S
M '). This is a solid-state image sensor 1
Is a configuration on the assumption that the smear component SM always exists in the output of. However, the solid-state imaging device 1
In some cases, the smear component SM does not exist in the output of the solid-state device 1 depending on the illumination state of the subject during the output of the solid-state device. In such a case, if the pseudo-smear signal SM 'is forcibly subtracted by the second subtractor 13, a noise component is added by the processing, and the pseudo-original long-time exposure which is the image signal after the subtraction processing is performed. Signal Slo
There is a possibility that the S / N ratio in ngO 'is deteriorated.

Therefore, in the present embodiment, the smear extracting means extracts the smear component (the pseudo smear signal S).
M ′) is detected, and the output of the pseudo smear signal SM ′ is permitted only when the signal level reaches an arbitrary level, thereby suppressing the deterioration of the S / N ratio described above. ing.

FIG. 4 is a configuration diagram showing the configuration of the smear extraction means of the solid-state imaging device according to the present embodiment. In the present embodiment, the configuration and operation of the solid-state imaging device other than the smear extracting unit are the same as those of the first embodiment, and thus description thereof will be omitted.

The smear signal extracting means 3 B included in the solid-state imaging device according to the present embodiment has a base clip circuit 20. The base clip circuit 20 detects the signal level of the pseudo smear signal SM ′ output from the first subtractor 12 and blocks the output of the pseudo smear signal SM ′ until the signal level reaches an arbitrary lower limit level SMO ₁ . extracted pseudo smear signal SM is configured to permit the output of the 'signal level of the pseudo smear signal SM Once you reach the any lower level SMO _1'. In the present embodiment, the level detection unit and the first smear suppression unit are configured by the base clip circuit 20.

Hereinafter, the operation of this embodiment will be described. The smear extracting means 3B receives the attenuation signal Slong / N and the short-time exposure signal Sshort as in the first embodiment, and extracts the pseudo smear signal SM 'by the first subtractor 12. Pseudo smear signal SM 'is input to the base clip circuit 20, where the pseudo smear signal SM' signal level pseudo smear signal if any lower level SM0 ₁ below S the
Block the output of M ', that is, the pseudo smear signal S
'While the output level forcibly zero, pseudo smear signal SM' M if the signal level of the lower limit level SM0 ₁ or more, suppresses a signal obtained by subtracting the lower limit level SM0 ₁ from the pseudo smear signal SM 'pseudo smear signal rSM Output as'. Note that the lower limit level SMO is derived from the pseudo smear signal SM '.
_The reason why ₁ is subtracted is to maintain the continuity of gradation. FIG. 5 shows the input / output characteristics of the base clip circuit 20.

As described above, the suppression pseudo-smear signal rSM ′ in which the noise component is suppressed is input to the second subtraction unit 13 of the smear removal means 4, where it is subtracted from the long exposure signal Slong. Same as 1.

When the smear component SM is removed as described above, an image signal in which the noise component is suppressed and the dynamic range is expanded can be obtained.

[0058] In this embodiment is adapted to set the signal level of the noise component contained in the smear component SM as the lower limit level SM0 _1. This lower level SMO ₁
Depends on the noise level of the image signal output from the solid-state imaging device 1. Therefore, it is conceivable to change this value according to the imaging conditions. However, since the lower-limit level SM0 ₁ following the smear component SM is not suppressed, it is necessary to adjust the level of the lower limit level SMO ₁ against the smear reduction effect and noise reduction effect.

Further, the long-time exposure signal Slong and the short-time exposure signal Ssh in the solid-state image pickup device 1 depend on photographing conditions and the like.
In some cases, it is not possible to completely match the exposure ratio (1: N) with the ort and the amount of attenuation in the adjusting unit 11 of the signal level adjusting means 2. In such a case, the smear component SM There is a possibility that an error will occur in the removal of. However, in the present embodiment, the base clip circuit 20
As a result, the signal level of the pseudo smear signal SM ′ can be suppressed, so that it is possible to suppress the error including the generated error.

In the present embodiment, the level detecting section and the first smear suppressing section are constituted by the base clip circuit 20, but the level detecting section and the first smear suppressing section are separately constituted separately. Needless to say.

Embodiment 3 FIG. 6 is a configuration diagram showing a configuration of a smear extracting means of the solid-state imaging device according to the present embodiment. In the present embodiment, the configuration and operation of the solid-state imaging device other than the smear extracting unit are the same as those of the first embodiment, and thus description thereof will be omitted.

In this embodiment, the smear extracting means detects the signal level of the extracted smear component (pseudo smear signal SM '), and the signal level is reduced to the lower limit level SMO.
Until reaching ₂ By configured to attenuation processing pseudo smear signal SM ', S which has been a problem in the second embodiment
This is to suppress the deterioration of the / N ratio with another configuration.

The smear extracting means 3C of the present embodiment detects the level of the pseudo smear signal SM ', and attenuates the pseudo smear signal SM' based on the detection result of the level detector 21. And a smear attenuation section 22. The level detection unit 21 is configured by, for example, a comparator. The smear attenuator 22 is composed of, for example, an attenuator.
Note that, in the present embodiment, the level detection unit is configured by the level detection unit 21, and the first smear suppression unit is configured by the smear attenuation unit 22.

Hereinafter, the operation of the present embodiment will be described. In the smear extraction means 3, the attenuation signal S
The long / N and short-time exposure signal Sshort are input, and the first attenuator 12 extracts the pseudo smear signal SM ′. The pseudo smear signal SM ′ is input to the level detector 21 and the smear attenuator 22. Then, the level detector 21, the signal level of the pseudo smear signal SM 'as compared with any lower limit level SM0 _2, the comparison result smear attenuation section 2
Output to 2. In the smear attenuator 22, the pseudo smear signal SM 'is changed to the lower limit level SM based on the input comparison result.
If O ₂ or less, a signal obtained by attenuating the signal level of the pseudo smear signal SM ′ at an arbitrary attenuation rate is converted to an attenuated pseudo smear signal dS.
Output as M '. On the other hand, without pseudo smear signal SM attenuates the signal level of 'a lower limit level SMO ₂ or if the pseudo smear signal SM', attenuated pseudo smear signal dS
Output as M '. FIG. 7 shows input / output characteristics of the smear attenuator 22.

In the present embodiment, the pseudo smear signal SM ′
Set as a lower limit level SM0 ₂ levels of noise components included in the same manner as the second embodiment, the pseudo smear signal S
The pseudo smear signal S whose M ′ does not reach the lower limit level SMO ₂
M ′ is determined to contain a large amount of noise components and is subjected to an attenuation process, thereby suppressing the noise components from being mixed into the pseudo smear signal SM ′. This makes it possible to obtain an image signal in which the noise component is suppressed and the dynamic range is expanded.

Embodiment 4 In Embodiments 1 to 3 described above, the comparator 15A
Offset short exposure signal Sshort 'and threshold level Ssa
By comparing and judging with t ′, it was judged whether or not the incident light amount reached the saturated light amount A. Here, the short-time exposure signal S
short has a relatively wide dynamic range but S / N
The long exposure signal Slong has a relatively narrow dynamic range but a good S / N ratio. Therefore, in order to further improve the accuracy of determining the saturated light amount A, the long-time exposure signal Slo having a good S / N ratio is used.
It is better to use a signal based on ng as a criterion. In the present embodiment, the position of the saturation light amount A is determined based on the pseudo original long-time exposure signal SlongO ′ from such a point of interest.

However, when the position of the saturated light amount A is determined based on the signal based on the long exposure signal Slong, there are the following inconveniences. That is, the long-time exposure signal Slong is saturated when the light intensity is equal to or more than the saturation light amount A, and has a characteristic that the signal level change has no linearity near the saturation light amount A. The signal SM 'cannot be obtained. Therefore, the pseudo original long-time exposure signal SlongO 'generated based on the pseudo smear signal SM' cannot have high generation accuracy, and the position determination accuracy of the saturated light amount A is not good.

Therefore, in the present embodiment, a high luminance region where the long exposure signal Slong is saturated (a region where the signal level change of the long exposure signal Slong has no linearity).
By suppressing the pseudo smear signal SM ', the malfunction of the smear suppression processing (error in determining the position of the saturated light amount A) due to the erroneous extraction of the pseudo smear signal SM' in the high luminance region is suppressed.

FIG. 8 is a configuration diagram showing the configuration of the solid-state imaging device according to the fourth embodiment of the present invention. In Embodiment 4,
The configuration and operation of the solid-state imaging device other than the smear extracting unit and the image synthesizing unit are the same as those of the first embodiment, and therefore, the same or equivalent parts are denoted by the same reference numerals and description thereof will be omitted. I do.

The image synthesizing means 5B of this solid-state imaging device is
Threshold value Ssat 'and pseudo original long exposure signal Slong
It has a comparator 15B as a first judgment unit for comparing and judging the signal level with O ′. Also, smear extraction means 3D
Cuts the short-time exposure signal Sshort at Ssat '/ N, and then applies the clipped signal cSshort to the first subtractor 12.
Is provided. Note that, in the present embodiment, the clipping circuit 23 outputs the second determination unit and the second
A smear suppression unit is configured.

Hereinafter, the operation of the solid-state imaging device according to the present embodiment will be described. As in the first embodiment, an attenuation signal Slong / N and a short-time exposure signal Sshort are input to the smear extraction unit 3D. In the clip circuit 23, the short-time exposure signal S
By determining whether or not the short has reached the threshold level Ssat '/ N, it is determined whether or not the signal level change of the long exposure signal Slong has linearity. Is not reached, the change in the signal level of the linearity exposure signal Slong is assumed to have linearity, and the clipping circuit 23 outputs the short-time exposure signal Sshort as it is as a clip short-time exposure signal cSshort without any processing. On the other hand, if the threshold level Ssat '/ N is exceeded, it is determined that the signal level change of the linear exposure signal does not have linearity, and the signal clipped at the signal level of Ssat' / N is clipped short exposure signal. Output from the clip circuit 23 as cSshort.

The clip short-time exposure signal cSshort created by the clip circuit 23 is input to the first subtraction unit 12,
Here, the decay signal Slong / N is subtracted from the clip short-time exposure signal cSshort 'to generate a clip pseudo smear signal cSM'. Clip pseudo smear signal cS
M ′ is input to the smear removing means 4.

In the smear removing means 4, the second subtractor 13 subtracts (removes) the clip pseudo smear signal cSM 'from the long exposure signal Slong (Slong-cS).
M ′) to generate and output a clip pseudo original long exposure signal cSlong0 ′, while passing the short exposure signal Sshort unprocessed.

Clip pseudo original long exposure signal c
SlongO ′ and the short-time exposure signal Sshort are input from the smear removing unit 4 to the image synthesizing unit 5B. In the image synthesizing means 5B, an offset short-time exposure signal Sshort 'is created in the adder 14 by adding a specific offset Soff to the input short-time exposure signal Sshort (Sshort + Soff), and a selector 16 is provided. Output to On the other hand, the input clip pseudo original long exposure signal cSlongO 'is output to the comparator 15B and the selector 16 without processing.

The comparator 15 B compares the input clip pseudo-original long-time exposure signal cSlongO ′ with the above-mentioned threshold level Ssat ′, and outputs the determination result to the selector 16. In the selector 16, in the comparator 15B, the clip pseudo original long exposure signal cSlong
O 'is greater than threshold level Ssat' (cSlongO '> S
If it is determined to be sat ′), the selector 16 selects the short-time offset exposure signal Sshort ′ as the composite signal Smix and outputs it, assuming that the signal level change of the long-time exposure signal Slong does not have linearity. On the other hand, the comparator 15A
In the clip pseudo original long exposure signal cSl
ongO 'is smaller than the threshold value Ssat' (cSlongO '<Ssa
t '), it is determined that the signal level change of the long exposure signal Slong has linearity, and the clip pseudo original long exposure signal cSlong0' is combined with the composite signal Smix.
And output.

FIG. 9 shows a clip short exposure signal cSshor.
The signal level change of t ', the attenuation signal Slong / N, and the clip pseudo smear signal cSM' is shown.

The threshold level Ssat '/ N is determined according to the first embodiment.
The threshold value Ssat ′ of the threshold value 3 is multiplied by the attenuation rate 1 / N in the attenuation unit 11. Threshold level Ssa
t ′ is set smaller than the signal level Ssat of the long-time exposure signal Slong at the saturation light amount A. for that reason,
The threshold level Ssat '/ N is smaller than the signal level when the short-time exposure signal Sshort reaches the saturation light amount A (≒ saturation signal level Ssat / N). As a result, the short-time exposure signal Sshort has an incident light amount B (B <B
It can be seen that the threshold level Ssat '/ N is reached in A).

The clip short exposure signal cSshort does not rise above the threshold level Ssat '/ N like the attenuation signal Slong / N, and the clip pseudo smear signal cSM' becomes saturated light amount A accordingly. It can be seen that the signal level becomes zero. Moreover, as described above, the short-time exposure signal Sshort reaches the threshold level Ssat '/ N at the incident light amount B lower than the saturated light amount A. Therefore, the clip pseudo smear signal _C SM 'is the amount of incident light B
Until the saturation light amount A, the signal level gradually decreases from the incident light amount B to the saturation light amount A. When the saturation light amount A is reached, the level finally becomes zero.

The incident light amount region from the incident light amount B to the saturation light amount A is a region where the change in the signal level of the long exposure signal Slong has no linearity, and is created based on such a long exposure signal Slong. Pseudo-smear signal S
The signal level of M 'is incorrect. On the contrary,
In the present embodiment, the clip pseudo smear signal in this area
'By reducing the level gradually, the pseudo smear signal _C SM' _C SM in a state to eliminate the instability of the signal level at, have created a clip pseudo original long exposure signal slongo '. Thus, even if the position of the saturated light amount A is determined based on the pseudo original long exposure signal SlongO ', the position can be accurately determined.

As described above, in the present embodiment, a high-luminance region where the long-time exposure signal Slong is saturated (a region where the signal level change of the long-time exposure signal does not have linearity)
Outputs a suppressed clip pseudo smear signal cSM ′. Therefore, the pseudo smear signal cSM ′ whose signal level has been suppressed in the high luminance region is the long exposure signal Slon.
will be removed from g. As a result, in the image synthesizing unit 5B, it is possible to suppress the smear component without causing a malfunction in the high luminance region (without erroneous determination of the position of the saturated light amount A), thereby expanding the dynamic range. be able to.

In this embodiment, the clip circuit 2
3, the second determination unit and the second smear suppression unit are configured.
It goes without saying that the second determination unit and the second smear suppression unit may be separately configured separately.

Embodiment 5 When a solid-state imaging device in which color filters having different spectral characteristics are arranged for each pixel is used as the imaging means, when the output signal is band-limited and smear is removed as a luminance signal,
Since the difference in smear level generated for each column cannot be removed, an unnatural color is added to a portion where smear has occurred.

Therefore, in the present embodiment, the smear extraction operation by the smear extraction means and the smear removal operation by the smear removal means are performed for each pixel, so that the smear signal of each column due to the color filter array of the solid-state imaging device is obtained. The generation of false colors due to the difference is suppressed.

The present embodiment will be described below with reference to the drawings. The configuration of the solid-state imaging device according to the present embodiment is the same as that of the first embodiment except for the solid-state imaging device 1 and the image synthesizing unit 5 as the imaging unit, and therefore the same or similar parts are denoted by the same reference numerals. Is omitted.

First, the configuration of the solid-state imaging device 30 as the imaging means of the present embodiment will be described with reference to FIG. The solid-state imaging device 30 constituting this solid-state imaging device includes a photoelectric conversion device in which color filters having four different spectral characteristics of magenta (Mg), green (Gr), yellow (Ye), and cyan (Cy) are arranged for each pixel. Although it has a conversion unit 31, its operation is exactly the same as that of the solid-state imaging device 1 described in the first embodiment. However, as the output signals, four signals of magenta + yellow (MY), magenta + cyan (MC), green + yellow (GY), and green + cyan (GC) are obtained by the pixel mixing operation of the solid-state imaging device 30. These are output as a long exposure signal Slong and a short exposure signal Sshort, respectively.

According to the arrangement of the color filters, a signal having MY and GC as one set and MC and GY as one set is
The output is alternately output for each line, and the set of MY and GC and the set of MC and GY are output alternately for each pixel. The spectral characteristics of each color filter are designed so that the sum of MY and GC and the sum of MC and GY become the spectral characteristics of the luminance signal Y. Therefore, the signal array of MY and GC and the signal sequence of MC and GY are output as signals in which the luminance signal component Y is modulated and superimposed on the color signal components Cr and Cb.

In the arrangement of the color filters as shown in FIG. 11, when viewed from the left end, only three types of color filters of Mg, Ye, and Gr are arranged in odd columns, and M
Only three types of color filters, g, Cy and Gr, are arranged. The smear component SM is generated due to leakage of electric charges from the photoelectric conversion unit 31 to the vertical transfer unit 7, and in such a color filter array, the smear component SM signal level differs for each column. On the other hand, the output signals of the solid-state imaging device 30 (long exposure signal Slong, short exposure signal Sshor)
It is conceivable to remove smear as a luminance signal by limiting the band of t). However, this method cannot remove the difference in the smear level for each pixel column, so that an unnatural color appears in the area where the smear occurs. It will stick.

On the other hand, in the solid-state imaging device according to the present embodiment, the smear extraction by the smear extraction means 3 and the smear removal by the smear removal means 4 are performed for each pixel, thereby removing the difference in the smear level for each pixel column. This makes it possible to completely suppress the smear component SM,
An output signal free from false colors can be obtained. The reason why the difference in smear level for each pixel column can be removed is as follows.

In general, when processing the output signal of a national body image pickup device in which such color filters are arranged, a color signal is obtained from a difference signal level between pixels of signals output from pixels of different filters. . Therefore, it is possible to perform processing in which the timing of each pixel column is strictly adjusted, or processing in which A / D conversion is performed for each pixel and no deviation occurs for each pixel column.

Next, a specific configuration of the present embodiment will be described. The image synthesizing unit 5C included in the solid-state imaging device according to the present embodiment is configured such that the long-time exposure signal Slo
amplifying section 32 that amplifies short-time exposure signal Sshort with a gain corresponding to the light amount ratio of short-time exposure signal Sshort and outputs amplified short-time exposure signal Sshort ″, amplified short-time exposure signal Sshort ″ and threshold value Ssat ′ 15C as a first judgment unit for comparing and judging the pseudo original long exposure signal SlongO ′ and the amplified short exposure signal Sshort ″ based on the judgment result of the comparator 15C to produce a composite signal Smix And a selector 16 for outputting the result.

Next, the operation of the image synthesizing means 5C will be described. The short-time exposure signal Ssh input to the image synthesizing means 5C
The ort is amplified by the amplifier 32 with a gain corresponding to the light amount ratio of the long exposure signal Slong and the short exposure signal Sshort. Exposure time of long exposure signal Slong is 1/60 second,
When the exposure time of the short-time exposure signal Sshort is 1/1000 second, the gain is 16 times.

The amplified short exposure signal Sshort ″ amplified by the amplifier 32 is compared with the threshold value Ssat ′ in the comparator 15C, and the result of the comparison is output to the selector 16. In the selector 16, the input comparison result is Ssh
ort ''> Ssat ', amplified short-time exposure signal Sshor
t '' is selected and output as a composite image signal Smix. On the other hand, when the input comparison result is Sshort '<Ssat', the pseudo original long-time exposure signal Slong0 'is selected and output as the composite image signal Smix. As a result, smear extraction by the smear extraction means 3 and smear removal by the smear removal means 4 can be performed for each pixel, and a smear signal can be completely suppressed to obtain an output signal free of false colors.

The configuration of the image synthesizing means 5C of the present embodiment is different from that of the first embodiment (short-time exposure signal S
The reason why the short is amplified by the amplifying unit 32 is that the output signal (Slong, Sshort) of the solid-state imaging device 30 has a modulation component due to a chrominance signal. There is no direct relationship with them.

In each of the above embodiments, the image synthesizing means 5A, 5B, 5C uses the long exposure signal Slong
The pseudo original long exposure signal SlongO 'and the offset short exposure signal Sshort' (amplified short exposure signal Ss) are determined based on whether or not the signal level change has linearity.
hort '') and output as a composite signal Smix. This is because it is effective to use the linearity of the change in the signal level of the long-time exposure signal Slong as a criterion in removing the smear component. However, the pseudo original long exposure signal SlongO 'and the offset short exposure signal Sshort' (the amplified short exposure signal Ss
The judgment criterion for switching to “hort”) may be finely adjusted based on other requirements in performing image adjustment. That is, depending on other image adjustment requirements, the long exposure signal Slo
ng based on a criterion (signal level) slightly adjusted from a signal level (threshold value Ssat ') serving as a criterion for determining whether or not the ng signal level change has linearity. The short exposure signal Sshort '(amplified short exposure signal Sshort'') may be switched.

[0095]

As described above, according to the present invention, the smear component is removed from the output signals having different exposure amounts output from the image pickup means, and then the synthesis processing is performed. The effect of being able to suppress is obtained.

[Brief description of the drawings]

FIG. 1 is a block diagram illustrating a configuration of a solid-state imaging device according to a first embodiment of the present invention.

FIG. 2 is a diagram illustrating a configuration of an imaging element as imaging means of the solid-state imaging device according to the first embodiment;

FIG. 3 is a photoelectric characteristic diagram illustrating a relationship between a signal level of each unit of the solid-state imaging device according to the first embodiment and an incident light amount;

FIG. 4 is a block diagram illustrating a configuration of a smear extracting unit of the solid-state imaging device according to the second embodiment of the present invention;

FIG. 5 is an input / output characteristic diagram of a smear extraction unit of the solid-state imaging device according to the second embodiment;

FIG. 6 is a block diagram illustrating a configuration of a smear extracting unit 3 of the solid-state imaging device according to the third embodiment of the present invention.

FIG. 7 is an input / output characteristic diagram of a smear extraction unit of the solid-state imaging device according to the third embodiment;

FIG. 8 is a configuration diagram illustrating a configuration of a smear extraction unit of the solid-state imaging device according to the fourth embodiment of the present invention;

FIG. 9 is a photoelectric characteristic diagram illustrating a relationship between a signal level of each unit of the solid-state imaging device according to the fourth embodiment and an incident light amount;

FIG. 10 is a block diagram illustrating a configuration of a solid-state imaging device according to a fifth embodiment of the present invention;

FIG. 11 is a diagram illustrating a configuration of a solid-state imaging device as an imaging unit according to a fifth embodiment.

FIG. 12 is a block diagram illustrating a configuration of a conventional solid-state imaging device.

FIG. 13 is a photoelectric characteristic diagram illustrating a relationship between a signal level of each unit of a conventional solid-state imaging device and an incident light amount.

[Explanation of symbols]

REFERENCE SIGNS LIST 1 solid-state imaging device 2 signal level adjusting means 3A smear extracting means 3B smear extracting means 3C smear extracting means 3D smear extracting means 4 smear removing means 5A image synthesizing means 5B image synthesizing means 5C pixel synthesizing means 6 photoelectric conversion unit 11 adjusting unit 12th 1 subtraction unit 13
Second subtraction unit 14 Adder 15A
Comparator 15B Comparator 16
Selector 20 Base clip circuit 21
Level detection unit 22 Smear attenuation unit 23
Clip circuit 30 solid-state image sensor

Claims (9)

[Claims] In a solid-state imaging device including an imaging unit that outputs a plurality of image signals having different exposure amounts during a period of capturing an image for one screen, a signal level of the image signals having different exposure amounts is adjusted. Signal level adjusting means, smear extracting means for extracting a smear component from image signals having different levels of light exposure, and smear removing means for removing the extracted smear components from the image signals having different light exposures. And an image synthesizing unit for synthesizing image signals having different exposure amounts. 2. The solid-state imaging device according to claim 1, wherein the signal level adjusting unit is configured to make a signal level of the image signal having a large exposure amount substantially coincide with a signal level of the image signal having a small exposure amount. A solid-state imaging device comprising an adjustment unit for adjusting. 3. The solid-state imaging device according to claim 2, wherein the adjustment unit includes an attenuator that attenuates a signal level of an image signal having a large exposure amount. 4. The solid-state imaging device according to claim 1, wherein the smear extracting unit converts an image signal having a small exposure amount into an image signal.
A solid-state imaging device, comprising: a first subtraction unit that performs subtraction processing on an image signal having a large exposure amount whose level has been adjusted by the signal level adjustment unit. 5. The solid-state imaging device according to claim 1, wherein the smear removing unit subtracts a smear component extracted by the smear extracting unit from an image signal having a large exposure amount. A solid-state imaging device comprising a two subtraction unit. 6. The solid-state imaging device according to claim 1, wherein said image synthesizing unit determines whether a signal level change of an image signal having a large amount of exposure has linearity. When the first determining unit and the first determining unit determine that the signal level change has linearity, an image signal with a large exposure amount from which a smear component has been removed is output as a combined signal, while the first signal is output.
When the determination unit determines that the signal level change does not have linearity, a signal that maintains the continuity of gradation with respect to the image signal with a large exposure with the smear component removed is added to the image signal with a small exposure. A solid-state imaging device comprising: a signal selection unit that performs processing and outputs the resultant signal as a composite signal. 7. The solid-state imaging device according to claim 1, wherein the smear extracting unit detects a signal level of the extracted smear component, and detects the signal level of the extracted smear component. A solid-state imaging device further comprising: a first smear suppressor that suppresses the smear component when the signal level of the smear component does not reach a predetermined level. 8. The solid-state imaging device according to claim 1, wherein the smear extracting unit determines whether a signal level change of an image signal having a large exposure amount has linearity. (2) a second smear suppressing section that suppresses and outputs the extracted smear component when the signal level change does not have linearity in the second judging section. Solid-state imaging device. 9. The solid-state imaging device according to claim 1, wherein the imaging unit has a solid-state imaging device in which color filters having different spectral characteristics are arranged for each pixel, and A solid-state imaging device, wherein the smear extraction performed by the smear extraction unit and the smear removal performed by the smear removal unit are performed for each pixel.