JP3156281B2 - Solid-state imaging device - Google Patents

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a solid-state imaging device for automatically correcting a defective pixel signal in a solid-state imaging device of a color difference sequential system.

[0002]

2. Description of the Related Art In recent years, in an image pickup apparatus such as a video camera, a solid-state image pickup device is often used as a means for photoelectric conversion. However, there is a pixel defect caused by a defect of the light receiving element of the solid-state imaging device or non-uniformity of the light receiving surface. For this reason, image defects called white points (white defects) and black points (black defects) occur in the luminance signal obtained by passing the signal output from the solid-state imaging device through a low-pass filter. Such an image defect is very conspicuous even if it is small, and an image pickup device having an image defect cannot be used as a product, so that the yield is reduced. In addition, an image defect may occur even after the product is shipped. . These problems are disadvantages peculiar to the solid-state imaging device and are a major obstacle in using the solid-state imaging device.

Conventionally, as means for solving the above problems,
As disclosed in JP-A-61-261974, there is known a method of detecting and removing a pixel defect of a target pixel by a simple comparison between a target pixel and an adjacent pixel group.

[0004]

However, in the above prior art, it is possible to remove a pixel defect by comparing a target pixel with a group of neighboring pixels as a first precondition. The neighboring pixel group is a signal of the same kind (for example, if the luminance signal, the luminance signals are compared with each other). As a second precondition, a pixel defect is generated sporadically and the correlation between the defective pixel and the neighboring pixel is low. Based on However, no consideration has been given to the color filter arrangement. Even if the output of the solid-state imaging device is directly input to the device according to the related art, different types of color difference information are included between neighboring pixels, and the first precondition is not satisfied, and it is difficult to remove noise. Therefore, it is necessary to extract a luminance signal from the output of the solid-state imaging device. In order to obtain a luminance signal, the signal is passed through a low-pass filter, but the noise component of the defective pixel is diffused to an adjacent pixel by passing through the filter, and the second precondition is not satisfied and the noise is removed by the conventional technique. It was difficult.

[0005] In view of the above, the present invention can effectively eliminate pulsative image defects such as white scratches or black scratches even if the image information is diffused to a plurality of pixels by filter processing without losing significant image information. The present invention is to provide a removing device.

[0006]

In order to achieve the above object, a solid-state imaging device according to the present invention comprises a first, second, and third low-pass filters having a pole frequency of 1/2 of a clock frequency of a digital signal. A noise removing device for removing a defective noise signal of a luminance signal by a horizontal and vertical correlation of a signal output from a low-pass filter, a pixel delay circuit for delaying a signal by two pixels in a horizontal direction, and a pixel for further delaying two pixels A delay circuit, an adder for adding a signal without pixel delay and a four-pixel delay signal, a selector receiving the added signal and the two-pixel delay signal, and a selector control signal from a defective noise signal detected based on the luminance signal And an edge detection circuit for removing a defective noise signal of a color signal by a selector.

[0007]

With the above arrangement, an image defect caused by a defect of the light receiving element of the solid-state imaging device or a non-uniform light receiving characteristic is detected by horizontal and vertical correlation characteristics of the image, and a defective pixel signal is detected by the horizontal and vertical signals. It is characterized by automatic correction.

Further, when removing an image defect, it is necessary to determine which image in the image signal contains the image defect. However, if the criteria for determining the image defect to be removed are unnecessarily widened, Since significant image information such as detail components in an image signal may be lost, the present invention has an effect of selectively removing white (black) scratches diffused to surrounding pixels by a filter. It is characterized by.

A luminance signal is generated from an output of a solid-state image sensor.
In general, the clock frequency of the digital signal is set to 1
/ 2 digital low-pass filter with pole frequency (1
+ Z ^-1) Scratch components are generated by applying
It is spread in the direction. The range of pixels diffused by the filter
Note that the area is only one pixel adjacent in the scanning line direction.
Near one pixel before and after excluding neighboring pixels including flaws
It is necessary to make a determination based on the neighboring pixel group. Neighboring images including scratches
To remove the element, if the target pixel is white (black),
Excluding pixels with large (small) pixel values in the next one pixel
And can be realized with In other words, the pixel of one pixel before and after
Note the small (large) pixels and other neighboring pixels
Compare with the eye pixel.

Next, if a certain pixel is determined to be flawed, the flaw can be eliminated by acting to replace the pixel value with surrounding pixel values. However, here
If there is a correlation between the surrounding pixels, it is necessary to eliminate the scratches while maintaining the correlation. Here, regarding the correlation, it is necessary to consider not only the horizontal direction but also the vertical direction or the oblique direction with respect to the scanning line. In order to realize this, if the pixel of interest is determined to be a white (black) defect, the pixel value of the pixel of interest is changed to a pixel value group obtained by removing the pixel value including the defect from the two-dimensional neighboring pixel group. This is achieved by acting to replace the pixel having the maximum (small) pixel value. The control signal is used to remove noise from the color signal.

[0011]

An embodiment of the present invention will be described below with reference to the drawings.

FIG. 1 is a schematic block diagram of the present invention. In FIG. 1, 1 is a solid-state imaging device, 2 is an automatic gain control circuit for keeping the level of the signal (a) output from the solid-state imaging device 1 constant, and 3 is an analog signal (output) from the automatic gain control circuit 2. AD converters for converting b) into digital signals; 4, 5 are first and second delay circuits for delaying the digital signal (c) output from the AD converter 3 for one horizontal scanning period; , 8 are first, second, third and third polarities which are の of the clock frequency of the digital signal.
A digital low-pass filter 9; a noise removal circuit 9 for removing a defective noise signal of a luminance signal by horizontal and vertical correlation of the signals (e) output from the low-pass filters 6, 7, 8; h1 to h3) A luminance signal processing circuit including a horizontal aperture circuit for performing contour correction in the horizontal direction, and a vertical aperture circuit for performing contour correction in the vertical direction from a signal from which noise has been removed. A color signal noise elimination circuit 13 controlled by the control signal output from the color signal noise elimination circuit 12 is a color signal processing circuit that processes a color signal based on the output signal (o) of the color signal noise elimination circuit 12.

FIG. 2 shows a color filter array of a solid-state image sensor used in the present invention. A first color filter array that horizontally repeats a first color filter for green light transmission and a second color filter for green light blocking provided on a two-dimensionally arranged light receiving element, and a third color filter for yellow light transmission A second color filter row that repeats a color filter and a fourth color filter that transmits cyan in the horizontal direction, and a third color that repeats the first and second color filters by inverting the first color filter row. A filter row and a color filter that repeats the same color filter row as the second color filter row in the vertical direction, and is vertically adjacent to the first to fourth color filters in one horizontal scanning period. This is a drive structure for sequentially reading signals of two color filter rows.

[0014] Figure 3 is a block diagram showing the configuration of a Brightness signal noise removal circuit in FIG. The signal (c) subjected to the AD conversion is delayed, and the signal (f) after passing through an LPF (not shown ) is obtained as a signal (h) from which noise is removed in each line. For each block,
This is performed using FIG.

FIG. 4 is a configuration diagram of the pixel delay circuit . This shows a configuration example when M = N = 3 in the description of the operation of the scanning line memory.
5 and six pixel delay units 24-29.
The pixel delay unit outputs a signal obtained by delaying the input signal by one pixel scanning time, and the line delay units 4 and 5 output a signal obtained by delaying the input signal by one horizontal scanning time. . Also, 8 to 10 shows an arrangement example of the pixel of interest and the neighboring pixel group scanning line, each delay pixel
The output of the signal is given numbers f1 to f9 as in FIG.

An image signal c input from the input terminal of FIG.
Is passed through the (1 + Z ^-1 ) LPF, the signal (e1) obtained is connected to the pixel delay unit 24, and the output of the pixel delay unit 24 is connected to the pixel delay unit 25. The input terminal of the pixel delay unit 24, the output terminal of the pixel delay unit 24, and the pixel delay unit 2
If signals are extracted from three places at the output end of No. 5, pixel values at positions corresponding to f1, f2, and f3 in FIGS. 8 to 10 are obtained. Further, the image signal (c) input from the input terminal of FIG. 4 is also connected to the line delay unit 4, and the output signal (d2) of the line delay unit 4 is converted into the (1 + Z ^-1 ) L signal as described above.
If the pixel delay units 26 and 27 are connected in series to the signal (e2) obtained by passing through the PF, similarly, FIGS.
The pixel values at the positions of f4, f5, and f6 are obtained. Further, the output terminal of the line delay unit 4 of FIG.
Is connected, and the output signal (d3) of the line delay unit 5 is changed to (1
If a signal obtained by passing through the LPF of + Z ^-1 ) is connected in series to the pixel delay units 28 and 29, pixel values at positions corresponding to f7, f8, and f9 in FIGS. . If the scanning line memory has the above configuration,
The target pixel and the neighboring pixel group can be simultaneously output from the plurality of output terminals.

In FIG. 3, a replacement candidate pixel value detector 18 is used.
Is the largest pixel among the smaller pixel values of f4 and f6 and the remaining neighboring pixel value groups from the pixel value groups f1 to f3 and f7 to f9 of the neighboring pixel groups output simultaneously from the scanning line memory. The value is output as Y _max , and the minimum pixel value is output as Y _min from among the pixels having the larger pixel value of f4 and f6 and the remaining neighboring pixel value groups. These Y _max and Y _{min form a} replacement candidate pixel value group.

FIG. 5 shows a detailed first example of the replacement candidate pixel value detector.
This is an example of the configuration. In FIG. 5, according to the description of the scanning line memory, the target pixel is set to f5, and the neighboring pixel group is set to 8 pixels (f
1 to f3, f7 to f9), a configuration example of a replacement candidate pixel value detector is shown, in which a large selection circuit 30 and a small selection circuit 31 having two input terminal groups, and seven It comprises a minimum value circuit 32 and a maximum value circuit 33 having an input terminal group. The small selection circuit 31 outputs a pixel having a smaller pixel value among f4 and f6 located one pixel before and after the pixel of interest from the input neighboring pixel value group, and the maximum value circuit 33 outputs the remaining neighboring pixel value group. From the outputs of f1 to f3, f7 to f9 and the small selection circuit 31, the largest pixel value (Y
_max (g4)) is extracted and output. The large selection circuit 30 outputs a pixel having a larger pixel value among f4 and f6 located one pixel before and after the pixel of interest from the input neighboring pixel value group, and the minimum value circuit 32 outputs the remaining neighboring pixel value group. f1-f
3, the minimum pixel value (Y _min (g3)) is extracted from the outputs of the large selection circuit 30 and f7 to f9 and output.

In FIG. 3, the noise detector 19 is connected to a scanning line.
The pixel value f5 of the target pixel output from the memory and the replacement
Y output from the complementary pixel value detector 18_max(G4) and
And Y_{m in}From (g3), the target pixel contains noise.
And outputs the result as a control signal.
You. The control signal is Y_maxIs a signal that selects_maxWhen
Y _minIs a signal that selects_minIt is composed of

FIG. 7 shows a noise detector 2 and a pixel value selector 20.
The noise detector 2 includes two subtractors 38 and 41, two comparators 40 and 43, and two certain value setting units 39 and 42. The subtractor 38 is Y
The pixel value of the target pixel (f5) is subtracted from _min (g3) to generate a difference (hereinafter, referred to as DIF _min ). Comparator 40
Compares the Arne (hereinafter referred to as TH _min) output from the DIF _min and Arne generator 39, DIF _min condition 1
> Signal SEL _min (1 to select TH if _min Y _min
1) is activated, and if condition 1 is not satisfied, S
EL _min (11) is made inactive and output. The subtractor 41 calculates Y _max from the pixel value of the target pixel (f5).
(G4) is subtracted to generate the difference (hereinafter referred to as DIF _max ). Comparator 43 DIF _max and Arne generator 42 (hereinafter referred to TH _max) Arne output from the comparing signal SEL _{m ax} for selecting the Y _max if DIF _max> TH _max condition 2 12 is made active, and if the condition 2 is not satisfied, SEL _max (12) is made inactive and output. Generally, the amplitude of the high-frequency component (detail) of the image signal output from the photoelectric conversion element becomes small due to the spatial low-pass filter effect due to the aperture ratio of the lens system and the photoelectric conversion unit. Is not included, it can be said that the difference between the neighboring pixel group and the target pixel is also small. Therefore, TH _max and T
If H _min is set too small with respect to the maximum pixel value, the detail component of the image signal may be lost. If H _min is set too large, the flaw itself cannot be detected, so it is necessary to set H _min to an appropriate value. is there.

The pixel value selector 20 includes one multiplexer 44. Multiplexer 44 inputs the pixel value of the pixel of interest, Y _max, the third signal of the Y _min, SEL _max
If both SEL _min and SEL _min are inactive, the pixel value of the target pixel is output to the output terminal (h2). If SEL _max is active and SEL _min is inactive, Y _max is output to the output terminal (h2). If SEL _max is inactive and SEL _min is active, Y _min is output to output terminal (h
Output to 2). Here, the Tokaru so the description of the noise detector 2, a condition for outputting the SEL _max and SEL _min, SEL _{ma x} and SEL _min is impossible both be active.

In FIG. 3, an output signal (h1) is an input signal to the vertical aperture circuit. The replacement candidate pixel value detector 15 calculates a pixel value group of the neighboring pixel groups f1 and f3 to f6 output from the scanning line memory at the same time. The maximum pixel value is output as Y _max from among them, and the minimum pixel value is output as Y _min from among the larger pixel values of f1 and f3 and the remaining neighboring pixel value groups. These Y _max and Y _{min form a} replacement candidate pixel value group.

FIG. 6 shows a detailed second example of the replacement candidate pixel value detector.
This is an example of the configuration. In FIG. 6, according to the description of the scanning line memory, the pixel of interest is set to f2, and the neighboring pixel group is composed of five pixels (f
1, f3 to f6) (see FIG. 9), showing a configuration example of a replacement candidate pixel value detector, in which a large selection circuit 34 and a small selection circuit 35 having two input terminal groups, 7
It comprises a minimum value circuit 36 and a maximum value circuit 37 having a number of input terminal groups. The small selection circuit 35 selects one of the input neighboring pixel values f
1 and f3, and outputs the pixel value with the smallest pixel value. The maximum value circuit 37 outputs the maximum pixel value (Y _max (g _max) from the remaining neighboring pixel value groups f4 to f6 and the output of the small selection circuit 35.
2)) is extracted and output. The large selection circuit 34 outputs a pixel having a larger pixel value among f1 and f3 located one pixel before and after the pixel of interest from the input neighboring pixel value group, and the minimum value circuit 36 outputs the remaining neighboring pixel value group. f4 to f6 and the smallest pixel value (Y _min (g
1)) is extracted and output.

In FIG. 3, the noise detector 16 has a scanning line
The pixel value f2 of the target pixel output from the memory and the replacement
Y output from the complementary pixel value detector 15_max(G2) and
And Y_{m in}From (g1), the target pixel contains noise.
And outputs the result as a control signal.
You. The control signal is Y_maxIs a signal that selects_maxWhen
Y _minIs a signal that selects_minIt is composed of

In FIG. 3, the pixel selector 17 is configured similarly to the pixel selector 20, and obtains an output signal (h1). The output signal (h3) is an input signal to the vertical aperture circuit similarly to h1. The replacement candidate pixel value detector 21 uses f8 of the signal output simultaneously from the scanning line memory as the target pixel, and calculates f7 from the pixel value groups f4 to f7 and f9 of the neighboring pixel group.
And f9, the maximum pixel value among the pixel values having a small pixel value and the remaining neighboring pixel value groups is Y _max , and further, f7 and f9
Among them, the smallest pixel value is output as Y _min from among those having a larger pixel value and the remaining neighboring pixel value groups. These Y _max and Y _{min form a} replacement candidate pixel value group as shown in FIG. The noise detector 22 and the pixel value selector 23 are configured similarly to the noise detector 16 and the pixel selector 17, and obtain an output signal (h3).

FIG. 11 is a part of a configuration diagram of a chrominance signal noise elimination circuit. SEL output by luminance noise elimination circuit
If either _min or SEL _max is active, the edge detector 49 for detecting the edge, the pixel delay units 45 to 48 for delaying the input signal (d) by four pixels, and the input signal (d) and the input signal It comprises an adder 50 for averaging the signal (s) delayed by four pixels and a selector 51 controlled by a signal output from the edge detector 49.

FIG. 12 illustrates the operation. SEL
_When either _min or SEL _max is active, the signal (u) detected by the edge detector 49 becomes active for one pixel period, and the signal (r) delayed by two pixels from the input signal is input. By replacing the signal (d) with the value (t) obtained by averaging the signal (s) delayed by four pixels in the adder 50, the noise of the color signal is removed from the signal within one horizontal scanning period. By controlling a similar circuit for each line, noise of the color signal is removed.

In FIG. 1, a horizontal aperture circuit for performing horizontal contour correction based on a signal from which the noise has been removed by the luminance signal noise removal circuit, and a vertical aperture circuit for performing vertical contour correction using a signal from which the noise has been removed. And a signal output from the chrominance signal noise elimination circuit 12
Through the sensor, an image defect caused by a defect in the light receiving element of the solid-state imaging device 1 or an uneven light receiving characteristic is detected by horizontal and vertical correlation characteristics of the image, and a defective pixel signal is detected by the horizontal and vertical signals. Automatically compensates for

Further, in the present invention, the processing is performed by focusing only on the two-dimensional or pseudo two-dimensional correlation of the image, and the time axis is not affected, so that the processing is not affected by the movement of the subject. .

[0030]

As is apparent from the above description, according to the present invention, regardless of the movement of the object, the image diffused by the filter, which is difficult in the prior art, without losing significant image information. Defects can be satisfactorily removed, and the practical effect is great.

[Brief description of the drawings]

FIG. 1 is a schematic block diagram of an embodiment of the present invention.

FIG. 2 is a layout diagram of color filters used in an embodiment of the present invention.

FIG. 3 is a configuration example of a luminance signal noise elimination circuit according to an embodiment of the present invention;

FIG. 4 is a block diagram showing a configuration of a pixel delay according to the embodiment of the present invention.

FIG. 5 shows a first example of a replacement candidate pixel value detector according to the embodiment of the present invention.
Block diagram showing a configuration example of

FIG. 6 shows a second example of the replacement candidate pixel value detector according to the embodiment of the present invention.
Block diagram showing a configuration example of

FIG. 7 is a block diagram illustrating a configuration of a pixel value selector according to an embodiment of the present invention.

FIG. 8 shows a first example of a pixel of interest and a group of neighboring pixels according to the embodiment of the present invention
Layout diagram showing a layout example

FIG. 9 shows a second example of the target pixel and the neighboring pixel group according to the embodiment of the present invention.
Layout diagram showing a layout example

FIG. 10 is an arrangement diagram showing a third arrangement example of a target pixel and a neighboring pixel group according to the embodiment of the present invention;

FIG. 11 is a block diagram of a color signal noise elimination circuit according to an embodiment of the present invention.

FIG. 12 is a diagram illustrating the operation of the color signal noise elimination circuit according to the embodiment of the present invention.

[Explanation of symbols]

 DESCRIPTION OF SYMBOLS 1 CCD 2 Automatic gain control circuit 3 A / D converter 4 1 line delay 5 1 line delay 6 Digital low pass filter 7 Digital low pass filter 8 Digital low pass filter 9 Luminance signal noise elimination circuit 11 Luminance signal processing circuit 12 Color signal noise elimination circuit Reference Signs List 13 color signal processing circuit 14 pixel delay 15 replacement candidate pixel value detector 16 noise detector 17 pixel value selector 18 replacement candidate pixel value detector 19 noise detector 20 pixel value selector 21 replacement candidate pixel value detector 22 noise Detector 23 Pixel value selector 24 Pixel delay device 25 Pixel delay device 26 Pixel delay device 27 Pixel delay device 28 Pixel delay device 29 Pixel delay device 30 Large selection circuit 31 Small selection circuit 32 Maximum value selection circuit 33 Minimum value selection circuit 34 Large selection circuit 35 Small selection circuit 36 Maximum value selection circuit 37 Maximum Value selection circuit 38 Subtractor 39 Some value setter 40 Comparator 41 Subtracter 42 Some value setter 43 Comparator 44 Multiplexer 45 Pixel delay circuit 46 Pixel delay circuit 47 Pixel delay circuit 48 Pixel delay circuit 49 Edge detector 50 Adder 51 Selector

──────────────────────────────────────────────────続 き Continued on front page (58) Field surveyed (Int.Cl. ⁷ , DB name) H04N ^9/ 04-9/11

Claims (2)

(57) [Claims] A first color filter array which horizontally repeats a green light transmitting first color filter and a green light blocking second color filter provided on a two-dimensionally arranged light receiving element; A second color filter row that horizontally repeats a third color filter for light transmission and a fourth color filter for cyan light transmission, and the first color filter row as the first and second color filters. And a third color filter row to be reversed and repeated, as in the second color filter row
Fourth, repeating the third and fourth color filters horizontally
The first, second, third, and second color filter rows
Color filters to repeat four of the color filter column in the vertical direction
It has the sequence, one from the first one horizontal scanning period of the fourth color filter rows, and the solid-state image sensor of sequentially reading drive structure signals of a color filter row of two rows adjacent in the vertical direction, the solid An automatic gain control circuit for keeping the signal level output from the element constant, an AD converter for converting an analog signal from the automatic gain control circuit into a digital signal, and a first serially connected to the AD converter. , The second
And one horizontal scanning period delay circuit, the output and the AD converter
Each of the first and second horizontal scanning period delay circuits is connected to the output of the delay circuit, and is １ ／ of the clock frequency of the digital signal.
First, second, and third low-pass filters each having a polar frequency as input, and three signals output from the first, second, and third low-pass filters as inputs, and detecting horizontal and vertical correlations of the three signals. and a luminance signal noise elimination circuit be complemented by the horizontal and vertical signals adjacent uncorrelated portion, said first
First and second horizontal scanning period delay circuit and AD conversion
A signal representing horizontal and vertical decorrelation detected by the luminance signal noise elimination circuit by inputting an output signal from the
A color signal noise elimination circuit that complements the noise signal of each horizontal line signal with an adjacent signal as a control signal , and detects an image defect caused by a defect in the light receiving element of the solid-state imaging device or a non-uniform light receiving characteristic. Detects defective pixel signals based on the horizontal and vertical correlation characteristics of the luminance signal.
A solid-state imaging apparatus characterized by automatically correcting the horizontal and vertical pixel signal adjacent to No.. 2. A color signal noise elimination circuit comprising: a first pixel delay circuit for delaying a signal by two pixels in a horizontal direction ;
A second pixel delay circuit connected in series to the pixel delay circuit for delaying the signal by two more pixels, and averaging the input signal to the first pixel delay circuit and the output signal of the second pixel delay circuit Supplied from adder and luminance signal noise elimination circuit
Of the adder as a control signal
Output signal or the output signal of the first pixel delay circuit.
Constituted by a selector for-option to output, the color signal noise
The solid-state imaging device according to claim 1, wherein the removing.