JPH0541868A - Solid-state image pickup device - Google Patents

Abstract

PURPOSE:To eliminate only a pulsive noise selectively generated from a solid- state image pickup element. CONSTITUTION:The device consists of 1st, 2nd delay circuits 4, 5 delaying a digital signal by one horizontal scanning period, 1st, 2nd, 3rd low pass filters 6, 7, 8 using a half of the clock frequency of the digital signal as a node frequency, a noise elimination device 9 eliminating a defect noise signal from a luminance signal with correlation between signals outputted from the low pass filters 6, 7, 8 in the horizontal and vertical directions, a horizontal aperture circuit 10 correcting contour in the vertical direction with the signal whose noise is eliminated, and a vertical aperture circuit 11 correcting contour in the vertical direction with the signal whose noise is eliminated, to detect a picture defect caused by the defect of a light receiving element of the solid-state image pickup element 1 or ununiformity of the light receiving characteristic based on the correlation characteristic in the horizontal and vertical direction of the picture and to correct automatically a defective picture element signal with the horizontal and vertical signals.

Description

Detailed Description of the Invention

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a solid-state image pickup device for automatically correcting a defective pixel signal in a color-difference sequential type solid-state image pickup device.

[0002]

2. Description of the Related Art In recent years, solid-state image pickup devices are often used as photoelectric conversion means in image pickup devices such as video cameras. However, there is a defect in the light receiving element of the solid-state imaging device or a pixel defect caused by nonuniformity of the light receiving surface. Therefore, image defects called white spots (black spots) and black spots (black spots) occur in the luminance signal obtained by passing the signal output from the solid-state image sensor through a low-pass filter. Such image defects are very conspicuous even if they are small, and an image pickup device having an image defect cannot be used as a product, which leads to a decrease in yield and may cause an image defect even after the product is shipped. .. These problems are disadvantages peculiar to the solid-state image pickup device, and are major obstacles in using the solid-state image pickup device.

Conventionally, as means for solving the above problems,
As disclosed in Japanese Unexamined Patent Publication No. 61-261974, there is known a method of detecting and removing a pixel defect of a target pixel by a simple comparison between the target pixel and an adjacent pixel group.

[0004]

SUMMARY OF THE INVENTION In the prior art,
It is possible to remove the image defect by comparing the pixel of interest and its neighboring pixel group.
The pixel of interest and its neighboring pixel group are of the same type of signal (for example, if the luminance signal is a comparison of luminance signals), the second precondition is that a pixel defect occurs sporadically and It is based on the premise that the correlation of is low. However, no consideration has been given to the color filter array,
Even if the output of the solid-state image sensor is directly input to the device according to the conventional technique, different types of color difference information are included between neighboring pixels, the first precondition is not satisfied, and it is difficult to remove noise. Therefore, it is necessary to extract the luminance signal from the output of the solid-state image sensor. Although a low-pass filter is used to obtain the luminance signal, the noise component of the defective pixel is diffused to adjacent pixels by passing the filter, and the second precondition is not satisfied, and noise is removed by the conventional technique. It was difficult.

In view of the above points, the present invention favors a pulsative image defect such as a white flaw or a black flaw even if it is diffused into a plurality of pixels by a filter process without impairing significant image information. It is in the provision of a device for removal.

[0006]

In order to solve the above-mentioned problems, a solid-state image pickup device of the present invention comprises a first low-pass filter, a second low-pass filter, and a third low-pass filter whose pole frequency is 1/2 of the clock frequency of a digital signal. , A noise eliminator that removes a defective noise signal of a luminance signal by the correlation of the signal output from the low-pass filter in the horizontal and vertical directions, a horizontal aperture circuit that performs horizontal contour correction using the denoised signal, and further noise It is characterized by being constituted by a vertical aperture circuit which performs contour correction in a vertical direction by the removed signal.

[0007]

With the above structure, the image defect caused by the defect of the light receiving element of the solid-state image pickup device or the nonuniformity of the light receiving characteristic is detected by the horizontal and vertical correlation characteristics of the image,
A feature is that the defective pixel signal is automatically corrected by the horizontal and vertical signals.

Further, when removing an image defect, it is necessary to determine whether or not each pixel in the image signal contains the image defect, but the criterion for determining the image defect to be removed is unnecessarily widened. However, since significant image information such as detail components in the image signal may be lost, the present invention uses white (black) diffused to surrounding pixels by the filter.
It is characterized by having an action of selectively removing scratches.

In order to generate a luminance signal from the output of the solid-state image sensor, generally, one of the clock frequencies of the digital signal is used.
Digital low-pass filter (1+
Since it is generated by applying Z ⁻¹ ), the flaw component is also diffused in the scanning line direction. Since the range of pixels diffused by the filter is only one pixel adjacent to each other in the scanning line direction, it is necessary to make a determination based on a neighboring pixel group excluding adjacent pixels including scratches before and after one pixel of the target pixel. .. In order to remove adjacent pixels that contain scratches, when the pixel of interest is a white (black) scratch,
It can be realized by excluding one having a large (small) pixel value from one pixel before and after. In other words, the pixel having the smaller (larger) pixel value and the neighboring pixel group other than the preceding and following one pixel are compared with the target pixel.

Next, if a pixel is determined to be a flaw, the flaw can be erased by acting to replace the pixel value with the surrounding pixel values. But here,
If a correlation is found between surrounding pixels, it is necessary to eliminate the flaw after maintaining the correlation. Here, regarding the correlation, it is necessary to consider not only the horizontal direction with respect to the scanning line but also the vertical direction or the oblique direction. To achieve this, if the pixel of interest is determined to be a white (black) flaw, the pixel value group of the pixel of interest is the pixel value group excluding the pixel value including the flaw from the two-dimensional neighboring pixel value group. Of the maximum (small) pixel value of the sub-pixel.

In the input signal of the vertical aperture circuit for performing contour correction in the vertical direction, a defect signal is detected by correlation of signals in the two horizontal scanning period signals without increasing the circuit scale.
Replace.

[0012]

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS An embodiment of the present invention will be described below with reference to the drawings.

FIG. 1 is a schematic block diagram of the present invention. In FIG. 1, 1 is a solid-state image sensor, 2 is an automatic gain control circuit for making the signal (a) level output from the solid-state image sensor 1 constant, and 3 is an analog signal output from the automatic gain control circuit 2 ( AD converters for converting b) into digital signals, 4, 5 are first and second delay circuits for delaying the digital signal (c) output from the AD converter 3 by one horizontal scanning period, 6, 7 , 8 are the first, second, and third whose pole frequency is 1/2 of the clock frequency of the digital signal.
, A noise removing circuit 9 for removing the defective noise signal of the luminance signal by the horizontal and vertical correlation of the signals (e) output from the low pass filters 6, 7, 8; A horizontal aperture circuit that performs horizontal contour correction from the removed signals (h1 to h3), a vertical aperture circuit 11 that performs vertical contour correction from the signal from which noise has been further removed by the horizontal aperture circuit 10, and a luminance signal. The processing unit 13 is an adder.

FIG. 2 shows a color filter array of the solid-state image pickup device used in the present invention. A first color filter row that repeats in the horizontal direction a first color filter that transmits green light and a second color filter that blocks green light provided on a two-dimensionally arranged light receiving element, and a third color filter that transmits yellow light. A second color filter row in which a color filter and a cyan color transmitting fourth color filter are repeated in the horizontal direction, and a third color filter row in which the first and second color filters are inverted and repeated with respect to the first color filter. A color filter row and a color filter row that repeats the same repeating color filter row as the second color filter row in the vertical direction, and among the first to fourth color filters in one horizontal scanning period, in the vertical direction. This is a drive structure for sequentially reading out signals from two adjacent color filter columns.

FIG. 3 shows a luminance signal noise elimination circuit. A
The D-converted signal (c) is delayed, and the signal (f) after passing through the LPF is subjected to noise removal in each line to obtain a signal (h). Each block is shown in Fig. 4 below.
~ It demonstrates in description of FIG.

FIG. 4 is a block diagram of the pixel delay. This shows an example of the configuration when M = N = 3 in the explanation of the operation of the scanning line memory, and shows two line delay units 4, 5
And six pixel delay devices 24 to 29. The pixel delay device outputs a signal obtained by delaying the input signal by a scanning time of one pixel, and the line delay device outputs a signal obtained by delaying the input signal by one horizontal scanning period. Further, FIGS. 8 to 10 show examples of the arrangement of the target pixel and the neighboring pixel group on the scanning line, which are denoted by the numbers f1 to f9.

Image signal c input from the input terminal of FIG.
The signal (e1) obtained by passing (1 + Z ⁻¹ ) of the LPF 6 is connected to the pixel delay device 24, and the output of the pixel delay device 24 is connected to the pixel delay device 25. If signals are taken out from the input end of the pixel delay unit 24, the output end of the pixel delay unit 24, and the output end of the pixel delay unit 25, the pixels at the positions corresponding to f1, f2, and f3 in FIGS. The value is obtained. The image signal (c) input from the input terminal of FIG. 4 is also connected to the line delay unit 4, and the output signal (d2) of the line delay unit 4 is L (1 + Z ^-1 ) as described above.
If the pixel delay devices 26 and 27 are connected in series to the signal (c2) obtained by passing the signal through the PF 7, then similarly to FIGS.
Pixel values at positions 0, f4, f5, and f6 are obtained. Further, the line delay unit 5 is connected to the output terminal of the line delay unit 4 of FIG.
Are connected, and the output signal (d3) of the line delay device 5 is changed to (1
If the signal obtained by passing the LPF 8 of + Z ⁻¹ ) is connected in series to the pixel delay units 28 and 29, similarly, as shown in FIG.
Pixel values at positions corresponding to f7, f8, and f9 in FIG. 10 are obtained. With the scanning line memory configured as described above, the target pixel and the neighboring pixel group can be simultaneously output from the plurality of output terminals.

In FIG. 3, the replacement candidate pixel value detector 18
Is the largest pixel among the pixel value groups f1 to f3 and f7 to f9 of the neighboring pixel group that are simultaneously output from the scanning line memory, among the pixel values of f4 and f6 having the smallest pixel value and the remaining neighboring pixel value group. The value is output as Ymax, and the minimum pixel value out of f4 and f6 having a larger pixel value and the remaining neighboring pixel value group is output as Ymin. This Ymax and Ym
in is the replacement candidate pixel value group.

FIG. 5 is a detailed first diagram of the replacement candidate pixel value detector.
It is a configuration example of. In FIG. 5, in accordance with the description of the scanning line memory, the target pixel is set to f5, and the neighboring pixel group has 8 pixels (f
1 to f3, f7 to f9), a configuration example of a replacement candidate pixel value detector is shown, in which a large selection circuit 30 and a small selection circuit 31 having two input terminal groups and seven It is composed of a minimum value circuit 32 and a maximum value circuit 33 having an input terminal group. The small selection circuit 31 selects f4 and f6 located one pixel before the target pixel from the input neighboring pixel value group.
The one with the smallest pixel value is output, and the maximum value circuit 33
Is the maximum pixel value (Ymax (g) from the remaining neighboring pixel value groups f1 to f3, f7 to f9 and the output of the small selection circuit 31.
4)) is extracted and output. The large selection circuit 30 outputs the one having a larger pixel value out of f4 and f6 located one pixel before and after the pixel of interest from the input neighboring pixel value group, and the minimum value circuit 32 outputs the remaining neighboring pixel value group. f1-f3, f7-
The minimum pixel value (Y
min (g3)) is extracted and output.

In FIG. 3, the noise detector 19 uses the pixel value f5 of the pixel of interest output from the scanning line memory and Ymax (g4) and Ymin (g3) output from the replacement candidate pixel value detector 18, It is determined whether or not the target pixel contains noise, and the determination result is output as a control signal. The control signal is SELm which is a signal for selecting Ymax.
It is composed of SELmin which is a signal for selecting ax and Ymin.

FIG. 7 shows a noise detector 19 and a pixel value selector 2
0 is a detailed configuration example of 0, and the noise detector 19 includes two subtractors 38 and 41, two comparators 40 and 43, and two constant value setters 39 and 42. The subtractor 38 subtracts the pixel value of the target pixel (f5) from Ymin (g3) to generate the difference (hereinafter referred to as DIFmin). The comparator 40 compares DIFmin with a certain value (hereinafter THmin) output from the certain value generator 39, and activates a signal SELmin (11) for selecting Ymin if DIFmin> THmin as condition 1. If the condition 1 is not satisfied, SELmin (11) is made inactive and output. Further, the subtractor 41 subtracts Ymax (g4) from the pixel value of the target pixel (f5) and generates the difference (hereinafter referred to as DIFmax). The comparator 43 compares DIFmax with a certain value (hereinafter THmax) output from the certain value generator 42, and as a condition 2, DIFmax is set.
A signal S for selecting Ymax if max> THmax
ELmax (12) is activated, and when the condition 2 is not satisfied, SELmax (12) is inactivated and output. Generally, the high-frequency component (detail) of the image signal output from the photoelectric conversion element is produced by the spatial low-pass filter effect due to the aperture ratio of the lens system and the photoelectric conversion unit.
Since the amplitude of is small, if the pixel of interest does not include pulse noise, it can be said that the difference between the pixel of interest and the neighboring pixel group is also small. Therefore, THmax and THm
If in is set too small with respect to the maximum pixel value, the detail component of the image signal may be lost, and if set too large, the flaw itself cannot be detected, so it must be set to an appropriate value. ..

The pixel value selector 20 is composed of one multiplexer 44. The multiplexer 44 inputs the pixel value of the pixel of interest, three signals of Ymax and Ymin, and SE
If both Lmax and SELmin are inactive, the pixel value of the target pixel is output to the output terminal (h2), and SEL
If max is active and SELmin is inactive, Ymax is output to the output terminal (h2) and SELm
If ax is inactive and SELmin is active, Ymin is output to the output terminal (h2). here,
As can be seen from the description of the noise detector 19, SELmax
From the condition of outputting SELmin and SELmin, it is impossible that SELmax and SELmin are both active.

In FIG. 3, the output signal (h1) is an input signal to the vertical aperture circuit 11. The replacement candidate pixel value detector 15 uses f1 and f3 from the pixel value groups f1 and f3 to f6 of the neighboring pixel group that are simultaneously output from the scanning line memory.
Ymax of the maximum pixel value from the one having the smaller pixel value and the remaining neighboring pixel group, and the smallest pixel from the one having the larger pixel value of f1 and f3 and the remaining neighboring pixel value group. The value is output as Ymin. The Ymax and Ymin form a replacement candidate pixel value group.

FIG. 6 is a detailed second configuration example of the replacement candidate pixel value detector 18. In FIG. 6, in accordance with the description of the scanning line memory, the configuration of the replacement candidate pixel value detector in the case where the pixel of interest is f2 and the neighboring pixel group is composed of 5 pixels (f1, f3 to f6) (see FIG. 9) An example is shown, and a large selection circuit 34 and a small selection circuit 35 having two input terminal groups,
It is composed of a minimum value circuit 36 and a maximum value circuit 37 having a group of seven input terminals. The small selection circuit 35 is positioned f before and after the pixel of interest from the input neighborhood pixel value group.
The maximum value circuit 37 outputs the one having the smallest pixel value out of 1 and f3, and the maximum value circuit 37 outputs the maximum pixel value (Ymax (g) from the remaining neighboring pixel value groups f4 to f6 and the output of the small selection circuit 35.
2)) is extracted and output. The large selection circuit 34 outputs one of the input neighboring pixel value groups having a larger pixel value out of f1 and f3 located one pixel before and after the pixel of interest, and the minimum value circuit 36 outputs the remaining neighboring pixel value group. From among the outputs of the large selection circuit 34 and f4 to f6, the minimum pixel value (Ymin (g
1)) is extracted and output.

In FIG. 3, the noise detector 16 uses the pixel value f2 of the pixel of interest output from the scanning line memory and Ymax (g2) and Ymin (g1) output from the replacement candidate pixel value detector 15. It is determined whether or not the target pixel contains noise, and the determination result is output as a control signal. The control signal is SELm which is a signal for selecting Ymax.
It is composed of SELmin which is a signal for selecting ax and Ymin. In FIG. 3, the pixel selector 17 is configured similarly to the pixel selector 20 to obtain the output signal (h1).

In FIG. 3, the output signal (h3) is also (h3
Similar to 1), it is an input signal to the vertical aperture circuit 11. The replacement candidate pixel value detector 21 sets f8 of the signals simultaneously output from the scanning line memory as a pixel of interest, and determines that the pixel value of f7 and f9 is smaller than the pixel value groups f4 to f7 and f9 of the neighboring pixel group. The maximum pixel value out of the remaining neighboring pixel value group is output as Ymax, and the minimum pixel value out of the remaining pixel value group of f7 and f9 having a large pixel value is output as Ymin. This Ymax and Ym
in is the replacement candidate pixel value group. (See FIG. 10) In FIG. 3, the noise detector 22 and the pixel value selector 23 are configured similarly to the noise detector 16 and the pixel selector 17, and obtain the output signal (h3).

In FIG. 1, a horizontal aperture circuit 10 which performs horizontal contour correction by the signal from which noise has been removed by the luminance signal noise removal circuit 9, and a vertical aperture circuit 11 which performs vertical contour correction by the signal from which noise has been removed.
The image defect caused by the defect of the light receiving element of the solid-state image sensor 1 or the nonuniformity of the light receiving characteristic is detected by the correlation characteristic in the horizontal and vertical directions of the image, and the defective pixel signal is automatically detected by the horizontal and vertical signals. Correct to.

Further, in the present invention, since the processing is performed by focusing on only the two-dimensional or pseudo two-dimensional correlation of the image and the time axis is not concerned, it is not affected by the movement of the subject. Absent.

[0029]

As is apparent from the above-described invention, according to the present invention, an image diffused by a filter, which is difficult by the conventional technique, can be obtained without impairing significant image information regardless of the movement of the object. It becomes possible to satisfactorily remove the defect, and the practical effect thereof is great.

[Brief description of drawings]

FIG. 1 is a schematic block diagram of an embodiment of the present invention.

FIG. 2 is a layout diagram of a color filter used in an embodiment of the present invention.

FIG. 3 is a block diagram showing a configuration of a luminance signal noise elimination circuit according to an embodiment of the present invention.

FIG. 4 is a block diagram showing a pixel delay configuration according to an embodiment of the present invention.

FIG. 5 is a block diagram showing a first configuration of a replacement candidate pixel value detector according to an embodiment of the present invention.

FIG. 6 is a block diagram showing a second configuration of a replacement candidate pixel value detector according to an embodiment of the present invention.

FIG. 7 is a block diagram showing the configuration of a pixel value selector according to an embodiment of the present invention.

FIG. 8 is a layout diagram showing a first layout example of a target pixel and a neighboring pixel group according to an embodiment of the present invention.

FIG. 9 is a layout diagram showing a second layout example of a target pixel and a neighboring pixel group according to an embodiment of the present invention.

FIG. 10 is a layout diagram showing a third layout example of a target pixel and a neighboring pixel group according to an embodiment of the present invention.

[Explanation of symbols]

 1 Solid-state image sensor 2 Automatic gain control circuit 3 AD converter 4 1 line delay device 5 1 line delay device 6 Digital low pass filter 7 Digital low pass filter 8 Digital low pass filter 9 Luminance signal noise elimination circuit 10 Horizontal aperture processing circuit 11 Vertical aperture processing Circuit 12 Luminance signal processing circuit 13 Aperture mix circuit 14 Pixel delay device 15 Replacement candidate pixel value detector 16 Noise detector 17 Pixel value selector 18 Replacement candidate pixel value detector 19 Noise detector 20 Pixel value detector 21 Replacement candidate pixel Value detector 22 Noise detector 23 Pixel value selector 24 Pixel delay device 25 Pixel delay device 26 Pixel delay device 27 Pixel delay device 28 Pixel delay device 29 Pixel delay device 30 Large selection circuit 31 Small selection circuit 32 Maximum value selection circuit 33 Minimum value selection circuit 34 Large selection circuit 3 Small selection circuit 36 the maximum value selection circuit 37 the minimum value selection circuit 38 subtractor 39 Arne setter 40 comparator 41 subtractor 42 Arne setter 43 comparator 44 multiplexer

Claims (2)

[Claims] 1. A first color filter row in which a first color filter for transmitting green light and a second color filter for blocking green light, which are provided on a two-dimensionally arranged light receiving element, are repeated in a horizontal direction,
A second color filter row in which a third color filter that transmits yellow light and a fourth color filter that transmits cyan light are horizontally repeated, and the first and second color filters are the first color filter row. And a third color filter row that is repeated by being inverted, and a color filter that vertically repeats the same repeated color filter row as the second color filter row, and has a color filter that repeats in the vertical direction. Of the four color filter arrays, a solid-state image sensor having a driving structure for sequentially reading out signals of two color filter arrays that are vertically adjacent to each other, and an automatic gain for making the signal level output from the solid-state image sensor constant. A control circuit, an AD converter that converts an analog signal from the automatic gain control circuit into a digital signal, first and second delay circuits that delay the digital signal by one horizontal scanning period, and a clock frequency of the digital signal First of 1/2 and the pole frequency of,
A second and a third low-pass filter, a noise removing circuit for removing the defective noise signal of the luminance signal by the correlation of the signal output from the low-pass filter in the horizontal and vertical directions, and a signal from which the noise is removed by the noise removing circuit. A horizontal aperture circuit that performs horizontal contour correction and a vertical aperture circuit that performs vertical contour correction by a noise-removed signal, similar to the horizontal aperture circuit, are configured, and a defect of the light receiving element of the solid-state imaging device or An image defect caused by non-uniformity of the light receiving characteristic is detected by the horizontal and vertical correlation characteristics of the image, and the defective pixel signal is automatically corrected by the horizontal and adjacent vertical color signals. .. 2. The input signal to the vertical aperture circuit is A
The first and second detecting D-converter and the first and second horizontal scanning period delay circuits detect defective pixel signals by correlation of signals in two horizontal scanning period signals in each of three consecutive horizontal scanning periods. A noise removal circuit comprising a noise detection circuit and first and second correction circuits for correcting noise components by utilizing horizontal and vertical correlations of two horizontal scanning period signals adjacent to each other by the signal detected by the circuit. The solid-state imaging device according to claim 1, wherein the solid-state imaging device is input via