JPH04304091A - Picture noise eliminating device - Google Patents

Abstract

PURPOSE:To selectively eliminate the impulsive noise that a solid-state image pickup device generates by providing a scanning line memory, a candidate picture element group detector, a noise detector and a picture element value selector. CONSTITUTION:A scanning line memory 2 holds a picture signal (a) temporarily and outputs a neighbourhood picture element value group (c) composed of picture element value (b) of a notice element, N picture element in the scanning direction with the notice picture element as the center and the value of MXN- 1 picture element that the notice picture element is eliminated from the picture element group closing two- dimensionally for M line in the right angle direction to the N picture element. A replacement candidate picture element value group detector 3 detects a maximum picture element value (d) and a minimum picture element value e from the picture element value group (c). A noise detector 4 detects the noise included in the notice picture element comparing with the prescribed threshold and generates control signals (f), (g). A picture element value selector 5 selects either one of picture element values (b), (c), (e) by the control signals (f), (g). By this constitution, necessary picture information is not eliminated by mistake as the two dimensional correlation of the original picture is noticed and the decision is performed by the threshold at the time of noise decision.

Description

[Detailed description of the invention]

0001

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to an image noise removal device for a video camera or the like having an optical-to-electrical conversion means.

0002

2. Description of the Related Art In recent years, solid-state imaging devices have been increasingly used as optical-to-electrical conversion means in imaging devices such as video cameras. When the output signal of a solid-state image sensor is displayed on a monitor device and the image signal is observed, image defects called white spots (white scratches) and black spots (black spots) may be observed in the image. Even if such image defects are small, they are very noticeable, and an image sensor with an image defect cannot be used as a product, resulting in a decrease in yield.In addition, image defects may occur even after the product is shipped. . These problems are shortcomings specific to solid-state image sensors, and are major obstacles to the use of solid-state image sensors.

[0003] Conventionally, as a means to solve the above problems,
As disclosed in Japanese Patent Application Laid-Open No. 61-261974, a method is known in which a pixel defect in a pixel of interest is detected and removed by a simple comparison between the pixel of interest and a group of adjacent pixels.

0004

[Problem to be solved by the invention] In the conventional technology,
The first prerequisite for being able to remove image defects by comparing the pixel of interest with its neighboring pixel group is the following:
The pixel of interest and its neighboring pixel group are signals of the same type (for example, in the case of luminance signals, the luminance signals are compared), and the second premise is that pixel defects occur singly and there is no correlation between the defective pixel and neighboring pixels. It is based on the premise that gender is low. However, no consideration has been given to the color filter array based on the single-plate color difference coloring method. In the single-chip color difference coloring method, even if the output of the solid-state image sensor is directly input to a device using the conventional technology, different types of color difference information are contained between neighboring pixels, and the first precondition does not hold. Since it is difficult to remove noise, it is necessary to extract the luminance signal or color difference signal from the output of the solid-state image sensor in advance. In order to obtain a luminance signal or a color difference signal, it is necessary to perform filter processing, but by passing through the filter, the noise component of the defective pixel will be diffused to the adjacent pixels, and the second precondition will not be met, so the conventional It was difficult to remove noise using this technology.

[0005] In view of the above, the present invention effectively removes pulsed image defects such as white scratches or black scratches, even if they are diffused into a plurality of pixels by filter processing, without impairing significant image information. The aim is to provide equipment for removing

[0006]

[Means for Solving the Problems] According to the present invention, there is provided a means for temporarily retaining pixel information of several scanning lines, and a means for temporarily retaining pixel information of a scanning line spanning several lines, a pixel of interest, and a plurality of pixel value groups two-dimensionally adjacent to the pixel of interest. A scanning line memory having a means for simultaneously outputting from an output terminal of the pixel, and a pixel value on which noise is superimposed among the neighboring pixel values, including a pixel value on which noise is superimposed and diffused to pixels other than the pixel of interest by a filter. A replacement candidate pixel value detector that estimates this by comparing the magnitude of adjacent pixel values before and after the pixel of interest, detects the maximum and minimum pixel values excluding the estimated pixel value, and outputs them as a replacement candidate pixel value group; The pixel value of the pixel of interest is greater than or equal to a predetermined value with respect to the maximum pixel value of the candidate pixel value group, or the pixel value of the pixel of interest is greater than or equal to the minimum pixel value of the replacement candidate pixel value group. a noise detector that determines whether or not a pixel of interest contains noise by comparing whether the pixel is greater than or equal to a value and outputs the determination result as a control signal;
It is characterized by comprising at least a pixel value selector that selects and outputs any one pixel value from the pixel value of the pixel of interest and the maximum and minimum pixel values among the replacement candidate pixel value group based on a control signal. shall be.

[0007]

[Operation] When removing an image defect, it is necessary to determine which pixel in the image signal contains the image defect, but if the criteria for determining the image defect to be removed is expanded unnecessarily, the image Since significant image information such as detail components in the signal may be lost, it is necessary to selectively remove only image defects that are easily noticeable when the image signal is displayed on a monitor device. . An example of an image defect with such characteristics is a white (black) flaw that has a pixel value that is noticeably brighter (darker) than the surrounding pixels. It is characterized by its ability to selectively remove scratches.

[0008] Based on the above-mentioned properties, a white flaw or a black flaw can be determined based on whether the pixel values of pixels surrounding the flaw are significantly brighter or darker than the pixel values. However, as an example, in order to generate a luminance signal from the output of a solid-state image sensor that has a color filter array based on a complete color-difference line sequential single-chip colorization method, it is generally necessary to use a digital filter with a characteristic of (1+Z-1). In order to generate a color difference signal, (1
-Z-1) Since it is generated by applying a digital filter having the characteristic, the flaw component is also diffused in the scanning line direction. Since the range of pixels diffused by the filter is only one pixel adjacent to the scanning line direction, it is necessary to make a determination based on a group of neighboring pixels excluding adjacent pixels that include scratches among the pixels before and after the pixel of interest. . In order to remove adjacent pixels containing scratches, when the pixel of interest has a white (black) scratch, this can be achieved by removing one pixel with a large (small) pixel value from among the preceding and succeeding pixels. In other words, a pixel with a small (large) pixel value among the preceding and following pixels and a group of other nearby pixels are compared with the pixel of interest.

Next, if a certain pixel is determined to be a flaw, the flaw can be erased by replacing the pixel value with surrounding pixel values. However, if there is a correlation between surrounding pixels, it is necessary to erase the scratches while maintaining the correlation. Regarding the correlation, it is necessary to consider not only the horizontal direction but also the vertical or oblique direction with respect to the scanning line. To achieve this, if the pixel of interest is determined to be a white (black) flaw, the pixel value of the pixel of interest is set in a pixel arrangement group that excludes pixel values that include the flaw from a two-dimensional neighboring pixel value group. This is achieved by replacing the pixel with the one with the largest (smallest) pixel value.

0010

DESCRIPTION OF THE PREFERRED EMBODIMENTS An embodiment of the present invention will be described below with reference to the drawings. (FIG. 1) is a schematic block diagram of an image noise removal device according to the present invention. In FIG. 1, the input terminal 1 receives, for example, a luminance signal or a color difference signal by applying a digital filter to the output of a solid-state image sensor having a color filter array based on a complete color difference line sequential single-chip coloring method. input.

In (FIG. 1), the scanning line memory takes in an image signal a input from an input terminal 1, and stores the pixel value b of the pixel of interest and N pixels in the scanning line direction around the pixel of interest, orthogonal to the scanning line. A neighboring pixel value group c consisting of pixel values of N×M−1 pixels is output, which is obtained by excluding the pixel of interest from a two-dimensionally neighboring pixel group for M lines in the direction of .

FIG. 2 shows a detailed configuration example of a scanning line memory. (FIG. 2) shows an example of the configuration when M=N=3 in the explanation of the operation of the scanning line memory, and 2
It consists of 1 line delay device and 6 pixel delay devices. The pixel delay device outputs a signal obtained by delaying the input signal by one pixel scanning time, and the line delay device outputs a signal obtained by delaying the input signal by one horizontal scanning time. In addition, (Fig. 6) shows an example of the arrangement of a pixel of interest and a group of neighboring pixels on a scanning line. The pixels with numbers 1 to 8 constitute the group of neighboring pixels, and the pixel of interest is in the center of the group of neighboring pixels. Located in Note that if the pixel of interest contains a flaw, it means that the neighboring pixel value c4 or c5 also contains a flaw.

Image signal a inputted from input terminal 2 (FIG. 2) is connected to pixel delay device 1, and the output of pixel delay device 1 is connected to pixel delay device 2. If signals are extracted from three locations: the input end of pixel delay device 1, the output end of pixel delay device 1, and the output end of pixel delay device 2, the signals at the positions corresponding to neighboring pixels 8, 7, and 6 in (Fig. 6) are obtained. The pixel value is obtained. or,(
The image signal inputted from the input terminal 2 of FIG. ), the pixel values at positions corresponding to the neighboring pixel 5, the pixel of interest, and the neighboring pixel 4 are obtained. Furthermore, if the line delay device 2 is connected to the output end of the line delay device 1 in (FIG. 2), and the pixel delay devices 5 and 6 are connected in series to the output end of the line delay device 2, similarly (FIG. 6 ) are obtained at positions corresponding to neighboring pixels 3, 2, and 1. With the scanning line memory configured as described above, the pixel value b of the pixel of interest and the pixel value group c of the neighboring pixel group can be simultaneously output from a plurality of output terminals.

In FIG. 1, the replacement candidate pixel value detector selects the smaller pixel value among c4 and c5 and the remaining neighboring pixel values from the pixel value group c of the neighboring pixel group simultaneously output from the scanning line memory. The maximum pixel value from the group is Ymax,
Further, the minimum pixel value from among c4 and c5, which have a larger pixel value, and the remaining neighboring pixel value group is output as Ymin. These Ymax and Ymin become a replacement candidate pixel value group.

FIG. 3 shows a detailed configuration example of a replacement candidate pixel value detector. (Fig. 3) shows an example of the configuration of a replacement candidate pixel value detector when the neighboring pixel group is composed of eight pixels, in accordance with the explanation of the scanning line memory, and two input terminal groups are used. It consists of a maximum value circuit 1 and a minimum value circuit 1, and a maximum value circuit 2 and a maximum value circuit 2 each having a group of seven input terminals. The minimum value circuit 1 outputs the smaller pixel value among c4 and c5 located one pixel before and after the pixel of interest from among the input neighboring pixel value group c, and the maximum value circuit 2 outputs the one with the smaller pixel value. Groups c1, c2, c3, c6, c
7. The maximum pixel value from c8 and the output of minimum value circuit 1 (
Ymax) is extracted and output. The maximum value circuit 1 outputs the one with the largest pixel value among c4 and c5 located one pixel before and after the pixel of interest from among the input neighboring pixel value group c, and the minimum value circuit 2 outputs the one with the largest pixel value. Groups c1, c2, c
3, c6, c7, c8, and the minimum pixel value (Ymin) is extracted from the output of the maximum value circuit 1 and output.

In FIG. 1, the noise detector determines whether the pixel of interest contains noise based on the pixel value b of the pixel of interest output from the scanning line memory and Ymax and Ymin output from the replacement candidate pixel value detector. It determines whether or not it is true, and outputs the determination result as a control signal. The control signal is Yma
It is composed of SELmax, which is a signal for selecting x, and SELmin, which is a signal for selecting Ymin.

FIG. 4 shows a detailed configuration example of the noise detector, which is composed of two subtracters, two comparators, and two value setters. The subtracter 1 subtracts Ymax from the pixel value of the pixel of interest and generates the difference (hereinafter referred to as DIFmax). Comparator 1 compares DIFmax with a certain value (hereinafter referred to as THmax) output from certain value generator 1,
Condition 1 is Ymax if DIFmax>THmax
A signal SELmax for selecting x is made active, and if condition 1 is not satisfied, SELmax is made inactive and output. Further, the subtracter 2 subtracts the pixel value of the pixel of interest from Ymin, and generates the difference (hereinafter referred to as DIFmin). Comparator 2 compares DIFmin with a certain value (hereinafter referred to as THmin) output from a certain value generator 2,
As condition 2, if DIFmin>THmin, Ymi
A signal SELmin for selecting n is made active, and if condition 2 is not satisfied, SELmin is made inactive and output.

Generally, the amplitude of the high-frequency components (details) of the image signal output from the photoelectric conversion element becomes small due to the spatial low-pass filter effect caused by the aperture ratio of the lens system and the photoelectric conversion element, so that the pixel of interest If the pixel does not include pulse noise, it can be said that the difference between the pixel of interest and the neighboring pixel group will also be small. Therefore, if THmax and THmin are set too small relative to the maximum pixel value, the detail components of the image signal may be lost, and if set too large, the scratches themselves cannot be detected, so set them to appropriate values. There is a need to.

In (FIG. 1), the pixel value selector inputs the pixel of interest output from the scanning line memory, Ymax and Ymin output from the maximum and minimum value detector, and SELmax and SELmin output from the noise detector. Then, based on SELmax and SELmin, the pixel of interest, Yma
The pixel value of any one of x and Ymin is output to the output terminal 1.

FIG. 5 shows a detailed configuration example of the pixel value selector, which is composed of one multiplexer. The multiplexer inputs the pixel value of the pixel of interest, Ymax, and Ymin, and if both SELmax and SELmin are inactive, outputs the pixel value of the pixel of interest to output terminal 2, and if SELmax is active and SELmin is inactive. If so, output Ymax to output terminal 2 and SE
If Lmax is inactive and SELmin is active, Ymin is output to output terminal 2. Here, as can be seen from the explanation of the noise detector, SELmax
From the conditions for outputting and SELmin, SELmax
It is impossible for both SELmin and SELmin to be active.

Next, the operation of the present invention will be explained using (FIGS. 7(a), (b), and (c)). (Figure 7(a)(b)(c)
) is a part of the image signal of one field, and represents the pixel value in the vertical direction, with the scanning line in the horizontal direction and the depth direction, and the axis of the vertical component in the scanning line. (FIG. 7A) shows, as an example, a signal output from a solid-state image sensor having a color filter array based on a complete color difference line sequential single-chip coloring method. Here, to simplify the explanation, it is assumed that a black and white image is captured. (Fig. 7(b)) is the signal of (Fig. 7(a)).
1+Z-1), and this signal is input to input terminal 1 of FIG. 1. (Figure 7(c)) shows that when the above signal is input, (
This is a signal output from output terminal 1 of FIG. 1).

[0022] In (FIG. 7(a)), S01, S11
, S21 expresses sharp line segments existing in the vertical direction of the scanning line, S14 expresses white scratches, S16 expresses black scratches, and S18 expresses fine details included in the original image. There is. Here S01, S11, S2
1. All pixel values of S14 are 200, and pixel values of S16 are 0.
, the pixel value of S18 is 140, and the pixel values of all remaining pixels are 100, in (FIG. 7(b)),
P01, P02, P11, by digital filter
The pixel values of P12, P21, P22, P14, and P15 are all 150, and the pixel values of P16 and P17 are 50, P18, and P15.
The pixel value of 19 is 70, and the pixel value of all remaining pixels is 100.
becomes.

[0023] Also, THmax and THmin are 30
Suppose it was. Now, if the pixel of interest is P11, Ymax output from the maximum/minimum value detector is 150,
Ymin becomes 100, and DIF
max is 0, DIFmin is -50, conditions 1 and 2
are not established, both SELmax and SELmin become inactive, and the pixel of interest P in the pixel value selector
11 pixel values are output. The pixel of interest is P01, P02,
Similarly, in the case of P12, P21, P22, the pixel value of the pixel of interest is output, and P01, P02, P11, P12
, P21, and P22, vertical line segments are retained. Note that the same argument holds true for line segments in the diagonal and horizontal directions with respect to the scanning line.

[0024] Furthermore, if the pixel of interest is P14 or P15, both Ymax and Ymin are 100,
DIFmax is 50, DIFmin is -50, only condition 1 is satisfied, SELmax is active and SEL
Since min becomes inactive and the pixel value selector outputs 100 as the pixel value, P14 and P15 are replaced so as to be flat with respect to the pixel values of the surrounding pixels. When the pixel of interest is P16 or P17, only condition 2 is satisfied, contrary to the operation when the pixel is P14 or P15, SELmax becomes inactive and SELmin becomes active, and the pixel value selector selects the pixel value as the pixel value. 100
, P16 and P17 are also replaced so as to be flat with respect to the pixel values of the surrounding pixels.

[0025] When the pixel of interest is P18, DIFmax is 20, DIFmin is -20,
Conditions 1 and 2 are not satisfied, the pixel value selector outputs the pixel value of P18 as it is, and the detail components included in the original image signal are retained.

Finally, an output as shown in FIG. 7(c) is obtained. Furthermore, in the present invention, processing is performed focusing only on the two-dimensional correlation of images and is not concerned with the time axis, so that it is not affected by the movement of the subject.

[0027]

Effects of the Invention As is clear from the above description, according to the present invention, an image can be diffused by a filter without losing significant image information, regardless of the movement of the subject, which is difficult to do with conventional techniques. It becomes possible to remove defects well, and the practical effect is great.

[Brief explanation of drawings]

FIG. 1 is a schematic block diagram of an embodiment of the invention.

FIG. 2 is a configuration diagram of a scanning line memory according to an embodiment of the present invention.

FIG. 3 is a configuration diagram of a replacement candidate pixel value detector according to an embodiment of the present invention.

FIG. 4 is a configuration diagram of a noise detector according to an embodiment of the present invention.

FIG. 5 is a configuration diagram of a pixel value selector according to an embodiment of the present invention.

FIG. 6 is an explanatory diagram of an arrangement example of a pixel of interest and a group of neighboring pixels according to an embodiment of the present invention.

FIG. 7 is an explanatory diagram of the operation of the embodiment of the present invention.

[Explanation of symbols]

1 Input terminal 1 2 Scanning line memory 3 Replacement candidate pixel value detector 4 Noise detector 5 Pixel value selector 6 Output terminal 1 10 Input terminal 2 11 Pixel delay device 1 12 Pixel delay device 2 13 Pixel delay device 3 14 Pixel delay device 4 15 Pixel delay device 5 16 Pixel delay device 6 17 Line delay device 1 18 Line delay device 2 19 Attention pixel value output terminal 20 Neighboring pixel value group output terminal 21 Neighboring pixel value group input terminal 22 Minimum value circuit 1 23 Maximum value circuit 1 24 Maximum value circuit 2 25 Minimum value circuit 2 26 Ymax output terminal 27 Ymin output terminal 30 Interested pixel value input terminal 1 31 Ymax input terminal 1 32 Ymin input terminal 1 33 Subtractor 1 34 Comparator 1 35 Some value setter 1 36 Subtractor 2 37 Comparator 2 38 Some value setter 2 39 SELmax output terminal 40 SELmin output terminal 41 Attention pixel value input terminal 2 42 Ymax input terminal 2 43 Ymin input terminal 2 44 SELmax input terminal 45 SELmin input terminal 46 Output terminal 2 47 Multiplexer 201 Scanning line 202 Pixel of interest 203 Neighboring pixel group a Input signal b Noticed pixel value c (c1 to c8) Neighboring pixel value group d Ymax e Ymin f　SELmax g SELmin h Output signal i DIFmax j　THmax k　DIFmin l　THmin

Claims (2)

[Claims] 1. A solid-state imaging device that is superimposed on an image signal such as a luminance signal or a color-difference signal obtained by applying a filter to a signal output from a solid-state imaging device having a color filter array based on a single-chip color difference coloring method. This is an image noise removal device that removes pulse-like noise generated in a pixel of interest and a group of neighboring pixels that are two-dimensionally adjacent to the pixel of interest from an image signal composed of a plurality of scanning lines. A pixel value group extraction means to be output, and a replacement candidate pixel value group in which pixel values including noise that are diffused to pixels other than the pixel of interest by a filter are estimated from the neighboring pixel value group, and the pixel values are removed from the neighboring pixel value group. noise determination means for determining whether or not the pixel of interest contains noise by detecting whether the pixel value of the pixel of interest differs by more than a predetermined value; An image noise removal device comprising: a pixel value selection means for selecting and outputting any one of pixel values that do not include noise among the pixel values of the pixel of interest and the pixel values of the replacement candidate pixel value group. 2. A solid-state imaging device that is superimposed on an image signal such as a luminance signal or a color-difference signal obtained by applying a filter to a signal output from a solid-state imaging device having a color filter array based on a single-chip color difference coloring method. An image noise removal device that removes pulse-like noise generated in an image signal comprising a plurality of scanning lines, and a pixel of interest and a pixel of interest. A scanning line memory having means for simultaneously outputting a group of pixel values that are two-dimensionally close to each other from multiple output terminals, and a means for simultaneously outputting a group of pixel values that are two-dimensionally close to the periphery of a It is estimated that noise is superimposed among the group of neighboring pixel values including the pixel of interest by comparing the magnitude of the pixel values adjacent to the front and back of the pixel of interest, and the maximum and minimum pixel values excluding the estimated pixel values are detected and replaced as candidate pixels. A replacement candidate pixel value detector that outputs a value group, and a detector that detects whether the pixel value of the pixel of interest is greater than a predetermined value with respect to the maximum pixel value among the replacement candidate pixel value group, or whether the pixel value of the target pixel is greater than a predetermined value or the minimum value among the replacement candidate pixel value group. Noise detection that determines whether or not the pixel of interest contains noise by comparing whether the pixel value of the pixel of interest is smaller than a predetermined value with respect to the pixel value of , and outputs the determination result as a control signal. and a pixel value selector that selects and outputs one pixel value from three pixel values, that is, the pixel value of the pixel of interest and the maximum and minimum pixel values among the replacement candidate pixel value group, according to a control signal. An image noise removal device characterized by: