JP2005341020A - Pixel defect detector and pixel defect correction device - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To realize the correction of pixel defects at two parts in filters with the same color by suppressing the degradation of image quality to a minimum while reducing a circuit scale. <P>SOLUTION: A first maximum value propriety determination/comparison reference value candidate selector 105 makes either of a maximum value Pm and a second value Ps, which is determined to be more proper than the other with the determination on the basis of the comparison between the maximum value Pm and the second value Ps, as a first comparison reference value candidate Pth1. A second maximum value propriety determination/comparison reference value candidate selector 109 makes a proper maximum value as a second comparison reference value candidate Pth2 with the determination on the basis of the comparison between a specific direction maximum value and a pixel value of a different color pixel by a specific direction maximum value detector 108. A comparison reference value selector 110 makes either of the first/second comparison reference value candidates, which is more proper than the other as the final comparison reference value Pth. A target pixel value propriety determining part 106 determines a target pixel value P0 on the basis of the final comparison reference value. A pixel value replacement control unit 107 replaces the target pixel value P0 with the comparison reference value Pth when a correction control signal Sc is effective. <P>COPYRIGHT: (C)2006,JPO&NCIPI

Description

  The present invention relates to a pixel defect detection device that detects pixel defects (particularly white defects) in an imaging device, a video device, and the like, and a pixel defect correction device that corrects defective pixels.

  As a conventional pixel defect correction apparatus, an apparatus using an adaptive filter is known. It targets a plurality of peripheral pixels of the same color as the target pixel, and uses the maximum value among the pixel values of the peripheral pixels. The pixel value of the target pixel is compared with the maximum value, and if the comparison result (difference) is equal to or greater than the threshold value, the target pixel is determined to be a defective pixel, the pixel value of the target pixel is replaced with the peripheral maximum value, and correction is performed. To do.

  A conventional pixel defect correction apparatus will be described with reference to FIG.

  The input video signal becomes a signal delayed by 1 line, 2 lines, or 3 lines in the line memory circuit 301, and is input to the image data holding circuit 302 including a combination of delay elements that are delayed in units of pixels, and held in the image data holding circuit 302. The resulting signals are arranged in the filter configuration 400 of FIG. In the case of the RGB Bayer arrangement, the pixel arrangement is similar to the filter configuration 401, and this pixel arrangement changes in units of one pixel. When P32 is the target pixel, peripheral pixels are set and processed, such as the G processing pixel 402, the R processing pixel 403, and the B processing pixel 404. Hereinafter, a case where the target pixel is P32 will be described.

  First, the maximum value detection unit 303 detects the maximum value from the five surrounding pixels of the peripheral pixels P12, P52, P14, P34, and P54, and sends the data to the comparison determination processing unit 304. The comparison determination processing unit 304 uses the maximum value Pm detected by the maximum value detection unit 303 and the pixel value P0 of the target pixel P32 in order to determine whether or not the target pixel is a defective pixel, and (P0−Pm)> Comparison of Th0 is performed. If the comparison result is larger than the target pixel determination threshold Th0, that is, if the target pixel value P0 is excessively larger than the maximum value Pm, this means white spots or the like. Therefore, the target pixel P32 is determined as a defective pixel, and a correction control signal Sc is sent to the pixel value replacement control unit 305. When the pixel value replacement control unit 305 receives the correction control signal Sc, the pixel value replacement control unit 305 performs a process of replacing the pixel value P0 of the target pixel value P32 with the maximum value Pm.

  Conversely, when the comparison result of the comparison determination processing unit 304 is equal to or less than the target pixel determination threshold Th0, it is determined that the target pixel P32 is not a defective pixel. In this case, pixel replacement does not occur, and the pixel value P0 of the target pixel P32 is output as it is.

  FIG. 9 is a flowchart showing the operation of the conventional pixel defect correction apparatus. In the conventional configuration, the peripheral maximum value Pm is detected in step S31. Next, in step S32, it is determined whether or not the absolute value D4 of the difference between the target pixel value P0 and the peripheral maximum value Pm exceeds the target pixel determination threshold Th0 (D4 = P0−Pm> Th0?). When the target pixel value P0 is excessively large with respect to the peripheral maximum value Pm, the target pixel is a defective pixel. Therefore, the process proceeds to step S33, and correction is performed to replace the pixel value P0 of the target pixel that is a defective pixel with the maximum value Pm. I do. On the other hand, when the target pixel value P0 does not change much with respect to the maximum value Pm, the target pixel is a normal pixel. Therefore, the process proceeds to step S34, and correction of pixel value replacement is not performed. Output as is.

In the case of this configuration, since the pixel value is replaced with the closest pixel value of the surrounding pixels, the frequency characteristics are hardly deteriorated and the image quality deterioration can be minimized.
JP 2001-86517 A (Page 4, FIG. 4)

  However, when there are two pixel defects in a peripheral pixel of the same color as the target pixel, the peripheral maximum value at that time is the pixel value of the defective pixel, and thus is inappropriate as a comparison criterion. That is, assuming that the target pixel is a defective pixel, the target pixel value approaches the peripheral maximum value. The difference (= target pixel value−maximum value) becomes small, and the comparison result in the comparison determination processing unit 304 becomes equal to or smaller than the target pixel determination threshold Th0. As a result, even if the target pixel is a defective pixel, it cannot be determined as a pixel defect and cannot be corrected.

  By the way, in order to enable correction of pixel defects at two or more locations, instead of using the maximum value Pm, the second largest value of peripheral pixel values is set as the second value Ps, and the comparison processing is performed using the second value Ps as a comparison reference. There is a method to do. According to this, even if there are two defective pixels, the maximum value Pm is not used as a comparison reference, and the second value Ps is used as a comparison reference. Therefore, it is possible to correct a target pixel that is a defective pixel.

  However, since the second value Ps is used as a comparison reference, the frequency characteristics of the video signal and the image quality are deteriorated as compared with the above processing.

  In order to prevent image quality degradation, there is a method of taking a wide filter area and changing a correction method based on pixel information of peripheral pixels (see, for example, Patent Document 1), but there is a problem that the circuit scale becomes large.

  The pixel size of image sensors is increasing, and correction of pixel defects has become a problem. On the other hand, downsizing of the system is also an issue.

  When it is configured with 4 lines or less, accuracy is deteriorated when surrounding second values are adopted. This is because in the vertical direction, the same color pixel data is only one pixel data with respect to the target pixel. In the case of a high-frequency signal having a large difference in the vertical direction, it is not possible to determine whether or not the data is defective data, and there is a possibility of erroneous correction. Particularly noticeable is the high-frequency signal of the achromatic vertical component.

  Further, regarding the pixel defect (white defect) of the sensor, the value of the defect data becomes larger as the exposure time becomes longer, so that it becomes noticeable as the image quality. The same applies when the signal is amplified by the amplifier operation.

  The present invention has been created in view of such circumstances, and is capable of correcting pixel defects at two locations in a filter of the same color while minimizing image quality degradation while reducing the circuit scale. An object of the present invention is to provide a correction device.

  The present invention takes the following means in order to solve the above problems.

A first pixel defect detection device according to the present invention includes:
Maximum value detecting means for detecting the maximum value of the pixel values of a plurality of pixels around the pixel of interest;
Second value detection means for detecting second values of pixel values of a plurality of pixels around the target pixel;
Based on the comparison between the maximum value and the second value, whether or not the maximum value is adopted is determined. When the maximum value is appropriate, the maximum value is adopted as a comparison reference value, and the maximum value is inappropriate. When the maximum value appropriateness determination / comparison reference value selection means adopting the second value as the comparison reference value,
And a target pixel value appropriate determining means for determining the appropriateness of the target pixel value based on the comparison reference value adopted by the maximum value appropriate determining / comparison reference value selecting means.

  As compensation when there are two or more defective pixels in the peripheral pixels, it is determined whether or not the maximum value is achieved and is appropriate as a comparison reference value for determining the appropriateness of the target pixel value. That is, the maximum value appropriateness determination / comparison reference value selection means compares the maximum value from the maximum value detection means with the second value from the second value detection means to determine whether the maximum value is appropriate as the comparison reference value. judge. The maximum value appropriateness determination / comparison reference value selection means adopts the maximum value as the comparison reference value when the maximum value is appropriate as the comparison reference value as a result of the above determination. When appropriate, the second value is adopted as the comparison reference value. Then, the adopted comparison reference value is given to the target pixel value appropriate determination means. The target pixel value appropriateness determination means determines the appropriateness of the target pixel value based on the adopted comparison reference value (maximum value or second value).

  A preferable aspect of the above configuration is that the maximum value appropriateness determination / comparison reference value selection unit is configured as follows. That is, the difference between the maximum value and the second value is compared with a first difference determination threshold value, and when the difference is less than the first difference determination threshold value, the maximum value is determined to be appropriate as the comparison reference value. Adopting the maximum value, when the difference is equal to or greater than the first difference determination threshold, it is determined that the maximum value is inappropriate, and the second value is used as the comparison reference value, so that the maximum value appropriateness determination / comparison is performed. The reference value selection means is configured.

  When the difference between the maximum value and the second value is excessively large, the maximum value may be abnormal. In this case, the second value is adopted as the comparison reference value for the target pixel value, and otherwise, the maximum value is adopted as the comparison reference value.

  In the above, the difference between the maximum value and the second value is broad, and may be the result of subtracting the second value from the maximum value, or may be the ratio of the maximum value to the second value.

  A preferable aspect of the above configuration is that the maximum value appropriateness determination / comparison reference value selection unit is configured to be able to vary the first difference determination threshold value.

  A preferred example is that the first difference determination threshold is made variable according to the exposure time of the image sensor.

  Another preferred example is that the first difference determination threshold is made variable in accordance with the video signal amplification factor.

  By adjusting the first difference determination threshold value, the criterion of which of the maximum value and the second value is adopted as the comparison reference value is changed. That is, when the exposure time is shortened in a bright place, the first difference determination threshold value is shifted to a larger value, and the possibility that the maximum value is appropriate is expanded. On the contrary, when the exposure time becomes long in a dark place, the first difference determination threshold value is shifted to a smaller value to widen the possibility that the maximum value is inappropriate and widen the possibility of adopting the second value.

  Further, when the video signal amplification factor is high, the first difference determination threshold value is shifted to a larger value to widen the possibility that the maximum value is appropriate. On the other hand, when the video signal amplification factor is low, the first difference determination threshold value is shifted to a smaller value to widen the possibility that the maximum value is inappropriate and widen the possibility of adopting the second value.

  A preferred aspect of the above configuration is that the pixel value appropriateness determining means is configured as follows. That is, the difference between the target pixel value and the comparison reference value adopted by the maximum value appropriate determination / comparison reference value selection unit is compared with a target pixel determination threshold, and the difference is equal to or less than the target pixel determination threshold. The pixel-of-interest value determination unit is configured to determine that the pixel-of-interest value is appropriate, and to determine that the pixel-of-interest value is inappropriate when the difference exceeds the pixel-of-interest determination threshold. It is.

  The following corresponds to the case where the filter region for pixel value determination is narrow.

The second pixel defect detection apparatus according to the present invention is:
Maximum value detecting means for detecting the maximum value of the pixel values of a plurality of pixels around the pixel of interest;
Second value detection means for detecting second values of pixel values of a plurality of pixels around the target pixel;
Based on a comparison between the maximum value and the second value, it is determined whether or not the maximum value is appropriate. When the maximum value is appropriate, the maximum value is used as a first comparison reference value candidate, and the maximum value is determined. Is not appropriate, a first maximum value appropriateness determination / comparison reference value candidate selection means that adopts the second value as the first comparison reference value candidate;
Specific direction maximum value detecting means for determining whether or not the pixel value of the same color pixel closest to the target pixel in the specific direction is the maximum value;
When the pixel value of the closest same color pixel in the specific direction of the target pixel is determined to be the maximum value by the specific direction maximum value detection means, the maximum value of the closest different color pixel in the specific direction of the target pixel is determined. It is determined whether or not the maximum value is appropriate as a second comparison reference value candidate in comparison with a pixel value, and when it is determined to be appropriate, the second maximum is adopted as the second comparison reference value candidate. Value appropriateness judgment / comparison reference value candidate selection means,
When the determination result of the second maximum value appropriateness determination / comparison reference value candidate selecting means makes the maximum value inappropriate as the second comparison reference value candidate, the first maximum value appropriateness determination / comparison reference When the first comparison reference value candidate adopted by the value candidate selection means is adopted as a final comparison reference value and the determination result makes the maximum value appropriate as a second comparison reference value candidate, the first comparison reference value candidate is adopted. A comparison reference value selection means that adopts the second comparison reference value candidate adopted by the maximum value appropriateness determination / comparison reference value candidate selection means of 2 as a final comparison reference value;
The apparatus includes a target pixel value appropriate determination unit that determines the appropriateness of the target pixel value based on the final comparison reference value adopted by the comparison reference value selection unit.

  When the filter region for pixel value determination is narrow, there may be only one peripheral pixel to be compared in the specific direction of the target pixel. For example, in the case of a filter area for four lines in the vertical direction, if the specific direction is the vertical direction, there is only one peripheral pixel to be compared. Therefore, when the pixel value of the closest same color pixel in the specific direction is the maximum value, it is difficult to determine whether or not the maximum value is appropriate as the comparison reference value. This is not limited to the vertical direction, but also applies to the horizontal direction.

  The second maximum value appropriateness determination / comparison reference value candidate selection means when the first maximum value appropriateness determination / comparison reference value candidate selection means determines that the maximum value is inappropriate and adopts the second value. To recheck the maximum value. Although the maximum value is the pixel value of the same color pixel, if it is the pixel value of the nearest same color pixel in the specific direction, the maximum value is compared with the pixel value of the nearest different color pixel in the specific direction. Thus, it is rechecked whether or not the determination of whether or not the maximum value is appropriate by the first maximum value appropriateness determination / comparison reference value candidate selection unit is appropriate. If the maximum value is still inappropriate, the second value is set as the final comparison reference value. If the maximum value is appropriate as a result, the maximum value is used as the final comparison reference value. In this way, even when the filter region for pixel value determination is narrow, the accuracy of pixel defect determination can be increased.

  A preferable aspect of the above configuration is that the second maximum value appropriateness determination / comparison reference value candidate selection unit is configured as follows. That is, the absolute value of the difference between the pixel value of the nearest same color pixel in the specific direction of the target pixel and the pixel value of the nearest different color pixel in the specific direction is compared with a second difference determination threshold, and the absolute value of the difference When the value is less than or equal to the second difference determination threshold, the maximum value of the pixel values of the nearest same color pixel is determined to be appropriate, and otherwise, the second maximum value is determined to be determined to be inappropriate The determination / comparison reference value candidate selection means is configured.

  In the above aspect, the specific direction maximum value detecting unit is configured to determine whether or not the pixel value of the closest same color pixel in the vertical direction of the target pixel is the maximum value. is there. here. The specific direction is assumed to be the vertical direction.

  The second pixel defect detection device is preferably configured as follows, similarly to the first pixel defect detection device described above.

  For the first maximum value appropriateness determination / comparison reference value candidate selection means, the difference between the maximum value and the second value is compared with a first difference determination threshold value, and the difference is less than the first difference determination threshold value. When the maximum value is appropriate, the maximum value is adopted as the first comparison reference value candidate, and when the difference is greater than or equal to the first difference determination threshold, the maximum value is inappropriate. It is preferable that the second value is adopted as the first comparison reference value candidate.

  The first maximum value appropriateness determination / comparison reference value candidate selection unit is preferably configured such that the first difference determination threshold is variable.

  Further, it is preferable that the first maximum value appropriateness determination / comparison reference value candidate selection unit is configured to vary the first difference determination threshold according to the exposure time of the image sensor.

  Moreover, it is preferable that the first maximum value appropriateness determination / comparison reference value candidate selection unit is configured to vary the first difference determination threshold according to a video signal amplification factor.

  Further, the target pixel value appropriateness determining unit compares a difference from the final comparison reference value selected by the comparison reference value selecting unit with a target pixel determination threshold value, and the difference is equal to or smaller than the target pixel determination threshold value. In this case, it is preferable that the target pixel value is determined to be appropriate, and that the target pixel value is determined to be inappropriate when the difference exceeds the target pixel determination threshold.

  The above is for the pixel defect detection device, but it can be developed as a pixel defect correction device as follows.

A first pixel defect correction apparatus according to the present invention includes:
A pixel defect correction apparatus of the first configuration;
When the pixel-of-interest value determination unit determines that the pixel-of-interest value is appropriate, pixel value replacement of the pixel-of-interest value is not performed, and when the pixel-of-interest value is determined inappropriate, the pixel-of-interest value is A pixel value replacement control unit that replaces the comparison reference value is provided.

  The pixel value of the defective pixel when there is a pixel defect can be corrected with an optimal correction pixel value. When the adopted comparison reference value is the maximum value, the maximum value replacement mode is given priority to image quality, and when the adopted comparison reference value is the second value, the second value replacement mode is given priority to defect correction.

The second pixel defect correcting device according to the present invention is
A pixel defect detection device of the second configuration;
When the pixel-of-interest value determination unit determines that the pixel-of-interest value is appropriate, pixel value replacement of the pixel-of-interest value is not performed, and when the pixel-of-interest value is determined inappropriate, the pixel-of-interest value is A pixel value replacement control unit that replaces the final comparison reference value is provided.

  When the filter region for pixel value determination is narrow, the pixel value of the defective pixel when there is a pixel defect can be corrected with an optimal correction pixel value.

  According to the present invention, the maximum value replacement mode and the second value replacement mode are switched using a plurality of types of threshold values, and can be realized with a small circuit configuration. This is advantageous for incorporation into a low-cost system. In addition, switching according to the exposure time of the sensor or the amplification factor of the sensor is possible, and an image with optimum image quality can be obtained.

  Embodiments of a pixel defect correction apparatus according to the present invention will be described below in detail with reference to the drawings.

(Embodiment 1)
FIG. 1 is a block diagram showing a configuration of a pixel defect correction apparatus according to Embodiment 1 of the present invention. FIG. 6 shows the positional relationship that the pixel defect correction circuit corresponding to the pixel defect correction apparatus occupies in the imaging apparatus. In FIG. 6, reference numeral 601 denotes an image sensor, 602 denotes a RAW data processing circuit, 603 denotes a line memory, 604 denotes a pixel defect correction circuit, 605 denotes a luminance signal processing circuit, 606 denotes a color difference signal processing circuit, and 607 denotes an output switching circuit.

  In FIG. 1, 101 is a line memory circuit, 102 is an image data holding circuit composed of a combination of delay elements, 103 is a maximum value detection unit, 104 is a second value detection unit, 105 is a maximum value appropriateness determination / comparison reference value selection unit, Reference numeral 106 denotes a target pixel value appropriateness determination unit, and 107 denotes a pixel value replacement control unit. The configuration excluding the pixel value replacement control unit 107 corresponds to a pixel defect detection device.

  The line memory circuit 101 connects a plurality of line memories in series and temporarily stores an input video signal for delaying one line, two lines, and three lines.

  The image data holding circuit 102 delays the line-delayed video signal from each line memory in units of pixels and holds the image data in units of pixels, and includes a flip-flop or the like.

  The maximum value detection unit 103 detects the maximum value Pm from the pixel values of a plurality of pixels of N rows and M columns held in the image data holding circuit 102.

  The second value detection unit 104 detects a second value Ps that is the next largest pixel value after the maximum value.

  The maximum value appropriateness determination / comparison reference value selection unit 105 has a first difference D1 (= Pm−Ps) between the maximum value Pm detected by the maximum value detection unit 103 and the second value Ps detected by the second value detection unit 104. It is determined whether or not the difference determination threshold value Th1 is less than the threshold value. When the maximum value Pm is appropriate and the difference D1 is less than the first difference determination threshold value Th1, the maximum value Pm is selected and output as the comparison reference value Pth. When Pm is inappropriate and the difference D1 is greater than or equal to the first difference determination threshold Th1, the second value Ps is selected and output as the comparison reference value Pth.

  The target pixel value appropriateness determination unit 106 receives the pixel value P0 of the target pixel held in the image data holding circuit 102 and the comparison reference value Pth from the maximum value appropriateness determination / comparison reference value selection unit 105, and receives the target pixel It is determined whether or not a difference D2 (= P0−Pth) between the value P0 and the comparison reference value Pth is greater than a predetermined threshold value for pixel determination Th0, and correction control indicating whether or not the target pixel value P0 needs to be corrected. The signal Sc is output to the pixel value replacement control unit 107.

  It is determined whether or not the difference D2 between the target pixel value P0 and the comparison reference value Pth (maximum value Pm or second value Ps) exceeds the target pixel determination threshold Th0, and the difference D2 exceeds the comparison reference value Pth. In some cases, it is necessary to perform pixel replacement, so that the correction control signal Sc is made active, and when it does not exceed, the correction control signal Sc is made inactive.

  When the correction control signal Sc from the target pixel value appropriateness determination unit 106 indicates that correction is necessary, the pixel value replacement control unit 107 performs correction by replacing the target pixel value P0 with the comparison reference value Pth. .

  The pixel defect correction apparatus according to the present embodiment is of a type that does not see a correlation in the vertical direction.

  Next, the operation of the pixel defect correction apparatus of the present embodiment configured as described above will be described.

  The input video signal becomes a signal delayed by one line, two lines, or three lines in the line memory circuit 101, and is input to the image data holding circuit 102 that holds the image data while being delayed in units of pixels. The signals delayed by the image data holding circuit 102 are arranged in the filter configuration 200 of FIG.

  In the case of the RGB Bayer array, a pixel array like the filter configuration 201 is obtained. In the case of the Bayer array, it changes in units of one pixel. For example, when P32 is the target pixel, peripheral pixels are set like the G processing pixel 202, the R processing pixel 203, and the B processing pixel 204, and processing is performed. At this time, the comparison pixel of the target pixel is different from the R pixel and the B pixel in the case of the G pixel, and the adjacent pixels P23 and P43 are used in the oblique direction (in the case of the R pixel and the B pixel). , P14, P54). With this configuration, the G pixel has a configuration in which two comparison pixels exist with respect to the target pixel.

  Hereinafter, a case where the target pixel is P32 will be described.

  First, the maximum value detection unit 103 detects the maximum value Pm from the five peripheral pixels of the peripheral pixels P12, P52, P23, P43, and P34 when the pixel is a G pixel. In the case of the R pixel and the B pixel, the maximum value Pm is detected from the five peripheral pixels of the peripheral pixels P12, P52, P14, P34, and P54. At the same time, the second value detection unit 104 detects the second largest value from the five surrounding pixels as the second value Ps.

The data detected by the maximum value detection unit 103 and the second value detection unit 104 is transferred to the maximum value appropriateness determination / comparison reference value selection unit 105,
D1 = Pm−Ps <Th1?
The determination process is performed. Th1 is a first difference determination threshold value for determining the difference D1 between the maximum value Pm and the second value Ps.

  The maximum value appropriateness determination / comparison reference value selection unit 105 determines that the maximum value Pm is not defective data when the determination result is “true”, that is, when the maximum value Pm does not become much larger than the second value Ps. Then, the maximum value Pm is selected as the comparison reference value Pth. This is the maximum value replacement mode, which is the image quality priority correction mode.

  On the other hand, when the determination result is “false”, that is, when the maximum value Pm is excessively larger than the second value Ps, it is determined that the maximum value Pm is defective data, and the comparison reference value Pth is the maximum value Pm. And a more appropriate second value Ps is selected instead. This is a second value replacement mode, and is a defect correction priority correction mode.

  As described above, the maximum value replacement mode and the second value replacement mode are switched by determining the difference D1 between the maximum value Pm and the second value Ps.

  Next, the operation of the pixel defect correction apparatus of the present embodiment configured as described above will be described with reference to the flowchart of FIG.

  In step S1, the maximum value Pm around the pixel of interest is detected, then in step S2, the peripheral second value Ps is detected, and in step S3, the difference D1 between the maximum value Pm and the second value Ps is the first value. It is determined whether it is less than the difference determination threshold Th1 (D1 = (Pm−Ps) <Th1?). This checks whether or not the maximum value Pm is appropriate. If the maximum value Pm is appropriate, the normal maximum value Pm can be used as the comparison reference value Pth, which is a reference value for comparison determination. Therefore, in this case, the process proceeds to step S4. If the maximum value Pm is abnormal, the difference D1 between the maximum value Pm and the second value Ps is equal to or greater than the first difference determination threshold Th1. In this case, using the maximum value Pm as the comparison reference value Pth causes erroneous determination. That is, the process proceeds to step S6 to use the second value Ps as the comparison reference value Pth.

  If the maximum value Pm can be confirmed to be normal, the maximum value Pm is set as the comparison reference value Pth in step S4, and the difference D2 between the target pixel value P0 and the comparison reference value Pth that is the normal maximum value Pm is determined as the target pixel determination. It is determined whether the threshold value Th0 is exceeded.

D2 = P0−Pth = P0−Pm> Th0?
When the difference D2 is larger than the target pixel determination threshold Th0 and the target pixel value P0 is excessively larger than the comparison reference value Pth (normal maximum value Pm), the target pixel is a defective pixel. Then, correction is performed to replace the pixel value P0 of the target pixel, which is a defective pixel, with the maximum value Pm (P0 ← Pm (maximum value) = Pth). This is the “maximum value replacement mode”. The flow from step S3 to step S4 and step S5 is the maximum value replacement mode.

  On the other hand, when the difference D2 is equal to or less than the target pixel determination threshold Th0 and the target pixel value P0 is not much different from the comparison reference value Pth (normal maximum value Pm), the target pixel is a normal pixel. Then, the normal target pixel value P0 is output as the target pixel value P0 without correcting the pixel value replacement.

  On the other hand, when the maximum value Pm is abnormal and cannot be adopted as the comparison reference value Pth in the determination in step S3, that is, when the second value Ps is adopted as the comparison reference value Pth, the process proceeds to step S6. In step S6, the second value Ps is set as the comparison reference value Pth, and it is determined whether or not the difference D2 between the target pixel value P0 and the comparison reference value Pth that is the second value Ps exceeds the target pixel determination threshold Th0. .

D2 = P0-Pth = P0-Ps> Th0?
When the difference D2 is larger than the target pixel determination threshold Th0 and the target pixel value P0 is excessively larger than the comparison reference value Pth (second value Ps), it is determined that the target pixel is still a defective pixel and the process proceeds to step S7. . In step S7, correction is performed to replace the pixel value P0 of the target pixel, which is a defective pixel, with the second value Ps (P0 ← Ps (second value) = Pth). This is the “second value replacement mode”. The flow from step S3 to step S6 and step S7 is the second value replacement mode.

  On the other hand, when the target pixel value P0 does not change much with respect to the comparison reference value Pth (second value Ps), the target pixel is actually a normal pixel. Therefore, the process proceeds to step S8, and the normal target pixel value P0 is the pixel. The value replacement is not corrected.

  As described above, the determination in step S3 is the starting point for selecting either the maximum value replacement mode or the second value replacement mode. For this determination, the maximum value Pm, the second value Ps, and the first difference determination threshold Th1 are used.

  In step S4 and step S6, the pixel-of-interest determination threshold Th0 may be different from each other.

(Embodiment 2)
FIG. 4 is a block diagram showing the configuration of the pixel defect correction apparatus according to Embodiment 2 of the present invention. In FIG. 4, the same reference numerals as those in FIG. The configuration excluding the pixel value replacement control unit 107 corresponds to a pixel defect detection device.

  In FIG. 4, 105 ′ is a first maximum value appropriateness determination / comparison reference value candidate selection unit, 108 is a specific direction maximum value detection unit, 109 is a second maximum value appropriateness determination / comparison reference value candidate selection unit, and 110 is It is a comparison reference value selection unit. The first maximum value appropriateness determination / comparison reference value candidate selection unit 105 ′ is equivalent to the maximum value appropriateness determination / comparison reference value selection unit 105 in the first embodiment. In this embodiment, the specific direction is the vertical direction.

  The specific direction maximum value detection unit 108 determines whether or not the pixel value Pv of the same color comparison pixel (the same color comparison pixel immediately below) in the vertical direction of the target pixel is the peripheral maximum value Pm. Here, a comparison pixel of the same color immediately below the pixel of interest in the vertical direction will be described with reference to FIG. When the pixel of interest is P32, the same color comparison pixel directly below the pixel of interest P32 is P34, and the different color comparison pixel directly below is P33. This is true for any of R, G, and B.

  The second maximum value appropriateness determination / comparison reference value candidate selection unit 109 determines that the immediately-same color comparison pixel value Pv, which is the maximum value Pm detected by the specific direction maximum value detection unit 108, is inappropriate as the final comparison reference value Pth. It is determined whether or not the signal is proper, and an appropriate / inappropriate determination signal Sj is output. When the signal is appropriate, the determination signal Sj is set to “1” and output to the comparison reference value selection unit 110. The comparison pixel value Pv (= maximum value Pm) immediately below is selected and output as the comparison reference value candidate Pth2, and “0” is output as the determination signal Sj when it is inappropriate.

  When the determination signal Sj from the second maximum value appropriateness determination / comparison reference value candidate selection unit 109 is “0”, the comparison reference value selection unit 110 is the first maximum value appropriateness determination / comparison reference value candidate selection unit. When the first comparison reference value candidate Pth1 (maximum value Pm or second value Ps) from 105 ′ is selected and the determination signal Sj is “1”, the second maximum value appropriateness determination / comparison reference value candidate selection unit 109 is selected. The second comparison reference value candidate Pth2 is selected.

  Next, the operation (see the correlation in the vertical direction) of the pixel defect correction apparatus of the present embodiment configured as described above will be described with reference to the flowchart of FIG.

  The flow from Step S11 to Step S15 through Steps S12, S13, and S14 is the same as the flow of Steps S1, S2, S3, S4, and S5 in the case of FIG. To explain again, in step S11, the surrounding maximum value Pm is detected, then in step S12 the surrounding second value Ps is detected, and in step S13, the maximum value Pm is checked to see if it is abnormal. It is determined whether or not the difference D1 between Pm and the second value Ps is less than the first difference determination threshold Th1. Since the maximum value Pm is normal and may be used as the final comparison reference value Pth, the process proceeds to step S14, and the difference D2 between the target pixel value P0 and the normal maximum value Pm is determined as the target pixel determination in order to see whether the target pixel is a defective pixel. It is determined whether the threshold value Th0 is exceeded. Since the target pixel is a defective pixel, the process proceeds to step S15, and correction is performed to replace the target pixel that is a defective pixel with the maximum value Pm (maximum value replacement mode).

  On the other hand, if the target pixel is a normal pixel, the process proceeds to step S20, the pixel value replacement is not corrected, and the target pixel value P0 is adopted as it is.

  If the maximum value Pm is excessively larger than the second value Ps in the determination in step S13, the maximum value Pm may be abnormal, but it may not always be determined that it is abnormal. This is because if the pixel detected as the maximum value Pm in step S11 is the same color pixel immediately below the pixel of interest, it is difficult to determine whether the maximum value Pm is normal or abnormal.

  Therefore, the process proceeds to step S16, and it is determined whether or not the maximum value Pm obtained in step S11 is the same as the pixel value Pv of the same color comparison pixel directly below. If the determination result is that the maximum value Pm is the same color comparison pixel value Pv immediately below, the process proceeds to step S17.

  In step S17, it is determined whether the maximum value Pm in step S13 is determined to be abnormal and there is no doubt. The determination in step S17 corresponds to the processing in the second maximum value appropriateness determination / comparison reference value candidate selection unit 109. In step S17, the absolute value D3 of the difference between the immediately lower same color comparison pixel value Pv (= maximum value Pm) and the immediately lower different color comparison pixel value Pd is taken, and the absolute value D3 of this difference is equal to or smaller than the second difference determination threshold Th2. It is determined whether or not. The affirmative or negative result of this determination corresponds to “1” or “0” of the determination signal Sj output from the second maximum value appropriateness determination / comparison reference value candidate selection unit 109.

  When the directly-same color comparison pixel value Pv, which is considered to be the maximum value Pm, does not have much difference from the immediately-lower adjacent color comparison pixel value Pd, the maximum value Pm is inappropriate. Instead, it may be adopted as the final comparison reference value Pth. This corresponds to the determination signal Sj = 1. In this case, the process returns to step S14.

  In order to reduce the circuit scale, when it is configured with four lines or less, the same color pixel with respect to the target pixel is only one pixel in the vertical direction. Therefore, it cannot be determined whether the high-frequency signal of the vertical component is defective data of two pixels. In this case, there remains a possibility that erroneous correction is performed on the high-frequency signal of the vertical component. Particularly noticeable is an achromatic vertical component high-frequency signal. Therefore, the erroneous correction is eliminated by comparing the pixels with different colors.

  If the determination result is negative in step S17, this corresponds to the determination signal Sj = 0, but the processing at this time is as follows. It is determined whether or not the absolute value D3 of the difference between the immediately lower same color comparison pixel value Pv (= maximum value Pm) and the immediately lower different color comparison pixel value Pd exceeds the second difference determination threshold Th2.

D3 = | Pv−Pd | ≦ Th2?
When the second maximum value appropriateness determination / comparison reference value candidate selection unit 109 performs a comparison between the immediately lower same color comparison pixel value Pv and the immediately lower different color comparison pixel value Pd, if the target pixel is G, the immediately lower different color comparison pixel value Pd. These pixels are not G pixels but R pixels or B pixels. However, in the case of an achromatic color, an approximate expression of G = R = B is established. It is detected whether an achromatic high-frequency signal is present in the vertical direction.

  As a result of the above determination, when the absolute value D3 of the difference exceeds the second difference determination threshold Th2, the immediately-same color comparison pixel value Pv as the maximum value Pm at that time is still the final comparison reference value Pth. It will not be appropriate. The reason is as follows. In step S17, the maximum value Pm is premised on that the difference from the second value Ps is excessively large. The difference between the maximum value Pm and the directly different color comparison pixel value Pd is excessive. This means that the maximum value Pm is excessively far from the two pixel values. Therefore, it can be concluded that the maximum value Pm is not appropriate.

  In this case, the process proceeds to step S18, the final comparison reference value Pth is set to the second value Ps (Pth ← Ps), and the difference D2 between the target pixel value P0 and the final comparison reference value Pth (= second value Ps). And whether or not the difference D2 exceeds the threshold value Th0 for the target pixel determination is determined.

D2 = P0-Pth = P0-Ps> Th0?
If the difference D2 is larger than the target pixel determination threshold Th0 and the target pixel value P0 is excessively larger than the final comparison reference value Pth (= second value Ps), the target pixel is a defective pixel, so step S19 Then, correction is performed to replace the pixel value P0 of the target pixel, which is a defective pixel, with the second value Ps (P0 ← Ps (second value) = Pth: second value replacement mode).

  On the other hand, when the target pixel value P0 does not change much with respect to the final comparison reference value Pth (second value Ps), the target pixel is a normal pixel. Therefore, the process proceeds to step S20, and the normal target pixel value P0 is determined. The pixel value replacement is not corrected.

  The flow from step S13 to step S14 corresponds to the case where the first maximum value appropriateness determination / comparison reference value candidate selection unit 105 ′ outputs the maximum value Pm as the first comparison reference value candidate Pth1 (Pth1 ← Pm). ).

  The flow of steps S13 → S16 → S18 corresponds to the case where the first maximum value appropriateness determination / comparison reference value candidate selection unit 105 ′ outputs the second value Ps as the first comparison reference value candidate Pth1 (Pth1 ← Ps). ).

  In the flow of steps S13 → S16 → S17, the second maximum value appropriateness determination / comparison reference value candidate selection unit 109 outputs the immediately lower same color comparison pixel value Pv (= maximum value Pm) as the second comparison reference value candidate Pth2. (Pth2 ← Pv (= Pm)). The determination result Yes corresponds to the determination signal Sj = 1, and the determination result No corresponds to the determination signal Sj = 0.

  In the above description, the correlation in the specific direction is limited to the vertical direction, but it is not necessarily limited to the vertical direction. If there is only one comparison pixel for the target pixel, it is effective in all directions.

  There is a configuration in which the second value Ps is not used in the case of a specific direction. This can be handled by setting the second difference determination threshold value Th2 to a sufficiently large value. This is because the determination in step S17 is always Yes.

  The specific direction maximum value detection unit 108 may be configured to always adopt the maximum value when the same color comparison pixel immediately below the target pixel has the maximum value. This corresponds to the case where step S17 is omitted in FIG. 5 and the flow of Yes in step S16 is connected to step S14. Malfunctions can also be prevented for chromatic colors.

  The second embodiment is effective when the filter area is 4 lines or less in the vertical direction. If there are 6 lines or more, the first embodiment is suitable. This is because there is no need to see the correlation in the vertical direction because there are two or more same-color comparison pixels for the target pixel in the vertical direction.

  The pixel defect correction apparatus of the present invention is useful as a pixel defect correction apparatus for an input signal from an image sensor such as an image sensor. The pixel defect detection device of the present invention is useful as a pixel defect detection device for an input signal from an image sensor such as an image sensor.

1 is a block diagram showing a configuration of a pixel defect correction apparatus according to Embodiment 1 of the present invention. Filter configuration diagram according to the embodiment of the present invention The flowchart which shows operation | movement of the pixel defect correction apparatus in Embodiment 1 of this invention. The block diagram which shows the structure of the pixel defect correction apparatus in Embodiment 2 of this invention. The flowchart which shows operation | movement of the pixel defect correction apparatus in Embodiment 2 of this invention. 1 is a block diagram showing a schematic configuration of an imaging device on which a pixel defect correction device according to an embodiment of the present invention is mounted. The block diagram which shows the structure of the pixel defect correction apparatus in a prior art Filter configuration diagram in the prior art The flowchart which shows operation | movement of the pixel defect correction apparatus in a prior art.

Explanation of symbols

101 Line memory circuit 102 Image data holding circuit 103 Maximum value detection unit 104 Second value detection unit 105 Maximum value appropriateness determination / comparison reference value selection unit 105 ′ First maximum value appropriateness determination / comparison reference value candidate selection unit 106 Target pixel value Appropriateness determination unit 107 Pixel value replacement control unit 108 Specific direction maximum value detection unit 109 Second maximum value appropriateness determination / comparison reference value candidate selection unit 110 Comparison reference value selection unit Pm Maximum value Ps Second value Pv Same color comparison pixel value immediately below Pd Directly different color comparison pixel value Pth comparison reference value (final comparison reference value)
Pth1 first comparison reference value candidate Pth2 second comparison reference value candidate Sc correction control signal Sj determination signal Th0 threshold for target pixel determination Th1 first difference determination threshold Th2 second difference determination threshold

Claims (16)

Maximum value detecting means for detecting the maximum value of the pixel values of a plurality of pixels around the pixel of interest;
Second value detection means for detecting second values of pixel values of a plurality of pixels around the target pixel;
Based on the comparison between the maximum value and the second value, whether or not the maximum value is adopted is determined. When the maximum value is appropriate, the maximum value is adopted as a comparison reference value, and the maximum value is inappropriate. When the maximum value appropriateness determination / comparison reference value selection means adopting the second value as the comparison reference value,
A pixel defect detection apparatus comprising: a target pixel value appropriate determination unit that determines the appropriateness of the target pixel value based on the comparison reference value employed by the maximum value appropriate determination / comparison reference value selection unit.   The maximum value appropriateness determination / comparison reference value selection means compares the difference between the maximum value and the second value with a first difference determination threshold, and when the difference is less than the first difference determination threshold, the maximum The maximum value is adopted as the comparison reference value assuming that the value is appropriate, and when the difference is equal to or greater than the first difference determination threshold value, the maximum value is inappropriate and the second value is used as the comparison reference value. The pixel defect detection device according to claim 1, which is employed.   The pixel defect detection device according to claim 2, wherein the maximum value appropriateness determination / comparison reference value selection unit is configured such that the first difference determination threshold value is variable.   The pixel defect detection device according to claim 3, wherein the maximum value appropriateness determination / comparison reference value selection unit is configured to vary the first difference determination threshold according to an exposure time of an image sensor.   The pixel defect detection device according to claim 3, wherein the maximum value appropriateness determination / comparison reference value selection unit is configured to vary the first difference determination threshold according to a video signal amplification factor.   The target pixel value appropriateness determination unit compares the difference between the target pixel value and the comparison reference value adopted by the maximum value appropriateness determination / comparison reference value selection unit with a target pixel determination threshold, and the difference is 6. The pixel-of-interest value is determined to be appropriate when the pixel-of-interest determination threshold is less than or equal to the pixel-of-interest determination threshold, and the pixel-of-interest value is determined to be inappropriate when the difference exceeds the pixel-of-interest determination threshold. The pixel defect detection device according to any one of the above. Maximum value detecting means for detecting the maximum value of the pixel values of a plurality of pixels around the pixel of interest;
Second value detection means for detecting second values of pixel values of a plurality of pixels around the target pixel;
Based on the comparison between the maximum value and the second value, it is determined whether or not the maximum value is adopted. When the maximum value is appropriate, the maximum value is adopted as a first comparison reference value candidate. Is not appropriate, a first maximum value appropriateness determination / comparison reference value candidate selection means that adopts the second value as the first comparison reference value candidate;
Specific direction maximum value detection means for determining whether or not the pixel value of the same color pixel closest to the target pixel in the specific direction is the maximum value;
When the pixel value of the closest same color pixel in the specific direction of the target pixel is determined to be the maximum value by the specific direction maximum value detection means, the maximum value of the closest different color pixel in the specific direction of the target pixel is determined. It is determined whether or not the maximum value is appropriate as a second comparison reference value candidate by comparing with a pixel value, and when it is determined to be appropriate, the second maximum is adopted as the second comparison reference value candidate. Value appropriateness judgment / comparison reference value candidate selection means,
When the determination result of the second maximum value appropriateness determination / comparison reference value candidate selecting means makes the maximum value inappropriate as the second comparison reference value candidate, the first maximum value appropriateness determination / comparison reference When the first comparison reference value candidate adopted by the value candidate selection means is adopted as a final comparison reference value and the determination result makes the maximum value appropriate as a second comparison reference value candidate, the first comparison reference value candidate is adopted. A comparison reference value selection means that adopts the second comparison reference value candidate adopted by the maximum value appropriateness determination / comparison reference value candidate selection means as a final comparison reference value;
A pixel defect detection apparatus comprising: a target pixel value appropriate determination unit that determines appropriateness of the target pixel value based on the final comparison reference value adopted by the comparison reference value selection unit.   The second maximum value appropriateness determination / comparison reference value candidate selecting unit is configured to calculate an absolute value of a difference between the pixel value of the closest same color pixel in the specific direction of the target pixel and the pixel value of the closest different color pixel in the specific direction. Is compared with a second difference determination threshold value, and when the absolute value of the difference is equal to or smaller than the second difference determination threshold value, the maximum value of the pixel values of the nearest same color pixel is determined to be appropriate, and otherwise The pixel defect detection device according to claim 7, wherein the pixel defect is determined to be inappropriate.   The said specific direction maximum value detection means is comprised so that the pixel value of the nearest same color pixel of the perpendicular direction of the said attention pixel may be determined whether it is the said maximum value. Pixel defect detection device.   The first maximum value appropriateness determination / comparison reference value candidate selection unit compares a difference between the maximum value and the second value with a first difference determination threshold value, and the difference is less than the first difference determination threshold value. When the maximum value is appropriate, the maximum value is adopted as the first comparison reference value candidate, and when the difference is equal to or greater than the first difference determination threshold, the maximum value is inappropriate. The pixel defect detection device according to claim 7, wherein the second value is adopted as a first comparison reference value candidate.   The pixel defect detection device according to claim 10, wherein the first maximum value appropriateness determination / comparison reference value candidate selection unit is configured such that the first difference determination threshold value is variable.   The pixel defect detection according to claim 11, wherein the first maximum value appropriateness determination / comparison reference value candidate selection unit is configured to vary the first difference determination threshold according to an exposure time of an image sensor. apparatus.   The pixel defect detection device according to claim 11, wherein the first maximum value appropriateness determination / comparison reference value candidate selection unit is configured to vary the first difference determination threshold according to a video signal amplification factor. .   The target pixel value appropriate determination unit compares a difference from the final comparison reference value selected by the comparison reference value selection unit with a target pixel determination threshold, and the difference is equal to or smaller than the target pixel determination threshold. The pixel defect according to any one of claims 7 to 13, wherein the pixel value is determined to be appropriate and the target pixel value is determined to be inappropriate when the difference exceeds the target pixel determination threshold value. Detection device. A pixel defect correction device according to any one of claims 1 to 6,
When the pixel-of-interest value determination unit determines that the pixel-of-interest value is appropriate, pixel value replacement of the pixel-of-interest value is not performed, and when the pixel-of-interest value is determined inappropriate, the pixel-of-interest value is A pixel defect correction apparatus comprising pixel value replacement control means for replacing the comparison reference value. A pixel defect correction device according to any one of claims 7 to 14,
When the pixel-of-interest value determination unit determines that the pixel-of-interest value is appropriate, pixel value replacement of the pixel-of-interest value is not performed, and when the pixel-of-interest value is determined inappropriate, the pixel-of-interest value is A pixel defect correction apparatus comprising pixel value replacement control means for replacing the final comparison reference value.

Images