KR100780224B1 - Method of detecting and restoring defect pixels of image sensor - Google Patents

Abstract

The present invention relates to a method for detecting and restoring a bad pixel of an image sensor. In the method of determining and restoring a bad pixel of an image sensor, a plurality of pixels respectively detecting red, green, and blue are arranged in a mosaic pattern of a Bayer pattern. In a method of detecting and restoring a bad pixel for each pixel in a predetermined image sensor, the pixel to be inspected is an edge region of the entire image by comparing signal levels of a plurality of pixels in a horizontal direction around the pixel to be inspected. A first restoring step of determining whether the control target pixel is included in the control unit, determining whether the inspection target pixel is defective when the inspection target pixel is not included in the edge region, and restoring the inspection target pixel when the inspection target pixel is determined to be defective; And comparing the signal levels of a plurality of pixels forming an m × n matrix array (m, n is a natural number of 3 or more) centering on the inspection target pixel after the first reconstruction step is completed for all of the plurality of pixels. Determine a format of a region located in the entire image including the target pixel, determine whether the inspection target pixel is defective according to the format of the region including the inspection pixel, and restore the inspection target pixel if it is determined to be defective. And a second restoration step.

Image sensor, bad pixels, restore

Description

Defective pixel detection and restoration method of image sensor {METHOD OF DETECTING AND RESTORING DEFECT PIXELS OF IMAGE SENSOR}

1 shows a pixel arrangement of an image sensor according to the invention;

2 is a flowchart of a method for detecting and restoring a bad pixel of an image sensor according to the present invention;

3 is a flowchart of a first restoration process according to the present invention.

4 is a flowchart of a second restoration process according to the present invention;

5 is a flowchart illustrating a method for detecting and restoring a bad pixel of a pixel in an image edge region in a second restoration step according to the present invention.

6 is a diagram illustrating a first window and a second window in a method for detecting and restoring a defective sensor of an image sensor according to the present invention;

The present invention relates to a method for detecting and restoring a bad pixel of an image sensor, and more particularly, to a method for detecting and restoring a bad pixel generated in an image sensor having a plurality of pixels formed in a mosaic array.

In general, each subject in the natural world has different brightness and wavelengths of light. An image sensor is an element that outputs different brightness and wavelengths of an object as an electrical value at a signal processing level by using a property of a semiconductor that reacts to light. Recently, portable devices (eg, digital cameras, mobile communication terminals, etc.) equipped with image sensors have been developed and sold, and image sensors having a high pixel count have become more common.

On the other hand, as the number of pixels of the image sensor increases, defective pixels generated due to manufacturing process problems also increase, affecting the yield of the product. In addition, during the use of the image sensor, defective pixels are generated later due to deterioration of the image sensor due to impact or use. Such defective pixels are a cause of degrading the image quality of the image picked up by the image sensor.

SUMMARY OF THE INVENTION The present invention has been made to solve the above-mentioned problems of the prior art, and an object thereof is to increase the product yield of an image sensor to reduce the cost of the product, and to improve the image picked up by the image sensor. To provide a method for detecting and restoring.

In order to achieve the above technical problem, the bad pixel determination and restoration method of the image sensor of the present invention, in the image sensor in which a plurality of pixels for detecting red, green and blue are arranged in a mosaic array of Bayer pattern, In the method for detecting and restoring a bad pixel for each pixel,

It is determined whether the inspection target pixel is included in an edge region of the entire image by comparing signal levels of a plurality of pixels in a horizontal direction with respect to the pixel to be inspected, and when the inspection target pixel is not included in the edge region, A first restoring step of determining whether the inspection target pixel is defective and restoring the inspection target pixel when it is determined to be defective; And

After the first reconstruction step is completed for all of the plurality of pixels, the inspection target is compared by comparing signal levels of a plurality of pixels forming an m × n matrix array (m, n is a natural number of 3 or more) centering on the inspection target pixel. Determining a format of a region located in the entire image including the pixel, determining whether the inspection target pixel is defective according to a format of the region including the inspection target pixel, and restoring the inspection target pixel if it is determined to be defective. And a second restoration step.

The first restoring step may include: forming a first window including at least three pixels continuously arranged in a horizontal direction of the mosaic array including the inspection target pixel at a center thereof; Comparing signal levels of the plurality of pixels included in the first window to determine whether the inspection target pixel is included in an edge region of the image; When the inspection target pixel is not included in the edge area, the signal level of the inspection target pixel and the same color as the inspection target pixel among the pixels of the first window are detected and disposed closest to the inspection target pixel. Comparing the signal levels of the pixels to determine whether the inspection target pixel is a bad pixel; And when the pixel to be inspected is a bad pixel, the signal level of the pixel to be inspected is the same as that of the pixel to be inspected among the pixels of the first window, and the signal level of the pixel disposed closest to the pixel to be inspected is determined. And restoring the inspection target pixel by changing to an average value.

The determining whether the image is included in the edge region of the image may include detecting the same color among pixels included in the first window, and when the signal level differences of pixels adjacent to each other are smaller than a preset threshold, the inspection target. The pixel may be determined not to be included in an edge region of the image.

In the determining of whether the pixel is a bad pixel, when the pixel to be inspected is not included in an edge region of the image, a signal level of the pixel to be inspected detects the same color as the pixel to be inspected among the pixels of the first window. The inspection object pixel may be determined to be a bad pixel when the signal level of the pixel disposed closest to the inspection object pixel is greater than or equal to a predetermined threshold value or less than the predetermined threshold value.

The second reconstructing step may further include forming a second window including a plurality of pixels forming an m × n matrix array (m, n is a natural number of 3 or more) centered on the inspection target pixel; Comparing signal levels of each pixel included in the second window to determine whether the inspection target pixel is included in an edge region of an image; When the inspection target pixel is not included in the edge area, the signal level of the inspection target pixel and the same color as the inspection target pixel among the pixels of the second window are detected and disposed closest to the inspection target pixel. Comparing the signal levels of the pixels to determine whether the inspection target pixel is a bad pixel; And when the pixel to be inspected is a bad pixel, the signal level of the pixel to be detected is the same as that of the pixel to be inspected among the pixels of the second window, and the signal level of the pixel disposed closest to the pixel to be inspected is determined. And restoring the inspection target pixel by changing to an average value.

Determining whether or not included in the edge area, the first vertical edge value that is the sum of the signal level difference of the pixels for detecting two consecutive green in the vertical direction of the second window, in the vertical direction of the second window A second vertical edge value that is the sum of the differences between signal levels of pixels detecting two consecutive reds and the signal levels of pixels detecting two consecutive blues, and continuous in the horizontal direction of the second window; The first horizontal edge value, which is the sum of the signal level differences of the pixels detecting two greens, and the difference in the signal levels of the pixels detecting two red consecutive in the horizontal direction of the second window, and in the horizontal direction of the second window. Calculating a second horizontal edge value that is a sum of differences of signal levels of pixels for detecting two consecutive blues; And determining that the inspection target pixel is not included in the edge area when the first vertical edge value, the second vertical edge value, the first horizontal edge value, and the second horizontal edge value are smaller than a preset threshold. It is characterized by including.

In the determining of whether the pixel is a bad pixel, when the pixel to be inspected is not included in the edge area, the signal level of the pixel to be inspected is the same color as that of the pixel to be inspected among the pixels of the second window, and the inspection is performed. The inspection object pixel may be determined to be a bad pixel when the threshold value is greater than or equal to the average value of the signal level of the pixel disposed closest to the target pixel or less than or equal to the predetermined threshold value.

In the second reconstructing step, when the inspection target pixel is included in the image edge region, the signal level of each pixel included in the second window is compared so that the inspection target pixel is a vertical edge region and a horizontal edge region of the image. And determining the format of the image area to determine which one of the diagonal edge areas is included. Determining whether the inspection target pixel is a bad pixel according to a format determination result of the image area; And restoring the inspected pixel when the inspected pixel is determined to be a bad pixel.

The determining of the format of the image area may include: a first vertical edge value that is a sum of signal level differences between pixels detecting two green colors continuous in a vertical direction of the second window, and a continuous value in the vertical direction of the second window; A second vertical edge value that is the sum of the differences between the signal levels of the pixels detecting the two reds and the differences in the signal levels of the pixels detecting the two blues, and two consecutive in the horizontal direction of the second window; The first horizontal edge value, which is the sum of the signal level differences of the pixels detecting green, and the difference in the signal levels of the pixels detecting two red consecutive in the horizontal direction of the second window, and the horizontal direction of the second window. Calculating a second horizontal edge value that is a sum of differences of signal levels of pixels detecting two blues; The pixel to be inspected when the first vertical edge value is greater than the sum of the first horizontal edge value and the preset threshold and the second vertical edge value is greater than the sum of the second horizontal edge value and the preset threshold. Determining to be in the horizontal edge region; The pixel to be inspected when the first horizontal edge value is greater than the sum of the first vertical edge value and the preset threshold and the second horizontal edge value is greater than the sum of the second vertical edge value and the preset threshold. Determining that it is in the vertical edge region; And determining that the inspection target pixel is in a diagonal edge region when the pixel to be inspected is not in the horizontal edge region and the vertical edge region.

When the pixel to be inspected is included in the horizontal edge region, determining whether the pixel to be inspected is the bad pixel may include detecting a color having the same signal level as that of the pixel to be inspected among the pixels of the second window. When the threshold level is greater than or equal to a predetermined threshold value or less than or equal to a predetermined threshold value than a signal level of a pixel disposed closest to the horizontal direction of the target pixel, determining the inspection target pixel as a bad pixel and restoring the inspection target pixel may include: The signal level of the pixel to be inspected is changed to an average value of the signal levels of pixels arranged closest to each other in the horizontal direction of the pixel to be detected by detecting the same color as the pixel to be inspected among the pixels of the second window. do.

In addition, when the inspection target pixel is included in the vertical edge area, the determining of whether the inspection pixel is the bad pixel may include detecting a color having the same signal level as the inspection pixel among the pixels of the second window. Determining the inspection target pixel as a bad pixel when the signal level is greater than or equal to a predetermined threshold value or less than or equal to a signal level of a pixel disposed most closely in the vertical direction of the inspection object pixel, and restoring the inspection object pixel; Detects the same color as the inspection target pixel among the pixels in the second window and changes the signal level of the inspection target pixel to an average value of the signal levels of the pixel disposed closest in the vertical direction of the inspection target pixel. It features.

In addition, when the inspection target pixel is included in the diagonal edge area, the determining of whether the inspection pixel is the bad pixel may include detecting a color having the same signal level as the inspection pixel among the pixels of the second window. Determining the inspection target pixel as a bad pixel when the signal level is greater than or equal to a predetermined threshold value or less than or equal to a signal level of a pixel disposed most closely in the vertical direction of the inspection object pixel, and restoring the inspection object pixel; The signal level of the pixel to be inspected may be changed to an average value of a signal level of a pixel surrounding the test pixel by detecting the same color as the pixel to be inspected among the pixels of the second window.

Hereinafter, various embodiments of the present invention will be described in more detail with reference to the accompanying drawings.

1 is a diagram illustrating a pixel array of an image sensor according to the present invention.

The pixels of the image sensor have a mosaic arrangement. Each pixel detects, filters, and converts one color among red (R), green (G), and blue (B) into an electrical signal in the subject. Pixels having a mosaic arrangement in the image sensor form a predetermined pattern according to the detected color, and the most common pattern is a Bayer pattern as shown in FIG. 1. That is, half of all the pixels detect green, and each quarter of the total number receives red and green. In order to obtain color information, image pixels are formed of a repetitive pattern of red, green, or blue.

The Bayer pattern is based on the premise that the human eye derives most of the luminance data from the green content of the subject. Thus, by allowing more of the pixels to process signals for green, an image of higher resolution can be generated as compared to the pattern in which pixels detecting the same number of red, green and red signals repeat.

2 is a flowchart illustrating a method for detecting and restoring a bad pixel of an image sensor according to the present invention. 2 illustrates a process of restoring a bad pixel through two restoration steps of an image captured by an image sensor.

3 is a flowchart showing a first restoration process according to the present invention. 3 illustrates a process of forming a first window for first restoration of a bad pixel, and detecting and restoring a bad pixel when the pixel to be inspected is not included in an edge region of the image.

4 is a flowchart showing a second restoration step process according to the present invention. 4 illustrates a process of forming a second window for second restoration of a bad pixel, and detecting and restoring a bad pixel when the pixel to be inspected is not included in an edge region of the image.

FIG. 5 is a flowchart illustrating a method of detecting and restoring a bad pixel of an inspection target pixel included in an image edge area in a second restoration step according to the present invention. 5 illustrates a process of determining a format of an edge region when the inspection target pixel is included in an image edge region, and detecting and restoring a bad pixel according to the format of each edge region.

6 is a diagram illustrating a first window and a second window for explaining a method of detecting and restoring a defective sensor of an image sensor according to the present invention. As shown in FIG. 6, the first window may include nine pixels (defined as P1, P2, P3, P4, P5, P6, P7, P8, and P9) which are continuous in the horizontal direction. P1, P3, P5, P7, and P9 are formed of the same color, and P2, P4, P6, and P8 are also formed of pixels of the same color.

 The second window is composed of 25 pixels in which each pixel is represented by Pij (i, j is a natural number less than or equal to 5), and each row is alternately arranged with pixels of two colors. The inspection target pixels in the first window and the second window are P5 and P33 which are pixels located at the center.

Hereinafter, the operation and effects of the present invention will be described in detail with reference to the accompanying drawings.

2 to 6, in order to determine and restore a bad pixel of the present invention, an image is first picked up using an image sensor (S100). Each pixel of the image sensor generates an electrical signal having a signal level corresponding to the corresponding color from the light of the subject. The electrical signal is an analog signal, which is converted into a digital value for image processing, and each pixel has a digitally converted signal level.

When each pixel has a signal level with respect to the image, it is determined whether the inspection target pixel is included in an edge region of the entire image by comparing signal levels of a plurality of pixels in a horizontal direction with respect to the pixel to be inspected. When the target pixel is not included in the edge area, the first restoration step (S200) of determining whether the inspection target pixel is defective or restoring the inspection target pixel when the determination is determined to be defective is started.

The first reconstructing step (S200) is a step of forming a first window including at least three or more pixels continuously arranged in a horizontal direction of the mosaic array including the inspection target pixel at the center (S210), and the first window. Determining whether the inspection target pixel is included in an edge region of the image by comparing a signal level of each pixel included in the operation (S220), and when the inspection target pixel is not included in an edge region of the image, the inspection It is determined whether the inspection target pixel is a bad pixel by comparing a signal level of a target pixel with a signal level of a pixel disposed closest to the inspection target pixel by detecting the same color as the inspection target pixel among the pixels of the first window. In step S230 and if the inspection target pixel is a bad pixel, the signal level of the inspection target pixel is determined among the pixels of the first window. And detecting the same color as the pixel to be inspected, and changing the signal to an average value of signal levels of a pixel disposed closest to the pixel to be inspected (S240).

In the step S210 of forming a first window, a first window including at least three or more pixels continuously arranged in a horizontal direction of a mosaic array is formed. The inspection target pixel is a pixel located at the center of the first window. Preferably, the first window may be composed of nine pixels.

In order to determine whether the pixel is a bad pixel, it is first determined whether the inspection target pixel is in an edge region of the image (S220). If the inspection target pixel is in an edge region of the image, the signal level of the inspection pixel is relatively different from any one of the signal levels of the pixel on the left or right side of the inspection pixel.

Whether the inspection target pixel is a bad pixel is determined by how much the signal level is different from the surrounding pixels. Therefore, even when the inspection target pixel is in an edge region of the image, the inspection pixel is a normal pixel. Since the signal level is different from the signal level of surrounding pixels, the signal level may be determined to be a bad pixel.

Therefore, in the first reconstruction step S200, when the inspection target pixel is included in the edge area of the image, the bad pixel determination and reconstruction are not performed, and the bad pixel determination is performed only when it is not included in the edge area of the image.

When the inspection pixel is not included in the edge region of the image, pixels having the same color adjacent to the inspection pixel have a similar signal level. Therefore, by comparing the signal levels of the pixels detecting the same color among the pixels included in the first window, it may be determined whether the inspection target pixel is in an edge region of the image.

Preferably, the pixel to be inspected is not included in an edge region of the image when each of the pixels included in the first window detects the same color and each of signal level differences of pixels adjacent to each other is smaller than a preset threshold. It may be to judge.

For example, | P2-P4 |, | P1-P3 |, | P6-P8 |, | P7-P9 | And || P2-P4 |-| P6-P8 || (P1, P2, P3, P4, P5, P6, P7, P8, and P9 indicate the signal level of the corresponding pixel for explanation). If smaller, the inspection target pixel P5 may be determined not to be included in the image edge region.

In the case where the inspection target pixel P5 is a bad pixel, a value different from a neighboring pixel that detects the same color among the pixels of the first window is different from that of the pixel when the pixel is included in the image edge region. Calculate it except because it affects. The threshold value is a setting value for determining the accuracy of determining the image edge area. The lower the value, the higher the precision. Hereinafter, various threshold values are used in the description of the method for detecting and restoring defective pixels of the image sensor of the present invention, and the threshold values according to each use may be different values. The lower the threshold value is, the higher the accuracy of detection of defective pixels is. However, if the threshold value is too low, the normal pixels can be determined as defective pixels, and are set appropriately so as to obtain an optimal result.

When the inspection target pixel is included in the image edge region, the first restoration step of the inspection pixel is terminated and the first restoration step of the next inspection pixel is performed.

If the inspection target pixel is not included in the image edge area, it is determined whether the inspection target pixel is a bad pixel (S230).

If the pixel to be inspected is not included in the image edge area, the signal level of the pixel to be inspected is similar to that of a pixel of the same color adjacent thereto, so that the signal level of the pixel to be inspected is equal to the signal level of an adjacent pixel having the same color. By comparing, it is possible to determine whether or not the defective pixel.

When the pixel to be inspected is not included in an edge region of the image, a signal level of the pixel to be inspected detects the same color as the pixel to be inspected among the pixels of the first window and is disposed closest to the pixel to be inspected. When the threshold level is greater than or equal to a predetermined threshold value or less than or equal to a signal level of the pixel, the inspection target pixel may be determined as a bad pixel.

For example, when {P5> P3 + threshold} and {P5> P7 + threshold} or {P5 + threshold <P3} and {P5 + threshold <P7}, the inspection target pixel is determined as a bad pixel. It may be. {P5> P3 + Threshold} and {P5> P7 + Threshold} indicate that the pixel P5 to be inspected is a bad pixel whose signal level is excessively higher than the surrounding pixels, and {P5 + threshold <P3} and { When P5 + threshold <P7}, it indicates that the inspection target pixel P5 is a bad pixel whose signal level is excessively lower than the surrounding pixels.

If it is determined that the inspection target pixel is a bad pixel, the inspection target pixel is restored (S240). The defective pixel is detected by changing a signal level of the inspection target pixel to a mean value of a signal level of a pixel disposed closest to the inspection target pixel by detecting the same color as the inspection target pixel among the pixels of the first window. It may be to restore.

For example, the signal level of the inspection target pixel may be changed to (P3 + P7) / 2. The pixels P1 and P9 have the same color as the pixel P5 to be inspected. However, when the pixels P1 and P9 are relatively far from P5 and restored, the restoration accuracy is inferior. Therefore, the signal level of the pixel P5 is changed to the average value of the signal levels of the adjacent pixels P3 and P7. It may be.

When the first restoration step S200 for the inspection target pixel ends, the first restoration step S200 for the next inspection pixel is repeated, and when the first restoration step for all the pixels is completed, the second restoration step S300. Is disclosed.

After the defective pixels are first detected and restored in the horizontal direction of the mosaic array, the second restoration step S300 is started, so that the second restoration step can more accurately detect and restore the defective pixels.

In the second reconstructing step (S300), forming a second window including a plurality of pixels forming an m × n matrix array (m, n is a natural number of 3 or more) around the inspection target pixel (S310); Comparing the signal levels of the pixels included in the two windows to determine whether the inspection target pixel is included in the image edge region (S320); and when the inspection target pixel is not included in the image edge region, In operation S330 of determining whether the pixel to be inspected is a bad pixel and when the pixel to be inspected is a bad pixel, the signal level of the pixel to be inspected is changed using the signal level of a pixel adjacent to the test pixel to restore the bad pixel. It includes a step (S340).

In the step S310 of forming a second window, a second window including a plurality of pixels forming an m × n matrix array (m, n is a natural number of 3 or more) is formed. Preferably, the matrix array is the same number. It may have rows and columns of. In addition, the 3x3 matrix array is not preferable because the number of pixels included is small, so that the accuracy of bad pixel detection is inferior, and the matrix array of 7x7 or more is not correlated with the inspection target pixel. Preferably the second window may be a 5 × 5 matrix arrangement. The pixel located at the center of the matrix array is the inspection target pixel.

After the second window is formed, it is determined whether the inspection target pixel is included in the image edge region (S320). In the first reconstruction step (S200), there is also a step (S220) of determining whether the inspection target pixel is included in the image edge area. However, the first reconstruction step (S200) compares the signal level of the pixel in the horizontal direction to the inspection target pixel. Is determined to be included in the image edge region, and is different from the second reconstruction step (S300) of determining whether the inspection target pixel is included in the image edge region by comparing signal levels of pixels in the horizontal and vertical directions.

Determining whether the image is included in the image edge area (S320) includes a first vertical edge value, which is a sum of signal level differences between pixels for detecting two consecutive greens in the vertical direction of the second window, and the vertical of the second window. A second vertical edge value that is a sum of differences between signal levels of pixels detecting two successive reds in a direction and a signal level of pixels detecting two successive blues in a vertical direction, in a horizontal direction of the second window; The first horizontal edge value that is the sum of the signal level differences of the pixels for detecting two consecutive greens, and the difference between the signal levels of the pixels for detecting two consecutive reds in the horizontal direction of the second window and the horizontal of the second window. Calculating a second horizontal edge value that is a sum of differences of signal levels of pixels for detecting two consecutive blues in the direction; And determining that the inspection target pixel is not included in the edge area when the first vertical edge value, the second vertical edge value, the first horizontal edge value, and the second horizontal edge value are smaller than a preset threshold. Include.

For example, if P33, the pixel to be inspected in FIG. 6B, is a pixel for detecting green color, P11, P13, P15, P22, P24, P31, P35, P42, P44, P51, P53, and P55 are green in the Bayer pattern. To be a pixel for detecting. If P12 is a pixel that detects blue, P14, P32, P34, P52, and P54 become pixels that detect blue, and P21, P23, P25, P41, P43, and P45 become pixels that detect red.

Since the first vertical edge value is the sum of the signal level differences of the pixels detecting two consecutive greens in the vertical direction of the second window, the first vertical edge value = | P11-P31 | + | P31-P51 | + | P22-P42 | + | P24-P44 | + | P15-P35 | + | P35-P55 |

Since the second vertical edge value is the sum of the differences between the signal levels of the pixels detecting two consecutive reds in the vertical direction of the second window and the differences in the signal levels of the pixels detecting two consecutive blues in the vertical direction, Second vertical edge value = | P21-P41 | + | P12-P32 | + | P32-P52 | + | P23-P43 | + | P14-P34 | + | P34-P54 | + | P25-P45 |

Since the first horizontal edge value is the sum of the signal level differences of the pixels detecting two consecutive greens in the horizontal direction of the second window, the first horizontal edge value = | P11-P13 | + | P13-P15 | + | P22-P24 | + | P42-P44 | + | P51-P53 | + | P53-P55 |

The second horizontal edge value is the difference between the signal levels of the pixels detecting two red consecutive in the horizontal direction of the second window and the signal levels of the pixels detecting two blue consecutive in the horizontal direction of the second window. Is the sum of the differences, so the second horizontal edge value = | P12-P14 | + | P21-P23 | + | P23-P25 | + | P32-P34 | + | P41-P43 | + | P43-P45 | + | P52-P54 |.

If the first vertical edge value, the second vertical edge value, the first horizontal edge value, and the second horizontal edge value are smaller than the preset threshold value, the difference in signal level between adjacent pixels is small. The inspection target pixel P33 may be determined not to be included in the image edge area.

If the inspection target pixel is not included in the image edge area, it is determined whether the inspection target pixel is a bad pixel (S330).

Since the pixels not included in the image edge region are close and have a similar signal level to the pixels detecting the same color, the signal level of the inspected pixel detects the same color as the inspected pixel among the pixels of the second window. The inspection target pixel may be determined to be a bad pixel when the threshold value is greater than or equal to the average value of the signal level of the pixel disposed closest to the inspection pixel or less than or equal to the predetermined threshold value.

For example, in FIG. 6B, when the inspection target pixel P33 is a pixel for detecting green, the pixels for detecting green closest to the inspection pixel are P13, P31, P35, and P53, and the signal level thereof is a signal of P33. It may be determined whether or not the defective pixel is compared with the level.

That is, when {P33> (P13 + P31 + P35 + P53) / 4 + threshold} or {P33 + threshold <(P13 + P31 + P35 + P53) / 4 + threshold}, the pixel to be inspected is defective. It may be determined as a pixel. {P33> (P13 + P31 + P35 + P53) / 4 + Threshold} indicates that the pixel P33 to be inspected is a bad pixel whose signal level is excessively higher than that of the surrounding pixels, and {P33 + Threshold <(P13 + P31 + P35 + P53) / 4 + threshold} indicates that the inspection target pixel P33 is a bad pixel having an excessively low signal level compared to the surrounding pixels.

If the inspection target pixel is determined to be a bad pixel, the inspection target pixel is restored (S340). If the pixel to be inspected is a bad pixel, the signal level of the pixel to be inspected is the average value of the signal level of the pixel disposed closest to the pixel to be detected by detecting the same color as the pixel to be inspected among the pixels of the second window. It may change to restore the inspection target pixel.

For example, since the pixels P13, P31, P35, and P53 are the closest pixels that detect the same color as the inspection target pixel P33, the signal level of the inspection target pixel is the average value of the signal levels of the pixels P13, P31, P35, and P53. It may be changed to (P13 + P31 + P35 + P53) / 4.

When the inspection target pixel is included in the image edge region, the format of the region located in the entire image including the inspection pixel is any one of a horizontal edge, a vertical edge, or a diagonal edge. Therefore, when the inspection target pixel is included in the image edge region, it is determined whether the inspection target pixel is included in a horizontal edge region, a vertical edge region, or a diagonal edge region (S500), and according to each image format. It determines whether it is a bad pixel and restores it (S510-S560).

For example, when {first vertical edge value> first horizontal edge value + threshold value} and {second vertical edge value> second horizontal edge value + threshold value}, the inspection target pixel is included in the horizontal edge area. May be judged to be.

Further, when the {first horizontal edge value <first vertical edge value + threshold value} or {second horizontal edge value> second vertical edge value + threshold value}, the inspection target pixel is included in the vertical edge area. It may be to judge.

If the inspection target pixel is not in the horizontal edge region or the vertical edge region, it may be determined that the pixel to be inspected is included in the diagonal edge region.

If it is determined that the inspection target pixel is in a horizontal edge area, it is determined whether the inspection pixel is a bad pixel by comparing the signal level of the inspection pixel with a signal level of a pixel detecting the same color closest to the inspection pixel in a horizontal direction ( S510).

If it is determined that the inspection target pixel is in a horizontal edge region, the signal level of the inspection target pixel detects the same color as the inspection target pixel among the pixels of the second window and is closest to the horizontal direction of the inspection target pixel. If the threshold level is greater than or equal to a predetermined threshold value or less than or equal to a predetermined signal level, the inspection target pixel is determined to be a bad pixel.

When the pixel is in the horizontal edge area, the pixel detecting the same color in the vertical direction is much different from the signal level of the pixel to be inspected, and even though the pixel to be inspected is not a bad pixel, the pixel may be determined as a bad pixel. It is determined whether or not the defective pixel is compared with the signal level of the pixel for detecting the same color.

Therefore, when the inspection target pixel is included in the horizontal edge region of the image, the signal level of the inspection target pixel is closer to the inspection pixel in the horizontal direction and is higher than the signal level of the pixel detecting the same color as the inspection target pixel. If the threshold value is greater than or equal to the preset threshold or less than the preset threshold, the inspection target pixel may be determined as a bad pixel.

For example, the pixels that detect the same color as the inspection target pixel P33 and are closest to P33 in the horizontal direction are P31 and P35, so {P33> P31 + threshold} and {P33> P35 + threshold}, or {P33 + When the threshold value <P31} and {P33 + threshold value <P35}, it may be determined that the inspection target pixel is a bad pixel.

{P33> P31 + threshold} and {P33> P35 + threshold} indicate that the pixel P33 to be inspected is a bad pixel having a higher signal level than the surrounding pixels, and {P33 + threshold <P31} and { When P33 + threshold <P35}, it indicates that the inspection target pixel P33 is a bad pixel having a lower signal level than the surrounding pixels.

If it is determined that the inspection target pixel is a bad pixel, the inspection target pixel is restored (S520). The signal level of the inspection target pixel is detected by detecting the same color as the inspection target pixel among the pixels of the second window, and changing the signal level to an average value of the signal levels of the pixel disposed closest to the horizontal direction of the inspection target pixel. It may be to restore the pixel.

For example, the signal level of the inspection target pixel may be changed to (P31 + P35) / 2. As described above, when the bad pixel is in the horizontal edge region at the time of restoring the bad pixel, the signal level of the adjacent pixel in the vertical direction is excluded, thereby making it possible to more accurately recover.

If it is determined that the inspection target pixel is in a vertical edge area, it is determined whether the inspection pixel is a bad pixel by comparing the signal level of the inspection target pixel with the signal level of a pixel that detects the same color closest to the inspection pixel in a vertical direction ( S530).

The determining of whether the pixel is a bad pixel, the signal level of the pixel to be detected is the same color as the pixel to be inspected among the pixels of the second window and the signal of the pixel disposed closest to the vertical direction of the pixel to be inspected When the threshold value is greater than or equal to the predetermined threshold value or less than the predetermined threshold value, the inspection target pixel may be determined as a bad pixel.

When the pixel is in the vertical edge area, the pixel detecting the same color in the horizontal direction is different from the signal level of the pixel to be inspected, and even though the pixel to be inspected is not a bad pixel, the pixel may be determined as a bad pixel. It is determined whether or not the defective pixel is compared with the signal level of the pixel for detecting the same color.

Therefore, when the inspection target pixel is included in the vertical edge area of the image, the signal level of the inspection target pixel is closer to the inspection pixel in the vertical direction and is higher than the signal level of the pixel detecting the same color as the inspection target pixel. If the threshold value is greater than or equal to the preset threshold or less than the preset threshold, the inspection target pixel may be determined as a bad pixel.

For example, the pixels that detect the same color as the inspection target pixel P33 and are closest to P33 in the vertical direction are P13 and P53, so {P33> P13 + threshold} and {P33> P53 + threshold}, or {P33 + When the threshold value <P13} and {P33 + threshold value <P53}, the inspection target pixel may be determined as a bad pixel.

{P33> P13 + Threshold} and {P33> P53 + Threshold} indicate that the pixel P33 to be inspected is a bad pixel having a higher signal level than the surrounding pixels, and {P33 + threshold <P13} and { P33 + threshold <P53} indicates that the pixel P33 is a defective pixel having a lower signal level than the surrounding pixels.

If it is determined that the inspection target pixel is a bad pixel, the inspection target pixel is restored (S540). The signal level of the inspection target pixel is detected by detecting the same color as the inspection pixel among the pixels of the second window, and changing the signal level to an average value of the signal levels of the pixel disposed closest to the vertical direction of the inspection pixel. It may be to restore the pixel. For example, the signal level of the inspection target pixel may be changed to (P13 + P53) / 2. As described above, when the bad pixel is in the vertical edge region at the time of restoring the bad pixel, the signal level of the adjacent pixel in the horizontal direction is excluded, thereby more accurately restoring.

When it is determined that the inspection target pixel is in the diagonal edge region, it is determined whether the inspection pixel is a bad pixel by comparing the signal level of the inspection target pixel with the signal level of the surrounding pixel.

The average value of the pixels surrounding the inspection target pixel and detecting the same color as the inspection target pixel among the pixels of the second window, and the average value of the pixels surrounding the inspection target pixel and detecting a color different from the inspection target pixel. It may be determined whether the inspection target pixel is a bad pixel compared with the signal level of the target pixel.

For example, when an average value of pixels surrounding the inspection target pixel and detecting the same color as the inspection target pixel is defined as a first average, a first average = (P22 + P24 + P42 + P44) / 4, and If the average value of pixels surrounding the inspection target pixel and detecting a color different from the inspection target pixel is defined as a second average, the second average = (P23 + P32 + P34 + P43) / 4. The test subject when {P33> first average + threshold} and {P33> second average + threshold}, or {P33 + threshold <first average} and {P33 + threshold <second average}. The pixel P33 may be determined to be a bad pixel.

When {P33> first average + threshold} and {P33> second average + threshold}, it indicates that the pixel P33 to be inspected is a bad pixel whose signal level is excessively high in surrounding pixels, and {P33 + threshold <1st average} and {P33 + threshold <2nd average} indicate that the pixel P33 is a defective pixel whose signal level is excessively low in the surrounding pixels.

In addition, the first average and the second average may be defined as first average = MEDIAN (P22, P24, P42, P44) and second average = MEDIAN (P23, P32, P34, P43), respectively.

If it is determined that the inspection target pixel is a bad pixel, the inspection target pixel is restored (S560). Restoring the inspection target pixel that detects the same color as the inspection target pixel among the pixels in the second window and changes the signal level of the inspection target pixel to an average value of the signal levels of the pixel surrounding the inspection target pixel. have. For example, the signal level of the inspection target pixel may be changed to the first average.

In each of the above cases, when the detection and restoration of the bad pixels are finished, the second restoration S300 for the next pixel is started, and when the second restoration S300 for all the pixels is finished, the detection of the bad pixels in the image sensor is completed. And restoration ends.

As described above, the defective pixels in the image sensor may be more effectively and accurately determined and restored by performing a first restoration in the horizontal pixel array and a second restoration of the pixels in the matrix array.

The present invention is not limited by the above-described embodiment and the accompanying drawings, but by the appended claims. Therefore, it will be apparent to those skilled in the art that various forms of substitution, modification, and alteration are possible without departing from the technical spirit of the present invention described in the claims, and the appended claims. Will belong to the technical spirit described in.

As described above, according to the present invention, by detecting and restoring the defective pixel of the image sensor, there is an effect of increasing the yield of the image sensor and improving the image quality of the captured image. In addition, by detecting and restoring the bad pixels from the horizontal pixels in a continuous direction, detecting and restoring the bad pixels from the pixels arranged in a matrix form, it is possible to increase the accuracy of detecting and restoring the bad pixels.

Claims (12)

In an image sensor in which a plurality of pixels respectively detecting red, green, and blue are arranged in a mosaic pattern of a Bayer pattern, a method of detecting and restoring a bad pixel for each pixel, It is determined whether the inspection target pixel is included in an edge region of the entire image by comparing signal levels of a plurality of pixels in a horizontal direction with respect to the pixel to be inspected, and when the inspection target pixel is not included in the edge region, A first restoring step of determining whether the inspection target pixel is defective and restoring the inspection target pixel when it is determined to be defective; And After the first reconstruction step is completed for all of the plurality of pixels, the inspection target is compared by comparing signal levels of a plurality of pixels forming an m × n matrix array (m, n is a natural number of 3 or more) centering on the inspection target pixel. It is determined whether the pixel is included in the edge region of the entire image. When the pixel to be inspected is not included in the edge region, the pixel to be inspected is determined to be a bad pixel using the signal level of a pixel that detects the same color in the matrix array. A second restoration step of determining and restoring whether or not the inspection target pixel is a bad pixel according to a direction of an edge including the inspection target pixel when the inspection target pixel is included in an edge region; Bad pixel determination and restoration method of the image sensor comprising a. The method of claim 1, The first restoration step, Forming a first window including at least three pixels continuously arranged in a horizontal direction of the mosaic array including the inspection target pixel at a center thereof; Comparing signal levels of the plurality of pixels included in the first window to determine whether the inspection target pixel is included in an edge region of an image; When the inspection target pixel is not included in the edge area, the signal level of the inspection target pixel and the same color as the inspection target pixel among the pixels of the first window are detected and disposed closest to the inspection target pixel. Comparing the signal levels of the pixels to determine whether the inspection target pixel is a bad pixel; And If the pixel to be inspected is a bad pixel, the signal level of the pixel to be detected is the same value as that of the pixel to be inspected among the pixels of the first window, and an average value of the signal levels of a pixel disposed closest to the pixel to be inspected. Restoring the inspection target pixel by changing to Defective pixel detection and restoration method of the image sensor comprising a. The method of claim 2, Determining whether it is included in the edge region of the image, When the same color is detected among the pixels included in the first window and each of signal level differences between pixels adjacent to each other is smaller than a preset threshold, it is determined that the inspected pixel is not included in an edge region of the image. A method for detecting and restoring defective pixels of an image sensor The method of claim 2, wherein the determining of whether the pixel is defective is performed by: When the pixel to be inspected is not included in an edge region of the image, a signal level of the pixel to be inspected detects the same color as the pixel to be inspected among the pixels of the first window and is disposed closest to the pixel to be inspected. And determining the inspection target pixel as a bad pixel when the signal level of the pixel is greater than or equal to a predetermined threshold value or less than a predetermined threshold value. The method of claim 1, wherein the second restoration step, Forming a second window including a plurality of pixels forming an m × n matrix array (m, n is a natural number of 3 or more) centering on the inspection target pixel; Comparing signal levels of each pixel included in the second window to determine whether the inspection target pixel is included in an edge region of an image; When the inspection target pixel is not included in the edge area, the signal level of the inspection target pixel and the same color as the inspection target pixel among the pixels of the second window are detected, and are vertically and horizontally centered about the inspection target pixel. Comparing signal levels of pixels arranged closest to each other in a direction to determine whether the inspection target pixel is a bad pixel; And If the inspected pixel is a bad pixel, the signal level of the inspected pixel is detected in the same color as the inspected pixel among the pixels of the second window, and is disposed closest to the inspected pixel in the vertical and horizontal directions. Restoring the inspection target pixel by changing to an average value of the signal levels of the pixels. Defective pixel detection and restoration method of the image sensor comprising a. The method of claim 5, wherein the determining of whether the edge region is included in the edge region comprises: The first vertical edge value which is the sum of the signal level differences of the pixels detecting two green consecutive in the vertical direction of the second window, and the signal level of the pixel detecting the two red continuous in the vertical direction of the second window. A second vertical edge value that is the sum of the differences between signal levels of the pixels detecting two blue colors that are continuous in the vertical direction, and a sum of the signal level differences of pixels that detect two green colors continuous in the horizontal direction of the second window; The difference between the signal level of the pixel detecting the first horizontal edge value and two red consecutive in the horizontal direction of the second window and the signal level of the pixel detecting the two blue continuous in the horizontal direction of the second window. Calculating a second horizontal edge value that is the sum of the differences; And Determining that the inspection target pixel is not included in the edge area when the first vertical edge value, the second vertical edge value, the first horizontal edge value, and the second horizontal edge value are smaller than a preset threshold. Defective pixel detection and restoration method of the image sensor comprising a. The method of claim 5, wherein the determining of whether the pixel is defective is performed. When the inspection target pixel is not included in the edge area, a pixel in which the signal level of the inspection target pixel is the same as the inspection target pixel among the pixels of the second window and detects the same color and is disposed closest to the inspection target pixel. And determining the pixel to be inspected as a bad pixel when the threshold value is greater than or equal to a predetermined threshold value or less than a mean value of a signal level. The method of claim 5, wherein the second restoration step, When the inspection target pixel is included in the image edge region, the signal level of each pixel included in the second window is compared, so that the inspection target pixel is any one of a vertical edge region, a horizontal edge region, and a diagonal edge region of the image. Determining a format of an image area to determine whether the image area is included in the image area; Determining whether the inspection target pixel is a bad pixel according to a format determination result of the image area; And Restoring the inspected pixel when the inspected pixel is determined to be a bad pixel; Bad pixel detection and restoration method of the image sensor, characterized in that it further comprises. The method of claim 8, wherein the determining of the format of the image area comprises: The first vertical edge value which is the sum of the signal level differences of the pixels detecting two green consecutive in the vertical direction of the second window, and the signal level of the pixel detecting the two red continuous in the vertical direction of the second window. A second vertical edge value that is the sum of the differences between signal levels of the pixels detecting two blue colors that are continuous in the vertical direction, and a sum of the signal level differences of pixels that detect two green colors continuous in the horizontal direction of the second window; The difference between the signal level of the pixel detecting the first horizontal edge value and two red consecutive in the horizontal direction of the second window and the signal level of the pixel detecting the two blue continuous in the horizontal direction of the second window. Calculating a second horizontal edge value that is the sum of the differences; When the first vertical edge value is greater than the sum of the first horizontal edge value and a preset threshold, and the second vertical edge value is greater than the sum of the second horizontal edge value and the preset threshold. Determining that the pixel is in a horizontal edge region; The pixel to be inspected when the first horizontal edge value is greater than the sum of the first vertical edge value and the preset threshold and the second horizontal edge value is greater than the sum of the second vertical edge value and the preset threshold. Determining that it is in the vertical edge region; And Determining that the pixel to be inspected is in a diagonal edge region when the pixel to be inspected is not in a horizontal edge region and a vertical edge region; Defective pixel detection and restoration method of the image sensor comprising a. The method of claim 8, wherein when the inspection target pixel is included in the horizontal edge area, The determining of whether the pixel is a bad pixel may include detecting a signal having the same color level as that of the pixel to be inspected among the pixels of the second window, and a signal of a pixel disposed closest to the horizontal direction of the pixel to be inspected. If the threshold value is greater than or equal to a predetermined threshold value or less than a predetermined threshold value, the inspection target pixel is determined as a bad pixel, The restoring of the pixel to be inspected may include detecting a signal level of the pixel to be inspected the same color as the pixel to be inspected among the pixels of the second window and arranging the pixel closest to the horizontal direction of the pixel to be inspected. A method for detecting and restoring a bad pixel of an image sensor, the method comprising changing to an average value of a signal level. The method of claim 8, wherein when the inspection target pixel is included in the vertical edge area, The determining of whether the pixel is a bad pixel, the signal level of the pixel to be detected is the same color as the pixel to be inspected among the pixels of the second window and the signal of the pixel disposed closest to the vertical direction of the pixel to be inspected If the threshold value is greater than or equal to a predetermined threshold value or less than a predetermined threshold value, the inspection target pixel is determined as a bad pixel, The restoring of the pixel to be inspected may include detecting a signal level of the pixel to be inspected the same color as the pixel to be inspected among the pixels of the second window, and determining the signal level of the pixel to be most closely disposed in a vertical direction of the pixel to be inspected. A method for detecting and restoring a bad pixel of an image sensor, characterized by changing the average value of the signal level. The method of claim 8, wherein when the inspection target pixel is included in the diagonal edge area, The determining of the bad pixel may include determining a test pixel as a bad pixel by comparing a signal level of the test target pixel with signal levels of pixels closest to each other in the vertical, horizontal, and diagonal directions about the test target pixel. , The restoring of the pixel to be inspected may include a signal of pixels that detect the same color as the pixel to be inspected among the pixels closest to each other in the vertical, horizontal, and diagonal directions of the signal to be inspected. A method for detecting and restoring defective pixels of an image sensor, characterized by changing the average value of the levels.

Images