JP4331120B2 - Defective pixel detection method - Google Patents

Description

  The present invention relates to a defective pixel detection method for detecting a defective pixel based on image data captured by an image sensor.

  In recent years, digital cameras have become popular. This digital camera is provided with an image sensor such as a CCD image sensor or a CMOS image sensor. Digital cameras are desired to improve image quality, and the number of pixels (photoelectric conversion elements) in an image sensor is increasing with the development of semiconductor technology. However, it is known that such an image sensor has a pixel defect that causes image quality degradation due to a local crystal defect of a semiconductor, etc. When the number of pixels of the image sensor increases, an image sensor without a defective pixel is Difficult to manufacture.

  In the defective pixel, since an integral bias voltage is always added to the output corresponding to the amount of incident light, it appears as a bright white point at a fixed position on the screen. Such white spots are generally called white scratches and are caused not only by manufacturing problems but also by factors such as aging.

  In order to prevent such a deterioration in image quality due to defective pixels, a method for correcting the pixel data corresponding to the defective pixels out of the image data captured by the image sensor by previously storing the positions of the defective pixels in the memory. It has been known. However, such a method can prevent deterioration in image quality due to defective pixels generated during manufacturing, but has a problem that it cannot cope with defective pixels generated after manufacturing.

In order to solve such a problem, a defective pixel detection method for detecting a defective pixel while performing normal photographing is known (see, for example, Patent Document 1 and Patent Document 2). In the defective pixel detection method described in Patent Document 1, when it is detected that the inspection target pixel has a frequency component higher than a preset threshold value, the inspection target pixel is determined to be a defective pixel. In addition, in the defective pixel detection method described in Patent Document 2, a difference in pixel data (luminance value) is calculated between the central pixel that is the inspection target pixel and each of the eight pixels around the central pixel, and the difference between these differences is calculated. Is larger than a preset threshold value, the central pixel is determined to be a defective pixel.
JP 2001-86517 A JP-A-6-319082

  However, in the defective pixel detection methods described in Patent Document 1 and Patent Document 2, threshold values are set in advance in order to determine that the pixel is a defective pixel. It is necessary to change according to the number of bits of data to be handled. For this reason, there is a problem that the detection of defective pixels becomes complicated and the detection process takes time.

  The present invention has been made in view of the above circumstances, and an object thereof is to provide a defective pixel detection method capable of reducing the time required for detecting defective pixels.

In order to solve the above problems, a defective pixel detection method of the present invention is a defective pixel detection method for detecting a defective pixel based on image data captured by an image sensor, and is output from a predetermined pixel of the image sensor. Pixel data and a plurality of pixel data output from a plurality of peripheral pixels located around the predetermined pixel are extracted from the image data, and the predetermined pixel and the peripheral pixels are extracted based on the size of each pixel data. grouped into three or more groups, the predetermined pixel, if they belong to the group including pixels having large most pixel data, groups the peripheral pixel belongs, and based on the number of pixels the peripheral pixels included in this group, It is determined whether or not the predetermined pixel is a defective pixel.

  Further, the plurality of groups are three groups of a maximum value group, an average value group, and a minimum value group, and the maximum value, the average value, and the minimum value of the pixel data are calculated. It is preferable that the predetermined pixels and the peripheral pixels are grouped into the maximum value group, the average value group, and the minimum value group.

  Further, after grouping the predetermined pixel and the surrounding pixels into the maximum value group, the average value group, and the minimum value group, an average value of pixel data of each group is calculated, and each pixel data Comparing which average value is closest to each other, the predetermined pixel and the surrounding pixels are grouped again into the maximum value group, the average value group, and the minimum value group, and the change rate of the average value of each group is It is preferable to repeat the grouping of the predetermined pixel and the peripheral pixels until the predetermined value or less is reached.

  Further, when the predetermined pixel belongs to the maximum value group, it is determined whether or not a pixel belonging to the maximum value group is in the peripheral pixel. When it is determined that the predetermined pixel is in the peripheral pixel, the predetermined pixel is defective. It is preferable to determine that the pixel is not a pixel.

  Further, when it is determined that the pixels belonging to the maximum value group are not in the peripheral pixels, the number of pixels belonging to the average value group among the four first peripheral pixels located closest to the predetermined pixel is determined. It is preferable to determine whether or not the predetermined pixel is a defective pixel based on the number of pixels.

  Further, among the four first peripheral pixels, when the number of pixels belonging to the average value group is less than three, the predetermined pixel is determined to be a defective pixel, and the number of pixels belonging to the average value group is four. Sometimes it is determined that the predetermined pixel is not a defective pixel.

  Further, among the four first peripheral pixels, when the number of pixels belonging to the average value group is three, the average of the four second peripheral pixels located at the second closest location to the predetermined pixel. It is preferable to determine whether or not the predetermined pixel is a defective pixel based on whether or not there are a predetermined number or more of pixels belonging to the value group.

  Moreover, it is preferable that the predetermined pixel and the peripheral pixel are pixels of the same color.

According to the defective pixel detection method of the present invention, pixel data output from a predetermined pixel of the image sensor and a plurality of pixel data output from a plurality of peripheral pixels located around the predetermined pixel are extracted, and each pixel data The predetermined pixels and the peripheral pixels are grouped into three or more groups based on the size of the pixel, and when the predetermined pixels belong to a group including a pixel having the largest pixel data, the group to which the peripheral pixels belong, and this group It is determined whether or not the predetermined pixel is a defective pixel based on the number of surrounding pixels included. For this reason, it is not necessary to set a threshold value in order to determine that the pixel is a defective pixel, so that the detection of the defective pixel is simplified and the time required for detecting the defective pixel can be shortened.

  In addition, the maximum value, the average value, and the minimum value of the pixel data are calculated, and each pixel data is compared to which value is closest, and the predetermined pixel and the surrounding pixels are set to the maximum value group, the average value group, and the minimum value group. Therefore, the grouping process can be simplified and the time required for the grouping process can be shortened.

  Furthermore, after grouping into a maximum value group, an average value group, and a minimum value group, the average value of the pixel data of each group is calculated, and each pixel data is compared to which of these average values is closest to a predetermined value. The pixels and the peripheral pixels are grouped again into the maximum value group, the average value group, and the minimum value group, and the grouping of the predetermined pixels and the peripheral pixels is repeatedly performed until the change rate of the average value of each group becomes a predetermined value or less. . For this reason, grouping can be performed accurately, and the accuracy of defective pixel detection can be improved.

  A pixel data correction device 10 shown in FIG. 1 includes a pixel data capturing circuit 11, a defective pixel detection circuit 12, a pixel data correction circuit 13, and an image data output circuit 14. Image data picked up by a CCD image sensor (hereinafter referred to as a CCD) 20 that is an image pickup device is converted from an analog signal to a digital signal by an A / D converter 21. Thereafter, the image data is stored in the memory 22 in the RAW data format.

  The pixel data capturing circuit 11 extracts pixel data (luminance value) of a central pixel that is a predetermined pixel of the CCD 20 and a plurality of peripheral pixels (for example, for example) located around the central pixel from the image data stored in the memory 22. 8 pixel) pixel data is extracted. The pixel data fetch circuit 11 outputs these pixel data to the defective pixel detection circuit 12. The defective pixel detection circuit 12 detects a defective pixel by a defective pixel detection method described later.

  The defective pixel detection circuit 12 outputs pixel data to the pixel data correction circuit 13 when a defective pixel is detected, and outputs the pixel data as it is to the image data output circuit 14 when no defective pixel is detected. The pixel data correction circuit 13 corrects the pixel data of the defective pixel and then outputs the pixel data to the image data output circuit 14. The image data output circuit 14 outputs the image data in which the pixel data of the defective pixel is corrected to an external memory or the like.

  Next, a defective pixel detection method in the defective pixel detection circuit 12 will be described using the flowchart of FIG. The pixel data capture circuit 11 extracts pixel data of 13 pixels (D (0) to D (12)) from the memory 22 for each predetermined number of pixels, for example, as shown in FIG. Note that the number of pixels to be captured may not be 13 pixels. For example, as shown in FIG. 4, while moving a 3 × 3 window 26 in the image 25, 9 pixels included in the window 26, that is, You may capture every nine pixels in total, including the central pixel P and the eight peripheral pixels PA to PH located in the vicinity thereof. Further, it is possible to capture every 25 pixels included in this window while moving a 5 × 5 window.

  The image data capture circuit 11 outputs pixel data of 13 pixels (D (0) to D (12)) to the defective pixel detection circuit 12. The defective pixel detection circuit 12 acquires the pixel data of the 13 pixels (D (0) to D (12)), and obtains the maximum value Max, the average value Ave, and the minimum value Min of these pixel data. In the pixel data shown in FIG. 3, the maximum value Max = 1158, the average value Ave = 144, and the minimum value Min = 5.

  After that, each pixel data is compared with the maximum value Max, the average value Ave, and the minimum value Min that are closest to each other, and 13 pixels (D (0) to D (12)) are compared with the maximum value group and the average value group. , And a minimum value group. Hereinafter, the maximum value group is referred to as class 0, the average value group is referred to as class 1, and the minimum value group is referred to as class 2. By this grouping, 13 pixels (D (0) to D (12)) are classified into classes 0 to 2 as shown in FIG.

  Further, when the defective pixel is determined, the characteristics described below are used. In the case of a point light source, as shown in FIG. 5, there are pixels belonging to class 0 and class 1 around the pixel with the highest pixel data level, and the pixel data level is wide. However, in the case of a defective pixel, as shown in FIG. 6, there is no pixel belonging to class 0 or class 1 around the pixel having the highest pixel data level, and the pixel of the central pixel compared to the peripheral pixels. Data is at a prominent level. That is, based on the group to which the central pixel and its peripheral pixels belong and the number of pixels belonging to each group, it can be determined whether or not the central pixel is a defective pixel.

  First, it is determined whether or not the center pixel P shown in FIG. When it is determined that the central pixel P does not belong to class 0, it is determined that the central pixel P is not a defective pixel. Thereafter, the process proceeds to a process for determining whether or not the detection of the defective pixel is completed for all the pixels of the image data.

  When it is determined that the central pixel P belongs to class 0, it is determined whether or not the pixels belonging to class 0 are eight peripheral pixels P1 and P2 located around the central pixel P. When it is determined that there is a pixel belonging to class 0, it is determined that the central pixel P is not a defective pixel, that is, a point light source. Thereafter, the process proceeds to a process for determining whether or not the detection of the defective pixel is completed.

  If it is determined that there are no pixels belonging to class 0, it is determined whether or not there are three or more pixels belonging to class 1 among the four first peripheral pixels P1 located closest to the center pixel P. . When it is determined that there are not three or more pixels belonging to class 1, that is, two or less, the central pixel P is determined to be a defective pixel, and it is determined whether or not the detection of the defective pixel has ended. Proceed to

  When it is determined that there are three or more pixels belonging to class 1, it is determined whether or not there are three pixels belonging to class 1. If it is determined that the number of pixels belonging to class 1 is not three, that is, four, it is determined that the center pixel P is not a defective pixel, that is, a point light source, and detection of the defective pixel is completed. The process proceeds to the determination process. Further, when it is determined that there are three pixels belonging to class 1, among the four second peripheral pixels P2 that are second closest to the central pixel P, N pixels (for example, three pixels) belonging to class 1 ) Determine whether or not. When there are three or more pixels, it is determined that the central pixel P is not a defective pixel, that is, a point light source, and the process proceeds to determine whether detection of the defective pixel is completed.

  When it is determined that there are not three or more pixels belonging to class 1, that is, two or less, the central pixel P is determined to be a defective pixel. Thereafter, the process proceeds to a process for determining whether or not the detection of the defective pixel is completed. When it is determined that the detection of the defective pixel is not completed, the process returns to the process for acquiring the pixel data again. If it is determined that the determination has been completed, the defective pixel detection process ends.

  In addition, although the case where a defective pixel is determined based on whether there are three or more pixels belonging to class 1 among the four second peripheral pixels P2 is described as an example, the present invention is not limited to this, and one or more Alternatively, the defective pixel may be determined based on whether it is two or more.

  In the above defective pixel detection processing, the group to which the four first peripheral pixels P1 located at the location closest to the central pixel P and the four second peripheral pixels P2 located at the second closest location belong, and Based on the number of pixels belonging to the group, it is determined whether or not the center pixel P is a defective pixel. For this reason, it is necessary to capture pixel data of a total of 9 pixels including the central pixel P and the surrounding 8 pixels at the minimum. However, the number of pixels to be captured is not limited to this. For example, it may be determined by capturing pixel data for 24 pixels around the center pixel P, that is, a total of 25 pixels of 5 × 5. By increasing the number of pixels to be captured in this way, it is possible to improve the accuracy of defective pixel determination.

  In the above-described defective pixel detection processing, whether or not the central pixel P is a defective pixel based on the group to which the central pixel P and the peripheral pixels P1 and P2 belong and the number of pixels belonging to the group. Is judged. Such a determination method is effective because the central pixel P and the peripheral pixels P1 and P2 are pixel data of the same color when the CCD 20 is a 3CCD system (3-plate system).

  However, when the CCD 20 is a single CCD (single plate type), the eight peripheral pixels P1 and P2 include pixels of a color different from that of the central pixel P. Therefore, there is no problem when the point light source is white light. For example, when the point light source is red (R) light, when red light is incident on the central pixel P, other than R around the central pixel P (G, In the pixel B), the level of the pixel data does not spread, and only the pixel data of the center pixel P protrudes. For this reason, it is erroneously determined that the center pixel P is a defective pixel.

  In order to prevent such an erroneous determination, for example, as shown in FIG. 8, when the CCD 20 is a 1 CCD type color CCD employing a Bayer type color filter array, the same color located closest to the center pixel G is used. Each pixel in the order of the first peripheral pixel G1, which is in contact with each corner of the central pixel G, and then the same color pixel located second closest to the central pixel G, that is, the second peripheral pixel G2. Based on the group to which it belongs and the number of pixels included in the group, it may be determined whether or not the center pixel G is a defective pixel. The same operation may be performed when the central pixel is a pixel other than G, that is, an R or B pixel. Further, not only the Bayer system but also a 1 CCD system color CCD employing another color filter arrangement may be used.

  In the description of the defective pixel detection process, the groups are classified into three groups of classes 0 to 2 based on the first captured pixel data. However, the present invention is not limited to this. In the following, another embodiment will be described with reference to the flowchart of FIG.

  The defective pixel detection circuit 12 acquires a predetermined number of pixel data from the pixel data fetch circuit 11, for example, pixel data of 13 pixels (D (0) to D (12)) as shown in FIG. The maximum value Max, the average value Ave, and the minimum value Min of these pixel data are calculated. In the pixel data shown in FIG. 3, the maximum value Max = 1158, the average value Ave = 144, and the minimum value Min = 5.

  Thereafter, each pixel data is compared to the closest value of the maximum value Max, the average value Ave, and the minimum value Min, and 13 pixels (D (0) to D (12)) are classified into class 0, class 1, and Classify into 3 groups of class 2.

  By this grouping, 13 pixels (D (0) to D (12)) are classified into groups of class 0 to 2, as shown in FIG. Further, the average value of the pixel data in each group of classes 0 to 2 is calculated. That is, the average value in class 0 is 1158, the average value in class 1 is 179, and the average value in class 2 is 19.

  After that, each pixel data is compared to which of these average values is closest, and the grouping is performed again into three groups of classes 0 to 2. Further, the average value of the pixel data in each class 0 to 2 is calculated. At this time, it is determined whether or not the change amount of the average value is equal to or less than a predetermined amount (for example, 5% or less) as compared with each previous average value.

  When it is determined that the change amount of the average value is greater than 5%, the process returns to the process of calculating the average value of the pixel data of each class. If the change amount is 5% or less, the grouping process is terminated. Subsequent to the grouping process, that is, the determination of the defective pixel is the same as the above-described defective pixel detection process, and thus detailed description thereof is omitted.

  Next, correction processing of defective pixel data in the image data correction circuit 13 will be described. When the defective pixel detection circuit 12 detects a defective pixel, pixel data is output to the image data correction circuit 13. The pixel data correction circuit 13 calculates the average value of the four first peripheral pixels P1 located closest to the defective pixel, and replaces the pixel data of the defective pixel with this average value.

  For example, as shown in FIG. 10A, when the pixel data of the defective pixel is 1028, and the pixel data of the four first peripheral pixels is 10, 58, 35, 58, the average value of the pixel data of the neighboring pixels (37.75) is calculated. At this time, the decimal point is rounded off and the average value is 40. After that, as shown in FIG. 10B, the pixel data of the defective pixel is replaced with 1028 to 40, and the pixel data of the defective pixel is corrected.

  Further, in a total of five pixels including the defective pixel and the four first peripheral pixels P1, an intermediate value of these pixel data may be obtained, and the pixel data of the defective pixel may be replaced with this intermediate value. That is, when the pixel data of 5 pixels are arranged in ascending order, they are in the order of 10, 35, 58, 58, and 1028, and the intermediate value 58 is obtained. Thereafter, as shown in FIG. 10C, the image data of the defective pixel is replaced with the intermediate value 58 from 1028, and the pixel data of the defective pixel is corrected.

  Further, an intermediate value of these pixel data may be obtained for a total of nine pixels including the defective pixel, the first peripheral pixel P1, and the second peripheral pixel P2, and the pixel data of the defective pixel may be replaced with this intermediate value. That is, as shown in FIG. 11A, when the pixel data of the defective pixel is 1028 and the pixel data of the pixels adjacent to the defective pixel are 3, 10, 12, 30, 35, 35, 58, 58, these The intermediate value in the pixel data of 9 pixels is 35. Thereafter, as shown in FIG. 11B, the pixel data of the defective pixel is corrected by replacing the image data of the defective pixel with the intermediate value 35 from 1028.

  Further, in the 1 CCD type color CCD employing the Bayer type color filter array shown in FIG. 8, when the central pixel G is a defective pixel, an average value is calculated for the four first peripheral pixels G1, and the center pixel G is calculated. The pixel data of the pixel G may be replaced with this average value. In addition, the average value of the pixel data of the eight pixels including the second nearest neighboring pixel G2 may be calculated, and the pixel data of the center pixel G may be replaced with this average value. Further, an intermediate value may be obtained from pixel data of a total of five pixels including the central pixel G and the first peripheral pixel G1, and the pixel data of the central pixel G may be replaced with the intermediate value, or the central pixel G and the first peripheral pixel G1. In addition, an intermediate value may be obtained from the pixel data of a total of nine pixels of the second peripheral pixel G2, and the pixel data of the center pixel G may be replaced with the intermediate value.

  In this embodiment, in order to perform the defective pixel detection process, the central pixel and the peripheral pixels are divided into three groups of a maximum value group, an average value group, and a minimum value group. However, the present invention is not limited to this. For example, the defective pixel detection process may be performed by grouping the central pixel and the peripheral pixels into four or more groups.

  In the present embodiment, the case where the defective pixel detection process and the pixel data correction process are performed by the image data correction apparatus has been described as an example. However, the present invention is not limited to this, and the image data captured by the CCD is taken into the personal computer. Thus, the defective pixel detection process and the image data correction process may be executed by software.

  Furthermore, in the present embodiment, the CCD is described as an example of the image pickup device, but the present invention is not limited to this, and the present invention can also be applied to defective pixel detection processing and pixel data correction processing of a CMOS image sensor.

It is a block diagram which shows the electric constitution of a pixel data correction apparatus. It is a flowchart explaining a defective pixel detection process. It is a data table which shows pixel data. It is explanatory drawing which shows a center pixel and a peripheral pixel. It is explanatory drawing which shows pixel data in case a center pixel is a point light source. It is explanatory drawing which shows pixel data in case a center pixel is a defective pixel. It is explanatory drawing which shows the position of a center pixel, a 1st periphery pixel, and a 2nd periphery pixel. It is explanatory drawing which shows the pixel array of CCD which employ | adopted the Bayer color filter array. It is a flowchart explaining a grouping process. It is explanatory drawing which shows the correction process which correct | amends the pixel data of a defective pixel. It is explanatory drawing which shows the correction process which correct | amends the pixel data of a defective pixel.

Explanation of symbols

DESCRIPTION OF SYMBOLS 10 Image data correction apparatus 11 Pixel data capture circuit 12 Defective pixel detection circuit 13 Pixel data correction circuit 14 Image data output circuit 20 CCD
P, G center pixel PA to PH peripheral pixel P1, G1 first peripheral pixel P2, G2 second peripheral pixel

Claims (8)

In a defective pixel detection method for detecting a defective pixel based on image data captured by an image sensor,
Pixel data output from the predetermined pixels of the image sensor and a plurality of pixel data output from a plurality of peripheral pixels located around the predetermined pixels are extracted from the image data, and based on the size of each pixel data The predetermined pixels and the peripheral pixels are grouped into three or more groups,
If the predetermined pixel belongs to a group including a pixel having the largest pixel data, whether the predetermined pixel is a defective pixel based on the group to which the peripheral pixel belongs and the number of pixels of the peripheral pixel included in the group A defective pixel detection method characterized by determining whether or not.   The plurality of groups are three groups, a maximum value group, an average value group, and a minimum value group. The maximum value, the average value, and the minimum value of the pixel data are calculated, and each pixel data has the highest value. 2. The defective pixel detection method according to claim 1, wherein the predetermined pixels and the peripheral pixels are grouped into the maximum value group, the average value group, and the minimum value group in comparison with each other.   After the predetermined pixel and the surrounding pixels are grouped into the maximum value group, the average value group, and the minimum value group, an average value of pixel data of each group is calculated, and each pixel data is an average of which The predetermined pixels and the peripheral pixels are grouped again into the maximum value group, the average value group, and the minimum value group, and the change rate of the average value of each group is a predetermined value. The defective pixel detection method according to claim 2, wherein grouping of the predetermined pixels and the peripheral pixels is repeatedly performed until the following is reached.   When the predetermined pixel belongs to the maximum value group, it is determined whether or not the pixel belonging to the maximum value group is in the peripheral pixel. When it is determined that the pixel is in the peripheral pixel, the predetermined pixel is not a defective pixel The defective pixel detection method according to claim 2, wherein the defective pixel detection method is determined.   When it is determined that the pixels belonging to the maximum value group are not in the peripheral pixels, the number of pixels belonging to the average value group among the four first peripheral pixels located closest to the predetermined pixel is determined. 5. The defective pixel detection method according to claim 4, wherein whether or not the predetermined pixel is a defective pixel is determined based on the number of pixels.   Among the four first peripheral pixels, when the number of pixels belonging to the average value group is less than three, it is determined that the predetermined pixel is a defective pixel, and when the number of pixels belonging to the average value group is four, 6. The defective pixel detection method according to claim 5, wherein the predetermined pixel is determined not to be a defective pixel.   Among the four first peripheral pixels, when the number of pixels belonging to the average value group is three, the average value group among the four second peripheral pixels located at the second closest position to the predetermined pixel 7. The defective pixel detection method according to claim 5, wherein whether or not the predetermined pixel is a defective pixel is determined based on whether or not there are a predetermined number or more of pixels. The defective pixel detection method according to claim 1, wherein the predetermined pixel and the peripheral pixel are pixels of the same color.

Images