TWI448154B - Method of detecting a bad pixel of a sensed image - Google Patents

Description

Image sensing method for detecting bad pixel points

The present invention relates to an image processing, and more particularly to an image sensing method for detecting bad pixels.

Each pixel of a color image can be represented using a vector containing three color components, such as the intensity of light in the red, green, and blue bands in the visible spectrum. In order to reduce size and cost, in a general image sensor, only one optical filtering unit is used for each pixel to capture a single chromatic value. Then, other color values are obtained by interpolation according to pixel values adjacent to the same color.

The first figure shows a photosensor array 10 of an image sensor covered with a color filter array (CFA) 12. Each color filter (CF) of the color filter array 12 is only capable of sensing one color of light by the corresponding light sensing element. Since the particular mosaic pattern arrangement of the color filter array 12 is not a complete image viewed by the human eye, color interpolation is necessary to reconstruct an image viewable by the human eye. Therefore, color interpolation is also commonly referred to as CFA interpolation, color reconstruction, or demosaicking.

The color filter array 12 illustrated in the first figure is one of Bayer filters. Kind, it is commonly used in digital cameras or cameras. The Bayer filter pattern is arranged by the red (R), green (G), and blue (B) filters as shown in the figure, which contains 50% green (G), 25% red (R), 25%. Blue (B) filter. The other color at each pixel location can be obtained from the surrounding pixels. Even though Bayer pattern interpolation is commonly used in the field of image processing, it has some disadvantages, such as the disappearance of high frequency details due to aliasing phenomena, and even the generation of new frequencies that severely damage the image.

In addition, image sensors typically have defective pixels or bad pixels. The traditional interpolation method cannot effectively distinguish the dead point and the edge image, so it is easy to misjudge the edge image as a bad point or mistake the dead point as an edge image.

Therefore, it is urgent to propose a method for effectively distinguishing between dead pixels and edge images, and to preserve image details when compensating for dead pixels.

In view of the above, one of the objects of the embodiments of the present invention is to provide a method for detecting image dead pixels, which is used to effectively distinguish between dead pixels and edge images, thereby correctly correcting or compensating for dead pixels.

According to an embodiment of the invention, the difference between the pixel value to be detected and the pixel value of the plurality of adjacent positions is examined. Then, the continuity between the pixel value to be detected and the plurality of adjacent color pixel values in a specific direction is examined and a specific direction is obtained. Finally, according to the specific direction obtained, the smoothness of the pixel value to be detected in the specific direction is examined. When the pixel value to be detected is determined to be a dead point, the pixel of the same color pixel value may be adjacent according to the complex point of the bad point to perform the compensation of the dead pixel.

10‧‧‧Light sensor array

12‧‧‧Color Filter Array

21-23‧‧‧Steps

211-215‧‧‧Steps

221-224‧‧‧Steps

231-232‧‧‧Steps

91-92‧‧‧Steps

101-104‧‧‧Steps

Nc‧‧‧ pixels to be detected / dead pixels

N0-N7‧‧‧ pixels

R‧‧‧Red

G‧‧‧Green

B‧‧‧Blue

The first figure shows an array of light sensing elements and a color filter array of an image sensor.

The second figure shows a flow chart of an image sensing bad pixel detection method according to an embodiment of the present invention.

The third to third D-pictures show the pixel value to be detected and the pixel value of the adjacent position.

The fourth figure shows a detailed flow chart of step 21 of the second figure.

The fifth to fifth C charts show the pixel values to be detected and the adjacent color pixel values.

The sixth A diagram and the sixth B diagram show the horizontal direction enhancement weight, the vertical direction enhancement weight, the upper right direction enhancement weight, and the upper left direction enhancement weight.

The seventh diagram shows a detailed flow chart of step 22 of the second figure.

The eighth figure shows a detailed flow chart of step 23 of the second figure.

The ninth figure shows the fill point flow when the dead point is blue or red.

The tenth figure shows the fill point flow when the dead point is green.

The eleventh figure shows the green dead pixel value and the adjacent color pixel value.

The second figure shows a flow chart of an image sensing bad pixel detection method according to an embodiment of the present invention. This embodiment is applicable to various image sensors such as a complementary metal oxide semiconductor (CMOS) image sensor. In this embodiment, the sensed image values of the respective pixels are obtained via a Bayer pattern color filter array. If a bad pixel ("bad point") is detected, the bad point can be further corrected or compensated. The corrected dead pixel values can be stored, displayed or further image processed after being output.

In step 21, the difference between the pixel value to be detected and the pixel value of the adjacent location is examined, Wherein, the pixel adjacent to the location may be in a different color from the pixel to be detected. In this embodiment, if the pixel to be detected is taken as the center, it will form a 3×3 array with the adjacent position pixel, as shown in the third A (blue) to be detected pixel value Nc and green (G), Red (R) adjacent position pixel values N0-N7.

The fourth figure shows a detailed flow chart of step 21. First, in step 211, a difference between the average value of the pixel value Nc to be detected and the vertical neighboring pixel values N1, N6 is obtained as the vertical neighboring metric value BM1. Wherein, if the difference is less than 0, the vertical neighboring metric value BM1 is 0. This step 211 can be expressed as follows: BM1=Nc-(N1+N6)/2 if BM1<0 then BM1=0.

In step 212, a difference between the average value of the pixel value to be detected Nc and the horizontal neighboring position pixel values N3, N4 is obtained as the horizontal neighboring metric value BM2. Wherein, if the difference is less than 0, the horizontal neighboring metric value BM2 is 0. This step 212 can be expressed as follows: BM2=Nc-(N3+N4)/2 if BM2<0 then BM2=0.

In step 213, the difference between the average value of the pixel values Nc to be detected and the four corner neighboring position pixel values N0, N2, N5, and N7 is obtained as the corner neighboring metric value BM3. Wherein, if the difference is less than 0, the corner neighboring metric value BM3 is 0. This step 213 can be expressed as follows: BM3=Nc-(N0+N2+N5+N7)/4 if BM3<0 then BM3=0.

Next, in step 214, the integrated neighboring metric value BM4 is obtained according to the vertical neighboring metric value BM1, the horizontal neighboring metric value BM2, and the corner neighboring metric value BM3. Wherein, if the value is greater than the pixel value to be detected Nc, the integrated neighboring metric value BM4 is equal to the pixel value to be detected Nc. This step 214 can be expressed as follows: BM4 = (BM1 + BM2) / 2 + BM3 if BM4 > Nc then BM4 = Nc.

Finally, in step 215, determining the integrated neighboring metric value BM4 and the pixel to be detected Whether the absolute difference of the value Nc is less than the neighboring metric threshold B_TH. If the determination is yes, the pixel to be detected may be a dead pixel. This step 215 can be expressed as follows: if ABS(BM4-Nc)<B_TH then Nc may be a dead point.

The above steps 211-215 are described with the pixel value Nc to be detected as blue (B). However, the steps are equally applicable to the case where the pixel value Nc to be detected is red (R), as shown in FIG. The red (R) pixel value to be detected Nc and the green (G), blue (B) adjacent position pixel values N0-N7. The only difference is that BM1-BM4 is replaced by RM1-RM4, and B_TH is replaced by R_TH.

When the pixel value to be detected Nc is green (G), as shown in the third C diagram or the third D diagram, wherein the green to-be-detected pixel value Nc in the third C diagram is adjacent to the blue level, and the third The green to-be-detected pixel value Nc in the D picture is adjacent to the red level. For the case shown in the third C picture, BM1-BM4 needs to be replaced by GM1-GM4, B_TH is replaced by GB_TH, and step 214 can be expressed as follows: GM4=GM1+GM2+GM3 if GM4>Nc then GM4=Nc.

For the case shown in the third D picture, BM1-BM4 needs to be replaced by GM1-GM4, B_TH is replaced by GR_TH, and step 214 can be expressed as follows: GM4=GM1+GM2+GM3 if GM4>Nc then GM4=Nc.

Next, returning to the flow shown in the second figure, in step 22, the continuity of the pixel value to be detected and the adjacent color pixel value in a specific direction is examined. This step 22 can reduce or avoid the misjudgment caused by the previous step 21. For example, if there is an oblique bright line through the pixel value to be detected Nc, after the previous step 21 is examined, the pixel value Nc to be detected may be misjudged as a bad point. After performing step 22, if the pixel value to be detected Nc is discontinuous, It may be a bad point, otherwise it is not a bad point. In this embodiment, if the pixel to be detected is taken as the center, it will form a 3×3 array with adjacent pixels of the same color, as shown in FIG. 5A, the blue (B) pixel value to be detected Nc and the adjacent color pixel value D0. -N7.

In the present embodiment, the specific direction refers to the horizontal direction (H), the vertical direction (V), the upper right direction (NE), and the upper left direction (NW). In order to view the characteristics of a particular direction, the present embodiment uses enhanced weights to enhance its pixel values for the desired viewing direction. The sixth graph A shows the horizontal direction enhancement weight (B_H), the vertical direction enhancement weight (B_V), the upper right direction enhancement weight (B_NE), and the upper left direction enhancement weight (B_NW), respectively.

The seventh diagram shows a detailed flow chart of step 22. First, in step 221, B_H is used to weight each row of pixel values, and the absolute sum value is taken. This step 221 can be expressed as follows: BH_1=ABS[(-1)*D0+2*D3+(-1)*D5]; BH_2=ABS[(-1)*D1+2*Nc+(-1)*D6] ;BH_3=ABS[(-1)*D2+2*D4+(-1)*D7].

Next, in step 222, the absolute sum value of each row is weighted using B_V, and the absolute sum value is taken as the continuous metric value B_H in the horizontal direction. As for the continuous metric B_V in the vertical direction, the continuous metric B_NE in the upper right direction, and the continuous metric B_NW in the upper left direction, it can be obtained by a similar principle. This step 222 can be expressed as follows: B_H=ABS[(-1)*BH_1+2*BH_2+(-1)*BH_3].

Next, in step 223, the maximum or minimum value among the continuous metric B_H in the horizontal direction, the continuous metric B_V in the vertical direction, the continuous metric B_NE in the upper right direction, and the continuous metric B_NW in the upper left direction is obtained as a comprehensive continuous metric. The value B (X, Y).

Finally, in step 224, it is determined whether the integrated continuous metric value B(X, Y) is greater than continuous Measure the threshold B_Line_TH. If the determination is YES, the pixel to be detected has discontinuity. Steps 223 and 224 can be expressed as follows: B(X, Y)_max=max(B_H, B_V, B_NE, B_NW); B(X, Y)_min=min(B_H, B_V, B_NE, B_NW); B(X, Y)=select_max_min? B(X,Y)_max: B(X,Y)_min; If B(X,Y)>B_Line_TH then Nc has discontinuity; wherein the variable select_max_min is used to select B(X,Y)_max or B( X, Y)_min. If you select B(X,Y)_min, you can generally retain more image details, but it is easier to misjudge.

The above steps 221-224 are described with the pixel value Nc to be detected as blue (B). However, the steps are equally applicable to the case where the pixel value Nc to be detected is red (R), as shown in FIG. The red (R) pixel value to be detected Nc and the adjacent color pixel value D0-N7. The only difference is that the B in each variable is replaced by R.

Although the fifth A picture is also applicable to the case where the pixel value Nc to be detected is green (G), however, since the number of green pixel points is twice the number of blue or red pixel points, the fifth C picture can be used. The green (G) to be detected pixel value Nc and the adjacent color pixel values D0-N7 are shown. The horizontal direction enhancement weight (B_H), the vertical direction enhancement weight (B_V), the upper right direction enhancement weight (B_NE), and the upper left direction enhancement weight (B_NW) shown in FIG. 6A are also applicable to the pixel value to be detected Nc being green ( B). Further, the horizontal direction enhancement weight (G_H), the vertical direction enhancement weight (G_V), the upper right direction enhancement weight (G_NE), and the upper left direction enhancement weight (G_NW) shown in FIG. 6B may be used.

Next, return to the flow shown in the second figure, in step 23, according to the previous step In a specific direction obtained in step 22, the smoothness of the pixel value to be detected in the specific direction is examined. After performing this step 23, if the pixel value to be detected Nc is non-smooth, it can be determined as a bad point, otherwise it is not a bad point.

The eighth diagram shows a detailed flow chart of step 23. First, in step 231, an edge value for each specific direction is obtained. As described above, the specific direction of the present embodiment refers to the horizontal direction (H), the vertical direction (V), the upper right direction (NE), and the upper left direction (NW). When the pixel value to be detected Nc is blue (B), as shown in FIG. 5A, the edge values in each direction can be obtained by the following equations: V=[ABS(Nc-D1)+ABS(Nc-D6)] /2;H=[ABS(Nc-D3)+ABS(Nc-D4)]/2; NE=[ABS(Nc-D2)+ABS(Nc-D5)]/2; NW=[ABS(Nc- D0)+ABS(Nc-D7)]/2.

Next, according to the specific direction obtained in step 22, in step 232, it is determined whether the edge value of the direction is greater than a critical value. In this embodiment, it is determined whether half of the edge value edge_value is greater than the edge threshold B_edge_value_th. If the determination is YES, the pixel to be detected has non-smoothness. Steps 231-232 can be expressed as follows: if B(X,Y)=B_H then edge_value=H; else if B(X,Y)=B_V thenedge_value=V;else if B(X,Y)=B_NE thenedge_value=NE; Else edge_value=NW; if edge_value/2>B_edge_value_th then Nc is non-smooth.

The above steps 231-232 are described with the pixel value Nc to be detected as blue (B). However, the steps are equally applicable to the case where the pixel value Nc to be detected is red (R), as in the first case. Figure 5B shows. The only difference is that the B in each variable is replaced by R. The above steps 231-232 are also applicable to the case where the pixel value Nc to be detected is green (G), as shown in FIG. 5C. The only difference is that the B in each variable is replaced by G.

According to the flow shown in the second figure above, if the various determinations of steps 21 to 23 can be met, it can be determined that the pixel value Nc to be detected is a bad point. When it is judged to be a dead point, the pixel value of the dead pixel can be corrected or compensated (referred to as "complement point"). In this embodiment, the compensation of the dead pixels is performed according to the adjacent color pixel values of the dead pixels.

The ninth figure shows the fill point flow when the dead point Nc is blue (as shown in Figure 5A) or red (as shown in Figure 5B). First, in step 91, eight adjacent color-matching pixel values D0-D7 are sorted to obtain a maximum value. Next, in step 92, according to the maximum value, five adjacent color-matching pixel values far from the maximum value are weighted and averaged together with the pixel value Nc of the dead pixels to obtain a compensation value of the dead pixels. For example, when D0 is the maximum value, the values of five adjacent color pixels are D2, D5, D4, D6, and D7; when D1 is the maximum value, the values of five adjacent color pixels are D3, D4, and D5. D6, D7. In one embodiment, the weight of the dead pixels is 3/8, and the weights of the five adjacent pixels of the same color are both 1/8. In another embodiment, the weight of the dead pixels is 1/16, and the weights of the five adjacent pixels of the same color are both 3/16.

The tenth graph shows the fill point flow when the dead point Nc is green. The eleventh figure shows the green (G) dead pixel value Nc and the adjacent color pixel values N0-N7. First, in step 101, the same color pixel values D0-D3 closest to the dead point Nc are sorted to obtain a sorting minimum value min1_g, which can be expressed as follows: min1_g=sorting(D0-D3).

In step 102, the same color pixel values D0-D3 closest to the four dead pixels Nc are entered. Row operation to get the nearest pixel average avg_g, which can be expressed as follows: avg_g=avg(D0-D3).

In step 103, the same color pixel values D0-D3 closest to the dead point Nc are obtained together with the median of the dead pixels Nc, and the same color pixel values D4-D7 near the bad point Nc are obtained. The average median of points Nc. Next, the average median_g of the two average medians is obtained.

Finally, in step 104, according to the partial statistical values obtained in the above steps 101-103, the compensation value of the dead point is obtained. In an embodiment, the compensation value of the dead point is equal to the average of both the average median_g of the average median and the minimum value of the order min1_g, that is, avg (media_g, min1_g). In another embodiment, the compensation value of the dead pixel is equal to the average of both the nearest pixel average value avg_g and the sort minimum value min1_g, that is, avg(avg_g, min1_g). In yet another embodiment, the compensation value of the dead point is equal to the ranking minimum min1_g; or equal to the average median_g of the average median; or equal to the nearest pixel average avg_g.

The above description is only the preferred embodiment of the present invention, and is not intended to limit the scope of the present invention; all other equivalent changes or modifications which are not departing from the spirit of the invention should be included in the following Within the scope of the patent application.

21-23‧‧‧Steps

Claims (12)

A method for detecting a bad pixel point of an image, comprising: checking a difference between a pixel value to be detected and a pixel value of a plurality of adjacent positions; and checking continuity of the pixel value to be detected and a plurality of adjacent color pixels in a specific direction and obtaining a continuity a specific direction; and, according to the specific direction obtained, the smoothness of the pixel value to be detected in the specific direction; wherein the step of viewing the continuity includes: using an enhancement weight to enhance the pixel value of the specific direction to be inspected; The specific direction described above includes a horizontal direction, a vertical direction, an upper right direction, and an upper left direction. The method for detecting an image of a bad pixel in the image of claim 1, wherein at least a portion of the adjacent pixel is in a different color from the pixel to be detected. The method for detecting an image of a bad pixel according to claim 1, wherein the pixel to be detected is centered, and forms a 3×3 array with the pixel at the adjacent position. The method for detecting an image of a bad pixel according to the first aspect of the invention, wherein the step of detecting the difference comprises: obtaining a difference between an average value of the pixel value to be detected and a pixel value of the vertical adjacent position, as a a vertical neighboring metric value; obtaining a difference between the pixel value to be detected and an average value of the horizontal neighboring pixel values as a horizontal neighboring metric value; obtaining an average value of the pixel value to be detected and the pixel value of the four corner neighboring positions The difference, as a corner neighbor metric; Obtaining a comprehensive neighboring metric value according to the vertical neighboring metric value, the horizontal neighboring metric value, and the corner neighboring metric value; and determining whether the absolute difference between the integrated neighboring metric value and the to-be-detected pixel value is less than a neighboring metric threshold The value, if the determination is yes, the pixel to be detected may be a bad point. The method for detecting a bad pixel point of the image according to claim 4, further comprising: if the difference between the average value of the pixel value to be detected and the pixel value of the vertical adjacent position is less than 0, the vertical proximity metric is obtained The value is 0; if the difference between the pixel value to be detected and the average value of the horizontal neighboring pixel values is less than 0, the horizontal neighboring metric value is 0; if the pixel value to be detected is adjacent to the pixel in the four corners The difference between the average values of the values is less than 0, so that the corner neighboring metric value is 0; and if the integrated neighboring metric value is greater than the to-be-detected pixel value, the integrated neighboring metric value is made equal to the to-be-detected pixel value. The method for detecting an image of a bad pixel according to claim 1, wherein the pixel to be detected is centered, and forms a 3×3 array with the adjacent pixel of the same color. The method for detecting an image of a bad pixel according to the first aspect of the patent application, wherein the step of checking the continuity comprises: weighting each row of pixel values, and taking an absolute sum; and the absolute sum of each row After the values are weighted, the absolute sum value is obtained, and a continuous metric value in the horizontal direction, a continuous metric value in the vertical direction, a continuous metric value in the upper right direction, and a continuous metric value in the upper left direction are obtained; the continuous direction in the horizontal direction is obtained. Metric value, continuous measure of the vertical direction, the upper right a continuous measure of the direction and a maximum or minimum value of the continuous measure in the upper left direction as a comprehensive continuous measure; and determining whether the integrated continuous measure is greater than a continuous measure threshold, and if the determination is yes, The pixel to be detected has discontinuity. The method for detecting an image of a bad pixel according to the first aspect of the invention, wherein the step of detecting the smoothness comprises: obtaining an edge value of each specific direction; and determining whether an edge value of the specific direction is greater than A threshold value, if the determination is YES, the pixel to be detected has non-smoothness. The method for detecting a bad pixel point of the image according to the first aspect of the invention, wherein the pixel value to be detected is different from the pixel value of the adjacent position, and the pixel value to be detected and the pixel value of the adjacent color are in a specific direction. If the pixel value to be detected is non-smooth in the specific direction, it is determined that the pixel value to be detected is a bad point. The method for detecting a bad pixel point of the image according to claim 9 further includes: compensating for a dead pixel according to a plurality of adjacent color pixel values of the dead point. The method for detecting a bad pixel point of the image according to claim 10, wherein when the dead point is blue or red, the step of complementing comprises: sorting the plurality of adjacent pixels of the same color to obtain a maximum sorting. a value; and according to the sorted maximum value, the complex adjacent color pixel values that are farther from the sorted maximum value are weighted and averaged together with the pixel value of the bad point to obtain a compensation value of the bad point. The method for detecting a bad pixel point of the image according to claim 10, wherein when the dead point is green, the step of complementing comprises: sorting the plurality of same-color pixel values closest to the bad point to obtain a Sorting the minimum value; calculating the same-color pixel value closest to the bad point to obtain a nearest pixel average value; obtaining a complex same-color pixel value closest to the bad point together with an average median value of the bad point (median), And obtaining a plurality of same-color pixel values that are next to the bad point together with an average median value of the bad points, and then obtaining an average of the two average median values; and according to the sorted minimum value, the nearest pixel average value, and the two averages The portion of the average of the median values to obtain the compensation value for the bad point.