JP2008131158A - Defective pixel detecting device, electronic apparatus, and defective pixel detecting program - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide a defective pixel detecting device capable of detecting a defective pixel even when a pixel of interest is a red component pixel or blue component pixel and the pixel of interest is present on the border of an image. <P>SOLUTION: The defective pixel detecting device includes: a first decision section which decides whether the pixel of interest is a defective pixel on the basis of pixel data of red (blue) color component pixel nearby the pixel of interest when the pixel of interest of image data of a Bayer array is the red (blue) color component pixel; a second decision section which decides whether the pixel of interest is present on the density border of a component of the same color with the pixel of interest, and the density border of a component of the same color with the pixel of interest and the density border of a green component are correlative; and a defective pixel detection section which detects the pixel of interest as a defective pixel when the first decision section does not make a negative decision or when the second decision section does not make a negative decision. <P>COPYRIGHT: (C)2008,JPO&INPIT

Description

  The present invention relates to a defective pixel detection apparatus that detects defective pixels in Bayer array image data, an electronic apparatus including the defective pixel detection apparatus, and a defective pixel detection program for causing a computer to function to detect defective pixels in Bayer array image data. Is.

An example of a conventional technique related to a defective pixel detection method for detecting defective pixels in Bayer array image data will be described below. If the target pixel is a red component pixel, defective pixel detection is performed using a 5 × 5 pixel block centered on the target pixel R33 shown in FIG. When the signal level of the target pixel R33 is extremely high with respect to all the signal levels of the four red component pixels R31, R53, R35, and R13 that are present closest to the target pixel R33, or four that are closest to the target pixel R33. When the signal level of all the red component pixels R31, R53, R35, and R13 is extremely low, the target pixel R33 is detected as a defective pixel.
JP 2005-123946 A

  However, in the above defective pixel detection method, as shown in FIG. 4, the target pixel R33 is on the boundary of the image (color and / or luminance boundary), and the signal level of the pixel R13 and the signal level of the pixels R31, R53, and R35. If there is a large difference, the determination is made that the target pixel R33 is not a defective pixel regardless of the value of the signal level of the target pixel R33. It cannot be detected. When the boundary of the image is in the vertical direction or the horizontal direction and a defective pixel is present on the boundary, the defective pixel becomes a conspicuous scratch, and thus it is necessary to prevent the detection omission as described above.

  Even when the target pixel is a blue component pixel, the same problem occurs because a defective pixel is detected by the same method.

  Another example of the related art relating to a defective pixel detection method for detecting defective pixels in Bayer array image data is a defective pixel detection method disclosed in Patent Document 1. The defective pixel detection method disclosed in Patent Document 1 uses a variance value of signal levels of peripheral pixels of the same color component as the target pixel to reduce erroneous detection of defective pixels, but the target pixel is red. When the pixel of interest is a component pixel or a blue component pixel and the target pixel is on the boundary of the image, the variance value is high, so that even if the target pixel on the boundary of the image is a defective pixel, it is detected There was a risk of not being able to.

  In view of the above situation, the present invention includes a defective pixel detection device capable of detecting a defective pixel even when the target pixel is a red component pixel or a blue component pixel and the target pixel is on the boundary of the image, and the same An object is to provide an electronic device and a defective pixel detection program.

  In order to achieve the above object, a defective pixel detection device according to the present invention is a defective pixel detection device that detects defective pixels in Bayer array image data, and the pixel of interest is a red component pixel or a blue component pixel A first determination unit that determines whether the pixel of interest is a defective pixel based on pixel data of a pixel having the same color component as the pixel of interest belonging to a pixel block that includes the pixel of interest and includes a predetermined number of pixels When the first determination unit determines that the target pixel is not a defective pixel, the target pixel is on a density boundary of the same color component as the target pixel, and the density boundary of the same color component as the target pixel is A second determination unit that determines whether or not there is a correlation with the density boundary of the green component, and when the first determination unit determines that the pixel of interest is a defective pixel, or by the second determination unit If the target pixel is on the same color component density boundary as the target pixel and it is determined that there is a correlation between the same color component density boundary and the green component density boundary as the target pixel, the target pixel is defective. The configuration includes a first detection unit that detects a pixel and a second detection unit that detects a defective pixel when the pixel of interest is a green component pixel.

  According to such a configuration, when the target pixel is on the density boundary of the same color component as the target pixel and there is a correlation between the density boundary of the same color component and the density boundary of the green component, the target pixel is determined as a defective pixel. Since detection is performed, a defective pixel can be detected even when the target pixel is a red component pixel or a blue component pixel and the target pixel is on the boundary of the image.

  In order to achieve the above object, a defective pixel detection device according to the present invention is a defective pixel detection device that detects defective pixels in Bayer array image data, and the pixel of interest is a red component pixel or a blue component pixel A first determination unit that determines whether the pixel of interest is a defective pixel based on pixel data of a pixel having the same color component as the pixel of interest belonging to a pixel block that includes the pixel of interest and includes a predetermined number of pixels When the first determination unit determines that the target pixel is not a defective pixel, the target pixel is on a density boundary of the same color component as the target pixel, and the density boundary of the same color component as the target pixel is A second determination unit that determines whether or not there is a correlation with a density boundary of a green component; and the second determination unit determines that the target pixel is on a density boundary of the same color component as the target pixel, and And the density boundary of the same color component and the density boundary of the green component are determined to be correlated with the target pixel belonging to the pixel block and having the same color component as the target pixel. A third determination unit that determines whether or not the target pixel is a defective pixel based only on pixel data of pixels belonging to the same side, and the target pixel is a defective pixel by the first determination unit or the third determination unit A first detection unit that detects the pixel of interest as a defective pixel and a second detection unit that detects a defective pixel when the pixel of interest is a green component pixel. .

  According to such a configuration, when the target pixel is on the density boundary of the same color component as the target pixel and there is a correlation between the density boundary of the same color component and the density boundary of the green component, the target pixel belonging to the pixel block The pixel of interest is a red component pixel because it is determined whether or not the pixel of interest is a defective pixel based only on pixel data of a pixel having the same color component as the pixel and belonging to the same side as the pixel of interest relative to the density boundary Alternatively, even when the pixel is a blue component and the target pixel is on the boundary of the image, the defective pixel can be detected.

  In order to achieve the above object, an electronic apparatus according to the present invention includes a defective pixel detection device having any one of the above configurations, and a correction unit that corrects pixel data of a defective pixel detected by the defective pixel detection device. It is set as the structure provided with the image processing apparatus which has. Examples of the electronic device according to the present invention include a mobile phone device with a camera, a digital still camera, a digital video camera, and the like.

  In order to achieve the above object, a defective pixel detection program according to the present invention is a defective pixel detection program for detecting defective pixels in image data of a Bayer array, wherein the target pixel is a red component pixel. Or, in the case of a blue component pixel, whether or not the target pixel is a defective pixel based on pixel data of a pixel having the same color component as the target pixel belonging to the pixel block including the target pixel and having a predetermined number of pixels And when the target pixel is determined not to be a defective pixel by the first determination unit, the target pixel is on a density boundary of the same color component as the target pixel, and the target pixel Second determination means for determining whether or not there is a correlation between the density boundary of the same color component and the density boundary of the green component; and the target pixel is a defective pixel by the first determination means. Or when the target pixel is on the density boundary of the same color component as the target pixel and the density boundary of the same color component and the density boundary of the green component are correlated with the target pixel. In the case where it is determined that the pixel of interest is a defective pixel, it functions as a first detector that detects the defective pixel, and a second detector that detects a defective pixel when the pixel of interest is a green component pixel.

  In order to achieve the above object, a defective pixel detection program according to the present invention is a defective pixel detection program for detecting defective pixels in image data of a Bayer array, wherein the target pixel is a red component pixel. Or, in the case of a blue component pixel, whether or not the target pixel is a defective pixel based on pixel data of a pixel having the same color component as the target pixel belonging to the pixel block including the target pixel and having a predetermined number of pixels And when the target pixel is determined not to be a defective pixel by the first determination unit, the target pixel is on a density boundary of the same color component as the target pixel, and the target pixel The second determination means for determining whether or not there is a correlation between the density boundary of the same color component and the density boundary of the green component, and the second determination means determines that the target pixel is the target image Of the same color component and the density boundary of the same color component as that of the target pixel and the density boundary of the green component are determined to be correlated with each other. Third determination means for determining whether or not the target pixel is a defective pixel based only on pixel data of a pixel belonging to the same side as the target pixel with respect to the density boundary, the first determination unit Alternatively, when the third determination unit determines that the target pixel is a defective pixel, a first detection unit that detects the target pixel as a defective pixel, and a defective pixel when the target pixel is a green component pixel It is made to function as the 2nd detection means to detect.

  According to the present invention, a defective pixel can be detected even when the target pixel is a red component pixel or a blue component pixel and the target pixel is on the boundary of the image.

  Embodiments of the present invention will be described below with reference to the drawings. Here, a camera-equipped mobile phone device will be described as an example of the electronic apparatus according to the present invention.

  FIG. 1 shows a schematic configuration example of a camera-equipped mobile phone device according to the present invention. A camera configured by an optical lens system 1, a solid-state imaging device 2 such as a CCD having a Bayer color filter, and an A / D conversion unit 3 that converts an analog signal output from the solid-state imaging device 2 into a digital signal. The module 4 outputs RAW format image data, which is digital image data in a Bayer array, to an image processing device 5 which is a semiconductor integrated circuit device.

  In the image processing device 5, the frame memory 6 temporarily stores RAW image data output from the camera module 4, and the defective pixel detection device 7 detects defective pixels in the RAW image data to detect defective pixels. The correction unit 8 detects the pixel of the defective pixel by correction processing such as replacing the pixel data of the pixel having the same color component as that of the defective pixel with data obtained by arithmetic processing. The data is corrected, and the image processing unit 9 creates RGB image data by performing various kinds of image processing such as color interpolation on the RAW image data in which the pixel data of the defective pixel is corrected.

  When the preview mode is set by the CPU 11, the image processing device 5 outputs the RGB image data created by the image processing unit 9 to the display device 12 such as a liquid crystal display. Thereby, the image captured by the camera module 4 is displayed on the display device 12. When image recording is instructed by the CPU 11, the image processing apparatus 5 converts the RGB image data created by the image processing unit 9 into compressed image data by compression processing in the data compression unit 10, and then a memory card or the like. The compressed image data is output to the storage medium interface 13 to which the external storage medium is detachably mounted, and the compressed image data is written to the external storage medium. Thereby, the compressed data of the image captured by the camera module 4 is recorded in the external storage medium.

  In addition, the camera-equipped mobile phone device according to the present invention shown in FIG. 1 includes an antenna 14, a radio unit 15 that transmits and receives to / from a base station via the antenna 14, a notification unit 16 that notifies incoming calls by sound or vibration, and a key. An input unit 17, a voice input unit 18 that inputs voice converted by an operator during a voice call and converts it into an electrical signal, and a voice output unit that converts an electrical signal corresponding to the voice of the other party into voice during voice call and outputs it 19. The display device 12 also displays a telephone number and the like under the control of the CPU 11.

  Next, the operation of the defective pixel detection device 7 will be described in detail. When the target pixel is a red component pixel, the defective pixel detection apparatus 7 detects the defective pixel according to the procedure of the flowchart shown in FIG. 2A using a 5 × 5 pixel block centered on the target pixel R33 shown in FIG. Do.

In step S10, the defective pixel detection device 7 determines whether or not the target pixel R33 is a defective pixel according to the defective pixel determination using the red component pixel in the 5 × 5 pixel block. As a specific example of such determination, the signal level L _{R33 of the} target pixel _R33 and the signal levels L _R31 , L _R53 , and L _{R35 of} the four red component pixels R31, R53, R35, and R13 that are closest to the target pixel R33. , L _R13 and the difference Δ1 to Δ4 are obtained, and if all the differences Δ1 to Δ4 are smaller than a preset first threshold value (negative value), the target pixel R33 is a defective pixel (more specifically, a white defective pixel). And determining that the target pixel R33 is a defective pixel (more specifically, a black defective pixel) if all of the differences Δ1 to Δ4 are greater than a preset second threshold value (positive value). Is mentioned. The differences Δ1 to Δ4 are obtained by the following equations (1) to (4).
Δ1 = L _R31 −L _R33 (1)
Δ2 = L _R53 −L _R33 (2)
Δ3 = L _R35 -L _R33 (3)
Δ4 = L _R13 −L _R33 (4)

  When it is determined in step S10 that the target pixel R33 is a defective pixel (YES in step S10), the process proceeds to step S40 described later.

  On the other hand, when it is not determined in step S10 that the target pixel R33 is a defective pixel (NO in step S10), the defective pixel detection device 7 determines whether the target pixel R33 may be a defective pixel on the boundary of the image. It is determined whether or not (step S20). Specifically, in step S20, the defective pixel detection device 7 determines whether or not the target pixel R33 is on the density boundary of the red component. As a specific example of such determination, when the signal level of three of the four red component pixels R31, R53, R35, R13 existing closest to the target pixel R33 is low and the remaining one signal level is high, or the target pixel When the signal level of three of the four red component pixels R31, R53, R35, and R13 closest to R33 is high and the remaining one is low, the target pixel R33 is a defective pixel on the boundary of the image. It is determined that there is a possibility. For example, if the threshold value Rth for determining the density boundary of the red component is set in advance and three of the differences Δ1 to Δ4 are smaller than the threshold value Rth and the remaining one is larger than the threshold value Rth, the four closest to the target pixel R33 are present. This corresponds to the case where three of the red component pixels R31, R53, R35, and R13 are low and the remaining one is high, and three of the differences Δ1 to Δ4 are larger than the threshold value Rth. If one is smaller than the threshold value Rth, this corresponds to the case where three signal levels are high among the four red component pixels R31, R53, R35, R13 existing closest to the target pixel R33 and the remaining one signal level is low. And it is sufficient.

  When it is determined in step S20 that the target pixel R33 is not a defective pixel on the boundary of the image (NO in step S20), the process proceeds to step S50 described later.

  On the other hand, when it is determined in step S20 that the target pixel R33 is likely to be a defective pixel on the boundary of the image (YES in step S20), the defective pixel detection device 7 determines that the density boundary of the green component and the red component It is determined whether or not there is a correlation with the density boundary (step S30). Here, when the green component shading pattern of the area divided by the density boundary of the green component and the red component shading pattern of the area divided by the density boundary of the red component are the same (for example, pixels R31, R53, R35 in FIG. 4). The signal level of the pixel R13 is low, the signal level of the pixels G32, G43, and G34 is high and the signal level of the pixel G23 is low), and there is a correlation between the density boundary of the green component and the density boundary of the red component It is determined that there is. As a specific example of such determination, the red component density pattern of the region divided by the density boundary of the red component is recognized based on the magnitude relationship between each of the above differences Δ1 to Δ4 and the threshold threshold value Rth for determining the density boundary of the red component. Then, the difference between all the patterns of the pixels G32, G43, G34, and G23 is obtained, and is divided by the density boundary of the green component according to the magnitude relationship between each difference and a preset green component density boundary discrimination threshold Gth. The green component density pattern in the image area, and based on the comparison result between the red component density pattern and the green component density pattern, determine whether there is a correlation between the density boundary of the green component and the density boundary of the red component .

  If it is determined in step S30 that there is a correlation between the density boundary of the green component and the density boundary of the red component (YES in step S30), the process proceeds to step S40 described later. On the other hand, if it is determined in step S30 that there is no correlation between the density boundary of the green component and the density boundary of the red component (NO in step S30), the process proceeds to step S50 described later.

  The defective pixel detection device 7 detects the target pixel R33 as a defective pixel in step S40, recognizes the target pixel R33 as a normal pixel in step S50, and ends the flow when the process of step S40 or step S50 ends.

  The defective pixel detection device 7 may perform defective pixel detection according to the procedure of the flowchart shown in FIG. 2B using a 5 × 5 pixel block centered on the target pixel R33 shown in FIG. 2B, the same steps as those in FIG. 2A are denoted by the same reference numerals, and detailed description thereof is omitted.

  The flowchart shown in FIG. 2B is obtained by adding step S35 to the flowchart shown in FIG. 2A. In the flowchart shown in FIG. 2B, when it is determined in step S30 that there is a correlation between the density boundary of the green component and the density boundary of the red component (YES in step S30), the defective pixel detection device 7 has 5 × 5 pixels. According to the determination of a defective pixel using only a red component pixel in the pixel block and a pixel belonging to the same side as the pixel R33 with respect to the density boundary of the red component, it is determined whether or not the target pixel R33 is a defective pixel. To do. As a specific example of such determination, only the pixels belonging to the same side as the pixel R33 with respect to the density boundary of the red component among the four red component pixels R31, R53, R35, R13 existing closest to the target pixel R33 are selected. If all the differences between the signal level of those pixels and the signal level of the target pixel R33 are smaller than a preset first threshold value (negative value), the target pixel R33 is a defective pixel (more specifically, A white defective pixel), and if it is larger than a preset second threshold value (positive value), the target pixel R33 is determined to be a defective pixel (more specifically, a black defective pixel). It is done.

  If it is determined in step S35 that the target pixel R33 is a defective pixel (YES in step S35), the process proceeds to step S40, and if it is not determined in step S35 that the target pixel R33 is a defective pixel (in step S35). NO) moves to step S50.

  When the pixel of interest is a blue component pixel, the defective pixel detection device 7 performs the same operation as when the pixel of interest described above is a red component pixel (replaces the red component pixel with a blue component pixel in the above-described operation). Thus, defective pixel detection may be performed.

When the target pixel is a green component pixel, the defective pixel detection device 7 uses the green component pixel in the pixel block of 5 × 5 pixels centered on the target pixel G33 shown in FIG. It is determined whether G33 is a defective pixel. As a specific example of such determination, the signal level L _{G33 of the} target pixel _G33 and the signal levels L _G22 , L _G42 , and L _{G24 of} the four green component pixels G22, G42, G24, and G44 that are closest to the target pixel G33. , L _G44 and the difference Δ5 to Δ8 are obtained, and if all the differences Δ5 to Δ8 are smaller than a preset third threshold value (negative value), the target pixel G33 is a defective pixel (more specifically, a white defective pixel). And if all the differences Δ5 to Δ8 are greater than a preset fourth threshold value (positive value), the target pixel G33 is determined to be a defective pixel (more specifically, a black defective pixel). Is mentioned. The differences Δ5 to Δ8 are obtained by the following equations (5) to (8).
Δ5 = L _G22 -L _G33 (5)
Δ6 = L _G42 −L _G33 (6)
Δ7 = L _G24 -L _G33 (7)
Δ8 = L _G44 -L _G33 (8)

  As another embodiment of the present invention, a computer inputs RAW format image data that is digital image data of a Bayer array output from a digital still camera, and the computer inputs the image processing device 5 in FIG. Embodiment which performs the same operation | movement is mentioned. In this case, the computer performs the same operation as the defective pixel detection apparatus 7 in FIG. 1 by the defective pixel detection program according to the present invention.

These are figures which show the example of schematic structure of the mobile telephone apparatus with a camera which concerns on this invention. These are the flowcharts which show the operation example of the defective pixel detection apparatus with which the mobile telephone apparatus with a camera shown in FIG. 1 is provided. These are the flowchart figures which show the other operation example of the defective pixel detection apparatus with which the mobile telephone apparatus with a camera shown in FIG. These are diagrams showing a pixel block of 5 × 5 pixels centering on a target pixel which is a red component pixel. FIG. 4 is a diagram illustrating a state in which a target pixel is on an image boundary in the pixel block illustrated in FIG. 3. FIG. 5 is a diagram illustrating a pixel block of 5 × 5 pixels centered on a target pixel which is a green component pixel.

Explanation of symbols

DESCRIPTION OF SYMBOLS 1 Optical lens system 2 Solid-state image sensor 3 A / D conversion part 4 Camera module 5 Image processing apparatus 6 Frame memory 7 Defective pixel detection apparatus 8 Correction part 9 Image processing part 10 Data compression part 11 CPU
DESCRIPTION OF SYMBOLS 12 Display apparatus 13 Storage medium interface 14 Antenna 15 Radio | wireless part 16 Notification | reporting part 17 Key input part 18 Voice input part 19 Voice output part

Claims (5)

A defective pixel detection device for detecting defective pixels in Bayer array image data,
When the pixel of interest is a red component pixel or a blue component pixel, the pixel of interest is defective based on pixel data of a pixel having the same color component as the pixel of interest belonging to the pixel block including the pixel of interest and having a predetermined number of pixels. A first determination unit for determining whether or not the pixel;
When the first determination unit determines that the target pixel is not a defective pixel, the target pixel is on the density boundary of the same color component as the target pixel, and the density boundary and green component of the same color component as the target pixel A second determination unit that determines whether or not there is a correlation with the density boundary;
When the first determination unit determines that the target pixel is a defective pixel, or the second determination unit determines that the target pixel is on the density boundary of the same color component as the target pixel and has the same color as the target pixel. A first detection unit that detects the target pixel as a defective pixel when it is determined that there is a correlation between the density boundary of the component and the density boundary of the green component;
A defective pixel detection apparatus comprising: a second detection unit configured to detect a defective pixel when the target pixel is a green component pixel. A defective pixel detection device for detecting defective pixels in Bayer array image data,
When the pixel of interest is a red component pixel or a blue component pixel, the pixel of interest is defective based on pixel data of a pixel having the same color component as the pixel of interest belonging to the pixel block including the pixel of interest and having a predetermined number of pixels. A first determination unit for determining whether or not the pixel;
When the first determination unit determines that the target pixel is not a defective pixel, the target pixel is on the density boundary of the same color component as the target pixel, and the density boundary and green component of the same color component as the target pixel A second determination unit that determines whether or not there is a correlation with the density boundary;
When the second determination unit determines that the target pixel is on the same color component density boundary as that of the target pixel and that there is a correlation between the target pixel and the same color component density boundary and the green component density boundary In addition, whether or not the target pixel is a defective pixel based only on pixel data of a pixel having the same color component as the target pixel belonging to the pixel block and belonging to the same side as the target pixel with respect to the density boundary A third determination unit for determining whether or not
A first detection unit that detects the pixel of interest as a defective pixel when the pixel of interest is determined to be a defective pixel by the first determination unit or the third determination unit;
A defective pixel detection apparatus comprising: a second detection unit configured to detect a defective pixel when the target pixel is a green component pixel.   An electronic apparatus comprising: the defective pixel detection device according to claim 1 or 2; and an image processing device including a correction unit that corrects pixel data of the defective pixel detected by the defective pixel detection device. . A defective pixel detection program for detecting defective pixels in Bayer array image data,
Computer
When the pixel of interest is a red component pixel or a blue component pixel, the pixel of interest is defective based on pixel data of a pixel having the same color component as the pixel of interest belonging to the pixel block including the pixel of interest and having a predetermined number of pixels. First determination means for determining whether or not the pixel;
When the first determination unit determines that the target pixel is not a defective pixel, the target pixel is on a density boundary of the same color component as the target pixel, and the density boundary and green component of the same color component as the target pixel Second determination means for determining whether there is a correlation with the density boundary of
When the first determination unit determines that the target pixel is a defective pixel, or when the second determination unit determines that the target pixel is on the density boundary of the same color component as the target pixel, the same color component as the target pixel A first detection means for detecting the pixel of interest as a defective pixel when it is determined that there is a correlation between the density boundary of the green component and the density boundary of the green component; and a defective pixel when the pixel of interest is a green component pixel A defective pixel detection program for functioning as second detection means for detection. A defective pixel detection program for detecting defective pixels in Bayer array image data,
Computer
When the pixel of interest is a red component pixel or a blue component pixel, the pixel of interest is defective based on pixel data of a pixel having the same color component as the pixel of interest belonging to the pixel block including the pixel of interest and having a predetermined number of pixels. First determination means for determining whether or not the pixel;
When the first determination unit determines that the target pixel is not a defective pixel, the target pixel is on a density boundary of the same color component as the target pixel, and the density boundary and green component of the same color component as the target pixel Second determination means for determining whether there is a correlation with the density boundary of
When the second determination unit determines that the target pixel is on the same color component density boundary as the target pixel and that there is a correlation between the target pixel and the same color component density boundary and the green component density boundary In addition, whether or not the target pixel is a defective pixel based only on pixel data of a pixel having the same color component as the target pixel belonging to the pixel block and belonging to the same side as the target pixel with respect to the density boundary Third determination means for determining
First detection means for detecting the target pixel as a defective pixel when the first determination unit or the third determination unit determines that the target pixel is a defective pixel; and the target pixel is a green component pixel A defective pixel detection program for functioning as second detection means for detecting defective pixels in some cases.

Images