JP2012248938A - Image processor - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide an image processor for obtaining an image of high quality by appropriately correcting a flaw.SOLUTION: In the embodiment, a pixel extraction section 21 respectively extracts signal values of a plurality of same color peripheral pixels and signal values of a plurality of different color peripheral pixels. A flaw determining section 22 determines whether a target pixel has the flaw or not by comparing the signal value of a target pixel with one of the signal values of the plurality of the same color peripheral pixels. A white flaw correction release determining section 23 and a black flaw correction release determining section 24, which are the flaw correction release determining sections, determine whether the flaw correction is to be released or not on the target pixel which is determined to have the flaw by the flaw determining section 22. The flaw correction release determining section determines the flaw correction release in accordance with a comparison result of the signal value of the first different color peripheral pixel and the signal value of the second different color peripheral pixel. The first different color peripheral pixel positions in a first range with the target pixel as a center, and the second different color peripheral pixel positions in a second range outside the first range.

Description

  Embodiments described herein relate generally to an image processing apparatus.

  In recent years, in a solid-state imaging device, pixel miniaturization has progressed due to an increase in the number of pixels. Under such circumstances, the solid-state imaging device has been regarded as a problem of the occurrence of a missing portion of the digital image signal (hereinafter referred to as “scratch” as appropriate) due to pixels that do not function normally. If more than the specified number of pixel defects are found in the defect inspection at the time of manufacturing the solid-state imaging device, it will be treated as a defective product. Will be increased. Therefore, conventionally, a technique for making a scratch inconspicuous by signal processing in a scratch correction circuit has been positively employed in a solid-state imaging device.

  The flaw correction circuit determines whether or not the target pixel is flawed by comparing the signal level between the target pixel to be flaw-corrected and the peripheral pixels located around the target pixel. Conventionally, a defect correction that can deal with not only a case where the target pixel is a scratch among the target pixel and a plurality of peripheral pixels but also a so-called two-pixel scratch in which the target pixel and one of the peripheral pixels are scratches. Circuits are also known. Strict defect determination level to enhance the effect of defect correction makes it possible to perform defect correction on many pixels, and easily causes resolution reduction due to erroneous correction.

JP 2004-134941 A

  An object of one embodiment of the present invention is to provide an image processing apparatus capable of obtaining a high-quality image by appropriate defect correction.

  According to one embodiment of the present invention, the image processing apparatus includes a scratch correction unit. The defect correcting unit performs defect correction on the target pixel. The scratch correction unit includes a pixel extraction unit, a scratch determination unit, and a scratch correction cancellation determination unit. The pixel extraction unit extracts signal values of a plurality of peripheral pixels of the same color and signal values of a plurality of peripheral pixels of different colors. The same color peripheral pixel is a pixel for detecting the same color light as the color light detected by the target pixel. Peripheral pixels of the same color are located around the target pixel. The different color peripheral pixel is a pixel for detecting color light different from the target pixel. The different color peripheral pixels are located around the target pixel. The flaw determination unit performs flaw determination as to whether or not the target pixel is flawed by comparing the signal value of the target pixel with the signal values of a plurality of surrounding pixels of the same color. The scratch correction cancellation determination unit performs a scratch correction cancellation determination as to whether or not to cancel the scratch correction for the target pixel determined as a scratch by the scratch determination unit. The defect correction cancellation determination unit performs the defect correction cancellation determination according to the comparison result between the signal value of the first different color peripheral pixel and the signal value of the second different color peripheral pixel. The first different-color peripheral pixels are located in a first range centered on the target pixel. The second different-color peripheral pixels are located in the second range outside the first range.

1 is a block diagram illustrating a configuration of a digital camera to which an image processing apparatus according to an embodiment is applied. The block diagram which shows the structure of a defect correction circuit. The figure explaining an object pixel, the same color surrounding pixel, and a different color surrounding pixel. The figure explaining the procedure of the signal processing for the flaw correction cancellation determination in a white flaw correction cancellation determination part. The figure explaining the combination of the different color surrounding pixel referred in the defect correction cancellation | release determination. The figure explaining the combination of the different color surrounding pixel referred in the defect correction cancellation | release determination. The figure explaining the procedure of the signal processing for the crack correction cancellation determination in a black crack correction cancellation determination part.

  Hereinafter, an image processing apparatus according to an embodiment will be described in detail with reference to the accompanying drawings. In addition, this invention is not limited by this embodiment.

(Embodiment)
FIG. 1 is a block diagram illustrating a configuration of a digital camera to which the image processing apparatus according to the embodiment is applied. The digital camera 1 includes a solid-state imaging device 2, a storage unit 3, and a display unit 4. The solid-state imaging device 2 captures a subject image. The storage unit 3 stores an image captured by the solid-state imaging device 2. The display unit 4 displays an image captured by the solid-state imaging device 2. The display unit 4 is, for example, a liquid crystal display.

  The solid-state imaging device 2 outputs an image signal to the storage unit 3 and the display unit 4 by capturing a subject image. The storage unit 3 outputs an image signal to the display unit 4 in accordance with a user operation or the like. The display unit 4 displays an image according to the image signal input from the solid-state imaging device 2 or the storage unit 3.

  The solid-state imaging device 2 includes a lens unit 5, a CMOS (complementary metal oxide semiconductor) sensor 6, and a digital signal processor (DSP) 7.

  The lens unit 5 takes in light from the subject and forms a subject image with the CMOS sensor 6. The CMOS sensor 6 converts the light captured by the lens unit 5 into a signal charge, and captures a subject image. The CMOS sensor 6 takes in the signal values of red (R), green (G), and blue (B) in the order corresponding to the Bayer array, thereby generating an image signal.

  The DSP 7 that is an image processing device performs various image processing on the image signal from the CMOS sensor 6. The DSP 7 includes, for example, a white balance adjustment circuit for adjusting the white balance of the image signal, in addition to a scratch correction circuit described later.

  FIG. 2 is a block diagram showing the configuration of the defect correction circuit. The line memory 10 and the scratch correction circuit 11 are provided in the DSP 7 (see FIG. 1). The line memory 10 holds four lines of image signals. The line memory 10 outputs a total of 5 lines (0H to 4H) of image signals including the retained 4 lines and one main line. The defect correction circuit (defect correction unit) 11 performs defect correction on the target pixel.

  FIG. 3 is a diagram illustrating the target pixel, the same color peripheral pixel, and the different color peripheral pixel. The Bayer array is configured in units of four pixels of Gr, R, Gb, and B. The R pixel is a pixel for detecting R light. The B pixel is a pixel for detecting B light. The Gr pixel and the Gb pixel are pixels for detecting G light. The Gr pixel is parallel to the R pixel in a line. Gb pixels are parallel to B pixels in a line. The image signal is input to the defect correction circuit 11 as a signal for each line (Gr / R line, Gb / B line).

  The defect correction circuit 11 sets the center pixel among the 25 pixels forming a matrix of 5 lines (0H to 4H) in the vertical direction and 5 pixels in the horizontal direction as the target pixel. The same color peripheral pixels are pixels for detecting the same color light as the color light detected by the target pixel in the matrix. The different color peripheral pixels are pixels for detecting a predetermined color light different from the color light detected by the target pixel in the matrix. The same color peripheral pixel and the different color peripheral pixel are located around the target pixel.

  In the illustrated example, the R pixel R22 is the target pixel, and the eight R pixels R00, R02, R04, R20, R24, R40, R42, and R44 are the same color peripheral pixels. Further, twelve G pixels Gr01, Gr03, Gb10, Gb12, Gb14, Gr21, Gr23, Gb30, Gb32, Gb34, Gr41, and Gr43 are different color peripheral pixels. The matrix includes one target pixel, eight same-color peripheral pixels, and twelve different-color peripheral pixels.

  The scratch correction circuit 11 includes a pixel extraction unit 21, a scratch determination unit 22, a white scratch correction cancellation determination unit 23, a black scratch correction cancellation determination unit 24, a white scratch correction cancellation unit 25, a black scratch correction cancellation unit 26, and an output pixel selection unit. 27.

  The pixel extraction unit 21 synchronizes signal values of 25 pixels forming a matrix from image signals of 5 lines (0H to 4H). The pixel extraction unit 21 extracts the signal value 31 of the target pixel and eight surrounding pixels of the same color from the signal values of 25 pixels, and outputs them to the scratch determination unit 22. The pixel extraction unit 21 extracts signal values 32 of 12 different-color peripheral pixels from the signal values of 25 pixels, and outputs them to the white defect correction cancellation determination unit 23 and the black defect correction cancellation determination unit 24.

  The scratch determination unit 22 compares the signal value of the target pixel with the signal values of eight peripheral pixels of the same color, and determines whether the target pixel is white or not, and the target pixel is black. Whether or not black scratches are determined. White scratches are scratches in which higher luminance is detected than when the pixel functions normally. Black scratches are scratches in which lower luminance is detected than when the pixel functions normally. The scratch determination unit 22 outputs a white scratch flag 33 when the white pixel is determined to indicate that the target pixel is white and a black scratch flag 34 when the black pixel is determined to indicate that the target pixel is black. .

  For example, when the signal value of the target pixel is larger than the maximum value of the signal values of the eight neighboring pixels of the same color, the scratch determination unit 22 determines that the target pixel is white. For example, when the signal value of the target pixel is smaller than the minimum value of the signal values of eight peripheral pixels of the same color, the scratch determination unit 22 determines that the target pixel is black. The scratch determination unit 22 may perform white scratch determination and black scratch determination by any method.

  The scratch determination unit 22 selects and outputs the white scratch replacement signal value 35 and the black scratch replacement signal value 36 from the signal values of the eight peripheral pixels of the same color. The white scratch replacement signal value 35 is a signal value used for replacement with the signal value of the target pixel when the scratch determination unit 22 determines that the target pixel is a white scratch. The black scratch replacement signal value 36 is a signal value used for replacement with the signal value of the target pixel when the scratch determination unit 22 determines that the target pixel is a black scratch.

  The white defect replacement signal value 35 is, for example, the maximum value of the signal values of eight peripheral pixels of the same color. The black defect replacement signal value 36 is, for example, the minimum value of the signal values of eight peripheral pixels of the same color. The white scratch replacement signal value 35 and the black scratch replacement signal value 36 may be any of the signal values of eight peripheral pixels of the same color, and at least one of the signal values of eight peripheral pixels of the same color is used. It may be calculated in this way.

  The white scratch correction cancellation determination unit 23 performs a scratch correction cancellation determination on whether or not to cancel the scratch correction for the target pixel determined as a white scratch by the scratch determination unit 22. The white defect correction cancellation determination unit 23 determines whether or not the target pixel is included in a high-luminance portion of the subject image, for example, a part of a white line or a white point by comparing the signal values of the different color peripheral pixels. Determine. When it is determined that the target pixel is included in the high luminance portion of the subject image, the white defect correction cancellation determination unit 23 determines to cancel the defect correction. The white flaw correction cancellation determination unit 23 outputs a white flaw correction cancellation flag 37 based on the determination to cancel the flaw correction.

  The black scratch correction cancellation determination unit 24 performs a scratch correction cancellation determination as to whether or not to cancel the scratch correction for the target pixel determined as a black scratch by the scratch determination unit 22. The black defect correction cancellation determination unit 24 determines whether or not the target pixel is included in a low-luminance part of the subject image, for example, a part of a black line or a black point by comparing the signal values of the different color peripheral pixels. Determine. When it is determined that the target pixel is included in the low luminance portion of the subject image, the black defect correction cancellation determination unit 24 determines to cancel the defect correction. The black scratch correction cancellation determination unit 24 outputs a black scratch correction cancellation flag 38 based on the determination to cancel the scratch correction.

  The white scratch correction cancellation unit 25 is a case where the white scratch flag 33 from the scratch determination unit 22 and the white scratch correction cancellation flag 37 from the white scratch correction cancellation determination unit 23 are input. When the scratch correction cancellation is valid, the white scratch correction is canceled by the mask of the white scratch flag 33. When the white scratch flag 33 is input from the scratch determination unit 22 and the white scratch correction cancellation flag 37 is not input from the white scratch correction cancellation determination unit 23, the white scratch correction cancellation unit 25 uses the white scratch flag 33 as it is. Output. The white defect correction cancellation unit 25 outputs the white defect flag 33 as it is even when the defect correction cancellation in the defect correction cancellation command 44 is invalid.

  The black scratch correction cancellation unit 26 is a case where the black scratch flag 34 from the scratch determination unit 22 and the black scratch correction cancellation flag 38 from the black scratch correction cancellation determination unit 24 are input. When the scratch correction cancellation is valid, the black scratch correction is canceled by the mask of the black scratch flag 34. If the black scratch flag 34 is input from the scratch determination unit 22 and the black scratch correction cancellation flag 38 is not input from the black scratch correction cancellation determination unit 24, the black scratch correction cancellation unit 26 uses the black scratch flag 34 as it is. Output. The black defect correction cancellation unit 26 outputs the black defect flag 34 as it is even when the defect correction cancellation in the defect correction cancellation command 44 is invalid.

  The output pixel selection unit 27 selects a signal value to be applied to the target pixel from the signal value 40 read from the line memory 10, the white defect replacement signal value 35, and the black defect replacement signal value 36. The signal value 40 is the signal value of the target pixel before defect correction. The output pixel selection unit 27 selects the white flaw replacement signal value 35 when the white flaw flag 33 is input from the white flaw correction cancellation unit 25.

  The output pixel selection unit 27 selects the black defect replacement signal value 36 when the black defect flag 34 is input from the black defect correction cancellation unit 26. The output pixel selection unit 27 selects the signal value 40 when neither the white scratch flag 33 nor the black scratch flag 34 is input. The defect correction circuit 11 uses the signal value selected by the output pixel selection unit 27 as the output signal 39.

  Here, the defect correction cancellation determination by the white defect correction cancellation determination unit 23 will be described. FIG. 4 is a diagram for explaining a procedure of signal processing for determining a defect correction cancellation in the white defect correction cancellation determination unit. 5 and 6 are diagrams illustrating combinations of different-color peripheral pixels referred to in the defect correction cancellation determination.

  The white defect correction cancellation determination unit 23 is a predetermined first different color periphery among pixels parallel to the horizontal direction that is the first direction and the vertical direction that is the second direction perpendicular to the first direction. An operation using the signal value of the pixel and the signal value of the peripheral pixel of the second different color is performed.

  The first different color peripheral pixel is a pixel located in a first range close to the target pixel among the twelve different color peripheral pixels. The second different-color peripheral pixel is a pixel located in a second range away from the target pixel among the twelve different-color peripheral pixels. The first range is a range included in 3 × 3 pixels centered on the target pixel. The second range is a range outside the first range among 5 × 5 pixels centered on the target pixel. The first different color peripheral pixel is close to the target pixel. The second different color peripheral pixel is located at a position away from the target pixel in the range of 5 × 5 pixels.

  The white defect correction cancellation determination unit 23 obtains the signal value of the first different-color peripheral pixel and the signal value of the second different-color peripheral pixel in the diagonally downward direction (hereinafter referred to as the first diagonal direction) in FIG. Compare. As shown in FIG. 5 by surrounding with a broken line, the white flaw correction cancellation determination unit 23 includes first different color neighboring pixels (Gr21 and Gr21) adjacent to each other in a diagonally upward direction (hereinafter referred to as a second diagonal direction). For Gb12, Gb32 and Gr23), the sum of signal values is obtained. Further, as shown by surrounding with a broken line in FIG. 5, the white flaw correction cancellation determination unit 23, for the second different color neighboring pixels (Gb10 and Gr01, Gr43 and Gb34) adjacent to each other in the second oblique direction, respectively. Calculate the sum of signal values.

The white defect correction cancellation determination unit 23 compares the signal values of the first different-color peripheral pixels Gr21 and Gb12 with the signal values of the second different-color peripheral pixels Gb10 and Gr01 in the first oblique direction. Here, the white defect correction cancellation determination unit 23 performs the following for the sum of the signal values of the first different color peripheral pixels Gr21 and Gb12 (Gr21 + Gb12) and the sum of the signal values of the second different color peripheral pixels Gb10 and Gr01 (Gb10 + Gr01): It is determined whether or not equation (1) is satisfied.
(Gr21 + Gb12)> (Gb10 + Gr01) + WCL (1)

Further, the white defect correction cancellation determination unit 23 compares the signal values of the first different-color peripheral pixels Gb32 and Gr23 with the signal values of the second different-color peripheral pixels Gr43 and Gb34 in the first oblique direction. Here, the white defect correction cancellation determination unit 23 performs the following for the sum of the signal values of the first different color peripheral pixels Gb32 and Gr23 (Gb32 + Gr23) and the sum of the signal values of the second different color peripheral pixels Gr43 and Gb34 (Gr43 + Gb34). It is determined whether or not Expression (2) is satisfied.
(Gb32 + Gr23)> (Gr43 + Gb34) + WCL (2)

  It is assumed that WCL is a white scratch correction cancellation determination level that is used as a reference for determining scratch correction cancellation. When the expressions (1) and (2) are both established, that is, when the difference between the signal value of the first different color peripheral pixel and the signal value of the second different color peripheral pixel is greater than WCL, the white defect correction cancellation determination unit 23 Then, it is determined that the target pixel is included in a high luminance portion of the subject image.

  The white defect correction cancellation determination unit 23 calculates a WCL corresponding to the MP by multiplying the fixed value preset in the white defect correction cancellation determination command 41 by MP. As the WCL is increased, the white defect correction cancellation determination unit 23 reduces the number of cases in which the equations (1) and (2) are satisfied, and decreases the frequency of the white defect correction cancellation. As the WCL is set to a larger value, the scratch correction circuit 11 can reduce the determination of the scratch correction cancellation and enable more active white defect correction.

  For example, MP is a coefficient linked to the analog gain 43 in capturing the subject image. When the shooting environment is low illuminance, scratches due to thermal noise or the like are likely to occur. Therefore, in many cases, it is more effective to emphasize the aggressiveness of the scratch correction than the influence of the erroneous correction of the scratch. The scratch correction circuit 11 can increase the scratch correction effect by making the MP smaller as the analog gain 43 is larger, by performing more strict white scratch correction cancellation determination.

  MP is not limited to a coefficient linked to the analog gain 43, and may be linked to, for example, the temperature of the CMOS sensor 6, the light amount accumulation time in the CMOS sensor 6, and the like. The higher the temperature of the CMOS sensor 6, the smaller the MP. In addition, the longer the accumulated amount of light in the CMOS sensor 6, the smaller the MP. Also in this case, the scratch correction circuit 11 can perform more strict scratch correction cancellation determination under a situation in which scratches due to thermal noise or the like are likely to occur, and can enhance the effect of the scratch correction.

  The white defect correction cancellation determination unit 23 compares the signal value of the first different-color peripheral pixel with the signal value of the second different-color peripheral pixel in the second oblique direction. The white defect correction cancellation determination unit 23, as surrounded by a broken line in FIG. 6, outputs signal values for the first different color neighboring pixels (Gr23 and Gb12, Gb32 and Gr21) adjacent to each other in the first oblique direction. Find the sum of Further, as shown in FIG. 6 by enclosing with a broken line, the white defect correction cancellation determination unit 23 performs the second different color neighboring pixels (Gb14 and Gr03, Gr41 and Gb30) adjacent to each other in the first oblique direction, respectively. Calculate the sum of signal values.

The white defect correction cancellation determination unit 23 compares the signal values of the first different-color peripheral pixels Gr23 and Gb12 with the signal values of the second different-color peripheral pixels Gb14 and Gr03 in the second oblique direction. Here, the white defect correction cancellation determination unit 23 performs the following for the sum of the signal values of the first different color peripheral pixels Gr23 and Gb12 (Gr23 + Gb12) and the sum of the signal values of the second different color peripheral pixels Gb14 and Gr03 (Gb14 + Gr03): It is determined whether or not Expression (3) is satisfied.
(Gr23 + Gb12)> (Gb14 + Gr03) + WCL (3)

In addition, the white defect correction cancellation determination unit 23 compares the signal values of the first different-color peripheral pixels Gb32 and Gr21 with the signal values of the second different-color peripheral pixels Gr41 and Gb30 in the second oblique direction. Here, the white defect correction cancellation determination unit 23 performs the following for the sum of the signal values of the first different color peripheral pixels Gb32 and Gr21 (Gb32 + Gr21) and the sum of the signal values of the second different color peripheral pixels Gr41 and Gb30 (Gr41 + Gb30): It is determined whether or not Expression (4) is satisfied.
(Gb32 + Gr21)> (Gr41 + Gb30) + WCL (4)

  WCL is set to a white defect correction cancellation determination level that has been multiplied by MP, as in the cases of equations (1) and (2). When Expressions (3) and (4) are both established, that is, when the difference between the signal value of the first different color peripheral pixel and the signal value of the second different color peripheral pixel is greater than WCL, the white defect correction cancellation determination unit 23 Then, it is determined that the target pixel is included in a high luminance portion of the subject image.

  The white defect correction cancellation determination unit 23 compares the signal value of the first different-color peripheral pixel and the signal value of the second different-color peripheral pixel in the first oblique direction (expressions (1) and (2)), Scratch correction cancellation determination is performed according to the comparison results (Equations (3) and (4)) comparing the signal values of the first different color peripheral pixels and the second different color peripheral pixels in the second diagonal direction.

  Here, the condition (5) is when the expressions (1) and (2) are both satisfied, and the condition (6) is when both the expressions (3) and (4) are satisfied. The white defect correction cancellation determination unit 23 performs a white defect correction cancellation selection command when both of the condition (5) and the condition (6) are satisfied and when either of the condition (5) and the condition (6) is satisfied. According to 45, the white defect correction cancellation flag 37 is selected and output.

  The case where the condition (5) and the condition (6) are satisfied is a case where the target pixel is included in, for example, a dot-like high luminance portion. The case where the condition (5) or the condition (6) is satisfied is a case where the target pixel is included in, for example, a linear high luminance portion. For example, the condition (5) can be satisfied when the high-luminance portion extends in the second oblique direction including the target pixel or in a direction close to the second oblique direction. Further, the condition (6) can be satisfied when the high-luminance portion extends in the first oblique direction or the direction close to the first oblique direction including the target pixel.

  The defect correction circuit 11 can suppress erroneous correction when the target pixel is included in the high-luminance portion of the subject image by performing the defect correction cancellation determination by the white defect correction cancellation determination unit 23.

  Next, the scratch correction cancellation determination by the black scratch correction cancellation determination unit 24 will be described. FIG. 7 is a diagram for explaining a signal processing procedure for determining a defect correction cancellation in the black defect correction cancellation determination unit. Similar to the white defect correction cancellation determination unit 23, the black defect correction cancellation determination unit 24 calculates the sum of the signal values of the first different color peripheral pixels and the sum of the signal values of the second different color peripheral pixels.

The black defect correction cancellation determination unit 24 compares the signal values of the first different-color peripheral pixels Gr21 and Gb12 with the signal values of the second different-color peripheral pixels Gb10 and Gr01 in the first oblique direction. Here, the black defect correction cancellation determination unit 24 performs the following for the sum of the signal values of the first different-color peripheral pixels Gr21 and Gb12 (Gr21 + Gb12) and the sum of the signal values of the second different-color peripheral pixels Gb10 and Gr01 (Gb10 + Gr01): It is determined whether or not the following equation (7) is satisfied.
(Gr21 + Gb12) <(Gb10 + Gr01) −BCL (7)

Further, the black defect correction cancellation determination unit 24 compares the signal values of the first different color peripheral pixels Gb32 and Gr23 with the signal values of the second different color peripheral pixels Gr43 and Gb34 in the first oblique direction. Here, the black defect correction cancellation determination unit 24 performs the following for the sum of the signal values of the first different color peripheral pixels Gb32 and Gr23 (Gb32 + Gr23) and the sum of the signal values of the second different color peripheral pixels Gr43 and Gb34 (Gr43 + Gb34): It is determined whether or not Expression (8) is satisfied.
(Gb32 + Gr23) <(Gr43 + Gb34) −BCL (8)

  BCL is assumed to be a black defect correction cancellation determination level used as a criterion for defect correction cancellation determination. When Expressions (7) and (8) are both established, that is, when the difference between the signal value of the first different color peripheral pixel and the signal value of the second different color peripheral pixel is larger than BCL, the black defect correction cancellation determination unit 24 Then, it is determined that the target pixel is included in the low luminance portion of the subject image.

  The black defect correction cancellation determination unit 24 calculates BCL by multiplying the fixed value preset in the black defect correction cancellation determination command 42 by MP. As the BCL is increased, the black defect correction cancellation determination unit 24 reduces the number of cases where the expressions (7) and (8) are satisfied, and decreases the frequency of the black defect correction cancellation. As the BCL is set to a larger value, the defect correction circuit 11 can reduce the determination of the defect correction cancellation and can perform more active black defect correction. MP is a coefficient linked to the analog gain 43, the temperature of the CMOS sensor 6, the accumulation time of the light amount in the CMOS sensor 6, and the like, as in the case of white defect correction.

The black defect correction cancellation determination unit 24 compares the signal values of the first different-color peripheral pixels Gr23 and Gb12 with the signal values of the second different-color peripheral pixels Gb14 and Gr03 in the second oblique direction. Here, the black defect correction cancellation determination unit 24 performs the following for the sum of the signal values of the first different-color peripheral pixels Gr23 and Gb12 (Gr23 + Gb12) and the sum of the signal values of the second different-color peripheral pixels Gb14 and Gr03 (Gb14 + Gr03): It is determined whether or not Expression (9) is satisfied.
(Gr23 + Gb12) <(Gb14 + Gr03) −BCL (9)

Further, the black defect correction cancellation determination unit 24 compares the signal values of the first different color peripheral pixels Gb32 and Gr21 with the signal values of the second different color peripheral pixels Gr41 and Gb30 in the second oblique direction. Here, the black defect correction cancellation determination unit 24 performs the following with respect to the sum of the signal values of the first different color peripheral pixels Gb32 and Gr21 (Gb32 + Gr21) and the sum of the signal values of the second different color peripheral pixels Gr41 and Gb30 (Gr41 + Gb30). It is determined whether or not Expression (10) is satisfied.
(Gb32 + Gr21) <(Gr41 + Gb30) −BCL (10)

  BCL is set to a black defect correction cancellation determination level that has been multiplied by MP as in the cases of Expressions (7) and (8). When the equations (9) and (10) are both established, that is, when the difference between the signal value of the first different color peripheral pixel and the signal value of the second different color peripheral pixel is larger than BCL, the black defect correction cancellation determination unit 24 Then, it is determined that the target pixel is included in the low luminance portion of the subject image.

  The black defect correction cancellation determination unit 24 compares the signal value of the first different color peripheral pixel and the signal value of the second different color peripheral pixel in the first oblique direction (expressions (7) and (8)), Scratch correction cancellation determination is performed according to the comparison results (Equations (9) and (10)) comparing the signal values of the first different color peripheral pixels and the second different color peripheral pixels in the second diagonal direction.

  Here, the condition (11) is satisfied when both the expressions (7) and (8) are satisfied, and the condition (12) is satisfied when both the expressions (9) and (10) are satisfied. The black scratch correction cancellation determination unit 24 performs a black scratch correction cancellation selection command when both of the condition (11) and the condition (12) are satisfied and when one of the condition (11) and the condition (12) is satisfied. 46, the black defect correction cancellation flag 38 is selected and output.

  The case where the condition (11) and the condition (12) are satisfied is a case where the target pixel is included in, for example, a dot-like low luminance portion. The case where the condition (11) or the condition (12) is satisfied is a case where the target pixel is included in, for example, a linear low luminance portion. For example, the condition (11) can be satisfied when the low-luminance portion extends in the second oblique direction including the target pixel or in a direction close to the second oblique direction. Further, when the low-luminance portion extends in the first oblique direction including the target pixel or in a direction close to the first oblique direction, the condition (12) can be satisfied.

  The defect correction circuit 11 can suppress erroneous correction when the target pixel is included in the low-luminance portion of the subject image by performing the defect correction cancellation determination by the black defect correction cancellation determination unit 24. The DSP 7 can obtain a high-quality image by suppressing erroneous correction of defects in the defect correction circuit 11 and enabling appropriate defect correction.

  When the target pixel is the B pixel, the defect correction circuit 11 performs the defect correction cancellation determination using the Gr pixel and the Gb pixel as different color peripheral pixels as in the case where the target pixel is the R pixel. In addition, when the target pixel is any one of the Gr pixel and the Gb pixel, the defect correction circuit 11 performs the defect correction cancellation determination using the R pixel and the B pixel as the different color peripheral pixels. As a result, the defect correction circuit 11 can perform arithmetic processing using pixels at the same position as different color peripheral pixels, regardless of which color light pixel is the target pixel in the Bayer array.

  The scratch correction circuit 11 is not limited to the case where the scratch correction cancellation determination is performed for both the white scratch and the black scratch, and may be any circuit that performs the scratch correction cancellation determination for at least one of the white scratch and the black scratch. Shall. The image processing apparatus according to the embodiment may be applied to an electronic device other than a digital camera, such as a mobile phone with a camera.

  Although the embodiment of the present invention has been described, this embodiment is presented as an example and is not intended to limit the scope of the invention. The novel embodiment can be implemented in various other forms, and various omissions, replacements, and changes can be made without departing from the scope of the invention. This embodiment and its modifications are included in the scope and gist of the invention, and are included in the invention described in the claims and the equivalents thereof.

  11 scratch correction circuit, 21 pixel extraction unit, 22 scratch determination unit, 23 white scratch correction cancellation determination unit, 24 black scratch correction cancellation determination unit.

Claims (5)

It has a scratch correction unit that performs scratch correction on the target pixel,
The scratch correction unit
Pixels for detecting the same color light as the color light detected by the target pixel, and for detecting signal values of a plurality of peripheral pixels of the same color located around the target pixel and color light different from the target pixel A pixel extraction unit that extracts signal values of a plurality of different-color peripheral pixels located around the target pixel, and
A scratch determination unit that performs a scratch determination as to whether or not the target pixel is scratched by comparing the signal value of the target pixel and the signal value of the plurality of surrounding pixels of the same color;
A scratch correction cancellation determination unit that performs a scratch correction cancellation determination as to whether or not to cancel the scratch correction for the target pixel determined to be a scratch by the scratch determination unit,
The scratch correction cancellation determination unit includes a signal value of a first different-color peripheral pixel located in a first range centered on the target pixel among the plurality of different-color peripheral pixels, and a signal value outside the first range. 2. The image processing apparatus according to claim 1, wherein the defect correction cancellation determination is performed according to a result of comparison with a signal value of a second different color peripheral pixel located in a range of 2.   The scratch correction cancellation determination unit determines that the difference between the signal value of the first different-color peripheral pixel and the signal value of the second different-color peripheral pixel is larger than a scratch correction cancellation determination level that is a reference for the scratch correction cancellation determination. The image processing apparatus according to claim 1, wherein determination is made to cancel the defect correction.   The scratch correction cancellation determination unit sets the scratch correction cancellation determination level according to at least one of an analog gain in capturing a subject image, a temperature of an image sensor that captures the subject image, and a light amount accumulation time in the image sensor. The image processing apparatus according to claim 2, wherein the calculation is performed. The target pixel, the plurality of same-color peripheral pixels, and the plurality of different-color peripheral pixels are arranged in a first direction and a second direction perpendicular to the first direction,
The scratch correction cancellation determination unit compares the signal value of the first different color peripheral pixel with the signal value of the second different color peripheral pixel in an oblique direction with respect to the first direction and the second direction. The image processing apparatus according to claim 1, wherein the image processing apparatus is characterized.   The defect correction cancellation determination unit compares the comparison result of the signal value of the first different color peripheral pixel with the signal value of the second different color peripheral pixel in the first diagonal direction, and is different from the first diagonal direction. The defect correction cancellation determination is performed according to a comparison result of comparing the signal value of the first different color peripheral pixel with the signal value of the second different color peripheral pixel in the second oblique direction. The image processing apparatus according to claim 4.

Images