JP2016192662A - Imaging device and defective pixel correction method - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide an imaging device and a defective pixel correction method that can properly correct a defective pixel group.SOLUTION: An imaging device includes imaging means, defective pixel storage means 131 for pre-storing position information of a defective pixel, and defective pixel correction means 134 for correcting the signal value of the defective pixel. The defective pixel correction means 134 selects, as a pixel of interest, a defective pixel adjacent to a largest number of normal pixels from a defective pixel group including a plurality of defective pixels, sets a block of interest comprising pixels in a replacement range, a search range broader than the replacement range and a range which contains the replacement range and narrower than the search range while the pixel of interest is set to substantially the center, extracts all comparison blocks having the same shape as the block of interest from the search range, determines the correlation degree between the block of interest and the plurality of comparison blocks, and corrects the signal values of a pixel group of interest comprising plural defective pixels of a defective pixel group contained in the replacement range by using the signal values of the normal pixels in at least one comparison blocks selected based on the correlation degree for the pixel group of interest.SELECTED DRAWING: Figure 3

Description

  The present invention relates to an imaging apparatus capable of correcting a defective pixel group in an imaging device and a defective pixel correction method.

  In general, an imaging element formed of a semiconductor such as a CMOS (Complementary Metal Oxide Semiconductor) sensor or a CCD (Charge Coupled Device) may include a plurality of so-called defective pixels caused by local crystal defects of the semiconductor. . When imaging is performed using an imaging device having a defective pixel, a normal image signal is not output from the defective pixel, causing deterioration in image quality of the obtained image.

  In an imaging apparatus including an imaging device equipped with a widely used Bayer color filter, it is necessary to consider the presence of a color filter when correcting defective pixels. Therefore, as a conventional method for correcting a defective pixel alone, position information of the defective pixel is stored in advance in an internal memory of the imaging device, and among the pixels around the defective pixel, the same color filter as the defective pixel In general, the signal value of the defective pixel is replaced by the average of the signal values of the normal pixels having the above. However, when such correction is performed, the number of adjacent pixels having a color filter of the same color as that of the defective pixel is reduced, so that it may not be possible to detect the direction with high accuracy.

  For example, in the invention disclosed in Patent Document 1, the positional information of defective pixels is stored in a memory or the like before shipping the imaging device to the user, and the output values of the defective pixels are output from the defective pixels by the correction circuit. It replaces with the correction signal generated by using. Further, a correlation value in a predetermined direction is obtained from surrounding pixels of a predetermined size, and a correction value for a defective pixel is calculated using surrounding pixels belonging to a direction in which the correlation is maximized. Thus, it is possible to provide a defective pixel correction apparatus capable of correcting high-quality defective pixels while minimizing restrictions due to image patterns and pixel arrangements.

  On the other hand, with the recent increase in the number of pixels in an image sensor, not only a single defective pixel but also a plurality of defective pixels are connected so that all the pixels in one line in the horizontal or vertical direction become defective pixels. A two-dimensionally continuous block of pixels may be a defective pixel. When such a defective pixel group is formed, the proportion of normal pixels in the pixels adjacent to a certain defective pixel is greatly reduced compared with the case of correcting a single defective pixel, and the correction accuracy is reduced. Resulting in.

  This means that it is difficult to obtain a satisfactory correction result only by using distance information (for example, an average value of surrounding pixels) of a single pixel. Furthermore, if all the pixels adjacent to the defective pixel are defective pixels, the signal value cannot be estimated in the first place.

  Therefore, in the invention disclosed in Patent Document 2, a defect pattern acquisition unit that acquires a pattern at a position of a plurality of defective pixels as a defect pattern in a pixel group used for correcting a focused defective pixel; A pixel selection unit that selects one or more pixels from pixels that do not correspond to the defective pixel, and a defective pixel that corrects the focused defective pixel using each of the selected pixels By including a correction unit, a defective pixel that can select a pixel that does not correspond to a defective pixel from the pixels in the correction block based on the defect pattern and correct the pixel of interest using each of the selected pixels The correction device can be provided. Further, even if there are a large number of defective pixels, it is not necessary to correct a circuit or a program for correcting the defective pixels, so that an increase in labor and time is suppressed.

JP-A-2005-142997 JP 2013-183282 A

  However, in the invention disclosed in Patent Document 2 described above, the correction coefficient to be applied differs depending on whether or not the acquired defective pixel arrangement corresponds to a predefined defect pattern. There is a possibility that inconvenience may arise when trying to correct defective pixels with a complicated arrangement that does not correspond to the pattern. In addition, when trying to deal with such complicatedly arranged defective pixels, the set of defect patterns and correction coefficients to be stored increases, and the time required for the correction processing becomes longer.

  Even if a defective pixel group exists in the imaging area of a subject where high-frequency information is dominant, such as a resolution chart, the signal value of the pixel continues smoothly, ignoring the global structure of the subject. Will result in such correction. This results in a blurred and unnatural image with lost resolution.

  The present invention has been made in view of such a situation, and can perform high-precision correction processing for a large defective pixel group while suppressing the amount of calculation, and more preferably, high-frequency information is dominant. An object of the present invention is to provide an imaging device and a defective pixel correction method capable of performing highly accurate correction processing on a simple subject.

  In order to achieve the above object, an imaging apparatus embodying the present invention includes an imaging unit that has a plurality of pixels arranged two-dimensionally to acquire a subject image, and positional information of defective pixels among the plurality of pixels. A defective pixel storage unit that stores in advance, and a defective pixel correction unit that corrects a signal value output from the defective pixel, and the defective pixel correction unit includes a plurality of defective pixels that exist in succession. From the pixel group, select the defective pixel adjacent to the most normal pixels around as a target pixel, a replacement range including a plurality of surrounding pixels, and a search wider than the replacement range, with the target pixel being substantially at the center A target block including a range and pixels in a range narrower than the search range, including the replacement range, and extracting all comparison blocks having the same shape as the target block from the search range; Determining a degree of correlation between the target block and the plurality of comparison blocks, and for the target pixel group including the plurality of defective pixels in the defective pixel group included in the replacement range, at least selected based on the correlation degree The signal value of the target pixel group is corrected using signal values of normal pixels in one or more of the comparison blocks.

  Furthermore, in the image pickup apparatus embodying the present invention, in the above invention, each of the plurality of pixels has a plurality of light receiving portions in the depth direction, and the image pickup unit is a vertical color separation type image pickup unit having no color filter. The defective pixel group includes a plurality of the defective pixels having a defect in the light receiving portion having at least one same depth, and the defective pixel correction unit is selected based on the correlation degree. The signal value of the light receiving unit having a defect in the target pixel group is corrected using the signal value of the normal light receiving unit having the same depth in two or more comparison blocks.

  The defective pixel correction method embodying the present invention includes a step of selecting, as a target pixel, the defective pixel adjacent to the largest number of normal pixels in the periphery from a defective pixel group including a plurality of defective pixels that exist continuously. , A replacement range including a plurality of surrounding pixels, a search range wider than the replacement range, and a pixel that includes the replacement range and is narrower than the search range, with the target pixel being substantially at the center A step of setting a block, a step of extracting all comparison blocks consisting of only normal pixels in the same shape as the target block from the search range, and determining a degree of correlation between the target block and the plurality of comparison blocks; At least one or more selected based on the degree of correlation for a target pixel group consisting of a plurality of the defective pixels in the defective pixel group included in the replacement range By use of the signal values of the normal pixels in the serial comparison block, characterized in that a step of correcting the signal value of the target pixel group.

  According to the imaging apparatus and the defective pixel correction method embodying the present invention, it is possible to perform high-precision correction processing for a large defective pixel group while suppressing a calculation amount, particularly in a subject in which high-frequency information is dominant. A highly accurate correction process can be provided.

1 is a block diagram illustrating a main configuration of an imaging apparatus that is an embodiment of the present invention. FIG. It is sectional drawing which simplified the single pixel of the image pick-up element mounted in the imaging device shown in FIG. It is the block diagram which showed the main structures of the defective pixel correction | amendment part shown in FIG. It is a conceptual diagram explaining the process in a block process part.

  The best mode for carrying out the present invention will be described below with reference to the accompanying drawings. In addition, this invention is not limited by this embodiment.

  The block diagram shown in FIG. 1 shows the main configuration of an imaging apparatus that is an embodiment of the present invention. The image pickup apparatus 100 shown in the figure includes a photographing optical system 110, an image pickup element 200, a defective pixel correction unit 130, a signal processing unit 150, a camera CPU 170, a user interface (I / F) 171, and a recording medium interface. (I / F) 172 and an image display unit 190 are provided.

  The photographing optical system 110 includes a plurality of lens groups (not shown) including a focus lens group and a zoom lens group. In this figure, only one lens is shown for simplicity.

  The image sensor 200 receives the light beam collected by the photographing optical system 110, photoelectrically converts it, and outputs it as a signal value. A CMOS image sensor is used as the imaging device 200 of the present embodiment.

  The light receiving surface of the image sensor 200 is composed of a large number of pixels. These pixels are vertical color separation type image sensors that can output RGB color component signals from a single pixel by using the difference in depth that is photoelectrically converted depending on the wavelength of incident light. Details of the vertical color separation type image sensor will be described later.

  The imaging device 200 according to the present embodiment includes a gain variable amplifier that amplifies a color component signal read from a pixel, a gain correction circuit that corrects a gain value, and an A / D converter that digitally converts an analog image signal. Yes.

  The defective pixel correction unit 130 performs a process of correcting the signal value of the defective pixel included in the image sensor 200 with respect to the color component signal output from the image sensor 200. Details of the correction processing of the defective pixel correction unit 130 will be described later.

  The signal processing unit 150 performs various types of image processing on the color component signal output from the defective pixel correction unit 130. Examples of processing performed here include processing for generating RAW data in a predetermined format from color component signals, white balance processing, and color reproduction processing. The signal processing unit 150 also performs development processing on image data in JPEG format or TIFF format.

  The camera CPU 170 performs comprehensive control of the entire imaging apparatus 100. In particular, the camera CPU 170 controls the defective pixel correction unit 130 and the signal processing unit 150.

  For example, the user I / F 171 includes operation members such as a release button, a power button, a command dial, and a cross key. When the user operates these operation members, the camera CPU 170 issues an instruction to perform a predetermined operation. .

  The recording medium I / F 172 records or reads RAW data and developed image data with a recording medium (not shown). This recording medium is a detachable recording medium such as a semiconductor memory.

  The image display unit 190 displays image data processed by the signal processing unit 150, image data read from a recording medium (not shown), and the like.

  Note that when the imaging element 200 that does not include the above-described variable gain amplifier, gain correction circuit, and A / D converter is employed, these devices may be mounted individually.

  FIG. 2 is a cross-sectional view schematically showing a single pixel of the image sensor 200 mounted on the image pickup apparatus 100. As described above, the image sensor 200 of the present embodiment is a so-called vertical color separation type image sensor, and each pixel is formed by stacking three photodiodes in the depth direction.

  When light enters a certain pixel, the blue (B) component in the incident light is photoelectrically converted mainly by the photodiode 210 located on the uppermost surface. Similarly, the green (G) component in the incident light is mainly photoelectrically converted by the photodiode 230 located at the intermediate depth, and the red (R) component is mainly photoelectrically converted by the photodiode 250 located at the lowermost layer. . These vertical color separations utilize the characteristics of silicon (Si) used as the material of the image sensor 200.

  These three photodiodes are formed by performing a predetermined doping process at different depths inside the Si substrate. Specifically, the B component photodiode 210 is formed to a depth of about 0.2 to 0.5 μm, and the G component photodiode 230 is formed to a depth of about 0.5 to 1.5 μm. The R component photodiode 250 is formed to a depth of about 1.5 to 3.0 μm.

  Therefore, the image sensor 200 according to the present embodiment can acquire color component signals of three colors of RGB with one pixel, while the color filter essential for the Bayer image sensor is unnecessary. Since each pixel can photoelectrically convert all three color wavelength components, there is a merit that it is not necessary to perform essential pixel interpolation in the Bayer image sensor.

  Note that the present invention is not limited to the above configuration, and a plurality of photoelectric conversion films having specific absorption characteristics formed of an organic material, an inorganic material, or the like may be stacked.

  FIG. 3 is a block diagram showing a main configuration of the defective pixel correction unit 130 in the present embodiment. The defective pixel correction unit 130 includes a defective pixel storage unit 131, a block processing unit 132, a correlation degree determination unit 133, and a signal correction unit 134. An outline of defective pixel correction according to the present invention will be described with reference to FIG.

  Hereinafter, a case where a defect occurs in the B component photodiode 210 among the three photodiodes in the pixel will be considered. That is, in the following embodiments, “defective pixel” refers to a pixel in which the B component signal has an abnormal value because the B component photodiode 210 is defective unless otherwise specified. Further, whether or not the G component photodiode 230 and the R component photodiode 250 in the defective pixel have a defect can include both cases.

  The defective pixel storage unit 131 stores position information of defective pixels acquired in advance in the factory adjustment process. As the stored position information, in addition to the position information of each defective pixel, the position information further includes position information regarding a defective pixel group in which adjacent defective pixels are connected and information regarding the size thereof. The position information of the defective pixel is information unique to each image sensor 200, and the defective pixel correction unit 130 performs a process of correcting the signal value of the defective pixel while appropriately referring to the position information.

  The color component signal output from the image sensor 200 is first input to the block processing unit 132 in the defective pixel correction unit 130. First, the block processing unit 132 determines whether or not the size of each defective pixel group is larger than a predetermined threshold from the position information of the defective pixels stored in the defective pixel storage unit 131. When there is a defective pixel group larger than the threshold value, the block processing unit 132 divides the defective pixel group into defective pixel groups smaller than the threshold value, and corrects the defective pixel group after the division. I do.

  When a defective pixel group is located on a subject where high-frequency information is dominant, it is desirable to provide a correction result that maintains a sense of resolution, while when it is located on an aperiodic subject, the signal value is smoothly continuous. I want to give a correction result. By dividing the defective pixel group that exceeds the threshold and repeating the correction of the defective pixel group, the resolution is maintained in the former case, and the smooth correction result is obtained in the latter case. Can be obtained efficiently.

  The position information related to the defective pixel group after the division may be written in different storage areas (not shown) in the defective pixel storage unit 131 as new position information. Alternatively, a temporary storage area may be provided in the block processing unit 132 and written to the temporary storage area.

  FIG. 4 is a conceptual diagram illustrating processing in the block processing unit 132. In the figure, pixels marked with “x” indicate defective pixels. The block processing unit 132 selects one defective pixel having the largest number of normal pixels around as a target pixel from the defective pixel group, and is included in a 3 × 3 pixel size replacement range centered on the target pixel. A defective pixel is set as a target pixel group. Further, the block processing unit 132 sets a target block X having a size of 5 × 5 pixels and a search range (not shown) having a size of 24 × 24 pixels around the target pixel.

  Further, the block processing unit 132 uses the input color component signal of each pixel and the position information of the defective pixel stored in the defective pixel storage unit 131 to have a size of 5 × 5 pixels included in the search range. All comparison blocks Y are extracted. In this comparison block Y, it is desirable that all 5 × 5 pixels are normal pixels.

  The correlation degree determination unit 133 compares the target block X set and extracted by the block processing unit 132 with the plurality of comparison blocks Y, respectively, and determines the correlation degree f between them. The method for determining the correlation degree f will be described later in detail.

  Subsequently, the signal correction unit 134 selects at least one comparison block Y based on the determined correlation degree f of each comparison block Y, and pays attention using the signal value of the normal pixel in the comparison block Y. The color component signal of each defective pixel which is the target pixel group in the block X is corrected. Details of the correction of the signal value will be described later.

  The signal correction unit 134 further writes the position information of a new defective pixel in which each corrected defective pixel is a normal pixel in a different storage area (not shown) in the defective pixel storage unit 131. The corrected color component signal is sent to the signal processing unit 150, and the subsequent processing is performed as described above.

  The defective pixel correction unit 130 similarly corrects the next pixel group of interest based on newly obtained defective pixel information. The above process is repeated until all defective pixels in the target block X to be corrected are corrected. Further, after all defective pixels are corrected, the defective pixel correction unit 130 selects another defective pixel group and performs the same processing.

  Note that the size of each block and search range is an example, and can be appropriately selected according to necessary correction accuracy, processing speed, and the like.

Example 1
Next, an embodiment will be described with respect to a method for determining the correlation degree f and correction of signal values based on the method. When the target block is X, the comparison block is Y, and the set of comparison blocks Y is W, the correlation degree f can be expressed as f (X, Y, W) as a function thereof.

At this time, assuming that the relative position of each defective pixel in the target block X is xi and the center absolute position of the comparison block Y is y0, the corrected signal value b ′ (xi) of each defective pixel group is expressed by the following equation (1). ) (I = 1, 2,..., M)

  Here, b (y0 + xi) means a pixel signal value at the same relative position xi as the defective pixel in the target block X in the comparison block Y.

The correlation degree determination unit 133 according to the present embodiment first calculates a determination value V according to the following equation (2) in order to determine the correlation degree f in accordance with the control of the camera CPU 170.

  Here, x′i is a relative position of a pixel not to be corrected in the target block X, and x0 is a center absolute position of the target block X. (I = 1, 2,..., M ′)

  That is, in this embodiment, the determination value V means that the square error of the signal value of the normal pixel at the corresponding position in each of the target block X and the comparison block Y is obtained.

Using the determination value V obtained in this way, the correlation degree determination unit 133 determines the correlation degree f by the following equation (3).

  Here, the correlation level determination unit 133 sets the correlation level f = 1 (maximum value) for the comparison block Y in which the calculated determination value V is the smallest value, and the correlation level f = for the other comparison blocks Y. 0 (minimum value).

  When the degree of correlation f is determined, the signal correction unit 134 compares the signal value of each defective pixel constituting the target pixel group in the target block X with the highest degree of correlation based on Expression (1). A process of replacing the signal value of the normal pixel having the same relative position in the block Y is performed.

  In the above-described embodiment, the determination value V is obtained as a square error of pixel signals located at locations corresponding to each other in the target block X and the comparison block Y. However, the determination value V is not limited to the square, and the target block X The size can be set as appropriate according to the size and the amount of noise.

  According to the above embodiment, it is possible to collectively correct each signal value of the pixel group of interest in which a plurality of defective pixels are connected. Specifically, according to the present invention, the determination value and the degree of correlation can be obtained in units of blocks, so there is no need to determine a correction coefficient for each defective pixel as in the prior art, and correction value determination For this reason, it is possible to reduce the number of computations for the calculation, and to use the calculation capacity corresponding to the improvement of the accuracy of appropriate comparison block search and correlation degree determination, so that efficient defective pixel correction can be performed with a small amount of calculation.

  Further, by obtaining the determination value and the degree of correlation in units of blocks, the correction accuracy for a subject in which high frequency information is dominant, such as a resolution chart, is improved. This is because a subject whose high-frequency information is dominant often has a specific shape periodically arranged, and for such a subject, determination of the degree of correlation in units of blocks is particularly effective.

  Further, when comparing the target block X with the comparison block Y, the target block X excludes the defective pixel group including the target pixel group and uses only normal pixels, so that the reliability of the determination value can be improved.

(Example 2)
Next, a method for determining the degree of correlation f different from the above-described embodiment will be described. In the above-described embodiment, the degree of correlation f is determined for the comparison block Y having the minimum determination value V, but it may be obtained by calculation.

For example, the correlation degree f may be obtained by the following formula (4).

  Here, sd and sv are scaling factors and can be arbitrarily determined. The larger the scaling factor sd is selected, the easier it is to select a comparison block whose distance from the block of interest is a candidate block. Furthermore, by making the scaling factor sd a function that decreases monotonously with respect to the image height, importance is attached to the distance information between the target block X and the comparison block Y at the center of the image, and it is ignored at the periphery of the image. .

Further, by making the scaling factor sv a function monotonously decreasing with respect to the image height, the correction result becomes smooth particularly in the peripheral portion of the image, and the robustness of defective pixel correction can be improved. Furthermore, the scaling factor sv is preferably set to a value corresponding to the amount of noise included in the signal value as shown in the following equation (5).

  When the standard deviation σ depends on the signal value, the sensor temperature, etc., the standard deviation σ corresponding thereto may be calculated for each defective pixel.

  According to the above embodiment, in addition to the above-described effects, it is possible to place importance on the distance information between the block of interest X and the comparison block Y when determining the correlation degree f, and further consider the amount of noise included in the signal value. This improves the correction accuracy.

Example 3
Next, a determination value determination method different from the above-described embodiment will be described. In the embodiment described above, when comparing the target block X and the comparison block Y in order to calculate the determination value V, as shown in the equation (2), the pixels having the same relative position in each block are compared. Although the signal values are compared, a comparison block Y that is rotated or inverted may be included as a comparison target with the block of interest X.

The formula for obtaining the determination value V ′ at this time can be expressed as, for example, the following formula (6).

  Here, p1 (Y) is a comparison block Y rotated 90 degrees to the right, p2 (Y) is a comparison block Y rotated 180 degrees to the right, and p3 (Y) is a comparison block Y rotated 270 degrees to the right. , P4 (Y) is a comparison block Y inverted vertically, and p5 (Y) is a comparison block Y inverted horizontally.

  Then, the determination value V is calculated for each of the rotated and inverted comparison blocks Y, and the determination value V obtained by the comparison block Y having the smallest value among them is determined as the later correlation degree f. It will be used for. For example, as in the above-described embodiment, the correlation degree f is determined by setting the correlation degree f = 1 (maximum value) for the comparison block Y having the smallest determination value V, and the same relative value after rotation / inversion of the comparison block Y. The signal value of the normal pixel at the position may be replaced.

  According to the present embodiment, in addition to the above-described effect, when a subject having a curved structure is included in the target block X, the correction accuracy can be improved by considering the case where the comparison block is rotated and inverted. improves.

Example 4
In the above-described embodiment, when comparing the target block X and the comparison block Y in order to calculate the determination value, the signal values of the corresponding pixels in each block are simply compared. Weighting may be performed according to the distance between the block X and the comparison block Y.

At this time, if the correction term relating to weighting is D (X, Y), the equation for obtaining the determination value V ′ can be expressed as the following equation (7).

Here, the correction term D can be expressed, for example, by the following equation (8).

Here, sd is a scaling factor and can be arbitrarily determined. The larger the value selected, the easier it is to select a comparison block that is closer to the target block as a candidate block. In the above equation (8), the correction term D is calculated using an exponential function. However, any non-negative monotonically decreasing function may be used. For example, the following equation (9) can be used.

  Further, the correction term D may be determined by a PSF (Point Spread Function) of the photographing optical system 110. For example, since the image becomes smooth at the periphery of the image due to the resolution of the photographing optical system 110, the scaling block sd is a function that is monotonously decreasing with respect to the image height in the above equation, so that the block of interest at the center of the image Emphasis is placed on distance information between X and the comparison block Y, and it is ignored at the periphery of the image.

  According to the present embodiment, in addition to the above-described effect, an idea that an object having a shape similar to the structure of the object in the target block X is more likely to be located at a distance closer to the target block X is considered. The correction accuracy can be improved by using the correction term D relating to weighting.

(Example 5)
Next, a signal value correction method different from the above-described embodiment will be described. In the above-described embodiment, the process of replacing the signal value of the normal pixel at the corresponding position in the comparison block Y with the highest correlation degree f with the signal value of the defective pixel in the target block X is performed. The signal value of the defective pixel in the target block X may be replaced with the weighted average value of the signal value of the normal pixel at the corresponding position in each of the two or more comparison blocks Y having a correlation degree f equal to or greater than the threshold value.

  According to the present embodiment, when the signal value of the central portion has a signal value that is extremely large or small compared to the surroundings, and there is a comparison block Y that greatly affects the correction result, If the degree of correlation f of these comparison blocks Y is small, such comparison blocks Y can be excluded when determining the comparison block Y used for correction.

  As a result, the correction accuracy of the signal value is improved and the robustness of the correction process is improved.

(Example 6)
In the embodiments described so far, the case where only the signal value of the defective B component photodiode 210 is corrected has been described. In this case, there is a possibility that the finally generated color becomes unnatural with respect to the surroundings due to the mismatch of the correction processing. Therefore, in this embodiment, not only the signal value of the defective B component but also all the signal values output from the three photodiodes at the same position are corrected, thereby suppressing the correction mismatch between the signal values. To do.

First, the relative position of a pixel having a defect in the B component in the target block X is xi, and the signal values of three photodiodes before correction at the position xi are r (xi), g (xi), and b (xi), respectively. , R ′ (xi), g ′ (xi), b ′ (xi) are signal values after correction at the same position, and signal values a (xi) and a ′ (xi) before and after correction at the same position. Are expressed as in equations (10) and (11), respectively. (I = 1, 2,..., M)

At this time, the corrected signal value a ′ (xi) can be expressed as in Expression (12) using the degree of correlation f (X, Y, W) as in the above-described embodiment. (I = 1, 2,..., M)

  Here, a (y0 + xi) means the signal value of each photodiode at the same relative position xi as the defective pixel in the target block X in the comparison block Y.

  As for the determination of the correlation degree f, the correlation between the target block X and the comparison block Y is determined by paying attention to the position xi of the defective B component photodiode 210 as in the above-described embodiment. Can be obtained.

  For example, as in the above-described embodiment, the degree of correlation f = 1 (maximum value) is set for the comparison block Y having the smallest determination value V, and all the signal values of the photodiodes at the same relative position in the comparison block Y are replaced. That's fine.

  According to the present embodiment, it is possible to avoid unnatural colors after correction due to mismatching of correction processing in the same pixel, and it is possible to improve final color correction accuracy. In the present embodiment, the case where the B component photodiode 210 has a defect has been described. However, the defect can be applied to either an R component photodiode or a G component photodiode. In that case, the comparison block Y may be extracted and selected using a signal component having a defect.

  In this embodiment, all three signal values are corrected for one defective photodiode. However, for example, when correction is performed by replacing signal values, if attention is paid to luminance, the pixel position before correction is different. It is replaced with the luminance of the position, and there is a possibility that the value is different from the luminance that should have been obtained conventionally. Therefore, it is also possible to correct the luminance at the defective pixel position with higher accuracy by using two more normal signal values.

That is, using the signal value a (xi) before correction and the signal value a ′ (xi) after correction, the signal value a ″ (xi) further corrected for luminance can be obtained from the following equation (13). it can.

  Here, ε common to the denominator numerator is a constant for avoiding the denominator being 0, and any value may be used as long as it is a minute positive value.

  According to this equation, since the ratio between the three RGB signal values does not change before and after the luminance correction, the luminance can be corrected while maintaining the color obtained as b '.

Further, the signal value a ″ (xi) whose luminance is further corrected can be obtained from the following equation (14).

  According to this equation, the ratio between the three RGB signal values before and after the luminance correction only approaches 1, so that although it changes from the color obtained as b ', it does not become unnatural.

  Note that the luminance correction described above can also be applied even when a photodiode other than the B component is defective. In that case, the luminance is corrected using signal values of two normal photodiodes that are not defective.

(Example 7)
In the above-described embodiment, the comparison block Y is extracted and selected using the signal component of the photodiode having a defect. However, the signal component of the normal photodiode having no defect is used to select and compare the comparison block Y. It is also possible to perform extraction and selection.

That is, the determination value V for determining the correlation degree f of the comparison block Y is obtained from the following equation (15).

  Here, x ″ is the relative position of the pixel in the block of interest X, x0 is the absolute center position of the block of interest X, and σr, σg, and σb are standard deviations for each signal component.

Further, χb (x) is the following condition, and χr (x) and χg (x) are the same condition.

  According to this equation, when the signal value is compared between the target block X and the comparison block Y in order to obtain the determination value V, if the photodiode at the position x0 + x ″ has a defect, the defective photodiode Comparison of the signal values is not performed.

  In this embodiment, the signal value difference is normalized by the standard deviation when comparing the signal values. However, the present invention is not limited to this. In the vertical color separation type image sensor, the amount of noise differs for each layer. By normalizing the signal value difference with the standard deviation of each layer, it is possible to avoid underestimating the signal difference of the noisy layer among the layers.

  According to this embodiment, the comparison block Y is extracted and selected using the signal component of a normal photodiode having no defect, so that the degree of correlation can be obtained with only one signal component (for example, the B signal component). Even in the case of a subject that is difficult to discriminate, it is possible to discriminate the comparison block with high accuracy, and as a result, it is possible to correct defective pixels with high accuracy.

  For example, the correlation degree f is determined by setting the correlation degree f = 1 (maximum value) for the comparison block Y having the smallest determination value V, as in the above-described embodiment, and the signal value of the normal pixel of the comparison block Y. Replace with.

  In the above-described embodiment, the determination value V is obtained as a square error of pixel signals located at locations corresponding to each other in the target block X and the comparison block Y. However, the determination value V is not limited to the square. The size can be set as appropriate according to the size and the amount of noise.

  As described above, according to the imaging device and the defective pixel correction method of the present invention, it is possible to perform a highly accurate correction process for a large defective pixel group while suppressing the amount of calculation. A highly accurate correction process can be performed on a subject whose information is dominant.

DESCRIPTION OF SYMBOLS 100: Imaging device, 110: Imaging optical system, 130: Reading control part, 131: Defect pixel memory | storage part, 132: Block processing part, 133: Correlation degree determination part, 134: Signal correction part, 150: Signal processing part, 170 : Camera CPU, 171: User interface (I / F), 172: Recording medium interface (I / F), 190: Image display unit, 200: Image sensor, 210: Photodiode for B component, 230: Photo for G component Diode, 250: Photodiode for R component

Claims (13)

An imaging unit having a plurality of pixels arranged in a two-dimensional manner to acquire a subject image;
Defective pixel storage means for previously storing position information of defective pixels among the plurality of pixels;
And defective pixel correction means for correcting a signal value output from the defective pixel,
The defective pixel correction means includes
From the defective pixel group consisting of a plurality of the defective pixels present in succession, select the defective pixel adjacent to the most normal pixels around as a target pixel,
A target block comprising a replacement range including a plurality of surrounding pixels, a search range wider than the replacement range, and pixels in a range that includes the replacement range and is narrower than the search range, with the target pixel being substantially at the center. Set
Extract all comparison blocks of the same shape as the block of interest from the search range,
Determining a degree of correlation between the block of interest and the plurality of comparison blocks;
Using a signal value of a normal pixel in at least one or more of the comparison blocks selected based on the degree of correlation for a target pixel group including a plurality of the defective pixels in the defective pixel group included in the replacement range. An image pickup apparatus that corrects a signal value of the pixel group of interest.   The imaging apparatus according to claim 1, wherein all the comparison blocks include only normal pixels.   The imaging apparatus according to claim 1, wherein the defective pixel group larger than a predetermined value among the defective pixel groups is divided into a plurality of the defective pixel groups smaller than the predetermined value.   The position information of the defective pixel updated with respect to the position information of the target pixel group that has been corrected is generated, and the updated position information is used when correcting another target pixel group in the defective pixel group. The imaging apparatus according to any one of claims 1 to 3.   The degree of correlation is the s-th power of the difference value between the signal value of the normal pixel in the target block and the signal value of the normal pixel in the plurality of comparison blocks having a predetermined positional relationship with the normal pixel in the target block. 5. The imaging apparatus according to claim 1, wherein the imaging device is calculated based on a sum of the two.   The imaging apparatus according to claim 1, wherein the correlation is calculated by weighting according to an inter-block distance between the block of interest and the plurality of comparison blocks.   The corrected signal values of the target pixel group are signal values of a plurality of normal pixels in the predetermined positional relationship with the plurality of defective pixels constituting the target pixel group in the comparison block having the highest degree of correlation. The imaging apparatus according to claim 1, wherein:   The corrected signal value of the pixel group of interest includes a plurality of the defective pixels constituting the pixel group of interest and the predetermined position in at least two comparison blocks having the correlation degree equal to or greater than a predetermined threshold value. The imaging apparatus according to claim 1, wherein a weighted average value of signal values of a plurality of related normal pixels is used. Each of the plurality of pixels has a plurality of light receiving portions in the depth direction,
The imaging means is a vertical color separation type imaging means having no color filter,
The defective pixel group is composed of a plurality of the defective pixels having a defect in the light receiving portion having at least one and the same depth,
The defective pixel correcting means uses a signal value of the normal light receiving unit having the same depth in at least one or more of the comparison blocks selected based on the degree of correlation to detect a defect in the target pixel group. The image pickup apparatus according to claim 1, wherein a signal value of the light receiving unit is corrected.   The defective pixel correction means further uses the signal value of the normal light receiving unit having the same depth in at least one or more of the comparison blocks selected based on the correlation degree. The imaging apparatus according to claim 9, wherein the signal value of the normal light receiving unit having a depth different from that of the light receiving unit having a defect is corrected.   The correlation degree is equal to the signal value of the light receiving unit that does not have a defect among the plurality of light receiving units in the block of interest and has the same positional relationship as the light receiving unit that does not have a defect. The imaging apparatus according to claim 9, wherein the imaging apparatus calculates the difference based on a sum of s-th power of a difference value from a signal value of the normal light receiving unit in the plurality of comparison blocks.   The predetermined positional relationship is a position between the comparison block that has been subjected to at least one of 0 degree rotation, 90 degree rotation, 180 degree rotation, 270 degree rotation, upside down, and left and right inversion, and the target block. The imaging apparatus according to claim 5, wherein the imaging apparatus has a relationship. Selecting the defective pixel adjacent to the most normal pixels around as a target pixel from a defective pixel group composed of a plurality of defective pixels present in succession; and
A target block comprising a replacement range including a plurality of surrounding pixels, a search range wider than the replacement range, and pixels in a range that includes the replacement range and is narrower than the search range, with the target pixel being substantially at the center. A process of setting
Extracting all comparison blocks consisting of only normal pixels in the same shape as the target block from the search range, and determining a degree of correlation between the target block and the plurality of comparison blocks;
Using a signal value of a normal pixel in at least one or more of the comparison blocks selected based on the degree of correlation for a target pixel group including a plurality of the defective pixels in the defective pixel group included in the replacement range. And a step of correcting the signal value of the pixel group of interest.

Images