JP6519276B2 - Image pickup apparatus and defect pixel correction method - Google Patents

Description

  The present invention relates to an imaging device capable of correcting a defective pixel group in an imaging device and a defective pixel correction method.

  Generally, an imaging device formed of a semiconductor such as a complementary metal oxide semiconductor (CMOS) sensor or a charge coupled device (CCD) may include a plurality of so-called defective pixels caused by local crystal defects of the semiconductor or the like. . When imaging is performed using an imaging element having a defective pixel, a normal image signal is not output from the defective pixel, which causes deterioration of the image quality of the obtained image.

  In an imaging apparatus provided with an imaging device mounted with a widely used Bayer type color filter, it is necessary to consider the presence of a color filter when correcting a defective pixel. Therefore, as a conventional method of single defective pixel correction, the position information of the defective pixel is stored in advance in the internal memory of the imaging device, and the color filter identical to the defective pixel among the pixels around the defective pixel It is common to replace the signal value of the defective pixel by the average of the signal values of the normal pixels having. However, when such correction is performed, the number of adjacent pixels having a color filter of the same color as the defective pixel may be reduced, and therefore, highly accurate direction detection may not be possible.

  For example, in the invention disclosed in Patent Document 1, the position information of the defective pixel is stored in a memory or the like before shipping the imaging device to the user, and the output value from the defective pixel is corrected by the correction circuit. Replace with the correction signal generated by using. Further, a correlation value in a predetermined direction is obtained from surrounding pixels of a predetermined size, and the correction value of the defective pixel is calculated using the surrounding pixels belonging to the direction in which the correlation becomes maximum. According to this, it is possible to provide a defective pixel correction device capable of correcting a high quality defective pixel by minimizing the restriction due to the pattern of the image and the arrangement of the pixels.

  On the other hand, with the recent increase in the number of pixels of image pickup elements, not only a single defective pixel but also a plurality of defective pixels are connected, so that all pixels in one horizontal or vertical line become defective pixels. In some cases, a two-dimensionally continuous block of pixels may be a defective pixel. When such a defective pixel group is formed, the proportion of normal pixels among the pixels adjacent to a certain defective pixel is greatly reduced as compared with the case where a single defective pixel is corrected, and the correction accuracy is lowered. Resulting in.

  This means that it is difficult to obtain a satisfactory correction result only by using distance information (for example, the average value of peripheral pixels) of the pixel alone. Furthermore, when all the pixels adjacent to the defective pixel are defective pixels, the signal value can not be estimated from the beginning.

  Therefore, in the invention disclosed in Patent Document 2, in the pixel group used for the correction of the focused defective pixel, a defect pattern acquisition unit that acquires patterns of positions of a plurality of defective pixels as a defect pattern; A pixel selection unit that selects one or more pixels among the pixels that do not correspond to the defective pixel according to the defect pattern, and a defective pixel that corrects the focused defective pixel using each of the selected pixels By providing a correction unit, a pixel which does not correspond to a defective pixel is selected from pixels in the correction block based on the defect pattern, and a defective pixel which can correct the pixel of interest using each of the selected pixels It is possible to provide a correction device. Further, even if there are a large number of defective pixels, it is not necessary to correct the circuit or program for correcting the defective pixels, so that it is possible to suppress an increase in labor and time.

JP 2005-142997 A JP, 2013-183282, A

  However, in the invention disclosed in Patent Document 2 described above, the correction factor to be applied is made different depending on whether or not the arrangement of the acquired defective pixels corresponds to a previously defined defect pattern, and thus the defined defect. There is a possibility that inconvenience may occur when it is attempted to correct a complicated arrangement of defective pixels that do not correspond to a pattern. In addition, when trying to cope with such a complicated arrangement of defective pixels, the number of sets of defect patterns and correction coefficients to be stored increases, and the time required for correction processing becomes longer.

  Also, even if there is a defective pixel group in the imaging area of the subject where high frequency information is dominant such as a resolution chart, the signal value of the pixel is smooth and continuous regardless of the global structure of the subject. It will be the correction result that As a result, it becomes a blurred unnatural image with a lost sense of resolution.

  The present invention has been made in view of such a situation, and it is possible to perform a highly accurate correction process on a large defective pixel group while suppressing the amount of calculation, and more preferably, high frequency information is dominant It is an object of the present invention to provide an imaging device and a defective pixel correction method capable of highly accurate correction processing on a subject.

  In order to achieve the above object, an image pickup apparatus embodying the present invention comprises: an image pickup means having a plurality of pixels arranged in a two-dimensional manner for acquiring an object image; and position information of defective pixels among the plurality of pixels And a defective pixel correction unit that corrects a signal value output from the defective pixel, wherein the defective pixel correction unit is a defect including a plurality of the defective pixels continuously present. From the pixel group, the defective pixel adjacent to the largest number of normal pixels in the periphery is selected as the pixel of interest, and a search range wider than the substitution range including the plurality of surrounding pixels and the substitution range A target block including pixels in a range and a range narrower than the search range is set, and all comparison blocks having the same shape as the target block are extracted from the search range. The degree of correlation between the block of interest and the plurality of comparison blocks is determined, and at least the group of pixels of interest including the plurality of defective pixels in the group of defective pixels included in the replacement range is selected based on the degree of correlation The signal value of the target pixel group is corrected using the signal value of the normal pixel in one or more of the comparison blocks.

  Furthermore, in the imaging apparatus according to the present invention, in the above-mentioned invention, the plurality of pixels each have a plurality of light receiving portions in the depth direction, and the imaging unit is a vertical color separation type imaging unit without a color filter. The defective pixel group includes a plurality of defective pixels having defects in at least one or more light receiving units having the same depth, and the defective pixel correction unit selects at least one selected based on the degree of correlation. The signal value of the light receiving unit having a defect in the pixel group of interest is corrected using the signal value of the normal light receiving unit of the same depth in one or more of the comparison blocks.

  The defective pixel correction method according to the present invention includes the steps of: selecting a defective pixel adjacent to the largest number of normal pixels as a target pixel from a defective pixel group consisting of a plurality of defective pixels continuously present; A replacement range including a plurality of surrounding pixels, a search range wider than the replacement range, and a pixel range including the replacement range and narrower than the search range, with the target pixel substantially centered Setting a block, extracting all comparison blocks consisting of only normal pixels all having the same shape as the target block from the search range, and determining the degree of correlation between the target block and the plurality of comparison blocks; At least one or more selected on the basis of the degree of correlation for a target pixel group consisting of a plurality of the defective pixels in the defective pixel group included in the replacement range By use of the signal values of the normal pixels in the serial comparison block, characterized in that a step of correcting the signal value of the target pixel group.

  According to the image pickup apparatus and the defective pixel correction method of the present invention, it is possible to highly accurately correct a large defective pixel group while suppressing the amount of calculation, and in particular, in a subject whose high frequency information is dominant. It is possible to provide highly accurate correction processing.

FIG. 1 is a block diagram showing the main configuration of an imaging device according to an embodiment of the present invention. It is sectional drawing which simplified the single pixel of the image pick-up element mounted in the imaging device shown in FIG. It is the block diagram which showed the main structures of the defective pixel correction | amendment part shown in FIG. It is a conceptual diagram explaining the process in a block process part.

  Hereinafter, the best mode for carrying out the present invention will be described according to the attached drawings. The present invention is not limited by the embodiment.

  The block diagram shown in FIG. 1 shows the main configuration of an imaging apparatus according to an embodiment of the present invention. The imaging apparatus 100 shown in this figure includes a photographing optical system 110, an imaging element 200, a defective pixel correction unit 130, a signal processing unit 150, a camera CPU 170, a user interface (I / F) 171, and a recording medium interface. (I / F) 172 and an image display unit 190.

  The photographing optical system 110 is configured of a plurality of lens groups (not shown) including a focus lens group and a zoom lens group. In the drawing, only one lens is shown for simplicity.

  The image sensor 200 receives the light beam collected by the photographing optical system 110, photoelectrically converts it, and outputs it as a signal value. A CMOS image sensor is used as the imaging device 200 of the present embodiment.

  The light receiving surface of the image sensor 200 is composed of a large number of pixels. These pixels are vertical color separation type image sensors capable of outputting each color component signal of RGB from a single pixel by using the difference in depth photoelectrically converted by the wavelength of incident light in the inside thereof. Details of the vertical color separation type image sensor will be described later.

  The image pickup device 200 of the present embodiment incorporates a gain variable amplifier for amplifying a color component signal read from a pixel, a gain correction circuit for correcting a gain value, and an A / D converter for converting analog image signals into digital. There is.

  The defective pixel correction unit 130 performs a process of correcting the signal value of the defective pixel included in the imaging device 200 on the color component signal output from the imaging device 200. Details of the correction processing of the defective pixel correction unit 130 will be described later.

  The signal processing unit 150 performs various types of image processing on the color component signal output from the defective pixel correction unit 130. Examples of processing to be performed here include processing of generating RAW data of a predetermined format from color component signals, white balance processing, and color reproduction processing. The signal processing unit 150 also performs development processing to image data of JPEG format or TIFF format.

  The camera CPU 170 performs comprehensive control of the entire imaging device 100. In particular, the camera CPU 170 controls the defective pixel correction unit 130 and the signal processing unit 150.

  The user I / F 171 has, for example, operating members such as a release button, a power button, a command dial, and a cross key, and when the user operates these operating members, the camera CPU 170 issues an instruction to perform a predetermined operation. .

  The recording medium I / F 172 records or reads RAW data and image data after development between the recording medium and the recording medium (not shown). This recording medium is a removable recording medium such as a semiconductor memory.

  The image display unit 190 displays image data processed by the signal processing unit 150, image data read from a recording medium (not shown), and the like.

  In addition, what is necessary is to mount these devices separately, when employ | adopting the image pick-up element 200 which does not incorporate a gain variable amplifier, a gain correction circuit, and an A / D converter which were mentioned above.

  FIG. 2 is a cross-sectional view showing a single pixel of the imaging device 200 mounted on the imaging device 100 in a simplified manner. As described above, the imaging device 200 of the present embodiment is a so-called vertical color separation type image sensor, and in each pixel, three photodiodes are stacked in the depth direction.

  When light is incident on a certain pixel, the blue (B) component in the incident light is photoelectrically converted mainly by the photodiode 210 located on the uppermost surface. Similarly, the green (G) component in the incident light is photoelectrically converted by the photodiode 230 located mainly at the intermediate depth, and the red (R) component is photoelectrically converted by the photodiode 250 mainly located at the lowermost layer . The color separation in the vertical direction utilizes the characteristics of silicon (Si) used as the material of the imaging device 200.

  These three photodiodes are formed by performing predetermined doping at different depths inside the Si substrate. Specifically, the B component photodiode 210 is formed to a depth between about 0.2 and 0.5 μm, and the G component photodiode 230 is formed to a depth between about 0.5 and 1.5 μm. The R component photodiode 250 is formed to a depth between about 1.5 and 3.0 μm.

  Therefore, the image pickup device 200 according to the present embodiment can obtain color component signals of three RGB colors with one pixel without requiring a color filter which is essential for the Bayer type image sensor. Since each pixel can photoelectrically convert wavelength components of all three colors, there is an advantage that it is not necessary to perform pixel interpolation which is essential in the Bayer type image sensor.

  Note that the present invention is not limited to the above-described configuration, and a plurality of photoelectric conversion films having specific absorption characteristics formed of an organic substance, an inorganic substance, or the like may be stacked.

  FIG. 3 is a block diagram showing the main configuration of the defective pixel correction unit 130 in the present embodiment. The defective pixel correction unit 130 includes a defective pixel storage unit 131, a block processing unit 132, a correlation degree determination unit 133, and a signal correction unit 134. The outline of the defective pixel correction of the present invention will be described using this figure.

  Hereinafter, it is assumed that a defect occurs in the B component photodiode 210 among the three photodiodes in the pixel. That is, in the following embodiments, “defective pixel” refers to a pixel in which the B component signal has an abnormal value because there is a defect in the B component photodiode 210 unless otherwise specified. Further, whether or not the G component photodiode 230 and the R component photodiode 250 in the defective pixel have a defect may include both cases.

  The defective pixel storage unit 131 stores positional information of defective pixels acquired in advance in the adjustment process of the factory. As stored positional information, in addition to positional information on each defective pixel, positional information on a defective pixel group in which adjacent defective pixels are connected and information on the size thereof is further included. The position information of the defective pixel is information unique to each imaging element 200, and the defective pixel correction unit 130 corrects the signal value of the defective pixel while referring to the position information as appropriate.

  The color component signal output from the imaging element 200 is first input to the block processing unit 132 in the defective pixel correction unit 130. First, the block processing unit 132 determines, from the position information of the defective pixel stored in the defective pixel storage unit 131, whether or not the size of each defective pixel group is larger than a predetermined threshold value. If there is a defective pixel group larger than the threshold value, the block processing unit 132 divides the defective pixel group into defective pixel groups smaller than the threshold value, and corrects the defective pixel groups after the division. I do.

  When the defective pixel group is positioned on the subject where high frequency information is dominant, it is desirable to give a correction result maintaining a sense of resolution, while when the defective pixel group is positioned on an aperiodic subject, the signal value is smoothly continuous. I want to give a correction result that By dividing the defective pixel group having a size exceeding the threshold value and repeating the correction of the defective pixel group having a fine size, the sense of resolution is maintained in the former case, and the gentle correction result is obtained in the latter case. Can be determined efficiently.

  The position information on the defective pixel group after division may be written as new position information in a different storage area (not shown) in the defective pixel storage unit 131. In addition, a temporary storage area may be provided in the block processing unit 132, and the temporary storage area may be written there.

  FIG. 4 is a conceptual diagram for explaining the processing in the block processing unit 132. As shown in FIG. In the drawing, the pixels marked with x indicate defective pixels. The block processing unit 132 selects one defective pixel having the largest number of normal pixels in the periphery as a target pixel from the group of defective pixels, and is included in a replacement range of 3 × 3 pixels centered on the target pixel. Defective pixels are set as a target pixel group. Further, the block processing unit 132 sets a target block X of 5 × 5 pixels and a search range (not shown) of 24 × 24 pixels around the target pixel.

  Furthermore, using the input color component signal of each pixel and the position information of the defective pixel stored in the defective pixel storage unit 131, the block processing unit 132 has a size of 5 × 5 pixels included in the search range. All comparison blocks Y are extracted. In this comparison block Y, it is desirable that all 5 × 5 pixels be normal pixels.

  The correlation degree determination unit 133 compares the target block X set and extracted in the block processing unit 132 with the plurality of comparison blocks Y to determine the correlation degree f between them. The method of determining the degree of correlation f will be described in detail later.

  Subsequently, the signal correction unit 134 selects at least one or more comparison blocks Y based on the determined degree of correlation f of each comparison block Y, and uses the signal values of the normal pixels in the comparison blocks Y to focus attention. The color component signal of each defective pixel which is the target pixel group in the block X is corrected. Details of the correction of the signal value will be described later.

  Further, the signal correction unit 134 writes the position information of a new defective pixel in which each corrected defective pixel is a normal pixel in a different storage area (not shown) in the defective pixel storage unit 131. The corrected color component signal is sent to the signal processing unit 150, and the processing of the latter stage is performed as described above.

  The defective pixel correction unit 130 similarly corrects the next target pixel group based on the newly obtained defective pixel information. The above process is repeated until all the defective pixels in the target block X to be corrected are corrected. In addition, after all the defective pixels are corrected, the defective pixel correction unit 130 selects another defective pixel group and performs the same process.

  The size of each block and search range is an example, and can be selected appropriately according to the required correction accuracy, processing speed, and the like.

Example 1
Next, one embodiment of the method of determining the degree of correlation f and the correction of the signal value based thereon will be described. Assuming that the block of interest is X, the comparison block is Y, and the set of comparison blocks Y is W, the degree of correlation f can be expressed as f (X, Y, W) as a function of them.

At this time, assuming that the relative position of each defective pixel in the block of interest X is xi and the central absolute position of the comparison block Y is y0, the corrected signal value b ′ (xi) of each defective pixel group is expressed by the following equation (1 ). (I = 1, 2, ..., m)

  Here, b (y0 + xi) means a pixel signal value at the same relative position xi as the defective pixel in the block of interest X in the comparison block Y.

In order to determine the degree of correlation f according to the control of the camera CPU 170, the degree of correlation determination unit 133 of the present embodiment first calculates the determination value V by the following equation (2).

  Here, x'i is the relative position of the pixel not to be corrected in the block of interest X, and x0 is the central absolute position of the block of interest X. (I = 1, 2, ..., m ')

  That is, in the present embodiment, the determination value V means finding the square error of the signal value of the normal pixel at the corresponding position in each of the target block X and the comparison block Y.

Using the determination value V obtained in this way, the correlation degree determination unit 133 determines the correlation degree f by the following equation (3).

  Here, the correlation degree determination unit 133 sets the correlation degree f = 1 (maximum value) for the comparison block Y in which the calculated determination value V is the smallest value, and the correlation degree f = for the other comparison blocks Y. Set to 0 (minimum value).

  When the degree of correlation f is determined, the signal correction unit 134 compares the signal value of each defective pixel constituting the target pixel group in the target block X with the highest degree of correlation based on the equation (1). A process of replacing the signal position of the normal pixel with the same relative position in the block Y is performed.

  In the embodiment described above, the determination value V is obtained as a square error of pixel signals located at corresponding locations in the target block X and the comparison block Y. However, the present invention is not limited to the square. Can be set appropriately depending on the size of the image, the amount of noise, and the like.

  According to the above embodiment, it is possible to collectively correct each signal value of the target pixel group in which a plurality of defective pixels are connected. More specifically, according to the present invention, the determination value and the degree of correlation can be determined in block units, so it is not necessary to determine the correction coefficient for each defective pixel as in the prior art, and the correction value is determined. It is possible to reduce the number of calculations for the calculation and to improve the calculation ability of the calculation for the accuracy of appropriate comparison block search and correlation degree determination, and efficient defect pixel correction can be performed with a small amount of calculation.

  Furthermore, by obtaining the determination value and the degree of correlation on a block basis, the correction accuracy for an object such as a resolution chart in which high frequency information is dominant is improved. This is because a subject whose high frequency information is dominant often has a specific shape periodically arranged, and the determination of the degree of correlation in block units is particularly effective for such a subject.

  In addition, when comparing the block of interest X and the comparison block Y, in the block of interest X except for the defective pixel group including the pixel of interest group and using only normal pixels, the reliability of the determination value can be improved.

(Example 2)
Next, a method of determining the degree of correlation f which is different from the above-described embodiment will be described. In the embodiment described above, the degree of correlation f is uniformly determined for the comparison block Y having the minimum value of the determination value V, but this may be determined by calculation.

For example, the correlation degree f may be obtained by the following equation (4).

  Here, sd and sv are scaling factors, which can be arbitrarily determined. The larger the scaling factor sd, the easier it is to select a comparison block closer to the target block as a candidate block. Furthermore, by setting the scaling factor sd as a monotonically decreasing function with respect to the image height, the distance information between the block of interest X and the comparison block Y is emphasized in the center of the image and ignored in the periphery of the image. .

Further, by making the scaling factor sv a monotonically decreasing function with respect to the image height, the correction result becomes smooth especially in the image peripheral portion, and the robustness of the defective pixel correction can be improved. Furthermore, it is preferable to set the scaling factor sv to a value according to the amount of noise contained in the signal value as shown in the following equation (5).

  When the standard deviation σ depends on the signal value, the temperature of the sensor or the like, the standard deviation σ corresponding to them may be calculated for each defective pixel.

  According to the above embodiment, in addition to the above-described effects, the distance information between the block X of interest and the comparison block Y can be emphasized in determining the degree of correlation f, and the noise amount included in the signal value is taken into consideration. Correction accuracy is improved.

(Example 3)
Next, the determination method of the determination value different from the embodiment described above will be described. In the above-described embodiment, when comparing the block of interest X and the comparison block Y in order to calculate the determination value V, as shown in the equation (2), the pixels having the same relative position in each block Although the signal values are compared, a rotated or inverted comparison block Y may be included as a target of comparison with the block of interest X.

The equation for obtaining the determination value V ′ at this time can be expressed, for example, as the following equation (6).

  Here, p1 (Y) is a comparison block Y rotated 90 degrees to the right, p2 (Y) is a comparison block Y rotated 180 degrees to the right, and p3 (Y) is a comparison block Y rotated 270 degrees to the right , P4 (Y) is the comparison block Y inverted vertically, and p5 (Y) is the comparison block Y inverted horizontally.

  Then, the determination value V is calculated for each of the rotated or inverted comparison blocks Y, and the determination value V obtained by the comparison block Y having the smallest value among them is determined as the degree of correlation f later. Will be used for For example, the correlation degree f is determined as the correlation degree f = 1 (maximum value) for the comparison block Y having the smallest determination value V, as in the above-described embodiment, and the same relative after rotation / inversion of the comparison block Y It may be replaced with the signal value of the normal pixel at the position.

  According to the present embodiment, in addition to the effects described above, when there is a subject having a curvilinear structure in the block of interest X, the correction accuracy is taken into consideration also in the case of rotating / inverting the comparison block. improves.

(Example 4)
Further, in the above-described embodiment, when comparing the block of interest X and the comparison block Y in order to calculate the determination value, the signal values of the corresponding pixels of the respective blocks are simply compared. Weighting may be performed according to the distance between the block X and the comparison block Y.

At this time, assuming that a correction term relating to weighting is D (X, Y), an equation for obtaining the determination value V ′ can be expressed as the following equation (7).

Here, the correction term D can be expressed as, for example, the following equation (8).

Here, sd is a scaling factor, which can be arbitrarily determined. As a larger value is selected, it becomes easier to select a comparison block closer to the target block as a candidate block. Further, although the correction term D is calculated using an exponential function in the above equation (8), any non-negative monotonous decreasing function may be used. For example, equation (9) below may be used.

  Further, the correction term D may be determined by PSF (Point Spread Function) of the photographing optical system 110. For example, since the image is smoothed in the peripheral portion of the image due to the resolution of the imaging optical system 110, the block of interest is focused at the central portion of the image by making the scaling factor sd monotonically decreasing with respect to the image height in the above equation. Emphasis is placed on the distance information between X and the comparison block Y, and is ignored at the periphery of the image.

  According to this embodiment, in addition to the above-described effects, it is considered that an object having a shape similar to the structure of the object in the block of interest X is likely to be located at a distance closer to the block of interest X The correction accuracy is improved by using the correction term D for weighting.

(Example 5)
Next, the correction method of the signal value different from the embodiment described above will be described. In the above-described embodiment, the signal value of the normal pixel at the corresponding position in the comparison block Y having the highest degree of correlation f is replaced with the signal value of the defective pixel of the target block X. The signal value of the defective pixel of the target block X may be replaced with a weighted average value of the signal values of the normal pixels at corresponding positions in two or more comparison blocks Y having a correlation degree f equal to or more than the threshold value.

  According to the present embodiment, when there is a comparison block Y that greatly affects the correction result because the signal value of the central portion has a signal value that is extremely large or small compared to the surrounding, If the degree of correlation f of those comparison blocks Y is small, such a comparison block Y can be excluded when determining the comparison block Y to be used for correction.

  As a result, the correction accuracy of the signal value is improved, and the robustness of the correction process is improved.

(Example 6)
Also, in the embodiments described so far, the case where only the signal value of the defective B component photodiode 210 is corrected has been described. In this case, there is a possibility that the finally generated color may be unnatural with respect to the surroundings due to the misalignment of the correction process. Therefore, in the present embodiment, not only the signal value of the defected B component but also the signal values output from the three photodiodes at the same position are all corrected, thereby suppressing the mismatch of the correction between the signal values. Do.

First, let xi be the relative position of the pixel having a defect in the B component in the block X of interest, and r (xi), g (xi), b (xi) be the signal values of the three photodiodes before correction at the position xi. , R '(xi), g' (xi), b '(xi) respectively after correcting the same position, and further, signal values a (xi) and a' (xi) before and after correction at the same position Are expressed as equations (10) and (11) respectively. (I = 1, 2, ..., m)

At this time, the corrected signal value a ′ (xi) can be expressed as Expression (12) using the degree of correlation f (X, Y, W) as in the above-described embodiment. (I = 1, 2, ..., m)

  Here, a (y0 + xi) means the signal value of each photodiode at the same relative position xi as the defective pixel in the target block X in the comparison block Y.

  As for the determination of the correlation degree f, the correlation between the block of interest X and the comparison block Y is determined by paying attention to the position xi of the B component photodiode 210 having a defect, as in the above-described embodiment. You can ask for

  For example, as in the above-described embodiment, the correlation degree f = 1 (maximum value) is set for the comparison block Y having the smallest determination value V, and all the signal values of the respective photodiodes in the same relative position in the comparison block Y are replaced. Just do it.

  According to this embodiment, it is possible to avoid unnatural color after correction due to mismatch in correction processing in the same pixel, and it is possible to improve final color correction accuracy. Further, although the case where the B component photodiode 210 has a defect has been described in this embodiment, it is possible to apply the defect to either the R component photodiode or the G component photodiode. In that case, the comparison block Y may be extracted and selected using a signal component having a defect.

  In the present embodiment, all three signal values are corrected with respect to one defect photodiode, but for example, when correction is performed by replacing signal values, when focusing on luminance, it is different from the pixel position before correction. The luminance of the position is replaced, and the luminance may have a value different from that conventionally obtained. Therefore, it is also possible to more accurately correct the luminance at the defective pixel position using two more normal signal values.

That is, using the signal value a (xi) before correction and the signal value a ′ (xi) after correction, the signal value a ′ ′ (xi) obtained by further correcting the luminance can be obtained from the following equation (13) it can.

  Here, ε common to the denominator and numerator is a constant for avoiding that the denominator becomes 0, and any value may be used as long as it is a minute positive value.

  According to this equation, since the ratio between the RGB three signal values does not change before and after the luminance correction, the luminance can be corrected while maintaining the color obtained as b ′.

Also, the signal value a ′ ′ (xi) in which the luminance is further corrected can be obtained also by the following equation (14).

  According to this equation, since the ratio between the RGB three signal values only approaches 1 before and after the luminance correction, the change from the color obtained as b 'does not become unnatural.

  The above-described luminance correction can be applied even when the photodiode other than the B component is defective. In that case, the luminance is corrected using the signal values of two normal photodiodes that are not defective.

(Example 7)
Although the extraction and selection of the comparison block Y is performed using the signal component of the photodiode having a defect in the above-described embodiment, the signal component of a normal photodiode having no defect is used for the comparison block Y It is also possible to perform extraction and selection.

That is, the determination value V for determining the degree of correlation f of the comparison block Y is determined from the following equation (15).

  Here, x ′ ′ is the relative position of the pixel in the block of interest X, x 0 is the central absolute position of the block of interest X, σ r, σ g, and σ b are the standard deviations of the respective signal components.

In addition, χb (x) is the following condition, and χr (x) and χg (x) are the same condition.

  According to this equation, when the signal value is compared between the block of interest X and the comparison block Y in order to obtain the determination value V, if the photodiode at the position x0 + x ′ ′ has a defect, the defect photodiode The comparison of the signal values of is not performed.

  Further, in the present embodiment, the difference between the signal values is normalized by the standard deviation when comparing the signal values, but the present invention is not limited to this. In the vertical color separation type imaging device, the noise amount is different for each layer. By normalizing the differences in signal values with the standard deviation of each layer, it is possible to avoid underestimating the signal differences in noisy layers of each layer.

  According to the present embodiment, the extraction and selection of the comparison block Y is performed using also the signal components of the normal photodiode having no defect, so that only one signal component (for example, the B signal component) has a degree of correlation. Even in the case of a subject that is difficult to determine, the comparison block can be determined with high accuracy, and as a result, highly accurate defect pixel correction can be performed.

  Note that the correlation degree f is determined, for example, by setting the correlation degree f = 1 (maximum value) for the comparison block Y having the smallest determination value V as in the above-described embodiment, and the signal value of the normal pixel of the comparison block Y Just replace with.

  Further, in the above-described embodiment, the determination value V is obtained as a square error of pixel signals located at corresponding locations in the target block X and the comparison block Y. However, the present invention is not limited to the square. Can be set appropriately depending on the size of the image, the amount of noise, and the like.

  As described above, according to the imaging device and the defective pixel correction method according to the present invention, it is possible to perform high-accuracy correction processing on a large defective pixel group while suppressing the amount of calculation, and in particular, high frequency A highly accurate correction process can be performed on a subject whose information is dominant.

100: imaging apparatus 110: imaging optical system 130: readout control unit 131: defective pixel storage unit 132: block processing unit 133: correlation degree determination unit 134: signal correction unit 150: signal processing unit 170 : Camera CPU, 171: User interface (I / F), 172: Recording medium interface (I / F), 190: Image display unit, 200: Image sensor, 210: Photodiode for B component, 230: Photo for G component Diode, 250: photodiode for R component

Claims (13)

An imaging unit that has a plurality of pixels arranged in a two-dimensional manner and acquires a subject image;
Defect pixel storage means for storing in advance position information of a defect pixel among the plurality of pixels;
Defect pixel correction means for correcting a signal value output from the defect pixel;
The defective pixel correction unit
The defective pixel group adjacent to the largest number of normal pixels in the periphery is selected as a target pixel from the defective pixel group consisting of the plurality of defective pixels continuously present,
A target block including a replacement range including a plurality of surrounding pixels, a search range wider than the replacement range, and a range narrower than the search range with the replacement pixel including the surrounding pixels. Set
All comparison blocks having the same shape as the target block are extracted from the search range,
Determine the degree of correlation between the block of interest and the plurality of comparison blocks;
The signal value of the normal pixel in at least one or more of the comparison blocks selected based on the degree of correlation is used for the target pixel group including a plurality of the defective pixels in the defective pixel group included in the replacement range An image pickup apparatus for correcting the signal value of the target pixel group;   The imaging device according to claim 1, wherein the comparison blocks are all composed of only normal pixels.   The imaging device according to claim 1, wherein the defective pixel group larger than a predetermined value among the defective pixel groups is divided into a plurality of defective pixel groups smaller than the predetermined value.   The position information of the defective pixel updated about the position information of the pixel group of interest for which the correction is completed is generated, and the updated position information is used when correcting another pixel group of interest in the defect pixel group. The imaging device according to any one of claims 1 to 3, wherein The degree of correlation is an absolute value of a difference between signal values of normal pixels in the block of interest and signal values of normal pixels in the plurality of comparison blocks in a predetermined positional relationship with normal pixels in the block of interest The image pickup apparatus according to any one of claims 1 to 4, which is calculated based on the sum of s powers of.   The imaging apparatus according to any one of claims 1 to 4, wherein the degree of correlation is calculated by weighting according to inter-block distances between the block of interest and the plurality of comparison blocks.   The corrected signal values of the pixel group of interest are signal values of a plurality of normal pixels having a predetermined positional relationship with the plurality of defective pixels constituting the pixel group of interest in the comparison block having the highest degree of correlation. The imaging device according to any one of claims 1 to 6, wherein:   The corrected signal value of the pixel group of interest has a predetermined positional relationship with a plurality of defective pixels constituting the pixel group of interest in at least two or more comparison blocks whose correlation degree is equal to or more than a predetermined threshold value. The image pickup apparatus according to any one of claims 1 to 6, wherein a weighted average value of signal values of a plurality of normal pixels in (4) is used. Each of the plurality of pixels has a plurality of light receiving units in the depth direction,
The imaging means is a vertical color separation type imaging means which does not have a color filter,
The defective pixel group includes a plurality of defective pixels having defects in the light receiving unit of at least one or more same depths,
The defective pixel correction means uses a signal value of the normal light receiving unit of the same depth in at least one or more of the comparison blocks selected based on the degree of correlation to detect a defect in the target pixel group. The image pickup apparatus according to any one of claims 1 to 8, wherein the signal value of the light receiving unit is corrected.   The defective pixel correction means further uses signal values of the normal light receiving unit of the same depth in at least one or more of the comparison blocks selected based on the degree of correlation, 10. The image pickup apparatus according to claim 9, wherein correction of a signal value of the normal light receiving unit having a depth different from that of the light receiving unit having a defect is performed. Among the plurality of light receiving units in the block of interest, the correlation degree has a plurality of signal values of the light receiving unit having no defect and the same depth and predetermined positional relationship as the light receiving unit having no defect. The image pickup apparatus according to claim 9, wherein the image pickup apparatus is calculated based on a sum of s-th powers of difference values with the signal value of the normal light receiving unit in the comparison block.
  The predetermined positional relationship is a position between the target block and the comparison block that has performed at least one of 0 degree rotation, 90 degree rotation, 180 degree rotation, 270 degree rotation, vertical reversal, or horizontal reversal. The imaging device according to any one of claims 5, 7, 8, and 11, which is a relationship. Selecting, as a target pixel, the defective pixel adjacent to the largest number of normal pixels in the periphery from the defective pixel group consisting of a plurality of defective pixels continuously present;
A target block including a replacement range including a plurality of surrounding pixels, a search range wider than the replacement range, and a range narrower than the search range with the replacement pixel including the surrounding pixels. Process of setting
Extracting all comparison blocks having the same shape as the target block and all normal pixels and having only the normal pixels from the search range, and determining the degree of correlation between the target block and the plurality of comparison blocks;
The signal value of the normal pixel in at least one or more of the comparison blocks selected based on the degree of correlation is used for the target pixel group including a plurality of the defective pixels in the defective pixel group included in the replacement range And correcting the signal value of the target pixel group.

Images