JP2005210164A - Correction apparatus for defective pixel - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide a correction apparatus for defective pixels, capable of obtaining an image with high quality without the need for using pseudo-pixel data. <P>SOLUTION: A defective pixel correcting section is configured with an input image memory 21 for storing an image of a plurality of frames resulting from photographing the same scene with different timings; a defect position memory 22 for storing the position of a defective pixel of a CCD area sensor 11; a deviated vector detecting section 23 for detecting a deviation vector of an image between two frames; first and second frames, arbitrarily selected from among images of a plurality of frames stored in the input image memory; and a defective pixel replacing section 24 for replacing defective pixels, on the basis of coordinate data of the defective pixel obtained from the defective position memory and the deviated vector obtained from the deviated vector detecting section. <P>COPYRIGHT: (C)2005,JPO&NCIPI

Description

  The present invention relates to a defective pixel correction apparatus that corrects a defective pixel of a solid-state imaging device.

  In a solid-state imaging device having a multi-pixel using a CCD element as a pixel, which is generally used in an imaging apparatus such as a digital camera, a level that is unique from the peripheral CCD elements is output despite a uniform incident light level. There is a problem in that the occurrence frequency of so-called defective pixels increases and the image quality deteriorates.

Conventionally, techniques relating to defective pixel correction have been proposed for such problems. For example, in Japanese Patent Laid-Open No. 2000-176586, a correlation value in a predetermined direction is obtained from surrounding pixels of a predetermined size, and a correction value for a defective pixel is calculated using surrounding pixels belonging to a direction in which the correlation is maximized by a direction calculation unit. A defective pixel correction apparatus for calculation is disclosed.
JP 2000-176586 A

  By the way, correction of defective pixels is generally performed based on information of four pixels located around the defective pixels. That is, for the defective pixel, the image output data relating to the position of the defective pixel is replaced by the average value of the signals of the four pixels adjacent to the vertical (vertical direction) and the left and right (horizontal direction). However, since the pixel data used for replacement is artificially created, the corrected image may be a peculiar image.

  The present invention has been made paying attention to this point, and an object thereof is to provide a defective pixel correction device capable of obtaining a high-quality image.

  In order to solve the above problems, the invention according to claim 1 is a defective pixel correction apparatus that corrects defective pixel data included in image data read from a solid-state imaging device having defective pixels, and uses the solid-state imaging device. Storage means for storing image data of a plurality of frames taken from the same scene at different timings, and images between first and second frames arbitrarily selected from the image data of a plurality of frames stored in the storage means; Shift vector detecting means for detecting a shift vector; and defective pixel replacement means for replacing defective pixel data of the first frame with pixel data of the second frame based on the shift vector detected by the shift vector detecting means. It is characterized by comprising. The embodiment of the invention according to claim 1 corresponds to the first embodiment.

  According to a second aspect of the present invention, in the defective pixel correction apparatus according to the first aspect, the shift vector detecting means selects the selected one when the shift vector is 0 in the selected first frame and second frame. A frame other than the second frame is selected as the second frame, and a shift vector is detected. The embodiment of the invention according to claim 2 corresponds to the first embodiment.

  According to a third aspect of the present invention, in the defective pixel correction device according to the first aspect, the solid-state imaging device has a color filter in which basic color blocks are repeatedly arranged in units of 2 × 2 pixels on the light receiving surface. It is characterized by being. A second embodiment corresponds to the embodiment of the invention according to claim 3.

  According to a fourth aspect of the present invention, in the defective pixel correction apparatus according to the third aspect, the defective pixel replacement unit has a pixel shift amount in the X-axis and Y-axis directions of the shift vector detected by the shift vector detection unit. If both units are even, the pixel data of the image corresponding to the image of the defective pixel position of the first frame in the second frame is replaced with the defective pixel data of the first frame, and the amount of shift of either one is If the pixel unit is an odd number, the pixel data of the same color as the defective pixel of the first frame adjacent to the image corresponding to the image of the defective pixel position of the first frame in the second frame is the defective pixel of the first frame. It is characterized by replacing data. The second embodiment corresponds to the embodiment of the invention according to claim 4.

  The invention according to claim 5 is a defective pixel correction device that corrects defective pixel data included in image data read from a solid-state imaging device having defective pixels, and the lens disposed in front of the solid-state imaging device or the solid-state imaging device. A movement driving means for moving at least one of the predetermined amount, and at least one of the solid-state imaging element or the lens is moved by a predetermined amount by the movement driving means, and the same scene is used by using the solid-state imaging element before and after the predetermined amount movement. Storage means for storing image data of the first and second frames obtained by photographing the image, and storage in the storage means based on a deviation vector between the two frames due to the movement of the predetermined amount by the movement driving means. And defective pixel replacement means for replacing the defective pixel data of the first frame with the pixel data of the second frame. It is an. A third embodiment corresponds to the embodiment of the invention according to claim 5.

  According to a sixth aspect of the present invention, in the defective pixel correction device according to the fifth aspect, the movement driving means moves at least one of the solid-state imaging device or the lens by a predetermined amount based on the defective pixel position of the first frame. It is characterized by this. A third embodiment corresponds to the embodiment of the invention according to claim 6.

  According to the defective pixel correction apparatus according to claim 1, pixel data corresponding to the defective pixel position of the first frame in the second frame based on the image shift vector between the first frame and the second frame. Can be replaced with defective pixel data of the first frame, and an image closer to the actual image can be obtained. According to the defective pixel correction apparatus of claim 2, when the shift vector becomes 0 between the first frame and the second frame selected first, the defect of the first frame in the second frame Since the pixel data corresponding to the pixel position is also defective pixel data, in this case, a frame other than the selected second frame in which the shift vector is not 0 is used as a new second frame to detect shift. This makes it possible to correct defective pixels.

  Further, according to the defective pixel correction apparatus according to the third aspect, even a color image can be replaced with the pixel data obtained in the second frame for the defective pixel, so that a color image closer to the actual image can be obtained. . According to the defective pixel correction apparatus of the fourth aspect, in the case of the second frame when the amount of deviation is an even number in pixel units with respect to the X-axis and Y-axis directions of the deviation vector detected by the deviation vector detecting means. The pixel data corresponding to the defective pixel position of the first frame can be replaced with the defective pixel data of the first frame, and if any one of the shift amounts is an odd number in units of pixels, it is not the same color, so Although it cannot be replaced, by replacing pixel data of the same color as the defective pixel of the first frame adjacent to the defective pixel position of the first frame in the second frame with defective pixel data of the first frame, It is possible to correct defective pixels.

  According to the defective pixel correction apparatus of the fifth aspect, since the solid-state imaging device or the lens is moved by a predetermined amount, this movement amount can be handled as a deviation vector, and the deviation vector needs to be detected using a detecting unit. Disappears. According to the defective pixel correction device of the sixth aspect, the movement amount can be set based on the defective pixel position of the first frame, and the absolute position of the defective pixel of the first frame and the second frame. Therefore, the defective pixel data of the first frame can always be replaced with the corresponding pixel data of the second frame moved by a predetermined amount.

  Next, the best mode for carrying out the invention will be described.

  First, a first embodiment of the defective pixel correction apparatus according to the present invention will be described. This embodiment corresponds to the inventions according to claims 1 and 2, and a monochrome CCD area sensor not using a filter is used as the solid-state imaging device.

  FIG. 1 is a block diagram illustrating a schematic configuration of an imaging apparatus to which the defective pixel correction apparatus according to the first embodiment of the present invention is applied. In FIG. 1, reference numeral 10 denotes a photographic lens for forming an image of light from a subject, and includes a peripheral circuit thereof. A CCD area sensor 11 includes its peripheral circuits. Reference numeral 12 denotes a preprocess circuit constituted by an analog circuit, reference numeral 13 denotes an A / D conversion circuit, reference numeral 14 denotes a defective pixel correction unit, reference numeral 15 denotes an image signal processing unit, and reference numeral 17 denotes a CPU that controls the overall operation.

  FIG. 2 is a block diagram showing the configuration of the defective pixel correction unit 14 in the first embodiment shown in FIG. 1, and the defective pixel correction unit 14 stores an input image memory from the A / D conversion circuit 13. 21, a defect position memory 22 that stores the position of the defective pixel on the CCD area sensor 11, a shift vector detection unit 23 that detects a shift of the input image, and coordinate data and a shift vector detection unit obtained from the defect position memory 22 And a defective pixel replacement unit 24 that replaces the defective pixel based on the deviation vector obtained from 23.

  Next, the operation of the first embodiment shown in FIGS. 1 and 2 will be described. The image signal output from the CCD area sensor 11 is input to the defective pixel correction unit 14 via the preprocess circuit 12 and the A / D conversion circuit 13. Here, when the pixel of the CCD area sensor 11 that has output the image signal is a normal pixel, it does not coincide with the position of the defective pixel output from the defect position memory 22, so the image signal is not defective. The pixel replacement unit 24 outputs the signal to the image signal processing unit 15 without any correction. On the other hand, if the pixel of the CCD area sensor 11 that has output the image signal is a defective pixel, it matches the position of the defective pixel input from the defective position memory 22, so that the image signal is received by the defective pixel replacement unit 24. Correction is applied.

  FIG. 3 is an explanatory diagram showing a mode of pixel defect correction executed in the defective pixel correction unit 14. The defective pixel correction unit 14 first stores, in the input image memory 21, images of a plurality of frames obtained by photographing the same scene at different timings. Using a first and second frame image arbitrarily selected from a plurality of frames stored in the input image memory 21, a shift vector detector 23 detects a shift vector of the image between two frames. Based on this deviation vector, the defective pixel data is replaced with data to be an actual image. As the first and second frames, the frame that was first shot and the frame that was shot next are usually selected.

  Specifically, the coordinates (a, b) of the defective pixel in the first frame and the X-axis and Y-axis components of the shift vector detected by the shift vector detection unit 23 from the first frame and the second frame. Based on a certain amount of movement (c, d), true data coordinates (a + c, b + d) corresponding to an image of defective pixels in the first frame in the second frame are obtained, and an actual image in the second frame is obtained. The pixel data at the coordinates (a + c, b + d) of the power data is replaced with defective pixel data at the coordinates (a, b) of the first frame.

  Furthermore, it demonstrates concretely using FIG. FIG. 4 shows a part of the pixel arrangement of the monochrome CCD area sensor 11. In this illustrated example, description will be made on the assumption that the pixel 34 is a defective pixel. It is assumed that the position information of the defective pixel is stored in the defect position memory 22 in advance. With respect to the first frame image and the second frame image selected from the images of the plurality of frames taken at the same scene at different timings, the shift vector detection unit 23 extracts the shift vector by image matching, and based on the extracted shift vector. Correct the defect. In the illustrated example of FIG. 4, the X-axis and Y-axis components of the shift vector are (−3, −2), and the data of the pixel 34 ′ is replaced with the data of the defective pixel 34 of the first frame. One frame is an image having no defect.

  A pixel corresponding to the defective pixel position of the first frame in the second frame when the shift vector is 0 between the first frame and the second frame selected from the captured images of the plurality of frames. Data also becomes defective pixel data. In this case, defect correction can be performed by using, as the second frame, a frame in which the deviation vector other than the first selected second frame does not become zero.

  Similarly, in the case of a cluster defect composed of continuous defective pixels, the pixel data corresponding to the defective pixel position of the first frame in the second frame may also be defective pixel data. In this case, by using a frame other than the first selected second frame as the second frame, the pixel data of the second frame corresponding to the image by the defective pixel of the first frame can be Defect correction can be performed by replacing defective pixel data in a frame.

  Next, Example 2 will be described. This embodiment corresponds to the inventions according to claims 3 and 4 and uses a color CCD image sensor in which a color filter is pasted on the light receiving surface to correct defective pixels in a color image. . Here, as the color filter, an RGB Bayer array filter configured by repeatedly arranging basic color blocks in units of 2 × 2 pixels is used.

  FIG. 5 shows a pixel array corresponding to the color filter of the color CCD area sensor. In this example, description will be made on the assumption that the pixel R33 is defective and the 2 × 2 pixel block surrounded by a thick line located at the upper left is the defective pixel block. In the figure, (a) corresponds to the pixel data of the first frame, and (b), (c), (d), and (e) correspond to the image of the defective pixel block position of the first frame in the second frame. The pixel data of the image is shown, and the upper left pixel of the four pixels shown in (b), (c), (d), and (e) corresponds to the image at the position of the defective pixel R33 in the first frame. It is pixel data of the image of the second frame.

  When the amount of movement in the X-axis and Y-axis directions of the pixel data corresponding to the image of the defective pixel position of the first frame in the second frame is an even number in pixel units (b), The pixel data having the same color filter as the defective pixel data in the first frame corresponding to the image of the defective pixel position in the first frame (R pixel data with hatching), and the defective pixel data in the first frame Can be replaced. On the other hand, when the movement amount in the X-axis or Y-axis direction of pixel data corresponding to the image at the defective pixel position of the first frame in the second frame is an odd number in units of pixels (c), (d) , (E) adds 1 to the movement amount in the X-axis and / or Y-axis direction, which is an odd number, and corrects the movement amount in the X-axis and Y-axis direction to an even number as shown in (f), Pixel data having a color filter of the same color as the defective pixel data in the first frame corresponding to the image of the defective pixel position in the first frame in the second frame (data of R pixels with hatching), It is possible to replace defective pixel data in the frame. As a result, replacement correction can be performed with pixel data of the same color as that of the defective pixel.

  Next, Example 3 will be described. This embodiment corresponds to the inventions according to claims 5 and 6. FIG. 6 is a block diagram illustrating a schematic configuration of an imaging apparatus to which the defective pixel correction apparatus according to the third embodiment is applied. This image pickup apparatus is the same as the image pickup apparatus according to the first embodiment shown in FIG. 1, except that a mechanical movement driving means 16 capable of moving at least one of the CCD area sensor 11 and the lens 10 by a predetermined amount is added. is there.

  By moving at least one of the CCD area sensor 11 or the lens 10 by a predetermined amount by the movement driving means 16, an image shift vector between two frames taken at different timings, that is, before and after the predetermined amount of movement is obtained. Since it can be controlled, it is not necessary to detect a shift vector using the shift vector detection unit 23 in the defective pixel correction unit 14 according to the first embodiment shown in FIG. Therefore, as shown in FIG. 7, the defective pixel correction unit 14 in the case where the movement driving means 16 capable of moving at least one of the CCD area sensor 11 or the lens 10 by a predetermined amount is provided as shown in FIG. The shift vector detection unit 23, which includes the memory 21, the defect position memory 22, and the defective pixel replacement unit 24 and is necessary for the defective pixel correction unit 14 in the first embodiment shown in FIG.

  Further, if at least one of the CCD area sensor 11 or the lens 10 is moved based on the position of the defect in the first frame, an image shift between the first frame and the second frame is necessarily generated. Therefore, defective pixels can be corrected only with two frames obtained by photographing the same scene at different timings. Note that the movement amount of at least one of the CCD area sensor or the lens is preferably set based on the defective pixel position in the first frame before the movement.

1 is a block diagram illustrating a schematic configuration of an imaging apparatus to which a defective pixel correction apparatus according to a first embodiment of the present invention is applied. FIG. 2 is a block diagram illustrating a configuration of a defective pixel correction unit according to the first exemplary embodiment illustrated in FIG. 1. FIG. 3 is an explanatory diagram for explaining an operation of the defective pixel correction unit illustrated in FIG. 2. FIG. 3 is an explanatory diagram for explaining the operation of the defective pixel correction unit shown in FIG. 2 in more detail. It is explanatory drawing for demonstrating operation | movement of the defective pixel correction | amendment part in Example 2 using a color CCD area sensor. It is a block diagram which shows schematic structure of the imaging device to which the defective pixel correction apparatus which concerns on Example 3 of this invention is applied. It is a block diagram which shows the structure of the defective pixel correction | amendment part in Example 3 shown in FIG.

Explanation of symbols

10 lenses
11 CCD area sensor
12 Preprocess circuit
13 A / D converter circuit
14 Defective pixel correction unit
15 Image signal processor
16 Movement drive means
17 CPU
21 Input image memory
22 Defect location memory
23 Deviation vector detector
24 Defective pixel replacement

Claims (6)

  In a defective pixel correction apparatus that corrects defective pixel data included in image data read from a solid-state imaging device having defective pixels, image data of a plurality of frames obtained by capturing the same scene at different timings using the solid-state imaging device is stored. Storage means, shift vector detection means for detecting a shift vector of an image between arbitrarily selected first and second frames in a plurality of frames of image data stored in the storage means, and the shift vector detection means A defective pixel correction device comprising: defective pixel replacement means for replacing defective pixel data of the first frame with pixel data of the second frame based on the shift vector detected in step (b).   The shift vector detecting means selects a frame other than the selected second frame as the second frame when the shift vector is 0 in the selected first frame and second frame, and shifts the shift vector. 2. The defective pixel correction apparatus according to claim 1, wherein a vector is detected.   2. The defective pixel correction apparatus according to claim 1, wherein the solid-state image pickup device has a color filter in which basic color blocks are repeatedly arranged in units of 2 × 2 pixels attached to a light receiving surface thereof.   The defective pixel replacement unit detects a defect of the first frame in the second frame when the amount of shift is an even number in pixel units with respect to the X-axis and Y-axis directions of the shift vector detected by the shift vector detection unit. When the pixel data of the image corresponding to the image at the pixel position is replaced with the defective pixel data of the first frame, and the displacement amount of either one is an odd number per pixel, the defect of the first frame in the second frame 4. The defective pixel correction apparatus according to claim 3, wherein pixel data having the same color as defective pixels in the first frame adjacent to an image corresponding to the image at the pixel position is replaced with defective pixel data in the first frame.   In a defective pixel correction apparatus that corrects defective pixel data included in image data read from a solid-state imaging device having defective pixels, at least one of the solid-state imaging device or a lens disposed in front of the solid-state imaging device is moved by a predetermined amount. A movement driving means, and at least one of the solid-state imaging element or the lens is moved by a predetermined amount by the movement driving means, and a first scene obtained by photographing the same scene using the solid-state imaging element before and after the movement by the predetermined amount. Storage means for storing image data of the first and second frames, and defective pixels of the first frame stored in the storage means based on a shift vector between the two frames due to the predetermined amount of movement by the movement driving means A defective pixel correction device comprising: defective pixel replacement means for replacing data with pixel data of the second frame .   6. The defective pixel correction apparatus according to claim 5, wherein the movement driving unit moves at least one of the solid-state imaging device or the lens by a predetermined amount based on the defective pixel position of the first frame.

Images