JP2010154536A - Imaging apparatus and signal processing method - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide an imaging apparatus and a signal processing method for efficiently holding the position information of a defective pixel included in a CCD sensor, a CMOS image sensor or the like. <P>SOLUTION: This imaging apparatus 1 includes: an imaging device 13 for performing the photoelectric conversion of input information; a defective pixel detection module 15 for detecting a defective pixel that the imaging device includes from an output of each of pixels of the imaging device; and a defective pixel position storage module 17 for detecting that the defective pixel of the imaging device supplied from the defective pixel detection module is continued to an adjacent pixel, and storing the position of the defective pixel in accordance with the position information of a leading pixel and determination data of two bits when the defective pixels are adjacent. <P>COPYRIGHT: (C)2010,JPO&INPIT

Description

  The present invention relates to an imaging apparatus and a signal processing method for detecting defective pixels in a CCD sensor or a CMOS image sensor and correcting the influence of defective pixels in an output image.

  Image pickup apparatuses using a CCD (Charge Coupled Device) sensor or a CMOS (Complementary metal-oxide Semiconductor) image sensor as an image pickup element have already been widely used. It is known that a CCD sensor or a CMOS image sensor includes a defective pixel from which a normal image signal cannot be obtained.

  Against this background, a method for correcting the influence of defective pixels when storing positional information of defective pixels and obtaining an output image has already been proposed.

  Patent Document 1 discloses an image signal processing apparatus that corrects defective pixels based on pixel information held in a holding unit and pixel position information synchronized with an image signal.

JP 2007-228269 A

  However, Patent Document 1 discloses that the position information of defective pixels is retained and that the defective pixels are corrected using peripheral pixels, but a method of efficiently retaining the position information of a plurality of defective pixels. That is, there is no description about a method for reducing the memory for storing position information of defective pixels.

  An object of the present invention is to provide an image pickup apparatus and a signal processing method that efficiently hold position information of defective pixels included in a CCD sensor or a CMOS image sensor.

  The present invention was made based on the above problems, an image sensor that photoelectrically converts input information, a defective pixel detection module that detects a defective pixel included in the image sensor from the output of each pixel of the image sensor, It is detected that the defective pixel of the image sensor supplied from the defective pixel detection module is continuous with an adjacent pixel, and the position information of the leading pixel and the determination data of a plurality of bits when the defective pixel is adjacent An imaging apparatus comprising: a defective pixel position storage module that stores the position of the defective pixel.

  According to one embodiment of the present invention, addresses of defective pixels included in an image sensor can be recorded with high density. Further, the address of the recorded defective pixel can be easily demodulated.

  Therefore, it is possible to obtain an image pickup apparatus that can record the address of the defective pixel without unnecessarily occupying a large amount of memory and can easily demodulate the address during reproduction.

BRIEF DESCRIPTION OF THE DRAWINGS Schematic explaining the imaging device with which one Embodiment of this invention is summarized. Schematic explaining an example of a method for detecting a defect of an image sensor of the image sensor of the imaging device shown in FIG. FIG. 2 is a schematic diagram illustrating an example of a method for recording addresses of consecutive defective pixels of the image sensor of the imaging apparatus illustrated in FIG. 1 at high density.

  Embodiments of the present invention will be described below with reference to the drawings.

  FIG. 1 shows an example of an imaging apparatus to which the present invention is applied. In each element described below, an element called “module” may be realized by hardware, or may be realized by software using a CPU or the like.

  The imaging apparatus 1 shown in FIG. 1 performs luminance conversion and color information by photoelectrically converting the information of the imaging target to which predetermined optical characteristics are given by the lens 11 that captures the imaging target information as light brightness and color and color information. An image sensor 13 that outputs information is included. The image sensor 13 is, for example, a CCD (Charge Coupled Device) sensor or a CMOS (Complementary metal-oxide Semiconductor) image sensor.

  The output of the image sensor 13 is an image processing circuit module (DSP) that detects defective pixels included in the image sensor 13 based on luminance information or color information output from the image sensor 13 and corrects an output image corresponding to the defective pixels. 15 is input.

  The position of the defective pixel, that is, the defective pixel address acquired by the DSP 15 is stored by the MPU (control module and memory unit) 17 in accordance with the continuous defective pixel display described later with reference to FIG. When output of the output image is requested, the defective pixel address is demodulated (from the continuous defective pixel display) by the MPU 17 and supplied to the DSP 15.

  The DSP 15 is supplied with a clock signal CLK from a clock oscillator (OSC) 19. The DSP 15 includes a synchronization signal generation module (Sync) 17a integrally or independently.

  An output image output from the DSP 15 is output as a visible image via an output interface (IF) 21 to a monitor device 23 such as an LCD (Liquid Crystal Display) panel or a CRT (Cathode Ray Cube). .

  More specifically, an example is shown in FIG. 2 when the imaging device 1 is activated, but the DSP 15 checks whether each pixel of the image sensor 13 is a defective pixel.

  That is, the output of each pixel of the image sensor 13 input to the DSP 15 is checked by the DSP 15 (Block 1).

  Subsequently, the address (position information) of the pixel determined as a defective pixel by the DSP 15 is stored in the MPU 17. The address to be stored by the MPU 17 is transmitted from the DSP 15 to the MPU 17 (Block 2).

  Thereafter, when the DSP 15 checks all pixels of the image sensor 13 for defects, the MPU 17 determines whether defects are continuous based on the defective pixel addresses received from the DSP 15, that is, continuous defective pixels. Is checked (all address scanning). Here, when the presence of an arbitrary number of consecutive defective pixels can be checked, it is stored according to the continuous defective pixel display by the method described below with reference to FIG.

  On the other hand, when outputting an image captured by the image sensor 13, the address of the defective pixel is sequentially transmitted from the MPU 17 to the DSP 15. At this time, the continuous defective pixels are demodulated from the continuous pixel defect display described with reference to FIG. 3 to the addresses of the individual defective pixels and transmitted to the DSP 15 (Block 11).

  Hereinafter, in the DSP 15, the output image from the pixel specified by the address of the continuous defective pixel or the single (independent) defective pixel is corrected using the output from the surrounding pixels (Block 12).

  FIG. 3 shows an example of a method for displaying addresses of consecutive defective pixels in which two or more defective pixels are continuous. Note that the continuous defective pixel display method shown in FIG. 3 stores the continuous defective pixels by adding a “2 bit” continuous (direction) determination flag to the “representative pixel”. The “representative pixel” is indicated by “V” in the vertical direction and “H” in the horizontal direction. At this time, the continuity (direction) determination flag uses a memory area used during defective pixel detection.

  FIG. 3A shows the case of a single (non-continuous) defective pixel.

  In the example shown in FIG. 3A, since only a single “pixel a” is a defective pixel, the continuous determination flag “00” is used in addition to the address (H, V) of “pixel a” for continuous defect display. Is added.

  FIG. 3B shows the case of continuous defective pixels that are continuous in the horizontal direction.

  The example shown in FIG. 3B is a case where there are two defective pixels in the horizontal direction, that is, “pixel a” that is adjacent to the “pixel a” that is the representative pixel, and the representative pixel “ In addition to the address (H, V) of the pixel “a”, “01” is added. At this time, the address of “pixel b” is (H + 1, V).

  FIG. 3C shows the case of continuous defective pixels that are continuous in the vertical direction.

  In the example shown in FIG. 3C, since there are two defective pixels in the vertical direction, “10” is added to the address (H, V) of the “pixel a” that is the representative pixel. At this time, the address of “pixel b” is (H, V + 1).

  FIG. 3D shows the case of continuous defective pixels that are continuous in the vertical direction and the horizontal direction, respectively.

  In the example shown in FIG. 3D, since the defective pixels are 2 × 2 four consecutive defective pixels, two in the vertical direction and in the horizontal direction, the address (H, V) of the “pixel a” that is the representative pixel. In addition, “11” is added.

  At this time, the address of “pixel b” adjacent to “pixel a” in the horizontal direction is (H + 1, V), and the address of “pixel c” adjacent to “pixel a” in the vertical direction is (H, V + 1). It is. In addition, when considering a 4 × 2 matrix in which the sides do not touch “pixel a”, the address of “pixel d” located on the diagonal line of “pixel a” is represented by (H + 1, V + 1). be able to.

  On the other hand, when an output image is output from the DSP 15, that is, when an image captured by the image sensor 13 is reproduced, the address of the defective pixel output by the MPU 17 to the DSP 15 for the single defective pixel shown in FIG. Is the address (H, V) of “pixel a”.

  On the other hand, when the address output from the MPU 17 to the DSP 15 is (H + 1, V), the DSP 15 in the horizontal direction of “pixel a” and “pixel a” as shown in FIG. It is determined that the adjacent “pixel b” is a defective pixel.

  Similarly, when the address output from the MPU 17 to the DSP 15 is (H, V + 1), the vertical direction of “pixel a” and “pixel a” in the DSP 15 as shown in FIG. It can be determined that the “pixel b” adjacent to is a defective pixel.

  As shown in FIGS. 3B and 3C, one pixel adjacent to the representative pixel in the horizontal direction or the vertical direction (two pixels continuous with the representative pixel) is a defective pixel. In this case, it is preferable that it is possible to clearly indicate on which side the pixel adjacent to the representative pixel is a defective pixel.

  Therefore, in the case where the address is demodulated in the MPU 17, only when the “pixel b” that is the defective pixel is adjacent to the right side in the horizontal direction of the “pixel a” that is the representative pixel, (H + 1, V) is used as an address, and (H, V + 1) described with reference to FIG. 3C is used only when “pixel b” that is a defective pixel is adjacent to “pixel a” that is a representative pixel in the vertical direction. The address is a rule in the display method of continuous defective pixels.

  In the display example shown in FIG. 3D, the representative pixel “pixel a” is defined as a mathematical quadrant (when a parallel coordinate system is defined on a plane, of the four parts defined by the coordinate axes. , X and y are positive in the first quadrant, x is negative and y is positive in the second quadrant, x and y are negative in the third quadrant, x is positive and y is negative (A certain part is shown as the fourth quadrant), “pixel a” is located in the second quadrant, “pixel c” is in the third quadrant, “pixel d” is in the fourth quadrant, and “ It can be seen that pixel b "is located in the first quadrant. Therefore, when 2 × 2 four pixels are continuous defects, “pixel b” is (H + 1, V), “pixel c” is (H, V + 1), and “pixel d” is (H + 1, V + 1), respectively. , Respectively.

  That is, based on the result of the DSP 15 receiving the defective pixel address demodulated by the MPU 17, the address (position information) of the defective pixel to be corrected can be easily specified.

  Note that adding a continuous determination flag of “2 bits” to the address of “representative pixel” described above for the continuous defective pixels limits the number of displayable defective pixels to a maximum of four (2 × 2). However, when the address of the defective pixel is demodulated by the MPU 17, the demodulation is very easy.

  In addition, by using the 2 × 2 four continuous defective pixel display shown in FIG. 3D, even when any one of the four continuous pixels is not defective, it can be displayed as a continuous defective pixel. is there. That is, in the case where there are substantially three defective pixels, the address is displayed (stored in the MPU 17) as 2 × 2 four consecutive defective pixels, thereby further increasing the capacity of the memory holding the defective pixel address. Can be reduced.

  As described above, by taking one embodiment of the present invention, the addresses of defective pixels included in the image sensor can be recorded with high density. Further, the address of the recorded defective pixel can be easily demodulated.

  Therefore, it is possible to obtain an image pickup apparatus that can record the address of the defective pixel without unnecessarily occupying a large amount of memory and can easily demodulate the address during reproduction.

  In addition, this invention is not limited to each embodiment mentioned above, A various deformation | transformation or change is possible in the range which does not deviate from the summary in the stage of the implementation. In addition, the embodiments may be implemented by appropriately combining them as much as possible, or by deleting a part thereof. In that case, various effects resulting from the combination or deletion can be obtained.

  DESCRIPTION OF SYMBOLS 1 ... Imaging device, 11 ... Lens, 13 ... Image sensor, 15 ... Image processing circuit module (DSP), 17 ... MPU (control module and memory unit), 17a ... Synchronous signal generation part (Sync), 19 ... Clock oscillator ( OSC), 21... Output interface (IF), 23.

Claims (13)

An image sensor that photoelectrically converts input information;
A defective pixel detection module for detecting a defective pixel included in the image sensor from an output of each pixel of the image sensor;
It is detected that the defective pixel of the image sensor supplied from the defective pixel detection module is continuous with an adjacent pixel, and based on positional information of the leading pixel and 2-bit determination data when the defective pixel is adjacent A defective pixel position storage module for storing the position of the defective pixel;
An imaging device comprising:   The defective pixel position storage module is configured such that only the top pixel when the defective pixel is adjacent to each other, two pixels adjacent to the right in the horizontal direction of the top pixel and the top pixel, and the vertical direction between the top pixel and the top pixel. The two adjacent pixels below, and the pixel adjacent to the right adjacent pixel in the horizontal direction of the first pixel and the pixel adjacent to the pixel adjacent in the lower vertical direction of the first pixel are specified by the 2-bit determination data. The imaging apparatus according to claim 1.   The defective pixel position storage module, when the first pixel of the defective pixel is located in the second quadrant according to the description of the mathematical quadrant, four defective pixels continuous in the first quadrant, the third quadrant, and the fourth quadrant, respectively. The imaging apparatus according to claim 2, wherein the imaging apparatus is capable of displaying.   The defective pixel position storage module is adjacent to the defective pixel detection module in which the defective pixel is adjacent to the right side in the horizontal direction of the first pixel and only the first pixel when the defective pixel is adjacent to the defective pixel. Two pixels that are adjacent to each other below the first pixel in the vertical direction of the first pixel, and a pixel that is adjacent to the right of the first pixel in the horizontal direction and a pixel that is adjacent to the lower side in the vertical direction of the first pixel. The imaging device according to claim 1, wherein the output is continuous in any state of four pixels each composed of pixels in contact with each other.   The defective pixel position storage module, when the first pixel of the defective pixel is located in the second quadrant according to the description of the mathematical quadrant, four defective pixels continuous in the first quadrant, the third quadrant, and the fourth quadrant, respectively. The imaging apparatus according to claim 4, wherein the imaging apparatus is capable of displaying. An image sensor that photoelectrically converts input information;
A defective pixel detection module for detecting a defective pixel included in the image sensor from an output of each pixel of the image sensor;
When the defective pixel of the image sensor supplied from the defective pixel detection module is continuous with an adjacent pixel, positional information of a leading pixel and a plurality of bits of determination data when the defective pixel is adjacent A defective pixel position storage module for storing the position of the defective pixel by:
An imaging device comprising:   The defective pixel position storage module includes a first predetermined number of pixels adjacent to the right in the horizontal direction of the first pixel and the first pixel, only the first pixel when the arrangement of the defective pixels is adjacent, and the first pixel A second predetermined number of pixels adjacent below the first pixel in the vertical direction and a second predetermined number of pixels adjacent below the first predetermined number of pixels adjacent to the right in the horizontal direction of the first pixel. The imaging apparatus according to claim 6, wherein an arrangement of defective pixels in the pixel is stored by the determination data of the plurality of bits.   The defective pixel position storage module includes a first predetermined number of pixels adjacent to the first pixel in the horizontal direction of the first pixel and only the first pixel when the defective pixel is adjacent to the defective pixel. , The second predetermined number of pixels adjacent to the top pixel in the vertical direction below the top pixel, and the first predetermined number of pixels adjacent to the right in the horizontal direction of the top pixel. The imaging apparatus according to claim 6, wherein in which state the arrangement of defective pixels among the predetermined number of pixels is continuous is output. An image sensor that photoelectrically converts input information;
If a defective pixel included in the image sensor is detected in advance and the previously detected defective pixel of the image sensor is continuous with an adjacent pixel, the first pixel in the case where the defective pixel is adjacent is detected. A defective pixel position storage module for storing the position of the defective pixel according to position information and determination data of a plurality of bits;
An imaging device comprising:   The defective pixel position storage module includes a first predetermined number of pixels adjacent to the right in the horizontal direction of the first pixel and the first pixel, only the first pixel when the arrangement of the defective pixels is adjacent, and the first pixel A second predetermined number of pixels adjacent below the first pixel in the vertical direction and a second predetermined number of pixels adjacent below the first predetermined number of pixels adjacent to the right in the horizontal direction of the first pixel. The imaging apparatus according to claim 9, wherein an arrangement of defective pixels in a pixel is stored by the plurality of bits of determination data. Furthermore, it has an image processing module,
The defective pixel position storage module is adjacent to the image processing module in which the defective pixel is adjacent to the right in the horizontal direction of the first pixel and only the first pixel when the defective pixel is adjacent. A first predetermined number of pixels, a second predetermined number of pixels adjacent to the first pixel in the vertical direction below the first pixel, and a first predetermined number of pixels adjacent to the right of the first pixel in the horizontal direction The imaging apparatus according to claim 9, which outputs in which state the arrangement of defective pixels in a second predetermined number of pixels adjacent below each other is continuous. Detect defective pixels included in the image sensor from the output of each pixel of the image sensor,
Detecting that the detected defective pixel of the image sensor is continuous with the adjacent pixel,
A signal processing method characterized by storing a position of a defective pixel by position information of a leading pixel and a plurality of bits of determination data when the defective pixel is adjacent.   13. The signal processing method according to claim 12, wherein the multi-bit determination data is 2-bit data.

Images