JP5446955B2 - Imaging device - Google Patents

Description

  The present invention relates to an imaging apparatus and an imaging method, and more particularly to an imaging apparatus such as a digital still camera having a defective pixel correction function and an imaging method thereof.

In an image pickup device used for a digital still camera, for example, a CCD (charge coupled device), not all pixels are manufactured with the same performance in a semiconductor manufacturing process, and defective pixels are also manufactured. Examples of the defective pixel include a black spot defective pixel that outputs a signal having a level lower than normal, and a bright spot defective pixel that outputs a signal having a level higher than normal.
However, from the viewpoint of product yield, when the number of defective pixels is less than a predetermined number, it is generally used as a non-defective CCD for a product such as a digital still camera.

  Therefore, in a digital still camera using a CCD having a defective pixel, defective pixel data such as ordinate and abscissa (address) on the CCD of the defective pixel is stored in advance in a storage unit of a defective pixel correction unit provided in the digital still camera. Stored in The defective pixel data stored in the storage unit is referred to by the defective pixel correction unit at the time of photographing. Then, the pixel data located at the address of the defective pixel data is interpolated or replaced with the pixel data (image signal) of the same adjacent color filter, and defective pixel correction processing is performed.

  In addition to the aforementioned black spot defective pixels and bright spot defective pixels, there are also defective pixels (temperature scratches) with poor dark current characteristics. A defective pixel having poor dark current characteristics is not a problem with a normal exposure time of about 1 second, but is a defective pixel in which the increase in dark current increases as the exposure time increases, and this is superimposed on the image data. It becomes a bright spot. For example, when shooting with long exposure in night scene shooting, noise appearing in a star shape is a defective pixel having poor dark current characteristics and appears as a bright spot on the image. As described above, since defective pixels with poor dark current characteristics do not always appear as defective pixels, it is necessary to change the pixels to be corrected (and the number thereof) according to imaging parameters such as exposure time. Corresponding defective pixels are stored and stored and used for correction.

  In other words, the defective pixel is preliminarily grasped by storing and storing the defective pixel in advance and performing the same processing on the stored and stored defective pixel every time shooting is performed, or by performing predetermined processing according to the shooting parameter. The defective pixel can be corrected with the image signal of the surrounding pixels (static defective pixel correction).

  By the way, in the defective pixel, there is a temperature defective pixel that dynamically changes, as in the example of the defective pixel having poor dark current characteristics. For such temperature-defective pixels that change dynamically, each pixel is automatically determined based on a predetermined threshold value from information (luminance value, etc.) of the image signal obtained for each shooting, and determined as a defective pixel. A pixel can be corrected with an image signal of a pixel around the defective pixel (dynamic defective pixel correction).

  However, when these static defective pixel correction and dynamic defective pixel correction are used, the pixels stored / stored as defective pixels or the periphery of the pixels determined to be defective based on automatic determination are also defective pixels. In some cases (for example, in the case of continuous defective pixels), there is a problem that correction is not performed correctly.

In order to correct such a problem, for example, in Patent Document 1, in order to correct a continuous defective pixel, a part of the continuous defective pixel is corrected using static defective pixel correction to be an isolated point (non-continuous defective pixel). Next, a technique for properly correcting all defective pixels by performing dynamic defective pixel correction is disclosed. According to Patent Document 1, even a defective pixel having a pattern that cannot be corrected sufficiently only by dynamic defective pixel correction or static defective pixel correction can be corrected by using both of them.
However, even with such a technique, there is a problem that a defective pixel having a specific pattern, such as a defective pixel having four or more horizontal rows or a defective pixel having three horizontal rows × two vertical rows, cannot be sufficiently corrected. There is a demand for higher accuracy in defective pixel correction. In addition, since such a technique uses dynamic defective pixel correction for correction after an isolated point, for example, when correcting defective pixels in three horizontal directions, the central defective pixel is corrected by interpolation. Therefore, there is a case where a good resolution cannot be obtained due to the necessity of, for example.

  The present invention has been made in view of the above-described problems in the prior art, and can correct a defective pixel due to a defect in an image sensor with high accuracy, and in particular, correct a defective pixel with three or more continuous defects with high accuracy. An object of the present invention is to provide an imaging apparatus and an imaging method capable of performing the above.

  In order to solve the above problems, the imaging apparatus and imaging method according to the present invention specifically have the technical features described in the following (1) to (14).

(1): An image pickup device having a pixel column, a defective pixel storage unit that stores in advance a position of a defective pixel included in the pixel column, and a defective pixel correction unit that corrects the defective pixel. , At least 0th, 1st,..., N−1th, Nth, N + 1th, pixels arranged in a predetermined direction, and the defective pixel storage means Among them, the 1st to Nth pixels are defined as defective pixels, the 0th pixel and the (N + 1) th pixel in the pixel column are defined as non-defective pixels, and these are stored as a specific defective pixel pattern. The image signal of the defective pixel of the 1st to Mth defective pixels among the 1st to Nth is corrected with the image signal of the 0th non-defective pixel with respect to the specific defective pixel pattern, and the M + 1 to Nth defective pixels are corrected. N + Th corrected image signal of the non-defective pixel, imaging and selects the order of correction by referring to another pixel values arranged vertically with respect to the pixel rows having the specific defective pixel pattern Device.
However, N is an integer greater than or equal to 3, and M is an integer greater than or equal to 1 and less than or equal to N-1.

(2): The defective pixel correction unit includes a correction target pixel registration unit that registers the mth pixel and the m + 1th defective pixel as correction target pixels, and outputs an image signal of the mth defective pixel to m The imaging apparatus according to (1), wherein the image signal of the −1st pixel is corrected, and the image signal of the m + 1th defective pixel is corrected with the image signal of the m + 2th pixel.
However, m = 1, 2,..., N−1.

(3): The correction target pixel registration unit includes, among the defective pixels that are not corrected with the image signal of the 0th non-defective pixel or the image signal of the (N + 1) th non-defective pixel in the 1st to Nth defective pixels. , Register two consecutive defective pixels arranged at both ends in the arrangement direction of the uncorrected defective pixels as correction target pixels, and the defective pixel correcting means includes two consecutive pixels arranged at one end. The imaging apparatus according to (2), wherein the defective pixel is corrected and two consecutive defective pixels arranged at the other end are corrected.

(4): The defective pixel correcting means repeatedly corrects the image signal of the first pixel from the image signal of the first pixel to the image signal of the Mth pixel in order, and from the image signal of the Nth pixel to the image signal of the M + 1th pixel in order. The imaging apparatus according to any one of (1) to (3), wherein correction is performed repeatedly.

(5): The defective pixel correcting unit cuts out an image signal of the specific defective pixel pattern from the image signal of the pixel column to form an image block, and corrects the image block. (1) It is an imaging device given in any 1 paragraph of (4).

(6): The imaging device has a plurality of pixel columns, and the defective pixel correction unit determines M with reference to another pixel column adjacent to the pixel column having the specific defective pixel pattern. The imaging apparatus according to any one of (1) to (5), characterized in that:

(7): A luminance value difference is equal to or greater than a predetermined threshold value compared to surrounding pixels, and a discontinuous isolated point consisting of a single pixel is detected, and the isolated point is interpolated with pixels around the isolated point. The imaging apparatus according to any one of (1) to (6), further including dynamic defective pixel correction means that performs correction by performing correction.

(8): An imaging method using an imaging device including an imaging device, wherein the imaging device has a pixel row, and the pixel row is at least 0th, 1st,..., N−1. A defective pixel storing step for storing in advance the positions of defective pixels included in the pixel column, including pixels arranged in a predetermined direction in order of the Nth, Nth, and N + 1th orders; and defective pixel correction for correcting the defective pixels And the defective pixel storing step includes 1st to Nth pixels in the pixel column as defective pixels, and 0th pixel and N + 1th pixel in the pixel columns as non-defective pixels. Are stored as a specific defective pixel pattern, and the defective pixel correction step outputs an image signal of the 1st to Mth defective pixels among the 1st to Nth defective pixels with respect to the specific defective pixel pattern. Compensate with image signal of defective pixel And corrects the image signal of M + 1 to N-th defective pixel in the image signal of the (N + 1) -th non-defective pixels, reference another pixel values arranged vertically with respect to the pixel rows having the specific defective pixel pattern This is an imaging method characterized in that the order of correction is selected .
However, N is an integer greater than or equal to 3, and M is an integer greater than or equal to 1 and less than or equal to N-1.

(9): The defective pixel correction step includes a correction target pixel registration step of registering the mth pixel and the m + 1th defective pixel as a correction target pixel, and an image signal of the mth defective pixel as the m−1th pixel. And a target pixel correction step of correcting the image signal of the m + 1-th defective pixel with the image signal of the m + 2-th pixel by correcting the image signal of the pixel with the image signal of the pixel. It is.
However, m = 1, 2,..., N−1.

(10): In the correction target pixel registration step, among the 1st to Nth defective pixels, among the defective pixels that are not corrected by the image signal of the 0th non-defective pixel or the image signal of the N + 1th non-defective pixel , Register two consecutive defective pixels arranged at both ends in the arrangement direction of the uncorrected defective pixels as correction target pixels, and the target pixel correction step includes two consecutive pixels arranged at one end. The imaging method according to (9), wherein the defective pixel is corrected and at the same time, two consecutive defective pixels arranged at the other end are corrected.

(11): The defective pixel correction step repeatedly corrects the image signal of the first pixel from the image signal of the Mth pixel in order and corrects the image signal of the M + 1th pixel in order from the image signal of the Nth pixel. The imaging method according to any one of the above (8) to (10), wherein the correction is repeatedly performed.

(12): In the defective pixel correction step, the image signal of the specific defective pixel pattern is cut out from the image signal of the pixel row to form an image block, and the image block is corrected. (8) The imaging method according to any one of (11).

(13): The imaging device has a plurality of pixel columns, and the defective pixel correction step determines M with reference to another pixel column adjacent to the pixel column having the specific defective pixel pattern. The imaging method according to any one of (8) to (12), characterized in that:

(14): A luminance value difference is equal to or greater than a predetermined threshold value compared with surrounding pixels, and a discontinuous isolated point consisting of a single pixel is detected, and the isolated point is interpolated with a pixel around the isolated point The imaging method according to any one of (8) to (13), further including a dynamic defective pixel correction step of performing correction.

  ADVANTAGE OF THE INVENTION According to this invention, the imaging device and imaging method which can correct | amend the defective pixel by the defect of an imaging device with high precision can be provided.

It is explanatory drawing which shows the digital still camera as one Embodiment of the imaging device which concerns on this invention, (a) is a front view, (b) is a top view, (c) is a rear view. It is a block diagram which shows the outline | summary of the system configuration | structure of the digital still camera of FIG. It is explanatory drawing for demonstrating an example of the static defective pixel correction | amendment in a single defective pixel. It is explanatory drawing for demonstrating an example of the static defective pixel correction | amendment in the defective pixel which continued 2 times in the horizontal direction. It is explanatory drawing for demonstrating an example of the static defective pixel correction | amendment in the 3 continuous defective pixels in a horizontal direction. It is explanatory drawing for demonstrating another example of the static defective pixel correction | amendment in the 3 continuous defective pixels in a horizontal direction. It is explanatory drawing for demonstrating an example of the static defective pixel correction | amendment in the 4 continuous defective pixels in a horizontal direction. It is explanatory drawing for demonstrating the other example of the static defective pixel correction | amendment in the defective pixel which continued 4 directions in the horizontal direction. It is explanatory drawing for demonstrating another example of the static defective pixel correction | amendment in the defective pixel which continued 4 times in the horizontal direction. It is explanatory drawing for demonstrating an example of the static defective pixel correction | amendment in the 5 continuous defective pixels in a horizontal direction. It is explanatory drawing for demonstrating the other example of the static defective pixel correction | amendment in the 5 continuous defective pixels in a horizontal direction. It is a flowchart of correction | amendment of a continuous defective pixel. It is explanatory drawing which shows the processing flow at the time of using all pixel data when correcting 3 continuous defective pixels. It is explanatory drawing which shows the processing flow at the time of using the pixel block cut out when correcting 3 continuous defective pixels.

An imaging apparatus according to the present invention includes an imaging device having a pixel column, a defective pixel storage unit that stores in advance a position of a defective pixel included in the pixel column, and a defective pixel correction unit that corrects the defective pixel, The pixel column has at least pixels arranged in a predetermined direction in the order of 0th, 1st,..., N−1th, Nth, N + 1th, and the defective pixel storage means includes: The 1st to Nth pixels in the pixel column are defined as defective pixels, the 0th pixel and the N + 1th pixel in the pixel column are defined as non-defective pixels, and these are stored as a specific defective pixel pattern, and the defective pixel correction is performed. The means corrects the image signal of the defective pixel of the 1st to Mth defective pixels out of the 1st to Nth with respect to the specific defective pixel pattern with the image signal of the 0th non-defective pixel, and M + 1 to Nth Picture of defective pixels And correcting the signal in the image signal of the (N + 1) -th non-defective pixel.
However, N is an integer greater than or equal to 3, and M is an integer greater than or equal to 1 and less than or equal to N-1.

The imaging method according to the present invention uses an imaging device including an imaging device, and the imaging device has a pixel column, and the pixel column is at least 0th, 1st,..., N−1th. , Nth, N + 1th pixel including pixels arranged in a predetermined direction, a defective pixel storing step for storing in advance the positions of defective pixels in the pixel column, and a defective pixel correcting step for correcting the defective pixels And the defective pixel storing step includes 1st to Nth pixels in the pixel column as defective pixels, and 0th pixel and N + 1th pixel in the pixel columns as non-defective pixels. The defective pixel correction step stores the image signal of the 1st to Mth defective pixels among the 1st to Nth defective pixels with respect to the specific defective pixel pattern. With pixel image signal Correct, and correcting the M + 1 through the image signal of the N-th defective pixel in the image signal of the (N + 1) -th non-defective pixel.
However, N is an integer greater than or equal to 3, and M is an integer greater than or equal to 1 and less than or equal to N-1.

Next, the imaging apparatus and imaging method according to the present invention will be described in more detail.
Although the embodiments described below are preferred embodiments of the present invention, various technically preferable limitations are attached thereto, but the scope of the present invention is intended to limit the present invention in the following description. Unless otherwise described, the present invention is not limited to these embodiments.

<< First Embodiment >>
First, prior to describing processing and configuration relating to correction of defective pixels in the present invention, a configuration of a digital still camera as an embodiment of an imaging apparatus according to the present invention will be described in detail with reference to the drawings.

<Digital still camera>
FIG. 1 is an explanatory view showing a digital still camera (hereinafter referred to as “digital camera 1”) as an embodiment of an imaging apparatus according to the present invention, (a) is a front view, and (b) is a front view. It is a top view, (c) is a rear view. FIG. 2 is a block diagram showing an outline of the system configuration of the digital camera 1 of FIG.

(Appearance structure of digital camera)
In the digital camera 1 of the present embodiment, as shown in FIG. 1, a release button (shutter button) 2, a power button 3, and a shooting / playback switching dial 4 are provided on the upper surface side, and on the front (front) side, A lens barrel unit 6 having a photographing lens system 5, a strobe light emitting unit (flash) 7, and an optical viewfinder 8 are provided.

  In the digital camera 1, on the rear side, a liquid crystal monitor (LCD) 9, an eyepiece 8a of the optical viewfinder 8, a wide-angle zoom (W) switch 10, a telephoto zoom (T) switch 11, a menu (MENU) A button 12, a confirmation button (OK button) 13, and the like are provided. Inside the side surface of the digital camera 1 is provided a memory card storage unit 15 for storing a memory card 14 (see FIG. 2) for storing captured image data. Note that the appearance of the imaging device according to the present invention is not necessarily limited to the present embodiment, and may have another appearance.

(Digital camera system configuration)
As shown in FIG. 2, the digital camera 1 includes a CCD (charge coupled device) 20 and a CCD 20 as a solid-state imaging device on which a subject image incident through a photographing lens system 5 of the lens barrel unit 6 forms an image on a light receiving surface. An analog front end unit 21 (hereinafter referred to as “AFE unit 21”) that processes the output electrical signal (analog RGB image signal) into a digital signal, a signal processing unit 22 that processes the digital signal output from the AFE unit 21, An SDRAM 23 that temporarily stores data, a ROM 24 that stores a control program, a motor driver 25 that drives the lens barrel unit 6, and a memory that can store and read data appropriately by a control unit 28 described later. A RAM 44 and the like are provided.

  The taking lens system 5 includes a zoom lens, a focus lens, and the like. The lens barrel unit 6 includes a photographic lens system 5, an aperture unit 26, and a mechanical shutter unit 27. The drive units of the photographic lens system 5, the aperture unit 26, and the mechanical shutter unit 27 are driven by a motor driver 25. . The motor driver 25 is driven and controlled by a drive signal from a control unit (CPU) 28 of the signal processing unit 22.

  The CCD 20 is a solid-state imaging device that converts a formed subject image into an electrical signal (image data, image signal) and outputs the electrical signal. The CCD 20 has an entire light receiving surface divided into grid-like areas called pixels (hereinafter also referred to as pixels 20a), and converts image data composed of pixel data, which is digital data, into electrical signals. Output as. In the CCD 20, color filters (RGB, CYM, etc.) are provided in each divided area (pixel) so as to form a Bayer array. In this embodiment, an RGB primary color filter (hereinafter referred to as “RGB”) having a Bayer array is provided. Filter "). Therefore, the CCD 20 outputs an electrical signal (analog RGB image signal) corresponding to the three primary colors of RGB from each pixel 20a. The CCD 20 can also be configured with a CMOS image sensor or the like as a solid-state image sensor.

  The AFE unit 21 is sampled by a TG (timing signal generating unit) 30 that drives the CCD 20, a CDS (correlated double sampling unit) 31 that samples an electrical signal (analog RGB image signal) output from the CCD 20, and the CDS 31. An AGC (analog gain control unit) 32 that adjusts the gain of the signal, and an A / D conversion unit 33 that converts the signal gain-adjusted by the AGC 32 into a digital signal (hereinafter referred to as “RAW-RGB data”).

  The signal processing unit 22 includes a CCD interface (hereinafter referred to as “CCD I / F”) 34, a memory controller 35, a YUV conversion unit 36, a resizing processing unit 37, a display output control unit 38, and a data compression unit 39. , A media interface (hereinafter referred to as “media I / F”) 40, a defective pixel correction unit 42, and a control unit (CPU) 28.

  The CCD I / F 34 outputs a screen horizontal synchronization signal (HD) and a screen vertical synchronization signal (VD) to the TG 30 of the AFE unit 21, and from the A / D conversion unit 33 of the AFE unit 21 according to these synchronization signals. The output RAW-RGB data is captured. The memory controller 35 controls the SDRAM 23. The YUV converter 36 converts the RAW-RGB data captured by the CCD I / F 34 into image data in YUV format that can be displayed and recorded. The resizing processing unit 37 changes the image size according to the size of image data to be displayed or recorded. The display output control unit 38 controls display output of image data. The data compression unit 39 converts the data so that the image data is recorded in the JPEG format or the like. The media I / F 40 writes image data to the memory card 14 or reads image data written to the memory card 14. The defective pixel correction unit 42 corrects defective pixels in a process in which image data (image signal) acquired by the CCD 20 propagates through the signal processing unit 22. The correction of the defective pixel will be described in detail later. The control unit 28 performs system control and the like of the entire digital camera 1 based on a control program stored in the ROM 24 in accordance with operation input information from the operation unit 41.

  The operation unit 41 includes a release button 2, a power button 3, a shooting / playback switching dial 4, a wide-angle zoom switch 10, a telephoto zoom switch 11, a menu button 12, and a confirmation button 13 provided on the external surface of the digital camera 1. Etc. (see FIG. 1), and a predetermined operation instruction signal corresponding to the operation of the photographer is input to the control unit 28.

  In the SDRAM 23, the RAW-RGB data captured by the CCD I / F 34 is stored, and the YUV data (YUV format image data) converted by the YUV conversion unit 36 is stored. In addition, the data compression unit 39 The image data such as JPEG formation that has been subjected to compression processing is saved. The YUV data is the luminance data (Y), the color difference (U) that is the difference between the luminance data and the blue (B) component data, and the color difference (V) that is the difference between the luminance data and the red (R) component data. ) Information to express color.

(Digital camera monitoring and still image shooting)
Next, the monitoring operation and still image shooting operation of the digital camera 1 will be described. In the still image shooting mode, the digital camera 1 performs a still image shooting operation while executing a monitoring operation as described below.

  First, when the photographer turns on the power button 3 and sets the photographing / playback switching dial 4 to the photographing mode, the digital camera 1 is activated in the recording mode. When the control unit 28 detects that the power button 3 is turned on and the photographing / playback switching dial 4 is set to the photographing mode, the control unit 28 outputs a control signal to the motor driver 25 to allow the lens barrel unit 6 to be photographed. And the CCD 20, the AFE unit 21, the signal processing unit 22, the SDRAM 23, the ROM 24, the liquid crystal monitor 9 and the like are activated.

  Then, the photographing lens system 5 of the lens barrel unit 6 is directed toward the subject, so that the subject image incident through the photographing lens system 5 is formed on the light receiving surface of each pixel of the CCD 20. Next, an electrical signal (analog RGB image signal) corresponding to the subject image output from the CCD 20 is input to the A / D conversion unit 33 via the CDS 31 and the AGC 32, and 12 bits (bits) by the A / D conversion unit 33. To RAW-RGB data.

  This RAW-RGB data is taken into the CCD I / F 34 of the signal processing unit 22 and stored in the SDRAM 23 via the memory controller 35. The RAW-RGB data read from the SDRAM 23 is converted into YUV data (YUV signal) by the YUV conversion unit 36 and then stored as YUV data in the SDRAM 23 via the memory controller 35.

  The YUV data read from the SDRAM 23 via the memory controller 35 is sent to the liquid crystal monitor (LCD) 9 via the display output control unit 38, and the photographed image is displayed on the liquid crystal monitor (LCD) 9. At the time of monitoring in which a captured image is displayed on the liquid crystal monitor (LCD) 9, one frame is read out in a time of 1/30 seconds by the pixel number thinning process by the CCD I / F 34. For this reason, the AFE unit 21 and the signal processing unit 22 function as an image generation unit under the control of the control unit 28.

  In this monitoring operation, the photographed image is only displayed on the liquid crystal monitor (LCD) 9 functioning as an electronic viewfinder, and the release button 2 is not yet pressed (including half-pressed).

  By displaying the photographed image on the liquid crystal monitor (LCD) 9, the photographer can check the composition for photographing the still image. Note that the digital camera 1 can output a TV video signal from the display output control unit 38 and display a photographed image on an external TV (television) (not shown) via a video cable.

  Shooting is performed by pressing the shutter button 2. When it is necessary to adjust the angle of view prior to shooting, the photographer operates the wide-angle zoom switch 10 and the telephoto zoom switch 11 to zoom the zoom lens included in the shooting lens system 5 and adjust the angle of view.

  When the shutter button 2 is half-pressed, the CPU 28 performs AE / AF processing, determines the aperture value and the shutter speed, and controls the aperture unit 26 and the mechanical shutter unit 27 according to this to obtain an appropriate exposure amount.

  Thereafter, when the photographer presses down the shutter button 2, the CPU 28 starts photographing and recording processing. In other words, the CPU 28 exposes the CCD 20 by controlling the aperture unit 26 and the mechanical shutter unit 27 to control the exposure time of the CCD 20.

  The image signal output from the CCD 20 is stored in the SDRAM 23 as YUV data by the same processing as in the monitoring operation described above. (At this time, the defective pixel correction unit 42 corrects defective pixels in the process in which the image data acquired by the CCD 20 propagates through the signal processing unit 22. This will be described in detail later).

The YUV data stored in the SDRAM 23 is compressed by the data compression unit 39 in accordance with a predetermined compression format (for example, JPEG format), then stored in the SDRAM 23 again to be an image file of a predetermined image recording format, and then the memory The image is recorded on the card 14 and the imaging operation is terminated.
The image recorded on the memory card 14 as described above can be reproduced and displayed on the LCD 9.

(Defective pixel correction)
Next, processing and configuration related to correction of defective pixels in the present invention will be described.
Defective Pixel The CCD 20 as an image sensor used in the digital camera 1 includes a black spot defective pixel that outputs a signal having a level lower than normal, and a bright spot defective pixel that outputs a signal having a level higher than normal. In addition to black spot defective pixels and bright spot defective pixels, there are also defective pixels (temperature scratches) with poor dark current characteristics. These defective pixels are generally about 60 for an image sensor with 3 million pixels, and the address of each defective pixel can be stored in the digital camera 1.

  In the present invention, a pixel in which a defect occurs due to the image pickup device (CCD) 20 such as a black spot defect image, a bright spot defect pixel, or a defective pixel with poor dark current characteristics is referred to as a defective pixel. In the present invention, pixels other than the defective pixel (normal pixels) are referred to as non-defective pixels.

Defect pixel detection / storage method (defective pixel storage means)
In a digital still camera using a CCD having a defective pixel, defective pixel data such as ordinate and abscissa (address, position) on the CCD of the defective pixel is stored in advance in the defective pixel correction unit 42 provided in the digital still camera. Stored in the unit 43 by the defective pixel storage means. The defective pixel data stored in the storage unit 43 is referred to by the defective pixel correction unit 42 at the time of photographing.

[Memory of black spot defect pixels and bright spot defect pixels]
Detection / storage of defective pixels (black spot defective pixels / bright spot defective pixels) generated at room temperature can be performed, for example, by the following method.
In YUV data expressed in 8 bits (256 gradations), the AE evaluation value (brightness value) is approximately 128 in the entire image (for example, a light box as a subject, exposure time and ISO on the digital camera 1 side) The AE evaluation value is controlled by adjusting the sensitivity) from the captured image. With regard to the luminance values of all the pixels constituting the image, a pixel that falls below a predetermined black spot threshold is determined as a black spot defective pixel, and a pixel that has a pixel value that exceeds a predetermined bright spot threshold (separate from the black spot threshold) is determined as a bright spot defective pixel. To do. Then, defective pixels (black spot defective pixels / bright spot defective pixels) detected from the entire image are stored in the storage unit 43 by the defective pixel storage means.

[Memory of defective pixels with poor dark current characteristics]
In addition, detection / storage of defective pixels (temperature scratches) with poor dark current characteristics can be performed, for example, by the following method.
This is performed for a true black image taken with light shielding. A pixel having a pixel value exceeding a specific threshold is determined as a defective pixel. Then, defective pixels detected from the entire image (defective pixels with poor dark current characteristics) are stored in the storage unit 43 by the defective pixel storage means.

-Specific defective pixel pattern In the present invention, continuous defective pixels can be corrected. In particular, three or more consecutive defective pixels or more than four consecutive defective pixels are corrected by defective pixel correction means described later in detail. be able to.
Here, in the present invention, in a pixel column in which pixels are arranged in a predetermined direction, N defective pixels are consecutive, and pixels adjacent to both ends are non-defective pixels (a pattern of N + 2 pixels). Is a specific defective pixel pattern. However, N is an integer greater than or equal to 3, and may be an integer greater than or equal to 4 which is not corrected appropriately by the conventional defective pixel correction.
In other words, the specific defective pixel pattern is a pattern in a part or all of a pixel row, in which three or more consecutive defective images are arranged in a state sandwiched between non-defective images.
In the present embodiment, the pixel columns are arranged in the horizontal direction in the illustrated example. However, the present invention is not limited to such a configuration, and the pixel columns are arranged in an arbitrary direction. May be. In order to simplify the description, an example of only the horizontal direction will be described. For example, even a pixel array arranged in the vertical direction can be implemented to achieve the effects of the present invention based on the same technical idea as the present embodiment. Needless to say.

By the way, in the above-described detection / storage of the defective pixel, what is necessary for storing the defective pixel is the position (address in the defective pixel data) information of the defective pixel on the image (or on the CCD 20). In other words, since it is a coordinate in the captured image, its position coordinate is known at the time of detecting the defective pixel, so for each detected defective pixel, by comparing its position coordinate with each other, If it is a continuous defective pixel or if it is a continuous defective pixel, it is possible to know information on how many continuous defective pixels it is.
Therefore, the specific defective pixel pattern can be determined by referring to the address in the defective pixel data stored by the defective pixel storage means.

・ Defective pixel correction (static defective pixel correction)
Next, defective pixel correction processing is performed by correcting pixel data (image signal) located at the address of the defective pixel data stored in the storage unit 43 with pixel data (image signal) of the same adjacent color filter. .
That is, the defective pixel stores and stores the defective pixel in advance, and performs the same processing by the defective pixel correction unit, which will be described in detail later for each shooting, or according to the shooting parameters. By performing the predetermined process, the image signal of the defective pixel grasped in advance can be corrected with the image signal of the surrounding pixels (static defective pixel correction).

・ Specific examples of static defective pixel correction (defective pixel correction means)
Three or more consecutive defective pixels can be corrected by repeatedly performing static defective pixel correction on the input image signal in RGB or YUV format by the defective pixel correction means.

[Single defective pixel, 2 consecutive defective pixels (conventional static defective pixel correction)]
FIG. 3 is an explanatory diagram for explaining an example of static defective pixel correction in a single defective pixel. FIG. 4 is an explanatory diagram for explaining an example of static defective pixel correction in two consecutive defective pixels in the horizontal direction.
In the case of correcting a single defective pixel N1, the static defective pixel is corrected by interpolating or replacing the image signal using the non-defective pixels (normal pixels) N0 and N2 adjacent in the horizontal direction as reference pixels ( FIG. 3; however, the example shown is interpolation).
Further, in the case of correcting two consecutive defective pixels N1 and N2 in the horizontal direction, it is performed by replacing the image signal using the non-defective pixels (normal pixels) N0 and N3 adjacent in the horizontal direction as reference pixels (FIG. 4). ).
In addition, although the static defective pixel correction | amendment in the case where a non-defective pixel and a defective pixel continue in the horizontal direction was described, this is a mere example and is not restricted to a horizontal direction. That is, even when non-defective pixels and defective pixels continue in an arbitrary direction, static defective pixel correction can be performed by the same method. The same applies to the example of defective pixels of three or more consecutive lines shown below.

In the example shown in FIG. 3, since the non-defective pixels N0 and N2 adjacent to the single defective pixel N1 are arranged, the single defective pixel N1 is interpolated and corrected by interpolation of these image signals. Alternatively, correction may be performed by replacing one image signal of the non-defective pixels N0 and N2 adjacent to each other.
In the example shown in FIG. 4, the non-defective pixels N0 and N3 are arranged at positions sandwiching the two consecutive defective pixels N1 and N2, so that the two consecutive defective pixels N1 and N2 are adjacent to each other. Correction is performed by replacing one image signal of the non-defective pixels N0 and N3. (The image signal of the non-defective pixel N0 is replaced with the defective pixel N1, and the image signal of the non-defective pixel N3 is replaced with the defective pixel N2.)
At this time, when correction by interpolation is performed instead of correction by replacement, for example, the defective pixel N1 is not preferable because correction by interpolation between the non-defective pixel N0 and the defective pixel N2 is performed.

[3 consecutive defective pixels]
FIG. 5 is an explanatory diagram for explaining an example of static defective pixel correction in three consecutive defective pixels in the horizontal direction.
For example, FIG. 5 considers correction of two consecutive defective pixels that is possible with conventional static defective pixel correction, and correction of three consecutive defective pixels that is impossible with conventional static defective pixel correction.
First, at the time of the first correction process, the two consecutive defective pixels (N1, N2) on the left side are registered (corrected target pixels) by the corrected pixel registration unit. Then, the defective pixel N1 is replaced with the image signal of the non-defective pixel N0 adjacent to the left, and the defective pixel N2 is replaced with the image signal of the defective pixel N3 (step 5-1).
Next, during the second correction process, the remaining two consecutive defective pixels are registered (corrected target pixels) by the correction target pixel registration unit. Then, the defective pixel N2 (already replaced with the defective pixel N3) is replaced with the image signal of the pixel adjacent to the left (defective pixel N1 corrected with the non-defective pixel N0), and the defective pixel N3 is displayed on the right side. It is replaced with the image signal of the adjacent non-defective pixel N4 (step 5-2).
In this way, correction of three consecutive defective pixels is realized by performing two consecutive defective pixel correction processes (replacement) twice on the three consecutive defective pixels (N1 to N3) of the same color.

On the other hand, FIG. 6 shows an explanatory diagram for explaining another example of static defective pixel correction in three consecutive defective pixels in the horizontal direction.
Even in the correction of the same three consecutive defective pixels, there is a method of registering correction target pixels in the order shown in FIG. 6, and the correction results are different. Therefore, either correction method can be selected in consideration of the resolution of the result.
The order of registering the correction target pixels is selected with reference to the luminance value of another pixel column arranged vertically in the direction shown in FIGS. 5 and 6 with respect to the pixel column including the defective pixels N0 to N4. Preferably, the resolution of the image after the static image defect correction processing changes depending on the selection.

[4 consecutive defective pixels]
Next, FIG. 7 shows an explanatory diagram for explaining an example of static defective pixel correction in four consecutive defective pixels in the horizontal direction.
In FIG. 7, first, of the four consecutive defective pixels (N1, N2, N3, N4), the two defective pixels (N1, N2) on the left side are registered (corrected target pixels) by the correction target pixel registration unit.
Then, the defective pixel N1 is replaced with the image signal of the non-defective pixel N0 adjacent to the left by the first defective pixel correction process, and the defective pixel N2 is replaced with the image signal of the defective pixel N3 (step 7-1). .
Three consecutive defective pixels (N2, N3, N4) remain after the first defective pixel correction process. For the remaining three consecutive defective pixels (N2, N3, N4), the above-described three consecutive defective pixel correction methods may be applied (step 7-2, step 7-3).

FIG. 8 is an explanatory diagram for explaining another example of static defective pixel correction in four consecutive defective pixels in the horizontal direction. Further, FIG. 9 is an explanatory diagram for explaining still another example of static defective pixel correction in four consecutive defective pixels in the horizontal direction.
As can be seen from comparison with FIG. 7, when four consecutive defective pixels are corrected, the correction result can be changed by changing the order of the correction target pixels registered by the correction target pixel registration unit.
In the example shown in FIG. 7, the defective pixels N1 to N3 are replaced with the non-defective pixel N0 and the defective pixel N4 is replaced with the non-defective pixel N5, whereas in the example shown in FIG. In the example shown in FIG. 9, the defective pixels N1 and N2 are replaced with the non-defective pixels N0, and the defective pixels N3 and N4 are replaced with the non-defective pixels N0 and the defective pixels N2 to N4 are replaced with the non-defective pixels N5. Is replaced with the non-defective pixel N5.

In the example shown in FIG. 7 and FIG. 8, among the corrected defective pixels N1 to N4, the replacement ratio of the normal pixels (non-defective pixels N0 and N5) is left / right unbalanced (3: 1 or 1: 3). On the other hand, in the example shown in FIG. 9, correction is performed with a good balance (2: 2).
It is also possible to adjust the resolution of the corrected image to some extent by controlling the order of defective pixels registered in each step when correcting consecutive defective pixels. It is preferable to refer to the luminance value for the resolution, as in the case of the static defective pixel correction process for the three consecutive defective pixels described above.

[5 consecutive defective pixels]
Next, FIG. 10 is an explanatory diagram for explaining an example of static defective pixel correction in five consecutive defective pixels in the horizontal direction.
In FIG. 10, first, among the five consecutive defective pixels (N1, N2, N3, N4, N5), the two defective pixels (N1, N2) on the left side are registered by the correction target pixel registration unit (correction target pixels). To do.
Then, the defective pixel N1 is replaced with the image signal of the non-defective pixel N0 adjacent to the left by the first defective pixel correction process, and the defective pixel N2 is replaced with the image signal of the defective pixel N3 (step 10-1). .
After the first defective pixel correction process, four consecutive defective pixels (N2, N3, N4, N5) remain. For the remaining four consecutive defective pixels (N2, N3, N4, N5), the above-described four consecutive defective pixel correction methods may be applied (step 10-2, step 10-3, step 10-4).

FIG. 11 is an explanatory diagram for explaining another example of static defective pixel correction in five consecutive defective pixels in the horizontal direction.
In the case of five consecutive defective pixels (or more than five consecutive defective pixels), two consecutive defective pixels (N1, N2) on the left side of the five consecutive defective pixels and two consecutive pixels on the right side during the first defective pixel correction. Any of the defective pixels (N4, N5) thus registered can be collectively registered (corrected pixel) by the correction target pixel registration unit (step 11-1 in FIG. 11).
Then, the defective pixel N1 is replaced with the image signal of the non-defective pixel N0 adjacent to the left by the first defective pixel correction process, the defective pixel N2 is replaced with the image signal of the defective pixel N3, and the defective pixel N5 Is replaced with the image signal of the non-defective pixel N6 adjacent to the right, and the defective pixel N4 is replaced with the image signal of the defective pixel N3 (step 11-1). As a result, three consecutive defective pixels (N2, N3, N4) remain. For the remaining three consecutive defective pixels (N2, N3, N4), the above-described three consecutive defective pixel correction methods may be applied (step 11-2, step 11-3).
Compared to the example shown in FIG. 10, five consecutive defective pixels (N1, N2, N3, N4, N5) can be corrected with a process of one step fewer.
Also, in the case of five consecutive defective pixels, the resolution of the correction result can be controlled by changing the order of registration as in the case of correcting three consecutive defective pixels and correcting four consecutive defective pixels.

[N consecutive defective pixels]
It is also possible to correct six or more consecutive defective pixels by repeating the same process.
That is, a case where N defective pixels are generalized will be described below. However, N is an integer greater than or equal to 3, and may be an integer greater than or equal to 4 that is not properly corrected by conventional defective pixel correction.

  The image sensor (CCD 20) has a plurality of pixel columns each having pixels arranged in a predetermined direction. At this time, the pixel column has pixels arranged in the order of 0th, 1st,..., N−1th, Nth, N + 1th, and the 0th and N + 1th pixels are not defective pixels. It is a defective pixel (normal pixel), and all of the 1st to Nth pixels are defective pixels.

  The defective pixel storage means stores all the 1st to Nth pixels in the pixel column as defective pixels, and stores the 0th and N + 1th pixels in the pixel column as non-defective pixels (defective pixels). Is not remembered). In other words, these defective pixels and non-defective pixels are stored as the specific defective pixel pattern described above.

  The defective pixel correction means corrects a defective pixel and enables correction of a specific defective pixel pattern. At this time, when the specific defective pixel pattern does not exist, correction (static defective pixel correction processing for repeated replacement) for the specific defective pixel pattern is not performed.

  The defective pixel correction means corrects the image signal of the 1st to Mth defective pixels among the 1st to Nth defective pixels with the image signal of the 0th non-defective pixel for the specific defective pixel pattern, and M + 1 to The image signal of the Nth defective pixel is corrected with the image signal of the (N + 1) th nondefective pixel. However, N is an integer greater than or equal to 3, and M is an integer greater than or equal to 1 and less than or equal to N-1. N may be 4 or more.

More specifically, the defective pixel correction means includes a correction target pixel registration unit that registers the m-th pixel and the m + 1-th defective pixel as correction target pixels, and the image signal of the m-th pixel is the m-1th. Correction is performed with the image signal of the pixel, and the image signal of the (m + 1) th pixel is corrected with the image signal of the (m + 2) th pixel. However, m = 1, 2,..., N−1.
That is, M represents any integer from 1 to N-1, while m represents all integers from 1 to N-1. Therefore, the defective pixel correcting means corrects in all cases where m = 1, m = 2,..., M = M,.
At this time, it is needless to say that m performs correction processing while increasing one by one in order from 1 and performs correction processing by decreasing one by one from N−1 in order to achieve the effect of the present invention. Therefore, the process of replacing any two consecutive defective pixels registered as correction target pixels with a defective pixel is not performed, and the process of registering a non-defective pixel as a correction target pixel is not performed.

  In other words, at this time, the defective pixel correction unit repeatedly corrects the first pixel from the Mth pixel in order and corrects it from the Nth pixel to the M + 1th pixel in order.

  For example, the first defective pixel and the second defective pixel are registered as correction target pixels, and the Nth defective pixel and the N−1th defective pixel are registered as correction target pixels. A defective pixel correction process may be performed. That is, according to the present invention, when the 1st to Nth defective pixels are corrected, two consecutive pixels located at both ends of a defective pixel that is not corrected by replacement (a defective pixel that is not replaced by a non-defective pixel) This does not prevent the defective pixels from being registered as correction target pixels and simultaneously performing replacement correction. However, in the case of static defective pixel correction for four consecutive defective pixels, if two consecutive defective pixels at both ends are registered as correction target pixels, the correction target pixels are adjacent to each other, and an image of the correction target pixels is displayed. This is not preferable because the signal is replaced. In other words, for five or more consecutive defective pixels, two consecutive defective pixels at both ends may be registered as correction target pixels and simultaneously replaced and corrected, thereby improving the correction processing speed.

  Although the correction processing speed is improved by registering two consecutive defective pixels at both ends as correction target pixels, the resolution of the correction result is controlled by changing the registration order as described above. Therefore, the improvement in correction processing speed and the improvement in resolution do not always coincide with each other. In other words, the improvement in resolution is a pixel column parallel to the arrangement direction of the pixel column having the specific defective pixel pattern, and the registration order is appropriately changed with reference to the luminance values of the adjacent ones (M: N− This is possible by controlling the ratio of M).

-Flow of static defective pixel correction processing FIG. 12 shows a flowchart of correction of continuous defective pixels.
In order to simplify the explanation, consider a case where four consecutive defective pixels are corrected.
First, the number of continuous defective pixels held (stored) at the time of detection (defective pixel pattern determination), in this case, “4 continuous” is acquired (S1). Next, the coordinates (position of the correction target pixel) to be registered (corrected) are acquired (S2). As described above, since the resolution of the image after correction can be controlled by the order of the coordinates to be registered, the coordinates to be registered by the correction target pixel registration unit are acquired here based on the target correction result. Then, the correction target pixel is corrected based on the acquired registered coordinates (S3), and it is determined whether all defective pixels in the continuous defective pixels are corrected based on the continuous number acquired in S1 (4 consecutive defective pixels). As shown in FIGS. 7, 8, and 9, the correction requires at least three steps of correction processing (S4). If the correction is not completed, the process of changing the registered coordinates and correcting is performed until all defective pixels are corrected.

-Dynamic defective pixel correction In this embodiment, in addition to the static defective pixel correction described above, dynamic defective pixel correction may be provided.
Regarding correction of defective pixels having poor dark current characteristics, there is a method of correcting after detecting / registering (storing) dynamically as follows.
With respect to an input image signal in RGB or YUV format, a single pixel (hereinafter also referred to as an isolated point) whose level is higher or lower than the surrounding luminance value is detected, and the horizontal direction is detected. A pixel adjacent in the vertical direction is used as a reference pixel, and correction is performed by interpolation from these luminance values.

The detection of the isolated point is determined by whether or not the difference in luminance value from the surrounding pixels exceeds a predetermined threshold value. A threshold can be set. For example, an isolated point caused by the influence of a defective pixel can be detected as much as possible by lowering the threshold. However, there is a concern that the image quality is deteriorated due to overcorrection. In addition, since correction is performed with reference to surrounding pixels, there is a problem that correction cannot be performed when there are a large number of isolated points in adjacent pixels referred to for correction of isolated points.
On the other hand, by increasing the threshold value, it is possible to prevent deterioration in image quality due to overcorrection, but it is not possible to sufficiently detect isolated points caused by defective pixels, and there is concern about deterioration in image quality due to insufficient correction. Is done.
Therefore, it is necessary to carefully determine how much the threshold value for detecting an isolated point is finally set.

<< Second Embodiment >>
Next, a second embodiment of the imaging apparatus according to the present invention will be described. Except for the mode for determining whether or not the specific defective pixel pattern described below needs to be corrected, the description of the first embodiment described above is redundant and will be omitted.

The occurrence of continuous defective pixels is likely to occur under shooting conditions in which defective pixels are likely to occur, such as high ISO sensitivity and long-time shooting.
On the other hand, the correction (specific defect pixel correction) for the specific defective pixel pattern described above performs more defective pixel correction processing than usual (repetitive execution), and accordingly, additional processing time is required (approximately 70 ms for a 10M image). ).

  Here, the correction for the specific defective pixel pattern is performed by static defective pixel correction. As described above, it is necessary to store the ordinate and abscissa of the defective pixel on the image sensor (CCD) 20 in a storage area (storage unit) in the image pickup apparatus (digital still camera) in advance. At this time, a defective pixel is detected from an image photographed by high ISO / long second photographing, in which a continuous defective pixel (particularly, a specific defective pixel pattern) is likely to occur, and the coordinates (position) of the defective pixel are determined in the imaging apparatus. When registering in the storage area, information indicating whether correction for the specific defective pixel pattern is necessary is also held. (Some imaging elements 20 should not generate a specific defective pixel pattern.)

Since the correction process for the specific defective pixel pattern requires an extra time of about 70 ms compared to the normal time, refer to correction necessity information for the specific defective pixel pattern and shooting conditions (ISO sensitivity and long time setting) at the time of correction. Then, the presence / absence of correction processing for the specific defective pixel pattern is switched.
That is, the specific defective pixel pattern is determined with reference to the photographing conditions (ISO sensitivity and long time setting), and when it is determined that there is no specific defective pixel pattern, the correction (repetitive replacement for repeated replacement) is performed on the specific defective pixel pattern. (Defective pixel correction processing) is not performed.

<< Third Embodiment >>
Next, a third embodiment of the imaging apparatus according to the present invention will be described. In addition, since the description is the same as that of the first embodiment described above except for the image block cutout in the correction for the specific defective pixel pattern described below, the description is omitted.

FIG. 13 is an explanatory diagram showing a processing flow when all pixel data is used when correcting three consecutive defective pixels.
Up to now, the correction of the specific defective pixel pattern has been described with respect to the form in which the static defective pixel correction process is performed a plurality of times on the same image. However, the processing time is a concern in this form. In consideration of inputting two pieces of image data to the defective pixel correction unit, when correcting three consecutive defective pixels, the processing time is simply about twice as long as when no correction is made. This is easily considered (FIG. 13).

  On the other hand, an image block of a specific defective pixel pattern (including continuous defective pixels and non-defective pixels referred to for correction) is cut out from the entire image data, and the cut out image block is taken as one image to the defective pixel correction unit. Consider the case of input. The processing time required for the defective pixel correction unit to perform correction depends on the size of the input image data to the defective pixel correction unit. It can be said that the processing time is at a level that hardly causes a problem. FIG. 14 is an explanatory diagram showing a processing flow when a pixel block cut out when correcting three consecutive defective pixels is used.

  By the way, in the method of static defective pixel correction, correction of a single defective pixel is performed by interpolating or replacing normal pixels adjacent in the horizontal direction as reference pixels (FIG. 3). Also, correction of two consecutive defective pixels in the horizontal direction is performed by replacing the adjacent pixel in the horizontal direction as a reference pixel (FIG. 4). Further, since the reference pixels used for correction are only pixels adjacent in the horizontal direction, it is not necessary to consider defective pixels that are continuous in the vertical direction.

From the above, the correction of the specific defective pixel pattern in the horizontal direction is performed if there is input image data of an image block having a height of 1 including a continuous defective pixel to be corrected and a reference pixel for correction (non-defective pixel). Necessary and sufficient. Therefore, the case where a plurality of static defect pixel correction processes are performed on all image data has been described so far for the correction of the specific defective pixel pattern. Only an image block having a height of 1 including a defective pixel (specific defective pixel pattern) may be input to the static defective pixel correction processing unit. Although it depends on the number of consecutive defective pixels and the memory access time, the processing time is hardly required (about 10 ms size requires about 70 ms, so, for example, about 1 × 10 ⁻⁴ ms when the image block size is 1 × 12). (Fig. 14)
In the case of the present embodiment, since correction processing does not require time, it is possible to cope with continuous shooting.

DESCRIPTION OF SYMBOLS 1 Digital still camera 21 Analog front end part 22 Signal processing part 23 SDRAM
24 ROM
28 Control Unit 42 Defective Pixel Correction Unit 43 Storage Unit 44 RAM

JP 2009-130553 A

Claims (14)

An image sensor having a pixel row;
Defective pixel storage means for storing in advance the positions of defective pixels of the pixel column;
And defective pixel correction means for correcting the defective pixel,
The pixel row includes at least pixels arranged in a predetermined direction in the order of 0th, 1st,..., N−1th, Nth, N + 1th,
The defective pixel storage means uses the 1st to Nth pixels in the pixel column as defective pixels, the 0th pixel and the N + 1th pixel in the pixel column as non-defective pixels, and these as a specific defective pixel pattern. Remember,
The defective pixel correction means corrects the image signal of the 1st to Mth defective pixels among the 1st to Nth defective pixels with the image signal of the 0th non-defective pixel with respect to the specific defective pixel pattern, The image signal of the (M + 1) th to Nth defective pixels is corrected with the image signal of the (N + 1) th non-defective pixel, and another pixel value arranged above and below the pixel column having the specific defective pixel pattern is referred to. An image pickup apparatus , wherein the order of correction is selected by the above.
However, N is an integer greater than or equal to 3, and M is an integer greater than or equal to 1 and less than or equal to N-1. The defective pixel correction means includes a correction target pixel registration unit that registers the m-th pixel and the m + 1-th defective pixel as correction target pixels, and outputs an image signal of the m-th defective pixel as the (m−1) -th image signal. The imaging apparatus according to claim 1, wherein the image signal of the pixel is corrected, and the image signal of the m + 1th defective pixel is corrected with the image signal of the m + 2th pixel.
However, m = 1, 2,..., N−1. The correction target pixel registration unit corrects the correction among the defective pixels that are not corrected by the image signal of the 0th non-defective pixel or the image signal of the N + 1 non-defective pixel in the 1st to Nth defective pixels. Register two consecutive defective pixels arranged at both ends of the non-defective pixel in the arrangement direction as correction target pixels,
The defective pixel correcting means corrects two consecutive defective pixels arranged at one end and simultaneously corrects two consecutive defective pixels arranged at the other end. 2. The imaging device according to 2.   The defective pixel correcting means repeatedly corrects the image signal of the first pixel from the image signal of the first pixel to the image signal of the Mth pixel, and also repeatedly corrects the image signal of the Nth pixel to the image signal of the M + 1th pixel in order. The image pickup apparatus according to claim 1, wherein the image pickup apparatus is an image pickup apparatus.   5. The defective pixel correcting unit cuts out an image signal of the specific defective pixel pattern from image signals of the pixel column to form an image block, and corrects the image block. The imaging device according to any one of the above. The image sensor has a plurality of pixel rows,
The said defective pixel correction | amendment means determines the said M with reference to the other pixel row | line | column adjacent to the pixel row | line | column which has the said specific defective pixel pattern, The one of Claim 1 thru | or 5 characterized by the above-mentioned. Imaging device.   The difference in luminance value compared to the surrounding pixels is equal to or greater than a predetermined threshold, and a discontinuous isolated point consisting of a single pixel is detected, and the isolated point is interpolated with the surrounding pixels of the isolated point and corrected. The imaging apparatus according to claim 1, further comprising dynamic defective pixel correction means for performing the correction. An imaging method using an imaging device including an imaging element,
The image sensor has a pixel row,
The pixel row includes at least pixels arranged in a predetermined direction in the order of 0th, 1st,..., N−1th, Nth, N + 1th,
A defective pixel storage step of previously storing the position of the defective pixel included in the pixel row;
A defective pixel correction step of correcting the defective pixel,
In the defective pixel storing step, the 1st to Nth pixels in the pixel column are defined as defective pixels, the 0th pixel and the N + 1th pixel in the pixel column are defined as non-defective pixels, and these are defined as specific defective pixel patterns. Remember,
The defective pixel correction step corrects the image signal of the 1st to Mth defective pixels among the 1st to Nth defective pixels with the image signal of the 0th non-defective pixel with respect to the specific defective pixel pattern, The image signal of the (M + 1) th to Nth defective pixels is corrected with the image signal of the (N + 1) th non-defective pixel, and another pixel value arranged above and below the pixel column having the specific defective pixel pattern is referred to. An imaging method, wherein the order of correction is selected by :
However, N is an integer greater than or equal to 3, and M is an integer greater than or equal to 1 and less than or equal to N-1. The defective pixel correction step includes a correction target pixel registration step of registering the mth pixel and the m + 1th defective pixel as a correction target pixel, and an image signal of the mth defective pixel as an image signal of the m−1th pixel. The imaging method according to claim 8, further comprising: a target pixel correction step that corrects the image signal of the (m + 1) th defective pixel with the image signal of the (m + 2) th pixel.
However, m = 1, 2,..., N−1. In the correction target pixel registration step, in the 1st to Nth defective pixels, the correction is performed among defective pixels that are not corrected by the image signal of the 0th non-defective pixel or the image signal of the N + 1th non-defective pixel. Register two consecutive defective pixels arranged at both ends of the non-defective pixel in the arrangement direction as correction target pixels,
The target pixel correcting step corrects two consecutive defective pixels arranged at one end and simultaneously corrects two consecutive defective pixels arranged at the other end. 10. The imaging method according to 9.   The defective pixel correction step repeatedly corrects the image signal of the first pixel from the image signal of the Mth pixel in order, and repeats the correction from the image signal of the Nth pixel to the image signal of the M + 1th pixel in order. The imaging method according to any one of claims 8 to 10, wherein:   12. The defect pixel correcting step, wherein the image signal of the specific defective pixel pattern is cut out from the image signal of the pixel column to form an image block, and the image block is corrected. The imaging method according to any one of the above. The image sensor has a plurality of pixel rows,
The said defective pixel correction process determines said M with reference to the other pixel row | line | column adjacent to the pixel row | line | column which has the said specific defective pixel pattern, The one of Claim 8 thru | or 12 characterized by the above-mentioned. Imaging device.   The difference in luminance value compared to the surrounding pixels is equal to or greater than a predetermined threshold, and a discontinuous isolated point consisting of a single pixel is detected, and the isolated point is interpolated with the surrounding pixels of the isolated point and corrected. The imaging apparatus according to claim 8, further comprising a dynamic defective pixel correction step to be performed.