JP2004015191A - Apparatus and method for correcting flaw in solid-state imaging device - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To effectively correct continuous flaws and a single (isolated) flaw that appear during a long-term exposure, without increasing the capacity of ROM or the like that stores information on defects in a pixel of a solid-state imaging device. <P>SOLUTION: Positional information on the continuous flaws (flaws that cause a plurality of continuous defects among adjacent pixels), that are difficult be detected from a photographed image among flaws appearing during a long-term exposure, is previously registered on a ROM. Positional information on the isolated flaw (a flaw of a single pixel with respect to other ones), that appears during the long-term exposure is not to be registered. During the long time exposure, such as at night view photographing or the like, normal flaws are first corrected, and then the continuous flaws of long time are corrected, based on the registered information. After that, the isolated flaw is further detected from within the image and corrected. <P>COPYRIGHT: (C)2004,JPO

Description

[0001]
BACKGROUND OF THE INVENTION
The present invention relates to a defect correction apparatus and method for a solid-state image sensor, and is particularly applied to a digital still camera, a movie camera, an image input apparatus, and the like, and is a signal process for correcting pixel defects (scratches) that become conspicuous during long exposure. Regarding technology.
[0002]
[Prior art]
In general, when a long-time exposure such as night scene shooting is performed using a CCD solid-state imaging device (hereinafter, referred to as a CCD), a pulse-like white flaw appears in a captured image due to a signal defect due to a defect in a light receiving element, Originally, there may be an image in which white dots are scattered in a black background area having no color change. For pixel defects (usually flaws) that are problematic in the range of exposure time used in normal shooting, the addresses (coordinates) of defective pixels on a solid-state image sensor have been previously stored in a nonvolatile memory such as a ROM. In addition, a method of correcting a signal based on the defect position information at the time of photographing is used (Japanese Patent Laid-Open No. 5-68209, etc.).
[0003]
However, if the above method is applied to even a fine defective pixel that becomes conspicuous for the first time after long exposure, and the addresses of such defective pixels are all stored in a ROM or the like, the number of defective pixels to be stored increases. And not realistic. Therefore, at the time of long exposure, after correcting normal scratches by the above conventional method, an isolated point scratch in the screen (white scratches that become noticeable only during long exposure that is not corrected by the conventional method) is detected. A correction method of replacing (refilling) isolated point scratches with an average value of surrounding pixel values of the same color is performed.
[0004]
[Problems to be solved by the invention]
However, under a higher temperature environment or when exposure is performed for a longer time, such as bulb photography, pulse-like white scratches appear adjacent to each other, and the conventional method (isolation point removal) corrects the scratches. There is a problem of being unable to understand. Normally, an isolated point scratch (single scratch) of only one pixel with respect to the surroundings can be easily determined as a scratch from a captured image, and the scratch is not so conspicuous, but a plurality of adjacent scratches. Consecutive point flaws (continuous flaws) in which pixels continuously become defective are difficult to distinguish from the information of the subject itself, so that there is a problem that it is difficult to automatically discriminate from captured images and the flaws are also conspicuous.
[0005]
The present invention has been made in view of such circumstances. A solid-state imaging device capable of efficiently correcting both continuous and single defects while avoiding an increase in the capacity of a ROM or the like that stores pixel defect information. An object is to provide a defect correction apparatus and method.
[0006]
[Means for Solving the Problems]
In order to achieve the above object, the defect correction apparatus for a solid-state imaging device according to the present invention is a position of only continuous flaws caused by a plurality of consecutive defective pixels among the defects of pixels that appear during long exposure for a predetermined time or longer. Continuous flaw position information storage means for storing information, and correction of the continuous flaw by using the position information of the continuous flaw stored in the continuous flaw position information storage means and signal values of surrounding pixels Continuous scratch correction processing means, isolated point scratch detection means for detecting isolated point scratches compared to surrounding pixels for each pixel in the screen, and using the signal values of the peripheral pixels of the detected isolated point scratch An isolated point flaw correction processing unit that corrects an isolated point flaw, and the normal flaw correction processing unit, the isolated point flaw unit, and the isolated point flaw when shooting is performed by long exposure for the predetermined time or longer. It is characterized in controlling the correcting means further comprising a correction control means for performing a correction of the continuous scratch and the isolated point flaw.
[0007]
During long-time exposure such as night scene photography, pixel defects (scratches) that do not stand out in normal exposure may become apparent. According to the present invention, among such long-term scratches, continuous scratches that are difficult to be corrected by isolated point removal by image analysis are registered in advance as long-term continuous scratches in continuous scratch position information storage means such as a ROM. deep. During long exposure, long flaw correction is performed by combining continuous flaw correction processing using registered continuous flaw information and isolated point flaw correction processing by image analysis. In order to register only continuous scratches among long-time exposure scratches (that is, because location information of isolated point scratches that appear in long-time exposure is not registered), positional information of all long-time exposure scratches including isolated point scratches and continuous scratches Compared with the case of registering the scratches, it is possible to reduce the capacity of continuous scratch position information storage means (for example, a nonvolatile memory such as a ROM) that stores the position information of scratches, and without preparing a large-capacity storage area, Continuous scratches and isolated point scratches can be corrected efficiently.
[0008]
In the correction device for a solid-state image pickup device according to the present invention, further, normal flaw position information storage means for storing position information of normal flaws that appear during normal exposure within the predetermined time, and the normal flaw position information storage means And normal flaw correction processing means for correcting the normal flaw using the position information of the normal flaw stored in the signal and the signal values of the surrounding pixels, and photographing by long exposure over the predetermined time First, the normal flaw correction is performed by the normal flaw correction processing unit, then the continuous flaw correction is performed by the continuous flaw correction processing unit, and then the normal flaw is further corrected. In addition, the isolated point scratch is detected by the isolated point scratch detection unit for the image after the processing for correcting the continuous scratches, and the isolated point scratch correction unit for the detected isolated point scratch is performed. Mode of control to perform the correction of the isolated point flaw is preferred.
[0009]
According to this aspect, it is possible to efficiently correct all of normal scratches, continuous scratches and isolated point scratches without preparing a large-capacity storage area.
[0010]
In the present invention, as one aspect for correcting the signal value of the scratch pixel, an average value is obtained from the signal values of a plurality of the same type (same color) pixels existing around the defective pixel on the pixel arrangement of the solid-state imaging device, and this peripheral type There is a mode in which an average value of pixel values is replaced with a signal value of a defective pixel. Also, as an aspect of detecting isolated point scratches, each pixel constituting the image is compared with a plurality of peripheral pixels, and pixels having a luminance of a certain level difference or more for each pixel type are identified as isolated point scratches. There is a mode of determination. The range of peripheral pixels to be compared is determined for each color in relation to the arrangement structure of the color separation color filters of the solid-state imaging device. It is preferable that the “predetermined time” that is a criterion for long exposure and the threshold value that is a criterion for determination of isolated point scratches are variably set according to the sensitivity of the solid-state imaging device.
[0011]
According to still another aspect of the present invention, there is provided a defect correction method for a solid-state imaging device, which includes position information of only continuous flaws caused by a plurality of continuous defective pixels among the defects of pixels that appear during long exposure for a predetermined time or longer. In addition to registering in advance, the position information of normal scratches that appear during normal exposure within the predetermined time is registered in advance. The normal flaw is corrected using the registered normal flaw position information and the signal values of the peripheral pixels, and then the registered flaw position information and the signal values of the peripheral pixels are used. Then, the continuous flaws are corrected, and after that, the isolated point flaws are detected by comparing each pixel with surrounding pixels in the image after the normal flaw and continuous flaw correction processing, and the detected isolated point flaws are detected. of By utilizing the signal value of side pixel it is characterized by performing the correction of the isolated point flaw.
[0012]
DETAILED DESCRIPTION OF THE INVENTION
A preferred embodiment of a defect correction apparatus and method for a solid-state imaging device according to the present invention will be described below in detail with reference to the accompanying drawings.
[0013]
FIG. 1 is a block diagram of an electronic camera to which a solid-state imaging device defect correction apparatus and method according to an embodiment of the present invention is applied. The camera 10 is a single-plate digital camera, and light that has passed through the photographing lens 12 and the shutter-use diaphragm mechanism 14 is imaged on the light receiving surface of the imaging device 16. The mechanical shutter prevents light from entering the imaging device 16 and generating smears when reading a signal from the imaging device 16. The diaphragm mechanism adjusts the amount of light incident on the imaging device 16.
[0014]
In this example, a CCD is used as the imaging device 16, but the device is not limited to the CCD type, and a device of another type such as a CMOS type may be applied. A large number of light receiving elements (photodiodes) are two-dimensionally arranged on the light receiving surface of the imaging device 16, and a color separation color filter is provided corresponding to each light receiving element. The subject image formed on the light receiving surface of the imaging device 16 is converted into a signal charge of an amount corresponding to the amount of incident light by each light receiving element. The signal charges accumulated in the respective light receiving elements in this way are read out to the transfer path by the read gate pulse applied from the drive circuit 18 and sequentially output as voltage signals (image signals) corresponding to the signal charges. The imaging device 16 has an electronic shutter function that controls the charge accumulation time (shutter speed) according to the timing of the shutter gate pulse.
[0015]
The drive circuit 18 includes a timing generator that generates a timing signal. The drive circuit 18 supplies a drive signal to the imaging device 16 in accordance with a command from the central processing unit (CPU) 20, and also provides a synchronization signal to the imaging device 16, the analog signal processing unit 22, and the like. give. The drive circuit 18 functions as a driver circuit for the imaging device 16 and at the same time functions as a drive circuit for operating the photographing lens 12, the shutter / aperture mechanism 14 and the strobe (flash device) 24. The strobe 24 can emit light automatically or when necessary, for example, at low illuminance, and irradiates the subject with auxiliary light.
[0016]
The imaging device 16 is driven based on the timing signal generated by the timing generator and outputs an image signal. The image signal output from the imaging device 16 is sent to the analog signal processing unit 22. The analog signal processing unit 22 includes a sampling hold circuit, a color separation circuit, a gain adjustment circuit, and the like. The image signal input to the analog signal processing unit 22 is subjected to correlated double sampling (CDS) processing, color separation processing into R, G, and B color signals, and signal level adjustment (pre-white balance processing) of each color signal is performed. Is called.
[0017]
The signal generated by the analog signal processing unit 22 is converted into a digital signal by the A / D converter 26 and then temporarily stored in the memory 30 via the bus (main bus inside the camera) 28. A part of the storage area of the memory 30 is also used as a calculation work area for the CPU 20.
[0018]
The image data stored in the memory 30 is sent to the digital signal processing unit 32 via the bus 28. The digital signal processing unit 32 performs correction (hereinafter referred to as defective pixel correction) processing, white balance processing, gamma conversion processing, and synchronization processing (single-plate imaging device) for interpolating defective pixel (scratch) data of the imaging device 16. Processing to calculate the color of each point by interpolating the spatial shift of the color signal due to the color filter array), luminance / color difference signal generation (YC conversion) processing, contour enhancement (aperture addition) processing, sharpness correction processing, This is a signal processing means for performing data compression / decompression processing, etc., and processes image signals while utilizing the memory 30 in accordance with commands from the CPU 20.
[0019]
The image data input to the digital signal processing unit 32 is subjected to predetermined processing such as defective pixel correction and YC conversion, and is then compressed according to a predetermined compression format such as JPEG format, via the memory card interface unit 34. It is recorded on the memory card 36. The compression format is not limited to JPEG, MPEG and other methods may be adopted, and a compression engine corresponding to the compression format used is used.
[0020]
The means for storing the image data is not limited to the semiconductor memory represented by the memory card 36, and various media such as a magnetic disk, an optical disk, and a magneto-optical disk can be used. Further, the recording medium (internal memory) built in the camera 10 is not limited to the removable medium.
[0021]
In the camera 10, raw image data that has not undergone image processing such as gamma conversion, synchronization, and YC conversion (CCD-RAW data in which only defective pixel correction is performed after A / D conversion) is stored in the memory card 36. A recording mode (RAW data recording mode) may be added.
[0022]
In the reproduction mode, image data is read from the memory card 36, decompressed by the digital signal processing unit 32, converted into a display signal, and output to the image display unit 38. A display device such as a liquid crystal monitor or an organic EL can be used for the image display unit 38. The image display unit 38 is also used as a display screen for a user interface.
[0023]
The camera 10 also includes a communication connection for transmitting and receiving data to and from a personal computer and other external devices, or a communication / option interface unit 40 for connecting an external option device. For the communication / option interface unit 40, for example, various wired or wireless wireless interfaces such as USB, IEEE 1394, and Bluetooth can be applied.
[0024]
The CPU 20 is a control unit that performs overall control of the camera system according to a predetermined program, and controls the operation of each circuit in the camera 10 based on input signals from the shutter switch 42 and other operation switches 44. The operation switches for the user to input various instructions to the camera 10 include, for example, a mode selection switch for selecting the operation mode of the camera 10, a menu switch for displaying a menu, and a menu item selection operation (cursor movement). Operation) and four-way key for inputting instructions such as frame advance / reverse of the playback image, execution key for instructing selection (registration) and execution of the selected item, and selection item, etc. There are a cancel key, a power switch, a zoom switch, a release switch, etc.
[0025]
The CPU 20 controls the imaging device 16 in accordance with various imaging conditions (exposure conditions, presence / absence of strobe light emission, imaging mode, etc.) in accordance with instruction signals input from the shutter switch 42 and the operation switch 44, etc., and automatic exposure (AE). Control, automatic focus adjustment (AF) control, auto white balance (AWB) control, lens drive control, image processing control, read / write control of the memory card 36, display control of the image display unit 38, communication control with external devices, etc. .
[0026]
The ROM 46 stores a program processed by the CPU 20 and various data necessary for control (position information of defective pixels, a threshold for determining scratches, adjustment value data, etc.). The ROM 46 as the nonvolatile storage means may be non-rewritable or may be rewritable like an EEPROM.
[0027]
Next, a method for correcting defective pixels of the imaging device 16 in the camera 10 configured as described above will be described.
[0028]
In the camera 10 of this embodiment, the pixel defect of the imaging device 16 is removed by combining three types of correction processing.
[0029]
The first correction process is intended to correct normal scratches that have been conventionally performed. That is, for normal flaws that appear in normal exposure, the flaw address (defective pixel coordinates) is stored in the ROM 46 in advance, and the signal value (pixel value) of the pixel at the corresponding address at the time of shooting is the signal of the surrounding same color pixel. Correction is performed by replacing with a representative value calculated from the value (for example, an average value of four pixels of the same surrounding color).
[0030]
For example, the data of the light shielding state is captured on the imaging device 16, and a pixel having a signal value equal to or higher than a predetermined threshold is scratched, and the scratch coordinates are recorded in the ROM 46 of the camera 10 (white scratches in the dark). Registration). Further, data is captured by irradiating the imaging device 16 with a certain amount of light, and pixels having a difference equal to or greater than a predetermined threshold are also scratched, and the scratch coordinates are recorded in the ROM 46 of the camera 10 (registration of bright modulation scratches). ). Note that the threshold value for determining dark white scratches and the threshold value for determining bright modulation scratches are set separately.
[0031]
In this way, the coordinates of dark white scratches and bright modulation scratches are registered in the ROM 46 in advance, and at the time of shooting, the output signal of the imaging device 16 is A / D converted and taken into the memory 30, and then stored in the ROM 46. The defect is corrected by performing the process of interpolating and refilling the defect pixel of the coordinates recorded in the above with the surrounding pixels of the same color.
[0032]
The second correction process is intended to correct continuous point flaws (hereinafter referred to as long-time continuous flaws) that occur during long-time exposure such as night scene photography. For this long-time continuous flaw, a representative value obtained by storing the flaw address in the ROM 46 in advance and calculating the signal value of the pixel at the corresponding address from the signal value of the surrounding same color pixel at the time of shooting by long exposure. Correction for replacement with (for example, an average value of four pixels in the surrounding same color) is performed.
[0033]
FIG. 2 is a flowchart showing a processing procedure for registering long-term continuous scratch information in the ROM 46. Registration of continuous scratches for a long time is not limited to the case of factory shipment (when the imaging device 16 is shipped or when the camera 10 is shipped), and is registered by the user operating the camera 10 after shipment (update of registered contents). ) Is also possible.
[0034]
As shown in FIG. 2, first, long-time exposure photography (long-time charge accumulation) of the imaging device 16 is performed in a light-shielding state where light is not incident on the imaging device 16 by closing a mechanical shutter or the like. Data is fetched (step S110). Then, normal scratches are corrected by the first correction process described above (step S112), and long-term continuous scratches are extracted (step S114). Note that the detection method similar to the above-described registration of dark white scratches is applied to the extraction of long-term continuous scratches.
[0035]
Next, the extracted long-time continuous flaw address is stored in the ROM 46 (step S116). For long-term scratches, the addresses of isolated points are not registered, but only for continuous scratches. As a result, the capacity of the ROM 46 can be reduced as compared with the case of registering information on all long-term scratches (continuous scratches and isolated point scratches).
[0036]
The third correction process is common to the second correction method in that it corrects flaws that occur during long exposure, but the second correction process described above corrects continuous flaws that are difficult to distinguish from captured images. In the third correction process, isolated point scratches that are easily discriminated from the captured image are corrected. That is, in the third correction process, the flaw address is not registered in advance, and an isolated point flaw is detected by analyzing an image obtained by long exposure shooting. Then, correction is performed to replace the detected isolated point scratch with a representative value obtained from the signal value of the surrounding same color pixel.
[0037]
For example, it is determined whether or not each pixel of an image obtained by photographing for a certain exposure time or longer is a white defect (an isolated point defect) compared to the surrounding eight pixels. The positional relationship of the peripheral eight pixels to be compared differs depending on the optical arrangement and color of each pixel on the imaging device 16. Then, the detected isolated point scratch is replaced with an average value of four pixels of the same surrounding color.
[0038]
Here, a specific example of detection and correction of isolated point scratches will be described.
[0039]
The configuration shown in FIG. 3 is a pixel array called a honeycomb array, in which the center point of the geometric shape of the light receiving element 50 is shifted by half the pixel pitch (1/2 pitch) every other row direction and column direction. Are arranged. That is, in the rows (or columns) of the light receiving elements 50 adjacent to each other, the element arrangement in one row (or column) is in the row direction (or column direction) with respect to the element arrangement in the other row (or column). The structure is arranged so as to be relatively shifted by about ½ of the arrangement interval.
[0040]
Each light receiving element 50 has an octagonal light receiving surface, and RGB primary color filters are arranged corresponding to each light receiving element 50. As shown in FIG. 3, the GGGG... Row is arranged in the next stage of the RBRB... Row in the horizontal direction, and the BRBR. In the column direction, the RBRB... Row, the GGGG... Row, and the BRBR. The opening shape of the light receiving element 50 is not limited to an octagon, and may be a polygon such as a quadrangle or a hexagon, or a circle. A microlens (not shown) is disposed on each light receiving element 50 so that incident light is efficiently incident on the light receiving element 50.
[0041]
When an isolated point scratch is detected in the imaging device 16 having the pixel arrangement structure shown in FIG. 3, four G pixels (G1, G2, G3, G4) on the top, bottom, left, and right adjacent to the target G0 pixel are detected for the G pixel. And two R pixels (R1, R2) and two B pixels (B1, B2) that are close to each other in the oblique direction are treated as eight peripheral pixels. As for the R or B pixel, as shown in FIG. 4, 8 pixels (G1, G2, G3, G4, R1, R2, R3, R4) at pixel positions close to each other in the oblique direction are treated as 8 peripheral pixels. The R pixel and the B pixel have the same relative position with respect to scratch detection and correction. That is, in the case of the B pixel, instead of R1 to R4 in FIG. 4, four B pixels at the same relative positions are included.
[0042]
In this way, each pixel is compared with the surrounding 8 pixels in the positional relationship set for each color, and a pixel having a luminance of a certain level difference or more for each pixel type (for each color) is determined as a white scratch, and the scratch The pixel value is replaced with an average value of surrounding similar pixel values.
[0043]
For example, when white scratches are determined for G pixels, the following conditions (1) and (2) are used as determination conditions.
[0044]
[Condition (1)] G0 is all greater than the sum of the threshold values THG_G for the neighboring G4 pixels (G1, G2, G3, G4) of the G0 pixel of interest. That is, for the four G pixels (G1, G2, G3, G4) adjacent to the G0 pixel in FIG. 3, the following equations (1-1) to (1-4) are all satisfied.
[0045]
## EQU1 ## G0> G1 + THG_G (1- 1)
[0046]
## EQU2 ## G0> G2 + THG_G (1-2)
[0047]
## EQU3 ## G0> G3 + THG_G (1-3)
[0048]
## EQU4 ## G0> G4 + THG_G (1-4)
[Condition (2)] G0 is all greater than the sum of the thresholds THG_RB for the four adjacent R or B pixels (R1, R2, B1, B2) of the G0 pixel of interest. That is, the following expressions (2-1) to (2-4) must all hold for the four pixels (R1, R2, B1, B2) close to the G0 pixel in FIG.
[0049]
## EQU5 ## G0> R1 + THG_RB (2-1)
[0050]
## EQU6 ## G0> R2 + THG_RB (2-2)
[0051]
G0> B1 + THG_RB (2-3)
[0052]
## EQU8 ## G0> B2 + THG_RB (2-4)
When both the above conditions (1) and (2) are satisfied, the G0 pixel is determined as an isolated point scratch pixel. The isolated point scratch pixel is replaced with the average value of the neighboring G4 pixels. That is, the pixel value of R0 is represented by the following equation (3-1).
[0053]
## EQU9 ## G0 = (G1 + G2 + G3 + G4) / 4 (3-1)
In the case of the R pixel, as shown in FIG. 4, eight peripheral pixels (G1, G2, G3, G4, R1, R2, R3, R4) are defined for the R0 pixel of interest. The following conditions (1) ′ and (2) ′ are used as judgment conditions.
[0054]
[Condition {circle over (1)}] R0 is all greater than those obtained by adding the threshold THRB_G to the adjacent G4 pixels (G1, G2, G3, G4) of the R0 pixel of interest. That is, for the four G pixels (G1, G2, G3, G4) adjacent to the R0 pixel in FIG. 4, all of the following expressions (4-1) to (4-4) are satisfied.
[0055]
R0> G1 + THRB_G (4-1)
[0056]
R0> G2 + THRB_G (4-2)
[0057]
R0> G3 + THRB_G (4-3)
[0058]
R0> G4 + THRB_G (4-4)
[Condition {circle over (2)}] R0 is all greater than the sum of the threshold values THRB_RB for the four adjacent R or B pixels (R1, R2, B1, B2) of the R0 pixel of interest. That is, the following equations (5-1) to (5-4) must all hold for the four pixels (R1, R2, B1, B2) close to the G0 pixel in FIG.
[0059]
## EQU14 ## R0> R1 + THRB_RB (5-1)
[0060]
R0> R2 + THRB_RB (5-2)
[0061]
R0> B1 + THRB_RB (5-3)
[0062]
R0> B2 + THRB_RB (5-4)
When both the above conditions (1) ′ and (2) ′ are satisfied, the R0 pixel is determined to be an isolated point scratch pixel. The isolated point scratch pixel is replaced with the average value of the four adjacent pixels of the same color. That is, the pixel value of R0 is represented by the following equation (6-1).
[0063]
R0 = (R1 + R2 + R3 + R4) / 4 (6-1)
Although the R pixel has been described in FIG. 4, the isolated point scratch is determined and corrected for the B pixel by a method similar to that for the R pixel (the R pixel is replaced by the B pixels B1 to B4).
[0064]
The exposure time and threshold value are variably set according to sensitivity. The camera 10 of this example is configured so that the sensitivity of the imaging system can be changed within a range corresponding to ISO 100 to ISO 1600, and the user can select a desired photographing sensitivity by operating the operation switch 44 or the like. The CPU 20 sets the gain in the analog signal processing unit 22 in accordance with the designated photographing sensitivity. Since the gain increases as the photographing sensitivity increases, the noise component also increases accordingly, so the determination threshold is also set to a large value.
[0065]
The camera 10 according to the present embodiment corrects various types of scratches by combining the first to third correction processes described above. The processing procedure will be described below.
[0066]
FIG. 5 is a signal processing flowchart of the camera 10. As shown in the figure, when the shutter switch 42 is pressed (step S210) and a shooting execution instruction is input, the CPU 20 controls the shooting operation to expose the imaging device 16. Then, the mechanical shutter is closed after exposure, data is read from the imaging device 16 in the shutter closed state, and the image data is stored in the memory 30 (step S212). First, normal flaw correction is performed on the image data stored in the memory 30 in this manner (step S214). The correction process here implements the first correction process described above.
[0067]
In step S214 of FIG. 5, after correcting the normal scratch and removing the scratch, the process proceeds to step S216, and it is determined whether or not the exposure time is longer than a predetermined determination reference value (for example, 1 second). If it is determined in step S216 that the exposure time is longer than the predetermined determination reference value (long exposure), a long-time scratch correction process is performed (step S218).
[0068]
FIG. 6 shows a process flowchart for long-term scratch correction. When the long-time scratch correction process shown in the figure starts, first, a continuous scratch correction process registered in the ROM 46 is performed (step S310). The correction process here implements the second correction process described above.
[0069]
In step S310, long-term continuous flaw correction is performed to remove continuous flaws, and then the process proceeds to step S312 to perform processing for correcting isolated point flaws. In this correction process, the third correction process described above is performed. The image is scanned (by comparing each pixel with surrounding pixels) to detect isolated point scratches, and the isolated point is surrounded by Replace with the average value of pixels of the same color. When the long-term continuous flaw and the isolated point flaw are corrected according to the above procedure, the long-time flaw correction subroutine is terminated and the process returns to the flowchart of FIG.
[0070]
After performing the long-time scratch correction (step S218), the process proceeds to step S220, and the image data stored in the memory 30 is subjected to white balance (WB) correction, luminance / color difference signal generation, gamma correction, contour enhancement, compression, and the like. The image data is recorded on the memory card 36 (step S220).
[0071]
On the other hand, when it is determined in step S216 that the exposure time is shorter than the predetermined determination reference value (during normal exposure), the long-time scratch correction process (step S218) is omitted, and the process proceeds to step S220. The image data is recorded on the memory card 36 by performing an appropriate signal processing. In this way, when the image data has been recorded on the memory card 36, this processing sequence ends.
[0072]
In implementing the present invention, the structure of the imaging device is not limited to the example described with reference to FIGS. 3 and 4, and the light receiving elements may be arranged in a square lattice pattern, and a color filter array for color separation. As for (CFA), the present invention can also be applied to various arrangement structures such as Bayer arrangement, interline arrangement, and G vertical stripe RB checkered.
[0073]
In the above embodiment, the digital still camera has been described as an example. However, the scope of application of the present invention is not limited to this, and the present invention can be applied to various apparatuses such as a movie camera and an image input apparatus that record electronic video.
[0074]
【The invention's effect】
As described above, according to the present invention, position information is registered in advance only for continuous scratches (long-time continuous scratches) that are difficult to detect from a photographed image among scratches that appear during long exposure. During time exposure, the correction of scratches is performed by combining the correction process for long-term continuous scratches based on the registered information and the isolated point scratch correction (isolation point removal) process by image analysis. Even if a large-capacity storage area is not prepared for the means for storing the position information, continuous flaws and single flaws (isolated flaws) can be corrected efficiently.
[Brief description of the drawings]
FIG. 1 is a block diagram of an electronic camera to which a solid-state image sensor defect correction apparatus and method according to an embodiment of the present invention is applied.
FIG. 2 is a flowchart showing a processing procedure when registering positional information of long-term continuous scratches in a ROM.
FIG. 3 is a schematic plan view illustrating an example of an optical arrangement of pixels on an imaging device.
FIG. 4 is a schematic plan view illustrating an example of an optical arrangement of pixels on an imaging device.
FIG. 5 is a flowchart showing a signal processing procedure of the camera according to the present embodiment.
FIG. 6 is a flowchart showing a processing procedure for correcting long-term continuous scratches.
[Explanation of symbols]
DESCRIPTION OF SYMBOLS 10 ... Camera, 16 ... Imaging device, 20 ... CPU, 22 ... Analog signal processing part, 26 ... A / D converter, 30 ... Memory, 32 ... Digital signal processing part, 46 ... ROM, 50 ... Light receiving element

Claims (3)

Continuous flaw position information storage means for storing position information of only continuous flaws caused by a plurality of continuous defective pixels among the defects of pixels that are manifested during long exposure for a predetermined time or longer; and
Continuous flaw correction processing means for correcting the continuous flaws using the position information of the continuous flaws stored in the continuous flaw position information storage means and signal values of surrounding pixels;
Isolated point scratch detecting means for detecting isolated point scratches for each pixel in the screen in comparison with surrounding pixels;
Isolated point scratch correction processing means for correcting the isolated point scratch using a signal value of a peripheral pixel of the detected isolated point scratch;
When the image is shot by long exposure for the predetermined time or longer, the normal scratch correction processing means, the isolated point scratch correction means, and the isolated point scratch correction processing means are controlled to control the continuous scratch and the isolated point scratch. Correction control means for performing the correction of
A defect correction apparatus for a solid-state image sensor, comprising: Normal flaw position information storage means for storing normal flaw position information that is expressed at the time of normal exposure within the predetermined time; and
Normal flaw correction processing means for correcting the normal flaw by using the normal flaw position information stored in the normal flaw position information storage means and signal values of surrounding pixels;
With
The correction control means first performs the normal flaw correction by the normal flaw correction processing means when shooting is performed by long exposure for the predetermined time or longer, and then the continuous flaw correction processing means. The isolated flaw is detected by the isolated point flaw detection means for the image after the normal flaw and continuous flaw correction processing, and the isolated point flaw is detected. 2. The defect correction apparatus for a solid-state imaging device according to claim 1, wherein control is performed so as to perform correction of isolated point scratches by the isolated point scratch correction processing means. Among pixel defects that appear during long exposure for a predetermined time or more, position information of only continuous flaws caused by a plurality of consecutive defective pixels is registered in advance, and it appears during normal exposure within the predetermined time. Register location information for normal scratches in advance,
When shooting is performed with a long exposure of the predetermined time or longer,
First, the normal scratch is corrected using the registered positional information of the normal scratch and the signal values of the surrounding pixels,
Next, using the registered position information of the continuous scratch and the signal value of the surrounding pixels, the continuous scratch is corrected,
Thereafter, each pixel is compared with surrounding pixels in the image after the normal scratch and continuous scratch correction processing, and isolated point scratches are detected, and the signal values of the peripheral pixels of the detected isolated point scratches are used. A defect correction method for a solid-state imaging device, wherein the isolated point scratch is corrected.

Images