JP2000244823A - Device for concealing defective pixel of imaging device - Google Patents

Abstract

PROBLEM TO BE SOLVED: To obtain a defective pixel concealing device to properly correct the values of defective pixels even when a plurality of defective cells are continuously generated in an imaging device. SOLUTION: When pieces of image data picked up by the imaging device (CCD) 12 and converted into a digital signal at an analog/digital converting part 18 are supplied to a digital signal processing part 20 provided in a digital camera 10, these pieces of data are stored in a frame memory(FM) 21. When positional information 114 in which pieces of coordinate data indicating the defective pixels of the imaging device 12 which are stored in a coordinate memory 28 are supplied to the signal processing part 20 from a control part 22, pixels adjacent to object pixels are judged whether they are defective in the case of performing interpolation processing of object pixels indicated by the position information 114, defective adjacent pixels are not used, the pixel values of normal adjacent pixels are adopted, the average of them are calculated, written in storage addresses of the object pixels and the pixel values of the defective pixels being the object of processing are interpolated.

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to an apparatus for correcting a defective pixel of an image sensor, and more particularly to an apparatus for correcting a defective pixel of an image sensor for correcting a pixel signal generated due to a pixel defect of a solid-state image sensor. .

[0002]

2. Description of the Related Art For example, there is known a two-dimensional solid-state imaging device in which a large number of photosensitive elements are arranged in an array in horizontal and vertical scanning directions to form pixels. The light receiving elements included in such an imaging cell array include cells that do not respond to incident light or cells that generate an abnormally large dark current without incident light due to the manufacturing process or the like. Often there are. These defective cells are referred to, for example, as “black defects” and “white defects”, respectively, and it is difficult for a solid-state imaging device to completely remove these defective cells themselves.

Therefore, in an image signal output from a solid-state image pickup device, a defective pixel which corrects a pixel signal of a defective pixel generated by these defective cells by using a pixel signal obtained from a surrounding image pickup cell. Has been proposed. In correcting a pixel signal of a defective cell, for example, there has been a method of simply averaging pixel signals of a plurality of imaging cells adjacent around the defective cell and replacing the average value with the pixel signal of the defective cell.

[0004]

However, in the above-described correction method, for example, when a defective cell is adjacent and a plurality of defective cells are in a burst shape, another defective cell adjacent to the defective cell to be processed is used. The average value will be averaged using the neighboring pixel values of the defective cell.
It was not appropriate to substitute the value of the target pixel.
Further, when the pixel value calculated in this manner is used as an adjacent pixel to correct another defective cell,
Correction errors were accumulated in the processing result, and appropriate defect correction could not be performed.

The present invention solves the above-mentioned drawbacks of the prior art, and appropriately corrects the value of a defective pixel even when a plurality of defective cells serving as defective pixels are continuously generated. It is an object of the present invention to provide a device for correcting a defective pixel of an imaging device.

[0006]

SUMMARY OF THE INVENTION In order to solve the above-mentioned problems, the present invention is directed to an image pickup device which receives a pixel signal output from an image pickup device and corrects a defective pixel value of the pixel signal by a defective pixel in the image pickup device. In the device for correcting a defective pixel of an element, the device obtains a signal storage unit that stores a pixel signal in a predetermined storage area and a storage position where a pixel signal corresponding to a defective pixel in the image sensor is stored in the storage area. A defect coordinate obtaining means for calculating an adjacent pixel position of a first adjacent pixel which is a pixel signal of a defective pixel stored in a storage position of the storage area and which is adjacent to a pixel signal to be processed in the image sensor; Means, a defect determining means for determining whether or not the first adjacent pixel is a defective pixel, and a pixel signal stored at an adjacent pixel position according to a determination result of the defect determining means. There are, characterized in that it comprises a correction means for correcting the pixel value of the pixel signal by a defective pixel to be processed.

[0007]

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS Next, an embodiment of an image pickup apparatus to which an apparatus for correcting defective pixels of an image pickup element according to the present invention is applied will be described in detail with reference to the accompanying drawings. Referring to FIG. 1, a block diagram of a digital camera is shown as an embodiment to which the present invention is applied. This digital camera 10 includes an imaging lens (not shown) and an imaging signal of an object scene captured by an imaging device 12. And an image capturing apparatus that records data representing a still image on a memory card 30 that is removable from the main body 10. More specifically, the camera 10 has a still image mode in which a high-resolution captured image is subjected to a still image encoding process and recorded on the memory card 30, and a moving image mode for performing framing or the like at the time of shooting. When the release switch provided in 26 is turned on, the still image mode is set.
In the moving image mode, framing, focus adjustment, exposure adjustment, and other shooting and imaging adjustments are performed by visually recognizing a monitor display that displays a captured moving image. When the mode shifts to the still image mode according to the release operation by the photographer,
The camera 10 processes a high-resolution still image of one frame or a plurality of frames captured after being framed in the moving image mode, and is removably mounted on the output unit 24.
To record. The camera 10 is an image pickup device that appropriately corrects defective pixels generated in the image pickup device 12 and outputs a good image. In the following description, parts not directly related to the present invention are not shown and described, and reference numerals of signals are represented by reference numerals of connection lines in which the signals appear.

[0008] The camera 10 includes an image sensor 12 for converting an image of a field formed by an image pickup lens disposed toward the field into an electric signal. And a photosensitive unit in which a plurality of photodiodes each forming a plurality of pixels in the vertical scanning direction are arranged in the horizontal and vertical directions, and a vertical transfer CCD for transferring charges generated by each photodiode in the vertical (V) direction, Each vertical transfer CCD
Transfer that transfers the charge transferred from the device in the horizontal (H) direction
This is a two-dimensional image sensor including a CCD and an output amplifier that converts electric charges transferred by the horizontal transfer CCD into electric signals and outputs the electric signals.

The image pickup device 12 includes defective pixels due to its manufacture. Pixels having a higher flaw level than other normal pixels are treated as defective pixels depending on the degree of the defect. The coordinate data specifying the position of the defective pixel is
The coordinate data is supplied from 12 manufacturers and stored in a coordinate memory 28 connected to a control unit (CPU) 22 described later. However, the present invention is not limited to this. For example, when incorporating the image sensor 12 into the camera 10, a defective pixel may be uniquely determined, and coordinate data indicating the position may be created in the coordinate memory 28.

The image pickup device 12 is driven in response to a drive signal input to the input 100, generates electric charges corresponding to the amount of light reaching the photodiode and the exposure time thereof, and generates electric charges corresponding to the electric charges. The signal is output to the output 102 as an RGB dot sequential imaging signal. Image sensor 12 in the present embodiment
In the moving image mode, each pixel is thinned in half in the horizontal and vertical directions in response to the supplied driving signal 100, and an RGB pixel signal of, for example, 640 × 460 pixels vertically and horizontally, which can be displayed on a monitor, is read. A moving image signal constituting one frame in a field is output to an output 102. In the still image mode, all pixels are read, and the effective pixels are 1280 × 1024
Outputs the RGB pixel signal of the pixel for each frame.

Of the pixel signals output from the image pickup device 12, pixel values obtained at the photosensitive section corresponding to positions registered in the coordinate memory 28 as defective pixels are sent to a digital signal processing section 20, which will be described later. Is corrected. In the case where the image pickup device 12 is an image pickup device that picks up a so-called monochrome image, this correction processing is performed as shown in FIG. 3 as a part of the pixel array of the image pickup device. V), the pixel (2,3) is a defective pixel, and the pixel (3,3) adjacent to this pixel is also a defective pixel.
In the correction method of (1), the pixel value of the target pixel (2, 3) is corrected using the pixel values obtained from the three pixels of the adjacent pixels (2, 2), (1, 3), and (2, 4). In this case, if the adjacent pixel (3,3)
Is not a defective pixel, the adjacent pixels (2,2),
The pixel value of the target pixel (2, 3) to be subjected to defect correction is corrected using each pixel value obtained by adding the adjacent pixel (3, 3) to (1, 3), (2, 4). In the first correction method, when there is no defective pixel around a defective pixel to be corrected, four pixels around the defective pixel to be corrected are averaged, and the pixel value of the target pixel is replaced with the calculation result. Do. However, when the defective pixels are adjacent in a burst, the adjacent defective pixels are not used, and the normal adjacent three pixels adjacent to the defective pixel to be corrected are used. Correct the pixel value. In this case, two neighboring pixels or one pixel adjacent to the target pixel may be used.

In the second correction method, for example, FIG.
As shown in the figure, the adjacent pixel (3, 3) adjacent to the target pixel (2, 3) to be defect corrected is a defective pixel, and the target pixel (2, 3)
When correcting the pixel value of (3), the pixel value of one of the adjacent pixels (3,2), (4,3), (3,4) further adjacent to the adjacent pixel (3,3) The pixel of the target pixel (2,3) is obtained by using the four pixel values of the obtained pixel value and the pixel values of the adjacent pixels (2,2), (1,3), and (2,4) of the target pixel. Correct the value. In this case, the arithmetic processing for correcting the defective pixel by taking the average of four pixels when the defective pixel exists alone can be used as it is. According to the conventional correction method, for example, as shown in FIG. 5, even if a pixel adjacent to a defect target is also a defective pixel, four pixels adjacent to the target pixel are averaged and the averaged result is obtained. Since the pixel value of the pixel was used, when the defective pixels were continuously arranged, the calculation result including the abnormal value of the defective pixel data was obtained. When correcting a defective pixel, an adjacent pixel registered as a defective pixel is not used, so that appropriate defective pixel correction can be performed. Such a pixel defect correction is not limited to a monochrome camera, but is applied to a multi-plate type imaging unit that separates incident light into, for example, RGB primary colors by a color separation optical system such as a dichroic prism and receives the separated primary color components. The present invention is also suitably applied to a case where a defective pixel generated in each mounted image sensor is corrected.

The above description has been given of the basic correction method when one unit of the image pickup device 12 is a monochrome image pickup device. The case where the image pickup device 12 is a so-called single-plate color image pickup device will be described below. I do. In this case, for example, RGB
A primary color filter and a complementary color filter are applied.
For example, as shown in FIG. 6 as an example, the image pickup device 12 is an image pickup device to which primary color filters of a G stripe R / B perfect checkerboard arrangement are attached. When correcting a defective pixel in an image sensor to which such a color filter is attached, attention is paid only to the same color component, respectively.
An adjacent pixel for correcting a correction target pixel is determined based on whether or not adjacent pixels of the same color component are consecutively defective pixels. In the image sensor having the filter arrangement shown in FIG. 3, when correcting the pixel value of a pixel (4, 5) which is a defective pixel as a correction target pixel, the adjacent pixel (4) adjacent to the target pixel with the same color component is used. , 4), (2,5), (6,5),
If the four pixels (4,6) are normal pixels, a process of averaging the pixel values of these four pixels and replacing the calculation result with the pixel value of the pixel (4,5) is performed. But the target pixel
If the adjacent pixel (6,5) of the same color component as (4,5) is also a defective pixel, the pixel value of this adjacent pixel (6,5) is not used and the other adjacent pixels (4,5) are not used. 4), (2,5), (4,6) to the target pixel (4,
5) Use as an adjacent pixel to correct. This is shown in FIG. That is, when the pixels (4,5) and (6,5) are defective pixels, the remaining pixels (4,4), (2,5) and (2,5) excluding the defective pixel (6,5) are excluded. The average value of the three pixel values (4, 6) is calculated, and the calculation result is replaced with the pixel value of the pixel (4, 5). This is basically the same as the above-described first correction method.

Further, a second correction method of the defect correction processing for the single-chip color image pickup device will be described with reference to FIG.
Similarly to the above description, when the pixels (4, 5) and (6, 5) are defective pixels and the pixel (4, 5) is corrected as a correction target pixel, the adjacent pixels (4, 4), In addition to the pixels (2,5) and (4,6), the pixels (6,4),
Use any pixel of (8,5) or (6,6). For example, the average value of the pixels (4, 4), (2, 5), (4, 6), and (6, 6) is replaced with the pixel value of the pixel (4, 5). In this case, as in the example shown in FIG. 4, the same arithmetic processing as usual can be employed for the averaging arithmetic processing. That is, the average of the pixel values can be calculated by dividing the sum of the four pixel values by the equation (4). In this case, it is preferable to use an adjacent pixel having a short distance between the target pixel and an adjacent pixel of the same color. Alternatively, weighting may be set according to the distance between the target pixel and the adjacent pixel, and the weighted values may be averaged so that the adjacent pixels closer to the target pixel are weighted. Further, in the example shown in FIG.
Instead of (6,5), one of the further adjacent pixels was used.
The average value of two or three pixels among (6,4), (8,5), and (6,6) may be used as a pixel value replacing the adjacent pixel (6,5).

In the conventional method, for example, as shown in FIG. 8, an adjacent pixel adjacent to the target pixel (4, 5) with the same color component
Even if (6,5) is a defective pixel, simply calculate the average value of four adjacent pixels adjacent to the target pixel (4,5) with the same color component, and calculate the result. Was used as the pixel value of the target pixel, but in this case, since the pixel value of the defective pixel was used, the value of the interpolated target pixel was an excessive correction value. However, in the first and second interpolation processing methods shown in FIGS. 3 and 4, when an adjacent pixel is a defective pixel, the adjacent defective pixel is not used, and the calculation method is switched to correct the target pixel. ,Also,
Since an appropriate adjacent pixel in place of the adjacent defective pixel is selected and the interpolation target pixel is corrected by the same arithmetic processing as usual,
Good correction results can be obtained.

The method of correcting a defective pixel based on the pixel arrangement and the filter arrangement on the image sensor 12 has been described above. Such a defective pixel correction process is performed by the digital signal processing unit 20 described below. The processing is performed. In the following description, it is assumed that the image pickup device 12 provided with the above-described color filter is used.

Returning to FIG. 1, an image signal including a pixel value obtained from a defective pixel according to the individual difference of the image sensor is output to the output 102 of the image sensor 12. This output 10
2 is connected to an analog processing unit 16, and the analog processing unit 16 performs correlated double sampling of an analog signal output from the image sensor 12 in synchronization with a pixel clock supplied from the timing control unit 14. Double sampling (CD
S) Has a circuit. The correlated double sampling circuit removes reset noise in the input image signal 102. Further, the analog processing unit 16 separates the image signal input in the RGB dot sequence into three lines for each color component in response to the color separation pulse supplied from the timing control unit 14, and a color separation circuit. A color balance adjustment circuit and a gamma correction circuit for adjusting the level of each color component signal, and a multiplexer for linearizing each of the three-lined color component signals are included in a control signal 104 such as an input pixel clock or a color separation pulse. In response, analog signal processing such as adjustment and correction processing is performed. The analog processing unit 16 converts each RGB signal obtained by pre-interpolating the defective pixel into a single line by switching the color components, for example, every one horizontal scanning period, and then converts the RGB signal to an analog / digital (A / D) conversion connected to the output 106. It is supplied to the unit 18 in a line sequential manner.

The A / D conversion section 18 is an analog / digital conversion processing section for converting an image signal input to the input 106 into a digital signal of a predetermined bit. A / in this embodiment
The D conversion unit 18 is an A / D supplied from the timing control unit 14.
In response to a control signal 108 such as a clock or a synchronization signal, each pixel signal of each color component, which is switched and supplied every horizontal scanning period, is sequentially converted into an 8-bit or 10-bit digital value, for example. The output 110 of the A / D conversion processing unit 18 is connected to the digital signal processing unit 20.

The digital signal processor 20 converts the input digital image data 110 into a control signal 112 from the controller 22.
Is a processing circuit that performs various digital signal processing based on The signal processing unit 20 has a frame memory (FM) 21 for temporarily storing each of the RGB image signals. When the still image mode is set, the color balance, the luminance, In addition, correction processing for correcting saturation and the like is performed according to signal processing parameters corresponding to the still image mode. Further, the signal processing unit 20 has a processing function of executing digital signal processing such as filter processing and contour enhancement processing on image data to create an appropriate still image. In the still image mode, the digital signal processing unit 20 converts the RGB primary color image data into luminance data Y.
And a YC separation processing function for converting the image data into color difference data C. In the moving image mode, when displaying a captured image on a general-purpose external display device, the YC data subjected to the YC conversion processing is supplied to the output unit 24. In addition, the signal processing unit 20 supplies RGB image data to the output unit 24 when displaying a captured image on a liquid crystal monitor device built in the output unit 24.
Further, the signal processing unit 20 in the still image mode includes an output unit 24.
The image data is converted to Y
The data is converted into C data and supplied to the output unit 24.

The digital signal processing unit 20 has a pixel interpolation function of representing each pixel by three primary colors RGB in accordance with the filter arrangement of the image sensor 12.
It has a pixel generation function of generating each RGB pixel at one pixel position from each primary color pixel by interpolation processing.

Further, the digital signal processing section 20 has a function of correcting a defective pixel generated in the image pickup device 12 before performing the above-described pixel interpolation processing, and uses a plurality of pixels around the defective pixel at that time. By performing the interpolation process, a still image that is well corrected is generated. Specifically, the signal processing unit 20
Using the position information 114 supplied from the control unit 22 in accordance with the coordinate data of the defective pixel stored in the coordinate memory 28 described later, the value of the defective pixel included in the imaging signal is converted to the value of the adjacent pixel of the same color component. Is performed to replace the value with a value corresponding to. For example, the signal processing unit 20 calculates the average of the same color component pixels in the up, down, left, and right directions adjacent to the defective pixel in the image data once stored in the frame memory 21 and calculates the average value of the defective pixels. Is written to the address where is stored, and an interpolation process of backfilling the defective pixel is performed. The signal processing unit 20 reads out one frame of still image data subjected to various image signal processing after the defect correction processing in this manner from the frame memory 21 and outputs it to the output unit 24 connected to the output 116.

The digital signal processing section 20 in the present embodiment
When processing a defective pixel, according to the state of the adjacent pixel necessary when interpolating the pixel to be interpolated, even if the adjacent pixel is a defective pixel, a good pixel defect correction is performed. Has a functional configuration for realizing the above-described first correction method. FIG. 2 shows a functional configuration of the signal processing unit 20 that performs such defect correction. The signal processing unit 20 includes a defect coordinate acquisition unit 200 that acquires position information 114 of a defective pixel supplied from the control unit 22; An adjacent address calculating unit 202 for calculating a storage address in the frame memory 21 of an adjacent pixel adjacent to the defective pixel to be processed in the same color component; and determining whether the adjacent pixel is a defective pixel based on the storage address. A defect judging unit 204 for judging and outputting the judgment result; a delay unit 206 for temporarily storing a pixel address and delaying and outputting it after a predetermined time according to a pixel value acquisition timing; And an interpolation processing unit 208 for performing interpolation processing according to the result.

As shown in FIG. 3, the interpolation processing unit 208 further includes a pixel value acquisition unit 210 for acquiring the pixel value of a normal adjacent pixel from the frame memory 21 according to the defect determination by the defect determination unit 204, and a defect determination unit 204. When it is determined that the four adjacent pixels are normal pixels, the pixel value acquisition unit 21
0, a first arithmetic processing unit 212 for performing a four-point interpolation operation using four pixel values sequentially acquired, and a pixel value acquiring unit when the determination result of the defect determination unit 204 includes one defective pixel.
A second operation processing unit 214 for performing a three-point interpolation operation using three pixel values sequentially obtained at 210, and a first operation processing unit 212
And the pixel value calculated by the second arithmetic processing unit 214 is written into the storage address of the pixel to be subjected to the interpolation processing, and the interpolation calculation result is embedded in the frame memory 21 in the interpolation value writing unit 216.
And

The interpolation processing unit 208 determines, for each defective pixel corresponding to the defective coordinate address, whether or not the adjacent pixel is a defective pixel, and determines the value of the remaining normal pixel excluding the defective pixel. use. At that time, in the first arithmetic processing unit 212, when all four pixels among the adjacent pixels are normal,
An arithmetic process for simply averaging four pixel values is performed.

The second arithmetic processing unit is activated when it is determined that one of the adjacent pixels is defective,
An arithmetic process for averaging the values of three normal pixels sequentially acquired by the pixel value acquisition unit 210 is performed. The second arithmetic processing unit
Without providing the 214, the first arithmetic processing unit is configured to
The plurality of normal pixel values acquired at 0
May be configured to change the parameter to be divided according to the result of the above. The interpolation value writing unit 216 writes the calculation value to the storage address given from the delay unit 206 when the calculation result is supplied only from the first calculation processing unit 212 each time the interpolation calculation process is performed on the target pixel. Then, the pixel value of the target pixel stored in the frame memory 21 is updated. When the calculation result is supplied from the second calculation processing unit, the interpolation value writing unit 216 stores the calculation value in the delay unit 20.
6 to update the pixel value of the target pixel stored in the frame memory 21.
When the pixel value is updated in this manner, the interpolation value writing unit 21
6 sends a control signal for correcting the next defective pixel to the defective pixel acquisition unit 200, and the defective pixel acquisition unit 200
The position information 114 of the unprocessed defective pixel is obtained from 28, and the signal processing unit 20 performs a correction process on the next defective pixel.

As described above, the pixel defect correction function in the digital signal processing unit 20 in the present embodiment performs the function of the storage address of the defective pixel supplied from the control unit 22 in the image data stored in the frame memory. When correcting the stored value, if the adjacent pixel used for correction is a value obtained from a defective pixel again, the pixel value obtained from the remaining normal adjacent pixel is averaged without using the adjacent pixel. Then, the average value is updated as the pixel value of the target defective pixel. Therefore, when correcting a defective pixel value, it is possible to prevent an excessive correction by an abnormal pixel value.

The digital signal processing section 20 performs the correction processing of the defective pixel in this manner, and further performs the above-described pixel generation processing, correction processing of color balance, luminance and saturation, and YC conversion processing. The image data that has been properly corrected and adjusted in this manner is read from the frame memory 21 and supplied to the output unit 24.

The output section 24 is a processing section for outputting the image data processed by the digital signal processing section 20 in a desired output form. More specifically, the output unit 24 includes a monitor display device that displays an image and a video represented by the image data, and displays an image of each frame corresponding to the RGB image data input to the input 116 on its display screen. The output unit 24
Converts the image data 116 into, for example, an NTSC video signal and outputs it to general-purpose video equipment such as a television monitor device, a video printer, and a video recorder, which is detachably connected to the output 118 thereof. Further, the output unit 24 converts the YC image data of the still image obtained by the still image shooting into, for example,
The encoded data is compressed and encoded into data of a predetermined length by the JPEG method, and the processed encoded data is stored in a storage medium 30 such as a memory card detachably connected to the output 120. Output unit 24
Has a function of reading information recorded in the storage medium 30, decoding the encoded data, and supplying the decoded image data to the digital signal processing unit 20. in this case,
The signal processing unit 20 re-converts the image data supplied from the output unit 24 into a signal format corresponding to the output destination from the output unit 24, and outputs the processing result to the output 116. The digital signal processing unit 20 may include a digital terminal for inputting / outputting a digital image signal, for example, a serial input / output terminal or a parallel input / output terminal capable of inputting / outputting moving image data of each continuous frame. In this case, it is preferable that the data be converted into a predetermined format according to the connection device and output from the input / output terminal.

The control unit 22 has a function of selectively setting a moving image mode or a still image mode in accordance with the operation state of the operator on the operation unit 26 and controlling each unit in accordance with the mode. Specifically, the control unit 22 sets the moving image mode when the camera 10 is turned on, and sets the still image mode when the release button arranged on the operation unit 26 is pressed. When the release switch of the operation unit 26 is constituted by a two-stage switch, the imaging adjustment in the moving image mode and the display processing of the captured image are performed by the first stroke, and the focus adjustment, the exposure adjustment, and the It is preferable to adjust the white balance or the like and shift to the still image mode in the second stroke when the release switch is further pressed.

When the moving image mode is set by the control unit 22, the timing control unit 14 generates a drive signal for driving the image pickup device 14 to thin out and read. In the still image mode, the control unit 22 converts the coordinate data recorded in the coordinate memory 28 into a storage address of the frame memory 21 and gives the position information obtained by the conversion to the digital signal processing unit 20 to perform signal processing. The unit 20 has a function of designating a storage address of each defective pixel in the frame memory 21.

In this embodiment, coordinate data indicating the position coordinates of the pixel defect is stored in the coordinate memory 28, and the pixel corresponding to the coordinate data is subjected to the pixel defect correction processing in the still image mode. However, the present invention is not limited to this. For example, even in the moving image mode, when the processing speed of the defective pixel is faster than the transfer speed of the moving image signal, the same arithmetic processing as in the still image mode is performed even in the moving image mode. May go. When displaying a moving image in the moving image mode, the image size is changed by thinning out the pixels of the image data in accordance with the display device or the like as the output destination. In this case, if the pixel to be decimated is a defective pixel, the calculation load is reduced by not setting that pixel as the target pixel. In preparation for such a case, the coordinate memory 28 may separately store defect coordinate data excluding defective pixels not to be processed in the moving image mode.

Next, a digital signal processing section to which the above-described second correction method is applied will be described below. As shown in FIG. 9, the digital signal processing unit 900 in this embodiment
The digital camera 10 is provided in place of the digital signal processing unit 20 shown in FIG. 1, and has a function of correcting the pixel value of a defective pixel in the image sensor 12 as in the above-described embodiment. It should be noted that the digital signal processing unit 900 has various signal processing functions in the above-described embodiment except for the functional configuration other than the defective pixel correction.

In the signal processing unit 900 according to the present embodiment, when performing the interpolation process of the defective pixel, the adjacent pixel to be used when correcting the defective pixel value adjacent to the interpolation target is the defective pixel. In such a case, the pixel value of the defective pixel to be corrected is corrected using the second adjacent pixel further adjacent to the adjacent pixel which is the defective pixel. More specifically, the signal processing unit 900 includes a defect coordinate acquisition unit that acquires coordinate data 114 indicating a storage address of a defective pixel to be processed in the frame memory 21 from the control unit 22.
200 and an adjacent address calculation unit that calculates the storage address of the adjacent pixel adjacent to the defective pixel to be processed with the same color component
902 and whether or not the storage address of the adjacent pixel calculated by the adjacent address calculation unit 902 is included in the position information 114 acquired by the defect coordinate acquisition unit 200. A defect determining unit 904 for determining whether the pixel is a pixel or not, and a delay unit 206 for temporarily storing a pixel address and outputting the pixel address after a predetermined time according to a pixel value acquisition timing.
And an interpolation processing unit 906 that corrects the defective pixel value of the target pixel based on the storage address supplied from the adjacent address calculation unit 902 and the defect determination unit 904.

Adjacent address calculation unit 902 in this embodiment
Supplies the storage address of the normal adjacent pixel to the first pixel value acquisition unit 908 in the interpolation processing unit 906 according to the result of the determination of the defective pixel by the defect determination unit 904, and does not output the storage address of the adjacent pixel which is a defective pixel. It is configured as follows.
Further, when correcting the defective pixel of the target pixel, the adjacent address calculation unit 902 sets the adjacent pixel adjacent to the target pixel to:
If the result of the determination by the defect determination unit 904 is a defective pixel,
It has an arithmetic processing function of calculating a storage address of a second adjacent pixel further adjacent to the pixel. Defect judgment unit 90
4 supplies the calculated storage address of the second adjacent pixel to the second pixel value acquisition unit 910 of the interpolation processing unit 906.

The interpolation processing unit 906 in this embodiment further includes a first pixel value acquisition unit 908 for reading out the pixel value stored in the storage address of the adjacent pixel calculated by the adjacent address calculation unit 902 from the frame memory 21. The pixel value stored in the storage address of the second adjacent pixel supplied from the defect determination unit 904 is read from the frame memory 21 by the second pixel value obtaining unit 910 and the first pixel value obtaining unit 908. An arithmetic processing unit 912 that calculates an interpolated pixel value of the target pixel by an averaging operation based on the pixel values of four adjacent pixels,
When the pixel value of the second adjacent pixel is supplied from the second pixel value obtaining unit 910, the three normal pixels obtained by the first pixel value obtaining unit 908 and the normal pixel values obtained by the second pixel value obtaining unit 910 are obtained. Arithmetic processing unit 912 for averaging arithmetic processing of each of the four pixel values obtained by summing the calculated pixel value and the pixel value of one pixel to calculate the interpolated pixel value of the target pixel
And an interpolation value writing unit 914 that updates the defective pixel value in the frame memory 21 by writing the calculation result in the calculation processing unit 912 as a pixel value for interpolating the target pixel to the storage address of the target pixel sent from the delay unit 206. And

With this configuration, in the present embodiment, adjacent pixels that replace the neighbors recognized in the first stage are recognized in the second stage without changing the arithmetic processing for averaging the pixel values of a plurality of adjacent pixels. By averaging the pixel values of the adjacent pixels recognized at each stage in the same manner as in the normal case,
Defect correction processing based on pixel averaging can be performed. In this case, a value that substitutes for the first neighboring pixel is obtained by averaging the value of the second neighboring pixel that is further neighboring the pixel, and
Using the original normal neighboring pixels for the pixel of interest,
By calculating these average values, an interpolation value for the target pixel may be obtained.

As described above, when correcting the value of a pixel to be interpolated, which is a defective pixel, by the value of an adjacent pixel adjacent to the pixel, it is determined whether or not the adjacent pixel is a defective pixel. If the pixel is a defective pixel, the pixel is excluded, and the pixel value of the interpolation target pixel is determined based on the value of the remaining normal adjacent pixel.
An interpolation process that is not affected by the defect of the adjacent pixel is performed on the interpolation result. As a result, in the interpolation process of the defective pixel, the defect correction is prevented from being excessively corrected due to the processing result, and the image quality is prevented from being deteriorated. Further, when the configuration is such that the adjacent pixel is further recognized as a defective pixel, the adjacent pixel value that is the defective pixel is not used for the correction processing, and the second pixel that is further adjacent to the defective adjacent pixel is not used. The interpolation pixel value for the target pixel is obtained using one of the adjacent pixels and the normal adjacent pixel to the target pixel. Therefore, even when a defective pixel occurs in a burst, the interpolation accuracy is low. A significant reduction is prevented.

In each of the above embodiments, each time a pixel defect is corrected, an adjacent address is calculated. However, the present invention is not limited to this. For example, an adjacent pixel used for defect correction may be used. Indicate storage address also coordinate memory
The storage address of the defective pixel and the storage address of the corresponding adjacent pixel are read out from the storage means in advance and stored in the storage means such as 28, and the digital signal processing unit 20 and the Storage means may be configured.

[0039]

As described above, according to the present invention, when correcting a defective pixel in an image sensor, it is determined whether or not an adjacent pixel necessary for interpolating a target pixel is a defective pixel.
The arithmetic processing according to the determination result or the second adjacent pixel that replaces the adjacent defective pixel is recognized, and based on a pixel value obtained from a normal pixel that is not the defective pixel,
Since the pixel value of the target pixel is calculated, an appropriate correction can be performed on the defective pixel to be processed without replacing the defective pixel with an excessive correction value. As a result, it is possible to appropriately remove or significantly reduce the adverse effect on the captured image due to the defective pixel of the image sensor. As described above, even when a plurality of defective cells serving as defective pixels are continuously generated,
By appropriately correcting the value of the defective pixel, a good corrected image can be obtained.

[Brief description of the drawings]

FIG. 1 is a block diagram showing an embodiment in which a pixel defect correction device for an image sensor according to the present invention is applied to an image pickup device such as a digital camera.

FIG. 2 is a block diagram showing an example of an internal configuration of a digital signal processing unit in the embodiment shown in FIG.

FIG. 3 is a diagram illustrating a first defect correction method for a defective pixel of the image sensor.

FIG. 4 is a diagram illustrating a second defect correction method for a defective pixel of the image sensor.

FIG. 5 is a diagram showing a conventional defect correction example.

FIG. 6 is a diagram illustrating a correction example when the first defect correction method is applied to a color image sensor.

FIG. 7 is a diagram illustrating a correction example when the second defect correction method is applied to a color image sensor.

FIG. 8 is a diagram illustrating a conventional correction example for a color imaging element.

FIG. 9 is a block diagram showing another example of the internal configuration of the digital signal processing unit.

[Explanation of symbols]

 10 Digital camera 12 Image sensor 14 Timing control unit 20 Digital signal processing unit 21 Frame memory (FM) 22 Control unit (CPU) 28 Coordinate memory 200 Defect coordinate acquisition unit

 ──────────────────────────────────────────────────続 き Continued on the front page F-term (reference) GG12 GG17 GG18 GG26 GG30 GG32

Claims (9)

[Claims] 1. A defective pixel correction device for an image sensor, which receives a pixel signal output from an image sensor and corrects a defective pixel value of the pixel signal due to a defective pixel in the image sensor. A signal storage unit that stores a signal in a predetermined storage area; a defect coordinate obtaining unit that obtains a storage position where a pixel signal corresponding to a defective pixel in the image sensor is stored in the storage area; An adjacent pixel calculation unit configured to calculate an adjacent pixel position of a first adjacent pixel adjacent to the pixel signal to be processed, which is a pixel signal of a defective pixel stored in a storage position, in the image sensor; Defect determination means for determining whether or not a pixel is a defective pixel; and, based on a determination result of the defect determination means, based on a pixel signal stored at the adjacent pixel position, Defective pixel correction apparatus of the image pickup element characterized by comprising a correction means for correcting the pixel value of the pixel signal by a defective pixel to be processed. 2. The apparatus according to claim 1, wherein said apparatus includes coordinate storage means for storing coordinate information for specifying a position of a defective pixel of said image sensor, and said defect coordinate obtaining means includes: A defective pixel correction device for an image sensor, wherein a storage position in the signal storage means corresponding to the above is acquired. 3. The apparatus according to claim 2, further comprising means for converting the coordinate information into information indicating the storage position, and supplying the conversion result to the defective pixel coordinate obtaining means. An apparatus for correcting a defective pixel of an image sensor. 4. The apparatus according to claim 1, wherein the adjacent pixel calculating unit calculates an adjacent pixel position of a first adjacent pixel adjacent to the image sensor using the same color component. An apparatus for correcting a defective pixel of an image sensor, comprising the steps of: 5. The apparatus according to claim 1, wherein the correction unit comprises: a first pixel value obtaining unit configured to obtain a pixel value of a first adjacent pixel at the adjacent pixel position; A first calculating unit that calculates a pixel value for interpolating the target pixel based on the first adjacent pixel obtained by the first pixel value obtaining unit; and an adjacent unit that is obtained by the pixel value obtaining unit. Calculating a pixel value for interpolating the target pixel based on remaining first neighboring pixels excluding the first neighboring pixel determined to be defective by the defect determining unit among the pixels; An arithmetic unit; and a first interpolation value writing unit that writes an arithmetic value calculated by the first arithmetic unit or the second arithmetic unit to a storage position of the target pixel. A defective pixel correction device for an image sensor. 6. The apparatus according to claim 5, wherein the first arithmetic unit performs an arithmetic operation to average pixel values of a plurality of first adjacent pixels adjacent to the target pixel, and performs the second arithmetic operation. The arithmetic processing unit calculates an average value of normal first adjacent pixels that have not been determined to be defective pixels by the defect determination unit among the first adjacent pixels. Defective pixel correction device. 7. The apparatus according to claim 1, wherein the defect determining unit determines whether a first adjacent pixel at the adjacent pixel position is a defective pixel, and The calculating means determines that the first adjacent pixel is a defective pixel when the defect determining means determines that the first adjacent pixel is a defective pixel.
Calculating a neighboring pixel position of a second neighboring pixel adjacent to the neighboring pixel of the first pixel value; a first pixel value acquiring unit for acquiring a pixel value of the first neighboring pixel; A second pixel value acquisition unit that acquires a pixel value of a pixel, based on a pixel value acquired by the first pixel value unit when a first adjacent pixel corresponding to the target pixel is normal. Calculating a pixel value for interpolating the target pixel
And second interpolation value writing means for writing the calculation result calculated by the third calculation means to the storage position of the target pixel, wherein the third calculation means comprises: The defect determining unit determines that any of the adjacent pixels is a defective pixel, and a pixel value of a second adjacent pixel adjacent to the first adjacent pixel is obtained by the second pixel value obtaining unit. Then, the first
A pixel value for interpolating the target pixel is calculated based on the pixel value obtained by the pixel value obtaining means and the pixel value obtained by the second pixel value obtaining means. A defective pixel interpolation device for an image sensor. 8. The apparatus according to claim 7, wherein the third arithmetic means performs the arithmetic processing by averaging the pixel values of the first adjacent pixels acquired by the first pixel value acquiring means. A pixel value for interpolating the target pixel is calculated, and when the pixel value of the second adjacent pixel is obtained by the second pixel value obtaining unit, the normal value obtained by the first pixel value obtaining unit is obtained. The average value of the pixel value of the first adjacent pixel and the pixel value of the second adjacent pixel is calculated, and the second interpolation value writing unit stores the calculation result in the storage position of the target pixel. An apparatus for correcting a defective pixel of an image sensor, wherein writing is performed. 9. The apparatus according to claim 1, wherein the apparatus includes the image sensor.