JP3128485B2 - Detecting device for defective pixels of solid-state image sensor - Google Patents

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a defective pixel detection circuit of a solid-state image pickup device suitable for application to an image pickup apparatus such as a video camera or an electronic still camera capable of detecting and correcting a defective pixel. .

[0002]

2. Description of the Related Art Video cameras, electronic still cameras, etc.
In an image pickup apparatus using a CD element, a so-called defective pixel which outputs a signal of a specific level in a state where light is not incident among the pixels of the CCD element has a high image quality of an image obtained by imaging. There was a problem of deterioration. Such defective pixels occur not only in initial failures but also over time. Therefore, the defective pixel is detected at the time of shipment of the imaging device body and periodically, the position information of the defective pixel is stored in the storage device, and after the image is actually captured by the imaging device and before signal processing is performed, the defective pixel is detected. In general, pixel position information is read from a storage device, and the defective pixel is one-dimensionally or two-dimensionally interpolated by an imaging signal of a surrounding non-defective pixel.

For example, Japanese Patent Application Laid-Open No. 5-260385 (H04N5 / 335) discloses that an image pickup signal level of each pixel of a CCD is compared with a predetermined threshold value in a state where light does not enter the CCD, and an abnormal level exceeding the threshold value is compared. The position of the pixel on the CCD is stored in a register, and the pixel corresponding to the position data stored in the register is stored a predetermined number of times (for example, 10 pixels).
A defective pixel detection method is disclosed which has a configuration in which when the abnormal level does not reach the abnormal level continuously, the position data is erased from the register to prevent erroneous detection due to the influence of noise or the like.

[0004]

Generally, it is necessary to include from the white level to the black level of the CCD output within the dynamic range. Therefore, in the output of the CCD at the time of shading, a normal pixel and a defect obtained by one photoelectric conversion are obtained. The level difference between the output levels of the pixels is not so large, so that it becomes very difficult to set the threshold value in the threshold processing as in the conventional example. For example, if the threshold is set too high, a defective pixel is erroneously determined to be a normal pixel. Conversely, if the threshold is set too low, a normal pixel is always erroneously recognized as a defective pixel. In particular, the output is, for example, 10
If the pixel cannot be recognized as a defective pixel unless it exceeds the threshold value consecutively, it is determined that the pixel is not a defective pixel if the value falls below the threshold value even for a moment, and the defective pixel is overlooked.

[0005]

SUMMARY OF THE INVENTION According to the present invention, there is provided a light-emitting device comprising:
By the second photoelectric conversion, the output level of each pixel of the CCD is compared with a predetermined first threshold, and the position information and the output level of the pixel lower than the first threshold are stored as a pair of defective pixels, and these candidates are stored. Only for the pixels, the cumulative value of the output level of the CCD obtained by the photoelectric conversion of a predetermined number of times thereafter is calculated for each candidate pixel, and these cumulative values are calculated from the average value of all the pixels of the CCD. In comparison with the threshold value, only candidate pixels having a cumulative value that is abnormally large with respect to the second threshold value are formally recognized as defective pixels.

[0006]

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS An embodiment of the present invention will be described below with reference to the drawings. FIG. 1 is a block diagram of a portion related to detection of a defective pixel, particularly, of an electronic still camera according to the present embodiment. In the figure, reference numeral 1 denotes an optical system composed of a plurality of lenses, and a mechanical light shielding mechanism 30 for shielding light from a subject is disposed at a position in front of the optical system 1, and the optical system 1 is manually or electrically driven to the optical system 1. Light can be blocked.

[0007] The incident light from the subject that has passed through the optical system 1 forms an image on the CCD 2 and is photoelectrically converted into an image pickup signal. still,
The CCD 2 is driven so as to photoelectrically convert incident light in one frame cycle. The CCD 2 is composed of a plurality of m pixels, and an image signal of each pixel is converted into a digital signal by an A / D converter 3 at a subsequent stage. The A / D conversion output is stored in the main image memory 4 as image data for each of all pixels. Here, the CCD 2 is driven and controlled by the reference timing signal generated from the timing signal generation circuit 5,
This reference timing signal is also supplied to the address generation circuit 6.

The address generation circuit 6 has a counter for counting a reference timing signal, and has a function of outputting the count value to each pixel of the CCD 2 as an address corresponding to one to one. The address of the pixel represented by the coordinates (1, 1) at the upper left corner is “1”, the address of the pixel at the center is “m / 2”, and the address of the pixel at the coordinates at the lower right corner is “m”.

Reference numeral 7 denotes a write / read control circuit for executing write / read control on the main image memory 4 using the address information from the address generation circuit 6 as a write address or a read address, and a system controller (not shown). The writing / reading of the main image memory 4 is controlled according to the mode signal issued from the system controller.

Reference numeral 20 denotes a switch for selecting an address output from the address generation circuit 6 and an address stored in a position memory unit 9 to be described later when supplying an address to the write / read control circuit 7. Is selectively switched to 20a, 20b, or 20c in accordance with the mode signal.

In performing the operation of detecting a defective pixel, the system controller selects one of three types of mode signals for sequentially executing three modes of a defect candidate pixel registration mode, a cumulative value calculation mode, and a defective pixel determination mode. Is output.

Reference numeral 8 denotes a position memory unit 9 for storing a pixel determined to be possibly defective as a defective candidate pixel and storing an address of the candidate pixel in the main image memory 4 as position information; Is a defect candidate pixel memory including a level memory unit 10 for storing the image data obtained in step (1) as level information.

Reference numeral 11 denotes a defect candidate pixel registration mode.
First preset threshold value THL1 and main image memory 4
Is a comparator for comparing the image data values of the respective pixels with the first threshold value THL1 based on the comparison result.
For the pixels exceeding, the first candidate pixel memory 8 stores
A registration command is issued, and in response to the first registration command, the defect candidate pixel memory 8 stores the address of the pixel and the image data value at predetermined positions in the position memory unit 9 and the level memory unit 10, respectively. When inputting the image data value from the main image memory 4 to the level memory section 10, the main image memory 4
A switch 22 that is switched by a mode signal from the system controller is disposed between the switch 22 and the level memory unit 10. In the defect candidate pixel registration mode, the switch 22 is switched to the fixed contact 22 a side and is read out from the main image memory 4. The input of the image data to the level memory unit 10 is permitted.

The switch 22 is switched to a contact point 22b in a later-described cumulative value calculation mode to allow the input of the output of the adder 14 to the level memory unit 10, and is opened when the mode is not any of these modes. The contact is switched to the contact 22c, and the data update of the level memory unit 10 is prohibited.

A switch 21 is closed only in the defect candidate pixel registration mode so that image data output from the main image memory 4 can be inputted to the subsequent adder 12 only in the defect candidate pixel registration mode. The opening and closing are controlled by.

The adder 12 calculates the sum of all the image data values of all the pixels sequentially read from the main image memory 4 in the defect candidate pixel registration mode, that is, performs digital integration, and is an adder for one screen (frame). When the sum total of the image data of all the pixels is calculated, the second threshold value THL2 used in the defective pixel determination mode is calculated by the following threshold calculation circuit 13 by the following equation (1). In this equation 1, D [i] (i
Is an integer of 1 to m) is an image data value of each pixel, m is a CCD
Indicates the total number of exposures in the defect candidate pixel registration mode and the cumulative value calculation mode, that is, the number of photoelectric conversions, and this number is preset. Q is a threshold offset value set in advance by an experiment.

[0017]

(Equation 1)

Reference numeral 14 denotes the CC in the cumulative value calculation mode.
Each time D2 is exposed every 1/30 second to output a photoelectric conversion output and update the storage content of the main image memory 4, the address of the defect candidate pixel stored in the position memory unit 9 is used as a read address. The adder reads out image data of the defective candidate pixel from the main image memory 4 and adds the read image data to the image data of the same defective candidate pixel stored in the level memory unit 10. Returning to step 10, the stored contents are updated. Therefore, the accumulated value of the defect candidate pixels is always stored in the level memory unit 10 for each defect candidate pixel.

In the cumulative value calculation mode, the switch 20
Is switched to the fixed contact 20b, and the write / read control circuit 7
, The address of the defective candidate pixel stored in the position memory unit 9 is sequentially input, and the write / read control circuit 7 uses this address as a read address to sequentially read only the image data of the defective candidate pixel.

In the defective pixel determination mode, the accumulated threshold value of the image data of each of the defective candidate pixels stored in the level memory unit 10 and the second threshold value THL2 calculated by the threshold value calculating circuit 13 are used as the defective candidate pixel values. This is a comparator that performs comparison every time, and issues a second registration command for a defect candidate pixel whose accumulated value exceeds the second threshold value THL2.

Reference numeral 16 denotes a defective pixel memory which receives the address corresponding to this pixel from the position memory unit 9 for the pixel issued from the comparator 15 and stores it as a defective pixel. When all of the accumulated value calculation mode and the defective pixel determination mode are completed, the address of the pixel determined to be defective is finally stored in the defective pixel memory 16.

Next, the operation of the above-described block diagram will be described with reference to the flowchart of FIG.

First, as shown in step S1, the light incident on the optical system 1 is cut off by the light-shielding mechanism 30. Then, as shown in step S2, the CCD 2 is exposed for the first time in this light-shielded state, that is, photoelectric conversion. I do. First, the system controller outputs a mode signal of the defect candidate pixel registration mode. In response to this, the switch 20 switches to the contact 20a side, and the write / read control circuit 7 controls the CCD 2 based on the address output from the address generation circuit 6. The outputs of all the pixels from are digitized and stored in the main image memory 4 for each pixel.

In FIG. 2, the image data of the pixel corresponding to the address i is represented by D [i], and the addresses are prepared from 1 to m as described above in order to enable the storage of the total number m of pixels. That is, there are only image data D [1] to D [m].

In step S3, the variables i and j, which are integers when executing the defect candidate pixel registration mode, are set to "1".
, The variable S is initialized to “0”.

When the image data of all the pixels is stored in the main image memory 4 in this way, the write / read control circuit 7 ends the writing of the image data in step S4, and then starts reading the data of each pixel. .

Next, as in step S4, the read image data D [i] of each pixel is compared with the first threshold value THL1 by the comparator 11, and only the pixels exceeding the threshold value THL1 are compared with the first threshold value THL1. In the candidate pixel memory 8, the address in the image memory 4 corresponding to the position on the CCD 2 and the image data value are paired as the defect candidate pixel, and in step S5, the corresponding address of this pixel is set as P [j] in the position memory unit. 9 and at the same time, the image data value of this pixel is L
It is stored in the level memory unit 10 as [j]. This operation is performed while incrementing the variable j.

For example, as shown in FIG. 2, D [i]> TH
When pixels exceeding L1 are represented by X and Y coordinates on the CCD 2, (x1, y1) (x2, y2) (x3, y3) (x
4, y4) (x5, y5) (x6, y6) (x7, y
7) In the case of (x8, y8) eight pixels, the addresses of these pixels on the main image memory 4 are respectively P [1].
~ P [8] in the position memory unit 9 and simultaneously store the image data values of these eight pixels in the level memory unit 10 respectively.
To memorize.

In step S6, while adding the image data of each pixel of the CCD 2, the total sum S of the image data values of all the pixels is added.
Then, while incrementing the variable i, step S is performed for the image data of all the pixels, ie, all the addresses.
The operation from step 4 to step S6 is repeated, and in step S7, the variable i is set to the total number of pixels m of the CCD 2, that is, the main image memory 4
Exceeds the total number of addresses prepared for storing image data, the first threshold value THL1 for image data of all pixels.
After the operation of registering a larger pixel as a defect candidate pixel is completed, step S8 is performed in preparation for a mode to be executed later.
A value U is calculated by multiplying the average value of the image data of all pixels by a predetermined integer N by U = S / m × N, and the variable k is set to “1”.
And the number J of candidate pixels stored in the candidate pixel memory 8 is derived from j-1.

Next, the mode shifts to the accumulation value calculation mode. In this mode, the variable j is initially set to "1" in step S9, and then the second exposure, that is, the CCD 2 is photoelectrically converted in the light-shielded state in step S10. The write / read control circuit 7 executes write control so as to write to the image memory 4, and the image data at all addresses of the main image memory 4 is updated with these new data.

In the cumulative value calculation mode, the switch 20
Is switched to the contact 20b side, and the switch 21 is opened. Along with this, the image data obtained by the second photoelectric conversion of the defect candidate pixels registered in the position memory unit 9 is read out from the main image memory 4, and the same candidate stored in the level memory unit 10 as in step S11. Pixel data, that is, image data obtained by the first photoelectric conversion is converted to a
4, where the two are added, and the value of the level memory unit 10 is updated with the added value. Then, the variable j is incremented. In step S12, the variable j is set to the total number of defective candidate pixels. J is determined, and the operation of step S11 is repeated for all the defect candidate pixels. When the addition of new image data for all the defect candidate pixels is completed,
In step S13, the variable k is incremented. In step S14, the variable k is compared with N-1.
The accumulation operation of 9 to S13 is repeated for (N-1) exposures.

Accordingly, the third, fourth,..., Nth photoelectric conversion outputs are accumulated for each defective candidate pixel, and the accumulated value obtained by adding all the N image data is stored in the level memory unit 10 for each candidate pixel. Will be remembered. In the example of FIG. 2, the address of the defective candidate pixel is still stored in the position memory unit 9 and the accumulated value of the eight pixels registered as the defective candidate pixel is held in the level memory unit 10. . In this cumulative value calculation mode, the switch 21
Are open, and the output from the level memory unit 10 is not output to the adder 12.

When the calculation of the accumulated value of the image data of the defective candidate pixel for N times is completed, the flow shifts to step S15 to shift to the defective pixel determination mode.

In step S15, the variables j and k are initially set to "1", and a second threshold value THL2 necessary for the determination operation in the defective pixel determination mode is first created. This second threshold value is created by adding a predetermined offset value Q to the value U created in step S9. The calculation of the second threshold value THL2 in steps S8 and S15 is as follows.
This corresponds to Equation 1 described above.

Next, in step S16, the level memory unit 1
Cumulative value L of each of the defect candidate pixels held at 0
[J] is compared with the second threshold value THL2 by the comparator 15, and a defective candidate pixel exceeding the second threshold value THL2 is regarded as a true defective pixel, and the address of this pixel is set to d in step S17.
[K] is newly transferred to the defective pixel memory 16 and the variable k is incremented. Conversely, it is determined that the defective candidate pixel having the second threshold value THL2 or less is not a true defective pixel, and the defective pixel memory 16 is transferred to the defective pixel memory 16. Do not transfer the address of

At step S18, the variable j is incremented, and at step S19, it is determined whether or not the variable j has exceeded the number J of all the candidate pixels, so that the determination operations of steps S16 to S18 are executed for all the defective candidate pixels. I do. As a result, the accumulated value among the defect candidate pixels becomes the second threshold value THL2.
Only the addresses of defective candidate pixels exceeding the number of defective candidate pixels are stored in the defective pixel memory 16.

In the example of FIG. 2, only the two pixels (x3, y3) and (x7, y7) out of the eight defect candidate pixels have the cumulative value of the image data values obtained by the N times exposure. If the value exceeds a second threshold value THL2 obtained by adding a predetermined offset value to the average value of all pixels obtained by the first exposure, only these two pixels are recognized as true defective pixels, and the addresses of these two pixels are stored in the defective pixel memory. 16 will be stored.

As described above, the output is not compared with the threshold value for each single exposure, but is compared with the cumulative value of the outputs of N exposures (multiple exposures). Even if a temporarily high output occurs, the average value gradually decreases during the continuation of the N exposures, whereas the defective pixel constantly maintains a high value. The values gradually become far apart when compared. Therefore, the accuracy of the determination by comparison with the second threshold value is improved. Further, by repeatedly adding, each output is multiplied by the number of repetitions, and the dynamic range is increased.

Further, the first threshold value THL1 used in setting the defective candidate pixel can be eventually removed even if a small number of normal pixels are included in order to prevent detection failure of the defective pixel. If it is lower, it is not necessary to set the accuracy so high.

If the above-described defective pixel detecting operation is performed before photographing each time normal photographing is performed, deterioration over time of the CCD can always be compensated, but pixel defects themselves occur so frequently in a normal CCD. However, if it is executed every time power is supplied to the camera body by turning on the power button of the electronic still camera of the present embodiment for photographing, power consumption will be a problem. Therefore, for example, after the battery of the electronic still camera is replaced, the power button is first turned on to perform the first normal shooting, and the light-shielding mechanism is operated manually or electrically to turn the CCD off and turn off the power button. Immediately thereafter, the above-described defective pixel detection operation is performed for a predetermined time. In this case, even when the power is turned off or the battery is replaced, the defective pixel memory 16
The defective pixel memory 16 is a non-volatile memory, that is, an EEP, so that the position data of the defective pixel stored therein can be stored.
It is composed of a ROM, a RAM with a battery backup, and the like.

Next, a description will be given of a case where the user performs a normal photographing with the electronic still camera with reference to FIG. By turning on the power switch provided on the electronic still camera main body to release the light shielding of the light shielding mechanism 30 and then pressing a shutter button (not shown),
The subject light imaged on the CCD 2 via the optical system 1 is photoelectrically converted and digitized by the A / D converter 3 for each pixel. These image data are stored in the main image memory 4 and all the pixels Is completed, the image data from the main image memory 4 is sequentially read out for each pixel.

Here, as shown in FIG. 4, the CCD 2 is provided with a color filter in which three primary colors of R, G, and B are arranged in a mosaic pattern, and each pixel of the CCD has R, G of these color filters. , B are mounted. Therefore, for example, when the defective pixel is R, as shown in FIG.
The image data of eight pixels that have passed through the R color filter of the same color at the oblique position is read out, a predetermined weighting is performed by the weighting amount set for each pixel to obtain a sum, and this sum is divided by the total weighting amount. This enables interpolation of defective pixels.

Therefore, when reading image data,
The write / read control circuit 7 always monitors the address of the defective pixel stored in the defective pixel memory 16 and, if the pixel to be read is not a defective pixel, reads out the image data of the address of the pixel as it is, If the pixel is defective, the image data at the address of the pixel is not read out, and the eight pixels (A11, A12, A13, A21, A23, A31,
A32, A33) are read out and input to the interpolation circuit 40 at the subsequent stage.

When the input image data is not a defective pixel, the interpolation circuit 40 outputs the input image data as it is without performing an interpolation operation. When the corresponding pixel is a defective pixel, the interpolation circuit 40 , The interpolation data of the defective pixel is calculated by substituting the image data values of the eight pixels input into the equation (2), and the image data in which the compensation of the defective pixel is completed is output. The processing is the same for the B and G pixels.

[0045]

(Equation 2)

The interpolation operation of the interpolation circuit 40 utilizes an interpolation operation using a two-dimensional digital filter. It is needless to say that the G interpolation operation in the interpolation circuit 40 can be the average value of the image data of the four pixels at the oblique position of the defective pixel as shown in FIG.

In FIG. 7, the block shown in the defective pixel detecting section 43 includes the defective candidate pixel memory 8, the comparators 11 and 15, the adders 12 and 14, and the threshold value calculating circuit 13 in FIG.
The operation here can be processed by software using a microcomputer, and the address generation circuit is not shown.

In the above-described embodiment, the address of the corresponding defective candidate pixel is transferred from the position memory unit 9 to the defective pixel memory 16 only when it is determined by the comparison in the comparator 15 that the accumulated value exceeds the second threshold value THL2. Although the second registration command for the transfer is output, the position memory unit 9 has the role of the defective pixel memory 16 and the defective pixel memory 16 is deleted in order to reduce the number of memories. Is also possible. FIG. 7 is a block diagram for realizing such a configuration. The difference from FIG. 1 is that the erase command is output from the comparator 15 to the defect candidate pixel memory 8 when the accumulated value is equal to or less than the second threshold value THL2. Upon receiving the erasure command, the defect candidate pixel memory 8 erases the address of the position memory section 9 on the assumption that the corresponding defect candidate pixel is not a true defective pixel. When the comparison of all the defective candidate pixels is completed by the comparator 15 in this way, the candidate pixels finally remaining in the position memory unit 9 are called true defective pixels. Will be used to execute the defective pixel interpolation operation.

[0049]

As described above, according to the present invention, a defect candidate element is once selected by rough threshold processing, and then the accumulated value of the image pickup signals obtained by a plurality of exposures is set as a comparison target, so that the threshold value can be set with particularly high accuracy. , It is possible to detect defective pixels with sufficiently high accuracy.

[Brief description of the drawings]

FIG. 1 is a block diagram of a portion related to defective pixel detection according to an embodiment of the present invention.

FIG. 2 is a diagram illustrating a defect candidate pixel according to an embodiment of the present invention.

FIG. 3 is a flowchart of one embodiment of the present invention.

FIG. 4 is a diagram illustrating an arrangement of color filters according to an embodiment of the present invention.

FIG. 5 is a diagram illustrating an interpolation operation in an interpolation circuit according to an embodiment of the present invention.

FIG. 6 is a diagram illustrating an interpolation operation in an interpolation circuit according to one embodiment of the present invention.

FIG. 7 is an overall block diagram according to an embodiment of the present invention.

FIG. 8 is a block diagram of a portion related to defective pixel detection according to another embodiment of the present invention.

[Explanation of symbols]

 Reference Signs List 30 light shielding mechanism 2 CCD 4 main image memory 11 comparator 8 defect candidate pixel memory 14 adder 15 comparator 16 defective pixel memory

Continuation of the front page (56) References JP-A-6-250926 (JP, A) JP-A-6-292088 (JP, A) JP-A-59-187220 (JP, A) JP-A-60-213174 (JP, A) JP-A-63-36687 (JP, A) JP-A-5-260385 (JP, A) JP-A-4-115785 (JP, A) (58) Fields investigated (Int. Cl. ⁷ , DB Name) H04N 17/00-17/06 H04N 5/30-5/335

Claims (2)

(57) [Claims] A light-blocking means for blocking light from entering the fixed image-capturing element; an image memory for storing, as image data, an image-capturing signal for each pixel of the solid-state image-capturing element obtained by exposure performed in a light-blocking state. Image data obtained by one exposure in a light-shielded state is read from the image memory, and a first comparison unit that compares the image data with a first threshold value for each pixel; A defect candidate pixel memory for storing a pixel exceeding the first threshold as a defect candidate pixel and storing position data and an image data value of the defect candidate pixel; and position data and image data value of all defect candidate pixels in the defect candidate pixel memory. Cumulative value calculating means for calculating a cumulative value of image data for each of the defect candidate pixels obtained by performing a predetermined number of exposures performed while maintaining the light-shielded state after the completion of the storage of Second comparing means for comparing the product value with a second threshold value for each of the defect candidate pixels; and comparing the position data of the defective candidate pixels whose cumulative value exceeds the second threshold value by the comparison by the second comparing means with the defective pixel position. A defective pixel detection device for a solid-state imaging device, comprising: a defective pixel memory that stores data as data. 2. A light-blocking means for blocking light from entering a fixed image pickup device, and an image memory for storing, as image data, an image signal for each pixel of the solid-state image pickup device obtained by exposure executed in a light-shielded state. Image data obtained by one exposure in a light-shielded state is read from the image memory, and a first comparison unit that compares the image data with a first threshold value for each pixel; A pixel exceeding the first threshold is regarded as a defective candidate pixel, and a defective pixel memory for storing position data and an image data value of the defective candidate pixel; and storing the position data and image data value of all the defective candidate pixels in the defective pixel memory. Is completed, a cumulative value calculating means for calculating a cumulative value of image data for each of the defect candidate pixels obtained by a predetermined number of exposures performed while maintaining the light-shielded state, A second comparing means for comparing each defective candidate pixel with a second threshold value, and comparing the position data of the defective candidate pixel whose accumulated value does not exceed the second threshold value by the comparison by the second comparing means. An apparatus for detecting a defective pixel of a solid-state imaging device, wherein the pixel is erased from a pixel memory.