JP2002218485A - Image data correction method and electronic camera - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide an image data correction method and an electronic camera that can properly correct image data with respect to a defective pixel whose number is increased/decreased depending on a change in an operating environment and narrows down image data to be corrected to ensure quick processing. SOLUTION: When the humidity around a solid-state image pickup element is low at actual image pickup, quick correction processing is applied to image data by reducing number of pixels that are an object of image data correction. On the other hand, when the humidity around the solid-state image pickup element is high, increasing number of target pixels for image data correction to a degree of necessity can enhance the image quality.

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

[0001] 1. Field of the Invention [0002] The present invention relates to an image data correction method for correcting image data from a solid-state imaging device containing a pixel defect and an electronic camera using the same.

[0002]

2. Description of the Related Art A solid-state imaging device provided in an electronic camera converts an optical image of a subject formed on a pixel into a charge (electric signal) by using a large number of pixels arranged two-dimensionally and outputs the charge. It has the function to do. By the way, some of such pixels may not be able to output a normal signal because they have a defect (pixel defect) based on dust adhesion, crystal defect or the like. For such a pixel defect, a signal that is obtained by adding an extra signal component to an output signal that should be output in accordance with the luminance of the subject is output, resulting in a white flaw that makes an image whitish, There is a black flaw that outputs a signal obtained by subtracting a certain signal component from an output signal that should be output in accordance with the luminance, thereby making an image blackish.

[0003] When many pixel defects occur, when an image captured using such a solid-state imaging device is reproduced, the image quality may be significantly reduced. On the other hand, since a solid-state imaging device that has recently been used has at least several hundred thousand pixels, it can be said that it is actually difficult to manufacture a solid-state imaging device having no pixel defect. Therefore, it is required to use a solid-state imaging device on the assumption that a certain degree of pixel defect always exists.

[0004] On the basis of such a premise, an electronic camera which has a correction circuit for correcting an electric signal output from a pixel having a pixel defect by post-processing to improve the image quality has already been developed. According to such an electronic camera, a pixel having a pixel defect (defective pixel) of the solid-state imaging device is detected by using a pixel defect inspection device at the time of shipment from a factory, and its position is stored as information in, for example, a ROM attached to the electronic camera. By storing the information, an output signal from the defective pixel is appropriately corrected at the time of actual imaging.

[0005]

By the way, it has been found that the above-mentioned defective pixels may fluctuate depending on the use environment of the solid-state imaging device. For example, when the humidity around the solid-state imaging device changes, the number of defective pixels may increase or decrease. In such a case, since a new defective pixel is generated at a position different from the position of the defective pixel detected under certain conditions at the time of factory shipment, sufficient correction may not be performed.

Further, there are some defective pixels which are known at the time of shipment from the factory and which cause almost no problem in actual imaging. For example, when image processing is performed on image data obtained from an image sensor by a specific image processing method during actual imaging, pixel defects may not be noticeable due to such image processing. In addition, a specific color may not be noticeable even if there is a pixel defect. Even in such a case, if it is necessary to store in the ROM up to the positions of all the defective pixels, it takes time and effort for detection,
There is a problem in that it is necessary to secure a certain amount of ROM capacity, and it takes time and effort to correct image data at the time of actual imaging.

SUMMARY OF THE INVENTION The present invention has been made in view of the above-mentioned problems of the prior art, and it is possible to appropriately correct image data with respect to defective pixels which increase or decrease in accordance with a change in a use environment. It is an object of the present invention to provide an image data correction method and an electronic camera capable of securing quick processing by narrowing down data.

[0008]

In order to achieve the above object, a first method of correcting image data according to the present invention is to remove defective pixels of the image sensor while changing the humidity around the image sensor of the electronic camera. Calculate and store the position of the defective pixel associated with the humidity, measure the humidity around the image sensor at the time of actual imaging, and based on the measured humidity and the position of the stored defective pixel, the image sensor Wherein the image data output from is corrected.

According to a second aspect of the present invention, there is provided an image data correcting method for obtaining a defective pixel for a pixel corresponding to a specific color among pixels of an image pickup device of an electronic camera provided with a plurality of color filters, and storing the position thereof. During actual imaging, the image data output from the image sensor is corrected based on the stored position of the defective pixel.

The image data correction method according to a third aspect of the present invention is characterized in that, after image processing of image data from an image sensor including defective pixels, when the level of pixel defects exceeds a predetermined range,
The position of the corresponding defective pixel is stored, and at the time of actual imaging, image data is obtained by correcting data from the defective pixel based on the stored position.

According to a fourth aspect of the present invention, there is provided a method for correcting image data including a case where the level of a pixel defect exceeds a predetermined range after image processing of image data from an image sensor including a defective pixel by a plurality of different image processing techniques. Indicates the position of the defective pixel,
The image data is stored in association with the image processing method used in the image processing, and the image data is obtained by correcting the data from the defective pixel based on the image processing method to be used and the stored position at the time of actual imaging. It is characterized by doing.

[0012]

According to the first image data correcting method of the present invention, a defective pixel of the image sensor is obtained while changing the humidity around the image sensor of the electronic camera, and the position of the defective pixel associated with the humidity is stored. During actual imaging, the humidity around the image sensor is measured, and based on the measured humidity and the position of the stored defective pixel, the image data output from the image sensor is corrected. If the humidity around the solid-state imaging device is low during imaging, the image data correction process can be performed quickly by reducing the number of pixels for which image data is to be corrected. On the other hand, when the humidity around the solid-state imaging device is high, the image quality can be improved by increasing the number of pixels for which image data is to be corrected to a necessary extent.

According to a second image data correction method of the present invention, a defective pixel is determined for a pixel corresponding to a specific color among pixels of an image sensor of an electronic camera provided with a plurality of color filters, and the position of the defective pixel is stored. At the time of actual imaging, since the image data output from the image sensor is corrected based on the position of the stored defective pixel, pixels corresponding to colors other than the specific color, which hardly affect image formation, are corrected. By not correcting the image data, the processing speed of the image data can be increased. In addition, for a pixel corresponding to a color other than the specific color, it is not necessary to detect the pixel in advance, and the capacity for storing the position of the defective pixel is small.

The specific color is RG as the color filter.
When a B filter is used, the color is preferably G color.
Note that R represents red, G represents green, and B represents blue.

The image data correction method according to the third aspect of the present invention includes a case where the level of a pixel defect exceeds a predetermined range (including a case where the level is estimated to exceed) after image processing is performed on image data from an image sensor including a defective pixel. ), The position of the corresponding defective pixel is stored, and at the time of actual imaging, image data is obtained by correcting data from the defective pixel based on the stored position, so that image processing is performed. If the level of the pixel defect falls within the predetermined range, it is not necessary to store the position of the corresponding pixel, so that the storage capacity can be reduced and the image processing time can be shortened.

The image data correction method according to the fourth aspect of the present invention provides a method for processing image data from an image pickup device including a defective pixel using a plurality of different image processing techniques, and thereafter, when the level of pixel defects exceeds a predetermined range ( (Including the case where it is estimated to exceed), the position of the defective pixel is stored in association with the image processing method used for image processing, and the image processing method to be used at the time of actual imaging and the stored position The image data is obtained by correcting the data from the defective pixel based on the above.If the level of the pixel defect falls within a predetermined range by performing the image processing using any of the image processing methods, It is not necessary to store the positions of the pixels to be processed, the storage capacity can be reduced, and the image processing time can be shortened. It is effective when a plurality of image processing methods are used.

Further, in order to determine whether or not the level of the pixel defect exceeds a predetermined range, it is preferable that a threshold value to be compared with the level is different for each pixel corresponding to each color of the color filter of the image sensor. .

[0018]

Embodiments of the present invention will be described below with reference to the drawings. FIG. 1 is a diagram showing a configuration of a digital still camera as an electronic camera according to the present embodiment. In FIG. 1, a CCD (Charge Coupled Device) as a solid-state imaging device is used.
The image sensor 1 performs a so-called photoelectric conversion, which converts an optical image of a subject formed on the pixel into an electric signal (generates an electric charge), and the drive circuit 2 generates a transfer pulse. And a circuit for supplying to the CCD1. C
The CD 1 outputs an analog electric signal based on the transfer pulse generated by the drive circuit 2.

A CDS (correlated double sampling) circuit 3 is a circuit for reducing noise, and is driven based on a driving pulse output from the driving circuit 2. The A / D conversion circuit 4 converts an input analog signal into a digital signal and outputs the digital signal. In the A / D conversion circuit 4 according to the present embodiment, it is assumed that the higher the intensity of the light incident on the pixels on the CCD 1, the larger the value of the light is converted to a digital signal. Image data for each pixel of the CCD 1 obtained through the A / D conversion circuit 4 is temporarily
It is stored in the image memory 5.

The image data stored in the image memory 5 is subjected to various types of image processing by the CPU 6 and is finally stored in a recording unit 7 composed of a recording medium such as a memory card or a magneto-optical disk.

Here, various types of image processing include processing for correcting image data of defective pixels of the CCD 1. As described later, the CPU 6 detects a defective pixel, and stores the position information (coordinates) of the defective pixel in a memory 9 provided in the control circuit 8, and corrects the image data of the defective pixel. In the processing, such position information is read out from the memory 9 to perform correction. still,
Various data used for detecting a defective pixel are also stored in the memory 9.

The liquid crystal display device 10 displays a captured image, necessary operation information, and the like. On the front surface of the CCD 1, a lens 11 for imaging light from a subject on a pixel, an aperture 12 for adjusting the amount of incident light, and a CCD
And a temperature sensor 13 for detecting the temperature of the temperature sensor 1. The detection signal of the temperature sensor 13 is input to the control circuit 8.

Further, a distance measuring means (not shown) to the subject is provided, and the control circuit 8 drives the lens driving circuit 14 to move the lens 11 to a focus position by a signal from the distance measuring means. It has become. Further, a power switch 15 and a mode switch 16 (mode selection means) for selecting a detection mode for detecting a pixel defect are provided, and a signal based on the operation of the switch is input to the control circuit 8. Has become. Meanwhile, the hygrometer 17
Are connected to the control circuit 8. The hygrometer 17 is a CCD
The humidity at the time of imaging of one area is detected, and a corresponding signal is output to the control circuit 8. The diffusion plate 21 is used for detecting a pixel defect called a black defect, and is not used for detecting a pixel defect called a white defect.

Next, detection of a defective pixel will be described.
When the power switch 15 is turned on in a state where the digital still camera is assembled, the switch signal is input to the control circuit 8, and the control circuit 8 controls the aperture 12 (light amount adjusting means) to be fully closed. In the state where incident light is restricted by the CCD 1, imaging is performed for a plurality of screens while changing humidity. The output from the CCD 1 by this imaging is
The CPU 6 compares the threshold value (reference data) for pixel defect determination stored in the memory 9 in advance with the image data for each pixel, that is, the output signal. Note that the reference data may be obtained by averaging the peripheral data of the pixel to be detected, using captured image data.

The method of obtaining the reference data will be described below with reference to the drawings. First, three colors as shown in FIG.
Consider a case in which color filters of colors B, C, and C) are arranged for each pixel. 3A to 3C show a state in which filters of the color A are two-dimensionally arranged as an example. For simplicity of description, filters of other colors are omitted. I have.

For example, in FIG. 3A, the target pixel AA
On the other hand, reference data is calculated using data of 24 pixels of the same color (color A) in a pixel of interest in an area around nine pixels around. First, a data average value of 24 pixels is obtained. This average value is used as first reference data for this pixel. The data of the pixel of interest is compared with the first reference data to determine a difference, and the difference is set as a flaw level. If it is estimated that the flaw level exceeds the threshold value when image processing performed during actual imaging is performed, it is recognized that the pixel defect has exceeded a predetermined allowable range, and the position information is stored in the memory. Note that after performing image processing performed at the time of actual imaging on the image data, the data of the target pixel is compared with the first reference data, and the difference is compared with a threshold as a defect level, thereby obtaining a pixel defect. It is also possible. Here, the color A is green (G) in the present embodiment.

Similarly, in FIGS. 3 (a) and 3 (b), with respect to the target pixel BB or CC, 24 same colors (B color or C color) in the target pixel in the area of 9 surrounding pixels are squared.
It is assumed that reference data is calculated using the pixel data of. First, a data average value of 24 pixels is obtained. This average value is used as first reference data for this pixel.
The data of the pixel of interest is compared with the first reference data to determine a difference, and the difference is set as a flaw level. If the flaw level exceeds the threshold value, it is recognized that the pixel defect has exceeded a predetermined allowable range, and the position information is stored in the memory. After performing image processing performed at the time of actual imaging on the image data, the data of the pixel of interest is compared with the first reference data,
It is also conceivable to obtain a pixel defect by comparing the difference with a threshold value as a flaw level.

The flaw level to be detected at this time may not be set to the same range for all the image data. Instead, a certain area may be divided and detected by a different reference for each area. For example, dividing the screen into a central part and a peripheral part,
The central part is less than 5%, the peripheral part is less than 10%,
The conditions for flaw detection can be changed. The percentage at this time is the ratio of the defect level to the data average value of the 24 pixels around the target pixel.

For colors other than A (B and C colors),
Pixel defects may not be detected. Red other than green,
This is because, when image processing is performed by a predetermined image processing method, the pixel defect related to blue is almost negligible to the human eye.

According to the above-described imaging, since the incident light does not reach the pixel of the CCD 1, the output signal from the pixel is less than the threshold as long as the pixel is normal.
Therefore, when the output signal is equal to or larger than the threshold, it is determined that there is a defect corresponding to a white defect. The white defect is detected by such a comparison. When a white defect is detected, the position information (two-dimensional coordinates) of the defective pixel corresponding to the white defect is associated with the humidity (for example, in the form of an LUT).
Will be stored in the memory 9. Note that black flaws can be similarly stored.

The position of the defective pixel detected as described above and stored in the memory 9 is read out at the time of actual imaging of the subject, and is used for a process of correcting an output signal relating to the image. Hereinafter, the embodiment will be described.

FIG. 4 is a flowchart showing a process for actually photographing a subject using the electronic camera according to the present embodiment. First, in step S101 in FIG. 4, the user turns on the power switch 15 (power O
N) starts the flow. Note that when the user half-presses a shutter button (not shown),
It is assumed that the switch SW1 is turned on (ON), and the switch SW2 is turned on (ON) by fully pressing the switch.

In step S102, the switch SW
When the switch 1 is not turned on, the electronic camera is maintained in a standby state. However, when the switch SW1 is turned on, in the next step S103, the exposure control by the control circuit 8 is performed, and the necessary aperture value and shutter speed are determined. , The exposure time is determined.

Further, in step S104, the humidity around the CCD 1 is measured using the hygrometer 17, and in step S105, a pixel whose output signal is to be corrected is determined. More specifically, the control circuit 8 includes a hygrometer 17
Is applied to the LUT stored in the memory 9 on the basis of the humidity measured by the above to obtain a defective pixel which causes a problem at the humidity.

In the following step S106, the control circuit 8 obtains the distance to the object by a distance measuring circuit (not shown), and drives the lens 11 to the in-focus position by the lens driving circuit 14.

In step S107, if the switch SW2 is not turned on (that is, the shutter button is not fully depressed), the electronic camera is kept in a standby state. However, when the switch SW2 is turned on, the process proceeds to step S108. The shutter is driven by the control circuit 8, and the exposure is performed for the exposure time T.

After the electric charges are accumulated in the CCD 1 by the exposure, the electric charges are transferred in step S109. The charges transferred in this manner are stored as image data for each pixel in the A / D converted image memory 5.

In step S110, the control circuit 8
Performs white flaw correction on the electric charge stored in the image memory 5 based on the information of the pixel to be corrected stored in the memory 9. The mode of this correction will be described later. After such correction, the control circuit 8 proceeds to step S111.
Then, the CPU 6 performs other signal processing, and compresses the signal in step S112. Control circuit 8
Finally, an image is recorded in the recording unit 7 in step S113, and thereafter, the flow returns to step S102.

Next, the manner of correcting white spots in step S110 will be described. First, the control circuit 8
Represents the pixel to be corrected selected based on the exposure time T,
The correction processing is performed on the charge amount corresponding to the pixel in the image data read from the memory 9 and stored in the image memory 5.

In the correction of the image data in the present embodiment, the charge amount of the defective pixel is newly obtained by averaging the charge amounts of the pixels adjacent to the defective pixel. . However,
It is also possible to correct image data by adding, subtracting, multiplying, and multiplying a correction value with respect to a charge amount from a defective pixel, or by multiplying and multiplying a correction term.

As shown in FIG. 2, the color mosaic filter is a filter of three colors of R, G and B, and a filter color A
When the pixel related to (here, green) is a defective pixel, the calculation can be performed based on any of the following formulas.

(I) First calculation formula _{An, n} = ( _{An-1, n-1} + _{An-1, n + 1} + _{An + 1, n-1} + _{An + 1, n + 1} ) / 4 (1) (ii) ) Second calculation formula _{An, n} = ( _{An-2, n-2} + _{An-2, n} + _{An-2, n + 2} + _{An-1, n-1} + _{An-1, n + 1} + _{An, n) −2} + _{An, n + 2} + _{An + 1, n−1} + _{An + 1, n + 1} + _{An + 2, n−2} + _{An + 2, n} + _{An + 2, n + 2} ) / 12 (2)

According to the average of the above equation (1), within the 3 × 3 pixel area centered on the defective pixel, the charge amount of the four pixels at the four corners provided with the same A color filter as the defective pixel is calculated. Since the average value is replaced by the charge amount of the defective pixel, it is possible to achieve harmony between improvement in image quality and improvement in processing speed.

On the other hand, according to the average of the above equation (2), within a 5 × 5 pixel area centered on the defective pixel, the average value of 12 pixels provided with the same A color filter as the defective pixel is calculated as The charge amount of the defective pixel is replaced, and the image quality can be improved.

Further, the color mosaic filters are R, G,
B is a filter of three colors, and when a pixel related to the filter color B (here, blue) is a defective pixel, the calculation can be performed based on any of the following formulas.

(I) First calculation formula B _{n, n} = (B _{n−1, n−1} + B _{n−1, n + 1} + B _{n + 1, n−1} + B _{n + 1, n + 1} ) / 4 (3) (ii) ) Second calculation formula _{Bn, n} = ( _{Bn-2, n-2} + _{Bn-2, n} + _{Bn-2, n + 2} + _{Bn-1, n-1} + _{Bn-1, n + 1} + _{Bn, n −2} + _{Bn, n + 2} + _{Bn + 1, n−1} + _{Bn + 1, n + 1} + _{Bn + 2, n−2} + _{Bn + 2, n} + _{Bn + 2, n + 2} ) / 12 (4)

According to the average of the above equation (3), within the 3 × 3 pixel area centered on the defective pixel, the charge amount of the four pixels at the four corners provided with the same B color filter as the defective pixel is calculated. Since the average value is replaced by the charge amount of the defective pixel, it is possible to achieve harmony between improvement in image quality and improvement in processing speed.

On the other hand, according to the average of the above equation (4), the average value of 12 pixels provided with the same B color filter as the defective pixel in a 5 × 5 pixel area centered on the defective pixel is calculated as follows: The charge amount of the defective pixel is replaced, and the image quality can be improved.

Further, the color mosaic filters are R, G,
B is a filter of three colors, and a filter color C (here, R
When the pixel related to (color) is a defective pixel, the calculation can be performed based on any of the following formulas.

(I) First calculation formula C _{n, n} = (C _{n−1, n−1} + C _{n−1, n + 1} + C _{n + 1, n−1} + C _{n + 1, n + 1} ) / 4 (5) (ii) ) Second calculation formula Cn _{, n} = (Cn _{-2, n-2} + Cn _{-2, n} + Cn _{-2, n + 2} + Cn _{-1, n-1} + Cn _{-1, n + 1} + Cn _{, n −2} + Cn _{, n + 2} + Cn _{+ 1, n−1} + Cn _{+ 1, n + 1} + Cn _{+ 2, n−2} + Cn _{+ 2, n} + Cn _{+ 2, n + 2} ) / 12 (6)

According to the average of the above equation (5), within the 3 × 3 pixel area centered on the defective pixel, the charge amount of the four pixels at the four corners provided with the same C color filter as the defective pixel is calculated. Since the average value is replaced by the charge amount of the defective pixel, it is possible to achieve harmony between improvement in image quality and improvement in processing speed.

On the other hand, according to the average of the above equation (6), within the 5 × 5 pixel area centered on the defective pixel, the average value of 12 pixels provided with the same C color filter as the defective pixel is calculated as: The charge amount of the defective pixel is replaced, and the image quality can be improved.

As described above, according to the present embodiment,
Since the number of defective pixels to be corrected is determined in consideration of humidity, it is possible to perform correction according to changes in the environment, thereby improving image quality.

Further, when an RGB color filter is used, by performing image processing, R (red), B
In the case where the defective pixel becomes inconspicuous with respect to the (blue) color, the defective pixel is limited to only the pixel corresponding to the G (green) color filter, and the correction is narrowed down to only the pixel corresponding to the G color to thereby reduce the image data. Correction processing can be performed quickly.

Further, as a modification of the present embodiment, FIG.
Image processing method used for the image processing performed in step 111 (for example, other than the normal image processing method, an image processing method for obtaining an image having a different contrast or color tone such as a sepia tone, an image processing method for obtaining a black and white image, etc. ), The image data for obtaining the pixel defect is subjected to image processing using all image processing methods, and only the image data exceeding the threshold value for the pixel defect is obtained. It is also conceivable to store the position of the corresponding pixel in the ROM (memory) 9 in association with the used image processing method. According to this, appropriate correction can be performed in accordance with a pixel defect that can change depending on the type of the image processing method, so that a higher quality image can be obtained.

[0056]

According to the electronic camera of the present invention, image data can be appropriately corrected with respect to defective pixels which increase or decrease in accordance with a change in the use environment, and rapid processing can be performed by narrowing down image data to be corrected. Image data correction method and an electronic camera that can ensure the image quality.

[Brief description of the drawings]

FIG. 1 is a diagram showing a configuration of a digital still camera as an electronic camera.

FIG. 2 is a diagram showing a state in which three color filters are arranged for each pixel.

FIG. 3 is a diagram illustrating a state where filters of A, B, and C colors are two-dimensionally arranged;

FIG. 4 is a flowchart illustrating a process when actually capturing an image of a subject using the electronic camera according to the embodiment;

[Explanation of symbols]

 Reference Signs List 1 CCD 2 drive circuit 3 CDS circuit 4 A / D converter circuit 5 image memory 6 CPU 7 storage unit 8 control circuit 9 memory 10 liquid crystal display device 11 lens 12 aperture 13 temperature sensor 14 lens drive circuit 15 power switch 16 mode switch 17 Hygrometer 21 Diffuser

Claims (7)

[Claims] 1. A method for determining a defective pixel of an image sensor while changing the humidity around the image sensor of an electronic camera, storing a position of the defective pixel associated with the humidity, and storing a position of the defective pixel at the time of actual imaging. And correcting the image data output from the image sensor based on the measured humidity and the stored position of the defective pixel. 2. A defective pixel is obtained for a pixel corresponding to a specific color among pixels of an image sensor of an electronic camera provided with a plurality of color filters, and the position is stored. An image data correction method, wherein image data output from the image sensor is corrected based on a position of a pixel. 3. The method according to claim 1, wherein the specific color is R.
3. The image data correction method according to claim 2, wherein the color is a G color when a GB filter is used. 4. After image processing of image data from an image sensor including a defective pixel, if the level of a pixel defect exceeds a predetermined range, the position of the corresponding defective pixel is stored. An image data correction method, wherein image data is obtained by correcting data from the defective pixel based on the determined position. 5. After performing image processing on image data from an image sensor including a defective pixel by a plurality of different image processing methods, if the level of a pixel defect exceeds a predetermined range, the position of the defective pixel is determined by image processing. And storing the image data by correcting the data from the defective pixel based on the image processing method to be used and the stored position at the time of actual image capturing. An image data correction method characterized by the following. 6. A threshold value to be compared with the level to determine whether the level of the pixel defect exceeds a predetermined range is different for each pixel corresponding to each color of the color filter of the image sensor. The image data correction method according to claim 4 or 5, wherein 7. An electronic camera using the image data correction method according to claim 1.