JP5057618B2 - Imaging device - Google Patents

Description

[0001]
BACKGROUND OF THE INVENTION
The present invention provides an imaging device. In place Related.
[0002]
[Prior art]
Imaging devices such as video cameras have been widely used conventionally. In recent years, electronic still cameras that mainly capture and record still images have become widespread, especially as digital cameras, and video movies that are mainly intended for moving image recording have a still image shooting and recording function. Long exposure, which is mainly used for still image shooting, increases the exposure time by increasing the charge accumulation time in the image sensor, so that it can be shot without using auxiliary lighting such as a strobe even under low illumination. Known as the technology to do.
[0003]
On the other hand, the image sensor has a dark output due to the presence of a so-called dark current, and this is superimposed on the image signal, resulting in image quality degradation. When there is a pixel with a large dark output level, it is called a pixel defect, and it is widely used to complement information using output information of neighboring pixels without using output information of the pixel. In the present specification, such a complementary process is referred to as pixel defect compensation. A standard exposure time that is often determined on the premise of moving image driving at a used frame rate (for example, 1/60 seconds for NTSC, or 1/15 seconds for a 4-fold margin based on this) Then, the dark output is evaluated, and the pixel defect compensation is applied to a pixel having a large level as a defective pixel.
[0004]
Further, Japanese Patent Application Laid-Open No. 06-038113 discloses a technique for improving the pixel defect because it is not sufficient to evaluate the defective pixel before shipment from the factory because it is dependent on temperature and changes with time. . In other words, this publication discloses a technique for compensating for defects by closing the iris immediately after turning on the power to shield the light receiving surface and detecting the defective pixels by evaluating the CCD dark output before using the camera. Has been.
[0005]
[Problems to be solved by the invention]
As a premise that the prior art has been used, pixel defects have been regarded as a destructive phenomenon. That is, there is a technical premise that a defect will not disappear at least under the same temperature condition once it has occurred even though there is a problem of the above-described temperature change and a late defect that has become apparent in recent years.
[0006]
However, a phenomenon that has been overlooked in the past has been found due to the recent increase in the number of pixels in the image sensor and the accompanying pixel miniaturization. This phenomenon is referred to as “flashing defect” in this specification. The cause (mechanism) of this phenomenon is not clear, but the blinking defect has an extremely low dark charge level when the image sensor is repeatedly read under the same conditions (same temperature, same accumulation time, under light shielding). This is a phenomenon in which there are pixels that increase or decrease. Such a pixel behaves like a blinking white defect, such as a normal pixel at some times and a white defect at some times.
[0007]
For such blinking white defects, if the defect is unlit at the same time by performing only one defect detection in the manufacturing process and the self-measuring function of the camera device as in the prior art In other words, it is overlooked because it cannot be detected, and it is not possible to prevent the occurrence of defects due to the lighting state during main imaging for image recording.
[0008]
An object of the present invention is to provide a high-performance imaging device in which all pixel defects including a blinking defect are compensated by performing defect detection capable of detecting a blinking defect.
[0009]
[Means for Solving the Problems]
In order to solve the above-mentioned problems, the present invention has taken the following measures.
[0014]
An image pickup apparatus according to the present invention includes an image pickup element, an image pickup optical system that inputs a subject image to the image pickup element, and defect data detection means that detects a defective pixel address of the image pickup element based on an output of the image pickup element. And an imaging apparatus comprising defect compensation means for performing compensation processing on neighboring pixel data of the defective pixel with respect to the defective pixel data based on the defect data detected by the defect data detection means, When the defect data detection means executes the defect detection operation of the imaging device, A predetermined number of times The detection result when the defect detection operation is executed The defective pixel address detected for the first time in the past detection operation is provisionally registered as a candidate for the final defective pixel address, and the temporary pixel is detected when the temporarily registered defective pixel address is detected in the next defect detection operation. Register the registered defective pixel address as the final defective pixel address It is characterized by that.
[0015]
In the above imaging device, The defect data detection means, for a plurality of image signals obtained under the same imaging conditions, For each image signal Defect judgment The number of times it was determined to be defective Less than the total number of defect judgments It is characterized in that a pixel address of a predetermined number of times or more is detected as a final defective pixel address. Note that the predetermined number of times is preferably one time, or the predetermined number of times is preferably two times.
[0016]
In each of the above-described imaging apparatuses, the imaging apparatus includes defect compensation means for performing compensation processing on neighboring pixel data of the defective pixel with respect to the defective pixel data based on the defect data detected by the defect data detection means. Features. The imaging apparatus further includes a nonvolatile memory for registering defective pixel address information of the imaging device as defect data, and a means for updating the defect data, and the defect compensation means is registered in the nonvolatile memory. Based on the defect data, the data of the defective pixel is compensated by neighboring pixel data of the defective pixel, and the updating unit detects defects in the newly detected defect data newly detected by the defect data detecting unit. It is preferable that only defective pixels that do not overlap with defective pixel addresses already registered in the nonvolatile memory among pixel addresses are additionally registered in the nonvolatile memory.
[0017]
In each of the above inventions, the same imaging conditions Before In light-shielding conditions that block the light input to the image sensor is there It is preferable.
[0018]
DETAILED DESCRIPTION OF THE INVENTION
Embodiments of the present invention will be described with reference to the drawings. FIG. 1 is a schematic block diagram of an imaging apparatus (electronic camera) according to an embodiment of the present invention.
[0019]
In FIG. 1, an optical image of a subject that has passed through a lens group 11 is converted into an electrical signal by an imaging device 12 such as a CCD, and converted into an analog image signal by an imaging circuit 13. The analog image signal is converted into a digital image signal (hereinafter also referred to as “image information”) by the A / D converter 14 and temporarily stored in the volatile built-in memory 25. The built-in memory 25 is a high-speed SDRAM (Synchronous Dynamic Random Access Memory), for example, and is also used as a work area for image processing to be described later.
[0020]
The image processing circuit 21 performs processing such as conversion of color information of image information temporarily stored in the built-in memory 25 and conversion of the number of pixels. The image information subjected to various image processing in the image processing circuit 21 is JPEG-compressed, for example, by the compression / decompression unit 22 and recorded in the removable memory 20 such as smart media.
[0021]
When displaying a captured image, the image information after the image processing is displayed on the image display LCD 30 via the LCD driver 31.
[0022]
When displaying an image recorded in the detachable memory 20, the image information read from the detachable memory 20 is expanded by the compression / decompression unit 22, for example, after predetermined image processing is performed by the image processing circuit 21. As in the case of shooting, an image is displayed on the image display LCD 30.
[0023]
The system controller 40 reads the basic control program of the electronic camera from the nonvolatile memory 41 such as a flash memory and controls the entire electronic camera.
[0024]
Furthermore, the system controller 40 receives an input from the operation unit 46 (including a release button and a cross key (not shown)) and performs control according to the input. The system controller 40 also controls the power supply of the entire electronic camera by controlling a power supply unit (not shown). Further, the system controller 40 controls driving of the lens group 11 via the lens driving circuit 15.
[0025]
In the electronic camera according to the embodiment of the present invention, exposure (charge accumulation) and signal readout are performed by controlling the shutter included in the lens group 11 and the image sensor 12 in particular, and this is performed on the image sensor 13 and the A / D. Using the information processed by the converter 14, the image processing circuit 21, etc., detection of pixel defects and the like described below are performed.
[0026]
In addition, the electronic camera has a pixel defect compensation unit 23, and the camera stored in the nonvolatile memory 41 (a storage unit that can retain the memory even when the power is turned off) is shipped from the factory based on a command from the system controller 40. Based on defect data relating to registered pixel defects (hereinafter referred to as “registered defects”) and defect data relating to pixel defects (hereinafter referred to as “detected defects”) detected by defect detection to be described later in detail after factory shipment. To perform pixel defect compensation processing. The pixel defect compensation unit 23 may be included in the image processing circuit 21. In addition, a temperature sensor (not shown) is provided at an appropriate place. The specification guarantee upper limit temperature of the electronic camera is 41 ° C., and the longest exposure time Tmax is 5 seconds (5 s).
[0027]
The camera is made up of three control states of a recording mode, a reproduction mode, and a standby mode. The recording mode is a mode in which an image is taken (image recording) by a photographing command generated by manual operation of a release button. The reproduction mode is a mode for reproducing a recorded image and displaying it on, for example, the image display LCD 30. In this playback mode, no image is taken even if the release button is operated. Therefore, for example, the playback frame may be advanced by operating the release button. For example, the camera is provided with a mode changeover switch for switching between recording / reproduction, whereby each mode is changed over and selected. The standby mode is a mode in which the minimum circuit necessary for monitoring is operated by turning off the power to the main circuit. The camera is provided with a power switch as an operation switch. By operating the power switch, the camera switches between each mode corresponding to the selection of the mode switch and the standby mode. In the recording mode or the reproduction mode, if each predetermined time (for example, 1 minute and 3 minutes) has passed without any operation, the apparatus automatically shifts to the standby mode. In the standby mode, no switch operations other than the power switch are accepted. Note that it is also possible to shift from the standby mode to the selected mode by operating the release button, and a specific switch for shifting from the standby mode to each mode can be arbitrarily set.
[0028]
Hereinafter, the control of the camera by the system controller 40 will be described focusing on the processing directly related to the detection and compensation of the pixel defect of the present invention. However, it is assumed that 8 bits (0 to 255) are applied to the digital signal level processing in this electronic camera. In addition, the description will be made assuming that the room temperature is assumed, except for the specially mentioned part.
[0029]
In the imaging mode, an exposure time required for imaging is set based on manual setting or photometry results prior to imaging. Next, it waits for a shooting command for main shooting. When receiving the command, exposure based on a predetermined exposure control value is performed, and an imaging signal is read and subjected to predetermined signal processing, and then recorded in the removable memory 20. At that time, pixel defect compensation is performed for defective pixels. Video signal processing up to recording after defect compensation is a well-known process that is used as needed, for example, color balance processing, conversion to a luminance-color difference signal by matrix calculation, or its inverse conversion processing, For example, false color removal or reduction processing by band limitation, various nonlinear processing represented by γ conversion, various information compression processing, and the like.
[0030]
In the electronic camera according to the present invention, it is assumed that defect compensation uses “a complement by neighboring pixels with respect to a pixel in which a defect address is registered”, which is a known defect compensation method. Specifically, “the closest pixels of the same color (four pixels closest to the defective pixel among the pixels of the same color: four G pixels adjacent in four diagonal directions with respect to G, R in the case of an RGB Bayer array, R For (or B), instead of directly adjoining in the four directions of up, down, left, and right, the average value of the four pixel information that is the next four R (or B) pixels positioned with one G in between is applied instead. As a result, defect compensation is performed.
[0031]
As a registration defect, the dark output level measured in the charge accumulation operation of the exposure time (example value: 1/60 s) of the camera shake limit shutter speed of the electronic camera at each of the dark temperatures of 25 ° C., 33 ° C., and 41 ° C. S is, for example,
S> 5 (≒ 2%)
Are registered for each temperature (hereinafter referred to as “registered defective pixel”). The address of the registered pixel is a defect address (hereinafter referred to as “registered defect address”, and this data is referred to as “registered defect data”). Therefore, when S ≦ 5, it is determined as non-defect. This means that the dark output level at the time of actual imaging is allowed to be about 2% of the maximum full range 255 at least when the exposure time is less than the exposure time (example value 1/60 s) of the camera shake limit shutter speed. Means that. Here, the determination reference level of about 2% of the output level is an example, and can be arbitrarily set according to the circumstances at the time of designing. However, an appropriate value of the above level (for example, about 5% or about 1 If “%” is also effective), the possibility of revealing the influence of dark output superimposed on the image is sufficiently low. Further, if the dark output level determination value is set to 0%, it is possible to completely eliminate pixels on which the dark output is superimposed. In this respect, this can also be cited as one preferred embodiment. However, when the dark output level determination value is set to 0% in this way, the pixel information is regarded as a defective pixel due to the superimposition of a dark signal, so that the total number of defective pixels increases. In some cases, the image quality may be degraded. Therefore, in reality, the dark output determination level is set in consideration of these trade-off factors.
[0032]
The camera detects defects when necessary, and additionally updates the registered defects based on the result. Defect detection is performed as follows.
[0033]
That is, after the light receiving surface of the image sensor is shielded by the shutter included in the lens group 11, test imaging is performed in the shielded state. In other words, the CCD driving circuit (not shown) performs a charge accumulation operation for the longest exposure time Tmax (setting is arbitrary: the example value 5s here) of the camera in the dark to read a test imaging signal (dark output signal), and the built-in memory 25 Once stored. Each output level is examined with respect to all the data of the stored effective output pixels, and it is determined whether or not it is a defect by performing a digital comparison with the reference level.
[0034]
Note that the address (final defect data) of the defective pixel to be used for compensation at the time of photographing is determined using both the registered defective pixel and the defective pixel data detected by the defect detection.
[0035]
The test imaging as described above and the defect determination associated therewith are repeated five times, and if it is determined as a defect even once, this pixel is detected as a final defect. In this case, as the number of test imaging increases, the probability of overlooking the flashing defect decreases, so it is preferable to increase the number of test imaging, but it takes a long time to detect it. Therefore, the number of determinations (test imaging) can be arbitrarily set in consideration of these viewpoints.
[0036]
On the other hand, if there is a possibility of erroneous determination in which the normal pixel is determined to be defective for some reason such as disturbance noise, the pixel determined to be defective even once is regarded as the final defect. In this case, a pixel that is normally normal may be erroneously detected as having a defect due to the erroneous determination. For this reason, it is preferable to determine that the number of test imaging is 10 and only pixels that are determined to be defective twice or more are defective pixels. The reason for this will be briefly described.
[0037]
Even when the number of test imaging is increased from 5 times to 10 times and the criterion is changed from 1 time to 2 times, the detection probabilities are both 1/5 and the detection time is doubled. Will not change. Note that, for example, a pixel is not specified for erroneous detection due to accidental noise mixing, and therefore, the probability that erroneous detection is performed twice out of 10 times in the same pixel among millions of pixels, for example, is extremely low, and is virtually 0. Can be considered. That is, when the determination criterion is set to once, the erroneous detection is excessively detected as a defective pixel, whereas when the determination criterion is set to two times, the probability of such erroneous detection is almost zero. be able to. In addition, even if this criterion is 3 times or more, the probability of false detection is drastically reduced as a numerical value when compared with the case of 2 times, but it is practically effective in the effect that it can be regarded as 0 in practice. This means that the detection time is simply increased. Therefore, by setting the determination criterion twice, the probability of overdetection (false detection) can be made extremely small while the detection probability of the blinking defect is substantially the same as in the above embodiment.
[0038]
In the above, “defect detection operation” of one time defines “predetermined multiple times of test imaging, defect determination corresponding to each time, and final defect detection based on these determination results”. If such a detection method is called a temporary detection method, the temporary detection method is
(1) A nonvolatile memory is not necessarily required. That is, for example, it is possible to perform defect detection every time the camera power is turned on, and apply the result to defect compensation in every shooting performed until the power is turned off.
(2) It is also suitable for application in the camera manufacturing process (adjustment process). By writing the detected defect address in the camera's non-volatile memory, it was possible to deal with flashing defects that could not be handled conventionally. You can get a camera. In this case, the restriction on the detection time is much gentler than the automatic adjustment function in the camera body, so that it is easy to perform detection with higher accuracy.
It has the characteristics. In both cases (1) and (2) (these can be used simultaneously in one camera), the basic control regarding detection is the same. FIG. 2 shows an example of a control flow when this temporary detection method is used.
[0039]
On the other hand, when implemented as an automatic adjustment function of the camera body as in the present embodiment, the “defect detection operation” is not necessarily completed once, but the past detection result (intermediate) is detected at each detection. It is also possible to reuse the data as a determination result in new detection, in other words, to use the accumulated information of the past for the next final defect detection. Such a detection method is referred to as a cumulative detection method.
[0040]
For example, in a modification using the cumulative detection method, each test imaging and defect determination is performed only once, and in the n-th defect detection, “the past n times (that is, all the past) including the current test” If it is determined that a defect has been detected twice or more in imaging and accompanying defect determination, this pixel is determined as a final defect ”. In this case, although the blinking defect that occurred later after shipment from the factory may be overlooked in several detections immediately after the occurrence, the probability of overlooking approaches zero as much as the number of subsequent detections increases. FIG. 3 (however, control of S11 to S15 ignoring branching in S14) shows a control flow in the case of such a method (referred to as cumulative detection method 1).
[0041]
In addition, in the cumulative detection method 1, in order to improve the above-mentioned problem that “the flashing defect that occurred after factory shipment may be overlooked in several detections immediately after the occurrence”, the first nine times Until the detection, the final defect may be recognized by only one detection. The branch (S16) in S14 in FIG. 3 is a process for that purpose. (However, in this case, there is a risk of erroneous detection until the first nine detections where this processing is effective.)
Further, in the cumulative detection method 1, all past data is used, but this may be applied only a predetermined number of times m. The control flow in this case (referred to as cumulative detection method 2) is shown in FIG. Note that the branch in S24 (S26) is a process that can be appropriately adopted for the same purpose as the branch in S14 (S16) in the cumulative detection method 1.
[0042]
An example of control for explaining at what timing the detection of defective pixels (including additional registration of defect data) is performed in the overall control of the camera will be described with reference to FIG. FIG. 5 is a flowchart showing a flow of defect data update.
[0043]
First, it is detected whether the camera mode is shifted from the recording mode to another mode (step A1), and defective pixels are detected when the recording mode is shifted to another mode (step A4). Here, examples of modes other than the recording mode include a playback mode and a standby mode. Then, when the detection of the defective pixel is completed, the defective pixel is additionally registered (step A5). Next, it is determined whether or not the current mode is the recording mode (step A2). If it is the recording mode, it is not preferable to detect the defect. Therefore, the process returns to step A1, and the transition from the recording mode to another mode is monitored. To do. In step A2, if it is a mode other than the recording mode, it is determined whether or not a predetermined time has passed (step A3). When the predetermined time has passed, defective pixels are detected and defective pixel data is updated (additional registration of addresses). (Step A4). In other words, what corresponds to this step A4 was the defect detection control of FIGS. 2 to 4 already described, but in the following, it will be supplemented especially regarding the registration of defect addresses performed at this time (for example, S04, S13, S23). explain.
[0044]
The registration of defective pixels in step A4 is performed as follows, for example.
Basically, an address obtained by removing a duplicated address from a registered defect is additionally registered in the nonvolatile memory 41. At this time, the detected defect data may be simply replaced with the existing registered data. However, when a defective pixel is detected, a detection error due to some cause, for example, noise occurs, and the deterioration at the time of shipment from the factory or until then. As a result, a pixel that was a defect may be treated as a non-defect, and the defect may be revealed. On the other hand, in the present invention, since the detected defect is removed from the existing registered defect and added to the existing data, it is possible to prevent the once registered defect from becoming obvious again. .
[0045]
As described above, in the present invention, defective pixels are detected when the camera shifts from the recording mode to the reproduction mode or the standby mode other than the recording mode. Since it is considered that shooting is basically not performed at the time of transition to this mode, even if a certain amount of time is required to detect a defective pixel, a problem of time lag (that is, missed photo opportunity) occurs. There is nothing. In this case, it is preferable to detect and register a defective pixel in the recording mode and then shift to a predetermined mode.
[0046]
The present invention is not limited to the above-described embodiments. In the embodiment described above, defective pixels are detected when a transition is made from a recording mode to a mode other than the recording mode, or when a predetermined time elapses in a mode other than the recording mode. In the case of transition (for example, transition from reproduction mode to standby mode or vice versa), defective pixels may be detected.
[0047]
In addition, on the basis of the initial registration data that is registered at the time of factory shipment (or a similar time), only detected defect data different from the initial registration data is added to the initial registration data every time. May be. As a result, even when a non-defective pixel is erroneously detected as a defective pixel, the result can be prevented from being accumulated, and at least with respect to the initial registration data, the defective pixel can be manifested as a non-defective pixel due to erroneous detection. It has the effect that it can be prevented.
[0048]
It should be noted that the quantization level of the A / D converter used in the above embodiment is actually the minimum in principle even if there is an error characteristic of the hardware of the A / D converter, or even if it does not exist. Considering that the quantization error is relatively equivalent to 100% in the vicinity of the quantization level, the A / D converter used for actual quantization in the above embodiment is the number of quantization bits of the image processing system. It is more preferable to use more than 10 bits (for example, 8 bits in the embodiment), for example, about 10 bits or 12 bits (or more). Can do.
[0049]
In the above embodiment, additional registration of detected defects is performed after defect detection. However, the present invention is not limited to this, and additional registration of detected defects may be performed at an arbitrary time after detection of a defective pixel, and is recorded. It is preferable to be an arbitrary time before setting the mode.
[0050]
In the above embodiment, data is updated every time the recording mode is changed to another mode. However, the data may be updated every predetermined number of times instead of every time the mode is changed. In this case, since the number of data updates is reduced, there is an effect that power consumption can be saved.
[0051]
Furthermore, in the above embodiment, only the so-called white defect is targeted by the defect detection using the light shielding unit. However, according to the defect detection using the white light (reference light) applying unit to the imaging surface, the so-called black defect is performed. Can be included.
[0052]
The present invention is not limited to the above-described embodiments. Of course, various modifications can be made without departing from the scope of the present invention.
[0053]
【Effect of the invention】
According to the present invention, since the test photographing is performed a plurality of times and the defective pixel is specified based on the predetermined determination criterion, the defect does not appear in the photographed image without overlooking the blinking defect.
[Brief description of the drawings]
FIG. 1 is a schematic block diagram of an imaging apparatus according to an embodiment of the present invention.
FIG. 2 is a flowchart showing a flow of defect detection by a temporary detection method.
FIG. 3 is a flowchart showing a flow of defect detection according to a cumulative detection method 1;
FIG. 4 is a flowchart showing a flow of defect detection by cumulative detection method 2;
FIG. 5 is a flowchart showing a flow of updating defect data.
[Explanation of symbols]
11 ... Lens group
12 ... Image sensor
13 ... Imaging circuit
14 ... D converter
15 ... Lens drive circuit
20 ... removable memory
21. Image processing circuit
22: Compression / decompression unit
23 ... Pixel defect compensation means
25 ... Built-in memory
30 ... LCD for image display
31 ... LCD driver
40 ... System controller
41 ... Non-volatile memory
46 ... operation unit

Claims (3)

An imaging device; an imaging optical system that inputs a subject image to the imaging device; a defect data detection unit that detects a defective pixel address of the imaging device based on an output of the imaging device; and the defect data detection unit An imaging apparatus comprising defect compensation means for performing compensation processing on neighboring pixel data of the defective pixel with respect to the defective pixel data based on the detected defective data,
The defect data detection means is a detection result when the defect detection operation is executed a predetermined number of times in the past when executing the defect detection operation of the imaging apparatus, and is a defective pixel address detected for the first time in the past detection operation. Is temporarily registered as a candidate for the final defective pixel address, and when the temporarily registered defective pixel address is detected in the next defect detection operation, the temporarily registered defective pixel address is used as the final defective pixel address. An image pickup apparatus that is registered as: The imaging device according to claim 1,
The defect data detecting means, for a plurality of image signals obtained by the same imaging conditions, the imaging device comprising a TURMERIC line defect determination at a predetermined criteria for each image signal. The imaging apparatus according to claim 1, further comprising: a nonvolatile memory that registers defective pixel address information of the imaging element as defect data; and a means for updating the defect data.
The defect compensation means performs a compensation process based on neighboring pixel data of the defective pixel with respect to the defective pixel data based on the defective data registered in the nonvolatile memory,
The update means includes only defective pixels that do not overlap with defective pixel addresses already registered in the nonvolatile memory among defective pixel addresses of newly detected defect data newly detected by the defective data detection means. An image pickup apparatus that is additionally registered with the camera.

Images