JP2001036820A - Image pickup device - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide an image pickup device capable of efficiently compensating a pixel defect including the case of long-time expoure. SOLUTION: This image pickup device is constituted by providing a CCD imaging device 5, exposure control mechanism 3 to control exposure for the imaging device, an operation switch system 14 to set exposure time in the case of image pickup, a pixel defect detecting part 8-1 to detect the defective pixel information in an actual operating state of the imaging device and a pixel defect compensating part 8-2 to compensate defects of image data outputted from the imaging device in the case of the image pickup based on the defective pixel information acquired by the pixel defect detecting part 8-1 when the exposure time set by the operation switch system exceeds a specified value.

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to an imaging device, and more particularly to an imaging device having a pixel defect compensation function.

[0002]

2. Description of the Related Art Imaging devices such as video cameras have been widely used. In recent years, electronic still cameras that mainly capture and record still images have also come into widespread use especially as digital cameras, and video movies that are mainly used for recording moving images also have still image capturing and recording functions. Long-time exposure, which is mainly used for still image shooting, increases the exposure time by lengthening the charge accumulation time in the image sensor, thereby shooting without using auxiliary lighting such as a strobe even under low illuminance. It is known as an enabling technology.

On the other hand, in an image pickup device, there is a dark output due to the presence of a so-called dark current or the like, which is superimposed on an image signal, thereby deteriorating the image quality. When a pixel having a large dark output level exists, it is called a pixel defect, and complementing information using output information of a neighboring pixel without using output information of the pixel has been widely used. In this specification, such processing is referred to as pixel defect compensation. A predetermined standard exposure time (for example, 1/60 second in NTSC, or 1/15 second for a 4 × margin based on this based on a predetermined margin) which is often determined on the assumption of moving image driving at the used frame rate. , The dark output is evaluated, and a pixel having a high level is regarded as a defective pixel, and the above-described pixel defect compensation is applied.

Further, Japanese Patent Application Laid-Open No. Hei 6-38113 discloses a technique for improving that defective pixels are not evaluated only before shipment from the factory because pixel defects are accompanied by temperature dependency and aging. It is disclosed in a gazette and known. That is, this publication discloses a technique of closing a iris immediately after power-on to shield a light receiving surface, and detecting a defective pixel by evaluating a CCD dark output prior to use of a camera to perform defect compensation. Has been described.

[0005]

By the way, even if a pixel is determined to have no defect or deterioration by the above-mentioned conventional technique, in the case of long-time exposure exceeding the standard exposure time, dark current is accumulated. Since the dark output level of the pixel increases, the pixel may newly become a deteriorated pixel. Since the deterioration caused by the dark current accumulation effect increases in proportion to the approximate exposure time, the longer the time, the larger the dark output level and the number of deteriorated pixels that become apparent, and the image quality is significantly reduced. The conventional technology has a problem that defect compensation cannot be performed effectively even in the case of such a long exposure.

SUMMARY OF THE INVENTION The present invention has been made to solve the above-mentioned problems in the conventional imaging apparatus, and has as its object to provide an imaging apparatus capable of compensating for a pixel defect corresponding to long-time exposure. So, to state the purpose of each claim,
It is as follows. That is, an object of the invention according to claim 1 is to provide an imaging device capable of performing pixel defect compensation according to a situation of actual use and capable of performing high-quality long-time exposure shooting. . It is another object of the present invention to provide an image pickup apparatus capable of more reliably acquiring defective pixel information corresponding to long-time exposure in an actual operation state. An object of the invention according to claim 3 is to provide an imaging apparatus which can be applied to a wide range of exposure time and can acquire defective pixel information corresponding to long exposure without error caused by calculation or the like. And A fourth object of the present invention is to provide an imaging apparatus which does not require test imaging at the time of normal exposure and does not cause a time lag, thereby obtaining high image quality.

[0007]

According to a first aspect of the present invention, there is provided an image pickup device, an exposure control unit for controlling exposure of the image pickup device, and an exposure time for image pickup. Imaging exposure time setting means, defective pixel information obtaining means for detecting defective pixel information in an actual operation state of the image sensor, and the defective pixel if the exposure time set by the imaging exposure time setting means exceeds a predetermined value. An image pickup apparatus is constituted by means for compensating for a defect in image data output from the image pickup element at the time of the image pickup based on the defective pixel information obtained by the information obtaining means. As described above, when long-time exposure equal to or more than the predetermined value is set, defective pixel information in the actual operation state of the image sensor is acquired, so that pixel defect compensation according to the actual use state is performed. And high-quality long-time exposure shooting can be performed.

According to a second aspect of the present invention, in the image pickup apparatus according to the first aspect, the defective pixel information obtaining means stores the charge in the image pickup element while exposing the image pickup element to exposure by the exposure control means. It is characterized in that a read operation is executed, and a pixel whose image sensor output level obtained in response to the read operation is equal to or higher than a predetermined level is determined as a defective pixel. As described above, since the dark output level of the test imaging in which the imaging signal is read under light shielding by an optical shutter or the like and the dark output level of which is equal to or more than a predetermined value is determined to be a defective pixel, it is possible to more reliably determine a long time in an actual operation situation. It becomes possible to acquire defective pixel information corresponding to exposure.

According to a third aspect of the present invention, in the image pickup apparatus according to the first or second aspect, the defective pixel information acquiring means includes:
It is characterized in that a defective pixel is determined based on an output of the imaging device in which charge accumulation is performed for a time corresponding to the exposure time set by the imaging exposure time setting means. As described above, the test imaging is performed in the light accumulation time corresponding to the actual exposure time under the light shielding, so that the test imaging can be applied to a wide range of exposure time, and a long time without an error caused by calculation or the like. Defective pixel information corresponding to exposure can be obtained.

According to a fourth aspect of the present invention, in the imaging apparatus according to any one of the first to third aspects, there is provided a storage unit for storing defective pixel information in a normal exposure state of the imaging element. When the exposure time set by the time setting unit is equal to or less than the predetermined value, a defect of the image data output from the imaging element at the time of the imaging is compensated based on the defective pixel information stored in the storage unit. It is characterized by comprising. In the imaging apparatus configured as described above, for normal exposure that is not long-time exposure, pixel defect compensation is performed using defective pixel information in the normal exposure state stored in the storage unit in advance. Test imaging is not required, so that high image quality can be obtained without a time lag associated therewith.

[0011]

Next, an embodiment will be described. FIG. 1 is a block diagram showing a digital camera according to a main embodiment of an imaging apparatus according to the present invention. 1 is a lens system, 2 is a lens driving mechanism, 3 is an exposure control mechanism, 4
Is a filter system, 5 is a CCD image sensor, 6 is a CCD driver, 7 is a pre-processing circuit including an A / D converter, 8
Is a digital process circuit that includes a memory as hardware and performs all digital process processing.
9 is a memory card interface, 10 is a memory card, 11 is an LCD image display system, 12 is a system controller including a microcomputer as a main configuration, 13 is an operation switch system, 14 is an operation display system including a display LCD, 15 is a strobe, 16 is a lens driver, 17 is an exposure control driver, and 18 is an EEPROM.

In the digital camera configured as described above, the system controller 12 performs overall control, and particularly includes a shutter device included in the exposure control mechanism 3 and a CCD image pickup device 5 by a CCD driver 6. To read (read out) a signal and store it in a digital process circuit 8 via a pre-process circuit 7 including an A / D converter. After processing,
The data is recorded on the LCD image display system 11 or the memory card 10. The various types of signal processing performed by the digital process circuit 8 include pixel defect detection processing and pixel defect compensation processing using output level information, which are essential parts of the present invention. That is, the digital process circuit 8 includes a pixel defect detector 8-1 for detecting a pixel defect using the output level information stored in the digital process circuit 8, and a pixel detected by the pixel defect detector 8-1. A pixel defect compensating unit 8-2 for compensating for pixel defects based on the defect information. In the pixel defect compensating section 8-2, based on the address data of the defective pixel relating to the defect at the time of the normal exposure (hereinafter referred to as a normal defect) stored in the EEPROM 18 in advance, at the time of the long-time exposure, furthermore, A pixel defect compensation process is performed based on pixel address data of a defective pixel relating to a defect (hereinafter, referred to as a detected defect) as a result of the pixel defect detection performed by the pixel defect detection unit 8-1.

Next, camera control by the system controller 12 will be described focusing on processing directly related to pixel defect detection and compensation in the present embodiment. However, the digital processing of the signal level in the digital camera according to the present embodiment is performed with 8 bits (0 to 255).

First, prior to photographing, an exposure time Ttotal necessary for photographing is set based on a manual setting or a photometric result (however, the longest exposure time is 10 seconds as a specification of this digital camera). It is determined whether or not this is longer than the standard exposure time Tstd (which can be arbitrarily set, but is 1/15 second in this example) in the present digital camera, and the following processing is performed.

(1) In the case of Ttotal ≦ Tstd, the apparatus waits for a shooting trigger command for the main image capturing, which is the same as the prior art, and performs exposure based on a predetermined exposure value in response to the command.
After the image pickup signal is read out and subjected to predetermined signal processing, it is recorded on the memory card 10. At that time, pixel defect compensation corresponding to the above, that is, only for the always defective pixel is accompanied. (2) If Ttotal> Tstd, first, prior to actual shooting, specifically, when a shooting trigger command is received,
First, test imaging is performed in a state where the light receiving surface of the image sensor is shielded from light by a shutter device included in the exposure control mechanism 3. That is, the predetermined exposure time Tte is determined by the CCD driver 6 in the dark.
A charge accumulation operation of st = 5 × Tstd is performed, a test image pickup signal (dark output signal) is read, and stored in the digital process circuit 8. Then, in the pixel defect detection section 8-1, each output level is checked for the effective output pixels excluding at least the normal defective pixels in all the stored data, and a defect is determined by performing a digital comparison with the reference level. .

The criteria are as follows. That is, assuming that the output level of the pixel of interest is S, a defect occurs if S> 25 × Ttest / Ttotal, otherwise (S ≦ 25 × Ttest / Ttotal)
At the time of the non-defect. This means that the dark output is almost proportional to the accumulation time, and the dark output level at the time of the main imaging is allowed to be 25 or less (about 10% of the full range 255). Therefore, for example, Ttotal = T
In the case of std, all pixels having a dark output level of 125 or less during test imaging are regarded as non-defective, and Ttotal is 25/3 (≒ 8.3
3) When the time exceeds 2 seconds, the dark output level during test imaging is 0
Is regarded as a non-defective pixel.

Here, the judgment criterion of the output level 25 (about 10%) is not necessarily the only absolute one, but can be arbitrarily set according to the circumstances at the time of design. If (for example, about 5% or 3% is also effective) is selected, the possibility that the influence of the dark output superimposed on the image becomes apparent becomes sufficiently low. If this is selected to be 0%, it is possible to completely eliminate the pixel on which the dark output is superimposed. In this regard, this can also be cited as a preferred modified example. Because of the increase, the resolution is likely to be degraded. In reality, the criterion is set in consideration of these trade-off factors.

The detected defective pixel address obtained in this manner is stored in an appropriate area of the digital process circuit 8, integrated with the normal defective address data stored in the EEPROM 18, and used for pixel defect compensation processing. You. At this time, the detected defective pixel address is temporarily set to EEPRO.
It may be used after being stored in M18. At the time of defect compensation, the ordinary defect and the detected defect are processed as completely the same defective pixel, and there is no difference.

As described above, after obtaining the address data of the defective pixel in this case, the main exposure based on the predetermined exposure value is performed and the image pickup is performed in the same manner as after the photographing trigger command in the case (1). After reading out the signal and performing predetermined signal processing, the signal is recorded on the memory card 10. At that time, the defective pixel corresponding to the above, that is, including the detected defective pixel, is accompanied by pixel defect compensation.

At this time, a known process can be used as the pixel defect compensation process itself. In this embodiment, however, the process is particularly performed as follows. That is, "the data of the pixel of the closest non-defective same-color pixel of the defective pixel is used as the data of the defective pixel. However, when there are a plurality of corresponding pixels, the representative value is used."
The closest non-defective same-color pixel is a pixel of the same color filter that is not a defective pixel (all pixels in the case of a monochrome CCD).
The pixel closest to the defective pixel. Further, the method of obtaining the representative value may be any of a selection method (for example, employing the data of the leftmost and uppermost pixel) and an interpolation method (for example, employing an average value of each data).

Incidentally, the known defect compensation imposes certain constraints (for example, two consecutive pixels do not exist) on the defective pixel itself, and an image sensor that does not satisfy this condition is regarded as defective. Since it was presupposed, it could be simplified so that it was simply complemented with the representative value of the closest same-color pixel (of the same-color pixels, the pixel closest to the defective pixel). Since a defect detected by the camera immediately before is also targeted, it is extremely difficult to impose an arbitrary constraint condition. For example, there may be a case where all the pixels of the same color closest to each other are defective pixels. For this reason, interpolation using the closest non-defective same-color pixels is used.

In each of the cases (1) and (2),
The video signal processing up to the recording after the defect compensation is processed in common, but the processing in the circuit at the subsequent stage is appropriately used according to the necessity. To a luminance-color difference signal or its inverse conversion processing, false color removal or reduction processing by band limitation, various nonlinear processing represented by γ conversion, various information compression processing, and the like.

According to the above-described embodiment, it is not necessary to perform test imaging for a normal exposure time shorter than the predetermined standard exposure time Tstd (therefore, there is no occurrence of a time lag associated with test imaging). High quality image with the same pixel defect compensation as
For long exposures that would otherwise cause image quality degradation,
By detecting a defective pixel according to the situation at that time by the test imaging, a high-quality image with appropriate pixel defect compensation can be obtained. In addition, all of these situation determinations are automatically performed, and the test imaging time is also short (1/3 second in the above example), so that no inconvenience or uncomfortable feeling occurs in the imaging operation. Further, since the timing of the test imaging is not at the time of turning on the power as in the conventional example disclosed in the above-mentioned publication, but immediately before the main imaging, the actual timing including the influence of the internal temperature rise due to the lapse of time after the power is turned on is considered. In this case, it is possible to perform compensation without malfunction according to the situation used.

In the above description, the charge accumulation time and the exposure time are regarded as the same for the sake of simplicity.
Strictly speaking, charge accumulation is started before the start of exposure using a mechanical shutter, or charge is transferred to the transfer path a predetermined time after the exposure is completed, or a predetermined time after the stored charge is transferred to the transfer path. In some cases, such as when a so-called delayed read method for starting transfer is used, the two may not always match. However, since the difference between the two is managed and recognized by the system controller, the above-described embodiment may be applied by specifically considering the difference as necessary.

Further, various embodiments of the present invention can be considered in addition to the above-described main embodiment. First, Tte
The time setting of st is 5 in the main embodiment.
A value of × Tstd = 1/3 seconds is shown, but this can be arbitrarily changed. In the case of the above main embodiment, since the longest exposure time of this camera is 10 seconds, Ttest is 1/3.
Seconds were available. With such a short time, it is possible to perform test imaging immediately before the main imaging in a normal camera sequence without making the user aware, and without reducing the operability of the camera at all. The present invention can be applied. On the other hand, in order to cope with a longer exposure time, Ttest can be set to an arbitrary longer time, test imaging can be performed several times with different Ttest values, and Ttest = Ttotal instead of a fixed value. Each of them is a preferred embodiment, such as a configuration in which image quality can be determined for an arbitrary exposure time.

The timing of the test imaging is preferably immediately before the actual imaging, but is not limited to this. For example, when the power is turned on, when switching to the shooting mode in a camera that can be switched from the reproduction mode, when the camera is switched to the shooting mode in the two-stage release switch camera (the second stage is a shooting trigger).
The operation may be performed at any time according to the purpose, such as at the time of operating the stage or operating a separately provided test imaging switch.

From a different viewpoint, the determination level of the always defective pixel and the determination level of the detected defective pixel may not be equal. Rather, for the always defective pixel,
It is not an image taken under the special condition of "long exposure" and determines the normal image quality. Therefore, in general, it is not merely determined by the dark output level but, for example, "a continuous defect exists. The imaging device is considered to be a non-defective product only when the distribution condition such as `` Do not exist '' and `` No more than n defective pixels must be present in the predetermined circle at the center of the screen '' is satisfied. It can be said that it is indispensable to adopt the inspection standards to be performed. On the other hand, in the case of automatic detection by a camera, such a judgment (which can also result in the quality of the image sensor) is impossible in the first place, and these judgments are simply compared in the same row. Can be difficult. However, in the sense of general requirements for product specifications, performance (in this case, image quality) requirements under “special conditions” such as “long exposure” are “basic performance (“ normal ”)
Therefore, it is one desirable example to set the determination level of the detected defective pixel to a higher level (a looser reference) than that at the time of the normal defective pixel determination. The fact that the present invention separates the processing between the case where the exposure time is equal to or less than the predetermined standard exposure time Tstd and the case where the processing time exceeds the standard exposure time Tstd has a great advantage that such setting is possible.

Note that supplementing the quantization level of the A / D converter used in the main embodiment, in reality, even if the error characteristic of the A / D converter hardware does not exist or if it does not exist, it is theoretically required. Near the minimum quantization level, the quantization error is relatively 100
Considering that the A / D converter also corresponds to%, the A / D converter used for the actual quantization in the main embodiment is larger than the number of quantization bits (8 bits in the main embodiment) of the image processing system, for example, 10 bits. It is more preferable to use bits or about 12 bits, so that the effects of errors can be sufficiently reduced in the calculation of each of the above arithmetic expressions.

Although the present invention has been described with reference to several embodiments, the present invention is not limited to these embodiments, and can take any form as long as it is described in the appended claims. Needless to say,

[0030]

As described above with reference to the embodiments, according to the present invention, it is possible to realize an imaging apparatus capable of compensating for a pixel defect corresponding to a long-time exposure. In particular, according to the first aspect of the present invention, when long-time exposure equal to or more than a predetermined value is set, defective pixel information is obtained in an actual operation state of the image sensor. In this case, it is possible to perform pixel defect compensation according to the situation, and to perform high-quality long-time exposure imaging. According to the invention of claim 2, since the dark output level of the test imaging in which the imaging signal is read out under light shielding by an optical shutter or the like is determined to be a defective pixel, the pixel is determined to be defective. It is possible to reliably obtain defective pixel information corresponding to long-time exposure in an actual operation situation. According to the invention according to claim 3, under light shielding,
Since the test imaging is performed during the charge accumulation time corresponding to the actual exposure time, it can be applied to a wide range of exposure times and has no errors caused by calculations, etc. Information can be obtained. According to the fourth aspect of the present invention, pixel defect compensation is performed for normal exposure that is not long-time exposure using defective pixel information in the normal exposure state stored in the storage unit in advance. At the time of normal exposure, test imaging is unnecessary, so that high image quality can be obtained without generating a time lag associated therewith.

[Brief description of the drawings]

FIG. 1 is a schematic block diagram showing an overall configuration of a digital camera according to an embodiment of an imaging apparatus according to the present invention.

[Explanation of symbols]

 DESCRIPTION OF SYMBOLS 1 Lens system 2 Lens drive mechanism 3 Exposure control mechanism 4 Filter system 5 CCD image sensor 6 CCD driver 7 Pre-process circuit 8 Digital process circuit 8-1 Pixel defect detector 8-2 Pixel defect compensator 9 Memory card interface 10 Memory card 11 LCD image display system 12 System controller 13 Operation switch system 14 Operation display system 15 Strobe 16 Lens driver 17 Exposure control driver 18 EEPROM

 ──────────────────────────────────────────────────続 き Continued on the front page F term (reference) 4M118 AA07 AA09 AB01 BA10 FA06 5C024 AA01 BA01 CA09 CA24 DA01 DA04 FA01 GA11 HA08 HA12 HA14 HA24

Claims (4)

[Claims] 1. An image pickup device, an exposure control unit for controlling exposure of the image pickup device, an image pickup exposure time setting unit for setting an exposure time at the time of image pickup, and detecting defective pixel information in an actual operation state of the image pickup device Defective pixel information acquisition means, and when the exposure time set by the imaging exposure time setting means exceeds a predetermined value, is output from the imaging element at the time of the imaging based on the defective pixel information acquired by the defective pixel information acquisition means. Means for compensating for a defect in the image data. 2. The defective pixel information acquiring means executes a charge accumulation and readout operation in the image pickup device in a state in which exposure to the image pickup device is cut off by the exposure control means, and an image pickup corresponding to the operation is performed. 2. The imaging apparatus according to claim 1, wherein a pixel whose element output level is equal to or higher than a predetermined level is determined as a defective pixel. 3. The defective pixel information acquiring means determines a defective pixel based on an output of the image pickup element which has accumulated electric charges for a time corresponding to the exposure time set by the imaging exposure time setting means. The imaging device according to claim 1, wherein the imaging device is configured. 4. A storage device for storing defective pixel information in a normal exposure state of the image pickup device, wherein when the exposure time set by the image pickup exposure time setting device is equal to or less than the predetermined value, the storage device 4. The imaging device according to claim 1, wherein the imaging device is configured to compensate for a defect in image data output from the imaging device at the time of the imaging based on the defective pixel information stored in the imaging device. apparatus.