JP2002281396A - Imaging system - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide an imaging system with high performance not to virtually generate degradation of image quality due to a defect by performing effective defect compensation. SOLUTION: The imaging system is provided with an image pickup device 12, an image pickup optical system 11 to input a subject image in the image pickup device, a defect data storage means 41 to register a pixel defect address of the image pickup device as defect data, a first defect data deciding means 40 to decide a first piece of defect data regarding photography based on the defect data registered in the defect data storage means, a defect data detecting means 40 to perform detection of the pixel defect address based on output of the image pickup device and a second defect data deciding means 40 to decide the final piece of defect data regarding the photography based on the second piece of defect data and the first piece of defect data as detection results of the defect data detecting means.

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to an imaging device, and more particularly to an imaging device having a pixel defect compensation function.

[0002]

2. Description of the Related Art Imaging devices such as video cameras have been widely used. In recent years, electronic still cameras that mainly capture and record still images have also come into widespread use especially as digital cameras, and video movies that have been mainly used for recording moving images also have a still image capturing and recording function. Long-time exposure, which is mainly used for shooting still images, increases the exposure time by increasing the charge accumulation time in the image sensor, so that shooting can be performed without using auxiliary lighting such as a strobe even under low illumination. It is known as a technology.

On the other hand, in the image pickup device, there is a dark output due to the presence of a so-called dark current, which is superimposed on an image signal, thereby deteriorating image quality. The presence of a pixel having a high dark output level is called a pixel defect, and it is widely practiced to supplement the information using the output information of neighboring pixels without using the output information of the pixel. In the present specification, such a complementing process is referred to as pixel defect compensation. A predetermined standard exposure time (for example, 1/60 second in NTSC, or 1/15 second in the case of a quadruple margin based on a predetermined margin based on the predetermined margin) which is often determined on the assumption that a moving image is driven at a used frame rate. And evaluates the dark output. Pixels having a higher level are regarded as defective pixels, and the pixel defect compensation is applied.

Further, Japanese Patent Application Laid-Open No. 06-038113 discloses a technique for improving the problem that the evaluation of defective pixels is not sufficient just before shipment from the factory because pixel defects are accompanied by temperature dependence and aging. It has been disclosed. This publication discloses that the light receiving surface is shielded by closing the iris immediately after the power is turned on, and the CC is used before the camera is used.
A technique for detecting a defective pixel by evaluating a D dark output and performing defect compensation is described.

[0005]

In a method of registering defect address information in a nonvolatile memory such as an EEPROM in advance (hereinafter referred to as a "registration method"), for example, a defect newly generated after shipment from a factory (later-occurring method). It is obvious that sexual deficiency cannot be dealt with at all. On the other hand, a camera device (as a completed product after shipment from a factory) that detects a defective pixel and performs defect compensation each time the power is turned on (hereinafter, referred to as a “detection method”) as in the above-mentioned publication. ) Can deal with late defects, but automatic detection in camera equipment generally has lower accuracy than detection in factories, and the possibility of erroneous detection of accidents (for example, due to noise). There was a problem such as high.

It should also be noted that the nature of the defect data differs slightly between the two methods. That is, in the registration method, the temperature and the exposure time (charge accumulation time) are extremely strictly controlled at the time of detecting a defect in a factory, and the defect is detected with high accuracy. And in order to adapt to the temperature environment at the time of using the camera, in general, a plurality of sets (sets) corresponding to a plurality of temperatures
Defective address data is registered. When the camera is applied to a camera, the temperature at the time of use of the camera is detected, and a corresponding one of the plurality of sets of defect data is detected.
One or more of them are selected and a predetermined conversion operation is performed, and the final defect address is determined later. On the other hand, since the defect data of the detection method is basically a defect address at the time of application, such processing is generally not required.

[0007] Accordingly, a composite defect compensation that effectively combines only the advantages of both has not been realized.

SUMMARY OF THE INVENTION It is an object of the present invention to provide a high-performance imaging apparatus in which the above-described two methods for pixel defect compensation are complementarily combined and effective defect compensation is performed so that image quality is not substantially degraded due to defects. Is to provide.

[0009]

According to the present invention, the following means have been taken in order to solve the above-mentioned problems. That is, in the present invention, the first registered result registered in the past is
After performing the respective condition judgments on the first and second defect data based on both the defect data and the second defect data newly obtained by detecting the pixel defect of the image sensor, A total defect address is determined, and based on this, defect compensation is performed.

Specifically, the electronic camera according to the present invention comprises:
An image pickup element, an image pickup optical system for inputting a subject image to the image pickup element, a defect data storage unit for registering a pixel defect address of the image pickup device as defect data, and a defect data registered in the defect data storage unit. First defect data determining means for determining first defect data relating to the photographing, defect data detecting means for detecting a pixel defect address based on an output of the image sensor, and detection of the defect data detecting means. And a second defect data determining means for determining final defect data relating to the imaging based on the second defect data as a result and the first defect data.

A preferred embodiment of the above electronic camera is as follows. In addition, each of the following embodiments may be applied independently or may be applied in an appropriate combination. (1) The second defect data determination means is configured to determine the final defect data so that at least the first defect data is all included in the defect data.

(2) The second defect data determination means determines the final defect data so that a pixel adjacent to the first defect data in the second defect data is not included in the defect data. Must be configured to

(3) The second defect data determining means sets the first defect data to the final defect data when the detection of the pixel defect address by the defect data detecting means is not performed for the photographing. It is configured to be decided.

(4) The detection of the defect data executed by the defect data detecting means is configured to be performed in response to power-on.

(5) The defect data detecting means is configured to detect the defect data immediately before the exposure for the photographing.

(6) The defect data detecting means, which is to be executed by the defect data detecting means, is configured to be performed immediately after exposure for the photographing.

(7) A light shielding means for blocking incident light from the image pickup optical system to the image pickup device, and the defect data detecting means is provided by the light shield means to input light to the image pickup device by the image pickup optical system. And detecting the pixel defect address by analyzing the output of the image pickup device obtained in this state while shutting off.

(8) Defect compensating means for compensating the output of the image sensor with respect to the photographing using neighboring pixel data based on the defect data registered in the storage means.

[0019]

Embodiments of the present invention will be described with reference to the drawings. FIG. 1 is a schematic block diagram of an electronic camera according to an embodiment of the present invention.

In FIG. 1, an optical image of a subject which has passed through a lens group 11 is converted into an electric signal by an image pickup device 12 such as a CCD, and is converted into an analog image signal by an image pickup circuit 13. The analog image signal is converted into a digital image signal (hereinafter, also referred to as “image information”) by the A / D converter 14 and is temporarily stored in the volatile built-in memory 25. This built-in memory 25 is a high-speed, for example, SDRAM (Sy
nchronousDynamic Random A
Access Memory), which is also used as a work area for image processing described later.

The image processing circuit 21 performs processing such as conversion of color information of image information temporarily stored in the built-in memory 25 and conversion of the number of pixels. The image information that has been subjected to various image processes in the image processing circuit 21 is, for example, JPEG-compressed by the compression / expansion unit 22 and recorded in the removable memory 20 such as a smart media.

When displaying a photographed image, the image information after the image processing is transmitted via the LCD driver 31.
The image is displayed on the image display LCD 30.

When displaying an image recorded in the detachable memory 20, the image information read from the detachable memory 20 is decompressed by the compression / decompression unit 22. For example, the image processing circuit 21 performs predetermined image processing. After that, the image is displayed on the image display LCD 30 in the same manner as in the shooting.

The system controller 40 reads a basic control program of the electronic camera from a nonvolatile memory 41 such as a flash memory and controls the entire electronic camera.

Further, the system controller 40 receives an input from an operation section 46 (including a release button, a cross key, etc., not shown) and performs control according to the input. The system controller 40 also controls a power supply unit (not shown) to manage the power supply of the entire electronic camera. Further, the system controller 40 controls the driving of the lens group 11 via the lens driving circuit 15.

In the electronic camera according to the embodiment of the present invention, in particular, the shutter included in the lens group 11 and the drive of the image pickup device 12 are controlled to perform exposure (charge accumulation) and signal readout, and the readout is performed by the image pickup circuit 13 and Using the information processed by the A / D converter 14, the image processing circuit 21, and the like, detection of a pixel defect described below is performed.

Further, the electronic camera has a pixel defect compensating means 23, and based on a command from the system controller 40, a non-volatile memory 41 (any other memory means can be used even if the power is turned off). Defect data relating to a defect (hereinafter referred to as “registered defect”) registered at the time of factory shipment of a camera stored in the camera, and details regarding defects detected by defect detection described below (hereinafter referred to as “detected defects”) after factory shipment. Pixel defect compensation processing is performed based on the defect data. The pixel defect compensating means 23
May be included in the image processing circuit 21. Further, a temperature sensor (not shown) is provided at an appropriate position. Note that the specification guarantee temperature of the electronic camera is 41 ° C., and the longest exposure time Tmax is 5 seconds (5 s).

Hereinafter, the control of the camera by the system controller 40 will be described focusing on the processing directly related to the detection and compensation of pixel defects according to the present invention. However, in this electronic camera, digital signal level processing is 8 bits (0 to 255)
Shall be applied. Except where otherwise noted, the description will be made assuming normal temperature.

The above-mentioned registration defects are caused by a dark medium temperature of 25.degree.
At each temperature of 3 ° C. and 41 ° C., the dark output level S measured in the charge storage operation of the exposure time (exemplary value: 1/60 s) of the camera shake limit shutter speed of the electronic camera is, for example, S> 5 (≒ 2%) is registered for each temperature (hereinafter, referred to as “registered defective pixel”).
The address of the registered pixel is a defective address (hereinafter, referred to as a defective address).
This data is referred to as “registered defect address” and this data is referred to as “registered defect data”). Therefore, when S ≦ 5, it is determined to be non-defective. This means that the dark output level at the time of actual imaging is allowed up to about 2% of the maximum full range 255, assuming at least an exposure time shorter than the exposure time (exemplary value 1/60 s) of the camera shake limit shutter speed. Means that. Here, the determination reference level of about 2% of the output level is an example, and can be arbitrarily set according to the circumstances at the time of design. % Is also effective), the possibility that the effect of the dark output superimposed on the image becomes obvious becomes sufficiently low. Further, if the determination value of the dark output level is set to 0%, it is possible to completely eliminate the pixel on which the dark output is superimposed. Therefore, such a setting is ideally preferable. However, by setting the determination value of the dark output level to 0% in this way, the pixel information is regarded as a defective pixel due to the superimposition of a slight dark signal. In some cases, the image quality may be degraded. Therefore, in reality, the dark output determination level is set in consideration of these trade-off factors.

Prior to photographing, an exposure time required for photographing is set based on a manual setting or a photometric result. Next, it waits for a shooting trigger command of the main shooting, and upon receiving the command, performs exposure based on a predetermined exposure control value, reads out an image pickup signal, performs predetermined signal processing, and then attaches / detaches the detachable memory 20.
To record. At this time, pixel defect compensation is performed for the defective pixel. The video signal processing up to the recording after the defect compensation is a known processing appropriately used as needed, for example, a color balance processing, a conversion into a luminance-color difference signal by a matrix operation or a reverse conversion processing thereof, Examples include false color removal or reduction processing by band limitation, various nonlinear processing represented by γ conversion, various information compression processing, and the like.

In the electronic camera according to the present invention,
It is assumed that the defect compensation employs a known defect compensation method of “complementing a pixel in which a defect address is registered with a neighboring pixel”. Specifically, “the closest closest color pixel (four pixels closest to the defective pixel among the same color pixels: in the case of an RGB Bayer array, for example, four G pixels obliquely adjacent to G, R Regarding (or B), it is not directly adjacent but four G
The defect compensation is performed by alternately applying the average value of the four pixel information corresponding to the four R (or B) pixels located next to each other.

The address (final defect data) of a defective pixel to be used for compensation at the time of photographing is determined by using both the above-mentioned registered defective pixel and data of a defective pixel detected by defect detection described later in detail. You.

The operation of the electronic camera according to the present invention will be described with reference to FIG. FIG. 2 is a flowchart showing the operation of the electronic camera of the present invention.

(1) First, it is determined whether or not the planned exposure time is equal to or less than the exposure time of the camera shake limit shutter speed (step A1). The data is used (step A2). In this case, defect address data (that is, first defect data) corresponding to the temperature is determined according to the detection result of the temperature sensor. If the temperature is 25 ° C or lower, the data of 25 ° C among the registered address data is
If the temperature is higher than 25 ° C. and equal to or lower than 33 ° C., the data of 33 ° C. is selected. That is, the first defect data is employed as it is as the final defect data.

In this case, it is not necessary to perform defect detection because data of a defect detected by an electronic camera, which will be described in detail later, is not necessary. For example, if the camera's exposure mode is the camera shake prevention priority mode (a mode in which the exposure time is within the camera shake limit shutter speed and the built-in strobe is automatically fired in the case of insufficient exposure in that case), a defect including the time of turning on the power supply is possible. It is preferable that the detection is not performed at all.

(2) In step A1, if the scheduled exposure time exceeds the exposure time at the camera shake limit shutter speed, the camera itself performs the process in addition to the above-described first defect data determined based on the registered defect data. The address of the defect data detected by the defect detection (hereinafter, this data (second defect data) is referred to as “detected defect data”) is also used (step A3).

Basically, the electronic camera according to the present invention performs defect detection when the power is turned on, and retains the result as detected defect data until the power is turned off. However, in consideration of a change in the temperature environment or a temperature rise due to self-heating, every time a predetermined time elapses after power-on, a defect is detected and data is updated. It should be noted that this defect detection may be performed immediately before or immediately after each photographing, irrespective of power-on. FIG. 3 shows a flow of defect detection.

First, for example, the light receiving surface of the image pickup device 12 is shielded from light by a shutter in the lens group 11 (step B1).
By performing test imaging in this state, a defect of each pixel can be detected. That is, in a state where no light is incident on the light receiving surface of the image sensor, the charge accumulation operation of the electronic camera, for example, for the longest exposure time Tmax (= 5 seconds) is performed, and the output signal from the image sensor (that is, the image signal ( = Dark output signal)) and temporarily stores it in, for example, the built-in memory 25. The output level and the reference level of the stored output data from all of the effective output pixels are compared to determine whether or not the data is defective (step B3). And
If it exceeds the reference level, it is detected as a defective pixel (step B4).

The criterion for determination is the same as that for the registered defect. However, when the output level of the pixel of interest is S, for example, in the case of a detected defect, the defect is determined when S> 5, and the other (S ≦ 5) Sometimes treated as non-defective. However, as described above, since the target shutter speed (exposure time) is different and the temperature is not controlled, even if the determination level, that is, the dark charge level at the time of detection is the same, the target is the same as the detection target. Defect levels of different pixels (dark current values, in other words, dark charge levels at the same temperature and the same exposure time) are different.

As described above, the detected defect data (second defect data), which is the pixel defect address obtained by the defect detection of the electronic camera, is the one whose address overlaps with the first defect data and the one adjacent thereto. After the removal, the first defect data is integrated with the first defect data and used as a final compensation target address (final defect data) at the time of actual imaging, thereby performing defect compensation.

As described above, in the electronic camera according to the present invention, since data detected as a new defect is adopted, it is possible to cope with a late defect. In addition, since the first defect data detected by a highly reliable detection at a factory or the like (and usually has a higher defect level) is always adopted, even if there is an erroneous detection, it is erroneously detected for defect compensation. There is no defect to remove from the target. Furthermore, since the adjacent pixels are removed, there is no possibility that defect compensation becomes impossible or the image quality is deteriorated by pixel compensation due to the defect. In this case, “adjacent” means that pixel data is provided as complementary data at the time of the defect compensation as described above.

In the electronic camera according to the present invention,
In addition to the configuration in which the defect is detected in accordance with the exposure condition, the execution of the defect detection may be arbitrarily selected as necessary (or purpose). For example, only when it is determined that the time for performing the defect detection is insufficient, the defect detection is not performed even if the defect is normally detected (or if the detection is being performed). This may be interrupted and stopped), and compensation may be performed using registered defect data without performing defect detection. An example of a case where it is determined that the time for performing the defect detection is insufficient is a predetermined time (for example, 5 seconds + slightly seconds) after the power is turned on.
There is a case where an imaging trigger command is issued before the time elapses, whereby priority can be given to not missing a so-called photo opportunity (rapid shooting). Although a slight pixel defect may be apparent in the captured image obtained in such a case, compensation is applied at least for the registered defect, so that the minimum image quality is secured by this. Become. Such a control may be performed in one of the modes selected by switching the mode by a switch operation, that is, in the "quick shooting priority mode", and in the other "image quality priority mode", the camera of the above embodiment may be controlled. preferable.

Note that the quantization level of the A / D converter used in the above embodiment actually has the error characteristic of the hardware of the A / D converter, and even if it does not exist, the principle does not matter. Considering that the quantization error relatively corresponds to 100% near the minimum quantization level, A / A used for actual quantization in the above embodiment is considered.
The D converter has a quantization bit number (8 in the embodiment) of the image processing system.
It is more preferable to use more than 10 bits, for example, about 10 bits or 12 bits (or more), so that the effects of errors can be sufficiently reduced in the calculation of each of the above arithmetic expressions.

The present invention is not limited to the above embodiment of the present invention. For example, in the above description, registered defect data is described as an embodiment in which only an initial defect in a factory is registered as defect data. However, the same pixel is detected many times as a defective pixel in detected defect data. In such a case, registered defect data may be used as in the case of the initial defective pixel. It goes without saying that various modifications can be made without departing from the scope of the present invention.

[0045]

According to the present invention, the following effects can be obtained.

As described above, in the electronic camera according to the present invention, since data detected as a new defect is employed, it is possible to cope with a late defect. In addition, since the first defect data detected by a highly reliable detection in a factory or the like (and usually has a higher defect level) is always adopted, even if there is an erroneous detection, the first defect data is erroneously subjected to defect compensation. There is no problem that it is removed from. Furthermore, since the adjacent pixels are removed, there is no possibility that defect compensation becomes impossible or the image quality is deteriorated by pixel compensation due to the defect.

[Brief description of the drawings]

FIG. 1 is a schematic block diagram of an electronic camera according to an embodiment of the present invention.

FIG. 2 is a flowchart showing the operation of the electronic camera of the present invention.

FIG. 3 is a flowchart showing a flow of defect detection.

[Explanation of symbols]

 DESCRIPTION OF SYMBOLS 11 ... Lens group 12 ... Imaging element 13 ... Imaging circuit 14 ... A / D converter 15 ... Lens etc. drive circuit 20 ... Removable memory 21 ... Image processing circuit 22 ... Compression / decompression unit 23 ... Pixel defect compensation means 25 ... Built-in memory 30 ... LCD for image display 31 ... LCD driver 40 ... System controller 41 ... Non-volatile memory 46 ... Operation unit

 ──────────────────────────────────────────────────続 き Continuation of the front page (72) Hideaki Yoshida 2-43-2 Hatagaya, Shibuya-ku, Tokyo Inside Olympus Optical Industrial Co., Ltd. (72) Takayuki Kijima 2-43-2 Hatagaya, Shibuya-ku, Tokyo Olympus Optical Co., Ltd. F term (reference) 5B047 BB04 DA06 5C024 AX01 BX01 CX23 CX24 CY44 GY01 HX14 HX29 HX59 5C072 AA01 DA15 EA05 EA08 FB16 UA11

Claims (9)

[Claims] An imaging optical system for inputting a subject image to the imaging device; a defect data storage unit for registering a pixel defect address of the imaging device as defect data; and an image data registered in the defect data storage unit. A first defect data for determining the first defect data related to the photographing based on the obtained defect data.
Defect data determination means, a defect data detection means for detecting a pixel defect address based on an output of the image sensor, second defect data which is a detection result of the defect data detection means, and the first defect And a second defect data determining means for determining final defect data relating to the photographing based on the data. 2. The method according to claim 1, wherein the second defect data determining means is configured to determine the final defect data such that at least the first defect data is all included in the defect data. The imaging device according to claim 1, wherein: 3. The second defect data determination means determines the final defect data such that a pixel adjacent to the first defect data in the second defect data is not included in the defect data. The imaging device according to claim 2, wherein the imaging device is configured as described above. 4. The second defect data determination means determines the first defect data as final defect data when the detection of the pixel defect address by the defect data detection means is not performed for the photographing. The imaging apparatus according to claim 2, wherein the imaging apparatus is configured to perform the operation. 5. The apparatus according to claim 1, wherein the detection of the defect data performed by the defect data detecting means is performed in response to power-on.
The imaging device according to any one of the above. 6. The apparatus according to claim 1, wherein the detection of the defect data performed by the defect data detecting means is performed immediately before the exposure relating to the photographing. An imaging device according to item 13. 7. The apparatus according to claim 1, wherein the detection of the defect data performed by the defect data detecting means is performed immediately after the exposure for the photographing. An imaging device according to item 13. 8. A light-shielding means for blocking incident light from the imaging optical system to the image sensor, wherein the defect data detecting means controls the incident light to the image sensor by the imaging optical system by the light-shielding means. 8. The apparatus according to claim 1, wherein the detection of the pixel defect address is performed by analyzing an output of the image sensor obtained in this state while shutting off. 2. The imaging device according to claim 1. 9. The image processing apparatus according to claim 1, further comprising a defect compensating unit that performs a compensation process based on the defect data registered in the storage unit with respect to the output of the image sensor relating to the photographing, based on neighboring pixel data. The imaging device according to claim 1.