JP2002185860A - Imaging device - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide an imaging device that attains effective defect correction where deterioration in the resolution cannot substantially be caused by utilizing information of effective pixels without waste, so as to minimize the deterioration in the S/N without the need for provision of a complicated hardware circuit needing much load. SOLUTION: The imaging device the sensitivity of which is enhanced through summation of pixel values, is provided with a read function that attains summation reading, where pixel information items of a unit pixel of a pixel in the unit of photoelectric conversion of a CCD image pickup element 105 picking up an object are summed. In a 1st photographing mode corresponding to an exposure time less than a prescribed time, summation reading can generate pixel information of pixels being after the summation. In a 2nd photographing mode corresponding to the exposure time being the prescribed time or over, non-summation reading is conducted, summing pixel information items of read unit pixels by an external arithmetic means generates pixel information for the pixels. In the case of the summation in the 2nd photographing mode, a defect compensation processing is conducted where the defect pixel information in the pixel information of the unit pixel is supplemented by pixel information of adjacent non-defective unit pixels.

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to an image pickup apparatus whose sensitivity is improved by adding pixels, and more particularly to an image pickup apparatus having a defect compensation function.

[0002]

2. Description of the Related Art In recent years, digital still cameras (electronic cameras) for capturing a subject image with an image sensor such as a CCD and obtaining a video signal have been actively developed. In an image pickup apparatus used in this type of electronic camera, compensation for pixel defects and pixel addition readout are known. That is, the former makes it possible to use even an imaging device having a defect, and realizes a cost that can be provided to the market.
In addition, the latter can improve the SN ratio and the dynamic range.

At this time, for example, when one pixel (this is referred to as a multi-pixel and the original pixel is referred to as a unit pixel) is obtained by adding and reading four pixels in two rows and two columns, one pixel is included in the multi-pixel. However, if a defective pixel is included, the multiple pixels become defective pixels. For this reason, if the conventional defect compensation technology is applied, the defective multi-pixel is applied (substituted) with information of the adjacent multi-pixel.

[0004]

In the above example, the defective pixel information is mixed with the defective pixel information by addition reading even though the defective multiple pixels include valid pixels that are not defective in the unit pixel. Therefore, these pieces of pixel information become invalid. Then, the correction for the defective multi-pixel is performed not based on the adjacent unit pixel closest to the defective unit pixel, but on the information of the farther adjacent multi-pixel. Therefore, the resolution was greatly deteriorated.

[0005] In order to solve this problem, in the case of using a signal readout unit from an image sensor, that is, a pixel signal addition using a sample and hold circuit, as a pixel signal addition and readout method, only a portion including a defective pixel is used in a normal manner. A technique has been proposed in which information corresponding to an effective unit pixel can be obtained by performing a sample-and-hold process different from the addition operation, thereby preventing resolution degradation. (Japanese Patent Laid-Open Publication No. H11-122538) However, this type of system has the following problems.
In other words, it is a delicate hardware circuit that contains technically advanced elements, and it is extremely difficult to ensure performance because a sample-and-hold circuit, which may be difficult to ensure stable basic performance, requires a more special switching operation. Could be Also, since it can be applied only to addition in the horizontal direction, for example, in the case of the above-described four-pixel addition in which this technology is applied most commonly after addition of two vertical lines, even if there is only one defective pixel, The information that can be effectively used is up to two pixels of information, which is wasteful. At this time, if, for example, two diagonal pixels among the four pixels are defective, the multiple pixels are defective pixels.

The present invention has been made in consideration of the above circumstances, and has as its object to eliminate the need for a complicated and heavy hardware circuit and to use the information of effective pixels without waste. It is an object of the present invention to provide an image pickup apparatus capable of performing effective defect correction with practically no resolution degradation while minimizing SN degradation.

[0007]

(Structure) In order to solve the above problem, the present invention employs the following structure.

That is, according to the present invention, in an imaging apparatus in which sensitivity is improved by pixel addition, an imaging element for imaging a subject, driving means for driving the imaging element, and photoelectric conversion of the imaging element using the driving means. A reading unit that can perform addition and reading while adding a plurality of pieces of pixel information of a unit pixel that is a pixel of a conversion unit; a storage unit that stores unit pixel defect information that is pixel defect information related to the unit pixel; In the photographing mode, pixel information of a multi-pixel, which is a pixel after addition, is generated by performing addition reading by controlling the reading means. In the second photographing mode, non-addition is performed by controlling the reading means. Image reading control means for performing read-out and adding a plurality of read-out pixel information of the unit pixel by an external arithmetic means to generate pixel information of a plurality of pixels. And wherein the door.

Here, preferred embodiments of the present invention include the following.

(1) The first photographing mode is when the exposure time is shorter than a predetermined value, and the second photographing mode is when the exposure time is longer than a predetermined value. (2) The first shooting mode is continuous shooting, and the second shooting mode is single shooting. (3) The first shooting mode is a moving image, and the second shooting mode is a still image.

(4) The imaging control means performs a defect compensation process in which defective pixel information in the pixel information of a unit pixel is applied with pixel information of a neighboring non-defective unit pixel when adding pixel information in the second imaging mode. It is configured to perform the addition process after the application.

(5) The defect compensation processing is to be applied with an average value of neighboring non-defect unit pixels belonging to the same multiple pixels. (6) The defect compensation process shall be applied using the average value of the neighboring non-defective unit pixels.

(7) The external operation means is a digital operation means.

The present invention also provides an image pickup device for picking up an object, a driving unit for driving the image pickup device, and a pixel of a photoelectric conversion unit of the image pickup device in an image pickup device in which sensitivity is improved by pixel addition. Storage means for storing unit pixel defect information, which is pixel defect information relating to the unit pixel,
Imaging control means for generating pixel information of multiple pixels by adding a plurality of pixel information of unit pixels read from the imaging element by an external arithmetic means, the imaging control means,
It is configured to perform an addition process after performing a defect compensation process that applies defective pixel information in the pixel information of the unit pixel from among the non-defective unit pixels belonging to the same multiple pixels among the neighboring non-defective unit pixels. It is characterized by.

(Operation) According to the present invention, addition reading is used in the first shooting mode such as normal exposure, but non-addition reading is used in the second shooting mode such as long-time exposure, so that external reading is performed. Perform digital addition. Therefore, in the case of long-time exposure or the like in which many defects occur, defect compensation can be performed at the unit pixel level, and deterioration in image quality due to defect compensation can be suppressed. Note that it takes time to perform addition externally after non-addition reading, but there is little problem with rate reduction in long-time exposure or the like.

As described above, the second photographing mode is suitable for compensating for a defective pixel, but takes time because of non-additional reading. Therefore, it is suitable for photographing that does not require high speed. Therefore, in continuous shooting and single shooting, if continuous shooting is the first shooting mode and single shooting is the second shooting mode,
The same effect as above can be expected. For moving images and still images, the same effects as described above can be expected if the moving image is set to the first shooting mode and the still image is set to the second shooting mode.

Further, when compensating for a defective pixel, by allocating the average value of the neighboring non-defective unit pixels with the average value of the adjacent non-defective unit pixels, the deterioration of the resolution can be minimized, and the same multiple pixels It is possible to substantially eliminate the deterioration of the resolution by applying the average value of those belonging to.

[0018]

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS The details of the present invention will be described below with reference to the illustrated embodiments.

FIG. 1 is a block diagram showing a schematic configuration of an electronic camera according to an embodiment of the present invention.

In the figure, reference numeral 101 denotes a lens system including various lenses; 102, a lens driving mechanism for driving the lens system 101; 103, an exposure control mechanism for controlling the aperture of the lens system 101; A filter for cutting, 105 is a CCD color image sensor having a built-in color filter, and 106 is a CCD for driving the image sensor 105.
CD driver 107, a pre-processing circuit including an A / D converter, etc., 108, a digital processing circuit for performing color signal generation processing, matrix conversion processing, and other various digital processing, 109, a card interface, 11
0 denotes a memory card such as a CF, and 111 denotes an LCD image display system.

In the figure, reference numeral 112 denotes a system controller (CPU) for comprehensively controlling each unit, 113 denotes an operation switch system including various switches, and 114 denotes an operation display for displaying an operation state, a mode state, and the like. System, 115 is a lens driver for controlling the lens driving mechanism 102, 116 is a strobe as a light emitting means, 117 is an exposure control driver for controlling the exposure control mechanism and the strobe 116, and 118 is various kinds of setting information and the like. Non-volatile memory (EEPROM) is shown.

In the electronic camera of the present embodiment, the system controller 112 performs overall control, and the CCD driver 106 controls the CCD image pickup device 10.
5 is controlled to perform exposure (charge accumulation) and signal readout, which are taken into a digital process circuit 108 via a preprocess circuit 107, subjected to various signal processings, and then subjected to a card interface 109 via a card interface 109. The data is recorded on the memory card 110.

FIG. 2 is a plan view showing the element structure of the CCD image pickup device 105. Photodiodes 201 are arranged in a matrix as light receiving elements.
A plurality of vertical CCDs 202 are arranged in the column direction between them,
One horizontal CCD in the row direction at the end of the vertical CCD 202
203 are arranged. And the photodiode 2
The signal charge stored in the first CCD 01 is read out to the vertical CCD 202 by the charge transfer pulse TG, and is transferred inside the vertical CCD 202 downward in the drawing. The signal charges transferred from the vertical CCD 202 are transferred to the horizontal CCD 203,
D203 is transferred leftward on the paper, and finally output via an FDA (floating diffusion amplifier) 204 which is a readout amplifier.

The basic configuration up to this point is the same as that of the conventional apparatus, but in this embodiment, in addition to this, an addition reading means for reading out the pixel information of the unit pixel while adding a plurality of pieces of pixel information to the CCD 105; And a pixel information correcting unit for correcting the pixel information. Note that the addition reading in the image sensor 105 is performed by the CCD driver 106 under the control of the system controller 112, and the addition processing and correction processing outside the image sensor are performed in the digital process circuit 108 under the control of the system controller 112. It has become.

It is assumed that the image sensor 105 used in this embodiment is of a square pixel type in which the vertical and horizontal pixel pitches are equal (value is p). Then, as shown in FIG. 3, the image pickup device 105 performs addition and readout with four 2 × 2 pixels as the minimum unit. When the minimum unit of the substantial image information is composed of a plurality of pixels (unit pixels) of the image sensor 102, this is referred to as a multi-pixel.

Next, the operation of reading out a pixel addition signal and correcting defects in the present embodiment will be described.

First, prior to image capturing (still image recording), using known photometric means (for example, so-called imager photometry, in which the system controller 112 analyzes the level of an image signal in continuous temporary image capturing, can be used). , The shutter speed, that is, the exposure time T is obtained.

If T <T0 (exemplary value T0 = 1/60 s) → first mode If the exposure time is sufficiently short, pixel defects are extremely small, so even if addition processing is performed at the time of reading, there is almost no problem in defect compensation. Absent. That is, since the number of occurrences of defects is only a very small number locally, even if they are left unchecked, there is a very small possibility that they will become apparent due to the averaging effect by addition. Alternatively, in this case, conventional defect processing (that is, in units of multiple pixels) may be used. In any case, even if the addition reading is performed, no problem is evident. Therefore, by performing the reading at a high speed, the next frame standby time is shortened and the quick shooting performance is secured.

The readout is an adjacent four-pixel addition readout by employing the well-known vertical two-pixel addition and horizontal two-pixel addition simultaneously. As shown in FIG. 4A, the vertical two-pixel addition is performed by outputting a V transfer clock for two pixels (two rows) in each horizontal blanking period. As shown in FIG. 4B, horizontal two-pixel addition is performed by outputting two units of horizontal transfer pulses between sampling gates in a CDS (correlated double sampling circuit), that is, between each reset gate pulse. Will be

If necessary, the readout data (multiple pixels) after the addition is subjected to the same defect compensation as before, or data correction is performed.

When T ≧ T0 → Second Mode As the exposure time becomes longer, the number of pixel defects increases. If a large number of defects occur, the effect on the image quality cannot be ignored even if the conventional defect processing is performed in units of multiple pixels, as well as when the defect is left unattended. At this time, in the defect processing in units of multiple pixels, as described above, a defect in one unit pixel has the same meaning as a defect in one multiple pixel (four unit pixels), which directly leads to serious image quality deterioration. . Therefore, in this case, the priority is given to ensuring image quality even at the expense of quick shooting. If the shutter is used for a long time, there is no problem since the demand for the next frame fast shooting characteristic is reduced.

For reading, non-addition reading is used, and an external digital operation is performed. That is, the unit pixel data corresponding to the non-addition reading is subjected to the defect compensation processing, and then four pixels are added. At that time, (1) the average value of other non-defective pixel data belonging to the same multiple pixel (other three pixels when the defect is only the one pixel) is applied. Thereafter, four-pixel addition is performed to obtain a multi-pixel value.
In this case, the value of the multiple pixels, which should be the sum of the four pixels, is obtained as 4/3 times the total value of the effective three pixels excluding the defective pixel. The advantage is that the resolution is not degraded because the aperture of the multiple pixels is not widened. (2) Instead of belonging to a plurality of pixels, any one of adjacent non-defective unit pixel data (for a color element, the same color and the closest pixel), for example, an average value of all target pixels may be used. In this case, there is an advantage that compensability is increased. If (1) is not possible, the process (2) is also desirable.

In the processing (1), for example, when all four pixels of the same multiple pixel are defective pixels, or when the adjacent pixels are all defective pixels in (2), the processing cannot be executed. Regards this multiple pixel as a defective multiple pixel and applies the conventional defect compensation process again in multiple pixel units.

In the above description, the acquisition of defect information is not described, but any known method can be used. For example, the data (including temperature and exposure time dependency) that was previously checked in the manufacturing process was stored in the memory,
The temperature and the exposure time during use may be referred to. In particular, this method is desirable for black defects. Further, data obtained by performing test imaging immediately before shooting, for example, in a light-shielded state at the expected use exposure time may be used. In particular, this method is excellent for white defects. It goes without saying that the data registered as each defect is the address data of the corresponding pixel.

A flowchart summarizing the above processing is as follows:
As shown in FIG. First, the exposure time T is determined, and if T <T0, addition reading is performed. If there is a defect, the defect is corrected based on the information of the adjacent multiple pixels. If there is no defect, the added and read signal is used as it is as the multi-pixel information.

When T ≧ T0, non-addition reading is performed. When there is a defect, the defect is corrected based on the information of the adjacent unit pixel. Multi-pixel information is generated by digital addition processing. If there is no defect, the read signals are added as they are to generate multi-pixel information.

As described above, according to the present embodiment, in the method of obtaining one multi-pixel by adding and reading four pixels,
In the first photographing mode such as the short-time exposure, by performing addition reading to generate pixel information of a plurality of pixels, the next frame standby time can be shortened and fast shooting performance can be secured. In the second imaging mode such as long exposure, which causes many defects, non-additional readout is performed, and pixel information of multiple pixels is generated by externally performing digital addition processing on the readout pixel information of the unit pixel. , Defect can be compensated at the unit pixel level, and deterioration can be prevented.

In particular, by correcting the defective multi-pixel based on the information of the unit pixel in the same multi-pixel instead of the adjacent multi-pixel information, the information of the effective pixel can be used without waste, and the degradation of SN can be reduced. Deterioration of resolution can be suppressed while keeping it to a minimum. Further, the present invention can be realized only by performing calculations using a simple calculation formula without requiring a complicated hardware configuration such as providing a special switching operation to the sample-and-hold circuit.

The present invention is not limited to the above embodiment. In the embodiment, the first mode and the second mode are switched according to the length of the exposure time, but another switching is also possible. In a camera having a continuous shooting mode and a single shooting mode as a shooting mode of the camera, it is needless to say that a higher next frame fast shooting property is required in the continuous shooting mode, but at the same time, a slightly higher tolerance to image quality deterioration can be expected. Therefore, it is preferable to set the first mode in the continuous shooting mode and the second mode in the single shooting mode. Similarly, when a moving image is used (in a continuous image output mode including a viewfinder), a higher frame rate is required than when a still image is recorded, but the image quality requirement such as resolution is relatively low. It is also preferable to set the mode to the second mode when recording a still image.

The present invention can be similarly applied to a color image sensor. In the case of color, the original addition is usually the same color addition processing. Depending on the coding pattern, addition other than adjacent addition is used, but the same can be applied to multiple pixels to be added. Further, the present invention is not limited to an electronic camera such as a digital still camera, and can be applied to a movie such as a video camera in the same manner.

In addition, various modifications can be made without departing from the spirit of the present invention.

[0042]

As described above in detail, according to the present invention, in an image pickup apparatus in which sensitivity is improved by pixel addition,
In the first photographing mode such as the short-time exposure, the readout unit is controlled to perform addition and reading to generate pixel information of a multi-pixel that is a pixel after addition, and to perform the second exposure such as the long-time exposure.
In the photographing mode, non-additional readout is performed by controlling the readout unit, and pixel information of a plurality of pixels is generated by adding a plurality of readout pixel information of the unit pixel by the external arithmetic unit. Therefore, it is possible to perform effective defect correction without using a complicated and heavy hardware circuit and effectively using information of effective pixels, thus minimizing SN degradation and causing virtually no resolution degradation. Becomes

[Brief description of the drawings]

FIG. 1 is a block diagram showing a circuit configuration of a digital camera according to an embodiment of the present invention.

FIG. 2 is a plan view showing an element structure of a CCD image sensor.

FIG. 3 is a diagram showing a relationship between a unit pixel and multiple pixels in a CCD image sensor.

FIG. 4 is a timing chart showing a pulse phase of addition driving.

FIG. 5 is a flowchart for selecting readout and correction calculation according to an exposure mode.

[Explanation of symbols]

 DESCRIPTION OF SYMBOLS 101 ... Lens system 102 ... Lens drive mechanism 103 ... Exposure control mechanism 104 ... Filter system 105 ... CCD color image sensor 106 ... CCD driver 107 ... Pre-process part 108 ... Digital process part 109 ... Card interface 110 ... Memory card 111 ... LCD image Display system 112 ... System controller (CPU) 113 ... Operation switch system 114 ... Operation display system 115 ... Lens driver 116 ... Strobe 117 ... Exposure control driver 118 ... Non-volatile memory (EEPROM) 201 ... Photodiode 202 ... Vertical CCD 203 ... Horizontal CCD 204: readout amplifier

 ──────────────────────────────────────────────────続 き Continued on the front page F term (reference) 4M118 AA05 AA07 AA10 AB01 BA10 CA02 DB01 FA06 GC11 GC20 5B057 BA02 BA13 CA12 CA16 CB12 CB16 CC03 CE06 CH09 DA03 DB02 DC05 5C024 AX01 BX01 CX37 CY37 EX42 GY01 HX14 HX57

Claims (7)

[Claims] An image pickup device for picking up an image of a subject, a driving unit for driving the image pickup device, and a plurality of pixel information of a unit pixel which is a pixel of a photoelectric conversion unit of the image pickup device are added by using the driving unit. A reading unit that can perform addition reading while reading, a storage unit that stores unit pixel defect information that is pixel defect information related to the unit pixel, and performs the addition reading by controlling the reading unit in the first imaging mode. In this way, pixel information of a multi-pixel that is a pixel after addition is generated, and in the second imaging mode, non-additional reading is performed by controlling the reading unit, and pixel information of the read unit pixel is externally calculated. An imaging control unit that generates pixel information of a plurality of pixels by adding a plurality of pieces of pixel information. 2. The first photographing mode is a case where the exposure time is shorter than a predetermined value, the second photographing mode is a case where the exposure time is longer than a predetermined value, the first photographing mode is continuous photographing, and the second photographing mode is The imaging apparatus according to claim 1, wherein the single shooting mode or the first shooting mode is a moving image, and the second shooting mode is a still image. 3. The image pickup control means performs a defect compensation process of applying defective pixel information in pixel information of a unit pixel with pixel information of a neighboring non-defective unit pixel when adding pixel information in a second imaging mode. 3. The image pickup apparatus according to claim 1, wherein the image pickup apparatus is configured to perform an addition process after the application. 4. The image pickup apparatus according to claim 3, wherein said defect compensation processing is performed by averaging non-defect unit pixels belonging to the same multi-pixel among neighboring non-defect unit pixels. 5. The image pickup apparatus according to claim 3, wherein the defect compensation processing is performed by using an average value of adjacent non-defective unit pixels. 6. An image pickup device for picking up an image of a subject, driving means for driving the image pickup device, and storage for storing unit pixel defect information as pixel defect information relating to a unit pixel which is a pixel of a photoelectric conversion unit of the image pickup device. Means, and imaging control means for generating pixel information of multiple pixels by adding a plurality of pixel information of unit pixels read from the image sensor by an external arithmetic means, the imaging control means, It is configured to perform the addition process after performing the defect compensation process that applies the defective pixel information in the pixel information of the unit pixel from among the neighboring non-defective unit pixels belonging to the same multiple pixels. Characteristic imaging device. 7. The imaging apparatus according to claim 1, wherein said external operation means is a digital operation means.