JP2009105582A - Noise correction circuit, imaging device and noise correction method - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To make it possible to correctly perform correction of any defective pixel regardless of existence of temperature characteristics. <P>SOLUTION: As for a defective pixel of an image sensor, a defective pixel with temperature characteristics which varies in level with temperature and a defective pixel with no temperature characteristics which does not depend upon temperature are discriminated and pieces of position information on the defective pixels are stored in a memory 30. When a video signal that the image sensor outputs is a signal of the defective pixel with the temperature characteristics stored in the memory 30, a subtracter 32 subtracts a correction signal having a gain set corresponding to temperature nearby the image sensor from the video signal. Further, when the video signal that the image sensor outputs is a signal of the defective pixel with no temperature characteristics stored in the memory 30, the subtracter 32 subtracts a correction signal having a constant gain from the video signal. <P>COPYRIGHT: (C)2009,JPO&INPIT

Description

  The present invention relates to a noise correction circuit that removes noise due to defective pixels included in a video signal output from an image sensor, an imaging apparatus including the noise correction circuit, and a noise correction method applied to the imaging apparatus.

Conventionally, various solid-state imaging devices using sensors such as a complementary metal oxide semiconductor (CMOS) image sensor and a charge coupled device (CCD) image sensor have been developed and put into practical use.
In the case of these solid-state imaging devices, there is fixed pattern noise that exists at the same pixel position regardless of time. One of the fixed pattern noises is due to pixel defects on the image sensor.

Noise due to pixel defects is classified into noise caused by defective pixels that are always high-luminance pixels (so-called white spot noise) and noise that is always defective pixels that are low-luminance pixels (so-called black spot noise).
A correction signal is added to the pixel signals of these defective pixels so as to perform correction processing to correct pixel signals.
Patent Document 1 describes an example of a correction method for this pixel defect.
Japanese Patent Laid-Open No. 5-268527

By the way, there are two types of defective pixels on the image sensor: a defective pixel that does not depend on temperature (hereinafter referred to as a defective pixel without temperature characteristics) and a defective pixel that depends on temperature. In the following description, a defective pixel that does not depend on temperature is referred to as a defective pixel without temperature characteristics, and a defective pixel that depends on temperature is referred to as a defective pixel with temperature characteristics.
However, conventionally, only one of the correction that considers the dependence due to the temperature or the correction that does not consider the dependence due to the temperature is basically performed.
A defective pixel having no temperature characteristic and a defective pixel having a temperature characteristic have different defect positions. That is, in the case of a defective pixel having no temperature characteristic, it is a defect of the light receiving element itself that constitutes the pixel. Part)).

In the conventional defective pixel correction, the presence / absence of such a temperature characteristic is not taken into consideration, and the correction is performed with uniform characteristics as having temperature characteristics or without temperature characteristics.
Therefore, for example, when a correction suitable for a defective pixel having a temperature characteristic is performed on a defective pixel having no temperature characteristic, there is a possibility that an erroneous correction state occurs. Similarly, when a correction suitable for a defective pixel having no temperature characteristic is performed on a defective pixel having a temperature characteristic, an erroneous correction state may occur.
As described above, if the correction state is not correct, the pixel level is not correct due to the defect correction process, and the image quality may be deteriorated.

  The present invention has been made in view of such a point, and an object thereof is to correct a defective pixel correctly for any pixel.

The present invention detects and corrects noise caused by defective pixels contained in a video signal output from an image sensor.
The processing is performed by discriminating between defective pixels with temperature characteristics whose level varies with temperature and defective pixels without temperature characteristics that do not depend on temperature, and storing the position information of each defective pixel. To do. When the video signal output from the image sensor is a signal of a stored defective pixel having temperature characteristics, a gain correction signal set corresponding to the temperature in the vicinity of the image sensor is subtracted from the video signal. Further, when the video signal output from the image sensor is a stored defective pixel signal without temperature characteristics, a correction signal having a certain gain is subtracted from the video signal.

  By doing in this way, the defective pixel with a temperature characteristic and the defective pixel without a temperature characteristic are correct | amended with the appropriate correction signal suitable for each defect state.

  According to the present invention, a defective pixel with temperature characteristics and a defective pixel without temperature characteristics are distinguished and stored, and correction suitable for each defect state is performed. Even when the defective pixels are mixedly present on the same image sensor, it is possible to output a video signal in which defect correction is appropriately performed.

Hereinafter, an example of an embodiment of the present invention will be described with reference to FIGS.
FIG. 1 is a diagram illustrating an overall configuration of an imaging apparatus 100 to which the example of the present embodiment is applied. The imaging device 100 is a three-plate type color imaging device provided with an image sensor for each primary color. The imaging 100 includes image sensors 101R, 101G, and 101B. The image sensors 101R, 101G, and 101B are image sensors that capture red, green, and blue images, respectively. In this embodiment, a CMOS image sensor is used as the image sensor. A temperature sensor 190 is arranged in the vicinity of the place where the image sensors 101R, 101G, and 101B are arranged, and the temperature of the place where the image sensors 101R, 101G, and 101B are arranged is measured.
Video signals (imaging signals) that are imaged and output by the image sensors 101R, 101G, and 101B are amplified to appropriate levels by the video amplifiers 102R, 102G, and 102B, and then each color is converted by the analog / digital converters 103R, 103G, and 103B. To digital data R, G, B. The converted data is subjected to video signal processing by the video processing unit 109.

  The video processing unit 109 includes a correction circuit 104 that corrects a video signal, a gain adjustment circuit 105 that performs gain adjustment such as white balance adjustment, a luminance adjustment circuit 106 that performs luminance adjustment, and a gamma correction circuit that performs gamma correction. 107 and an output signal generation circuit 108 for generating an output format video signal. Examples of correction processing in the correction circuit 104 include imager defective pixel correction processing. Imager defective pixel correction processing in the correction circuit 104 will be described later. The output signal generation circuit 108 outputs, for example, luminance data Y and color difference data Cr and Cb.

The operation of the imaging apparatus 100 will be briefly described. Image light from a subject obtained by an optical system (not shown) is incident on the imaging surfaces of different image sensors 101R, 101G, and 101B for each primary color. Then, red, green, and blue primary color images are formed. The primary color image is read out as a pixel signal by the light receiving elements that constitute the pixels arranged in a matrix on the imaging surface of the image sensor, and the pixel signal obtained by each sensor is read out in order, whereby a video signal in units of one frame. Is output.
The video signals output from the image sensors 101R, 101G, and 101B are converted into digital data by the analog / digital converters 103R, 103G, and 103B, and then the above-described video processing is performed in each circuit in the video processing unit 109.

Next, the circuit configuration of the pixel portion of the CMOS image sensor will be described with reference to FIG. FIG. 2 shows a configuration of one pixel unit. The pixel unit includes a photodiode (PD) 11 that is a light receiving element disposed on the imaging surface, and corresponds to the amount of light received by the photodiode 11. The obtained signal (charge) is read out by controlling each of the transistors 12-15.
A signal obtained by the photodiode 11 is connected to the readout transistor 12, the amplification transistor 14, and the row selection transistor 15 in this order. The row selection transistor 15 is arranged at the intersection of a line to which a row selection signal is supplied and a line to which a column selection signal is supplied.
Further, a reset transistor 13 is connected between the read transistor 12 and the amplification transistor 14.

The read transistor 12 is controlled by a read signal. The reset transistor 13 is controlled by a reset signal. The amplification transistor 14 performs an amplification operation by supplying the power supply VDD. The row selection transistor 15 is controlled by a row selection signal, and the electric charge obtained by the photodiode 11 is read from the column signal line as a voltage signal.
The electric charge obtained by light reception by the photodiode 11 is accumulated in a floating diffusion portion (hereinafter referred to as an FD portion) configured in the vicinity of the base portion of the amplification transistor 14.

  Here, with reference to FIG. 5, a processing state in which a signal is read from each pixel during normal imaging will be described. FIG. 5 shows a process of reading a signal accumulated in one pixel, and FIG. 5A shows a read signal for controlling the read transistor 12. During the period when the pulses P1 and P2 rising to the high level “H” are obtained as the read signal, the charge received by the photodiode 11 and accumulated in the FD portion is read. The signal read by the first pulse P1 is a P-phase signal, and the signal read by the next pulse P2 is a D-phase signal.

  FIG. 5B shows a row selection signal. When this row selection signal is at a high level “H”, it indicates that reading from the pixel in the corresponding row is selected. FIG. 5C shows a reset signal, and a reset pulse rises at the beginning and end of the light receiving period, and reset processing is performed.

  FIG. 5D shows the potential of the FD portion. When the accumulation period starts in conjunction with the rising edge of the reset signal, a signal V11 having a potential corresponding to the amount of received light at that time is obtained. In the example of FIG. 5, the state at the time of imaging in a state where the image sensor is shielded from light with the iris of the imaging apparatus 100 closed is shown. When the light is shielded, the potential of the signal V11 is constant and does not change unless the FD portion of each pixel is defective. On the other hand, the signal V12 in the case where the FD portion has a defect causes a leak current even when it is shielded from light, and the output potential decreases with time.

  The period after the pulse P1 of the read signal rises becomes the D phase. Even in this D phase period, the potential of the signal V11 is constant and does not change as long as there is no defect in the D portion. On the other hand, the signal V12 in the case where the FD portion has a defect causes a leak current even when it is shielded from light, and the output potential decreases with time.

  Then, as shown in FIG. 5E, a signal corresponding to the potential of the FD section is read as a column signal, and the signal is sampled and output. During normal imaging, an image sensor output video signal (imaging signal) is generated by the output circuit of the image sensor using the signal read in the P phase and the signal read in the D phase.

Next, FIG. 3 shows a configuration for detecting a pixel defect on the image sensor based on a signal read from the image sensor of this example. The configuration illustrated in FIG. 3 includes, for example, the correction circuit 104 of the imaging device 100 illustrated in FIG. 1 and its peripheral circuits.
Referring to the configuration of FIG. 3, a video signal obtained by imaging with an image sensor is input from the input terminal 21 and sampled by the sampling unit 22. The sampling unit 22 is provided on the output circuit (not shown) side of the image sensor. The sampling unit 22 samples at a timing at which a P-phase signal and a D-phase signal are obtained.

The signal sampled by the sampling unit 22 is directly supplied to the memory control unit 26 and also supplied to the comparator 24 via the high-pass filter (HPF) 23 that passes the high-frequency component of the signal, and is supplied to the threshold input terminal 25. Compared to the resulting threshold. The result of the comparison is supplied to the memory control unit 26.
The memory control unit 26 functions as a control unit that controls the defective pixel detection operation, and stores the detected defective pixel information in the memory 30.
In this case, the memory controller 26 is supplied with a count value obtained by counting the synchronization signal of the video signal obtained at the synchronization signal input terminal 27 by the address counter 28. By supplying the count value, the memory control unit 26 determines which pixel the signal supplied from the sampling unit 22 and the comparator 24 is.
The synchronization signal obtained at the synchronization signal input terminal 27 is also supplied to the readout signal control unit 29, and the readout signal of each pixel shown in FIG. 2 is generated in synchronization with the synchronization signal.
Note that the defect detection configuration shown in FIG. 3 is an example, and the same detection may be performed with other configurations. For example, the high-pass filter 23 may not be provided. Or it is good also as a structure which does not judge the threshold value in the comparator 24 but performs defect determination directly from judgment of a signal level.

  FIG. 4 is a flowchart showing an example of processing for detecting defective pixels using the detection circuit shown in FIG. FIG. 4 is an example of defective pixel detection processing without temperature characteristics. The defective pixel detection process without temperature characteristics and the defective pixel detection process with temperature characteristics are performed as separate processes. Description of the defective pixel detection process with temperature characteristics is omitted here.

An example of defective pixel detection processing without temperature characteristics will be described with reference to FIG. 4. First, the iris of the imaging apparatus 100 is closed and the image sensor is shielded from light. After that, reading from the PD section in the P phase is stopped (step S11). The reading stop from the PD unit in the P phase is executed by stopping the supply of the pulse P1 shown in FIG.
In this way, the imaging operation is executed, and the potential of the FD part is read in the P phase, and further, the potential of the FD part is read in the D phase (step S12). Then, the P-phase potential and the D-phase potential are detected (step S13), the difference between both potentials is compared with a threshold value, and the memory control unit 26 determines whether or not the value exceeds the threshold value ( Step S14).

  If the threshold value is exceeded, it is determined that the corresponding pixel signal is a defective pixel signal without temperature characteristics (step S15). Based on this determination, the address of the corresponding pixel, a flag indicating no temperature characteristic, and defect level information are stored in the memory 30 (step S16). The level of the defect is determined from, for example, the potential difference between the read signals. In step S15, determining that the pixel is a defective pixel having no temperature characteristic corresponds to determining that the FD portion of the corresponding image is defective.

  Then, it is determined whether there is another pixel that has not been determined to be defective (step S17), and the processing from step S11 is repeated until the determination of all the pixels is completed. The defective pixel determination process ends.

FIG. 6 shows an example of pixel driving at the time of detecting a defective pixel having no temperature characteristic described in the flowchart of FIG.
The signals shown in FIGS. 6A to 6E correspond to the signals shown in FIGS. 5A to 5E. As shown in FIG. 6A, when a defect having no temperature characteristic is determined, the supply of the pulse P1 shown in FIG. 5A is stopped, the process in the P phase is not performed, and the period of the D phase is entered. However, charges are accumulated in the FD portion as they are. By accumulating in this way, when there is no defect in the FD portion, a light shielding process is performed. Therefore, as shown in FIG. 6D, the potential V31 of the FD portion is a constant potential corresponding to the black level. Does not change. On the other hand, when the FD portion has a defect, the potential becomes a potential V32 that gradually decreases with time due to the influence of the leakage current.
As shown in FIG. 6E, the potential V41 of the FD portion read as a column signal when there is no defect remains a constant potential corresponding to the black level, and the FD portion has a defect. The potential V42 in the case where there is a potential is a potential that gradually decreases with the passage of time. Although not shown in FIG. 6, when the photodiode 11 has a defect, an abnormal constant level signal is detected in the entire reading period according to the defect state.

  In this way, information on defective pixels is stored in the memory 30 by distinguishing between information on pixels with temperature characteristics and information on pixels without temperature characteristics. The stored information is held in the memory 30 until update processing is performed.

Here, the detection processing state of the defective pixel in this example will be described using equations.
First, each condition is explained.
Tp, the accumulation time from reset to P phase
Td, the accumulation time from the P phase readout to the D phase
The voltage of the column signal output in the P phase is Vp,
The voltage of the column signal output in the D phase is Vc,
The voltage that increases per unit time due to a defect in the FD portion is Vfd.
And

The conditions at the time of normal use and at the time of normal defect detection (that is, at the time of defect detection with temperature characteristics) are expressed by the following equations.
P-phase output voltage Vp = reset voltage + (Vfd × tp)
D-phase output voltage Vd = reset voltage + (Vfd × tp) + PD part voltage + (Vfd × td)
Output voltage difference between both phases Vd−Vp = Δ1 = PD section voltage + (Vfd × td)

On the other hand, the condition at the time of defect detection without temperature characteristics is expressed by the following equation.
P-phase output voltage Vp = reset voltage + (Vfd x tp)
D phase output voltage Vp = reset voltage + (Vfd × tp) + (Vfd × td)
Output voltage difference between both phases Vd−Vp = Δ2 = Vfd × td

This Δ2 is detected by the current defect detection process as the defect level of the non-thermal defect, and this is subtracted from the video signal Δ1 during correction. As a result, only the “PD section voltage”, which is the true video signal, is obtained.
By detecting in this way, a “PD portion voltage (PD portion defect level)” having temperature characteristics and a “Vfd × tp” level having no temperature characteristics are obtained. Separation from defects having no characteristics becomes possible, and correction accuracy is improved.

Next, a process for correcting the video signal obtained by imaging at the time of imaging based on the information on the defective pixel stored in the memory 30 in this way will be described.
FIG. 7 shows an example of the configuration of a defective pixel signal correction unit according to this embodiment. This correction unit is configured, for example, in the correction circuit 104 shown in FIG.
The memory controller 26 is supplied with a count value obtained by counting the synchronization signal obtained at the terminal 27 by the address counter 28, and it is determined from which pixel the video signal read out is supplied.

In this determination, when it is determined that the pixel is stored as a defective pixel in the memory 30, the storage information about the pixel is read to determine the presence / absence of a temperature characteristic, the defect level, and the like. When it is determined that the pixel is a defective pixel, a signal corresponding to the defect level is supplied from the memory control unit 26 to the multiplier 34, and the multiplier 34 multiplies the master gain obtained at the terminal 35.
The signal multiplied by the multiplier 34 is supplied to one input terminal of the selector 36 and also supplied to the other input terminal of the selector 36 via the multiplier 37. The multiplier 37 is a multiplier that multiplies a gain signal that changes according to the temperature obtained at the terminal 38. The gain signal that changes according to the temperature obtained at the terminal 38 is a signal that changes according to the temperature detected by the temperature sensor 190 shown in FIG.

Selection by the selector 36 is controlled by the memory control unit 26. That is, when it is determined that the current pixel signal is a defective pixel having no temperature characteristic, one input terminal is selected and the output of the multiplier 34 is directly selected. When it is determined that the current pixel signal is a defective pixel having temperature characteristics, the other input terminal is selected and the output of the multiplier 37 is directly selected.
The signal selected in this way by the selector 36 is supplied to the subtracter 32 and subtracted from the video signal obtained at the input terminal 31 to correct the signal in the corresponding pixel section, and the corrected video. The signal is supplied from the output terminal 33 to the subsequent circuit.

The flowchart of FIG. 8 shows an example of correction processing executed by the circuit of FIG. 7 based on control by the memory control unit 26.
In the following, referring to the flowchart of FIG. 8, the memory controller 26 determines the count value in the address counter 27 (step S21), and determines whether the address position is a defective pixel (step S22). If this is not the case, the process returns to step S21 to determine the next address.
If it is determined in step S22 that the pixel is defective, it is determined from the flag stored in the memory 30 whether the defective pixel has a temperature characteristic or does not have a temperature characteristic (step S23). If it is determined that the defect has a temperature characteristic, a correction signal in which a gain corresponding to the current temperature is set by multiplication by the multiplier 37 is selected by the selector 36 (step S24). If it is determined that the defect has no temperature characteristic, the selector 36 selects a correction signal in which a constant gain is set by multiplication by the multiplier 34 (step S25). Then, the selected signal is supplied to the subtracter 32 for correction (step S26), and the process returns to step S21.

  By performing the correction process in this way, the defective pixel having the temperature characteristic is correctly corrected with the correction signal having the temperature characteristic, and the defective pixel having no temperature characteristic is corrected with the correction signal that does not change with temperature. Correction processing is performed correctly, and correction processing is performed correctly for any defective pixel. Therefore, the image signal corrected for defective pixels does not deteriorate in image quality due to the correction, and good imaging can be performed.

In the correction processing configuration shown in FIG. 7, the selector 36 selects two types of a correction signal having temperature characteristics and a correction signal having no temperature characteristics, and corrects using either of the signals. However, when correcting a pixel having temperature characteristics, correction may be performed using a correction signal obtained by mixing a correction signal having temperature characteristics and a correction signal having no temperature characteristics.
That is, for example, as shown in FIG. 9, a correction signal obtained by multiplying the gain value corresponding to the temperature by the multiplier 37 is further supplied to the adder 39 and added to the correction signal output from the multiplier 34. At the time of addition by the adder 39, the addition ratio of both signals is set to a preset ratio. Then, the correction signal added by the adder 39 is supplied to the other input terminal of the selector 36.
In FIG. 9, the other parts are configured in the same manner as in FIG.

  Even when configured as shown in FIG. 9, the characteristics of the correction signal can be changed according to the presence or absence of the temperature characteristics, and appropriate correction according to the respective defect states is possible.

It is a block diagram which shows the structural example of the imaging device by the example of one embodiment of this invention. It is a circuit diagram which shows the circuit structural example of the pixel part by the example of one embodiment of this invention. It is a block diagram which shows the example of a defective pixel detection structure by the example of one embodiment of this invention. It is a flowchart which shows the example of a defective pixel detection process by one embodiment of this invention. It is a timing chart which shows the example of a process at the time of the imaging by one embodiment of the present invention. It is a timing chart which shows the example of a process at the time of the defect detection by one embodiment of this invention. It is a block diagram which shows the example of the correction | amendment part by one embodiment of this invention. It is a flowchart which shows the example of a correction process by one embodiment of this invention. It is a flowchart which shows the example of a correction process by other embodiment of this invention.

Explanation of symbols

  DESCRIPTION OF SYMBOLS 11 ... Photodiode (PD), 12 ... Read-out transistor, 13 ... Reset transistor, 14 ... Amplification transistor, 15 ... Row selection transistor, 21 ... Input terminal, 22 ... Sampling part, 23 ... High pass filter, 24 ... Comparator, 25 Threshold input terminal, 26 Memory control unit, 27 Sync signal input terminal, 28 Address counter, 29 Read signal control unit, 30 Memory, 31 Video signal input terminal, 32 Subtractor, 33 Corrected Signal output terminal, 34 ... multiplier, 35 ... master gain input terminal, 36 ... selector, 37 ... multiplier, 38 ... temperature characteristic gain input terminal, 39 ... adder

Claims (7)

In a noise correction circuit that detects and corrects noise of defective pixels included in a video signal output by an image sensor,
A storage unit that distinguishes between a defective pixel having a temperature characteristic whose level varies with temperature and a defective pixel having no temperature characteristic that does not depend on temperature as the defective pixel of the image sensor, and stores position information of each defective pixel; ,
When the video signal output from the image sensor is a signal of a defective pixel with temperature characteristics stored in the storage unit, a gain correction signal set corresponding to the temperature in the vicinity of the image sensor is used as the video signal. And a correction unit that subtracts a correction signal having a constant gain from the video signal when the signal is a defective pixel signal without temperature characteristics stored in the storage unit and subtracted from the video signal. circuit. The noise correction circuit according to claim 1,
The storage unit
Along with the position information of the defective pixel, information on the defect level of the defective pixel is stored,
In the case of a defective pixel having a temperature characteristic, the correction unit sets a correction signal in which a gain corresponding to a temperature in the vicinity of the image sensor is set to a defect level signal stored in the storage unit. In the case of a defective pixel having no defect, the noise correction circuit is characterized in that a correction signal in which the constant gain is set to a signal of a defect level stored in the storage unit is used. In the noise correction circuit according to claim 2,
A noise correction circuit, wherein the detected pixel level is determined as a defect level and stored in the storage unit. In an imaging device including an image sensor unit,
A storage unit that distinguishes between a defective pixel having a temperature characteristic whose level varies with temperature and a defective pixel having no temperature characteristic that does not depend on temperature as the defective pixel of the image sensor, and stores position information of each defective pixel; ,
When the video signal output from the image sensor is a signal of a defective pixel with temperature characteristics stored in the storage unit, a gain correction signal set corresponding to the temperature in the vicinity of the image sensor is used as the video signal. And a correction unit that subtracts a correction signal having a constant gain from the video signal when the signal is a signal of a defective pixel having no temperature characteristic stored in the storage unit and subtracted from the video signal. . The imaging apparatus according to claim 4.
The storage unit
Along with the position information of the defective pixel, information on the defect level of the defective pixel is stored,
In the case of a defective pixel having a temperature characteristic, the correction unit sets a correction signal in which a gain corresponding to a temperature in the vicinity of the image sensor is set to a defect level signal stored in the storage unit. In the case of a defective pixel having no image, the image pickup apparatus is characterized in that a correction signal in which the constant gain is set to a signal of a defect level stored in the storage unit. In the imaging device according to claim 5,
An imaging apparatus, wherein the detected pixel level is determined as a defect level and is stored in the storage unit. In a noise correction method for detecting and correcting defective pixel noise included in a video signal output from an image sensor,
As a defective pixel of the image sensor, a defective pixel with a temperature characteristic whose level varies with temperature and a defective pixel without a temperature characteristic that does not depend on temperature are distinguished, and positional information of each defective pixel is stored.
When the video signal output from the image sensor is a signal of the stored defective pixel having temperature characteristics, a gain correction signal set corresponding to the temperature in the vicinity of the image sensor is subtracted from the video signal. A noise correction method comprising: subtracting a correction signal having a constant gain from the video signal when the signal is a stored defective pixel signal without temperature characteristics.

Images