JP2010011161A - Image capturing apparatus and image sensor temperature measuring method - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide an image capturing apparatus and an image sensor temperature measuring method which accurately obtain an internal temperature of an image sensor itself, prevent image quality from being reduced, and optimize image processing for image quality improvement. <P>SOLUTION: The image capturing apparatus 10 includes an image sensor 12 and an image processing section 15. The image processing section 15 includes: an area selection section 151 which selects a predetermined area for detecting a dark current output from an imaging plane of the image sensor; a dark current detection section 152 for detecting the dark current output from an output signal of the selected predetermined area; a singular pixel extraction section 153 for extracting a singular output pixel signal from the output signal of the selected predetermined area; and a processing section 154 for calculating a variance value by canceling the singular pixel signal component extracted from the detected dark current, and for obtaining a temperature of the image sensor from the calculated variance value of the dark current. <P>COPYRIGHT: (C)2010,JPO&INPIT

Description

  The present invention relates to an imaging apparatus and an imaging element temperature measuring method for generating an image by an imaging element such as a CCD or a CMOS sensor.

In response to the digitization of information, which has been rapidly developing in recent years, the response in the video field is also remarkable.
In particular, as symbolized by a digital camera, the imaging surface is changed to a film, and a CCD (Charge Coupled Device) or CMOS (Complementary Metal Oxide Semiconductor) sensor, which is a solid-state imaging device, is used in most cases.

  An image sensor that photoelectrically converts a subject image into an electrical signal generates an electrical signal based on the charge that was originally generated by receiving reflected light from the subject, but the charge is not actually received. Will be generated.

Such a charge is called dark current. The magnitude of the dark current depends not only on the imaging accumulation time but also on the temperature of the imaging device itself.
In particular, the temperature is said to increase by a factor of 2 from 8 to 10 ° C. If the image is not processed by any method, the image quality under high temperature will increase the dark current, resulting in a decrease in the S / N ratio. Or a decrease in dynamic range.

  Therefore, it is necessary for the imaging apparatus to accurately grasp the operating environment temperature of the imaging element in order to obtain a desired captured image, excluding the influence of dark current.

  As means for measuring the temperature of the image sensor, a technique for measuring the ambient temperature of the image sensor by providing a temperature sensor capable of detecting the temperature around the image sensor is known.

  In Patent Document 1, the number of pixel defects of the image sensor is obtained, and the internal temperature of the image sensor itself is calculated from a relationship table between the number of defective pixels obtained in advance and the temperature provided in the image pickup apparatus. A method is disclosed.

Patent Document 2 discloses a method of calculating a standard temperature or an average value of dark current, comparing the relationship between the dark current value and the temperature of the image sensor using a temperature table, and calculating the internal temperature of the image sensor itself. It is disclosed.
JP 2006-121292 A JP 2007-201735 A

  However, since the external temperature measuring means cannot measure the temperature of the image sensor itself, it cannot accurately measure the temperature of the image sensor that is necessary.

On the other hand, in Patent Document 1, it is considered that the internal temperature of the image pickup device itself is obtained from the number of pixel defects using a table provided in the image pickup apparatus.
However, in this case, it is impossible to consider the sensitivity variation among the individual pixels of the image sensor and the number of pixel defects that are newly generated due to cosmic rays. .
Therefore, the technique of Patent Document 1 has a disadvantage that the internal temperature of the image sensor itself required for image processing for preventing image quality deterioration of the image capturing apparatus and improving the image quality cannot be obtained accurately.

Further, by estimating and calculating the internal temperature of the image sensor based on the standard deviation or average value of the dark current from the image pickup pixel in Patent Document 2, the influence of a specific output pixel signal such as fixed pattern noise or incomplete pixel defects is obtained. Accordingly, an appropriate dark current value cannot be calculated.
As a result, in the technique of Patent Document 2, the internal temperature of the image pickup device itself is not correlated with the internal temperature of the image pickup device itself, which prevents image quality deterioration of the image pickup apparatus and is required for image processing to improve image quality. There is a disadvantage that it cannot be accurately determined.

  The present invention can accurately determine the internal temperature of an image sensor itself, can prevent deterioration in image quality, and can optimize image processing for improving image quality, and an image sensor temperature measurement method Is to provide.

  An imaging device according to a first aspect of the present invention includes an imaging device, a selection unit that selects a predetermined region for detecting a dark current output from a pixel region of the imaging device, and an output signal of the selected predetermined region A detection unit for detecting a dark current output; an extraction unit for extracting a specific output pixel signal from the output signal of the selected predetermined region; and removing the extracted specific pixel signal from the detected dark current. A processing unit that calculates a dispersion value and obtains the temperature of the image sensor from the calculated dispersion value of the dark current.

  Preferably, the selected predetermined area includes an optical light shielding portion of the image sensor.

  Preferably, the selected predetermined area includes an area in which an image output level is equal to or lower than a predetermined value in an effective imaging area of the imaging device with an accumulation time equal to or longer than a predetermined time.

  Preferably, the processing unit calculates the variance value of the dark current for each successive frame, and performs a conversion average by changing the number of frames used according to the relative speed between the image sensor and the subject.

  Preferably, a temperature measurement function unit that controls image processing according to a temperature detection result of the processing unit is provided.

  An image sensor temperature measuring method according to a second aspect of the present invention includes a step of selecting a predetermined area for detecting a dark current output from a pixel area of the image sensor, and a dark current output from an output signal of the selected predetermined area. Detecting a singular output pixel signal from the output signal of the selected predetermined region, and calculating a variance value by removing the extracted singular pixel signal from the detected dark current And obtaining the temperature of the image sensor from the calculated dispersion value of the dark current.

  According to the present invention, the internal temperature of the image sensor itself can be accurately obtained, deterioration of image quality can be prevented, and image processing for improving image quality can be optimized.

  Hereinafter, embodiments of the present invention will be described with reference to the drawings.

  FIG. 1 is a block diagram illustrating a configuration example of an imaging apparatus according to an embodiment of the present invention.

  As illustrated in FIG. 1, the imaging apparatus 10 according to the present embodiment includes an optical system 11, an imaging element 12, an analog front end unit (AFE) 13, an analog / digital conversion unit (ADC) 14, an image processing unit 15, and an image. A display unit 16 is included.

  The optical system 11 forms an image of the photographed subject OBJ on the light receiving surface of the image sensor 12.

The image sensor 12 is formed by a CCD or CMOS sensor.
FIG. 2 is a diagram illustrating an example of the image array unit of the image sensor 12.
As shown in FIG. 2, the imaging device 12 includes a pixel array unit 120 in which pixels are arranged in an array.
The pixel array unit 120 includes a central effective pixel region 121 and a peripheral light-shielded pixel region 122.
The image sensor 12 generally has an effective pixel region 121 and a light-shielded pixel region 122. The effective pixel area 121 is an area where an image of a subject is formed, and the light-shielded pixel area 122 is an area that is optically shielded to serve as a reference for pixel output.

The image sensor 12 photoelectrically converts the subject image captured by the optical system 11 and outputs it to the AFE 13 as an image signal.
Specifically, the image sensor 12 obtains an image signal by controlling a predetermined imaging accumulation and reading time by an image sensor driving timing control circuit (not shown).

  The AFE 13 receives the image signal from the image sensor 12, performs a reset noise removal process and a predetermined amplification process on the image sensor, and outputs the processed image signal to the ADC 14.

  The ADC 14 converts the analog image signal from the AFE 13 into a digital signal and outputs the digital signal to the image processing unit 15.

  The image processing unit 15 receives the digital image signal from the ADC 14, performs predetermined image processing, specifically color interpolation, white balance, YCbCr conversion processing, compression, filing, and the like, and outputs an image to the image display unit 16. Display or record to a memory (not shown).

The image processing unit 15 includes a temperature measurement function unit (temperature measurement function unit) of the image sensor 12.
The temperature measuring function unit of the image processing unit 15 selects a region for detecting a dark current from the image signal, detects a dark current value from the image signal in the selected region, and detects a specific pixel signal for the detected dark current value. The dispersion value is obtained by removing the portion, and the temperature of the image sensor 12 is obtained from the calculated dispersion value of the dark current.

  The temperature measuring function unit compares the environmental temperature of the image sensor obtained by correlation with a predetermined temperature table or the like with a predetermined reference value, and the image sensor 12 or the AFE 13 or ADC 14 or the post-imaging that is the configuration of the imaging pre-processing unit. Control processing for black level setting, black level offset correction, and noise reduction is performed on the image processing unit 15 as a processing unit in order to prevent image degradation due to dark current and improve image quality.

The image processing unit 15 can accurately measure the temperature of the image sensor 12 by the above-described processing of the temperature measuring function unit, and the processing system of the imaging apparatus can be stopped by the temperature measured as a result. It is also possible to restore the system.
For example, if the temperature of the image sensor is measured correctly and the temperature becomes too high to ensure image quality, stop the system in advance, and if the temperature reaches a level that can ensure image quality again, start the system as soon as possible. Therefore, a lean image processing system can be constructed.

  FIG. 3 is a diagram illustrating a configuration example of the temperature measuring function unit of the image processing unit 15 of the present embodiment.

  As shown in FIG. 3, the temperature measurement function unit 150 includes a region selection unit 151, a dark current detection unit 152, a singular pixel extraction unit 153, and a processing unit 154.

  The region selection unit 151 selects a predetermined region for detecting dark current output from the imaging surface of the image sensor 12 and supplies an image signal corresponding to the selected pixel region to the dark current detection unit 152 and the specific pixel extraction unit 153. To do.

FIGS. 4A and 4B are diagrams illustrating specific examples of the selection region by the region selection unit.
As illustrated in FIG. 4A, the region selection unit 151 selects a predetermined region of the light-shielded pixel region 122 that is an optical light-shielding portion of the image sensor 12 as the selection region 1221.
Alternatively, as illustrated in FIG. 4B, the region selection unit 151 ensures that a certain level of image output level is secured in the effective pixel region 121 of the image sensor 12 with an accumulation time equal to or longer than a predetermined time, and a predetermined level. An area that is equal to or smaller than the value is selected as the selection area 1211.

As described above, the image sensor 12 is a pixel that is optically shielded among the two-dimensionally arranged pixels, and an effective pixel that receives and photoelectrically converts an image captured via the optical system. And a region 121.
When temperature detection is performed in the effective pixel region 122, temperature detection can be performed in the normal processing algorithm because there is no processing for extracting the light shielding region.
However, depending on the subject, it is conceivable that a correct dispersion value cannot be detected when detecting in a region such as a high-luminance subject.
Therefore, a stable dispersion value is required to determine a region for selecting temperature detection. If the level of the image signal is high, there are pixels that saturate, so there is a possibility that the dispersion value for temperature detection cannot be obtained. In this case, it is preferable to select the light-shielded pixel region 122.

  Further, in order to obtain a stable dispersion value, an exposure time longer than a certain value is required. If the exposure time is short, the dispersion value becomes extremely small, and temperature detection may not be performed correctly. In this case, it is preferable to select the effective pixel region 121.

  The dark current detection unit 152 detects the dark current output from the output signal of the predetermined region selected by the region selection unit 151, and outputs the result to the processing unit 154.

  The singular pixel extraction unit 153 extracts a singular output pixel signal from the output signal of the predetermined region selected by the region selection unit 151, and outputs the extracted signal to the processing unit 152.

  The processing unit 154 removes the singular pixel signal extracted by the singular pixel extraction unit 153 from the dark current detected by the dark current detection unit 152, calculates a variance value, and captures an image from the calculated variance value of the dark current. The temperature of the element 12 is obtained.

  Hereinafter, the dark current output, the singular pixel output, the temperature acquisition process from the dispersion value, and the like will be described more specifically.

  FIG. 5 is a flowchart of dark current dispersion value calculation processing in the basic image sensor temperature measuring method according to the embodiment of the present invention.

<Step ST1>
In step ST <b> 1, the area selection unit 151 selects a dark current detection target area in the light-shielding partial area 122 or the effective imaging area 121 at a light-shielding pixel readout timing of an image sensor drive control timing (not shown).

<Step ST2>
Next, in step ST2, the dark current detector 152 detects the dark current in the selected area. Since the environmental temperature change of the image sensor 12 is gentler than the imaging frame rate, the singular pixel extraction unit 153 performs singular pixel signals such as fixed pattern noise and imperfect pixel defects by filtering or, for example, several frame integration techniques. Recognizes the output pixel timing.

<Step ST3>
In step ST3, the processing unit 154 performs, for example, gating only the pixel output signal recognized as the singular pixel signal in step ST2 from the output signal from the imaging element 12 that is sequentially output after the singular pixel can be recognized. Remove by the method of.

<Step ST4>
In step ST4, the processing unit 154 calculates the dark current dispersion value of the predetermined light-shielding partial region pixel based on the dark current noise pixel signal from which the singular pixel is removed.

  FIG. 6 is a diagram illustrating characteristic data of dark current output with respect to the temperature of the image sensor. FIG. 6 shows dark current values from room temperature to high temperature.

  As shown in FIG. 6, the dark current output tends to increase as the temperature increases.

  7A and 7B are diagrams for explaining the singular pixel output.

In order to obtain an accurate dispersion value of the dark current, it is necessary to exclude a singular pixel output from a pixel that outputs an abnormal value due to some defect.
As shown in FIG. 7A, when the temperature of the image sensor 12 is low, the level of the singular pixel output is small as well as the dark current output.
When the temperature of the image sensor 12 is high, as shown in FIG. 7B, the level of the singular pixel output is increased together with the dark current output, and there is a high possibility that the influence cannot be ignored.
Therefore, in the present embodiment, a singular output threshold level VTH that does not exceed the output of normal pixels even when the temperature is high is provided, and pixel outputs that exceed this level VTH are excluded as singular pixel outputs.

  FIG. 8 is a diagram conceptually showing that the dispersion of the warm current varies depending on the temperature of the image sensor.

FIG. 8 shows a histogram in which the horizontal axis indicates the output value of each pixel and the vertical axis indicates the frequency (number of pixels), and shows the distribution state of the pixel output value at normal temperature and at high temperature.
In FIG. 8, for convenience of explanation, the graph at normal temperature and the graph at high temperature are expressed by combining the position in the horizontal axis direction with the center value of the distribution.

When the temperature of the image sensor 12 is normal temperature, the distribution is a vertically long shape with high frequency near the center of the distribution and little variation.
Compared to this, when the temperature of the image sensor is high, the frequency is low near the center of the distribution, and the distribution has a wide shape with many variations.
That is, the dispersion is larger as the temperature of the image sensor 12 is higher, and the dispersion is smaller as it is lower.
Therefore, by using this relationship, it is possible to estimate the temperature of the image sensor from the value of the dispersion of the dark current output.
By using the dispersion value, it becomes less susceptible to the influence of an abnormal output as compared with the case of using an average value or the like, so that temperature detection with high accuracy and high resolution becomes possible.

  FIG. 9 is a diagram illustrating the relationship between the noise variance value and the temperature.

It has a certain width due to individual differences of elements. The width is shown by the upper and lower lines in FIG.
However, the noise variance value at each element can be expressed as an approximately constant mathematical expression.
In this case, it is possible to approximate with two formulas at a certain temperature.

[Equation 1]
y = B * X + C

[Equation 2]
y = A * x exp3.6

  For example, the temperature is calculated from the dispersion value using this approximate expression.

  Next, a method of performing temperature detection with higher accuracy using data of a plurality of frames obtained in time series will be described.

  FIG. 10 is a diagram illustrating an algorithm of a method for performing temperature detection with higher accuracy using data of a plurality of frames obtained in time series.

In this method, the dispersion value of the dark current is calculated for each successive frame, and the conversion average is performed by changing the number of frames used according to the relative speed between the image sensor and the subject.
Basically, the series processing of steps ST1 to ST4 in FIG. 5 is repeated in time series.
Then, an average process is performed by detecting a difference between variance values of adjacent series processes.

In order to improve the accuracy of temperature detection according to the present embodiment, not only the result of one time (one frame screen) but also the result of plural times (multiple frames) is used, for example, by performing addition averaging or the like It is also possible to take a method of improving the reliability of data.
In that case, since the degree of variation of the result also changes depending on the degree of change of the captured image with time, it is effective to change the number of results to be used (number of frames) according to the relative speed between the image sensor 12 and the subject OBJ. It becomes.

In the algorithm of FIG. 10, when the relative speed is fast, the image is updated quickly, in other words, the movement between the subject OBJ and the imaging device is fast, and thus the difference accumulation amount is increased.
For example, in the variance value calculation algorithm of FIG. 10, when the difference output in each frame is used, the difference output is averaged in a large number of frames.
In this case, by averaging and calculating the values 1, 2, and 3 obtained in steps ST5, ST6, ST7, it is possible to approach the desired temperature dispersion value, so that precise temperature detection is possible.

  On the other hand, when the relative speed is slow, the image update is slow, so even if the accumulation amount is small, it is close to the desired temperature dispersion value. It becomes possible to reduce the burden of processing.

When there are n pieces of data x ₁ , x ₂ ,..., x _n and / x (/ indicates a symbol instead of a bar) is an arithmetic average of the data, (/ x−x _i ) The average of ² is called the sample variance and is given by the following formula.

  In this embodiment, the variance value is obtained by obtaining this sample variance.

  According to the present embodiment, the image sensor 12 and the image processing unit 15 are included. Then, the image processing unit 15 selects an area selection unit 151 for selecting a predetermined area for detecting dark current output from the imaging surface of the image sensor, and a dark current for detecting dark current output from the output signal of the selected predetermined area. The detection unit 152, the singular pixel extraction unit 153 that extracts the singular output pixel signal from the output signal of the selected predetermined region, and the variance value is calculated by removing the singular pixel signal extracted from the detected dark current In addition, since the processing unit 154 that obtains the temperature of the image sensor from the calculated dispersion value of the dark current is provided, the following effects can be obtained.

  That is, the internal temperature of the image sensor itself can be obtained accurately. Therefore, an appropriate temperature noise removal process can be performed, the image quality of the image sensor can be prevented from being deteriorated at a high temperature, and the image processing for improving the image quality can be optimized.

The image processing unit 15 can accurately measure the temperature of the image sensor 12 by the above-described processing of the temperature measuring function unit, and the processing system of the imaging apparatus can be stopped by the temperature measured as a result. It is also possible to configure so that the system can be restored.
For example, if the temperature of the image sensor is measured correctly and the temperature becomes too high to ensure image quality, stop the system in advance, and if the temperature reaches a level that can ensure image quality again, start the system as soon as possible. Therefore, a lean image processing system can be constructed.

It is a block diagram which shows the structural example of the imaging device which concerns on embodiment of this invention. It is a figure which shows an example of the image array part of an image pick-up element. It is a figure which shows the structural example of the temperature measurement function part of the image processing part of this embodiment. It is a figure which shows the specific example of the selection area | region by an area | region selection part. It is a flowchart of the dispersion | distribution value calculation process of the dark current in the basic image pick-up element temperature measuring method which concerns on embodiment of this invention. It is a figure which shows the characteristic data of the dark current output with respect to the temperature of an image sensor. It is a figure for demonstrating a specific pixel output. It is a figure which shows notionally that dispersion | distribution of a warm current changes with the temperature of an image pick-up element. It is a figure which shows the relationship between a noise dispersion value and temperature. It is a figure which shows the algorithm of the method of performing a more accurate temperature detection using the data of the several flame | frame obtained in time series.

Explanation of symbols

  DESCRIPTION OF SYMBOLS 10 ... Imaging device, 11 ... Optical system, 12 ... Imaging element, 13 ... Analog front end part (AFE), 14 ... Analog-digital conversion part (ADC), 15 ... Image processing unit 150 ... Temperature measuring function unit 151 ... Region selection unit 152 ... Dark current detection unit 153 ... Specific pixel extraction unit 154 ... Processing unit 16 ... Image display section.

Claims (6)

An image sensor;
A selection unit for selecting a predetermined region for detecting a dark current output from a pixel region of the image sensor;
A detection unit for detecting a dark current output from the output signal of the selected predetermined region;
An extraction unit for extracting a singular output pixel signal from the output signal of the selected predetermined region;
An imaging apparatus comprising: a processing unit that calculates a dispersion value by removing the extracted singular pixel signal from the detected dark current, and obtains the temperature of the imaging element from the calculated dispersion value of the dark current. The imaging apparatus according to claim 1, wherein the selected predetermined area includes an optical light shielding portion of the imaging element. The imaging apparatus according to claim 1, wherein the selected predetermined area includes an area in which an image output level is equal to or lower than a predetermined value in an accumulation time equal to or longer than a predetermined time in an effective imaging area of the image sensor. The processor is
The variance value of the dark current is calculated for each successive frame, and the conversion average is performed by changing the number of frames used according to the relative speed between the image sensor and the subject. Imaging device. The imaging apparatus according to claim 1, further comprising a temperature measurement function unit that controls image processing according to a temperature detection result of the processing unit. Selecting a predetermined area for detecting dark current output from the image area of the image sensor;
Detecting a dark current output from an output signal of the selected predetermined region;
Extracting a singular output pixel signal from the output signal of the selected predetermined region;
Removing the extracted singular pixel signal from the detected dark current to calculate a variance value;
Obtaining a temperature of the image sensor from the calculated dispersion value of the dark current.

Images