JP2008034942A - Image processor - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To solve the problem of a conventional CMOS sensor, wherein white balance is unbalanced because the switching speed of a transistor of an XY address signal generating circuit changes with the rise in the temperature, a video signal output is delayed and a sampling position of a waveform is deviated, so that the output of the digital signal at A/D conversion is decreased. <P>SOLUTION: A white balance gain is corrected, according to the temperature difference between a sensor temperature at factory adjustment and the present temperature in an AWB mode and a preset WB mode. Furthermore, the WB is corrected, according to the temperature difference between a sensor temperature at WB setting and the present temperature. <P>COPYRIGHT: (C)2008,JPO&INPIT

Description

  The present invention relates to white balance control in an imaging apparatus that captures, records, and plays back still images and moving images, such as a digital video camera and a digital camera.

  As a white balance (WB) control method for image capture devices such as digital video cameras and digital cameras, the CCD signal output is separated into three components: R (red), G (green), and B (blue). Of these, WB control is generally performed by changing the gains of R and B as WB gains.

Then, an auto white balance (hereinafter referred to as AWB) mode in which the WB gain automatically follows the color temperature of the illumination light of the subject, and a fixed WB gain suitable for the color temperature of a specific illumination light such as halogen light or sunlight. There are a preset WB mode and a WB set mode in which the WB gain matches the color temperature of the illumination light.
In video cameras and digital video cameras, CCD sensors are often used for photoelectric conversion of optical images.

In recent years, high-definition photography and high-pixel still image photography have been demanded, but in the case of a CCD, charge transfer deterioration increases due to an increase in the horizontal transfer frequency, which is unsuitable for high-speed transfer.
In such applications, for the reasons described above, XY address type CMOS sensors (hereinafter referred to as CMOS sensors) capable of high-speed reading are beginning to be used.

In addition, when the direct current component of the luminance signal or the color difference signal fluctuates due to a temperature change or the like, the one that compensates for the fluctuation due to the temperature change by subtracting the black level from the color difference signal and then performing WB. Yes (Patent Document 1).
JP 10-065958

  In order to drive the image sensor, a reference clock signal is supplied from the outside by a timing generator (hereinafter referred to as TG), but this clock is very high when shooting a high-pixel moving image such as a high-definition moving image. Become.

  The output waveform of the video signal of the image sensor at this time is shown in FIG. Originally, it should be output as shown by the broken line in the figure, but since the output is high speed, the output waveform is narrow and sufficient time for rising cannot be taken, as shown by the solid line in the figure, The output signal waveform is distorted.

  Further, driving the image sensor at high speed also increases the heat generation of the circuit.

  In a CMOS sensor, unlike a CCD sensor, a logic circuit can be easily incorporated by a CMOS process. Therefore, a circuit for generating the XY address signal described above is often incorporated on a CMOS sensor.

  At this time, a phenomenon occurs in which the switching speed of the transistors of the XY address signal generation circuit changes with an increase in temperature, and the video signal output is delayed. In addition, since the number of transistors in the circuit is large, the delay amount is integrated and becomes large.

The waveform of the output signal at this time is shown in FIG.
When the temperature is low, the switching speed of the transistor is fast, so the waveform of the broken line is a waveform with a phase shift as shown by the solid line because of the delay due to the temperature rise.

  In order to perform various digital image processing, A / D conversion is performed to convert the output signal into a digital signal. For the above-described reason, the phase of the output waveform shifts due to the temperature rise, and the sampling position of the waveform shifts. As shown in FIG. 8, the output of the digital signal becomes small.

At this time, in the case of an image sensor with a color filter, there is a problem that WB shifts.
Taking an image sensor with a primary color filter as an example, there are R pixels that are sensitive to red, G pixels that are sensitive to green, and B pixels that are sensitive to blue, but the sensitivity ratio to natural light is Generally, R and B are lower than G. Here, as an example, the sensitivity ratio of each pixel is R: G: B = 1: 2: 1.

  At this time, in order to take WB with respect to natural light, the gains of R and B must be doubled with G as a reference.

  If the temperature rises in a state where WB is taken at a certain temperature and the output of the image sensor becomes small for the above reason, R and B apply a gain twice that of G. , R and B decrease twice as much as G, and a phenomenon occurs in which WB shifts.

  Therefore, in a mode that gives a fixed WB gain, such as the WB set mode and the preset WB mode, there is a problem that the WB shifts as the temperature rises.

  Also, in the AWB mode that automatically follows WB, some amount of deviation may be absorbed, but depending on the control, the control amount of the WB gain is limited along the black body radiation curve to improve performance. In some cases, when such control is performed, a WB shift due to temperature causes a deterioration in AWB performance.

  Since this phenomenon also occurs in the waveform of OB, it is the same even after subtracting OB, and cannot be improved by the method in Patent Document 1.

  Although it is conceivable to absorb the delay by changing the sample hold position, since the delay amount due to temperature is smaller than the switch speed of the transistor, the sample hold position cannot be accurately controlled.

An image processing apparatus of the present invention includes an image sensor that performs photoelectric conversion, a temperature sensor that detects a temperature of the image sensor,
A white balance circuit and a white balance for white balance by giving a gain to a means for storing the temperature state of the image sensor, and a video signal output from the image sensor separated into R, G and B signals. An automatic white balance gain setting unit that automatically follows, stores the temperature at the time of factory adjustment, and corrects the white balance gain according to the difference from the current temperature.

  Another feature of the present invention is that an image sensor that performs photoelectric conversion, a temperature sensor that detects the temperature of the image sensor, a unit that stores the temperature state of the image sensor, and the image sensor that outputs the temperature sensor. For a video signal separated into R, G, and B signals, a white balance circuit that performs white balance by giving a gain and a means for giving a fixed white balance gain are provided, and the temperature at the time of factory adjustment is stored. The white balance gain is corrected according to the difference from the current temperature.

  Another feature of the present invention is that an image sensor that performs photoelectric conversion, a temperature sensor that detects the temperature of the image sensor, a unit that stores the temperature state of the image sensor, and the image sensor that outputs the temperature sensor. A white balance circuit that performs white balance by giving a gain to each video signal separated into R, G, and B signals, and a white balance gain that sets the R, G, and B signals to 1: 1: 1 Storing the temperature when the white balance gain is set so that the R, G, B signals are 1: 1: 1, and the white balance gain is set according to the difference from the current temperature. It is characterized by correction.

  Another feature of the present invention is that, in addition to the above features, the correction amount is changed in accordance with the white balance gain.

  Another feature of the present invention is that, in addition to the above-described features, white balance correction is performed linearly with respect to temperature.

  Another feature of the present invention is that in addition to the above feature, a means for determining whether the temperature of the temperature sensor is within the range is provided, and when it is determined that the temperature is out of the range, the correction amount is made constant. It is a feature.

  Another feature of the present invention is that, in addition to the above features, a temperature sensor for detecting the temperature of the image sensor is integrated with the image sensor.

  Another feature of the present invention is that, in addition to the above features, the imaging device is a CCD (Charge Coupled Device).

Another feature of the present invention is that the imaging device is an XY address type CMOS sensor.

  As described above, according to the present invention, the temperature of the sensor changes in the AWB mode and the preset WB mode by correcting the white balance gain in accordance with the temperature difference between the temperature at the time of factory adjustment and the current temperature. Even if it does, it becomes possible to set it as appropriate WB.

  The difference between the temperature at the time of factory adjustment and the current temperature is controlled because the AWB gain and the preset WB gain are generated from the WB adjustment parameters at the time of factory adjustment. Thus, the WB gain can be corrected satisfactorily.

  In addition, by correcting the WB according to the temperature difference between the temperature at the time of WB setting and the current temperature, even if the temperature of the sensor changes since the WB setting, it is possible to obtain an appropriate WB.

FIG. 2 is a diagram showing the overall configuration of the present invention.
The details will be described below.

  Reference numeral 201 denotes a lens for forming an image to be picked up, 202 denotes an iris / mechanical shutter that limits the amount of incident light, and 203 denotes a lens / iris drive circuit that drives the lens 201 and the iris / mechanical shutter 202. .

  Reference numeral 204 denotes an image sensor that photoelectrically converts the imaged light, such as a CCD or CMOS sensor. Reference numeral 205 denotes a CDS / AGC / A / D circuit that performs correlated double sampling (CDS), gain control, and A / D conversion on signals from the image sensor 204.

A TG (timing generator) 206 supplies horizontal / vertical transfer pulses, shutter pulses, sample hold signals, horizontal / vertical synchronization signals, etc. to the image sensor 204 and the CDS / AGC / A / D circuit 205. .
Reference numeral 207 denotes a signal processing circuit that receives image data from the CDS / AGC / A / D circuit 205, performs resize processing, aperture processing, WB processing, color signal processing, and the like to generate a still image and a moving image.

Reference numeral 208 denotes a temperature sensor that measures the temperature of the image sensor 204 and is disposed in the vicinity of the image sensor 204 or integrated with the image sensor 204.
A system controller 209 controls the lens / iris driving circuit 203, the TG 206, the signal processing circuit 207, and the like.

  An image input / output controller 210 controls the recording medium 211 and inputs / outputs moving images and still images. 211 is a recording medium, such as a magnetic tape, an optical disk, or a memory card.

  Reference numeral 212 denotes a panel / video output circuit that displays captured and reproduced images or outputs them externally. Reference numeral 213 denotes a WB mode changeover switch for the user to instruct an AWB mode, a preset WB mode, a WB set mode, and the like.

  Reference numeral 214 denotes a WB set execution switch for the user to instruct execution of the WB set.

  FIG. 1 is a more detailed view of the signal processing circuit 207 and the system controller 209 related to the present invention. Each will be described.

  Reference numeral 2071 denotes a WB circuit that performs WB processing on the input image data from the WB gain generated by the system controller.

  Reference numeral 2072 denotes a color matrix circuit that converts an RGB image signal output from the WB circuit and from which the WB is taken into a luminance signal Y and color difference signals RY and BY.

  Reference numeral 2091 denotes a WB control circuit that calculates an optimum WB using the output of the color matrix circuit 2072 as color detection data, or generates a WB setting operation and a preset WB gain.

  Reference numeral 2092 denotes an overall control circuit that controls the WB control circuit 2091, the correction amount calculation circuit 2093, and the temperature data changeover switch 2096 based on the inputs of the WB mode changeover switch 213 and the WB set execution switch 214.

  Reference numeral 2093 denotes a WB correction amount calculation circuit that generates WB correction data based on the temperature data of the temperature sensor 208, correction amount data 2097, WB gain data, adjustment temperature data 2095, and WB set execution temperature data 2094. It is.

  Reference numeral 2094 denotes WB set execution temperature data for storing data of the temperature sensor 208 at the time of execution of the WB set. Reference numeral 2095 denotes WB factory adjustment to be performed to correct variations in characteristics of the image sensor and the lens. This is factory adjustment temperature data that stores the adjustment temperature.

  Reference numeral 2096 denotes a temperature data changeover switch that selects and outputs either the WB set execution temperature data 2094 or the factory adjustment temperature data 2095 to the WB correction amount calculation circuit.

Reference numeral 2097 denotes correction amount data serving as a reference for calculating WB correction data.
Reference numeral 2098 denotes WB gain data at the time of executing the WB set for storing the WB gain data set at the time of executing the WB set.
Reference numeral 2099 denotes a non-volatile ROM for storing WB set execution temperature data 2094, factory adjustment temperature data 2095, correction amount data 2097, and WB set execution WB gain data.

FIG. 3 shows a flowchart of the operation in the present invention.
The invention described in claims 1 and 2 will be described according to the flowchart of FIG. 3 with reference to FIG.

When the AWB mode or the preset WB mode is selected by the WB mode changeover switch 213 (F301),
The temperature data changeover switch 2096 is switched to a and set in the WB correction amount calculation circuit 2093. The WB correction amount calculation circuit 2093 reads the factory adjustment temperature data 2095 (F302).

The WB correction amount calculation circuit 2093 generates temperature difference data ΔT between the current temperature data obtained by the temperature sensor 208 and the factory adjustment temperature data 2095. (F304)
At this time, if the current temperature is outside the correction range, it is regarded as a temperature change up to the correction range, a temperature difference ΔT is generated, and the correction data amount is limited (F305).

  For example, when the adjustment temperature is 20 ° C. and the correction range is 0 ° C. to 40 ° C. and the current temperature is 45 ° C., 40 ° C., which is the upper limit of the correction range, is regarded as the current temperature, and ΔT = 40 ° C. To do.

  In this way, when the temperature is out of the correction range, the correction amount can be made constant and unauthorized correction can be prevented.

  Subsequently, WB correction data is generated based on the temperature difference data ΔT, the WB gains R and B generated by the WB control circuit 2091, and the correction amount data 2097 (F306). The generation method will be described later.

  Finally, the WB gain data generated by the WB control circuit 2091 and the WB correction data generated by the WB correction amount calculation circuit 2093 are added and set in the WB circuit 2071 (F307), and these operations are repeated.

Next, details of generation of WB correction data will be described.
First, correction coefficients K_R and K_B, which are correction amounts per 1 ° C. sensor temperature at the current color temperature, are calculated.

  This is calculated from WB gain data Rgain, Bgain from the current WB control circuit and correction coefficient data 2097.

  In the case of the present embodiment, the correction coefficient data includes CMP_R_L and CMP_B_L as the correction coefficients for the respective WB gains at the low color temperature, and R_L and B_L as the WB gains at this time. The correction coefficients of the WB gains at the time of high color temperature are CMP_R_H and CMP_B_H, and the WB gains at this time are R_H and B_H.

From these parameters, R gain and B gain correction coefficients K_R and K_B are expressed by (Expression 1) and (Expression 2).
K_R = (CMP_R_H−CMP_R_L) * (Rgain−R_L) / (R_H−R_L) + CMP_R_L (Formula 1)
K_B = (CMP_B_H−CMP_B_L) * (Bgain−B_L) / (B_H−B_L) + CMP_B_L (Formula 2)
For reference, a graph showing the parameters in (Equation 1) is shown in FIG.

  Subsequently, WB correction data CMP_R and CMP_B are calculated.

This is because K_R and K_B are correction amounts per 1 ° C., and if the difference between the current temperature and the factory adjustment is ΔT, the WB correction data CMP_R and CMP_B are:
CMP_R = K_R * ΔT (Formula 3)
CMP_B = K_B * ΔT (Formula 4)
It becomes.

  Finally, CMP_R is added to Rgain and CMP_B is added to Bgain as WB correction data and set in the WB circuit 2071, so that WB can be corrected satisfactorily.

  The reason for controlling by the difference between the temperature at the time of factory adjustment and the current temperature is that the preset WB gain and AWB gain are generated from the WB adjustment parameters at the time of factory adjustment, and control is performed by the difference from the temperature at this time. Thus, the preset WB gain and AWB can be corrected satisfactorily.

  In this embodiment, the correction coefficient is calculated as described above. However, correction may be performed by storing the correction coefficient as table data with the sensor temperature and the WB gain as axes.

  As described above, in the AWB mode and the preset WB mode, even if the sensor temperature changes from the factory adjustment temperature by correcting the WB according to the temperature difference between the factory adjustment temperature and the current temperature, Appropriate WB can be achieved.

  4 and 5 show flowcharts of operations in the present invention.

According to the third aspect of the present invention, a WB set mode, that is, a white board is photographed, and a WB gain is set so that the image at this time becomes white (this operation is called WB set execution, and so on) ), And an embodiment of the present invention in a mode for holding or resetting the gain,
First, the operation at the time of executing the WB set will be described according to the flowchart of FIG. 4 with reference to FIG.

  When the WB set mode is selected by the WB mode changeover switch 213 and execution of WB set is instructed by the input of the WB set execution switch 214 (F401), the WB correction amount calculation circuit 2093 sets the output of the WB correction data to 0. (F402).

  The WB control circuit 2091 generates WB gain data so that the color detection data is white (F403). The operation here is the same as a general WB setting operation.

  The generated WB gain data is stored in the set WB gain data 2098 (F404).

  The WB correction amount calculation circuit 2093 stores the current temperature data obtained by the temperature sensor 208 in the WB setting temperature data 2094 (F405).

The above is the operation when executing the WB set.
Subsequently, the operation when the WB gain is reset after transition from the AWB mode, the preset WB mode, and the like after the execution of the WB setting is performed according to the flowchart of FIG. 5 with reference to FIG. I will explain.

  First, the temperature data changeover switch 2096 is switched to b, and the WB correction amount calculation circuit 2093 reads the WB set execution temperature data 2094 (F502).

  The WB correction amount calculation circuit 2093 generates temperature difference data ΔT between the current temperature data obtained by the temperature sensor 208 and the WB setting temperature data 2094 (F504).

  At this time, if the current temperature is outside the operation compensation range, it is regarded as a temperature change up to the compensation range, temperature difference data ΔT is generated, and the correction data amount is limited (F505).

  Subsequently, WB correction data is generated based on the temperature difference data ΔT, the WB gains Rgain and Bgain when the WB control circuit 2091 generated the previous WB, and the correction amount data 2097 (F506).

  The generation method is the same except that the reference ΔT in the first embodiment is the difference between the factory adjustment temperature data 2095 and the WB setting temperature data 2094.

  Finally, the set WB gain data 2098 is read from the nonvolatile ROM 2099, added to the WB correction data generated by the WB correction amount calculation circuit 2093, and set in the WB circuit 2071 (F507). These operations are repeated unless the mode can be switched.

  As described above, by correcting the WB according to the temperature difference between the temperature at the time of WB setting and the current temperature, even if the temperature of the sensor changes since the WB setting, it is possible to obtain an appropriate WB. .

  Further, in the temperature measurement by the temperature sensor 208 of the above embodiment, by providing the measurement data with hysteresis, data fluctuation due to noise or the like can be prevented.

  As described above, the image processing apparatus according to the present embodiment includes an image sensor that performs photoelectric conversion, a temperature sensor that detects the temperature of the image sensor, a unit that stores the temperature state of the image sensor, and an output from the image sensor. Factory-adjusted with white balance circuit that performs white balance by giving gain to each of the separated video signal into R, G, and B signals, and auto white balance gain setting means that automatically follows white balance The temperature of the hour is stored, and the white balance gain is corrected according to the difference from the current temperature.

  The image processing apparatus according to the present embodiment includes an image sensor that performs photoelectric conversion, a temperature sensor that detects the temperature of the image sensor, a unit that stores the temperature state of the image sensor, and an image output from the image sensor. For the signal separated into R, G, and B signals, a white balance circuit that performs white balance by giving a gain and a means for giving a fixed white balance gain are provided, and the temperature at the time of factory adjustment is stored, The white balance gain is corrected according to the difference from the current temperature.

  The image processing apparatus according to the present embodiment includes an image sensor that performs photoelectric conversion, a temperature sensor that detects the temperature of the image sensor, a unit that stores the temperature state of the image sensor, and an image output from the image sensor. A white balance circuit that performs white balance by giving a gain to each of the signals separated into R, G, and B signals, and a white balance gain that sets the R, G, and B signals to 1: 1: 1. The temperature when the white balance gain is set so that the R, G, B signal is 1: 1: 1 is stored, and the white balance gain is corrected according to the difference from the current temperature. It is characterized by doing.

Further, the image processing apparatus according to the present embodiment is characterized in that the correction amount is changed according to the white balance gain.
In addition, the image processing apparatus according to the present embodiment is characterized in that in white balance correction, correction is performed linearly with respect to temperature.
In addition, the image processing apparatus according to the present embodiment includes means for determining whether the temperature of the temperature sensor is within the range, and when it is determined that the temperature is out of the range, the correction amount is constant.
Further, the image processing apparatus according to the present embodiment is characterized in that the temperature sensor that detects the temperature of the image sensor is integrated with the image sensor.
In addition, the image processing apparatus according to the present embodiment is characterized in that the imaging element is a CCD (charge coupled device).
The image processing apparatus according to the present embodiment is characterized in that the imaging element is an XY address type CMOS sensor.

Detail view Overall view Operation flowchart during AWB Operation flowchart when executing WB set Operation flowchart when holding WB set Graph showing parameters in (Equation 1) Diagram showing output waveform of image sensor The figure which shows the difference of the output when A / D conversion

Explanation of symbols

207 Signal processing circuit 2071 WB circuit 2072 Color matrix circuit 209 System controller 2091 WB control circuit 2092 Overall control circuit 2093 WB correction amount calculation circuit 2094 WB set execution temperature data 2095 Factory adjustment temperature data 2096 Temperature data changeover switch 2097 Correction coefficient data 2098 WB gain data when executing WB set 2099 Non-volatile ROM
213 WB mode change switch 214 WB set execution switch

Claims (9)

An image sensor that performs photoelectric conversion;
A temperature sensor for detecting the temperature of the image sensor;
Means for storing a temperature state of the image sensor;
A white balance circuit for performing white balance by giving a gain to an image signal output from the image sensor separated into R, G, and B signals, and auto white balance gain setting means for automatically following the white balance. Prepared,
An image processing apparatus that stores a temperature at the time of factory adjustment and corrects a white balance gain in accordance with a difference from the current temperature. An image sensor that performs photoelectric conversion;
A temperature sensor for detecting the temperature of the image sensor;
Means for storing a temperature state of the image sensor;
A white balance circuit for performing white balance by giving a gain to an image signal output from the image sensor separated into R, G, and B signals;
With a means to give a fixed white balance gain,
An image processing apparatus that stores a temperature at the time of factory adjustment and corrects a white balance gain in accordance with a difference from the current temperature. An image sensor that performs photoelectric conversion;
A temperature sensor for detecting the temperature of the image sensor;
Means for storing the temperature state of the image sensor;
A white balance circuit that performs white balance by giving a gain to each of the video signals output from the image sensor separated into R, G, and B signals;
Means for setting the white balance gain so that the R, G, B signals are 1: 1: 1;
The temperature when the white balance gain is set so that the R, G, B signals are 1: 1: 1 is stored, and the white balance gain is corrected according to the difference from the current temperature. An image processing apparatus. The image processing apparatus according to claim 1, wherein the correction amount is changed in accordance with the white balance gain. 4. The image processing apparatus according to claim 1, wherein the white balance is corrected linearly with respect to the temperature. Means for determining whether the temperature of the temperature sensor is within the range;
4. The image processing apparatus according to claim 1, wherein the correction amount is constant when it is determined that the value is out of the range. The image processing apparatus according to claim 1, wherein the temperature sensor that detects the temperature of the image sensor is integrated with the image sensor. The image processing apparatus according to claim 1, wherein the image pickup device is a CCD (Charge Coupled Device). The image processing apparatus according to claim 1, wherein the image sensor is an XY address type CMOS sensor.

Images