JP2008206055A - Image processor and imaging apparatus - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To accurately perform white balance adjustment of a display image even when an environmental color temperature is varied. <P>SOLUTION: An image processor includes: a primary color separator for separating a chrominance signal from a pixel signal; an integration circuit for integrating the separated chrominance signal for a predetermined period of time; a coordinate axis transforming section for transforming the integrated chrominance signal into color coordinate data; a storage circuit which measures a primary color signal beforehand and stores color temperature data of the measured data as evaluation reference data; a color temperature comparing section for comparing the evaluation reference data with color temperature data newly output from the coordinate axis transforming section; and a control section which selects a specified region of the image as an integration region, controls the integration circuit to integrate a chrominance signal in the integration region and estimates a color temperature in the specific region of the image in accordance with comparative data output from the color temperature comparing section, wherein white balance adjustment is performed by accurately estimating a color temperature even when the environmental color temperature is varied. <P>COPYRIGHT: (C)2008,JPO&INPIT

Description

  The present invention relates to an image processing apparatus and an imaging apparatus using a solid-state imaging device and the like, and in particular, an image processing apparatus and an imaging apparatus that estimate a color temperature from a change in color of a reference value of a fixed subject using information on the fixed subject of the image. Relates to the device.

In a solid-state imaging device such as a CCD (Charge Coupled Device) or CMOS (Complementary Metal Oxide Semiconductor) device or an imaging device (digital camera or video camera) using the same, white is accurately projected in white In this way, the white balance is controlled and corrected.
FIG. 11 shows a system configuration of signal processing for performing white balance control.
As shown in FIG. 11, an electric signal obtained by a solid-state image sensor, for example, a CCD type image sensor or the like is subjected to signal processing and is input to a primary color separator 201 as a digital signal. The primary color separator 201 separates the input digital signal into R (red), G (green), and B (blue) primary color signals. The separated color signal is input to a white balance amplifier (WB amplifier) 202.
The R, G, and B signals that have been separated into primary colors are also supplied to an optical detector circuit (OPD) 203 for use in white balance adjustment. The optical detector circuit 203 includes an integration circuit that integrates R, G, and D signals for each field. The integral value data obtained by the optical detector circuit 203 is supplied to the controller 204 at the next stage.

Based on the integral value data of the R ′, B ′, and G ′ color signals supplied from the optical detector circuit 203, the controller 204 generates (R′−B ′) / G ′ data and (R ′ + B ′). -2G ') / G' data is calculated and the result is output, and (R'-B ') / G', (R '+ B'-2G') / G 'coordinate data A gain setting function for setting the gains of the R signal, G signal, and B signal is provided. Note that these functions are executed by software. Each gain setting value obtained by software execution of the controller 204 is supplied to each white balance amplifier 202 to adjust each gain relating to the R, G, and B signals.
Here, adjusting the white balance means adjusting the white balance amplifier 202 so that the ratios of the R, G, and B signals of the estimated color temperature are equal.

As a method of estimating the color temperature, there is a method of estimating using a black body radiation curve as disclosed in Patent Document 1.
In this method, when adjusting the white balance, the estimated color temperature is set by the following equation.

R signal x R gain-G signal x G gain = B signal x B gain-G signal x G gain = 0
... (1)

That is, the white balance amplifier 202 is controlled so that R: G: B = 1: 1: 1.
In Cited Document 2, the white wall of the background of the amusement camera is used as a fixed subject, and white balance control is performed on the premise that the camera is all white within the effective area of the amusement camera. Is disclosed.
JP 2004-320671 A JP 2000-59807 A

However, in the example described above, since the subject cannot be specified, the color temperature (light source) is estimated on the assumption that the ratio of R: G: B of all pixels in the effective area is 1: 1: 1. Was. As a result, accurate color temperature cannot be estimated, and there is a problem that the original color of the subject is destroyed by performing white balance control with R: G: B = 1: 1: 1.
In view of the above problems, it is an object of the present invention to provide an image processing apparatus and an imaging apparatus that can estimate a color temperature with high accuracy for a fixed subject of any color and perform white balance adjustment that matches the color temperature.

  An image processing apparatus of the present invention includes a primary color separator that separates a color signal from a pixel signal, an integration circuit that integrates the separated color signal for a predetermined period, and a coordinate axis conversion unit that converts the integrated color signal into color coordinate data. And a storage circuit that measures the color signal in advance and stores the color temperature of the measured color signal as evaluation reference data, and compares the evaluation reference data with the color temperature data newly output from the coordinate axis conversion unit. The color temperature comparison unit and the specified region of the image are selected as the integration region, the integration circuit is controlled to integrate the color signal in the integration region, and according to the comparison data output from the color temperature comparison unit And a controller for estimating the color temperature of the specific area of the image.

  An image pickup apparatus according to the present invention includes an image pickup element, a primary color separator that separates a pixel signal output from the image pickup element into a color signal, an integration circuit that integrates the separated color signal for a predetermined period, and an integrated color signal. A coordinate axis conversion unit that converts the color coordinate data into a color coordinate data, a storage circuit that measures the pixel signal of the integration region in advance and stores the measured color temperature data as evaluation reference data, and outputs from the evaluation reference data and the coordinate axis conversion circuit A color temperature comparison unit that compares the color temperature data that has been generated, and the specified region of the image is selected as an integration region, the integration circuit is controlled to integrate the color signal in the integration region, and the color temperature comparison A control unit that estimates a color temperature of a specific area of the image according to the comparison data output from the unit, a white balance is adjusted by the color temperature estimated by the control unit, and the adjusted color signal and the above Signal processing the luminance signal Motoshingo and a signal processing circuit for generating an image signal.

  The image processing apparatus and the imaging apparatus of the present invention allow a predetermined fixed subject to be present in the display image, and set an integration region within the fixed subject. The color temperature of the pixel signal in this integration area when the ambient temperature is varied is accumulated as reference data, and the color temperature is estimated by comparing the color temperature data of the specified integration area during shooting with the accumulated reference data. The white balance is adjusted according to the estimated temperature.

The image processing apparatus and the imaging apparatus of the present invention can estimate the color temperature with high accuracy from the change in the color of the specified integration region. Also, the color temperature can be estimated from an identified integration region color change even in an imaging system without an infrared cut filter, and highly accurate white balance control independent of the subject can be performed.
According to the present invention, in an image processing apparatus and an imaging apparatus of a solid-state imaging device using a CCD, a CMOS device, or the like, white balance control can be performed with high accuracy particularly in a change in imaging environment.

FIG. 1 shows a schematic configuration diagram of a solid-state imaging device (camera device) 10 according to an embodiment of the present invention. Hereinafter, a solid-state imaging device using a CCD device will be described, but the present invention can also be applied to an imaging device using other imaging devices.
The solid-state imaging device 10 includes, for example, a lens 11, an imaging device 12, a preamplifier 13 having a CDS (correlated double sampling) and a preamplifier, an A / D (analog / digital) converter 14, a preprocessing unit 15, and primary color separation. And a signal processing unit (50) having a luminance signal processing unit (70) and a color signal processing unit (80). The signal processing unit (50) will be described later.
A part of the color signal processing unit (80) includes a white balance amplifier 17, an optical detector circuit 18, and a controller 19, as shown in FIG.

  The lens 11 projects an image of a subject (not shown) on the imaging surface of the imaging element 12. The image sensor 12 is composed of, for example, a CCD or a CMOS device, converts an image transmitted through the lens 11 into an electric signal, and supplies it as a pixel signal to a preamplifier 13 having a CDS (correlated double sampling), a preamplifier and the like.

  The preamplifier 13 samples and holds the pixel signal from the image sensor 12 to extract necessary data, performs gain control (AGC) to adjust to an appropriate level, and also performs black level adjustment. The output signal of the preamplifier 13 is output to the A / D converter 14 at the subsequent stage.

  Since the A / D converter 14 handles pixel (image) signals, generally, an A / D converter having an accuracy of 10 to 12 bits is adopted, and the output signal supplied from the preamplifier 13 is converted from an analog signal to a digital signal, and the digital signal is converted. The signal is output to the signal processing unit 50 connected to the subsequent stage.

  The signal processing unit 50 includes, for example, a preprocessing unit 15, a primary color separator 16, a luminance signal processing unit 70, and a color signal processing unit 80, and performs digital signal processing.

  In the preprocessing unit 15, black detection, digital gain control (Digital Gain Control), shading correction generated in the lens 11, and a delay line are used to separate a luminance signal (data) and a color signal (data), and a delay line Is used to correct pixel defects.

  The primary color separator 16 extracts primary color signals of red (R), blue (B), and green (G) from the digital signal corrected by the preprocessing unit 15.

  The luminance signal processing unit 70 corrects vertical / horizontal (direction) contours of Y (luminance) signals (VH aperture control), mixes (mixes; adds) processing of Y luminance signals and vertical / horizontal contour correction signals, and γ (gamma) corrections. Various image processing such as luminance (brightness) key processing, solarization for processing a part of the image to an arbitrary luminance, and negative processing for inverting the image are performed. Details of the luminance signal processing unit 70 will be described later.

  The color signal processing unit 80 performs color separation and clamping processing, removal of color signal noise and color false signals, RGB matrix (Matrix) processing, white balance (WB) adjustment for varying the coefficients of R, G, and B colors, γ (gamma) correction, RG / BG conversion, color false signal suppression processing, color difference signal (Cr / Cb) generation, chroma suppression (suppression) processing, Hue / Gain adjustment, monotone effect processing, etc. . Details of the color signal processing unit 80 will be described later.

FIG. 2 shows a block configuration example of the signal processing unit 50.
The signal processing unit 50 is mainly composed of three blocks: a preprocessing unit 60, a Y-block (luminance signal processing unit) 70, and a C-block (color signal processing unit) 80.

  The pre-processing unit 60 includes a black detection unit 61, an adder 62, a digital gain adjustment unit 63, a shading correction unit 64, a delay line / defect correction unit 65, and the like.

  The black detector 61 uses the digital data (ADin) output from the A / D converter 14 to calculate the black level of the input data (signal), and outputs data for calculating the clamp level.

  The adder 62 subtracts the black level from the digital data (ADin), and generates a new pixel signal (data) based on the black level (clamped to the black level).

  The digital gain adjustment unit 63 adjusts the brightness by digitally changing the gain of the data clamped at the black level output from the adder 62.

  The shading correction unit 64 has a luminance difference (unevenness) in the peripheral portion with respect to the central portion of the image due to the aperture of the lens 11, and corrects this unevenness in luminance.

  The delay line / defect correction unit 65 separates input data into a luminance signal (data) and a color signal (data) using a delay line and a calculator (adder / subtracter). Also, the input data is delayed by, for example, one pixel or two pixels by the delay element, a value obtained by averaging the data before and after the defective pixel is obtained, and the value is replaced with the defective pixel to perform pixel defect correction.

  The data corrected by the delay line / defect correction unit 65 and further separated into the luminance signal and the color signal are supplied to the luminance signal processing unit (Y-block) 70 and the color signal processing unit (C-block) 80, respectively. The

  A luminance signal processing unit (Y-block) 70 includes a YLPF (luminance signal low pass filter) 71, a vertical / horizontal contour correction (VH aperture control; VH aperture control) unit 72, and a contour correction adder (contour correction). Mix; aperture control (Mix) 73, γ (gamma) correction unit 74, image effect processing unit 75, chroma suppress signal generation unit 76, and the like.

  The YLPF 71 performs addition calculation processing on the data relating to the luminance signal to equivalently remove noise.

  The vertical / horizontal contour correcting unit 72 generates a pulse for emphasizing a horizontal contour portion of an image by using a delay element and an arithmetic unit for addition / subtraction processing, and a pulse for emphasizing the vertical contour portion of the image. Also generate.

  The contour correction adder (contour correction mix) 73 is supplied with the horizontal contour correction pulse and the vertical contour correction pulse output from the vertical / horizontal contour correction unit 72 and the luminance data supplied from the YLPF 71. Are added and output as luminance data with an enhanced outline.

  For example, if the display device is a CRT (Cathode Ray Tube), the gamma (γ) correction unit 74 has a γ characteristic of 2.2 for this CRT, so that 1 / γ = 0.45 on the imaging side beforehand. The input / output characteristic curve is corrected so that the gradation of the image is correctly reproduced on the CRT side, and correction is performed so that a correct reproduced image is obtained.

  The image effect processing unit 75 performs luminance (brightness) key processing, solar processing for processing a partial area of the image to an arbitrary luminance, and negative processing for inverting the image.

  The chroma suppress signal generation unit 76 generates a control signal for deleting the chroma (color) data from the pulse signal generated by the vertical / horizontal contour correction unit 72 using the pulse of the image contour.

Next, the color signal processing unit 80 shown in FIG. 2 will be described.
The color signal processing unit 80 receives the color signal processed by the delay line / defect correction unit 65 and receives the primary color separator 16, optical detector circuit 18, controller 19, white balance amplifier (white balance adjustment unit) 17, gamma ( (γ) correction unit 85, RG / BG conversion unit 86, Cr / Cb generation unit 87, chroma suppress, Hue (hue control) / Gain (gain) adjustment unit (monotone effect) 88, and the like.

  The primary color separator 16 color-separates the pixel signal (data) output from the delay line / defect correction unit 65 of the preprocessing unit 60 and clamps it to a predetermined value.

The integrating circuit of the optical detector circuit 18 integrates the R signal, G signal, and B signal output from the primary color separator 16 for one field (or one frame) period, and outputs the integrated color signal to the controller 19. .
3 is supplied with R, G, B signals of the fixed subject frame. When the address of the fixed frame of the fixed subject is reached, the fixed-subject frame integration unit 101 starts operation, and the R, G, and B signals supplied from the primary color separator 16 are integrated within the fixed subject frame.

The controller 19 is provided with a microcomputer, a storage device for storing control programs and data, and the like. As shown in FIG. 3, the controller 19 includes a coordinate axis conversion unit 102 that performs data processing on color signals, a color temperature comparison unit 103, an evaluation data storage device 104, and the like.
The controller 19 supplies a control signal to the black detector 61 in accordance with the signal detected by the optical detector circuit 18 to control the black level. In addition, the controller 19 controls the operation of the integration unit 101 in the fixed subject frame of the optical detector circuit 18, performs address counting and integration operation control of the integration region b of the fixed subject, and calculates the integrated color signal. Coordinate conversion is performed, and the color temperature of the display image is estimated from this value and the evaluation color temperature stored in the evaluation data storage device.
The fixed subject frame integration unit 101, the coordinate axis conversion unit 102, the color temperature comparison unit 103, and the like can be configured by software using a microcomputer, but of course, can be configured by hardware other than this.

  The white balance amplifier (WB amplifier) 17 includes an R signal amplifier 17r, a G signal amplifier 17g, and a B signal amplifier 17b, and a gain is set according to the color temperature information output from the controller 19.

Next, the main part of white balance adjustment will be described.
As shown in FIGS. 1 and 2, the main part of white balance adjustment includes an optical detector circuit 18 and a controller 19.
FIG. 3 shows a specific configuration example of a main part of white balance adjustment. 3, the optical detector circuit 18 shown in FIG. 2 includes a normal integration circuit and a fixed subject frame integration unit 101, and a part of the controller 19 includes a coordinate axis conversion unit 102, a color temperature comparison unit 103, evaluation data. The storage device 104 is configured.

  The coordinate axis conversion unit 102 provided in the controller 19 is integrated data (integrated color signals) R ′ and B ′ in the fixed subject frame supplied from the integration unit 101 in the fixed subject frame by a control circuit such as a microcomputer. , G ′ signals, the horizontal axis is converted to (R ′ + B′−2G ′) / G ′, the vertical axis is converted to (R′−B ′) / G ′, and color coordinates are converted. The converted coordinate values are output to the color temperature comparison unit 103.

Next, a method for obtaining the color temperature from the coordinate data obtained by the coordinate axis conversion unit 102 will be described. FIG. 4 shows an example of color temperature coordinates. In the coordinate axis conversion unit 102, the black body radiation curve is approximated by a straight line (hereinafter referred to as a color temperature change straight line LB) to estimate the color temperature.
In FIG. 4, when the X axis is set to (R ′ + B′−2G ′) / G ′ and the Y axis is set to (R′−B ′) / G ′, the coordinate area “A” is in the third quadrant. This is a region on the left side of the color temperature change line LB within the range of W1 and within the range of the frame W1 in the fourth quadrant.
The integration value conversion data obtained by converting the integration data (color signals) R ′, G ′, and B ′ into coordinate data was at the position indicated by a circle (circle) in the above-described area “A”. In this case, as indicated by an arrow, a point that translates along the X axis of (R ′ + B′−2G ′) / G ′ and intersects the color temperature change line LB is estimated as the current color temperature.
On the other hand, the area “B” on the coordinates is an area on the right side of the color temperature change line LB within the range of the frame W1 in the fourth quadrant.
When the integration value conversion data obtained by converting the integration data R ′, G ′, B ′ into coordinate data is at the position indicated by Δ in the above-mentioned region “B”, as indicated by an arrow ( R ′ + B′−2G ′) / X ′ axis of G ′ and Y axis of (R′−B ′) / G ′ are moved in the negative direction by the same amount and intersect with the color temperature change line LB Calculate the color temperature from
Specifically, from the position of Δ (triangle mark) in the region “B”, it is moved 45 ° to the lower left on the coordinates, and the point where the extension line and the color temperature change line LB intersect is determined as the current color temperature. Infer.

Here, adjusting the white balance means adjusting the white balance amplifier so that the ratios of the R, G, and B signals of the color temperature obtained by estimation are equal as described above.
This color temperature estimation method is a method of estimating using a black body radiation characteristic approximated by a straight line, but the present invention should not be limited to this.

  In this signal processing circuit, when adjusting the white balance, the estimated color temperature is

R signal × R gain = G signal × G gain = B signal × B gain (2)
Ie

R signal × R gain−G signal × G gain = B signal × B gain−G signal × G gain = 0
... (3)
The gain of the white balance amplifier 17 is set so that

In order to estimate the color temperature, it is necessary to prepare a reference value (evaluation data) for the fixed subject when the color temperature is changed in advance. An evaluation data storage device 104 is provided for storing the evaluation data.
In order to prepare evaluation values (evaluation data) for a fixed subject, first, (r−g) / g and (r + b−2g) / g are obtained from r, g, and b of the fixed subject in an environment where the color temperature is low. The color temperature value and the data of (r−g) / g, (r + b−2g) / g are stored in the evaluation data storage device 104, for example, a memory. Next, the color temperature value is increased, and data at each color temperature is stored in the memory in order. The result obtained by repeating this operation is shown in FIG. Such an operation is performed by the coordinate axis conversion unit 102.

  Next, the color temperature comparison unit 103 shown in FIG. 3 performs coordinate conversion on the actual integral value in the fixed subject frame output from the coordinate axis conversion unit 102 to (R′−B ′) / G ′, (R ′). The color temperature obtained from (+ B′−2G ′) / G ′ and the evaluation data at each color temperature of the fixed subject stored in advance are compared to estimate the color temperature.

Then, the actual color temperature of the solid subject is corrected based on the evaluation data by the control signal of the controller 19, and the R color signal, the G gain signal, and the B gain signal of the corrected color temperature estimation value are shown in FIG. The white balance is adjusted by outputting to the R signal amplifier 17r, the G signal amplifier 17g, and the B signal amplifier 17b of the white balance amplifier 17, and setting the gain.
This white balanced digital signal is output to the γ correction unit 85 in the next stage, where the input-output characteristics are corrected according to the gradation characteristics of the display with respect to the input signal level.

  The R-G / B-G conversion unit 86 γ-corrects the color data output from the white balance amplifier 17 and then suppresses the color false signal due to mixing of other color signals between pixels of the image sensor. Then, it is converted into color signals of R (red) -G (green) and B (blue) -G (green).

The Cr / Cb generator 87 generates RY and BY color difference signals. Here, Cr = R−Y, Cb = B−Y (Y is a luminance signal).
The chroma suppress, Hue / Gain adjustment, and monotone effect processing unit 89 prevents the output of Cr and Cb color difference signal data by the control signal output from the chroma suppress signal generation unit 76.
In addition, with respect to the data of the color difference signal, Hue (hue) adjustment and Gain (gain) adjustment are performed, and a monotone effect processing for removing a color to make a black and white image is also performed.
The processed (color) data is output to an encoder (not shown), encoded using the luminance (Y) signal output from the luminance signal processing unit (Y-block) 70, and a component or composite image signal is generated. Generated.

Next, an operation when the vehicle body is a fixed subject in the in-vehicle camera will be described. In addition, although the color of the body a of a car is demonstrated as red, it is not limited to this color.
FIG. 5A is a diagram illustrating an in-vehicle camera using a 180-degree wide-angle lens as an example. When a rear (rear) camera is installed because the wide-angle lens is used, the car body also enters the effective area of the camera, and the car body is always effective even if the car moves and the surrounding scenery changes It becomes a fixed subject in the area.
As shown in FIGS. 5A and 5B, the vehicle body a is a fixed subject, and an integration region b is set in a partial region thereof. For example, as shown in FIG. 5B, an integration area b indicated by a broken line is set in the body (a) of the car of the display image taken by the rear camera. In this case, the integration region b is not limited in place and color in the body of the car.

  FIG. 6 shows another example of the fixed subject in the captured image. In FIG. 6, for example, a specific region indicated by a broken line in the building a ′ is set as an integration region b ′, and temperature evaluation can be performed in the same manner as in the case of the in-vehicle camera. Therefore, detailed description of the building a 'will be omitted.

FIG. 7 shows a flowchart for calculating values of (R ′ + B′−2G ′) / G ′ and (R′−B ′) / G ′ related to coordinate conversion of color signals. The operation relating to the coordinate transformation will be described with reference to FIGS.
The pixel signal from the A / D converter 14 is supplied to the primary color separator 16 via the preprocessing unit 15, and the color signal separated into primary colors is supplied to the optical detector circuit 18 (ST-12).
Further, horizontal and vertical addresses relating to the coordinate position of the pixel signal (or color signal) are measured (ST-14).
Next, the horizontal / vertical address value of the pixel signal input within the effective area of the camera is counted, and the address of the integration frame b of the fixed subject a is set (ST-16).
When the pixel signal supplied from the primary color separator 16 becomes the address of the integration frame (inner) b, the fixed subject frame integration unit 101 starts to operate under the control of the controller 19 and integrates the color signal. An enable signal is generated (ST-18).
Next, as the in-frame integration enable signal is set and the R, G, and B signals after color separation are input, the horizontal and vertical address values are counted within the effective pixel area of the camera, and within the integration frame It is determined whether or not there is (ST-20).
When the address value of the color signal coincides with the integration area (inside frame) b of the fixed subject a, an in-frame integration enable signal that becomes active is generated. The R, G, and B signals color-separated by the primary color separator 16 based on the enable signal are integrated for each color by the integration unit 101 in the fixed subject frame of the optical detector circuit 18, and the integration region b in the fixed subject is integrated. Integral values R ′, G ′, and B ′ are generated at (ST-22).
Further, the integrated values R ′, G ′, B ′ are supplied to the coordinate axis conversion unit 102, where (R′−G ′) / B ′, (R ′ + B′−2B ′) / G ′ based on the coordinate axes. Is calculated (ST-24). The color temperatures indicated by (R′−G ′) / B ′ and (R ′ + B′−2B ′) / G ′ on the coordinate axes can be obtained from the description of FIG.

Although the method for estimating the color temperature has been described so far, it is necessary to prepare a reference value (evaluation data) for the integration region b of the fixed subject a when the color temperature is changed in advance in order to estimate the color temperature.
FIG. 8A shows a flowchart for preparing evaluation data for the integration region b. A description of the flowchart relating to this color temperature estimation will be given with reference to FIG.
First, in an environment where the color temperature is low, for example, an image of the integration region b of the fixed subject a is captured from a captured image at 1000 ° K (Kelvin), and the pixel signal separated from the primary color separator 16 is captured by the fixed subject of the optical detector circuit 18. This is supplied to the in-frame integration unit 101 (ST-42). For example, the fixed subject a is a vehicle body a, and the integration region b is a region surrounded by a broken line in the body a. Then, the color signals are integrated by the fixed subject frame integration unit 101 to obtain the r, b, and g color signals. Here, r, b, and g are red, blue, and green color signals when measuring the integration region b (or the fixed subject a).

  Next, the previously processed evaluation color temperature and the currently set evaluation color temperature are added (ST-44). Initially, since the previous evaluation color temperature does not exist, the temperature is set to 0 degrees K and addition processing is performed. From the color signals of r, b, and g in the integration region b obtained by integration by the fixed subject frame integration unit 101 at 1000 degrees K (Kelvin) in this case, (r−g) / b , (R + b−2g) / g. The resulting color signal is plotted on the color coordinates (corresponding to the 1000 ° K point in FIG. 8B), and the color temperature and the corresponding coordinate data are stored in a storage device such as a memory (ST -46).

  Next, it is determined whether or not the color temperature is maximum (ST-48). Since the evaluation color temperature is 1000 degrees K now, assuming that the maximum evaluation color temperature is 9000 degrees K, it is lower than the maximum evaluation color temperature, so the process proceeds to ST-44.

  Since the color temperature at the time of calculation processing in step ST-46 was 1000 degrees K, an arbitrary evaluation temperature step, for example, 1000 degrees K, is added to this time (ST-44). Next, the color signals of r, b, and g in the integration region b surrounded by the broken line of the car body a when the environmental color temperature is 2000 degrees K are obtained. Using the r, b, and g color signals, (r−g) / b and (r + b−2g) / g are calculated to obtain data on color coordinates, and the calculation results are plotted (ST) -46). The color coordinate data corresponds to a point of 2000 degrees K in FIG. The measurement data and environmental color temperature are stored in the memory.

Thereafter, the environmental temperature is compared with the maximum color temperature (ST-48), the same operation is repeated until the maximum temperature, for example, 9000 degrees K, and the creation of the evaluation data is completed (ST-50).

FIG. 8B is a diagram plotting color coordinate data when the environment of the color temperature is varied from 1000 degrees K to 9000 degrees K in 1000 degrees K steps.
The color signal (data) of the color coordinates stored in the memory is used as reference data for the evaluation of the color temperature.

  The reference data at the time of color temperature evaluation is created also in a fixed camera such as the surveillance camera shown in FIG. Since the surveillance camera is fixed, a building (building) a ′, a tree c, or the like is used as a fixed subject, and the color signal when the environmental color temperature is similarly changed for the integration region b ′ in the fixed subject frame is measured. Then, the color temperature data obtained as a result is stored as reference data.

Next, FIGS. 9A and 9B are flowcharts for estimating the color temperature of the subject of the display image displayed by the imaging apparatus or the like. In the display image, the color coordinate data of (R′−B ′) / G ′, (R ′ + B′−2G ′) / G ′ is shown in FIG. For example, when it is in the vicinity of 5000 degrees K or less of the position shown in b), it is determined as follows.
The color signals R, G, B of the integration area b in the fixed subject frame of the body a of the traveling vehicle are integrated by the integration unit 101 in the fixed subject frame to obtain R ′, B ′, G ′ signals. The R ′, B ′, and G ′ signals are processed by the coordinate axis conversion unit 102 to perform coordinate conversion. Specifically, the value of (R′−B ′) / G ′ corresponding to the Y-axis value of color coordinates and the value of (R ′ + B′−2G ′) / G ′ corresponding to the X-axis value are obtained. Then, the converted color signal is supplied to the color temperature comparison unit 103 (ST-62).
In the first determination, first, (R ′ + B′−2G ′) / G ′ corresponding to the value of the X axis on the color coordinate and the value of the X axis (r + b−2) / G / of the evaluation data previously stored in the memory or the like. g are compared, and a candidate value of the estimated color temperature is selected. Here, 2000 degrees K, 3000 degrees K, 4000 degrees K, and 5000 degrees K are selected as candidate color temperature values (ST-64).
Further, in the second determination, (R′−B ′) / G ′ corresponding to the Y-axis value on the color coordinate is compared with the Y-axis value (r−g) / g of the evaluation data stored in advance in a memory or the like. (ST-68). Then, an estimated color temperature is selected from the candidate values. As a result, 5000 degrees K closest to the evaluation value is selected from the X-axis value and the Y-axis value (ST-70).
By using this estimated color temperature value, white balance is more accurate than color temperature (light source) estimation based on the assumption that the ratio of R: G: B of all pixels in the effective area is 1: 1: 1. Control can be performed.

  Similarly, in the case of a fixed camera such as a surveillance camera in FIG. 6, the color temperature determination of the integration region b ′ of the building a ′ is performed in accordance with the change of the environmental color temperature in the same manner as in the case of the on-vehicle camera, and the estimated color temperature is Can be estimated.

Furthermore, as another modified example, color temperature estimation can be performed also in an imaging apparatus (camera apparatus) that is not equipped with an infrared light cut filter.
In the camera device without the infrared cut filter, the color of the subject changes due to the influence of the infrared light component, and the conventional color temperature estimation method cannot estimate the accurate color temperature. However, by using the color temperature estimation method described above and preparing an evaluation value without an infrared light cut filter, an accurate color temperature can be estimated even if the color of the subject changes due to the influence of the infrared component. can do.

Next, an example when the estimated color temperature value is applied to white balance adjustment will be described.
FIG. 10A is a graph showing the characteristics of the color signal gain with respect to the black body radiation temperature. In FIG. 10A, the horizontal axis represents color temperature, and the vertical axis represents R gain and B gain. At this time, it is assumed that the G gain is 1.
The R gain increases in a curve as the color temperature on the horizontal axis increases. On the other hand, the B gain decreases in a curve as the temperature increases.
As shown in FIG. 10A, an R gain value f and a B gain value g for a color temperature of 5000 degrees K on the X coordinate are obtained.
The gains (gain coefficients) of the R signal amplifier 17r, the G signal amplifier 17g, and the B signal amplifier 17b of the white balance amplifier 17 are obtained from the R gain f and B gain g corresponding to the estimated color temperature of 5000 ° K. Supply to the amplifier.

FIG. 10B shows a flowchart for performing white balance adjustment using each estimated color temperature.
First, an operation for obtaining the minimum and maximum evaluation color temperature is started (ST-82).
The currently set color temperature is added to the previously evaluated evaluation color temperature (ST-84). For example, if the set temperature of the previous color temperature is 4000 degrees K, if the color temperature step added this time is 500 degrees K, it is set to 4500 degrees K. The step color temperature to be added can be arbitrarily set depending on accuracy.
A fixed subject and a Macbeth chart (trade name) are arranged under each color temperature, and R gain and B gain are adjusted so as to obtain an optimum color by visual evaluation (ST-86).
The adjusted R gain and B gain are stored in the memory (ST-88).
It is determined whether the used evaluation color temperature of 4500 degrees K is equal to or lower than the maximum color temperature (5000 degrees K) (ST-90).
When the evaluation color temperature is equal to or lower than the maximum color temperature, the R gain and the B gain are reset to 1, and the process proceeds to ST-84 (ST-92). In ST-84, a color temperature (addition step temperature) with a predetermined accuracy is added to the evaluation color temperature of 4500 degrees K. By adding 500 degrees K as the addition step temperature, the evaluation color temperature is set to 5000 degrees K, and thereafter, ST-86 to ST-90 are similarly repeated. On the other hand, when the evaluation color temperature reaches the maximum color temperature, in this case, 5000 ° K., a series of operation flow ends (ST-94).

As described above, according to the present invention, a fixed subject region is provided in the image display region, a specific integration region is set in the fixed subject region, color temperature data with respect to the environmental temperature of the integration region is measured in advance, and evaluation data is obtained. Even if the environment changes and the color temperature of the integration region changes, the color temperature is estimated by referring to the stored evaluation data, and the optimum R and B gains are selected, so that the accuracy independent of the subject is selected. High white balance control.
Further, according to the present invention, the color temperature can be estimated even from an imaging system without an infrared cut filter from the change in the color of the fixed subject, and highly accurate white balance control independent of the subject can be performed.

It is a figure which shows the block structure of an imaging device. It is a figure which shows the block configuration of the signal processing part of an imaging device. It is a figure which shows the block configuration of the controller part of a signal processing part. It is a color coordinate diagram for demonstrating color temperature estimation. It is a figure which shows the display image containing a vehicle-mounted camera and a fixed subject. It is a figure which shows the other display image containing a fixed subject. It is a flowchart for demonstrating color signal integration operation | movement and coordinate transformation. It is the figure which displayed the flowchart and evaluation color temperature which calculate evaluation data on a color coordinate. It is the figure displayed on the flowchart for demonstrating the color temperature estimation of a display image, and a color coordinate. It is the graph and flowchart which obtain | require each color gain with respect to evaluation color temperature. It is a figure which shows the block configuration of the conventional imaging device.

Explanation of symbols

DESCRIPTION OF SYMBOLS 10,200 ... Solid-state imaging device, 11 ... Lens, 12 ... Image pick-up element (light receiving element), 13 ... Preamplifier, 14 ... A / D (analog / digital) converter, 15, 60 ... Pre-processing part, 16, 201 ... Primary color separator, 17 (17r, 17g, 17b), 202 ... white balance amplifier, 18, 203 ... optical detector circuit, 19, 204 ... controller, 50 ... signal processing unit, 61 ... black detection unit, 62 ... adder, 63 ... Digital gain adjustment unit, 64 ... Shading correction unit, 65 ... Delay line / defect correction unit, 70 ... Luminance signal processing unit, 71 ... Luminance signal low pass filter (YLPF), 72 ... Vertical / horizontal contour correction unit, 73 ... Contour correction adder (Mix), 74, 85... Gamma (γ) correction unit, 75... Image effect processing unit, 76. DESCRIPTION OF SYMBOLS 0 ... Color signal processing part, 86 ... RG / BG conversion part, 87 ... Cr / Cb (color difference signal) generation part, 88 ... Chroma suppress, Hue / Gain adjustment part, 101 ... Fixed subject frame integration part , 102 ... coordinate axis conversion unit, 103 ... color temperature comparison unit, 104 ... evaluation data storage device.

Claims (12)

A primary color separator for separating a color signal from a pixel signal;
An integration circuit for integrating the separated color signal for a predetermined period;
A coordinate axis converter that converts the integrated color signal into color coordinate data;
A storage circuit for measuring the color signal in advance and storing the measured color temperature of the color signal as evaluation reference data;
A color temperature comparison unit that compares the evaluation reference data with the color temperature data newly output from the coordinate axis conversion unit;
The specified region of the image is selected as the integration region, the integration circuit is controlled to integrate the color signal in the integration region, and the specific region of the image is selected according to the comparison data output from the color temperature comparison unit An image processing apparatus comprising: a control unit that estimates a color temperature of the image. The image processing apparatus according to claim 1, wherein the control unit sets the color temperature data output from the coordinate axis conversion unit to a color temperature of evaluation reference data close to the color temperature. The image processing apparatus according to claim 1, wherein the control unit measures a horizontal / vertical address of a captured image and specifies the integration region in the subject. The image processing apparatus according to claim 3, wherein the integration region is in an image of a known subject of a captured image. The image processing apparatus according to claim 4, wherein the subject is a fixed subject of a captured image. The image processing apparatus includes a white balance circuit, and the white balance circuit sets gains of R (red) and B (blue) signals according to an estimated temperature with reference to a G (green) signal. The image processing apparatus according to 1. An image sensor;
A primary color separator that separates pixel signals output from the image sensor into color signals;
An integration circuit for integrating the separated color signal for a predetermined period;
A coordinate axis converter that converts the integrated color signal into color coordinate data;
A storage circuit that measures the pixel signal of the integration region in advance and stores the measured color temperature data as evaluation reference data;
A color temperature comparison unit that compares the evaluation reference data with the color temperature data output from the coordinate axis conversion circuit;
The specified region of the image is selected as the integration region, the integration circuit is controlled to integrate the color signal in the integration region, and the specific region of the image is selected according to the comparison data output from the color temperature comparison unit A controller for estimating the color temperature of
An image pickup apparatus comprising: a signal processing circuit that performs white balance adjustment according to the color temperature estimated by the control unit, and generates an image signal by performing signal processing on the adjusted color signal and the luminance signal of the pixel signal. The imaging apparatus according to claim 7, wherein the control unit sets the color temperature data output from the coordinate axis conversion unit to a color temperature of evaluation reference data close to the color temperature. The imaging apparatus according to claim 7, wherein the control circuit measures a horizontal / vertical address of a captured image and specifies the integration region in the subject. The imaging apparatus according to claim 9, wherein the integration region is in an image of a known subject of a captured image. The imaging device according to claim 10, wherein the subject is a fixed subject of a captured image. 8. The imaging apparatus includes a white balance circuit, and the white balance circuit sets gains of R (red) and B (blue) signals according to an estimated temperature with reference to a G (green) signal. The imaging device described.

Images