JP2001320717A - Imaging unit, signal processor and signal conditioning method - Google Patents

Abstract

PROBLEM TO BE SOLVED: To calculate a coefficient for an optimum white balance correction to an object having all color temperatures. SOLUTION: In a plurality of imaging units using a imaging element provided with a color filter, a signal processing method to adjust so that the imaging element output of a second imaging unit other than a first imaging unit is combined with the imaging element output of the first imaging unit serving as a basis among the plurality of imaging units has a first photographing stage to photograph an object having a basic color by using a first light source having a first color temperature (S11), a second photographing stage to photograph the object by using a second light source having a second color temperature (S11), and an adjustment stage to adjust signal processing data based on the imaging element output of the plurality of imaging units obtained at the first and second photographing stages so that the imaging element output of the second imaging unit is combined with the first imaging element output (S12-S18).

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to an image pickup apparatus, a signal processing apparatus, and a signal adjustment method, and more particularly to a method of adjusting a variation in output signals from image pickup elements in a plurality of image pickup apparatuses to a predetermined output signal. The present invention relates to an adjustable imaging device, a signal processing device, and a signal adjustment method.

[0002]

2. Description of the Related Art FIG. 8 is a block diagram showing the structure of a conventional single-panel digital camera. CCD image sensor 501
Is subjected to a correction gain by the spectral sensitivity adjustment circuit 522 for each color corresponding to each color filter. Thereafter, the white gain is adjusted by the white balance circuit 502 and sent to the luminance notch circuit 510. The luminance notch circuit 510 performs a process of reducing the signal gain of the frequency near the Nyquist of the color in the vertical direction by using a vertical low-pass filter (VLPF). Similarly, in the horizontal direction, the gain is reduced by a horizontal low-pass filter (HLPF). Hereinafter, this filter is called a luminance notch filter. Thereafter, the horizontal band-pass filter (HBPF) circuit 511 and the vertical band-pass filter (VBPF) circuit 514 raise a frequency slightly lower than the Nyquist frequency weakened by the notch filter.

After that, both horizontal and vertical PP (Aperture
Peak) Gain circuits 512 and 515 adjust the amplitude, and base clip (BC) circuits 513 and 516 cut the small amplitude to remove noise. Then, adder 5
At 17 the horizontal and vertical components are added and the APC (Apertu
re Control) The main gain is applied in the MainGain circuit 518, and the result is added to the baseband signal in the adder 519. Thereafter, gamma conversion is performed by a gamma conversion circuit 520, and a luminance correction (YCOMP) circuit 521 performs luminance signal level correction by color.

As a color signal processing, a color interpolation circuit 50 is used.
3, the color conversion matrix (MTX) circuit 50 is interpolated so that all color pixel values exist for all pixels.
4, the complementary color signals are the luminance signal (Y) and the color difference signal (Cr,
Cb). Then chroma suppression (CSUP: Ch
roma suppress circuit 505 suppresses the color difference gain in the low luminance and high luminance regions, and the band is limited by a chroma low pass filter (CLPF) circuit 506. The band-limited chroma signal is supplied to a gamma conversion circuit 507.
Gamma conversion is performed simultaneously with conversion to RGB signals. The RGB signals after the gamma conversion are converted again into Y, Cr, and Cb signals, and CGainKnee (ChromaGain Knee)
The saturation gain is adjusted by the circuit and the LCMTX (Linear Cl
An ip Matrix) circuit 509 corrects the hue minutely and corrects the hue shift due to the individual difference of the image sensor.

FIG. 9 shows a spectral sensitivity characteristic of an image pickup device having a general-purpose complementary color filter as shown in FIG. 11, for example.
However, even when using the same type of imaging device,
This spectral sensitivity characteristic varies in the process of creating a color filter. In addition, variations in the spectral transmittance of the lens, the optical LPF, and the infrared cut filter are added, and the output characteristics of the image sensor 501 are different from each other. This variation deteriorates the accuracy of the subsequent white balance processing by the white balance circuit 502, and as a result, large variations occur in the color reproduction of the reproduced image.

Therefore, a reference imaging device (reference imaging device)
Is determined, and when the subject of the same light source is photographed under the same conditions, the spectral sensitivity is adjusted so that the output of the image pickup device of the reference image pickup device is matched with the output of the image pickup device of another image pickup device (non-reference image pickup device). The circuit 522 performs color gain adjustment.

[0007]

However, in the conventional method, since the adjustment is performed using one light source, the farther the light source color temperature of the object is from the color temperature of the light source used at the time of the adjustment, the more the color filter becomes. Has a disadvantage that the variation correction error increases. FIG. 10 shows that the wavelength is about 5
FIG. 7 shows an example of relative values of magenta output from a reference imaging device at various color temperatures and magenta output of a non-reference imaging device after adjustment when adjustment is performed using a light source mainly emitting 50 nm light. However, it can be seen that there is a substantial difference near the wavelength portion of 550 nm, but a difference occurs at a portion apart from 550 nm. As described above, there has been a problem that the accuracy of the white balance processing is reduced for a subject having a color temperature distant from the adjustment light source, and color reproduction varies.

SUMMARY OF THE INVENTION The present invention has been made in view of the above problems, and has as its object to calculate an optimal white balance correction coefficient for a subject having any color temperature.

[0009]

In order to achieve the above object, in a plurality of image pickup apparatuses using an image pickup device provided with a color filter, of the plurality of image pickup apparatuses, a first one serving as a reference is used.
The signal adjustment method of the present invention for adjusting the output of the imaging device of the second imaging device other than the first imaging device to match the output of the imaging device of the first imaging device includes the first imaging device, A first photographing step of photographing a subject of a reference color using a first light source having a color temperature, and a second photographing having a second color temperature different from the first color temperature by the plurality of imaging devices. A second photographing step of photographing the subject using a light source, and an image pickup device output of the second image pickup device based on the image pickup device output obtained in the first and second photographing steps. And adjusting the signal processing data to match the output of the image sensor.

[0010] According to another aspect of the present invention, in a plurality of image pickup apparatuses using an image pickup device provided with a color filter,
Among the plurality of imaging devices, a signal adjustment method for adjusting the imaging device output of the second imaging device other than the first imaging device to match the imaging device output of the first imaging device serving as a reference, A first acquisition step of acquiring, by the plurality of imaging devices, an image sensor output obtained by photographing a subject of a reference color using a first light source having a first color temperature; and the plurality of imaging devices. A second acquisition step of acquiring an image sensor output obtained by photographing the subject using a second light source having a second color temperature different from the first color temperature; Based on the image sensor output obtained in the second acquisition step, the image sensor output of the second image pickup device
And adjusting the signal processing data to match the output of the image sensor.

Further, in a plurality of image pickup apparatuses using an image pickup device provided with a color filter, among the plurality of image pickup apparatuses,
The first image pickup device output of the first image pickup device serving as a reference
The signal processing device of the present invention, which adjusts the output of the imaging device of the second imaging device other than the imaging device of the present invention to match, is obtained by photographing the subject of the reference color using at least two light sources having different color temperatures. The signal processing data is adjusted based on the image sensor outputs of the first and second image sensors so that the image sensor output of the second image sensor matches the image sensor output of the first image sensor. It has adjusting means.

The present invention further includes an image pickup device having a color filter, and an adjustment device for adjusting the output of the image pickup device to match the output of the image pickup device of an external reference image pickup device serving as a reference having the image pickup device having the color filter. The image capturing apparatus according to the present invention is configured to capture an image of the reference color object using at least two light sources having different color temperatures. An image pickup apparatus, comprising: an adjusting unit that adjusts signal processing data so that an element output matches an image pickup element output of the reference image pickup apparatus.

According to a preferred aspect of the present invention, the signal adjusting method further includes a step of acquiring signal processing data used in the first imaging device.

Alternatively, the signal processing data used in the first imaging device or the reference imaging device is set in advance.

According to a preferred aspect of the present invention, the adjusting step includes a step of calculating a plurality of correction coefficients based on the output of the image sensor obtained in the first and second photographing steps. Converting the signal processing data used in the first imaging device into the signal processing data used in the second imaging device using the plurality of correction coefficients, Means for calculating a plurality of correction coefficients based on the imaging device or the reference imaging device and the output of the imaging device of the second imaging device or the imaging device;
Using the plurality of correction coefficients, the signal processing data used in the first imaging device or the reference imaging device is stored in the second imaging device.
And means for converting the data into signal processing data used by the imaging device.

Further, according to a preferred aspect of the present invention, the adjusting step includes the step of determining the first and second image-capturing steps based on an image sensor output obtained from the first image-capturing device. A conversion step of converting the signal processing data of the first imaging device into reference data, and the reference data based on an image sensor output obtained from the second imaging device in the first and second imaging steps. Converting the data into signal processing data of a second imaging device, wherein the adjusting unit is configured to output the first imaging device or the first imaging device based on an imaging element output of the first imaging device or the reference imaging device. Means for converting the signal processing data of the reference imaging device into reference data, and, based on the output of the second imaging device or the imaging device of the imaging device, converting the reference data to a signal of the second imaging device or the imaging device. place And means for converting the data.

Preferably, the subject is a white subject.

Preferably, the signal processing data is data indicating white, and in the adjusting step, the data indicating white in the first imaging device is converted into data indicating white in the second imaging device. The adjustment unit converts data indicating white in the first imaging device or the reference imaging device into data indicating white in the second imaging device or the imaging device.

According to a preferred aspect of the present invention, the signal adjusting method further includes a step of calculating a parameter used in white balance correction based on the data converted in the adjusting step, The processing device and the imaging device further include a unit that calculates a parameter used in white balance correction based on the data converted by the adjustment unit.

Further, according to a preferred aspect of the present invention, the data indicating white is a parameter for specifying a plurality of straight lines indicating white, and more specifically, the data indicating white is a parameter of a color filter. These are parameters for specifying a plurality of straight lines indicating white for each color.

Further, the signal adjusting method further includes a storing step of storing the signal processing data adjusted in the adjusting step in the second imaging device, wherein the signal processing device and the imaging device are controlled by the adjusting means. There is further provided storage means for storing the adjusted signal processing data in the second imaging device or the imaging device.

According to the above configuration, in order to adjust the spectral sensitivity variation of the image sensor based on the output of the image sensor obtained using two types of light sources having different color temperatures, the conventional adjustment using one type of light source is performed. Compared with the method, the correction accuracy is greatly improved.

[0023]

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS Preferred embodiments of the present invention will be described below in detail with reference to the accompanying drawings.

<First Embodiment> In an embodiment of the present invention, a reference imaging device (hereinafter, referred to as a "reference imaging device") is set, and imaging devices other than the reference imaging device (hereinafter, referred to as "reference imaging device"). A method of adjusting the output of the image sensor of the “non-reference imaging device”) to match the reference imaging device will be described.

In particular, in the present embodiment, two light sources are used, and using a spectral sensitivity correction coefficient obtained under each light source, an operation parameter group (white detection axis and white detection axis) of the white balance (WB) of the non-reference imaging device is used. The method for calculating the control value axis will be described.

In the present embodiment, the white detection axis is the axis at the center of the white area where the color signal data output from the image sensor is determined to be "white". In the present embodiment, in order to output the color signal data determined to be “white” in the white detection area as an appropriate white balance control value, a control value area having a narrower range is set in the white detection area. Is set. That is, when the color signal data determined to be “white” exists outside the control value area, the white balance control value is output after being limited to the coordinates of the control value area near the color signal data. Become. The control ground axis is an axis that is the center of the control value area. In this embodiment, the white detection axis will be described. However, the object of the present invention can be achieved by the same method for the control value axis.

FIG. 5 is a block diagram showing a configuration example of the imaging system according to the embodiment of the present invention.

In FIG. 1, reference numeral 1 denotes a non-reference imaging device, 2 denotes a reference imaging device, and 3 denotes a signal for calculating adjustment data for adjusting the output of the imaging device of the non-reference imaging device 1 to the output of the reference imaging device 2. The processing device 4 is an A light source having a first color temperature, 5 is a B light source having a second color temperature different from the first color temperature, and 6 is a white object such as a white reference plate.

FIG. 6 is a block diagram showing the configuration of the non-reference imaging device 1 and the reference imaging device 2. The difference from FIG. 8 showing a conventional device is that the spectral sensitivity correction circuit 522 is not required. .

The signal processing device 3 includes a ROM 207, a RAM
208, a central processing unit 209, a disk device 210, a bus 211, an I / O port 212, an I / F 213, and an external storage device 214. The ROM 207 holds a program for implementing the adjustment procedure shown in FIGS. 3 and 4 described below. The RAM 207 provides a storage area necessary for the operation of the program. The central processing unit 209 has a ROM 20
The processing is performed in accordance with the program stored in.
The bus 211 connects the above-described components, and enables transmission and reception of data between the components. Reference numeral 212 denotes an I / O port for communicating with the camera system. I / F213
Is connected to an input unit 220 such as a mouse or a keyboard. The external storage device 214 is a floppy disk or C
This is for driving an external storage medium such as a D-ROM. As described above, when the control program is stored in the external storage medium instead of being stored in the ROM 207 in advance, it is read and downloaded.

The adjustment data obtained by the signal processing device 3 by a method described later is sent to the non-reference imaging device 1 and set.

In FIG. 5, the signal processing device 3 is independent of the non-reference imaging device 1 and the reference imaging device 2, but the signal processing device 3 is included in the non-reference imaging device 1 or the reference imaging device 2. You may comprise.

First, how to calculate the parameters of the white balance correction in the reference imaging device 2 will be described.

FIG. 1 is a graph showing a white detection area in the reference imaging device 2. To find this white detection area,
First, a white object 6 as a reference is photographed using light sources having different color temperatures from a light source having a low color temperature to a light source having a high color temperature, and the color evaluation values Ex and Ey are calculated based on the output from the light source according to the equation (1). Calculated based on Ex = (Ye-Cy) / Yl, Ey = (Mg-G) / Yl, where Yl = (Mg + G + Cy + Ye) / 4… (1)

The plot of Ex obtained for each light source on the X-axis and the plot of Ey on the Y-axis are connected by a straight line, or the plotted points are approximated using a plurality of straight lines. A white detection axis indicating the evaluation value is obtained. The white detection axis having a certain width in the Y-axis direction is defined as a white detection area. Next, the subject is photographed. When a complementary color filter as shown in FIG. 11 is used, for example, Mg, G,
Using the color signal values for each of the Cy and Ye array patterns, and using the result of averaging the signal values for each color in each of the small regions when the screen is divided into a plurality of arbitrary small regions,
The color evaluation values Ex and Ey are calculated by the equation (1). Then, it is determined whether or not the calculated color evaluation value is included in the white detection area shown in FIG. 1, and when it is determined that the calculated color evaluation value is included, the signal value of the area or the signal value obtained by averaging is integrated for each color. To go. For the entire captured image, the signal values of the regions included in the white detection region are sequentially integrated for each color, and the integrated values are averaged by the number of regions determined to be included in the white detection region.
The value obtained in this manner is used as a coefficient to perform white balance correction.

However, as described above, the output from the image pickup device differs for each image pickup device due to the difference in the spectral transmittance of the color filter and the like.
x and Ey differ for each imaging device even when the same subject is photographed using the same light source. That is, the parameters used for setting the white detection axis and the like differ for each imaging device.

Therefore, in the present embodiment, a process of converting the white detection axis of the reference imaging device 2 into a white detection axis for the non-reference imaging device 2 using a coefficient for correcting the spectral sensitivity distribution is performed.

Hereinafter, this conversion method will be described with reference to FIGS. FIG. 3 is a flowchart for explaining the white detection axis conversion processing in the first embodiment, and FIG. 4 is a flowchart for explaining the processing of step S14 in FIG.

(1) Calculation of spectral sensitivity correction coefficient group

First, in step S11 in FIG. 3, the reference light source 4 and the non-reference
Then, the white subject 6 is photographed under the B light source 5 and the average pixel value of the output from the image sensor at the center of the screen is calculated (step S12). Thereafter, in step S13, the pixel value of the reference imaging device 2 = the pixel value of the non-reference imaging device 1 ×
Correction factor

A correction coefficient that satisfies is calculated. That is, the output pixel value of a certain imaging device is multiplied by a correction coefficient to make the same value as the output value of the reference imaging device 2, and a coefficient for adjusting the white level to the reference imaging device 2 is calculated. Specifically, when a white image is captured by the reference imaging device 2 using the A light source 4,
The average pixel value of each color of the filter, that is, magenta, green, cyan, and yellow is calculated as refA_Mg, refA_G, refA_Cy, refA_Ye (2)

In the case where white light is photographed by the reference image pickup device 2 using the B light source 5, the average value of the pixels of each filter, that is, magenta, green, cyan, and yellow, is calculated as refB_Mg, refB_G, refB_Cy, refB_Ye. (3) In the case where white light is captured by the non-reference imaging device 1 using the A light source 4, the pixel average value of each color of the filter, that is, magenta, green, cyan, and yellow, is represented by stA_Mg, stA_G, stA_Cy, stA_Ye. .. (4), and when the non-reference imaging device 1 captures white light using the B light source 5, the average pixel value of each of the filters, that is, magenta, green, cyan, and yellow, is represented by stB_Mg, stB_G, stB_Cy, stB_Ye (5) When the A light source 4 is used, the spectral sensitivity correction coefficient for each color, kA_Mg, kA_G, kA_Cy, kA_Y
From Equations (2) and (4), e is given by: kA_Mg = refA_Mg / stA_Mg, kA_G = refA_G / stA_G, kA_Cy = refA_Cy / stA_Cy, kA_Ye = refA_Ye / stA_Ye For each color light source 5, The spectral sensitivity correction coefficient of k
From the equations (3) and (5), B_Mg, kB_G, kB_Cy, and kB_Ye can be expressed as kB_Mg = refB_Mg / stB_Mg, kB_G = refB_G / stB_G, kB_Cy = refB_Cy / stB_Cy, kB_Ye = refB_Ye / st and 7B. it can. (2) Any point P on the white detection axis
Calculation of pixel value of n reference imaging device 2

Next, in step S14, the pixel value at an arbitrary point Pn on the white detection axis as shown in FIG.
Ex including the pixel values when the A light source 4 (point P3) and the B light source 5 (point P0) are used as shown in FIG.
It is calculated by a linear operation expression using a pixel value straight line.

FIG. 2 shows a color evaluation value Ex of the reference image pickup device 2.
FIG. 7 is a diagram showing a relationship between the color evaluation value Ex and each color pixel value corresponding to the color evaluation value Ex. The data of this graph is obtained in advance based on each color pixel value obtained by imaging the white subject 6 when obtaining the white detection axis as shown in FIG.

In the present embodiment, it is assumed that pixel values for four division points are stored in advance as shown in FIG. 2, but the number of division points is not limited to this and may be changed as appropriate. can do.

The operation of step S14 will be described in detail with reference to FIG.

First, a group of straight lines for calculating a pixel value at an arbitrary point Pn is selected. Any point P
If the color evaluation value Ex at n is refPn_Ex,
If refPn_Ex <P1 (Y in step S21)
ES) Select straight line group 1 (step S22), and P1 ≦ r
If efPn_Ex <P2 (N in step S21)
O, YES in step S23) Select line group 2 (step S24), and if P2 ≦ refPn_Ex, select line group 3 (step S25).

The straight line groups 1 to 3 are given by the following equations.

[0049]

[0050]

[0051]

Here, terms including a and b are coefficients for specifying each linear function, x is a value of an arbitrary color evaluation value Ex, and y is a pixel value of each color.

When the group of straight lines to be used is determined in any of steps S22, S24, and S25, r is determined using a linear function representing the group of straight lines (i) determined in step S26.
pixel value re of the reference imaging device 2 corresponding to efPn_Ex
fPn_Mg, refPn_G, refPn_Cy, r
efPn_Ye is calculated using the following equation (9). refPn_Mg = aMg (i) × refPn_Ex + bMg (i), refPn_G = aG (i) × refPn_Ex + bG (i), refPn_Cy = aCy (i) × refPn_Ex + bCy (i), refPn_Ye = aYe (i) × ref + bYe (i),

Note that aMg (i), bMg (i), aG (i), bG (i), aCy (i), bCy (i), aYe (i), and bYe (i) are selected straight line groups. Coefficient of (9)

Is calculated. (3) Calculation of correction coefficient at arbitrary point Pn on white detection axis

Next, in step S15 of FIG. 3, the correction coefficients obtained by using the A light source 4 and the B light source 5 obtained by the equations (6) and (7) correspond to the A light source 4 and the B light source 5, respectively. Color evaluation values refA_Ex and refB_E
x (corresponding to points P0 and P3 respectively in FIG. 2) and a color evaluation value refPn_Ex of an arbitrary point Pn
A correction coefficient at an arbitrary point Pn is calculated from the distance between the points. The correction coefficients to be calculated are kPn_Mg and kPn for magenta, green, cyan and yellow, respectively.
G, kPn_Cy, kPn_Ye, kPn_Mg = (α × kA_Mg + β × kB_Mg) / (α + β), kPn_G = (α × kA_G + β × kB_G) / (α + β), kPn_Cy = (α × kA_Cy_β × Bk ) / (α + β), kPn_Ye = (α × kA_Ye + β × kB_Ye) / (α + β),

Where α = | refB_Ex-refPn_Ex |, β = | refA_Ex-refPn_Ex | (10)

(4) Calculation of the pixel value of the non-reference imaging device 1 at the color evaluation value refPn_Ex corresponding to an arbitrary point Pn on the white detection axis

Next, in step S16, equation (9)
The respective color pixel values at an arbitrary point Pn calculated by the above equation are expressed by the formula (1).
The pixel value of the non-reference imaging device 1 at the color evaluation value refPn_Ex corresponding to the arbitrary point Pn on the white detection axis is calculated by dividing by the correction coefficient corresponding to the arbitrary point Pn calculated in 0). stPn_Mg = refPn_Mg / kPn_Mg, stPn_G = refPn_G / kPn_G, stPn_Cy = refPn_Cy / kPn_Cy, stPn_Ye = refPn_Ye / kPn_Ye… (11)

(5) Calculation of color evaluation value of non-reference imaging device 1 corresponding to arbitrary point Pn

Next, in step S17, the color evaluation values stPn_Ex, stP of the non-reference imaging device 1 at the color evaluation value refPn_Ex corresponding to an arbitrary point Pn.
Calculate n_Ey. stPn_Ex = (stPn_Ye-stPn_Cy) / stPn_Yl stPn_Ey = (stPn_Mg-stPn_G) / stPn_Yl

Here, stEn_Yl = (stPn_Mg + stPn_G + stPn_Cy + stPn_Ye) / 4 (12) The color evaluation value of the non-reference imaging device 1 at the arbitrary point Pn obtained as described above is plotted in step S18. I do. Then, in step S19, it is determined whether the calculation of the color evaluation value has been completed for all the points set in advance or arbitrarily designated by the operator before step S13. If not, the process returns to step S14 to return to step S14. The same calculation is performed for an arbitrary point Pn, and if the calculation is completed, the process ends.

As described above, the white detection axis of the reference imaging device 2 is converted to the white detection axis of the non-reference imaging device 1. Then, similarly to the case of the reference imaging device 2, the obtained white detection axis of the non-reference imaging device 1 having a certain width in the Y-axis direction is defined as a white detection area of the non-reference imaging device 1, Obtain parameters used for balance correction.

As described above, according to the first embodiment, in order to adjust the spectral sensitivity variation of the image sensor based on the output of the image sensor obtained by using two types of light sources having different color temperatures, a conventional one is used. The correction accuracy is greatly improved as compared with the adjustment method using different types of light sources.

<Second Embodiment> Next, a method of converting a group of white balance parameters according to a second embodiment of the present invention will be described. The imaging system used in the second embodiment has the same configuration as that described in the first embodiment, and a description thereof will not be repeated.

First, a white subject 6 is photographed under the A light source 4 and the B light source 5 by the reference imaging device 2. Then, for each color of the filter, that is, the pixel average value of magenta, green, cyan, and yellow is calculated as in the first embodiment by refA_M
g, refA_G, refA_Cy, refA_Ye, refB_Mg, refB_G, re
Color evaluation values refA_Ex, refA_E when fB_Cy, refB_Ye
y, refB_Ex, and refB_Ey are calculated by the following equation (13). That is, refA_Ex = (refA_Ye-refA_Cy) / refAYi refA_Ey = (refA_Mg-refA_G) / refAYi

Where refAYi = (refA_Mg + refA_G + refA_Cy
+ refA_Ye) / 4 refB_Ex = (refB_Ye- refB_Cy) / refBYi refB_Ey = (refB_Mg-refB_G) / refBYi

However, refBYi = (refB_Mg + refB_G + refB_Cy + refB_Ye) / 4 (13)

Next, the color evaluation values Ex and Ey are converted into normalized axes Wx and Wy. Here, as shown in FIG. 7, the color evaluation values refB_Ex and refB_
When Ey is 0 and the A light source 4 is used, axis conversion is performed such that the color evaluation values refA_Ex and refA_Ey of b become 1024. In the axis conversion, the color evaluation values of the standard imaging device 2 when the light source n of an arbitrary color temperature is used are ref (n) _Ex and ref (n) _Ey, and the axis of the light source f (n) of an arbitrary color temperature is used. Convert color evaluation value to Wx
(n), Wy (n), can be realized by the following equation (Wx (n) = 1024 × (ref (n) _Ex-refB_Ex) / (refA_Ex-refB_Ex) Wy (n) = 1024 × (ref (n) _Ey-refB_Ey) / (refA_Ey-refB_Ey) ... (14)

Here, in the second embodiment, the parameter group (Ex (n), Ey (n)) related to the white detection axis of the white balance in the reference imaging device 2 is represented by (Wx
(N), Wy (n))

The color evaluation values Ex and Ey for the reference imaging device 2
Can be obtained by performing an inverse calculation on the above equation (14). That is, ref (n) _Ex = Wx (n) × (refA_Ex-refB_Ex) / 1024 + refB_Ex ref (n) _Ey = Wy (n) × (refA_Ey-refB_Ey) / 1024 + refB_Ey

Next, a white object 6 is photographed under the A light source 4 and the B light source 5 by the non-reference
The color evaluation value st of the non-reference imaging device 1 is calculated by the same calculation as
Non-reference imaging device 1 by A_Ex, stA_Ey, stB_Ex, stB_Ey
Is calculated. Then, the color evaluation values refA_Ex, refA_Ey, and refB obtained under the A light source 4 and the B light source 5 by the reference imaging device 2 described above.
_Ex, refB_Ey, instead of values obtained by non-reference imaging device 1 stA_Ex, stA_Ey, stB_E
By using x and stB_Ey, a group of white balance parameters of the non-reference imaging device 1 can be calculated. That is, st (n) _Ex = Wx (n) × (stA_Ex-stB_Ex) / 1024 + stB_Ex st (n) _Ey = Wy (n) × (stA_Ey-stB_Ey) / 1024 + stB_Ey

FIG. 7 shows a color evaluation value r obtained by the above conversion method.
It is a conceptual diagram of conversion of ef (n) _Ex into a color evaluation value st (n) _Ex.

As described above, according to the second embodiment,
The pixel values obtained under the A light source and the B light source by the reference imaging device and the pixel values of the A light source and the B light source by the non-reference imaging device are converted into white evaluation values Ex and Ey. Using the white balance axis setting parameter group set by the apparatus, the white balance axis setting parameter group of the non-reference imaging apparatus can be calculated by a simple calculation.

As described above, the white detection axis of the reference imaging device 2 is converted to the white detection axis of the non-reference imaging device 1, and the obtained non-reference imaging device 1 is converted in the same manner as the reference imaging device 2. A white detection axis having a certain width in the Y-axis direction is defined as a white detection area of the non-reference imaging apparatus 1, and parameters used for white balance correction are obtained.

Note that the parameters used for the white balance correction calculated in the first and second embodiments may be calculated in advance at the time of manufacturing the imaging device and recorded in the imaging device, or the imaging device may be powered on. It may be calculated sometimes and recorded in a memory or the like. Further, a configuration may be adopted in which a correction coefficient or the like is recorded in the captured image data and corrected on a personal computer, or the corrected value is additionally recorded in the image data.

Further, in the first and second embodiments, the correction example of the image pickup apparatus provided with the complementary color filter has been described. However, it is obvious that the correction can be performed by the same calculation using the pure color filter.

In the first and second embodiments, two
Although different types of light sources are used, the present invention is not limited to this, and it goes without saying that correction can be performed by similar calculations using three or more types of light sources having different wavelengths.

Further, by converting the white detection axis of the WB circuit 502 to the white detection axis of the non-reference imaging device using the two kinds of adjustment values, the output gain from the WB circuit 502 reduces the variation of the color filter. This becomes the absorbed value, and the optimum WB gain is obtained for the subject having any color temperature. Further, unlike the related art, it is not necessary to separately provide a spectral sensitivity correction circuit 522 for correcting variations in color filters before the WB circuit 502.

[0080]

[Other Embodiments] The present invention may be applied to a system constituted by a plurality of devices (for example, a host computer, an interface device, a camera, etc.) or to an apparatus constituted by one device. .

The procedure relating to the white detection axis conversion processing shown in FIGS. 3 and 4 can be used to manufacture an image pickup device. Therefore, the present invention also includes a manufacturing process of an imaging device having the white detection axis conversion process shown in FIGS. 3 and 4 and an imaging device manufactured by the manufacturing process.

Further, an object of the present invention is to supply a storage medium (or a recording medium) recording software program codes for realizing the functions of the above-described embodiments to a system or an apparatus, and to provide a computer (a computer) of the system or the apparatus. Or a CPU or MPU) reads out and executes the program code stored in the storage medium,
Needless to say, this is achieved. In this case, the program code itself read from the storage medium implements the functions of the above-described embodiment, and the storage medium storing the program code constitutes the present invention.
In addition, by the computer executing the readout program code, not only the functions of the above-described embodiments are realized, but also based on the instructions of the program code,
The operating system (OS) running on the computer performs part or all of the actual processing,
It goes without saying that a case where the function of the above-described embodiment is realized by the processing is also included.

Further, after the program code read from the storage medium is written into a memory provided in a function expansion card inserted into the computer or a function expansion unit connected to the computer, the program code is read based on the instruction of the program code. , The CPU provided in the function expansion card or the function expansion unit performs part or all of the actual processing,
It goes without saying that a case where the function of the above-described embodiment is realized by the processing is also included.

When the present invention is applied to the storage medium, the storage medium stores program codes corresponding to the flowcharts shown in FIGS. 3 and 4 described above.

[0085]

As described above, according to the present invention, in order to adjust the dispersion of the spectral sensitivity of the image sensor based on the output of the image sensor obtained using two types of light sources having different color temperatures, a conventional one type Compared with the adjustment method using the light source, the correction accuracy is greatly improved, and the optimum white balance correction coefficient can be calculated for the subject having any color temperature.

[Brief description of the drawings]

FIG. 1 is a diagram illustrating a white detection area in a reference imaging apparatus.

FIG. 2 is a diagram illustrating a relationship between a color evaluation value and each color pixel value according to the first embodiment of the present invention.

FIG. 3 is a flowchart illustrating a white detection axis conversion process according to the first embodiment of the present invention.

FIG. 4 is a flowchart illustrating a process for calculating each color pixel value at an arbitrary point performed in the process of FIG. 3;

FIG. 5 is a block diagram illustrating a configuration example of an imaging system according to the first embodiment of the present invention.

FIG. 6 is a block diagram illustrating a configuration of an imaging device according to the first embodiment of the present invention.

FIG. 7 is a conceptual diagram of a conversion method according to a second embodiment of the present invention.

FIG. 8 is a block diagram illustrating a configuration of a conventional digital camera.

FIG. 9 is a diagram illustrating an example of spectral sensitivity characteristics of a complementary color sensor.

FIG. 10 is a diagram showing a shift in spectral sensitivity in a conventional adjustment method.

FIG. 11 is a diagram illustrating a configuration example of a complementary color filter.

[Explanation of symbols]

 Reference Signs List 501 Image sensor 502 White balance circuit 503 Color interpolation circuit 504 Color conversion matrix circuit 505 Chroma suppression circuit 506 Chroma low pass filter circuit 507,520 Gamma conversion circuit 508 CGainKnee circuit 509 LCMTX circuit 510 Luminance notch circuit 511 Horizontal band pass filter circuit 512,515 PPGain circuit 513, 516 BC circuit 514 Vertical band-pass filter circuit 517, 519 Adder 518 APCMainGain circuit 521 Brightness correction circuit 522 Spectral sensitivity correction circuit

──────────────────────────────────────────────────の Continued on the front page (51) Int.Cl. ⁷ Identification symbol FI Theme coat ゛ (Reference) H04N 9/73 H04N 1/40 D 5C077 17/02 1/46 A 5C079 (72) Inventor Takaaki Fukui Tokyo 3-30-2 Shimomaruko, Ota-ku F-term (reference) in Canon Inc. DD02 EE07 EE20 GG15 GG17 GG22 GG23 GG32 5C066 AA01 AA15 CA00 CA08 EA14 GA02 GA05 KA12 KE02 KE03 KE05 KE17 KM02 KM06 KM10 5C077 MP08 PP32 PP33 PP37 PP46 PQ12 TT09 5C079 HB03 MA03 MA03 MA02 MA03 NA03

Claims (34)

[Claims] 1. A plurality of image pickup devices using an image pickup device provided with a color filter, wherein, among the plurality of image pickup devices, an image pickup device output of a first image pickup device serving as a reference, except for the first image pickup device. A signal adjustment method for adjusting an image sensor output of a second imaging device to match the first imaging device, wherein the plurality of imaging devices have a first color temperature.
A first photographing step of photographing a subject of a reference color using the light source of the first embodiment, and the plurality of imaging devices using a second light source having a second color temperature different from the first color temperature. A second photographing step of photographing the image, and adjusting an image pickup element output of the second image pickup apparatus to the first image pickup element output based on the image pickup element outputs obtained in the first and second photographing steps. And adjusting the signal processing data as described above. 2. A plurality of image pickup devices using an image pickup device provided with a color filter, among the plurality of image pickup devices, an image pickup device output of a first image pickup device serving as a reference, except for the first image pickup device. A signal adjustment method for adjusting an image sensor output of a second imaging device to match the first imaging device, wherein the plurality of imaging devices have a first color temperature.
A first acquisition step of acquiring an image sensor output obtained by photographing an object having a reference color using the light source of the first embodiment, and a second color temperature different from the first color temperature by the plurality of imaging devices. A second acquisition step of acquiring an image sensor output obtained by photographing the subject using the second light source having the image sensor output based on the image sensor outputs obtained in the first and second acquisition steps. An adjustment step of adjusting signal processing data so that an output of the image sensor of the second imaging device matches the output of the first image sensor. 3. The signal adjustment method according to claim 1, further comprising a step of acquiring signal processing data used in the first imaging device. 4. The adjusting step includes: calculating a plurality of correction coefficients based on an image sensor output obtained in the first and second photographing steps; and using the plurality of correction coefficients, Converting the signal processing data used in the first imaging device into signal processing data used in the second imaging device. 4. The signal adjustment method according to claim 1, further comprising: 5. The method according to claim 1, wherein the adjusting step includes the step of: converting signal processing data of the first imaging device into reference data based on an image sensor output obtained from the first imaging device in the first and second imaging processes. Converting the reference data into signal processing data of the second imaging device based on an image sensor output obtained from the second imaging device in the first and second imaging processes. 4. The signal adjusting method according to claim 3, further comprising the step of: 6. The signal adjustment method according to claim 1, wherein the subject is a white subject. 7. The signal processing data is data indicating white, and in the adjusting step, the data indicating white in the first imaging device is converted into data indicating white in the second imaging device. The signal adjustment method according to claim 1, wherein: 8. The signal adjusting method according to claim 7, further comprising a step of calculating a parameter used in white balance correction based on the data converted in the adjusting step. 9. The signal adjustment method according to claim 7, wherein the data indicating white is a parameter for specifying a plurality of straight lines indicating white. 10. The signal adjustment method according to claim 9, wherein the data indicating white is a parameter for specifying a plurality of straight lines indicating white for each color of the color filter. 11. The signal adjustment method according to claim 1, further comprising a storage step of storing the signal processing data adjusted in the adjustment step in the second imaging device. 12. In a plurality of image pickup devices using an image pickup device provided with a color filter, among the plurality of image pickup devices, an image pickup device output of a first image pickup device serving as a reference, except for the first image pickup device. A signal processing device that adjusts an image sensor output of the second imaging device so as to match the first and second images obtained by photographing a subject of a reference color using at least two light sources having different color temperatures. Adjusting means for adjusting signal processing data based on an image sensor output of the second image sensor so as to match an image sensor output of the second image sensor with an image sensor output of the first image sensor; A signal processing device characterized by the above-mentioned. 13. The signal processing device according to claim 12, wherein the signal processing data used in the first imaging device is set in advance. 14. The image processing apparatus according to claim 1, wherein the adjusting unit calculates a plurality of correction coefficients based on the output of the image sensor of the first imaging device and the second imaging device; 14. The signal processing apparatus according to claim 12, further comprising means for converting signal processing data used in the first imaging apparatus into signal processing data used in the second imaging apparatus. 15. The adjustment unit, comprising: a unit configured to convert signal processing data of the first imaging device into reference data based on an output of an imaging device of the first imaging device; 14. The signal processing device according to claim 12, further comprising: a unit configured to convert the reference data into signal processing data of the second imaging device based on an image sensor output. 16. The signal processing device according to claim 12, wherein the subject is a white subject. 17. The signal processing data is data indicating white, and the adjusting means converts data indicating white in the first imaging device into data indicating white in the second imaging device. 13. The method according to claim 12, wherein
17. The signal processing device according to any one of claims 16 to 16. 18. The apparatus according to claim 1, further comprising: means for calculating a parameter used in white balance correction based on the data converted by said adjusting means.
8. The signal processing device according to 7. 19. The signal processing apparatus according to claim 17, wherein the data indicating white is a parameter for specifying a plurality of straight lines indicating white. 20. The signal processing apparatus according to claim 19, wherein the data indicating white is a parameter for specifying a plurality of straight lines indicating white for each color of a color filter. 21. The signal processing apparatus according to claim 12, further comprising a storage unit that stores the signal processing data adjusted by the adjustment unit in the second imaging device. 22. An imaging device having a color filter,
An image pickup device that adjusts an output of the image pickup device to match an image pickup device output of an external reference image pickup device serving as a reference having an image pickup device provided with a color filter, using at least two light sources having different color temperatures. Based on the imaging device and the imaging device output of the reference imaging device, each of which is obtained by photographing the subject of the reference color, a signal is output to match the imaging device output of the imaging device with the imaging device output of the reference imaging device. An imaging apparatus comprising an adjustment unit for adjusting processing data. 23. The signal processing data used in the reference imaging device is set in advance.
3. The imaging device according to 2. 24. The adjustment unit, comprising: a unit configured to calculate a plurality of correction coefficients based on the reference imaging device and an output of an imaging device of the imaging device; Means for converting signal processing data to be used into signal processing data to be used in the imaging apparatus.
The imaging device according to 2 or 23. 25. The adjustment unit, comprising: a unit configured to convert signal processing data of the reference imaging device into reference data based on an imaging device output of the reference imaging device; and 24. A signal processing apparatus according to claim 22, further comprising means for converting the reference data into signal processing data of the imaging device. 26. The imaging apparatus according to claim 22, wherein the subject is a white subject. 27. The signal processing data is data indicating white, and the adjusting means converts data indicating white in the reference image pickup device into data indicating white in the image pickup device. Items 22 to 26
The imaging device according to any one of the above. 28. The apparatus according to claim 2, further comprising: means for calculating a parameter used in white balance correction based on the data converted by said adjusting means.
8. The imaging device according to 7. 29. The imaging apparatus according to claim 27, wherein the data indicating white is a parameter for specifying a plurality of straight lines indicating white. 30. The imaging apparatus according to claim 29, wherein the data indicating white is a parameter for specifying a plurality of straight lines indicating white for each color of a color filter. 31. The imaging apparatus according to claim 22, further comprising a storage unit configured to store the signal processing data adjusted by the adjustment unit. 32. A storage medium storing a program code for implementing the signal adjustment method according to claim 2. 33. A method for manufacturing an imaging device using the signal adjustment method according to claim 1. 34. An imaging device manufactured by using the manufacturing method according to claim 33.