JP2001320591A - Color balance adjustment method and device, and recording medium - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide a color balance adjustment device that can properly adjust color balance of an image even when a parameter for color balance adjustment is more or less deviated. SOLUTION: A database 9 stores contents of a matrix A and a matrix B whose off-diagonal terms also have a value not 0 on the basis of the color adaptation prediction expression by von Kries. A read means 3 reads image data S0 and an LMS conversion means 4 converts the data into LMS 3-stimulus values. First and second color balance adjustment processing means 5, 6 receive the contents of the matrices A, B from the database 9 on the basis of information of a photographing light source received from an input means 1. The 1 st and 2nd color balance adjustment processing means 5, 6 convert the LMS 3-stimulus values on the basis of the contents of the matrices A, B, and a weight summing means 7 applies weight summing to the converted 3-stimulus values with a prescribed ratio α to obtain summed 3-stimulus values. An RGB conversion means converts the converted 3-stimulus values into an RGB color space, from which processed image data S3 can be obtained.

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a color balance adjusting method and apparatus for adjusting the color balance of an image, and a computer-readable recording medium storing a program for causing a computer to execute the color balance adjusting method. .

[0002]

2. Description of the Related Art In a digital electronic still camera (hereinafter referred to as a digital camera), an image obtained by photographing is stored as digital image data in a recording medium such as an internal memory or an IC card provided inside the digital camera. Based on the obtained digital image data, an image obtained by shooting as a hard copy such as a print or as a soft copy on a display can be reproduced. As described above, when reproducing an image obtained by a digital camera, it is expected that the image has the same high-quality image quality as a photograph printed from a negative film.

On the other hand, human visual perception is called chromatic adaptation. Even when a white paper is viewed under daylight or under fluorescent light or tungsten light having a different spectral energy distribution, it is perceived as bluish or reddish. Instead, a mechanism for basically recognizing as white works. In consideration of such chromatic adaptation, even if the photographing light source is different such as tungsten light, fluorescent light or outdoor daylight, an image representing the photographed image can be obtained so that an image independent of the light source color can be obtained. A function (AWB) that automatically performs white balance adjustment processing (AWB processing) according to the shooting light source on data
Digital camera having a function).

As such a white balance adjustment method, a method according to the von Kries color adaptation prediction formula is used. This method uses the light source 1 and the light source 2 when observing an image captured by the light source 1 under the light source 2.
The tristimulus value based on the physiological primary color of the white point of L _1w ,
Assuming that M _1w and S _1w and L _2w , M _2w and S _2w , the LMS tristimulus value (L, M, S) ₁ of the image under the light source 1 before white balance adjustment is _calculated by the following equation (1). As shown in FIG. 5, the image under the light source 2 after white balance adjustment is converted into LMS tristimulus values (L, M, S) ₂ .

[0005]

(Equation 1) However, the method according to the von Kries color adaptation prediction formula can appropriately adjust the color of a face etc. other than white in the case of a daylight light source, for example, under a special light source such as a fluorescent lamp or tungsten light. There is a problem that the color of the face etc. other than white is converted into an unnatural color with respect to the image taken in.

For this reason, instead of using a matrix having only non-diagonal terms having non-zero values as shown in the above equation (1), a matrix having non-diagonal terms having non-zero values is used for conversion of physiological primary colors. A method of adjusting the color balance of an image by using the method has also been proposed. According to this method, even if an image is captured under a special light source, by generating a matrix in consideration of the light source, it is possible to perform a color balance adjustment process to obtain an appropriate color. it can.

[0007]

In the method according to the von Kries color adaptation prediction formula, the matrix has non-zero values only in the diagonal terms, and the components of the tristimulus values L, M, and S are represented by white points. Are multiplied by a constant according to the components L _w , M _w , and S _w of the light source. Therefore, in the color balance adjustment processing, the white point of the light source assumed to be the light source during shooting and the light source Even when the white point is slightly deviated, it is unlikely that the processing will fail and an image having an unnatural color will be obtained. On the other hand, when the conversion is performed using a matrix in which the off-diagonal terms also have non-zero values, the tristimulus values L, M, and S are converted including not only the diagonal terms but also the off-diagonal terms. Therefore, if the white point of the light source assumed as the light source at the time of shooting is different from the white point of the light source at the time of actual shooting, the shift is amplified by the conversion, and the white color of the image obtained by processing is There is a problem that it tastes.

For example, conversion based on the von Kries chromatic adaptation prediction formula is performed by using a conversion parameter consisting of a matrix represented by the following equation (2). It is assumed that the conversion is performed using a conversion parameter including a matrix shown in Expression (3). Then, assuming that L of the white point before the conversion is shifted by 1%, in the case of Expression (2), the color after the conversion is L: 0.99
%, M: 0%, and S: 0%, but the formula (3)
In the case of, the color after conversion is L: 1.22%, M: -0.11
%, S: -0.05%. Therefore, the off-diagonal term is also 0
In a transformation using a matrix with non-
If the conversion parameters are incorrectly set due to a slight shift in the estimation of the white point, the converted white will take on a tint.

[0009]

(Equation 2)

(Equation 3) The present invention has been made in view of the above circumstances, and provides a color balance adjustment method and apparatus capable of appropriately adjusting a color balance even when a conversion parameter is set somewhat incorrectly, and a program for causing a computer to execute the color balance adjustment method. It is an object of the present invention to provide a recorded computer-readable recording medium.

[0010]

According to the color balance adjusting method of the present invention, image data representing an image obtained by photographing a subject or color balance adjusting processing premised on observation under a predetermined light source is performed. In a color balance adjustment method of performing color balance adjustment processing on the applied image data to obtain processed image data, a color balance adjustment processing using a first conversion parameter based on a diagonal matrix, and at least one 0
Obtaining the processed image data by applying a mixing process in which a color balance adjustment process based on a non-diagonal matrix including a non-diagonal term including a non-diagonal term is mixed at a predetermined ratio to the image data. It is characterized by the following.

The "first conversion parameter based on the diagonal matrix" is, for example, a matrix used in the von Kries chromatic adaptation prediction formula described above, which has a value other than 0 only in the diagonal term, and There is a matrix that can match the white point of the image before and after, but in order to match the white point of the image before and after the conversion, a matrix generated based on a matrix having only non-zero values in the diagonal terms If so, a matrix having non-diagonal terms having non-zero values may also be used.

The "second conversion parameter based on an off-diagonal matrix including at least one off-zero off-diagonal term" means that an appropriate color balance adjustment process can be performed in consideration of a light source at the time of photographing. Are matrices that have non-zero values for off-diagonal terms as well.

[0013] In the color balance adjusting method according to the present invention, the mixing is performed by converting the image data using the first conversion parameter.
Is obtained by converting the image data according to the second conversion parameter, second intermediate image data is obtained, and the first and second intermediate image data are weighted and added according to the predetermined ratio. Processing may be performed.

[0014] In the mixing process, the first conversion parameter and the second conversion parameter are weighted and added according to the predetermined ratio to obtain an addition parameter, and the image data is converted by the addition parameter. It may be.

Further, in the mixing process, first intermediate image data is obtained by converting the image data by the first conversion parameter, and the first intermediate image data is converted into the first and second image data. The second intermediate image data is obtained by converting the white image generated based on the conversion parameter by using a third conversion parameter for storing the white image, and the first intermediate image data is converted into the third conversion parameter, the unit parameter, and The third intermediate image data is obtained by converting with a fourth conversion parameter that stores the white color generated based on the above, and the second and third intermediate image data are weighted and added by the predetermined ratio. Is also good.

"Third Conversion Parameter for Preserving White"
Is a conversion parameter that allows color balance adjustment processing to be performed without changing white color before and after conversion. When processing is performed using the first conversion parameter and the third conversion parameter, the second It is generated so that a process corresponding to the process based on the conversion parameter is performed.

A "unit parameter" is a matrix in which the diagonal terms have a value of 1 and the off-diagonal terms have a value of 0.

"Fourth Conversion Parameter for Preserving White"
Is a conversion parameter that allows color balance adjustment processing to be performed before and after conversion without changing the white color. When processing is performed using the first conversion parameter and the fourth conversion parameter, the first It is generated by subtracting the third parameter from the unit parameter so that a process corresponding to the process of subtracting the second conversion parameter from the conversion parameter is performed.

Further, in the mixing process, first image data is obtained by converting the image data using the first conversion parameter, and a white color generated based on the first and second conversion parameters is obtained. And a fourth conversion parameter for storing white generated based on the third conversion parameter and the unit parameter are weighted and added according to the predetermined ratio to obtain an addition parameter. , The first intermediate image data may be converted by the addition parameter.

Further, in the mixing process, the first intermediate image data is converted by converting the image data by a third conversion parameter for storing white generated based on the first and second conversion parameters. And converting the image data with a fourth conversion parameter for storing white generated based on the third conversion parameter and the unit parameter, thereby obtaining second intermediate image data. The weighted addition of the second intermediate image data and the second intermediate image data according to the predetermined ratio may be performed to obtain added image data, and the added image data may be converted by the first conversion parameter.

Further, the mixing process is performed based on a third conversion parameter for storing a white color generated based on the first and second conversion parameters, and the third conversion parameter and a unit parameter. A fourth conversion parameter for storing the generated white color is weighted and added according to the predetermined ratio to obtain an addition parameter, and the image data is converted by the addition parameter to obtain conversion image data. May be converted by the first conversion parameter.

Further, in the color balance adjusting method according to the present invention, the mixing process may be performed in a color space related to a spectral characteristic of a photographing device that has photographed the image.

The "color space related to the spectral characteristics of the imaging device that performs photographing" includes, for example, an RGB color space and a CMY color space.

Further, in the color balance adjusting method according to the present invention, the mixing process may be performed in a color space related to human visual characteristics.

The "color space related to human visual characteristics" is, for example, an LMS color space based on physiological primary colors.

In the color balance adjusting method according to the present invention, the first conversion parameter may be a color balance adjusting process performed based on the diagonal matrix in a color space related to human visual characteristics. As a parameter to be performed in a color space related to the spectral characteristic of the imaging device that has captured the image, and the second conversion parameter is
The color balance adjustment processing performed based on the off-diagonal matrix in a color space related to human visual characteristics may be set as a parameter for performing the color balance related to the spectral characteristics of an imaging device that has captured the image.

The "color balance adjustment processing performed based on the diagonal matrix in a color space related to human visual characteristics" means, for example, an LMS like the von Kries color adaptation prediction equation shown in the above equation (1). This is a process for adjusting the color balance in the color space. Here, when the obtained image data is composed of RGB color signals as in a digital camera, for example, it is necessary to convert the RGB color signals into the LMS color space in order to perform the color balance adjustment processing. Therefore, “parameters for performing color balance adjustment processing performed based on the diagonal matrix in a color space related to human visual characteristics in a color space related to spectral characteristics of an imaging device that has captured the image” Parameters for performing a conversion process from a color space related to the spectral characteristics of the imaging device to a color space related to the human visual characteristics;
This is a parameter in which color space conversion and color balance adjustment processing are performed simultaneously by synthesizing a parameter for performing color balance adjustment processing such as the n Kries color adaptation prediction equation.

[0028] Also, "the color balance adjustment processing performed based on the non-diagonal matrix in a color space related to human visual characteristics is performed in a color space related to the spectral characteristics of a photographing apparatus that has photographed the image. Are parameters for performing a conversion process from the color space related to the spectral characteristics of the imaging device to the color space related to the human visual characteristics, and a matrix such as the above-described non-diagonal term having a non-zero value. This is a parameter in which color space conversion and color balance adjustment processing are performed simultaneously by synthesizing parameters for performing color balance adjustment processing.

Further, in the color balance adjusting method according to the present invention, a white point of a photographing light source at the time of photographing the image data is estimated, and the first and second conversion parameters are generated based on the white point. You may.

Further, in the color balance adjusting method according to the present invention, designation of a photographing light source at the time of photographing the image data is received, and the predetermined ratio is changed according to the designated photographing light source. You may.

Here, the designation of the photographing light source can be accepted, for example, by accepting an input from a keyboard, a key for designating the light source, etc. in an apparatus for implementing the present invention. In addition, information about the photographing light source may be added to the image data at the time of photographing, and the designation of the photographing light source may be received by reading information about the photographing light source added to the image data when the processing parameter is generated.

The color balance adjusting device according to the present invention comprises:
A process of performing color balance adjustment processing on image data representing an image obtained by photographing a subject or image data on which color balance adjustment processing has been performed on the assumption that observation is performed under a predetermined light source. A color balance adjustment device for obtaining finished image data, a color balance adjustment process using a first conversion parameter based on a diagonal matrix, and a second conversion parameter based on an off-diagonal matrix including at least one non-zero off-diagonal term And a mixing processing unit that obtains the processed image data by performing a mixing process of mixing the color balance adjustment process with the image data at a predetermined ratio on the image data.

In the color balance adjusting device according to the present invention, the mixing processing means includes a first conversion means for converting the image data by the first conversion parameter to obtain first intermediate image data. A second conversion unit that obtains second intermediate image data by converting the image data using the second conversion parameter; and weights and adds the first and second intermediate image data according to the predetermined ratio. A weighting and adding unit may be provided.

In addition, the mixing processing means includes weighted addition means for obtaining an addition parameter by weighting and adding the first conversion parameter and the second conversion parameter at the predetermined ratio, and adding the image data to the image data. Conversion means for performing conversion by parameters may be provided.

Further, the mixing processing means converts the image data by the first conversion parameter to obtain first intermediate image data,
A second conversion for obtaining the second intermediate image data by converting the first intermediate image data by a third conversion parameter for storing a white color generated based on the first and second conversion parameters; Means for converting the first intermediate image data by a fourth conversion parameter for storing a white color generated based on the third conversion parameter and the unit parameter to obtain third intermediate image data The image processing apparatus may further include third conversion means, and weighting addition means for weighting and adding the second and third intermediate image data at the predetermined ratio.

Further, the mixing processing means includes: first conversion means for obtaining the first intermediate image data by converting the image data using the first conversion parameter; and the first and second conversion parameters. Weighting the third conversion parameter for storing the white color generated based on the third conversion parameter and the fourth conversion parameter for storing the white color generated based on the third conversion parameter and the unit parameter by the predetermined ratio The image processing apparatus may further include a weighting addition unit that obtains an addition parameter by adding, and a second conversion unit that converts the first intermediate image data using the addition parameter.

Further, the mixing processing means converts the image data by using a third conversion parameter for storing a white color generated based on the first and second conversion parameters, thereby obtaining a first intermediate image. A first conversion unit for obtaining data; and a second intermediate image by converting the image data by a fourth conversion parameter for storing white generated based on the third conversion parameter and the unit parameter. Second conversion means for obtaining data;
Weighted addition means for obtaining added image data by weighting and adding the first and second intermediate image data according to the predetermined ratio; and
And a third conversion unit that performs conversion using the conversion parameter of (3).

Further, the mixing processing means includes a third conversion parameter for storing the white color generated based on the first and second conversion parameters, and a third conversion parameter based on the third conversion parameter and the unit parameter. Weighting and adding means for obtaining an addition parameter by weighting and adding the fourth conversion parameter for storing the generated white color according to the predetermined ratio; and obtaining the converted image data by converting the image data using the addition parameter. 1 conversion means;
Second conversion means for converting the converted image data using the first conversion parameter may be provided.

Further, in the color balance adjusting device according to the present invention, the mixing processing means may be means for performing the mixing processing in a color space related to the spectral characteristics of the photographing apparatus that has photographed the image.

Further, the mixing processing means may be means for performing the mixing processing in a color space related to human visual characteristics.

Furthermore, the first conversion parameter is converted into a color balance adjustment process performed based on the diagonal matrix in a color space related to human visual characteristics, and the spectral characteristics of a photographing device that has photographed the image are converted into a color balance adjustment process. As a parameter to be performed in a related color space, the second conversion parameter is subjected to a color balance adjustment process performed based on the non-diagonal matrix in a color space related to human visual characteristics, and the image is captured. It may be a parameter for performing in a color space related to the spectral characteristics of the imaging device.

Further, in the color balance adjusting device according to the present invention, a white point estimating means for estimating a white point of a photographing light source when the image data is photographed, and the first and the second based on the white point. And a parameter generation unit that generates a conversion parameter.

Further, in the color balance adjusting device according to the present invention, a receiving means for receiving designation of a photographing light source when photographing the image data, and changing the predetermined ratio in accordance with the designated photographing light source. And changing means.

The program for causing a computer to execute the color balance adjustment method according to the present invention may be provided by being recorded on a computer-readable recording medium.

Further, the color balance adjusting device according to the present invention may be provided in an imaging device such as a digital camera.

Further, the color balance adjusting device according to the present invention may be provided in an output device such as a printer or a monitor.

[0047]

According to the present invention, the first diagonal matrix is used.
And a color balance adjustment process based on a non-diagonal matrix including at least one non-zero diagonal term and a color balance adjustment process based on a non-diagonal matrix including at least one non-diagonal matrix. Alternatively, processed image data is obtained by applying the image data to which color balance adjustment processing has been performed on the assumption that observation is performed under a predetermined light source. Here, since the color balance adjustment processing based on the first conversion parameter is based on a diagonal matrix, the white point of the light source assumed to be a light source during shooting in the color balance adjustment processing and the light source during actual shooting Even if the white point of the image slightly deviates, it is unlikely that the processing will fail and an image having an unnatural color will be obtained. Color is unnaturally converted. On the other hand, in the color balance adjustment processing using the second conversion parameter, if the white point of the light source assumed as the light source at the time of shooting is slightly different from the white point of the light source at the time of actual shooting, an image obtained by the processing is obtained. Is unnatural, but it is possible to convert an image obtained under a special light source so that colors other than white are also appropriate colors. Therefore, by performing a mixing process in which these color balance adjustment processes are mixed at a predetermined ratio, the white point of the light source assumed to be the light source during shooting and the white point of the light source during actual shooting slightly shift. Even if it is, or even an image obtained under a special light source, even if the image has been subjected to a color balance adjustment processing that is further premised on observation under a predetermined light source, The color balance adjustment processing can be performed without causing the white color of the image obtained by the processing to take on a tint and generating an unnatural color.

Further, by performing the mixing process in the color space related to the human visual characteristics, the color balance adjustment process can be performed based on the color that a human recognizes when he / she sees the object. A color balance adjustment process that reproduces an image faithful to the image can be performed.

Further, by performing a mixing process in a color space related to the spectral characteristic of the photographing device that has photographed the image, the color balance can be obtained without performing any color conversion on the data acquired by the photographing device such as a digital camera. Since the adjustment process can be performed, the configuration of the device can be simplified, and the process can be performed at high speed.

Further, by changing the predetermined ratio in accordance with the designation of a photographing light source, for example, an image obtained by photographing in daylight or a photographing assumed when the second conversion parameter is generated. For an image obtained by shooting under a light source, the ratio of the mixing process can be changed so that a color balance adjustment process suitable for the light source can be performed, so that a more appropriate color balance adjustment process can be performed.

[0051]

Embodiments of the present invention will be described below with reference to the drawings.

FIG. 1 is a schematic block diagram showing the configuration of the color balance adjusting device according to the first embodiment of the present invention. As shown in FIG. 1, a color balance adjustment device 1 according to the present embodiment includes an RGB color signal R0, G0,
This is for performing color balance adjustment processing on the image data S0 composed of B0 to obtain processed image data S3.
A, reading means 3 such as a card slot for reading image data S0 from A, and color signals R0, G constituting image data S0.
0, B0 is the tristimulus value L of the physiological primary color in the LMS color space
An LMS conversion means 4 for converting the tristimulus values L0, M0, S0, and a first color balance adjustment process on the tristimulus values L0, M0, S0 to obtain processed tristimulus values L1, M1, S1. Color balance adjustment processing means 5 and tristimulus value L
A second color balance adjustment process is performed on 0, M0, and S0 to obtain processed tristimulus values L2, M2, and S2.
Color balance adjustment processing means 6, processed tristimulus values L1, M1, S1 and processed tristimulus values L2, M2, S2
Weighted addition means 7 for obtaining the added tristimulus values L3, M3, S3 by weighting and adding
S3 is converted to a signal in the RGB color space, and the converted signal R
RGB conversion means 8 for obtaining processed image data S3 comprising G3, G3 and B3, and matrices A and B for color balance adjustment processing used in the first and second color balance adjustment processing means 5 and 6, The database 9 includes a database 9 stored according to the type of imaging light source, and input means 10 for inputting the type of imaging light source into the database 9.

The LMS conversion means 4 converts the image data S0 into tristimulus values L0, M0, S0 in the LMS color space as described below. The image data S0 acquired by the digital camera 2 is based on ITU Rec. RG conforming to 709 standard
The LMS conversion means 4 is composed of 8-bit color signals for each of the B colors.
G0 and B0 are converted into a color signal R1 normalized to values of 0 to 1 by the following equations (4) and (5).

[0054]

(Equation 4) Although only the color signal R1 has been described here, the color signals G1 and B1 are obtained by the same calculation as the color signal R1.

Next, the color signals R1, G1, and B1 are converted into CIE1931XYZ tristimulus values by the following equation (6) to obtain tristimulus values X1, Y1, and Z1.

[0056]

(Equation 5) Further, the tristimulus values X1, Y1 and Z1 are converted into tristimulus values based on physiological primary colors by the following equation (7), and these are converted into tristimulus values L0 and M0 based on physiological primary colors of the image represented by the image data S0. , S0.

[0057]

(Equation 6) The first color balance adjustment processing means 5 calculates the tristimulus values L0, M0, and S0 by a matrix A according to the von Kries chromatic adaptation prediction equation shown in the above equation (1) as shown in the following equation (8). After conversion, the color balance adjustment processing is performed to obtain processed tristimulus values L1, M1, and S1.

[0058]

(Equation 7) The second color balance adjustment processing means 6 calculates the tristimulus values L0, M0, S0 as von Kr as shown in the following equation (9).
The non-diagonal terms different from the chromatic adaptation prediction formula of ies are converted by a matrix B having a non-zero value to perform color balance adjustment processing to obtain processed tristimulus values L2, M2, and S2.

[0059]

(Equation 8) The weighting and adding means 7 calculates the processed tristimulus values L1, M1,
S1 and the processed tristimulus values L2, M2, and S2 are weighted and added at a predetermined ratio α as shown in the following equation (10) to obtain added tristimulus values L3, M3, and S3. In this embodiment, α = 0.5.

[0060]

(Equation 9) The RGB conversion means 8 calculates, as shown in the following equation (11),
By solving equations (7) and (6) in reverse, and further solving equations (4) and (5) in reverse, processed color signals R3, G3, and B3 constituting processed image data S3 are obtained. .

[0061]

(Equation 10) The database 9 includes a photographing light source (hereinafter, referred to as a light source 1) when the image data S0 is acquired by the digital camera 2,
A matrix A based on the von Kries chromatic adaptation prediction formula and a matrix B having non-diagonal non-zero values, which are generated according to an observation light source (hereinafter, light source 2) when observing the processed image data S3. Are stored. Hereinafter, a method of creating the matrices A and B will be described. First, a method of creating the matrix A based on the von Kries color adaptation prediction formula will be described.

First, image data S11 is obtained by photographing under the light source 1, and the color signals R11, G11, B11 constituting the image data S11 are obtained by the above equation (4).
Signals R11 'and G1 converted and normalized by (5)
1 'and B11' are obtained. Then, using the total number N of pixels of the image represented by the image data S11, the following expression (1)
The average values Rm11, Gm11, Bm11 of the color signals R11 ', G11', B11 'are calculated according to 2).

[0063]

[Equation 11] Then, the calculated average values Rm11, Gm11, Bm1
1 is converted into CIE1931XYZ tristimulus values by the above equation (6), and these are converted into tristimulus values X _11w , Y _11w , Z of a white point in the image represented by the image data S11.
_11w . Further, tristimulus values X _11w , Y _11w ,
The Z _{11 w} converted to tristimulus values based on the physiological primary colors according to the above equation (7), which tristimulus values _{L 1} 1 w based on the physiological primary colors of the white point of the light source _1, M 11 _w, and _{S 11 w.}

Similarly, photographing under the light source 2
To obtain image data S12.
Stimulus value based on the physiological primary color of the white point of light source 2
L_12w, M _12w, S_12wGet.

[0065] Then, the tristimulus values _L 11 w based on the physiological primary colors of the white point of the light sources _{1,2, M 11 w, S 11w} , L
_{12w, M 12w,} seek matrix shown in the equation (1) from _{S 12w,} which is a matrix A based on the color adaptation prediction equation of von Kries.

Next, a method of creating a matrix B having non-diagonal terms having values other than 0 will be described. It is generally known that the spectral reflectance of an object in the natural world can be represented by three main components. Therefore, the spectral reflectance of the photographed object is determined by three principal components x ₁ (λ), x ₂ (λ), and x ₃
It can be expressed as the following equation (13) as a linear combination of (λ) (λ: spectral frequency).

[0067]

(Equation 12) Here, a ₁ , a ₂ , and a ₃ are weights at the time of weighting and adding each principal component.

Let F (λ) be the spectral radiation distribution of the light source 1, and let L-cone, M-cone and S-cone of the tristimulus values be respectively

(Equation 13) Becomes In addition, although the integration in Equations (14) to (16) is integration in the visible region, integration may be performed in the entire spectral region. Here, the spectral radiation distribution F of the light source 1
(Λ), each principal component x ₁ (λ), x ₂ (λ), x ₃ (λ)
and

[Equation 14] Can be expressed as follows, the spectral reflectance x under the light source 1
The LMS tristimulus value when looking at the object of (λ) is

(Equation 15) Becomes Similarly, the spectral reflectance x (λ) under the light source 2
The LMS tristimulus value when looking at the object of

(Equation 16) Becomes From equations (18) and (19), a ₁ , a ₂ , a
_If you delete ₃ ,

[Equation 17] Next, converting the LMS tristimulus values under the light source 1 from (a _{y 1)} LMS tristimulus values under the light source 2 (the _{y 2)} is the equation (20),

(Equation 18) It can be carried out by the matrix P ₂ P ₁ ^-1 of 3 × 3 having a non-diagonal terms also non-zero value as. Thus, the matrix _P 2 _P ^{1 -1} obtained in this way a matrix B.

Then, a plurality of matrices A and B are calculated for various photographing light sources and observation light sources, and these are stored in the database 9. Examples of the matrices A and B stored in the database 9 are shown below. Here, the shooting light source is a daylight white fluorescent lamp (correlated color temperature 5000K) and a daylight fluorescent lamp (correlated color temperature 6500K), and the processed image data S
3 is assumed to be printed out, and the observation light source is CI
E-D50. In the case where the processed image data S3 is reproduced on a monitor, CIE-D65 is used. In this embodiment, the observation light source is fixed to CIE-D50,
It is assumed that the matrices A and B generated using the observation light source as CIE-D50 are stored in the database 9.

[0070]

[Equation 19] Incidentally, for example, using a number color spectral reflectance data as 1269 colors in Munsell color chips, seek LMS tristimulus values 1 below and the light source 2 below the light source, the tristimulus values y ₁ by a least square method the matrix to convert the tristimulus value y ₂ may be obtained as a matrix B.

The matrix can be determined so that the white color of the light source 1 is converted to the white color of the light source 2. It first uses multiple color spectral reflectance data, such as the Munsell color chips, seek LMS tristimulus values under and the light source 2 below the light source 1, the tristimulus tristimulus values y ₁ by a least square method Request matrix for converting the value y _2. At this time, as shown in the following equation (22), the white color of the light source 1 is converted to the white color of the light source 2
A matrix V ₁ based matrix for obtaining the chromatic adaptation prediction equation of von Kries, perpendicular to the tristimulus values y ₁ matrix V
₁ ′ to obtain matrices V ₁ and V ₁ ′.

[0072] _{y 2 = (V 1 + V} 1 ') y 1 (22) _where, V _{1' y} 1 = 0 In this case, the ^{_{P 2 P 1 -1 = V 1}} + V 1 '(23). By determining the matrix such that the white color of the light source 1 is converted to the white color of the light source 2, the white color under the light source 1 and the white color under the light source 2 can be matched, which is more preferable.

The input means 10 is for inputting information on the photographing light source to the database 9, and comprises a keyboard, keys for designating the light source, and the like. The database 9 reads out the matrices A and B according to the type of the imaging light source input from the input unit 10 and inputs the matrices A and B to the first and second color balance adjusting units 5 and 6. In addition, information regarding the photographing light source may be added to the image data S0 at the time of photographing, and this information may also be read at the time of reading the image data S0 from the memory card 2A, and may be input to the database 9 via the input means 10. . In this case, the matrices A and B are read from the database 9 based on the input information.

Next, the operation of this embodiment will be described. FIG. 2 is a flowchart showing the operation of the first embodiment. First, from the memory card 2A in which image data S0 is stored by photographing in the digital camera 2,
The image data S0 is read by the reading means 3 (step S1). Then, the LMS conversion means 4 converts the color signals R0, G0, B0 constituting the image data S0 into tristimulus values L0, M0, S0 based on physiological primary colors (step S2). On the other hand, the matrices A and B are read from the database 9 and input to the first and second color balance adjusting means 5 and 6 in accordance with the type of imaging light source input from the input means 10 (step S3).

The tristimulus values L0, M0, S0 are input to first and second color balance adjusting means 5, 6, where color balance adjustment processing is performed by matrices A and B, respectively, and the processed tristimulus values are processed. Values L1, M1, S1, L
2, M2 and S2 are obtained (step S4). The processed tristimulus values L1, M1, S1, L2, M2, and S2 are input to the weighting and adding means 7, where weighted addition is performed to obtain added tristimulus values L3, M3, and S3 (step S5). The added tristimulus values L3, M3, and S3 are converted into the RGB color space by the RGB conversion means 8 to obtain processed image data S3 (step S6), and the process ends.

Here, the first color balance adjusting means 5
Since the color balance adjustment processing by the matrix A in the above is based on the von Kries color adaptation prediction formula, the white point of the light source assumed to be a light source at the time of shooting in the color balance adjustment processing and the light source at the time of actual shooting Even when the white point is slightly deviated, it is unlikely that the processing will fail and an image having an unnatural color will be obtained, but the image obtained under a special light source such as a fluorescent light is In some cases, colors other than white are unnaturally converted. On the other hand, in the color balance adjustment processing using the matrix B in the second color balance adjustment means 6, the white point of the light source assumed to be the light source at the time of shooting and the white point of the light source at the time of actual shooting slightly differ. Although the color of the image obtained by the processing becomes unnatural, the image obtained under a special light source can be converted so that colors other than white are also appropriate colors. . Therefore, as in the first embodiment, the tristimulus values L1, M1, and S1 obtained by the first color balance adjusting unit 5 and the tristimulus values L2, M2, and S2 obtained by the second color balance adjusting unit 6 are used.
By weighting and adding S2 with a predetermined ratio α, even when the white point of the light source assumed to be the light source at the time of shooting and the white point of the light source at the time of actual shooting are slightly different, Further, even if the image is obtained under a special light source, the color balance adjustment processing is performed without causing the white color of the image represented by the processed image data S3 to take on a tint and generating an unnatural color. It can be performed.

Further, since the color balance adjustment processing is performed in the LMS color space, which is a color space related to human visual characteristics, it is necessary to perform color balance adjustment based on the color that a human recognizes when looking at an object. As a result, it is possible to perform a color balance adjustment process such that an image faithful to a visual impression is reproduced.

Next, a second embodiment of the present invention will be described. FIG. 3 is a schematic block diagram illustrating a configuration of a color balance adjustment device according to a second embodiment of the present invention.
In the second embodiment, the same components as those in the first embodiment are denoted by the same reference numerals, and the detailed description is omitted. The color balance adjustment device 1 according to the second embodiment includes first and second color balance adjustment units 5,
6 and the weighting and adding means 7,
Weighting and adding means 7 'for obtaining the addition matrix M0 by weighting and adding the matrices A and B read from
The tristimulus values L0, M0,
S0 is subjected to color balance adjustment processing by the addition matrix M0, and processed tristimulus values L4, M4, S4
Is that the color balance adjusting means 11 for obtaining
This is different from the embodiment.

The weighting and adding means 7 'is calculated by the following equation (24).
To obtain an addition matrix M0.

M0 = αA + (1−α) B (24) The color balance adjusting means 11 calculates the tristimulus values L0, M0, S0 as an addition matrix M as shown in the following equation (25).
By performing conversion by 0, color balance adjustment processing is performed to obtain processed tristimulus values L4, M4, and S4.

[0081]

(Equation 20) Note that the matrices A and B in the first embodiment are
If the matrices A and B are the same, the processed tristimulus values L4 and
M4, S4 and the added tristimulus values L3, M3, S3 are the same. Therefore, the processed image data obtained by the RGB conversion means 8 is the same as that of the first embodiment. Therefore, the processed image data of the second embodiment is denoted by the same reference numerals as in the first embodiment. S3
Shall be used.

Next, the operation of the second embodiment will be described. FIG. 4 is a flowchart showing the operation of the second embodiment. First, the image data S0 is read out by the reading means 3 from the memory card 2A in which image data S0 is stored by photographing in the digital camera 2 (step S11). Then, the LMS conversion means 4 converts the color signals R0, G0, B0 constituting the image data S0 into tristimulus values L0, M0, S0 based on physiological primary colors (step S12). On the other hand, the matrices A and B are read from the database 9 in accordance with the type of the imaging light source input from the input means 10 (step S13), and weighted and added by the weighting and adding means 7 'to obtain an addition matrix M0 (step S13). S14). Step S
Steps S13 and S14 may be performed before steps S11 and S12, or they may be performed in parallel.

The addition matrix M0 is input to the color balance adjusting means 11, where the tristimulus values L0, M0, S0
Is subjected to color balance adjustment processing by the addition matrix M0, and processed tristimulus values L4, M4, and S4 are obtained (step S15). Treated tristimulus value L4
M4 and S4 are converted to the RGB color space by the RGB conversion means 8 to obtain processed image data S3 (step S16), and the process is terminated.

As described above, in the second embodiment,
The matrices A and B are weighted and added by a predetermined ratio α to obtain an addition matrix M0.
The color balance adjustment is performed by using a special light source even if the white point of the light source assumed to be the light source at the time of shooting is slightly different from the white point of the light source at the time of actual shooting. Even for the image obtained below, the color balance adjustment processing can be performed without causing the white color of the image represented by the processed image data S3 to take on a tint and generating an unnatural color. .

In the first embodiment, the conversion for normalizing the color signals R0, G0 and B0 according to the equations (4) and (5) is the conversion X, and the tristimulus values X1 and Y according to the equation (6).
1, conversion to Z1; conversion to tristimulus values L0, M0, S0 based on physiological primary colors according to equation (7); conversion to matrix A in first color balance adjustment means 5; Assuming that the conversion by the matrix B in the second color balance adjusting means 6 is conversion B, the system from the input of the image data S0 to the output of the processed image data S3 can be represented as shown in FIG. The conversion in this system can be represented by one equation as shown in the following equation (26).

X ^-1 Y ^-1 Z ^-1 (αA + (1-α) B) ZYX (26) Further, this equation (26) can be modified as shown in the following equation (27).

X ⁻¹ (αY ⁻¹ Z ⁻¹ AZY + (1−α) Y ⁻¹ Z ⁻¹ BZY) X (27) Here, the conversion Y, Z, A, and B are performed in a 3 × 3 matrix. It is. Therefore, the configuration of the color balance adjustment device 1 according to the first embodiment can be changed as shown in FIG. Hereinafter, this will be described as a third embodiment.

As shown in FIG. 6, the color balance adjusting device according to the third embodiment has the same readout means 3, first color balance adjusting means 5, and second color balance adjusting means 6 as in the first embodiment. , Weighting and adding means 7, database 9 and input means 10, and color signals R0, G based on ITU Rec 709 standard constituting image data S0.
Normalizing means 12 for normalizing 0 and B0 to obtain color signals R1, G1 and B1, and color signals R6, G6 and B6 obtained by weighting and adding means 7 as described later in Rec 709.
RG for converting into color signals R7, G7, B7 conforming to the standard
And a B conversion unit 13, a matrix ^Y -1 ^Z instead of the matrix A, B in the database 9 ^- stores 1 Azy and ^Y -1 ^Z -1 BZY, first and second color balance adjustment means 5 in the 6, these matrix ^Y -1 ^Z -1 AZY and ^{Y -} is obtained to perform color balance adjustment processing on the color signals R1, G1, B1 based on 1 ^Z -1 BZY.

The normalizing means 12 normalizes the color signals R0, G0, B0 as shown in the above equations (4), (5) to obtain normalized color signals R1, G1, B1.

In the first color balance adjustment processing means 5, as shown in the following equation (28), the matrix Y
^The color balance adjustment processing is performed by converting the color signals R1, G1, and B1 using -1Z- ¹ AZY.
The processed color signals R4, G4, B4 are obtained. In the third embodiment, the matrix Y ^-1 Z ^-1 AZ
Y is a non-diagonal term having a non-zero value.

[0091]

(Equation 21) In the second color balance adjustment processing means 6, as shown in the following equation (29), the matrix Y ^-1 Z ^-1 BZ
The color balance adjustment processing is performed by converting the color signals R1, G1, and B1 by Y, and processed color signals R5, G5, and B5 are obtained.

[0092]

(Equation 22) In the weighting and adding means 7, the processed color signal R4,
G4, B4 and the processed color signals R5, G5, B5 are weighted and added at a predetermined ratio α as shown in the following equation (30) to obtain added color signals R6, G6, B6.

[0093]

(Equation 23) In the RGB conversion means 14, the added color signals R6, G
6, B6 are processed in a manner opposite to the above equations (4) and (5), and the color signals R7, G conforming to the Rec709 standard.
7, B7 are obtained. Here, the color signals R7, G7,
B7 is a color signal R3 obtained in the first embodiment.
G3 and B3, so that the color signals R7 and G3
The reference numeral S3 is used as the processed image data composed of 7 and B7 as in the first embodiment.

Next, the operation of the third embodiment will be described. FIG. 7 is a flowchart showing the operation of the third embodiment. First, the image data S0 is read by the reading means 3 from the memory card 2A in which the image is taken by the digital camera 2 and the image data S0 is stored (step S21). Then, the color signals R0, G0, B0 constituting the image data S0 are normalized by the normalizing means 12 to obtain normalized color signals R1, G1, B1 (step S22). On the other hand, according to the imaging light source type input from the input unit 10, the matrix Y ^- 1Z ^- 1A
ZY and ^{Y -} 1 ^Z -1 BZY is input to the first and second color balance adjustment means 5, 6 is read from the database 9 (Step S23).

The color signals R1, G1, and B1 are the first and second color signals.
Is input to the color balance adjustment means 5 and 6, wherein each matrix ^Y -1 ^Z -1 AZY and ^{Y -1} Z
^{-1 The} color balance adjustment processing by BZY is performed,
The processed color signals R4, G4, B4, R5, G5, B5 are obtained (step S24). Processed color signals R4, G
4, B4, R5, G5, and B5 are input to weighting and adding means 7, where weighted addition is performed, and the added color signal R
6, G6, and B6 are obtained (step S25). The added color signals R6, G6, and B6 are converted by the RGB conversion means 13 into I
Color signals R7, G7, compliant with TU Rec 709 standard
The processed image data S3 is obtained by conversion to B7 (step S26), and the processing is terminated.

On the other hand, the configuration of the color balance adjusting device 1 according to the second embodiment can be changed as shown in FIG. Hereinafter, this will be described as a fourth embodiment.

As shown in FIG. 8, the camera according to the fourth embodiment
The balance adjustment device performs the same readout as in the second embodiment.
Means 3, color balance adjusting means 11, weighting and adding means
Step 7 ', database 9 and input means 10, and a third
Normalization means 12 and RGB conversion means similar to the embodiment
13 and the database 9 has the same configuration as the third embodiment.
Matrix A instead of matrices A and B^-1Z^-1
AZY and Y^-1Z ^-1BZY is stored and weighted
In the calculating means 7 ', these matrices Y ^-1Z
^-1AZY and Y^-1Z^-1BZY is weighted and added
Thus, an addition matrix M1 is obtained.

In the third and fourth embodiments, the matrices Y ⁻¹ Z ⁻¹ AZY and Y ⁻¹
If Z ⁻¹ BZY is the same, the processed color signal obtained by the color balance adjustment processing unit 11 and the added color signals R6, G6, and B6 in the third embodiment are the same, and the RGB conversion unit 13 The processed image data composed of the obtained color signals is also the same. Therefore, in the fourth embodiment, the processed color signals obtained by the color balance adjustment processing unit 11 are denoted by reference numerals R6, G6, and B6, similarly to the third embodiment.
Is used for the color signals obtained by the RGB conversion means 13, and the reference sign S3 is used for the processed image data.

Next, the operation of the fourth embodiment will be described. FIG. 9 is a flowchart showing the operation of the fourth embodiment. First, the image data S0 is read by the reading means 3 from the memory card 2A in which the image is taken by the digital camera 2 and the image data S0 is stored (step S31). Then, the color signals R0, G0, B0 constituting the image data S0 are normalized by the normalizing means 12 to obtain normalized color signals R1, G1, B1 (step S32). On the other hand, according to the imaging light source type input from the input unit 10, the matrix Y ^- 1Z ^- 1A
ZY and ^{Y -} 1 ^Z -1 BZY is read from the database 9 (Step S33), the weighted sum in the weighted addition unit 7 'adds matrix M1 is obtained (Step S34). Steps S33 and S34
May be performed prior to steps S31 and S32, or may be performed in parallel.

The addition matrix M1 is input to the color balance adjusting means 11, where the color signals R1, G1 and B1 are subjected to color balance adjustment processing by the addition matrix M1, and the processed color signals R6, G6 and B6 are processed. Is obtained (step S35). Processed color signals R6, G6,
B6 is converted by the RGB conversion means 13 into ITU Rec 70
The image data is converted into color signals R7, G7, B7 conforming to the N.9 standard to obtain processed image data S3 (step S36).
The process ends.

Next, a fifth embodiment of the present invention will be described. FIG. 10 is a schematic block diagram illustrating a configuration of a color balance adjustment device according to a fifth embodiment of the present invention. In the fifth embodiment, the same components as those in the first embodiment are denoted by the same reference numerals, and detailed description is omitted. In the first embodiment, the color balance adjustment processing is performed using the matrices A and B stored in the database 9, but in the fifth embodiment, the matrices A and B are generated and the color balance adjustment processing is performed. It is like that. For this reason, the color balance adjusting apparatus 1 according to the fifth embodiment is different from the database 9 and the input unit 10 in the first embodiment in that the white point for estimating the white point of the photographing light source at the time of acquiring the image data S0 is used. Estimating means 14, parameter generating means 15 for generating matrices A and B based on the white point estimated by white point estimating means 14, and parameters necessary for generating matrices A and B as described later are stored. A database 16 is provided.

The white point estimating means 14 calculates the above equation (4),
The image data S0 is normalized by (5) to obtain the color signals R1,
G1 and B1, and the color signals R1, G1,
The average value of B1 is obtained, and this average value is calculated by the above equation (6).
The tristimulus values L _1w , based on the physiological primary color of the white point of the imaging light source when the image data S0 is obtained by conversion according to (7),
M _{1 w} and S _{1 w} are obtained, and are set as the white point of the imaging light source when the image data S0 is obtained. In the present embodiment, a description will be given with the imaging light source as a light source 1 and the observation light source as a light source 2.

The database 16 stores tristimulus values of a plurality of light sources (including observation light sources, hereinafter referred to as known light sources) based on physiological primaries of a white point. Also stored is a matrix for converting tristimulus values under each known light source into tristimulus values under an observation light source for observing an image. This matrix is a matrix having non-diagonal terms calculated by the above equations (13) to (21) and having values other than 0, and is hereinafter referred to as a known light source parameter.

The parameter generating means 15 calculates the white color of the light source 1
Point tristimulus value L_1w, M_1w, S_{1 w}Matri based on
And A and B are generated. First, about matrix A
Observes the processed image data S3 in the database 16
To the physiological primary color of the white point for the observation light source (light source 2)
Based tristimulus value L_2w, M_2w, S_2wIs remembered
Therefore, the white of the light source 1 when the image data S0 is obtained
Tristimulus value L based on physiological primaries of color points_1w, M_1w, S
_1wAnd the tristimulus value L of the light source 2 as the observation light source_2w,
M _2w, S_2wFrom this, von Kries's equation (1)
Find a matrix based on the chromatic adaptation prediction formula,
Box A.

Next, a method of generating the matrix B will be described. First, the parameter generation unit 15 outputs the light source 1 input from the white point estimation unit 14 in the LMS color space.
The distance between the white point tristimulus values L _1w , M _1w , S _1w and the white point tristimulus values of the known light sources stored in the database 16 is determined, and the two known light sources having the smallest distance are selected. . Specifically, the tristimulus values and the tristimulus values L _1w , M of the white point of the known light source stored in the database 16
_1w and S _1w are plotted in the LMS color space, and a known light source located closest to the white point of the light source 1 in the LMS color space, that is, can be regarded as a similar light source is selected. Here, as shown in FIG. 11, known light sources KA, KB
It is assumed that has been selected.

When the two known light sources KA and KB are selected as described above, the known light sources KA and K are selected on the LMS color space.
Straight line KAKB connecting the white points of B (points KA and KB)
And a straight line K from the white point of light source 1 (the point x in FIG. 11).
A crossing point KC with a perpendicular line dropped to AKB is obtained. Here, assuming that the straight line KAKB is divided by the intersection KC at a division ratio of 1-β: β, the intersection KC is divided into the spectral intensity distribution of the known light source KA and the spectral intensity distribution of the known light source KB as shown in FIG. Corresponds to the white point of a light source having a spectral intensity distribution synthesized at a ratio of β: 1−β.

When the intersection KC is obtained as described above, the known light source parameters MA, M of the known light sources KA, KB
By performing an interpolation operation on B, a matrix at the light source corresponding to the intersection KC is calculated.

Specifically, based on β calculated as described above, a matrix Pc for converting the LMS tristimulus values under the light source corresponding to the intersection KC into the LMS tristimulus values under the light source 2 is as follows. It is determined by equation (31).

Pc = βMA + (1-β) MB (31) Therefore, the LMS under the light source corresponding to the intersection KC
Tristimulus values y _C can be converted to the LMS tristimulus values under the light source 2 by the following equation (32).

Y ₂ = Pcy _C (32) On the other hand, the intersection KC is the known light source K in the LMS color space.
It is on the straight line KAKB connecting the white points of A and KB, and does not coincide with the light source 1. Therefore, the matrix Pc is corrected as described below so that the white color under the light source 1 is converted to the white color under the light source 2. First, the matrix P
The correction matrix ΔP is added to c, and the following equation (33) is obtained.
As shown in FIG. 5, consider that white under the light source 1 is converted to white under the light source 2.

[0111]

(Equation 24) Next, the matrices Pc and ΔP are calculated by the following equation (3).
4) As shown in (35), vo from the light source 1 to the light source 2
It is decomposed into a component V according to the n Kries color adaptation prediction formula, and other components Qc and ΔQ.

Pc = QcV (34) ΔP = ΔQV (35) Here, the tristimulus values L _1w , M _1w , S _1w and the tristimulus value L
_{Since 2w} , M _2w , and S _2w have the relationship shown in the following equation (36), equations (34) and (35) are replaced by equation (3).
Substituting into 3) gives the following equation (37).

[0113]

(Equation 25) By transforming equation (37), the following equation (38) is obtained.

[0114]

(Equation 26) However, E: unit determinant From the equation (34), since Qc = PcV ⁻¹ (39), the right side of the equation (38) is known. Therefore,

[Equation 27] If you replace like this,

[Equation 28] Becomes Here, the vectors (q00, q01, q0
2), (q10, q11, q12), (q20, q2
When the value of (1, q22) is the minimum, the component of the correction matrix ΔP is also the minimum. Therefore, the white point under the light source 1 can be converted to the white point under the light source 2 only by applying a small correction to the matrix Pc. .

Vectors (q00, q01, q02),
(Q10, q11, q12), (q20, q21, q2
The value of 2) is the smallest because these vectors are vector
Le (L _2w, M_2w, S_2w). I
Therefore, the vector is calculated by the following equations (40) to (41).
(Q00, q01, q02), (q10, q11, q1
2) By determining (q20, q21, q22)
Then, a matrix ΔQ is obtained.

[0116]

(Equation 29) Therefore, the correction matrix ΔP can be obtained by the above equation (35). Thereby, LM under the light source 1
A matrix B for converting the S tristimulus values into the LMS tristimulus values under the light source 2 can be obtained by B = Pc + ΔP (43).

This matrix B converts the tristimulus values of the white point under the light source 1 into the tristimulus values of the white point under the light source 2. L under the light source corresponding to the intersection KC
Since the matrix Pc for converting the MS tristimulus values into the LMS tristimulus values under the light source 2 is corrected,
The LMS tristimulus value under light source 1 is calculated as L under light source 2
It also has a feature of converting to an MS tristimulus value.

The matrices A and B generated in this way are input to the first and second color balance adjustment processing means 5 and 6, and the tristimulus values L0, M0, Color balance adjustment processing is performed on S0, and the processed tristimulus values L1, M1, S1, L2,
M2 and S2 are obtained. The processed tristimulus values L1, M1,
S1, L2, M2, and S2 are input to the weighting and adding means 7, where weighted addition is performed, and the added tristimulus value L3,
M3 and S3 are obtained. Additive tristimulus values L3, M3, S3
Is converted into the RGB color space by the RGB conversion means 8 to obtain the processed image data S3.

Next, a sixth embodiment of the present invention will be described. FIG. 13 is a schematic block diagram illustrating a configuration of a color balance adjustment device according to the sixth embodiment of the present invention. Note that, in the sixth embodiment, the same components as those of the second embodiment are denoted by the same reference numerals, and detailed description thereof will be omitted. In the second embodiment, the matrices A and B stored in the database 9 are weighted and added to obtain the addition matrix M0. However, in the sixth embodiment, the matrices A and B are used similarly to the fifth embodiment. B is generated, and an addition matrix M0 is obtained from the generated matrices A and B. For this reason, the color balance adjusting device 1 according to the sixth embodiment replaces the database 9 and the input unit 10 with the white point estimating unit 14 and the parameter generating unit 1 as in the fifth embodiment.
5 and a database 16.

The matrices A and B generated in the same manner as in the fifth embodiment are input to the weighting and adding means 7 ', where they are weighted and added as in the second embodiment to obtain an addition matrix M0. The addition matrix M0 is input to the color balance adjustment unit 11, where the color balance adjustment processing is performed by the addition matrix M0 to obtain processed tristimulus values L4, M4, and S4. The processed tristimulus values L4, M4, S4
The converted image data S3 is obtained after being converted into the GB color space.

Next, a seventh embodiment of the present invention will be described. FIG. 14 is a schematic block diagram showing the configuration of the color balance adjustment device according to the seventh embodiment of the present invention. In the seventh embodiment, the same components as those in the third embodiment are denoted by the same reference numerals, and detailed description thereof will be omitted. In the third embodiment, had sought matrix ^Y -1 ^Z -1 AZY and ^Y -1 ^Z -1 adder matrix M0 BZY a weighted sum stored in the database 9, in the seventh embodiment Generates matrices Y ^-1 Z ^-1 AZY and Y ^-1 Z ^-1 BZY in the same manner as in the fifth embodiment, and generates the generated matrix Y
^The color balance adjustment processing is performed based on -1Z- ¹ AZY and Y ^- 1Z- ¹ BZY.
Therefore, the color balance adjusting device 1 according to the seventh embodiment includes a white point estimating unit 14, a parameter generating unit 15, and a database 16 instead of the database 9 and the input unit 10, as in the fifth embodiment. Prepare.

Note that the matrices Y ^-1 Z ^-1 AZY and Y ^-1 Z ^-1 BZY are the matrices A and B generated in the same manner as in the fifth embodiment and the matrix Y for performing the conversion shown in equation (6). , And a matrix Z that performs the conversion shown in equation (7). The matrices Y and Z are stored in the database 16
Should be memorized.

The matrices Y ^-1 Z ^-1 AZY and Y ^-1 Z ^-1 B generated by the parameter generating means 15
ZY is input to the first and second color balance adjustment processing means 5 and 6, and color balance adjustment processing is performed on the normalized color signals R1, G1, and B1 as in the third embodiment. , Processed color signals R4, G4, B4, R
5, G5, B5 are obtained. Processed color signals R4, G
4, B4, R5, G5, and B5 are input to the weighting and adding means 7, where they are subjected to weighting and addition, and the color signals R6, G
6, B6 are obtained. The color signals R6, G6 and B6 are RGB
The conversion means 13 converts the color signals into color signals R7, G7, B7 conforming to the ITU Rec 709 standard to obtain processed image data S3.

Next, an eighth embodiment of the present invention will be described. FIG. 15 is a schematic block diagram showing the configuration of the color balance adjustment device according to the eighth embodiment of the present invention. In the eighth embodiment, the same components as those of the fourth embodiment are denoted by the same reference numerals, and the detailed description will be omitted. In the fourth embodiment, it had sought matrix ^Y -1 ^Z -1 AZY and ^Y -1 ^Z to -1 BZY weighted addition by adding matrix M1 stored in the database 9, in the eighth embodiment Generates matrices Y ^-1 Z ^-1 AZY and Y ^-1 Z ^-1 BZY similarly to the seventh embodiment, and generates the generated matrix Y
^-1 is obtained so as to obtain the Z ^-1 Azy and ^Y -1 ^Z to -1 BZY weighted addition by adding the matrix M1. For this reason, the color balance adjusting device 1 according to the eighth embodiment differs from the seventh embodiment in that the white point estimating unit 14 is replaced with the database 9 and the input unit 10,
It comprises a parameter generating means 15 and a database 16.

[0125] similarly to the seventh embodiment-generated matrix ^Y -1 ^Z -1 AZY of and ^{Y -} 1 ^Z -1 BZY is
It is input to the weighting and adding means 7 ', where it is weighted and added in the same manner as in the fourth embodiment to obtain an addition matrix M1. The addition matrix M1 is input to the color balance adjustment unit 11, where the color balance adjustment processing by the addition matrix M1 is performed, and the processed color signals R6 and R6 are processed.
G6 and B6 are obtained. Processed color signals R6, G6, B
Reference numeral 6 denotes ITU Rec 709 in the RGB conversion means 13.
The processed image data S3 is obtained by being converted into color signals R7, G7, B7 conforming to the standard.

Next, a ninth embodiment of the present invention will be described. FIG. 16 is a schematic block diagram showing the configuration of the color balance adjustment device according to the ninth embodiment of the present invention. In the ninth embodiment, the same components as those in the fifth embodiment are denoted by the same reference numerals, and a detailed description thereof will be omitted. As shown in FIG. 16, a color balance adjusting device 1 according to a ninth embodiment of the present invention includes a reading unit 3, an LMS conversion unit 4, a first color balance adjustment processing unit 5, and RGB similar to those of the fifth embodiment. The converting means 8, the white point estimating means 14, the parameter generating means 15, the database 16 and the processed tristimulus values L1, M1, S1 obtained by the first color balance adjustment processing means 5 are obtained by a matrix C which will be described later. Third to perform color balance adjustment processing to obtain processed tristimulus values L6, M6, and S6
And the processed tristimulus values L1, M1, and S1 are subjected to a color balance adjustment process using a matrix D, which will be described later, and the processed tristimulus values L
The fourth color balance adjustment processing means 18 for obtaining 7, M7, S7 and the processed tristimulus values L7, M7, S7 are compared with a ratio α.
Processed tristimulus values L6, M6, S6
And weighted adding means 27 for obtaining the added tristimulus values L8, M8, S8.

Here, the parameter generating means 15 generates matrices A and B based on the white point estimated by the white point estimating means 14 and the parameters stored in the database 16 in the same manner as in the fifth embodiment. At the same time, matrices C and D are generated. Here, the matrix C is obtained from the matrices A and B by the following equation (4).
It is generated as shown in 4).

C = BA ^-1 (44) This matrix C has the characteristics shown in the following equation (45). Therefore, other colors can be changed without changing the white point.

[0129]

[Equation 30] Here, L _w , M _w , and S _w are tristimulus values based on the physiological primary color of the white point. On the other hand, the matrix D is obtained from the matrix C by the following equation (4).
It is generated as shown in 6).

D = E−C (46) where E: unit matrix Here, the matrix D can change other colors without changing the white point like the matrix C.

Third color balance adjustment processing means 17
Is the processed tristimulus value L as shown in equation (47) below.
1, M1 and S1 are converted by a matrix C to perform a color balance adjustment process, and the processed tristimulus values L6, M
6, S6 is obtained. The processed tristimulus values L6, M6, S
Reference numeral 6 denotes processed tristimulus values L2, M2, and 2 obtained by performing conversion by the matrix B on the tristimulus values L0, M0, and S0 in the second color balance adjustment processing means 6 of the first embodiment. This is equivalent to S2.

[0132]

(Equation 31) The fourth color balance adjustment processing means 18 calculates the processed tristimulus values L1, M1, S as shown in the following equation (48).
1 is converted by the matrix D to perform color balance adjustment processing to obtain processed tristimulus values L7, M7, and S7. The processed tristimulus values L7, M7, S7 are obtained by converting the tristimulus values L0, M0, S0 by the matrix A in the first color balance adjustment processing means 5 of the first embodiment. Processed tristimulus value L
1, M1, S1 and processed tristimulus values L2, M2, S2
Is equivalent to the difference between

[0133]

(Equation 32) The weighting and adding means 27 calculates the processed tristimulus values L6, M
6, S6 and the processed tristimulus values L7, M7, S7 are weighted and added at a predetermined ratio α as shown in the following equation (49) to obtain added tristimulus values L8, M8, S8.

[0134]

[Equation 33] Here, in the ninth embodiment, the first color balance adjustment processing means 5 first corrects the white color, and the third and fourth color balance adjustment processing means 17 and 18 preserve the white color while excluding the white color. The correction is made for the color.

The added tristimulus values L8, M8, S8 obtained in the ninth embodiment are the same as the added tristimulus values L3, M3, S3 obtained in the first embodiment. Therefore, the processed image data obtained by the RGB conversion means 8 is the same as that of the first embodiment, and the processed image data obtained by the RGB conversion means 8 of the ninth embodiment is the first image data. The reference numeral S3 is used as in the embodiment.

Next, the operation of the ninth embodiment will be described. FIG. 17 is a flowchart showing the operation of the ninth embodiment. Here, the parameter generation unit 15
In the description below, it is assumed that the matrices A, C, and D have already been generated. First, the image data S0 is read out by the reading means 3 from the memory card 2A in which the image data S0 has been captured by the digital camera 2 and stored (step S41). Then, in the LMS conversion means 4, the color signals R0, G constituting the image data S0 are obtained.
0, B0 are tristimulus values L0, M0, S0 based on physiological primary colors
(Step S42). On the other hand, the matrices A, C, and D generated by the parameter generating means 15 are the first, third, and fourth color balance adjusting means 5, 1
7 and 18 (step S43).

A color balance adjustment process is performed on the tristimulus values L0, M0, S0 as follows (step S44). First, the tristimulus values L0, M0, S0 are inputted to the first color balance adjusting means 5, where the matrix A
Is performed, and the processed tristimulus values L1, M1, and S1 are obtained. The processed tristimulus values L1, M1, and S1 are input to third and fourth color balance adjustment units 17 and 18, where color balance adjustment processing is performed by matrices C and D, respectively.
Processed tristimulus values L6, M6, S6, L7, M7, S7
Is obtained. Treated tristimulus values L6, M6, S6, L
7, M7, S7 are input to the weighting and adding means 27, where weighted addition is performed, and the added tristimulus values L8, M8,
S8 is obtained (step S45). Additive tristimulus value L
8, M8, and S8 are converted into the RGB color space by the RGB conversion means 8 to obtain processed image data S3 (step S46), and the process ends.

Next, a tenth embodiment of the present invention will be described. FIG. 18 is a schematic block diagram showing the configuration of the color balance adjustment device according to the tenth embodiment of the present invention. Note that, in the tenth embodiment, the same configurations as those in the ninth embodiment are denoted by the same reference numerals, and detailed description is omitted. In the color balance adjusting apparatus 1 according to the tenth embodiment, the matrices C and D generated by the parameter generating means 15 are weighted and added by the weighting and adding means 27 'to obtain an addition matrix M2.
In the color balance adjustment processing means 19, an addition matrix M2 is added to the processed tristimulus values L1, M1, and S1 obtained in the first color balance adjustment processing means 5.
Is different from the ninth embodiment in that color balance adjustment processing is performed to obtain processed tristimulus values L8, M8, and S8.

The weighting and adding means 27 'is calculated by the following equation (5)
0), the matrices C and D are weighted and added to obtain an addition matrix M2.

M2 = C + αD (50) The fifth color balance adjusting means 19 calculates the following equation (5)
As shown in 1), the processed tristimulus values L1, M1 and S1 are converted by the addition matrix M2 and color balance adjustment processing is performed to obtain processed tristimulus values L8, M8 and S8.

[0141]

(Equation 34) Note that the processed tristimulus values obtained by the fifth color balance adjustment processing means 19 of the tenth embodiment are the same as the added tristimulus values L8, M8, S8 obtained by the weighted addition means 27 of the ninth embodiment. , The same reference numerals are used. Further, the processed image data obtained by the RGB conversion means 8 is the same as the processed image data obtained in the first embodiment, so that the reference numeral S3 is used.

Next, the operation of the tenth embodiment will be described. FIG. 19 is a flowchart showing the operation of the tenth embodiment. Here, the description will be made on the assumption that the matrices A, C, and D have already been generated by the parameter generating means 15. First, the image data S0 is read by the reading means 3 from the memory card 2A in which the image is taken by the digital camera 2 and the image data S0 is stored (step S51). Then, in the LMS conversion means 4, the color signals R0,
G0, B0 are tristimulus values L0, M0, S based on physiological primary colors
It is converted to 0 (step S52). On the other hand, the matrices A, C, and D generated by the parameter generation means 15
Is input to the first color balance adjusting means 5 and the weighting and adding means 27 '(step S53).

The matrices C and D input to the weighting and adding means 27 'are weighted and added to obtain an addition matrix M2.
Is obtained (step S54). And the tristimulus value L
Color balance adjustment processing is performed on 0, M0, and S0 as follows (step S55). First, the tristimulus values L0, M input to the first color balance adjusting means 5
0, S0 are subjected to the color balance adjustment processing by the matrix A, and the processed tristimulus values L1, M1, S
1 is obtained. The processed tristimulus values L1, M1, and S1 are
The data is input to the fifth color balance adjustment means 19, where the color balance adjustment processing is performed by the addition matrix M2, and the processed tristimulus values L8, M8, S8 are obtained. The processed tristimulus values L8, M8, and S8 are converted into the RGB color space by the RGB conversion means 8 to obtain processed image data S3 (step S56), and the process ends.

Next, an eleventh embodiment of the present invention will be described. FIG. 20 is a schematic block diagram showing the configuration of the color balance adjustment device according to the eleventh embodiment of the present invention. In the eleventh embodiment, the same components as those in the ninth embodiment are denoted by the same reference numerals, and a detailed description thereof will be omitted. As shown in FIG.
The color balance adjusting device 1 according to the present embodiment includes a reading unit 3, an LMS conversion unit 4, and a first
Color balance adjustment processing means 5, weight addition means 2
7, an RGB conversion means 8, a white point estimating means 14, a parameter generating means 15, and a database 16
Tristimulus values L0, M0, S0 obtained by conversion means 4
The sixth color balance adjustment processing means 17 'performs a color balance adjustment process using a matrix C' to be described later to process the processed tristimulus values L6 ', M6', S6 '.
And the tristimulus value L obtained by the LMS conversion means 4
The seventh color balance adjustment processing means 18 'performs color balance adjustment processing using a matrix D' described later on 0, M0, and S0, and processes the processed tristimulus values L7 ',
M7 'and S7' are obtained, and the weighted addition means 27 weights and adds the processed tristimulus values L6 ', M6' and S6 'and the processed tristimulus values L7', M7 'and S7' to obtain an added tristimulus value. L9, M9 and S9 are obtained, and the added tristimulus value L
The first, color balance adjustment processing means 5 performs color balance adjustment processing using a matrix A on the pixels 9, M9, S9 to obtain processed tristimulus values L8, M8, S8.

Here, since the sixth color balance adjustment processing means corresponds to the third color balance adjustment processing means in the ninth embodiment, 17 'is used as a reference numeral. Since the seventh color balance adjustment processing means corresponds to the fourth color balance adjustment processing means in the ninth embodiment, reference numeral 18 'is used.

In the parameter generating means 15, the matrices C 'and D' are obtained from the matrices A and B by the following equation (5).
2), generated as shown in (53).

C ′ = A ⁻¹ B (52) D ′ = E−C (53) The matrices C ′ and D ′ are the same as the matrices C and D in the ninth and tenth embodiments. Other colors can be changed without changing the point.

Sixth color balance adjustment processing means 17 '
, The tristimulus value L as shown in the following equation (54)
0, M0, and S0 are converted by the matrix C 'to perform color balance adjustment processing, and the processed tristimulus values L
6 ', M6' and S6 'are obtained.

[0149]

(Equation 35) In the seventh color balance adjustment processing means 18 ',
As shown in the following equation (55), tristimulus values L0, M0, S
0 is converted by the matrix D 'to perform color balance adjustment processing, and the processed tristimulus values L7', M7 ', S
7 'is obtained.

[0150]

[Equation 36] In the weighting and adding means 27, the processed tristimulus value L
6 ', M6', S6 'and the processed tristimulus values L7', M
7 ′ and S7 ′ are determined by the following equation (56) according to a predetermined ratio α.
Are weighted and added as shown in FIG.
9, S9 are obtained.

[0151]

(37) The first color balance adjustment processing unit 5 converts the added tristimulus values L9, M9, and S9 using the matrix A and performs the processed tristimulus values L8, M that have been subjected to the color balance adjustment processing.
8, S8 is obtained.

Here, in the eleventh embodiment, the sixth and seventh color balance adjustment processing means 17 ', 1
8 ', the correction ratio for colors other than white is changed while preserving white, and the first color balance adjustment processing means 5 corrects white.

The processed tristimulus values obtained by the first color balance adjustment processing means 5 of the eleventh embodiment and the added tristimulus values L8, M8,... Obtained by the weighting addition means 27 of the ninth embodiment. Since S8 is the same, the same reference numeral is used. Further, the processed image data obtained by the RGB conversion means 8 is the same as the processed image data obtained in the first embodiment, so that the reference numeral S3 is used.

Next, the operation of the eleventh embodiment will be described. Note that, in the eleventh embodiment, only the color balance adjustment processing in the ninth embodiment is different, so only the operation of the color balance adjustment processing will be described. In the LMS conversion means 4, the image data S
The tristimulus values L0, M0, S0 obtained by converting the color signals R0, G0, B0 constituting 0 are input to the sixth and seventh color balance adjustment processing means 17 ', 18'. The tristimulus values L0, M0, S0 are obtained by the sixth and seventh color balance adjusting means 17 ', 18'.
A color balance adjustment process is performed using the matrices C ′ and D ′, and the processed tristimulus values L6 ′, M6 ′,
S6 ', L7', M7 ', and S7' are obtained. Processed tristimulus values L6 ', M6', S6 ', L7', M7 ', S
7 'is input to the weighted addition means 27, where weighted addition is performed to obtain added tristimulus values L9, M9, S9. The added tristimulus values L9, M9, S9 are input to the first color balance adjusting means 5, where the color balance adjustment processing is performed by the matrix A, and the processed tristimulus values L8, M8, S8 are obtained. Treated tristimulus value L
8, M8 and S8 are converted to an RGB color space by an RGB conversion means 8 to obtain processed image data S3.

Next, a twelfth embodiment of the present invention will be described. FIG. 21 is a schematic block diagram showing the configuration of the color balance adjustment device according to the twelfth embodiment of the present invention. In the twelfth embodiment, the same components as those in the tenth embodiment are denoted by the same reference numerals, and detailed description thereof will be omitted. The color balance adjusting apparatus 1 according to the twelfth embodiment is configured such that the matrices C ′ and D ′ generated by the parameter
7 ', weighted addition is performed to obtain an addition matrix M3. Eighth color balance adjustment processing means 19' outputs tristimulus values L0, L0,
M0 and S0 are subjected to color balance adjustment processing by an addition matrix M3, and processed tristimulus values L10 and M
10, S10, and the processed tristimulus values L10, M1
The first color balance adjustment processing means 5 performs a color balance adjustment process using a matrix A on 0 and S10 to obtain processed tristimulus values L8, M8 and S8.

Here, since the eighth color balance adjustment processing means corresponds to the fifth color balance adjustment processing means in the tenth embodiment, reference numeral 19 'is used.

In the weighting addition means 27 ', the matrices C' and D 'are weighted and added according to the following equation (57) to obtain an addition matrix M3.

M3 = C '+ αD' (57) In the eighth color balance adjusting means 19 ', the tristimulus values L0, M0, S0 are converted by the addition matrix M3 as shown in the following equation (58). A color balance adjustment process is performed, and the processed tristimulus values L10, M10,
S10 is obtained.

[0159]

(38) The processed tristimulus values obtained by the first color balance adjustment processing means 5 of the twelfth embodiment are the same as the added tristimulus values L8, M8, S8 obtained by the weighting addition means 27 of the ninth embodiment. , The same reference numerals are used. The processed image data obtained by the RGB conversion means 8 is the same as the processed image data obtained in the first embodiment.
It is assumed that reference numeral S3 is used.

Next, the operation of the twelfth embodiment will be described. In the twelfth embodiment, only the color balance adjustment processing in the tenth embodiment is different, and therefore only the operation of the color balance adjustment processing will be described. The matrices C 'and D' generated by the parameter generating means 15 are weighted and added by 27 '.
Are weighted and added to obtain an addition matrix M3. The tristimulus values L0, M0, S0 obtained by converting the color signals R0, G0, B0 constituting the image data S0 in the LMS conversion means 4 are input to an eighth color balance adjustment processing means 19 '. Where the tristimulus value L
0, M0, and S0 are subjected to color balance adjustment processing by an addition matrix M3, and the processed tristimulus values L1
0, M10 and S10 are obtained. Treated tristimulus value L1
0, M10, and S10 are input to the first color balance adjustment processing means 5, where the color balance adjustment processing is performed by the matrix A, and the processed tristimulus values L8, M
8, S8 are obtained. Processed tristimulus values L8, M8, S
8 is converted to an RGB color space by an RGB conversion means 8 to obtain processed image data S3.

Note that the ninth to twelfth embodiments are
It has white point estimating means 14, parameter generating means 15, and database 16, and generates matrices A, C, D, C ', D' based on white points calculated from image data S0. As in the embodiment, a database 9 and an input unit 10 are provided, and matrices A, C, D, C ′, and D ′ are stored in the database 9, and the matrixes are stored in accordance with the input of the imaging light source from the input unit 10. A, C, D, C ', D' may be read out and input to the color balance adjustment processing means.

In the first to twelfth embodiments, the ratio α of the weighting addition in the weighting and adding means 7, 7 ', 27, 27' is fixed. May be changed. Hereinafter, the ratio α
Embodiment 13 of the color balance adjusting device for changing
An embodiment will be described.

FIG. 22 is a schematic block diagram showing the configuration of a color balance adjusting device according to the thirteenth embodiment of the present invention. The change of the ratio α can be performed in any of the color balance adjusting devices of the first to twelfth embodiments. However, here, the change of the ratio α in the color balance adjusting device of the first embodiment is described. It will be described as. As shown in FIG. 22, a color balance adjustment device 1 according to a thirteenth embodiment of the present invention includes a reading unit 3, an LMS conversion unit 4, a first color balance adjustment processing unit 5, and a 2 color balance adjustment processing means 6, weighting and adding means 7, RGB converting means 8, database 9, input means 10, and the ratio of weighted addition in weighting and adding means 7 based on information on the imaging light source inputted from input means 10 Changing means 2 for changing α
0, and the processed tristimulus values L1, M1, S1 and the processed tristimulus values L2, M2, S2 according to the information of the imaging light source.
Is changed.

That is, when the photographing light source is daylight, the color rarely becomes unnatural even if the processing is performed only by the matrix A, and the ratio α is preferably close to 1. On the other hand, when the imaging light source is a fluorescent light, the ratio α is preferably close to zero. Therefore, in the changing unit 20, for example, when the imaging light source is daylight, the ratio α is close to 1 based on the information of the imaging light source input from the input unit 10, and when the imaging light source is a fluorescent lamp, The ratio α in the weighting and adding means 7 is changed so that the ratio α is close to 0. Thereby, more appropriate color balance adjustment processing can be performed according to the type of the imaging light source.

In the first to thirteenth embodiments, the color balance adjusting device according to the present invention is used as a single device. However, the color balance adjusting device according to the present invention may be provided in a printer.

The digital camera 2 may be provided with a color balance adjusting device according to the present invention. Hereinafter, this will be described as a fourteenth embodiment. FIG. 23 shows the first embodiment of the present invention.
FIG. 14 is a schematic block diagram illustrating a configuration of a digital camera including a color balance adjustment device according to a fourth embodiment. Note that the digital camera 2 can be provided with any of the color balance adjustment devices of the first to thirteenth embodiments. Here, the description will be made assuming that the color balance adjustment device according to the first embodiment is provided. As shown in FIG. 23, the digital camera 2 includes an imaging unit 21 such as a CCD,
Readout means 3, LMS conversion means 4, first color balance adjustment processing means 5, second color balance adjustment processing means 6, weighting addition means 7, RG similar to the first embodiment.
It comprises a B conversion means 8, a database 9 and an input means 10, and stores the processed image data S3 obtained by the RGB conversion means 8 in the memory card 2A.

Hereinafter, the operation of the fourteenth embodiment will be described. FIG. 24 is a flowchart showing the operation of the fifth embodiment. First, a subject is imaged by the imaging means 21 to obtain image data S0 (step S6).
1). Then, the image data S
The color signals R0, G0, B0 constituting 0 are converted into tristimulus values L0, M0, S0 based on physiological primary colors (step S).
62). On the other hand, the matrices A and B are read out from the database 9 in accordance with the type of photographing light source input from the input means 10, and the first and second color balance adjusting means 5,
6 (step S63).

The tristimulus values L0, M0, S0 are input to first and second color balance adjusting means 5, 6, where color balance adjustment processing is performed by matrices A and B, respectively, and the processed tristimulus values are processed. Values L1, M1, S1, L
2, M2 and S2 are obtained (step S64). The processed tristimulus values L1, M1, S1, L2, M2, and S2 are input to the weighting and adding means 7, where weighted addition is performed to obtain the added tristimulus values L3, M3, and S3 (step S65). The added tristimulus values L3, M3, and S3 are converted into the RGB color space by the RGB conversion means 8 to obtain processed image data S3 (step S66). The processed image data S3 is stored in the memory card 2A (step S67), and the process ends.

In each of the above embodiments, the LMS
Although the color balance adjustment processing is performed in the color space, especially when the digital camera is provided with the color balance adjustment device according to the present invention as in the fourteenth embodiment,
It is preferable to perform color balance adjustment processing in the RGB color space in order to simplify the configuration of the device. Hereinafter, a color balance adjustment device that performs a color balance adjustment process in the RGB color space will be described as a fifteenth embodiment. The color balance adjustment processing in the RGB color space can be performed in any of the color balance adjustment devices according to the first to thirteenth embodiments and the digital camera including the color balance adjustment device according to the fourteenth embodiment. Although it is possible, a description will be given here assuming that a digital camera is provided with a color balance adjusting device in an RGB color space.

FIG. 25 is a schematic block diagram showing the configuration of a digital camera provided with a color balance adjusting device according to the fifteenth embodiment of the present invention. As shown in FIG. 25, the digital camera 2 includes an imaging unit 21, a first color balance adjustment processing unit 5 ', a second color balance adjustment processing unit 6', a weighting addition unit 7, an input unit 10, and a database 9 Is provided. The database 9 stores RG for the color signals R0, G0, B0 constituting the image data S0.
Matrixes A 'and B' for performing color balance adjustment processing in the B color space are stored. Here, the matrix A 'corresponds to the matrix A for performing the color balance adjustment processing in the LMS color space, and the matrix B' corresponds to the matrix B. Then, in the first color balance adjustment processing means 5 ', the color signal R is calculated by the matrix A'.
0, G0, and B0 are subjected to color balance adjustment processing to obtain processed color signals R21, G21, and B21. The color signals R0, G0, and B0 are processed by the matrix B 'in the second color balance adjustment processing means 6'. Is subjected to color balance adjustment processing, and processed color signals R22, G22,
B22, and these processed color signals R21, G21,
B21, R22, G22, and B22 are weighted and added by means 7
Are added to obtain added color signals R23, G23, and B23, which are used as processed image data S3.

As described above, by performing the color balance adjustment processing in the RGB color space, the color conversion of the image data S0 from the RGB color space to the LMS color space and the LM
Since there is no need to perform color conversion from the S color space to the RGB color space, the configuration of the device can be simplified,
In addition, processing can be performed at high speed.

In the fifth to twelfth embodiments, the tristimulus values of the white point of the image data S0 are calculated by the above equations (4), (5), (12), (6) and (7). Although the estimation is performed, for example, the tristimulus values of the white point of the image data S0 may be estimated by the white point estimation method described in Japanese Patent Application No. 11-70186. Hereafter, Japanese Patent Application No. 11-701
No. 86 describes a method of estimating tristimulus values of a white point.

First, the color signal R constituting the image data S0
The color signals R1, G1, and B1 normalized by the equations (4) and (5) are obtained from 0, G0, and B0, and the color signals R1, G1, and B1 are obtained.
Is converted to a CIE1931XYZ tristimulus value by the equation (6), and the CIE1976 is calculated by the following equation (59).
uv chromaticity value.

U = 4X / (X + 15Y + 3Z) v = 9Y / (X + 15Y + 3Z) (59) Then, the chromaticity values u and v obtained by the above equation (59)
Is calculated by the following equation (60).

U0 = Σu / n v0 = Σv / n (60) where n is the number of pixels, and then the average values u0 and v0 are calculated by the following equations (61) to (63).
To convert them into physiological primary colors L0, M0, S0.

Sx = 9.0 × u0 (6.0 × u0-16.0 × v0 + 12.0) sy = 4.0 × v0 (6.0 × u0-16.0 × v0 + 12.0) (61) X0 = sx / sy Y0 = 100.0 (62) ) Z0 = (1.0−sx−sy) / sy L0 X0 M0 = Z · Y0 (63) S0 Z0 Here, the matrix Z is the matrix in the equation (7). And the physiological primary color L obtained in this way
0, M0, and S0 are the tristimulus values of the white point of the image data S0.

In each of the above embodiments, RG
The color balance adjustment processing is performed on the image data S0 composed of the B color signal. However, the present invention is not limited to this. The color balance adjustment processing is performed on the image data S0 composed of the CMY color signals. It may be.

In each of the above embodiments, image data to be subjected to color balance adjustment processing may be image data to which color balance adjustment processing has already been performed on the assumption that observation is performed under a predetermined light source.

[Brief description of the drawings]

FIG. 1 is a schematic block diagram illustrating a configuration of a color balance adjustment device according to a first embodiment of the present invention.

FIG. 2 is a flowchart showing the operation of the first embodiment;

FIG. 3 is a schematic block diagram illustrating a configuration of a color balance adjustment device according to a second embodiment of the present invention.

FIG. 4 is a flowchart showing the operation of the second embodiment.

FIG. 5 is a diagram showing a system for converting image data according to the first embodiment;

FIG. 6 is a schematic block diagram illustrating a configuration of a color balance adjustment device according to a third embodiment of the present invention.

FIG. 7 is a flowchart showing the operation of the third embodiment.

FIG. 8 is a schematic block diagram illustrating a configuration of a color balance adjustment device according to a fourth embodiment of the present invention.

FIG. 9 is a flowchart showing the operation of the fourth embodiment.

FIG. 10 is a schematic block diagram illustrating a configuration of a color balance adjustment device according to a fifth embodiment of the present invention.

FIG. 11 is a diagram for explaining generation of a matrix in an LMS color space (part 1)

FIG. 12 is a diagram for explaining generation of a matrix in the LMS color space (part 2)

FIG. 13 is a schematic block diagram illustrating a configuration of a color balance adjustment device according to a sixth embodiment of the present invention.

FIG. 14 is a schematic block diagram illustrating a configuration of a color balance adjustment device according to a seventh embodiment of the present invention.

FIG. 15 is a schematic block diagram illustrating a configuration of a color balance adjustment device according to an eighth embodiment of the present invention.

FIG. 16 is a schematic block diagram illustrating a configuration of a color balance adjustment device according to a ninth embodiment of the present invention.

FIG. 17 is a flowchart showing the operation of the ninth embodiment;

FIG. 18 is a schematic block diagram illustrating a configuration of a color balance adjustment device according to a tenth embodiment of the present invention.

FIG. 19 is a flowchart showing the operation of the tenth embodiment.

FIG. 20 is a schematic block diagram illustrating a configuration of a color balance adjustment device according to an eleventh embodiment of the present invention.

FIG. 21 is a schematic block diagram illustrating a configuration of a color balance adjustment device according to a twelfth embodiment of the present invention.

FIG. 22 is a schematic block diagram illustrating a configuration of a color balance adjustment device according to a thirteenth embodiment of the present invention.

FIG. 23 is a schematic block diagram illustrating a configuration of a color balance adjustment device according to a fourteenth embodiment of the present invention.

FIG. 24 is a flowchart showing the operation of the fourteenth embodiment.

FIG. 25 is a schematic block diagram illustrating a configuration of a color balance adjustment device according to a fifteenth embodiment of the present invention.

[Explanation of symbols]

 DESCRIPTION OF SYMBOLS 1 Color balance adjustment device 2 Digital camera 3 Reading means 4 LMS conversion means 5 First color balance adjustment processing means 6 Second color balance adjustment processing means 7, 7 ', 27, 27' Weighting addition means 8, 13 RGB conversion Means 9, 16 Database 10 Input means 11 Color balance adjustment processing means 12 Normalization means 14 White point estimation means 15 Parameter generation means 17 Third color balance adjustment processing means 18 Fourth color balance adjustment processing means 19 Fifth color Balance adjustment processing means 20 changing means 21 imaging means

──────────────────────────────────────────────────続 き Continued on the front page (51) Int.Cl. ⁷ Identification symbol FI Theme coat ゛ (Reference) H04N 9/79 H04N 1/40 D 5C079 // H04N 101: 00 9/79 G F-term (Reference) 5B057 BA28 CA01 CB01 CC01 CE16 CH08 5C055 AA07 BA06 GA00 HA00 HA37 5C065 AA03 BB01 CC01 CC08 CC09 DD01 EE03 GG21 GG23 GG30 GG32 HH02 5C066 AA01 AA03 AA05 BA20 CA08 CA17 EA03 EA13 EC01 FA02 GA01 KA03 KE03 KA02 PQ18 TT09 5C079 HB01 HB11 JA25 LA23 LB02 MA01 MA11 NA02

Claims (38)

[Claims] 1. A color balance is adjusted for image data representing an image obtained by photographing a subject or image data subjected to a color balance adjustment process on the assumption that the image is observed under a predetermined light source. A color balance adjustment method for performing processing to obtain processed image data, comprising: a color balance adjustment process based on a first conversion parameter based on a diagonal matrix, and a color balance adjustment process based on a non-diagonal matrix including at least one non-zero off-diagonal term. A color balance adjustment method, wherein the processed image data is obtained by subjecting the image data to a mixing process in which a color balance adjustment process based on a second conversion parameter is mixed at a predetermined ratio. 2. The mixing process obtains first intermediate image data by converting the image data using the first conversion parameter, and converts the image data by using the second conversion parameter. 2. The color balance adjustment method according to claim 1, wherein the second and third intermediate image data are obtained by weighting and adding the first and second intermediate image data according to the predetermined ratio. 3. The process of weighting and adding the first conversion parameter and the second conversion parameter according to the predetermined ratio to obtain an addition parameter, and converting the image data using the addition parameter. 2. The color balance adjusting method according to claim 1, wherein: 4. The mixing process obtains first intermediate image data by converting the image data using the first conversion parameter, and converts the first intermediate image data into the first and second image data. The second intermediate image data is obtained by converting the white image generated based on the conversion parameter by a third conversion parameter that stores the white, and the first intermediate image data is converted into the third conversion parameter, the unit parameter, and The third intermediate image data is obtained by converting with a fourth conversion parameter that stores the white color generated based on the second intermediate image data, and the second and third intermediate image data are weighted and added according to the predetermined ratio. 2. The color balance adjusting method according to claim 1, further comprising: 5. The mixing process obtains first intermediate image data by converting the image data according to the first conversion parameter, and generates a white image generated based on the first and second conversion parameters. And a fourth conversion parameter for storing white generated based on the third conversion parameter and the unit parameter are weighted and added according to the predetermined ratio to obtain an addition parameter. 2. The color balance adjustment method according to claim 1, wherein the first intermediate image data is converted by the addition parameter. 6. The image processing device according to claim 1, wherein the mixing process includes:
A first intermediate image data is obtained by converting the white data generated based on the first and second conversion parameters using a third conversion parameter that stores the white color. The second intermediate image data is obtained by converting the white image generated based on the unit parameter with a fourth conversion parameter that stores the white color, and the first and second intermediate image data are weighted and added by the predetermined ratio. 2. The color balance adjustment method according to claim 1, further comprising obtaining additional image data by performing the conversion, and converting the additional image data using the first conversion parameter. 7. The mixing process according to claim 1, wherein the first and second mixing processes are performed.
A third conversion parameter for storing the white color generated based on the conversion parameter of the above, and a fourth conversion parameter for storing the white color generated based on the third conversion parameter and the unit parameter, A weighted addition based on a ratio to obtain an addition parameter; converting the image data using the addition parameter to obtain converted image data; and converting the converted image data using the first conversion parameter. The color balance adjustment method according to claim 1. 8. The color balance adjustment method according to claim 1, wherein the mixing process is performed in a color space related to a spectral characteristic of a photographing device that has photographed the image. 9. The color balance adjustment method according to claim 1, wherein the mixing process is performed in a color space related to human visual characteristics. 10. The first conversion parameter relates to a color balance adjustment process performed based on the diagonal matrix in a color space related to a human visual characteristic, and relates to a spectral characteristic of a photographing apparatus that has photographed the image. The second conversion parameter is a color space adjustment process performed based on the off-diagonal matrix in a color space related to human visual characteristics, the image is captured 4. The color balance adjusting method according to claim 1, wherein the parameter is a parameter to be performed in a color space related to a spectral characteristic of the photographing device. 11. The method according to claim 1, further comprising: estimating a white point of a photographing light source when photographing the image data, and generating the first and second conversion parameters based on the white point. The color balance adjusting method according to any one of the above items. 12. The apparatus according to claim 1, wherein a designation of a photographing light source at the time of photographing the image data is received, and the predetermined ratio is changed in accordance with the designated photographing light source. 2. The color balance adjustment method according to claim 1. 13. A color balance is adjusted for image data representing an image obtained by photographing a subject, or image data subjected to color balance adjustment processing on the assumption that the image is observed under a predetermined light source. A color balance adjustment device for performing processing to obtain processed image data, comprising: a color balance adjustment process based on a first conversion parameter based on a diagonal matrix; and a color balance adjustment process including at least one nonzero off-diagonal term. A color processing unit that obtains the processed image data by performing a mixing process on the image data by mixing a color balance adjustment process based on a second conversion parameter with a predetermined ratio. apparatus. 14. The image processing apparatus according to claim 1, wherein the mixing processing unit converts the image data using the first conversion parameter to obtain first intermediate image data, and converts the image data into the second conversion data. A second conversion unit that obtains second intermediate image data by performing conversion with a parameter; and a weighting addition unit that weights and adds the first and second intermediate image data according to the predetermined ratio. The color balance adjusting device according to claim 13, wherein 15. The weighting and adding means for obtaining an addition parameter by weighting and adding the first conversion parameter and the second conversion parameter according to the predetermined ratio; and adding the image data to the image data. 14. The color balance adjustment device according to claim 13, further comprising a conversion unit configured to perform conversion by a parameter. 16. The mixing processing means, wherein the first conversion means obtains first intermediate image data by converting the image data using the first conversion parameter; and A second conversion unit that obtains second intermediate image data by converting a white color generated based on the first and second conversion parameters using a third conversion parameter that stores the white color; and the first intermediate image. A third conversion unit that obtains third intermediate image data by converting the data by using a fourth conversion parameter that stores white generated based on the third conversion parameter and the unit parameter; 14. The color balun according to claim 13, further comprising: weighting and adding means for weighting and adding the second and third intermediate image data according to the predetermined ratio. Adjustment device. 17. The first conversion unit for obtaining first intermediate image data by converting the image data by using the first conversion parameter, the mixing processing unit comprising: Weighting the third conversion parameter for storing the white color generated based on the third conversion parameter and the fourth conversion parameter for storing the white color generated based on the third conversion parameter and the unit parameter by the predetermined ratio 14. The color balance adjusting device according to claim 13, further comprising: a weighting addition unit that obtains an addition parameter by adding, and a second conversion unit that converts the first intermediate image data using the addition parameter. 18. The first intermediate image by converting the image data by a third conversion parameter for storing white generated based on the first and second conversion parameters. First to get data
And a second conversion unit that converts the image data with a fourth conversion parameter that stores white generated based on the third conversion parameter and the unit parameter to obtain second intermediate image data. Conversion means; weighted addition means for obtaining added image data by weighting and adding the first and second intermediate image data according to the predetermined ratio; and converting the added image data by the first conversion parameter. 14. The color balance adjusting device according to claim 13, further comprising a third conversion unit. 19. The mixing processing means according to claim 3, further comprising: a third conversion parameter for storing a white color generated based on the first and second conversion parameters; and a third conversion parameter and a unit parameter. Weighting and adding means for obtaining an addition parameter by weighting and adding a fourth conversion parameter for storing the generated white color according to the predetermined ratio; and obtaining converted image data by converting the image data using the addition parameter. 14. The color balance adjusting device according to claim 13, further comprising: a first conversion unit; and a second conversion unit configured to convert the converted image data using the first conversion parameter. 20. The image processing apparatus according to claim 19, wherein the mixing processing unit includes:
20. The color balance adjustment device according to claim 13, wherein the color balance adjustment device is a unit that performs the mixing process. 21. The color balance adjustment according to claim 13, wherein said mixing processing means is means for performing said mixing processing in a color space related to human visual characteristics. apparatus. 22. The first conversion parameter relates to a color balance adjustment process performed based on the diagonal matrix in a color space related to human visual characteristics, and relates to a spectral characteristic of a photographing device that has photographed the image. The second conversion parameter is a color space adjustment process performed based on the off-diagonal matrix in a color space related to human visual characteristics, the image is captured 16. The color balance adjusting device according to claim 13, wherein the parameter is a parameter to be performed in a color space related to a spectral characteristic of the photographing device. 23. A white point estimating means for estimating a white point of a photographing light source when the image data is photographed, and a parameter generating means for generating the first and second conversion parameters based on the white point. 23. The color balance adjustment device according to claim 13, wherein the color balance adjustment device is provided. 24. An image forming apparatus comprising: receiving means for receiving designation of a photographing light source at the time of photographing the image data; and changing means for changing the predetermined ratio according to the designated photographing light source. 24. The color balance adjustment device according to claim 13, wherein 25. A color balance is adjusted for image data representing an image obtained by photographing a subject or image data subjected to color balance adjustment processing on the assumption that the image data is observed under a predetermined light source. A computer-readable recording medium storing a program for causing a computer to execute a color balance adjustment method of performing processing to obtain processed image data, the program comprising: a color balance using a first conversion parameter based on a diagonal matrix. Performing, on the image data, a mixing process in which an adjustment process and a color balance adjustment process using a second conversion parameter based on a non-diagonal matrix including at least one non-zero non-diagonal term are mixed at a predetermined ratio. A step of obtaining the processed image data. Computer-readable recording medium. 26. The step of performing the mixing process includes the steps of: converting the image data by the first conversion parameter to obtain first intermediate image data; and the step of converting the image data by the second conversion parameter. 26. The computer-readable medium according to claim 25, further comprising: a step of obtaining second intermediate image data by conversion; and a step of weighting and adding the first and second intermediate image data according to the predetermined ratio. Possible recording medium. 27. The step of performing the mixing process includes a step of obtaining an addition parameter by weighting and adding the first conversion parameter and the second conversion parameter according to the predetermined ratio; and a step of adding the image data. 26. The computer-readable recording medium according to claim 25, further comprising a step of performing conversion by a parameter. 28. The step of performing the mixing process includes a step of obtaining the first intermediate image data by converting the image data using the first conversion parameter; and a step of converting the first intermediate image data into the first intermediate image data. Obtaining a second intermediate image data by converting the first intermediate image data by converting the white image generated based on the first and second conversion parameters with a third conversion parameter for storing the white color. Obtaining third intermediate image data by converting with a fourth conversion parameter that stores white generated based on the conversion parameter and the unit parameter of the second and third intermediate image data. 26. The computer-readable recording medium according to claim 25, further comprising a step of performing weighted addition by a predetermined ratio. 29. The step of performing the mixing process includes a step of converting the image data using the first conversion parameter to obtain first intermediate image data, and a step of obtaining the first intermediate image data based on the first and second conversion parameters. Weighting and adding the third conversion parameter for storing the white color generated in accordance with the predetermined ratio with the fourth conversion parameter for storing the white color generated based on the third conversion parameter and the unit parameter. 26. The computer-readable recording medium according to claim 25, further comprising: a step of obtaining an addition parameter by using the addition parameter; and a step of converting the first intermediate image data using the addition parameter. 30. A step of performing the mixing process, wherein the image data is converted by a first conversion parameter for storing a white color generated based on the first and second conversion parameters. Obtaining intermediate image data; and converting the image data by a fourth conversion parameter for storing a white color generated based on the third conversion parameter and the unit parameter. Obtaining, adding and weighting the first and second intermediate image data according to the predetermined ratio to obtain added image data, and converting the added image data by the first conversion parameter. 26. The computer-readable recording medium according to claim 25, comprising: 31. The step of performing the mixing process includes a step of storing a third conversion parameter for storing a white color generated based on the first and second conversion parameters, and the third conversion parameter and a unit parameter. A procedure for obtaining an addition parameter by weighting and adding a fourth conversion parameter for storing the white color generated based on the predetermined ratio, and a procedure for obtaining converted image data by converting the image data using the addition parameter. 26. The computer-readable recording medium according to claim 25, further comprising: converting the converted image data using the first conversion parameter. 32. The method according to claim 25, wherein the step of performing the mixing process is a step of performing the mixing process in a color space related to a spectral characteristic of a photographing apparatus that has photographed the image. The computer-readable recording medium according to claim 1. 33. The computer according to claim 25, wherein the step of performing the mixing process is a step of performing the mixing process in a color space related to human visual characteristics. A readable recording medium. 34. The first conversion parameter relates to a color balance adjustment process performed based on the diagonal matrix in a color space related to human visual characteristics, and relates to a spectral characteristic of a photographing device that has photographed the image. The second conversion parameter is a color space adjustment process performed based on the off-diagonal matrix in a color space related to human visual characteristics, the image is captured The computer-readable recording medium according to any one of claims 25 to 27, wherein the parameter is a parameter to be performed in a color space related to a spectral characteristic of an imaging device. 35. A method comprising: estimating a white point of an imaging light source when imaging the image data; and generating the first and second conversion parameters based on the white point. Claim 2
35. The computer-readable recording medium according to any one of items 5 to 34. 36. A method according to claim 36, further comprising: a step of receiving designation of a photographing light source when the image data is photographed; and a step of changing the predetermined ratio in accordance with the designated photographing light source. 25 to 3
6. The computer-readable recording medium according to any one of claims 5 to 7. 37. An imaging apparatus comprising the color balance adjustment device according to claim 13. Description: 38. An output device comprising the color balance adjustment device according to any one of claims 13 to 24.