JP2002027264A - Image processing method and image processor - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide an image processing method and an image processor in which an input image is converted into a color space independent from the image processor and illumination with high accuracy. SOLUTION: Image data and colorimetric value of a color chip composed of a plurality of colors are acquired previously, ideal image data being predicted using a signal generation model of an input unit is calculated from the colorimetric value of the color chip. Image data of the color chip is then converted into the ideal image data through a neural network or a function and a matrix in order to remove nonlinearity of the image data. An image processor of higher order than the image data or color data independent from the image processor and illumination can be estimated with high accuracy using the image data from which nonlinearity is removed.

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

[0001] 1. Field of the Invention [0002] The present invention relates to an image processing method for converting color image data input from a commercially available input device into a color space independent of the device and illumination with high accuracy, and a device therefor.

[0002]

2. Description of the Related Art In recent years, various devices, such as scanners, digital cameras, printers, and displays, have been used as devices for processing color images. According to a technique for exchanging image data between these devices, color image data input from an input device is once converted into an independent color space independent of the device, and then converted into color image data to be output to an output device. There is something.

As described above, if the conversion between the signal of the image input device and the color space independent of the device is established, the data can be transferred to any image output device. There is no need to determine the number of color conversion processes.

If the color image data input from the image input device is converted into an independent color space not depending on the device but also on the illumination, an image under illumination different from the illumination at the time of image input can be output to the output device. You can also output from.

[0005] As an independent color space independent of the device,
XYZ tristimulus values specified by the International Standards Institute CIE, and L ^* a ^* b ^* color system, L ^* u ^* v ^* color system, or it is common to use a color appearance model such as CAM97s, Since the attribute values of the color appearance model are calculated from the XYZ tristimulus values, if the XYZ tristimulus values can be estimated from the signals of the image input / output device, the above color conversion becomes possible.

As a color space that does not depend on the device and the illumination, it is general to use the spectral reflectance of the subject. By integrating the spectral reflectance with the desired illumination, XYZ
Tristimulus values can be calculated.

Estimating the XYZ tristimulus values or the spectral reflectance of a subject from the color space of each image input / output device in this manner is called characterization.

The present invention relates to the characterization of an image input device such as a digital camera, a multispectral camera, and a scanner.

Conventional characterization techniques for an image input device include, for example, a colorimetric method for skin and a method for estimating a spectral reflection spectrum described in Japanese Patent Application Laid-Open No. 7-174631, and a color simulation described in Japanese Patent Application Laid-Open No. 9-233490. There is a device. Japanese Patent Application Laid-Open No. 7-174631 discloses a method of estimating a spectral reflection spectrum of skin from an image input from an image input device.

The procedure will be described with reference to FIG. First,
The image data RGB input in step 701 is converted into a signal linear in luminance by a quadratic function. The quadratic function described in the above publication is shown in (Equation 1). (Equation 1) is determined so that the XYZ tristimulus values of the 9-tone color chip of achromatic color are measured, and the luminance is linear with the Y value.

[0011]

(Equation 1)

Next, in step 702, from the luminance linear signal,
XY with multiple regression matrix using at least quadratic terms
Calculate Z tristimulus values. Finally, in step 703, the spectral reflectance is estimated from the XYZ tristimulus values. The dimension of the spectral reflectance is, for example, 31 dimensions even if the range of visible light from 400 nm to 700 nm is sampled every 10 nm, which is considerably large and difficult to estimate. The following expression method is used. Since the cumulative contribution ratio of the spectral reflectance of the skin as the subject to the third principal component is 99.5%, m = 3 is sufficient, so that the basis coefficient can be uniquely obtained from the XYZ tristimulus values.

As described above, the conventional characterization method converts the input image data into a signal linear in luminance.
(R ', G', B ') by a quadratic function to remove nonlinearity,
It consists of two stages of color estimation processing for estimating the XYZ values or the spectral reflectance from the (R ', G', B ') signal from which the non-linearity has been removed.

In the color simulation device described in Japanese Patent Application Laid-Open No. 9-233490, an illumination simulation for converting an image input from an image input device into a color under a desired light source and outputting the color on a display is disclosed. I have. The procedure will be described with reference to FIG.

Similar to the method described in Japanese Patent Application Laid-Open No. 7-174631, the spectral reflectance is subjected to principal component analysis, and is represented by an m-th order base lower than the 31st order. Then, in step 801, a m-th order basis coefficient vector is estimated from the input image data using a neural network. Next, in step 802, the spectral reflectance is calculated from the estimated m-order vector. The obtained spectral reflectance is multiplied by a desired light source vector to obtain XYZ tristimulus values, which are converted into display drive signals using the color characteristics of the display.

As described above, unlike the conventional example, the neural network estimates the basis coefficient of the spectral reflectance directly from the input image data without separating the nonlinearity removal processing and the color estimation processing.

[0017]

Problems to be Solved by the Invention
In the method described in Japanese Patent Application Laid-Open Publication No. H10-157, nonlinearity of input image data is removed by a quadratic function, but a signal output from an image input device is subjected to not only gamma conversion but also white balance and other color correction processing. Simple R, G,
It is impossible to remove nonlinearity by conversion using a quadratic function for each B signal. If the non-linearity remains, a large error occurs in the XYZ tristimulus values and the spectral reflectance, which are final estimation purposes.

In the color estimation processing after the non-linearity removal processing, since the subject is limited to the skin, the number of dimensions of the spectral reflectance can be reduced to three dimensions by principal component analysis. In other subjects, it is impossible to express the spectral reflectance in three dimensions. For example, journal
Of Optical Society America A Vol.3 No.1
0 (1986) Page 1673 "Evaluation of
Linear Models of Surfaces Spectral Reflectance with Small Numbers of Parameters states that at least six or eight dimensions are required to represent the spectral reflectance of any object. . Therefore, there is a problem in that an image input apparatus having a band number of less than 6 to 8 has to estimate higher-order information.

In the method described in Japanese Patent Application Laid-Open No. 9-233490, the basis coefficient of the spectral reflectance is directly estimated from the image data by the neural network. However, the relationship between the image data and the basis coefficient is originally nonlinear and linear. Therefore, when a multi-layer perceptron is used as a neural network, for example, a high-precision estimation cannot be performed unless the number of units and the number of layers in the intermediate layer is large and the number of learnings is not large.

As described above, the method of calculating the XYZ tristimulus values and the spectral reflectance of the subject from the image data of the image input device with high accuracy has not been solved yet.

The present invention has been made in view of the above points, and has as its object to provide an image processing method for converting color image data input from an input device into a color space that does not depend on the device and illumination with high accuracy. And

[0022]

According to the present invention, image data and colorimetric values of a color chart composed of a plurality of colors are obtained in advance, and a signal generation model of an input device is obtained from the colorimetric values of the color chart. By calculating the ideal image data predicted by using, the image data of the color chart is converted to the ideal image data by a neural network or a function and a matrix, thereby removing the nonlinearity of the image data, An image processing method for estimating color data that is independent of the image data and a device that is higher in dimension than the image data or that is independent of the device and the illumination using the image data from which the nonlinearity has been removed.

According to the present invention, the color image data input from the input device can be converted into color data independent of the device or the device and the illumination with high accuracy.

[0024]

According to the first aspect of the present invention, when estimating color data independent of a device or a device and illumination from image data input by an image input device, the dimension of the image data is determined by the device or the device. If the dimension is lower than the dimension of the color data independent of the illumination, the color data is estimated from the image data up to the same dimension as the image data, and the remaining dimensions are up to the same dimension as the image data already estimated. This is an image processing method of estimating from color data, and has an effect that a signal of an image input device can be converted to a color space independent of the device or the device and the illumination with high accuracy.

According to a second aspect of the present invention, when estimating color data independent of the device or the device and the illumination from the image data input by the image input device, the dimension of the image data is independent of the device or the device and the illumination. If it is lower than the dimension of the color data, the image processing method is to separately estimate the data of the remaining dimension up to the same dimension as the image data,
This has the effect that the signal of the image input device can be converted into a color space independent of the device or the device and the illumination with high accuracy.

According to a third aspect of the present invention, when color data independent of a device or a device and illumination is estimated from image data input by an image input device, a color independent of the device or a device and illumination is estimated from the image data. This is an image processing method in which conversion to data is performed by matrix conversion extended to second-order or higher terms, and it is possible to convert a signal of an image input device to a color space independent of a device or a device and illumination with high accuracy. Has the effect of being able to.

According to a fourth aspect of the present invention, the image data used for conversion into color data independent of the device or the device and the illumination is obtained by removing the non-linearity of the image data input from the image input device. An image processing method according to any one of claims 1 to 3, wherein the signal of the image input device can be converted into a color space that does not depend on the device or the device and the illumination with high accuracy. It has the action of:

According to a fifth aspect of the present invention, when the color data independent of the apparatus or the apparatus and the illumination is estimated after removing the nonlinearity of the image data input by the image input apparatus, the nonlinearity is removed. This is an image processing method performed by a neural network. In the conventional example, there is also a method of directly estimating the basis coefficient of the spectral reflectance by the neural network. However, in the conventional method, the network structure is complicated. While this method requires learning data and requires a long learning time, this method has an effect that a simple network can be obtained, and a good estimation result can be obtained with a small number of learning data and a small learning time. .

According to a sixth aspect of the present invention, when the color data independent of the apparatus or the apparatus and the illumination is estimated after removing the nonlinearity of the image data input by the image input apparatus, the nonlinearity is removed. This is an image processing method performed by a function and a matrix conversion, and can remove nonlinearity of input image data with high accuracy.

According to a seventh aspect of the present invention, there is provided the image processing apparatus according to the sixth aspect, wherein the matrix used for removing the non-linearity is expanded to a second or higher order term of the input image data. This method can remove nonlinearity of input image data with high accuracy.

According to the present invention, image data and colorimetric values of a color chart composed of a plurality of colors are obtained in advance, and a signal generation model of an input device is obtained from the colorimetric values of the color chart. Calculate ideal image data to be predicted using, and convert the image data of the color chart to the ideal image data to remove the nonlinearity of the image data, and remove the nonlinearity from the image. This is an image processing method for estimating color data that is independent of the device or the device and the illumination using the data. In this way, the input image data is converted into data suitable for the signal generation model of the image input device, and then spectral analysis is performed. There is an effect that the estimation accuracy can be improved by estimating the reflectance.

According to a ninth aspect of the present invention, there is provided a non-linearity removing means for removing non-linearity of image data input from an image input device, and a device which is higher in order than the image data based on the non-linearity-removed image data. Alternatively, the image processing apparatus includes an apparatus and color estimation means for estimating color data independent of illumination, and converts a signal of the image input apparatus into a color space independent of the apparatus or the apparatus and the illumination with high accuracy. It has the effect of being able to.

Hereinafter, embodiments of the present invention will be described in detail with reference to the drawings.

(Embodiment 1) A method of estimating the spectral reflectance of a subject, which is data independent of the device and illumination, from RGB image data, assuming that the image input device is a digital camera having RGB three-band outputs will be described. .

FIG. 1 shows an input device according to the first embodiment.
FIG. 3 is a block diagram of an image processing device that estimates spectral reflectance image data from RGB image data. In FIG. 1, 101
Is image input means such as a digital camera for acquiring image data of RGB three bands, 102 is image data input from the image input means 101, 103 is data which does not depend on a device and illumination, that is, estimates spectral reflectance from the image data 102 Color estimating means 104;
Is an image recording unit for recording the spectral reflectance image data 104; and 106 is one of the components of the color estimating unit 103, and removes nonlinearity from the input image data. Nonlinearity removing unit,
Reference numeral 07 denotes a non-linear removal processing parameter storage unit that stores parameters such as weights, threshold values, functions, and matrices of a neural network used for non-linearity removal. Reference numeral 108 denotes image data from which the non-linearity obtained by the non-linearity removal unit 106 has been removed; This is hereinafter referred to as scalar image data.

Reference numeral 109 denotes one of the components of the color estimating means 103.
A spectral reflectance estimating unit for estimating the spectral reflectance of the object from the scalar image data 108; a camera 110 for storing camera parameters such as the spectral sensitivity of the camera used in the spectral reflectance estimating unit 105 and illumination during shooting; This is a parameter storage unit.

The operation of the color estimating means of FIG. 1 will be described. The image data 102 input from the image input unit 101 is converted by the color estimation unit 103 into spectral reflectance image data 104 which is a color space independent of the device and illumination, and is recorded in the image recording unit 105.

The color estimating means 103, which is a feature of the present invention,
It comprises a non-linear elimination unit 106, a non-linear elimination processing parameter storage unit 107, a spectral reflectance estimating unit 109, and a camera parameter storage unit 110.

The operation of the color estimating means 103 will be described. First, the image data 102 input to the color estimating means 103 is
The scalar image data 104 by the non-linearity removing unit 106
Then, the spectral reflectance estimating unit 105 converts the scalar image data into the spectral reflectance image data 107.

The detailed operation of the non-linearity removing section 106 will be described. In the nonlinearity removing unit 106, the image data 1
02 is converted to scalar image data 108.

A case where a multilayer perceptron, which is a neural network, is used as one of the methods for realizing the conversion will be described. The multi-layer perceptron previously learns weights and thresholds, which are parameters of neurons, and performs estimation using the parameters obtained by the learning. Therefore, first, a parameter learning procedure of the multi-layer perceptron will be described with reference to FIG.

In step 201, image data and colorimetric values of a color chart composed of a plurality of colors are obtained in advance. Next, in step 202, ideal image data predicted using the signal generation model of the input device is calculated from the colorimetric values of the color chart.
Next, in step 203, weights and thresholds are learned using the image data 102 as input data and ideal image data as teacher data. The weight and threshold obtained by this learning are stored in the non-linearity removal processing parameter storage unit 107.

The method of calculating ideal image data in step 202 will be described.

The ideal image data can be obtained by substituting the colorimetric values of the color chart into the signal generation model of the input device. The signal generation model includes the spectral reflectance R (λ) of the object, the spectral distribution S (λ) of the illumination, and the spectral sensitivity C _R of the three RGB bands of the camera.
Using (λ), C _G (λ), and C _B (λ), it can be written by (Equation 2). It is assumed that the spectral sensitivity of the camera and the spectral distribution of the illumination are known.

[0045]

(Equation 2)

R ′, G ′, B ′ obtained by substituting the spectral reflectance of the color chart for R (λ) in (Equation 2) become ideal image data.

FIG. 3 is an explanatory diagram of the signal generation model of (Expression 2).
Shown in In FIG. 3, reference numeral 301 denotes the spectral reflectance of a color chart to be substituted into the signal generation model, 302 denotes the spectral sensitivity characteristic of a known input device, 303 denotes the illumination characteristic at the time of capturing the image data 102, and 304 denotes the signal generation of Equation (2). A model 305 is ideal image data of a color chart.

As shown in FIG. 3, by giving the spectral sensitivity characteristic 302 and the illumination characteristic 303 of the input device and the spectral reflectance 301 of the object to the signal generation model, that is, (Equation 2), ideal image data 305 of the object is obtained. Is obtained.

Next, estimation of weights and threshold values learned as described above using a neural network will be described. The non-linearity removing unit 106 reads out the weights and thresholds stored in the non-linearity removal processing parameter storage unit 107, and inputs the image data 102 to the neural network using the weights and the threshold values. As a result, scalar image data 108, which is image data from which nonlinearity has been removed, is obtained at the output of the neural network.

In the method described in JP-A-9-233490, the basis coefficient of the spectral reflectance is directly estimated from the image data by a neural network. There must be multiple stages of processing including non-linear and linear. If all such complicated operations are to be learned by a single neural network, it is necessary to increase the number of layers and neurons, or to increase the number of times of learning. Processing is extremely time-consuming.

On the other hand, in the method of the present embodiment, the process up to obtaining the scalar image data from the image data is performed by a neural network, and the estimation of the spectral reflectance from the scalar image data is performed, for example, as shown in the spectral reflectance estimating unit 109. By using a simple analytical method, the structure of the neural network can be simplified, and a good estimation result can be obtained even if the number of learning data and the learning time are small. For example, experiments have shown that a good estimation result can be obtained by learning 5,000 to 10,000 times with a multilayer perceptron of three layers having 10 neurons in the hidden layer. On the other hand, a multi-layer perceptron having the same structure according to the method described in
You cannot get a satisfactory estimation result even after learning 000 times.

Next, as another method for converting the image data 102 into the scalar image data 108 by the nonlinearity removing unit 106, a method using a one-dimensional function and a matrix will be described. The one-dimensional conversion function is a function corresponding to the inverse gamma of the camera, and is a non-linear function as shown in FIG. In this method, first, image data is converted into (R '', G '', B '') by this nonlinear one-dimensional conversion function, and further, (R '',
G ″, B ″) are converted to scalar image data (R ′, G ′, B ′) and output from the nonlinear elimination unit 106. It is assumed that the one-dimensional conversion function and the matrix used for estimation are determined in advance and stored in the non-linear removal processing parameter storage unit 107.

The procedure for determining the function and the matrix will be described with reference to FIG. In step 501, image data and colorimetric values of a gray scale color chart are obtained in advance. In step 502, ideal image data predicted from the colorimetric values of the color chart using the signal generation model (Equation 2) of the input device is calculated. Step 50
In step 3, a one-dimensional conversion function that uses the image data 102 as input data and outputs ideal image data is determined for each color. In step 504, image data and colorimetric values of a color chart composed of a plurality of colors are obtained in advance. In step 505, ideal image data to be predicted is calculated from the colorimetric values of the color patches of the plurality of colors using the signal generation model (Equation 2) of the input device. Step 5
In step 06, the image data of the color patches of a plurality of colors is input to the one-dimensional conversion function obtained in step 503 to obtain (R '', G '', B ''). Step 5
At 07, a matrix for estimating the ideal image data (R ′, G ′, B ′) in step 505 from (R ″, G ″, B ″) is determined.

The method of calculating ideal image data in steps 502 and 504 is the same as in FIG. Step 5
For the determination of the one-dimensional conversion function in 03, gray gradation is used as a color chart. The one-dimensional conversion function shown in FIG. 4 can be obtained by taking the image data of the gray gradation color chart on the horizontal axis and the ideal image data on the vertical axis.

The value obtained by the one-dimensional conversion function is treated as the one in which the non-linearity is removed in the method described in Japanese Patent Application Laid-Open No. 7-174631, but not only gamma but also complicated Since the processing is incorporated, it is not possible to sufficiently remove the non-linearity using only the one-dimensional conversion function. Experiments have shown that leaving the non-linearity greatly reduces the accuracy of the finally obtained spectral reflectance.

Therefore, in the present invention, the output (R ", G", B ") of the one-dimensional conversion function is further subjected to matrix conversion in step 507 to remove nonlinearity. A method for determining a matrix in step 507 will be described.

In step 506, the image data of the color chart composed of a plurality of colors has already been substituted into the one-dimensional conversion function determined in step 503 to obtain (R ", G", B "). . On the other hand, step 5
At 05, the ideal image data (R ', G', B ') of the same color chart has already been obtained. A matrix S for converting these two coordinate values is defined by (Equation 3).

[0058]

(Equation 3)

Each component of the matrix S is represented on the left side by each color of the color chart.
A matrix created by (R ', G', B ') ^t arranged side by side, and (R'',G'',B'') ^t of each color of the color chart arranged side by side in the second term on the right side The matrix S may be obtained using (Equation 4) so as to minimize the square error using the matrix. In (Equation 4), + means a general inverse matrix of Moore Penrose.

[0060]

(Equation 4)

Alternatively, instead of (Equation 2), (R ″, G ″,
B ″) may be extended to include the square terms and the product terms as shown in (Equation 5) to remove nonlinearity.

[0062]

(Equation 5)

The operation of the non-linearity removing section 106 has been described above.

Next, details of the spectral reflectance estimating unit 109 will be described. The spectral reflectance estimating unit 109 estimates the spectral reflectance of the object from the scalar image data 108 calculated by the nonlinearity removing unit 106. This corresponds to the left-hand side (R ',
G ′, B ′) when the scalar image data 104 is substituted
This is the inverse problem of estimating (λ). In (Equation 2), since the scalar image data and the spectral reflectance have a linear relationship,
If (Equation 2) is rewritten into a discrete matrix expression, (Equation 6) is obtained.

[0065]

(Equation 6)

The left side of (Equation 6) is the scalar image data 10
8, (R1, R2,..., Rn) ^T is a discrete expression of the spectral reflectance of the object, and each component represents the reflectance at a wavelength of, for example, every 10 nm. Matrix A is a matrix determined by the spectral sensitivity of the camera and the spectral distribution of illumination.

By using (Equation 6), scalar image data 1
The problem of estimating the spectral reflectance R of the object from the color image 08 is a linear inverse problem.
(1, R2, ..., Rn) ^T has a dimension number much larger than three, and it is difficult to estimate it.

As a method for solving this problem, for example,
If the spectral reflectance of the object is expressed by a basis function having a lower order than n, the number of dimensions of data to be obtained can be reduced.
For example, the basis functions are defined as 6-dimensional O ₁ (λ), O ₂ (λ), O ₃ (λ), O
₄ (λ), O ₅ (λ), O ₆ (λ), (Equation 2) can be rewritten as (Equation 7), and the data to be estimated are (a, b, c, d,
e, f) is a sixth-order vector. Therefore, (Equation 6) is
Equation 8 can be rewritten. In Equation 8, the matrix B is a matrix determined by the spectral sensitivity of the camera, the spectral distribution of illumination, and the above basis function.

[0069]

(Equation 7)

[0070]

(Equation 8)

In (Equation 7), if the basis function is three-dimensional, the number of dimensions of input data and the number of dimensions of output data are both equal, so that matrix B becomes a square matrix and a solution can be uniquely obtained. If the spectral reflectance is deviated like skin, three basis functions are sufficient, but it is difficult to express the spectral reflectance of other existing objects by using three bases. For example, Journal of Optical Society America A Vol.3 No.10 (1986), page 1673, "Evaluation of Linear Models of Surfaces, Spectral Reflectance With
The "Small Numbers of Parameters" requires at least six dimensions to represent the spectral reflectance of many objects,
The fact that about eight dimensions are required is described.

Therefore, unless the subject is limited to the skin, higher-order information must be estimated from the three-dimensional information of the image input device. For example, when the basis function is set to 6, the dimension number 6-3 = 3, which is larger than the input dimension number 3, becomes a kernel (kernel) component and cannot be theoretically calculated. It is possible. The kernel component is a subspace in the basis coefficient space projected to the zero vector 0 = (0,0,0) ^T by the matrix B, that is, (a, b,...) , f) is a set of ^T.

[0073]

(Equation 9)

In order to linearly estimate higher-order information from lower-order information, a method using a Moore-Penrose generalized inverse matrix is generally used. In the Moore-Penrose generalized inverse matrix, a solution is obtained under the constraint that the norm of the data to be estimated is minimized.

However, this constraint is not an appropriate constraint for the present problem, and causes an error. Therefore, in the present invention,
A method for estimating data having a number of dimensions larger than the number of input dimensions 3 is disclosed. First, the solution (a1,
b1, c1, d1, e1, f1). There are innumerable solutions that satisfy (Equation 8). For example, a solution based on the general inverse matrix of Moore Penrose and other solutions may be used. It has been found from experiments that the estimation accuracy is high. Therefore, here, the case where the basic solution is used is taken as an example of the solution.

A method for calculating the basic solution will be described. First, (d, e, f), which is a number of dimensions larger than the number of input dimensions, is ignored, and it is assumed that the basis function has only the same degree as the input (Equation 10). The coefficients of a, b, c) are determined in advance.
The solution obtained with (a, b, c) and (d, e, f) = (0,0,0) satisfies (Equation 8) and is generally called a basic solution. This solution is called (a1,
b1, c1, d1, e1, f1).

[0077]

(Equation 10)

Next, a kernel component is estimated from the obtained (a1, b1, c1, d1, e1, f1). To make this estimate,
A precondition is that there is a statistical correlation between (a1, b1, c1, d1, e1, f1) and a kernel component.

A matrix D representing the correlation is created. In that way
explain about. First, let the dimension of the matrix B be R. Do
And the Nth non-negative symmetric matrix B^tThe dimension of B is also R, B ^tZero of B
Eigenspaces belonging to eigenvalues have N-R dimensions. B^tB is non-negative
Since it is a symmetric matrix, B^tRN normal consisting of eigenvectors of B
Of the orthogonal basis [vn] n = 1, .., N, n = N-R, .., N is the kernel (Kerr
F) the orthonormal basis of (B), while n = 1, .., R are kernels
(Kernel) An orthonormal basis of the orthogonal complement space of (B).
What is necessary is to find the matrix D that connects these two spaces.
Become.

A specific calculation procedure will be described with reference to FIG. In step 601, a basic solution (a1, b1, c1, d1, e1, f1) is obtained from scalar image data of a color chart composed of a plurality of colors.
In step 602, the spectral reflectance of each color on the color chart was measured with a colorimeter, and the spectral reflectance was represented by six basis functions (a, b, c, d,
The sixth-order data of e, f) is obtained in advance. This is (ar, br, cr, d
r, er, fr).

Next, in step 603, the eigenvector of B ^t B is obtained. From the maximum eigenvalue of this eigenvector, (a1, b1, c1, d1, e1, f1) is projected onto a space represented by three eigenvectors. The obtained coordinate values are set as (a1P, b1P, c1P). In step 604, (a1, b1, c1, d1, e1, f) is calculated from (ar, br, cr, dr, er, fr).
The sixth-order data obtained by subtracting 1) is projected onto a space represented by the fourth, fifth, and sixth eigenvectors from the maximum eigenvalue of the eigenvector. The obtained coordinate values are expressed as (d0P, e
0P, f0P). Finally, in step 605, (a1P, b1P, c1P) and (d
0P, e0P, f0P) is defined by the following equation (11).
Ask.

[0082]

[Equation 11]

The matrix D is expressed by (Equation 11) for each color of the color chart.
Can be determined so as to minimize the estimated squared error.

In the present embodiment, since the input device has three bands and the spectral reflectance is represented by a sixth-order basis, D is a square matrix. In this way, if a base having twice the number of bands as the number of bands of the input device is used, D can be a square matrix, and estimation can be easily performed. As the above modification, for example, (Equation 1)
Similar estimation can be made by using scalar RGB data as it is as the second term on the left side of 1).

According to the above-described method, even if the spectral reflectance of the object to be obtained or the number of dimensions of its basis function is larger than the number of bands of the image data, the number of dimensions up to the same dimension as the image data in the color data is obtained. The spectral reflectance of the object can be obtained with higher accuracy by estimating from the image data and statistically estimating the remaining dimensions from the already estimated color data.

Next, as another method of realizing the spectral reflectance estimating unit, a method of performing matrix conversion using terms of square or more or products of scalar image data will be described.

The Wiener estimation is used for estimating the spectral reflectance. In the Wiener estimation, it is not necessary to previously obtain a matrix B composed of the basis functions of the spectral sensitivity of the camera, the spectral distribution of the illumination, and the spectral reflectance of the object as in the method described above. Specifically, a matrix G (corresponding to an inverse matrix of the matrix B) for estimating the spectral reflectance of an object from scalar image data is converted into known input / output data, that is, a scalar image data of each color of a color chart and a basis of the spectral reflectance. Estimate from coefficients.

A specific calculation procedure will be described. The scalar image data and the vertical vector of the basis coefficient of the spectral reflectance are arranged horizontally for each color of the color chart to form a matrix.
If they are called E and F, (Equation 8) is rewritten as (Equation 12) using a matrix G for estimating the spectral reflectance of the object from the scalar image data.

[0089]

(Equation 12)

The matrix G of (Equation 12) can be obtained by (Equation 13) by Wiener estimation. In (Equation 13),
R means a cross-correlation matrix.

[0091]

(Equation 13)

In the Wiener estimation, in (Equation 12),
The vertical dimension of matrix E, that is, the scalar dimension,
Even when the vertical dimension of the matrix F, that is, the dimension of the basis coefficient of the spectral reflectance to be determined is large, it is possible to perform estimation as long as the properties of the color data given as a color chart are good.

However, when it is desired to perform the estimation with higher accuracy, the following may be performed. For example, if the scalar image data is three-dimensional and it is desired to obtain six basis coefficients of the spectral reflectance, in order to increase the input order to six dimensions, the square term of the scalar image data or the term of each product is used. Used. That is, the estimation matrix G may be obtained by Wiener estimation in the same manner as described above, using (Expression 14) instead of (Expression 8).

[0094]

[Equation 14]

As described above, the dimension of the input, which is originally three-dimensional, is expanded to polynomial and is regarded as virtually six-dimensional, so that the six-dimensional input data is converted to the six-dimensional output data (a, b,
, F) can be estimated.

As described above, according to the present invention, even if the spectral reflectance of the object to be obtained or the number of dimensions of its basis function is larger than the number of bands of the image data, the term or product of the square or more of the scalar image data is obtained. The spectral reflectance of the object can be obtained with higher accuracy by the matrix conversion using the term. This concludes the detailed description of the operation of spectral reflectance estimating section 105.

In this embodiment, the image input device for RGB three bands is described.
This method can be similarly applied not only to RGB but also to a larger number.

The image input device is not limited to a digital camera, but may be a scanner or data obtained by digitizing an analog output. Further, the present process may be applied to each image of a moving image.

The color estimating means 103 generates spectral reflectance image data 104 which is a color space independent of the device and illumination.
, And recorded in the image recording unit 105. However, wired (for example, IEEE1394, USB)
Alternatively, the spectral reflectance image data 104 may be transmitted using a wireless (for example, Bluetooth, IrDA) communication unit.

According to the present invention, the color estimating means is stored on a CD-ROM, the program stored on the CD-ROM is downloaded to a RAM on a PC, and the processing on the color estimating means is performed by a CPU on the PC. It is.

Alternatively, the color estimating means may be replaced by a RO in the image input device.
This is stored in M and causes the CPU in the image input device to perform the processing of the color estimating means. In this case, the image data output from the image input device is not a color space display unique to the input device, but is image data in a color space independent of the device or the device and the illumination. Since it is not necessary to install the color estimation means in the computer, it has an advantage that it can be easily handled by general users who are not familiar with computers and color conversion. However, if the RGB image data of the input device can be obtained by changing the mode, consistency with the conventional device may be obtained.

As described above, according to the present embodiment, the image data and colorimetric values of the color chart composed of a plurality of colors are obtained in advance, and the signal of the input device is obtained from the colorimetric values of the color chart. Calculate ideal image data predicted using an occurrence model and remove the nonlinearity of the image data by converting the image data of the color chart into the ideal image data by a neural network or a function and a matrix. Then, the color image data input from the input device is estimated by using the image data from which the nonlinearity has been removed, and estimating color data independent of the device or the device and the illumination with higher dimensions than the image data using the image data. And can be converted to color data independent of illumination with high precision.

[0103]

As described above, according to the present invention, there is provided an image processing method and an image input device for converting color image data input from an input device with high accuracy into a device-independent color space. be able to.

[Brief description of the drawings]

FIG. 1 is a block diagram of an image processing apparatus according to a first embodiment of the present invention;

FIG. 2 is a flowchart showing a learning procedure of the nonlinear elimination neural network according to the first embodiment;

FIG. 3 is a diagram for explaining a signal generation model according to the first embodiment;

FIG. 4 is a diagram showing a one-dimensional conversion function according to the first embodiment;

FIG. 5 is a flowchart showing a procedure for determining a nonlinear elimination function and a matrix according to the first embodiment;

FIG. 6 is a flowchart showing a procedure for estimating the spectral reflectance according to the first embodiment;

FIG. 7 is a flowchart showing the procedure of a color estimation process in Conventional Example 1;

FIG. 8 is a flowchart showing a procedure of a color estimation process in Conventional Example 2;

[Explanation of symbols]

 Reference Signs List 101 Image input means 102 Image data 103 Color estimation means 104 Spectral reflectance image data 105 Image recording means 106 Nonlinearity removal unit 107 Nonlinear removal processing parameter storage unit 108 Scalar RGB image data 109 Spectral reflectance estimation unit 110 Camera parameter storage unit

 ──────────────────────────────────────────────────続 き Continued on the front page F term (reference) 5B057 CA01 CA08 CA12 CA16 CB01 CB08 CB12 CB16 CC01 CE17 5C066 AA11 CA00 CA08 GA01 HA03 KE04 KE07 5C077 MM27 MP08 PP32 PP37 PP71 PQ12 PQ15 PQ22 TT09 5C079 HB01 MA02

Claims (9)

[Claims] When estimating color data independent of a device or a device and illumination from image data input by an image input device, the dimension of the image data is greater than the dimension of the color data independent of the device or the device and illumination. If the image data is low, the image data is estimated from the image data up to the same number of dimensions as the image data, and the remaining dimensions are estimated from the color data up to the same number of dimensions as the already estimated image data. Processing method. 2. When estimating color data independent of a device or a device and illumination from image data input by an image input device, the dimension of the image data is greater than the dimension of the color data independent of the device or the device and illumination. An image processing method characterized by estimating data of the remaining dimension up to the same dimension as the image data when it is low. 3. Estimating color data independent of a device or a device and illumination from image data input by an image input device, and converting the image data into color data independent of a device or device and illumination in a second order. An image processing method characterized by performing by matrix transformation extended to the above terms. 4. The apparatus according to claim 1, wherein said image data used for conversion into color data independent of a device or a device and illumination is obtained by removing nonlinearity of image data input from an image input device. 4. The image processing method according to any one of 1 to 3. 5. The method according to claim 1, wherein when removing the nonlinearity of the image data input by the image input device and then estimating the color data independent of the device or the device and the illumination, the nonlinearity is removed by a neural network. Image processing method. 6. Eliminating nonlinearity by a function and matrix conversion when estimating color data independent of the device or the device and illumination after removing the nonlinearity of the image data input by the image input device. An image processing method characterized by the following. 7. The image processing method according to claim 6, wherein the matrix used for removing the non-linearity is expanded to a second-order or higher-order term of the input image data. 8. The image data and colorimetric values of a color chart composed of a plurality of colors are obtained in advance, and an ideal image is predicted from the colorimetric values of the color chart using a signal generation model of an input device. Calculating non-linearity of the image data by converting the image data of the color chart into the ideal image data, and removing the non-linearity of the image data. An image processing method for estimating color data independent of illumination and illumination. 9. A non-linearity removing means for removing non-linearity of image data input from an image input device, and a device or a device which is higher in order than the image data from the non-linearity-removed image data and independent of illumination and An image processing apparatus comprising: color estimation means for estimating color data.