JP2002077660A - Image processor - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide an image processor to convert inputted images at high accuracy into a color space which is not dependent on apparatuses or on apparatuses and illumination. SOLUTION: In the processor, a specific object is recognized through images inputted from an image input device, and each pixel of the inputted images by selecting an estimating means based on recognized results is converted into color data which is not dependent on the apparatus or the apparatus and illumination, so that the inputted data can be converted at high accuracy.

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to an image processing device for converting color image data input from an image input device into a color space independent of the device or the device and the illumination with high accuracy.

[0002]

2. Description of the Related Art In recent years, various devices, such as scanners, digital cameras, printers, and displays, have been used as devices for processing color images. According to a technique for exchanging image data between these devices, color image data input from an input device is once converted into an independent color space independent of the device, and then converted into color image data to be output to an output device. There is something.

As described above, if the conversion between the signal of the image input device and the color space independent of the device is established, the data can be transferred to any image output device. There is no need to determine the number of color conversion processes.

If the color image data input from the image input device is converted into an independent color space not depending on the device but also on the illumination, an image under illumination different from the illumination at the time of image input can be output to the output device. You can also output from.

[0005] As an independent color space independent of the device,
XYZ tristimulus values specified by the International Standards Institute CIE, and L ^* a ^* b ^* color system, L ^* u ^* v ^* color system, or it is common to use a color appearance model such as CAM97s, Since the attribute values of the color appearance model are calculated from the XYZ tristimulus values, if the XYZ tristimulus values can be estimated from the signals of the image input / output device, the above color conversion becomes possible.

As a color space that does not depend on the device and the illumination, it is general to use the spectral reflectance of the subject. By integrating the spectral reflectance with the desired illumination, XYZ
Tristimulus values can be calculated.

Estimating the XYZ tristimulus values or the spectral reflectance of a subject from the color space of each image input / output device in this manner is called characterization.

The present invention relates to the characterization of an image input device such as a digital camera, a multispectral camera, and a scanner.

As a conventional characterization technique for an image input device, there is a color reproduction device described in Japanese Patent Application Laid-Open No. H11-89552. FIG. 12 shows a block diagram of a color reproducing apparatus disclosed in Japanese Patent Application Laid-Open No. 11-89552, which will be described below.

In FIG. 12, reference numeral 1201 indicates that the number of bands is three.
An image input device 1202 larger than the band 1202 is an input image input from the image input device 1201, an image processing device 1203 estimates spectral reflectance data of each pixel from the input image 1202, and outputs it as a spectral reflectance image 1204. An input image storage unit for storing the input image 1202, 12
05 is the image data of interest read out one pixel at a time from the input image storage unit 1204, 1206 is the image data of interest 120
XYZ estimator for estimating XYZ tristimulus value data from 5, 120
7 is a parameter storage unit for storing parameters used in the XYZ estimation unit 1206, and 1209 is an XYZ estimation unit 1206
XYZ data estimated in the above, 1210 is an image recording unit that sequentially stores the estimated XYZ data, 1211 is an XYZ image having the estimated XYZ data as a pixel value, and 1212 is the estimation process of the target image data is completed. Control signal that tells
Reference numeral 1213 denotes an overall control unit which receives a control signal 1212 and instructs to read out image data to be processed next from the input image storage unit 1204.

The operation of the apparatus will be described. First, the image input device 1201 increases the number of image bands from three general RGB bands to N bands (N> 3), and is called a multispectral camera or a hyperspectral camera. An N-dimensional input image 1202 per pixel input from the image input device 1201 is stored in the input image storage unit 1204, and in accordance with an instruction from the overall control unit 1213, one pixel at a time starting from the upper left pixel as target image data 1205. Output to XYZ estimation section 1206. In the XYZ estimating unit 1206, the parameter storage unit 120
The XYZ data is estimated using the matrix stored in 7 as the estimation parameter 1208. The obtained XYZ data 1209 is sequentially stored in the image recording unit 1210, and is output as an XYZ image when all pixels have been processed.

Next, the detailed operation of the XYZ estimator 1206 will be described. The XYZ estimator 1206 calculates XYZ tristimulus values from the N-th image data. The estimation matrix used at this time is calculated as follows.

First, the tristimulus value T and the image data vector I
Can be expressed as (Equation 1). In equation (1), E _O lighting matrix during observation, X is the matrix of the color matching function and a lateral vector, f is the spectral reflectance, E _m is the illumination matrix at the time of photographing, S is an image input device spectral sensitivity This is a matrix as a horizontal vector. By substituting (Equation 1) into the multiple regression matrix (Equation 2), (Equation 3) is obtained. In this (Equation 3), R _ff is
It is a correlation matrix of the spectral reflectance of the subject. By calculating in advance the correlation matrix of the spectral reflectance of the subject, which is a main component of the input image, and substituting it for R _ff in (Equation 3),
A matrix (Equation 3) for predicting XYZ tristimulus values can be obtained from the image data.

[0014]

(Equation 1)

[0015]

(Equation 2)

[0016]

(Equation 3)

As described above, in the characterization method of the conventional image input apparatus, a main subject of the input image is assumed in advance, and the image data and the spectral reflectance of the subject are measured in advance to obtain (Equation 3) XY from the image data shown
The matrix for estimating the Z tristimulus values is determined.

In the above method, in order to estimate color data with high accuracy, the number of bands of the sensor is set to be larger than that of the normal three RGB bands. This is called multibanding. There are two reasons for using a multi-band sensor.

First, even when the number of bases necessary to represent the spectral reflectance of the subject is large, the base coefficient can be uniquely calculated by setting the number of bands of the camera to the same number. Become like

Secondly, the possibility that subjects having different spectral reflectances have the same signal value of the image input device is reduced. This is called sensor metamerism. For example, even though the lips and the apple have the same red color, they are different objects and therefore have different spectral reflectances. Nevertheless,
In the case of an image input device of RGB 3 band, the same signal value may be obtained. On the other hand, if the number of bands of the sensor is increased, there is a high possibility that the signal values of these two objects differ.

However, the image input device is very expensive and large-scaled due to the multi-band configuration. Therefore, the XYZ tristimulus value and the spectral reflectance of the object can be obtained with an image input device having as few bands as possible. It is desirable to estimate.

[0022]

In the above conventional method,
A method of calculating XYZ tristimulus values and spectral reflectance of a subject with high accuracy from image data of an image input device having a small number of bands has not been solved.

The present invention has been made in view of the above points, and has as its object to provide an image processing method for converting color image data input from an input device into a color space that does not depend on the device and illumination with high accuracy. And

[0024]

SUMMARY OF THE INVENTION The present invention solves the problem that objects having different spectral reflectances have the same signal value of the image input device, and are consequently estimated to have the same spectral reflectance. That is, even if the number of sensor bands is small, a specific object is recognized from an image input from an image input device, and an estimating unit is selected and input based on the recognition result. The color data independent of the device or the device and the illumination is estimated from each pixel of the image data.

In this way, the recognized object can be estimated with extremely high accuracy and the other objects can be estimated with high accuracy, and color data independent of the device or the device and the illumination can be estimated from the input image data. .

[0026]

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS The invention according to claim 1 is an object recognizing means for recognizing an object from an input image, and a color conversion processing for selecting a different color conversion processing according to a recognition result of the object recognizing means. Selecting means, and a color conversion processing means for converting the color of the pixel of interest of the input image in the color conversion processing selected by the color conversion processing selection means,
This has the effect that the color of the image signal input from the image input device can be converted with high accuracy.

According to a second aspect of the present invention, in the color conversion processing, the input image is converted into color data independent of a device or a device and illumination.
The image processing apparatus described above has an effect that an input image can be converted into color data that does not depend on the apparatus or the apparatus and the illumination with high accuracy.

According to a third aspect of the present invention, the color conversion processing selecting means selects a color conversion processing for a pixel belonging to the object recognized by the object recognizing means by using a statistical property of a desired object. The image processing apparatus according to claim 2 has an effect that a recognized object can be color-converted with particularly high accuracy.

According to a fourth aspect of the present invention, the color conversion processing selection means selects a color conversion processing for pixels belonging to the object recognized by the object recognition means by multiple regression analysis. Item 2 is an image processing apparatus according to Item 2, which has an effect that a recognized object can be subjected to particularly high-precision color conversion.

The invention according to claim 5 is characterized in that the object recognizing means recognizes an object from the input image using shape information of the object known in advance. An image processing apparatus according to the above item (1), which has an effect that a desired object in an input image can be recognized with high accuracy.

According to a sixth aspect of the present invention, when the input image is a moving image, the object recognizing means recognizes the object from the input image using motion information of the object which is known in advance. An image processing apparatus according to any one of claims 1 to 4, having an effect that a desired object in an input image can be recognized with high accuracy.

Hereinafter, embodiments of the present invention will be described with reference to the drawings.

(Embodiment 1) In this embodiment, a digital camera having an output of three RGB bands is assumed as the most common image input system, and a subject which is data independent of a device and illumination from RGB image data is assumed. Is estimated.

At this time, a human face is recognized from the input image, and for the pixels belonging to the skin, means for estimating the spectral reflectance from the statistical properties of the RGB image data of the skin prepared in advance is selected. Then, means for estimating the spectral reflectance for general use is selected. As a result, the spectral reflectance of the skin can be estimated with extremely high accuracy, and the spectral reflectance of other pixels can be estimated with high accuracy.

FIG. 1 is a block diagram of an image processing apparatus for estimating a spectral reflectance image from RGB image data of the image input apparatus according to the first embodiment.

In FIG. 1, reference numeral 101 denotes an image input device for acquiring image data of three RGB bands, 102 denotes an input image, 103 denotes an image processing device for estimating a spectral reflectance image from an input image, which is a feature of the present invention, and 104 denotes an image processing device. Input image 10
2, an image recognition unit for recognizing a face;
A recognition image in which a part of the face recognized in 04 has a pixel value of 1 and a part other than the face has a pixel value of 0, 106 is a recognition image storage unit that stores a recognition image 105, and 107 is a recognition image according to an instruction from the overall control unit 121. It is a pixel value sequentially read from the recognition image 105 stored in the storage unit 106, and is a recognition signal indicating whether or not the image data of interest is a part of the recognition object.
08 is a processing selection unit for selecting a method of estimating the spectral reflectance according to the attention recognition signal 107, and 109 is a processing selection unit 10.
8, a processing selection signal indicating the processing method selected by
Reference numeral 10 denotes a parameter storage unit that stores parameters for skin and non-skin for estimating the spectral reflectance from a signal of the image input device calculated in advance, and 111 denotes a parameter from the parameter storage unit 110 according to the processing selection signal 109. Estimated parameters used for estimating the read spectral reflectance, 112
Is an input image storage unit that stores the input image 102; 113 is target image data that is a pixel value of each pixel read from the input image storage unit 112 according to an instruction of the overall control unit 121;
Reference numeral 114 denotes a non-linearity removing unit for removing non-linearity from the image data of interest 113, and reference numeral 115 denotes image data from which the non-linearity obtained by the non-linearity removing unit 114 has been removed, which will be hereinafter referred to as scalar image data. 116
Is the scalar image data 1 using the estimation parameter 111
Spectral reflectance estimating unit for performing spectral reflectance estimation from 15, 11
7 is the estimated spectral reflectance data of the image data of interest, 1
Reference numeral 18 denotes an image recording unit that records spectral reflectance data, 119
Is a spectral reflectance image output as a result of the processing of the image processing apparatus 103, 120 is a control signal indicating that the spectral reflectance estimation of the target image data 113 has been completed, and 121 is processing control of each pixel of the input image 102. This is the overall control unit that performs the following.

The operation of the image processing apparatus shown in FIG. 1 will be described.

First, a subject is imaged by the image input device 101.

FIG. 2 is an example of an input image 102 input by the image input device 101. FIG. 2 shows a human upper body, which is a main subject, and a round object in the background. This object is a beige color close to skin color.

Next, the input image 102 is sent to the image recognition unit 104.
, And the position of the face is recognized from the input image 102. Then, a recognized image in which the recognized face part has a pixel value of 1 and a part other than the face has a pixel value of 0 is created and output. FIG. 3 is an example of the recognition image 105. The image recognizing unit 104 does not determine whether the face is a face or a face based on the color. Instead, the image recognizing unit 104 uses shape and position information such as eyes, nose, and mouth as described later, or in a continuous image in the case of a moving image. The position of the human face is recognized from the motion information.

The input image 102 is also stored in the input image storage unit 112. Then, in accordance with an instruction from the overall control unit 121, pixel values are sequentially read out from the upper left pixel to the non-linear removal unit 114 as the target image data 113.

The non-linearity removing unit 114 converts the image data of interest 113 into scalar image data 115 and outputs it to the spectral reflectance estimating unit 116.

The overall control unit 121 sequentially reads out recognition signals corresponding to the image data of interest 113 from the recognition image 105 stored in the recognition image storage unit 106 as the attention recognition signal 107. The processing selection unit 108 selects a different spectral reflectance estimation method depending on whether the target recognition signal 107 is 1 or 0, and outputs the selected signal as the processing selection signal 109. In the parameter storage unit 110, when the processing selection signal is 1 in advance, that is, when the spectral selection parameter is 0 when the processing selection signal is 0,
That is, since the parameters for estimating the spectral reflectance in cases other than the skin are stored, the parameter for estimation 111 is output to the spectral reflectance estimating unit 116 in accordance with the processing selection signal 109.

The spectral reflectance estimating section 116 estimates the spectral reflectance from the scalar image data 115 by using the estimation parameter 111 and outputs it as spectral reflectance data 117. The obtained spectral reflectance data 117 is recorded in the image recording unit 118. When the series of processes is completed for the image data of interest 113, a control signal 120 indicating the end of the process is output to the overall control unit 121. Upon receiving the control signal 120, the overall control unit 121 reads out image data to be processed next from the input image storage unit 112 as attention image data 113, and checks whether the attention image data is a face from the recognition image storage unit 106. , Is read out.

The above processing is repeated for all the pixels of the input image 102.

Next, an example of a detailed operation of the nonlinear elimination unit 106 will be described.

The processing in the non-linearity removing section 114 can be performed using, for example, a multi-layer perceptron. The multi-layer perceptron is a kind of neural network, in which weights and thresholds, which are parameters of neurons, are learned in advance, and estimation is performed using the parameters obtained by the learning. The multi-layer perceptron used here has an input as image data and an output as data obtained by removing non-linearity from the image data, that is, scalar image data.

The parameter learning procedure of such a multi-layer perceptron will be described with reference to FIG.
In step 401, image data and a colorimetric value (spectral reflectance) of a color chart composed of a plurality of colors are obtained in advance. Next, in step 402, ideal image data predicted using the signal generation model of the image input apparatus 101 is calculated from the colorimetric value (spectral reflectance) of the color chart. Next, in step 403, weights and thresholds are learned using the image data 102 as input data and ideal image data as teacher data.

The method of calculating ideal image data in step 402 will be described in detail.

The ideal image data is stored in the image input device 101
Can be obtained by substituting the colorimetric value (spectral reflectance) of the color chart into the signal generation model. The signal generation model includes the spectral reflectance R (λ) of the subject, the spectral distribution S (λ) of the illumination, and the spectral sensitivities C _R (λ) and C of the three RGB bands of the image input apparatus 101.
_{Using G} (λ) and C _B (λ), it can be written by (Equation 4). It is assumed that the spectral sensitivity of the image input device 101 and the spectral distribution of illumination are known.

[0051]

(Equation 4)

R ′, G ′, B ′ obtained by substituting the spectral reflectance of the color chart for R (λ) in (Equation 4) become ideal image data.
FIG. 5 is an explanatory diagram of the signal generation model of (Equation 4). FIG.
, 501 is the spectral reflectance of the color chart to be substituted into the signal generation model, 502 is the spectral sensitivity characteristic of the known image input device 101, 503 is the illumination characteristic at the time of capturing the image data 102,
504 is a signal generation model of (Equation 4), and 505 is ideal image data of a color chart. As shown in FIG. 5, the spectral sensitivity characteristic 502, the illumination characteristic 503, and the spectral reflectance 501 of the image input apparatus 101 are given to the signal generation model, that is, (Equation 4), so that the ideal image of the object is obtained. Data 505 is obtained.

The operation of the non-linearity removing unit 114 performed using the neural network of the weight and the threshold value learned as described above is performed by inputting the target image data 113 to the learned neural network and outputting the neural network. Scalar image data 115 is obtained.

The operation of the nonlinearity removing section 114 has been described above.

Next, the processing selecting section 10 which is a feature of the present invention.
8. Parameter storage unit 110, spectral reflectance estimating unit 116
The operation of will be described in detail.

The feature of the present invention is that the face is recognized by the image recognition unit 104, and as a result of the recognition, different spectral reflectance estimation methods are used for pixels belonging to the skin and pixels belonging to other than the skin.

The estimation matrix for the skin or for the area other than the skin is stored in the parameter storage unit 110 in advance, and when the processing selection unit 108 receives the attention recognition signal 107 which is the result of recognition by the image recognition unit 104, If the attention recognition signal 107 is 1 among the plurality of estimation matrices stored in the parameter storage unit 110, that is, if it is a face, the estimation matrix for skin is read, and if the attention recognition signal 107 is 0, that is, If it is not a face, a general-purpose estimation matrix for other than the skin is passed to the spectral reflectance estimating unit 116. The spectral reflectance estimating unit 116 estimates the spectral reflectance of the subject from the scalar image data 115 using the designated matrix.

The spectral reflectance is estimated by calculating the left side (R ′,
When the scalar image data 115 is substituted for G ′, B ′), a problem arises in obtaining the spectral reflectance R (λ).

In (Equation 4), since the scalar image data and the spectral reflectance have a linear relationship, (Equation 5) is obtained by rewriting (Equation 4) into a discrete matrix expression.

[0060]

(Equation 5)

The left side of (Equation 5) is the scalar image data 11
5, (R1, R2,..., Rn) ^T is a discrete expression of the spectral reflectance of the object, and each component represents the reflectance at each wavelength of, for example, 400 nm to 700 nm every 10 nm. Matrix A is image input device 1
It is a matrix determined by the spectral sensitivity of 01 and the spectral distribution of illumination.

Using (Equation 5), scalar image data 1
The problem of estimating the spectral reflectance R of the subject from 15 is a linear inverse problem. If the image data is, for example, RGB three bands,
(R1, R2,..., Rn) It is difficult to estimate ^T with a dimension number much larger than three.

As a method for solving this problem, for example,
There is a method of expressing the spectral reflectance of a subject by a basis function having a lower order than n. According to this method, the number of dimensions of data to be obtained can be reduced.

For example, the basis functions are defined as three-dimensional O ₁ (λ), O ₂ (λ),
If O ₃ (λ), (Equation 4) can be rewritten as (Equation 6), and the data to be estimated is a cubic vector of (a, b, c). Therefore, (Equation 5) can be rewritten as (Equation 7). In Equation 7, the matrix B is a matrix determined by the spectral sensitivity of the image input apparatus 101, the spectral distribution of illumination, and the basis function.

[0065]

(Equation 6)

[0066]

(Equation 7)

As the basis functions, for example, those described in “Measurement and Analysis of Object Reflectance Spectra” by Wihel, pp. 4 to 9, Magazine Color Research and Application, Volume 19, Number 1, 1994, can be used. . Since these basis functions are calculated by measuring many natural subjects and artificial subjects, they can be said to be very general-purpose basis functions.

Alternatively, the spectral reflectance of the Macbeth chart may be measured, the obtained spectral reflectance may be subjected to principal component analysis, and the principal component vectors of the top several components may be used as basis functions.

Since (Equation 7) obtained in this way does not particularly limit the subject, the accuracy of the estimated spectral reflectance is almost the same for all subjects. Also,
Since the matrix B is a square matrix, a unique solution can be obtained.

However, since the basis function cannot represent all the objects in the third order, the expression is based on the assumption that the objects are completely diffused, and furthermore, since noise is not considered, a highly accurate Estimation is not possible.

On the other hand, when the subject is limited, rather than solving the signal generation model (Equation 4) as described above, multiple regression analysis or neural regression for estimating from the statistical properties of image data and spectral reflectance data relating to the subject is performed. A network provides a more accurate solution. However, in this case, extremely accurate estimation can be obtained for a limited subject, but the error becomes extremely large for other subjects.

Therefore, in the present invention, a highly accurate estimation method is prepared in advance for the specified subject, that is, the skin, using a multiple regression analysis, a neural network, or the like. Then, the image recognition unit 115 determines whether each pixel in the input image is a face, and if it is determined that the pixel is a part of the face, performs estimation using the multiple regression analysis or the neural network. , If it is determined that it is not a face,
Estimation according to (Equation 7) is performed.

In this manner, extremely high-precision estimation can be performed for the specified subject, and an almost accurate solution can be obtained for other subjects.

Next, when the target pixel is a part of the face,
That is, the attention recognition signal 107 and the processing selection signal 10
An estimation method using multiple regression analysis or a neural network when 9 is 1 will be described.

As described above, a plurality of estimation methods are conceivable, such as a multiple regression analysis and a neural network. Here, a method based on the multiple regression analysis will be described with reference to FIG.

In step 601, a plurality of image data of a subject belonging to a category is obtained from an image input unit, and spectral reflectance data is obtained using a colorimeter. This is shown in FIG. In step 602, the image data is converted into scalar image data by the same processing as that of the nonlinear elimination unit 106.
In step 603, the spectral reflectance data is converted into basis coefficients. As for the basis function, the same as in (Equation 7) may be used, or the spectral reflectance of a previously specified subject (skin in the present embodiment) may be subjected to principal component analysis to obtain a higher number. The principal component vector of the component may be used. In the latter case, since the basis functions are also specialized for the skin, the expression accuracy is higher. In step 604, a matrix for estimating the basis coefficient of the spectral reflectance data from the image data is created by multiple regression analysis.

The details of the procedure 604 will be described. Step 602
X is a matrix in which a plurality of scalar image data (vertical vectors) calculated in the above are arranged horizontally by the number of data, and R is a matrix in which base coefficients (longitudinal vectors) of the spectral reflectances calculated in the step 603 are arranged horizontally by the number of data. In other words, the matrix M for estimating the basis coefficient from the image data is represented by (Equation 8). In (Equation 8), R _xx represents a correlation matrix. For example, R _RX is a correlation matrix of T and X, and is defined by (Equation 9).

[0078]

(Equation 8)

[0079]

(Equation 9)

M in (Equation 8) is a multiple regression matrix determined so as to minimize the error between the estimated basis coefficient and the basis coefficient calculated in step 603.

Using the matrix M obtained by the above procedure, from the arbitrary scalar image data x = (R ′, G ′, B ′) ^t , the basis coefficient r = (a, b, c) of the spectral reflectance ^t is estimated by (Equation 10).

[0082]

(Equation 10)

The spectral reflectance is calculated from the obtained basis coefficients. This concludes the description of a method for creating an estimation matrix for skin by multiple regression analysis.

Here, the estimation matrices (Equation 8) and (Equation 10) for the specified subject (for the skin) and the general estimation matrix (Formula) used when it is determined that the subject is other than the specified subject (other than the skin). The relationship 7) will be described.

When a matrix in which the sensitivities of the respective bands of the image input apparatus 101 are arranged as vertical vectors is C and a matrix in which the basis functions are arranged as vertical vectors is P, (Equation 4) Can be written as (Equation 11). Here, X is a matrix in which a plurality of scalar image data (vertical vectors) calculated in step 602 are arranged horizontally by the number of data, and R is a base coefficient (vertical vector) of the spectral reflectance calculated in step 603 by the number of data. It is a matrix arranged side by side.

[0086]

[Equation 11]

In (Equation 10), (Equation 8) and (Equation 11)
Is substituted into (Equation 12).

[0088]

(Equation 12)

In (Equation 12), if the subject is not limited, the correlation matrix R _RR of the basis coefficient of the spectral reflectance can be regarded as a unit matrix, so that (Equation 13) is obtained. In (Equation 13), + means the general inverse matrix of Moore Penrose.

If the basis function is used up to the same third order in the number of bands of the image input device, the general inverse matrix can be interpreted as a normal inverse matrix because P ^t C is a symmetric matrix.

[0091]

(Equation 13)

It should be noted that if the basis functions used in (Equation 8) and (Equation 7) are the same, (Equation 13) is equivalent to (Equation 7). That is, in the estimation matrices (Equation 8) and (Equation 10) for each category, the case where the correlation matrix of the basis coefficients is a unit matrix corresponds to the signal generation model solution of (Equation 7).

Next, the image recognition unit 10 which is a feature of the present invention is described.
4 will be described in detail.

The image recognizing section 104 recognizes the position of the face from the input image 102, and sets the portion corresponding to the face to 1 and the rest to 0.
And the recognition image 105 having the pixel value is output.

As a method for the image recognition unit 104 to recognize the position of the face, for example, there is a method using template matching. The procedure of this recognition method will be described with reference to FIG.

First, in step 801, R, R
The G and B values are averaged and the color image is converted to an 8-bit black and white image. Next, in step 802, m × n
An image is created with an average value of pixels for each block as a pixel value. This is called a block image. Step 803
Then, from the upper left of the block image obtained in step 802, a correlation with a plurality of face templates prepared in advance is obtained,
Find the position with the highest correlation value. FIG. 9 is an example of a face template. In step 804, the maximum correlation value calculated in step 803 is compared with a predetermined threshold value. If the maximum correlation value is larger than the threshold value, it is determined that a face is present at the position indicating the maximum correlation value. In step 805, in the input image 102,
Pixel values that are close to the pixel value RGB of the position determined to have a face and that are adjacent to the position determined to have a face are recognized as human faces and skin, and merged. An image having a pixel value of 1 and other pixel values of 0 is output as a recognition image 105.

Alternatively, if the input image 102 is a moving image, an inter-frame difference is obtained, it is determined that there is a human at a position where the difference value is large, and the pixel value RG at the position where it is determined that there is a human.
A pixel having a pixel value close to B and adjacent to the position where it is determined that a human is present is recognized as a human face and skin, and its pixel value is set to 1, and the other pixel values are set to 0. The image described as above may be used as the recognition image 105.

The description of the operation of each part of the image processing apparatus of FIG. 1 has been completed.

The effects of the processing by the image processing apparatus will be described.

For example, in the example of the input image shown in FIG.
Since the background object is beige and has a color close to human skin color, the image signal from the image input device is almost the same. However, human skin has a spectral reflectance such that the reflectance slightly drops at around 500 nm due to absorption by hemoglobin, while the object is an artificial object, and there is no such reduction. The two objects originally have different spectral reflectances.

According to the conventional image processing apparatus, these two objects have different spectral reflectances, but if the image input device outputs the same image signal value, the two objects have two different spectral reflectances. Since the object can no longer be distinguished, the same spectral reflectance is estimated.

On the other hand, according to the image processing apparatus of the present invention, as shown in FIG. 3, in the recognition image, the two objects are recognized as being different from each other.
And the estimation method represented by (Equation 10) is applied, and the estimation method represented by (Equation 7) is applied to an object. That is, both spectral reflectances can be accurately reproduced.

FIG. 1 shows a first application example of the image processing apparatus of FIG.
0 is shown. 10, reference numeral 1001 denotes an image input device 1
Reference numerals 01 and 1002 denote image processing devices 118 and 1003, a display, and 1004, a printer. Image input device 1
In step 001, an image of a subject is input, and the image processing apparatus 100
2, the spectral reflectance image 119 is output.
The spectral reflectance image 119 is passed to a display 1003 or a printer 1004, converted into a signal of each device, displayed on the display 1003, or output from the printer 1004. The conversion of the signal from the spectral reflectance image 119 to each device is performed on the display 10.
02 or the CPU inside the printer 1003, or may be performed by an arithmetic processing device such as a separate personal computer and then transferred to the display 1002 or the printer 1003. In addition, all processing of the image processing apparatus 1002 is performed by the image input apparatus 1001.
It may be performed by the internal CPU.

In such an image input / output device of FIG. 10,
Once the input image is converted into color data that does not depend on the device or the device and illumination, and then converted into signals from each output device, the true color of the subject can be output to a display or printer. become. In particular, important colors that are easily noticed when observing an image, such as skin colors, can be reproduced with higher accuracy.

FIG. 11 shows a remote medical image processing apparatus which is a second application example of the image processing apparatus shown in FIG. 11
01 is an image input device, 1102 is an image processing device of the present invention, 1103 is a display, 1104 is a doctor, 110
5 is a patient. The doctor 1104 has a display 1103
At the same time, it exists in a remote place away from the patient 1105.

An image of the patient 1105 is input to the image input device 110.
1, the input image is converted into a spectral reflectance image by the image processing device 1102, transmitted, and then converted into a signal for display using the color characteristics of the display 1103 and displayed. Alternatively, after transmitting the input image, the doctor 1104 may convert the image into a spectral reflectance image by the image processing device 1102.

For medical use, it is necessary to know the exact color of human skin. For this reason, the image processing device acquires an accurate spectral reflectance image of the skin and reproduces the exact color of the patient's skin, thereby erroneously diagnosing and erroneously determining the patient's complexion, skin disease, and other medical conditions of the affected part. Can be reduced.

Although the present embodiment has been described with reference to an image input unit for three RGB bands, the filter is set to RG.
The present method can be similarly applied to not only B but also to a larger number. Further, the image input unit may be not only a digital camera but also a scanner or data obtained by digitizing an analog output, or the present process may be applied to each image of a moving image.

Further, according to the present invention, the operation of the image processing apparatus in FIG. 1 is stored in a CD-ROM, a program stored in the CD-ROM is downloaded to a RAM on a PC, and color estimation is performed on a CPU on the PC. The processing of the means is performed. Alternatively, it is stored in a ROM in the image input device, and causes the CPU in the image input device to perform the processing of the means. In this case, the image data output from the image input device is not a color space display specific to the input device, but is image data in a color space independent of the device or the device and the illumination. Since it is not necessary to install the color estimation means in the computer, it has an advantage that it can be easily handled by general users who are not familiar with computers and color conversion. However, if the RGB image data of the input device can be obtained by changing the mode, consistency with the conventional device may be obtained.

As described above, according to the present embodiment, a specific object is recognized from an image input from an image input device, and an estimating unit is selected based on the recognition result, and each of the input image data is recognized. By estimating the color data that does not depend on the device or the device and the illumination from the pixels, the color data that does not depend on the device or the device and the illumination can be estimated with high accuracy even if the number of bands of the input image is small.

[0111]

As described above, according to the present invention, there is provided an image processing apparatus for converting color image data input from an image input apparatus into a color space which does not depend on the apparatus or the apparatus and the illumination with high accuracy. can do.

[Brief description of the drawings]

FIG. 1 is a block diagram of an image processing apparatus according to a first embodiment of the present invention;

FIG. 2 is a diagram showing an example of an input image 102 according to the first embodiment;

FIG. 3 is a diagram showing an example of a recognition image 105 according to the first embodiment;

FIG. 4 is a diagram showing a learning procedure of the nonlinear elimination neural network according to the first embodiment;

FIG. 5 is an explanatory diagram of a signal generation model according to the first embodiment;

FIG. 6 is a diagram showing a procedure for creating an estimation matrix for a specific object according to the first embodiment;

FIG. 7 is an explanatory diagram of acquisition of image data and a colorimetric value of a specific object.

FIG. 8 is a diagram illustrating an operation procedure of the image recognition unit according to the first embodiment;

FIG. 9 is a diagram showing an example of a face template according to the first embodiment;

FIG. 10 is a configuration diagram of an image input / output device of a first application example of the image processing device according to the first embodiment;

FIG. 11 is a configuration diagram of a medical image processing apparatus according to a second application example of the image processing apparatus according to the first embodiment;

FIG. 12 is a block diagram of an image processing apparatus according to a conventional example.

[Explanation of symbols]

 Reference Signs List 101 Image input device 102 Input image 103 Image processing device 104 Image recognition unit 105 Recognition image 106 Recognition image storage unit 107 Attention recognition signal 108 Processing selection unit 109 Processing selection signal 110 Parameter storage unit 111 Estimation parameter 112 Input image storage unit 113 Attention image Data 114 Nonlinear removal unit 115 Scalar RGB image data 116 Spectral reflectance estimating unit 117 Spectral reflectance data 118 Image recording unit 119 Spectral reflectance image 120 Control signal 121 Overall control unit

──────────────────────────────────────────────────続 き Continued on the front page (51) Int.Cl. ⁷ Identification symbol FI Theme coat ゛ (Reference) H04N 9/64 H04N 1/46 Z F-term (Reference) 5B057 CA01 CA08 CA12 CB01 CB08 CB12 CE18 DA11 DC09 5C066 AA11 CA27 DD01 EE01 GA01 KD06 KE01 KE04 KM02 5C077 MM27 MP08 PP31 PP32 PP36 PP55 PP65 PP71 PQ08 PQ12 PQ15 PQ22 SS01 TT09 5C079 HB01 HB05 HB08 HB09 HB11 LA01 LA31 LB02 MA13 NA03 NA29 5L096 AA02 FA

Claims (6)

[Claims] An object recognizing means for recognizing a desired object from an input image, a color conversion processing selecting means for selecting a different color conversion processing according to a recognition result of the object recognizing means,
An image processing apparatus comprising: a color conversion processing unit configured to convert a color of a pixel of interest of the input image by a color conversion process selected by the color conversion process selection unit. 2. The image processing device according to claim 1, wherein the color conversion process is a process of converting an input image into color data that does not depend on a device or a device and illumination. 3. The color conversion processing selecting unit selects a color conversion process for a pixel belonging to an object recognized by the object recognition unit by using a statistical property of the object. 3. The image processing device according to 1 or 2. 4. The image according to claim 1, wherein said color conversion processing selection means selects color conversion processing for pixels belonging to the object recognized by said object recognition means by multiple regression analysis. Processing equipment. 5. The image processing apparatus according to claim 1, wherein the object recognition unit recognizes the object from the input image using shape information of the object that is known in advance. . 6. The apparatus according to claim 1, wherein the object recognizing means recognizes the object from the input image using motion information of the object which is known in advance when the input image is a moving image. An image processing device according to any one of the above.