Method for obtaining high-precision chromaticity and spectrum image
Technical Field
The invention relates to the field of chromaticity and spectrum information, in particular to a method for acquiring high-precision chromaticity and spectrum images.
Background
Due to the metamerism phenomenon, the chromaticity information of the object acquired by the traditional color camera is difficult to represent the original information of the object reliably, and the spectral reflectance is a physical quantity which is irrelevant to equipment and a capture light source, so that the original information of the object can be represented. Therefore, accurate acquisition of the spectral reflectance of the object can realize high-precision color measurement, and can faithfully reproduce the color appearance of the object under any light source. The multispectral camera can improve the accuracy of object chromaticity information and spectral reflectance acquisition due to the fact that more channels are added for imaging. For a multispectral camera, the chromaticity and spectral images of the scene may be acquired based on the chromaticity and spectral characterization models, respectively. The traditional multispectral camera has a complex and bloated device structure due to the existence of the color filter wheel, and the multispectral image acquisition speed is low due to the fact that the color filters of different wave bands are switched through rotation of the color filter wheel. The multispectral camera based on the multispectral color filter array has compact structure because the color filter is closely attached to the surface of the sensor, improves the portability of the multispectral camera, can obtain images of all wave bands through one-time exposure, and greatly improves the acquisition efficiency of the multispectral images.
Under the environment of controllable light, the multispectral camera is effectively applied, but the chromaticity and spectrum characterization models are related to light sources, namely the chromaticity and spectrum characterization models of the same equipment under different light sources are different, so that the acquisition accuracy of the chromaticity and spectrum images of the multispectral camera under an unknown light source is restricted.
Disclosure of Invention
The invention mainly solves the problem of low acquisition precision of the chromaticity and the spectral image of the multispectral camera based on the multispectral color filter array under an unknown light source, and provides a method for acquiring the chromaticity and the spectral image with high precision based on the multispectral color filter array.
The technical scheme adopted by the invention for solving the technical problems is that the method for acquiring the high-precision chromaticity and spectrum image comprises the following steps:
s1: multispectral image I based on scene under multispectral color filter array instantaneous acquisition unknown light sourcex∈RH×W×1H and W are the height and width of the image, respectively;
s2: recovering a full-channel response value by using a multispectral demosaicing algorithm;
s3: multispectral image I under unknown light source by using spectrum adaptation algorithmxSwitching to a reference light source;
s4: respectively establishing a chrominance characterization model and a spectral characterization model based on a multispectral color filter array under a reference light source;
s5: the chromaticity and spectral images were calculated under a reference light source.
The multispectral image of a scene under an unknown light source can be collected instantly, and the acquisition accuracy of object colors and spectrums is improved.
As a preferable scheme of the foregoing scheme, in step S2, the sensor response values of the other n-1 channels of each pixel point select the response value of the corresponding channel of the pixel point nearest to the sensor response value, each pixel point forms an n × 1-dimensional response value vector, the entire color filter array forms an n × n-dimensional response value matrix, and the multispectral image I is obtained by integrating the response values of the n × n-dimensional response value matrix and the corresponding channel of the pixel point nearest to the pixel pointxHas a dimension of H × W × n, i.e. Ix∈RH×W×nAnd n is the number of color filters in the multi-spectral color filter array.
As a preferable mode of the above, the step S3 includes the steps of:
s31: calculating illuminant color filter mapping and conversion vectors under unknown light source and reference light source
Wherein, Fx∈R1×nAnd Fc∈R1×nRespectively representing the illuminant color filter mapping under an unknown light source and a reference light source, T is a transposed symbol,/represents the division of corresponding elements of a vector, R is belonged to R1×nRepresenting a translation vector, Lx∈Rm×1And Lc∈Rm×1Spectral power distribution of an unknown light source and a reference light source respectively, m is the sampling number of the spectral power distribution on the wavelength, and S belongs to Rm×nIs the spectral sensitivity of the multi-spectral color filter array;
s32: multispectral image IxConversion into a multispectral matrix Ix1;
S33: combining a multi-spectral matrix Ix1Conversion from unknown source to reference source
Wherein r isd∈Rn×nA diagonal matrix representing the conversion vector r, diag () representing the conversion of the vectorIs a diagonal matrix, Ic∈Rp×nIs a multispectral matrix under the reference light source after conversion.
As a preferable embodiment of the foregoing solution, in the step S4, the chromaticity characterization and spectrum characterization model is
Wherein Q ∈ Rk×nFor training the multi-channel response values of the samples, each row represents a sample, each column represents the response value of one channel, superscript + represents the pseudo-inverse of the matrix, Mu∈Rn×qFor the chrominance transformation matrix, Mv∈Rn×mFor the spectral transformation matrix, U ∈ Rk×qThe colorimetric values of the training samples obtained under the reference light source are obtained, k is the number of the training samples, q is the dimensionality of the colorimetric values, and V epsilon Rk×mTo obtain the spectral reflectance of the training sample under the reference light source, m is the number of samples of the spectral reflectance over the wavelength, as well as the number of samples of the spectral power distribution over the wavelength.
As a preferable mode of the above, the step S5 includes the steps of:
s51: characterizing model M with chrominanceuAnd spectral characterization model MvAnd converting into multispectral matrix I under reference light sourcecObtaining a chromaticity and spectral image matrix
Wherein, Iu∈Rp×qFor a chrominance image matrix of a scene, each line representing a pixel and each column representing one dimension of the chrominance value, Iv∈Rp×mA spectral image matrix for the scene, each row representing a pixel, each column representing a spectral reflectance at a respective sampling wavelength;
s52: respectively mixing IuAnd IvDimension of (d) is converted into H x W x q and H x W x m, i.e. Iu∈RH×W×q,Iv∈RH×W×mAnd obtaining a final chrominance image and a final spectrum image of the scene.
As a preferable solution of the above solution, in the step S32, the multispectral image I is processedxIs converted from H multiplied by W multiplied by n to q multiplied by n, q is H multiplied by W, and a multispectral matrix I is obtainedx1∈Rq×n。
The invention has the advantages that: the method can improve the accuracy of obtaining scene chromaticity and spectral images under an unknown light source and promote the effective application of the multispectral imaging technology in the unknown light environment.
Drawings
Fig. 1 is a schematic flow chart of a method for obtaining high-precision chromaticity and spectrum images in the embodiment.
Fig. 2 is a schematic diagram illustrating an effect of the demosaicing algorithm in the embodiment.
Fig. 3 is a schematic flow chart of converting a multispectral image under an unknown light source to a reference light source according to the embodiment.
Detailed Description
The technical solution of the present invention is further described below by way of examples with reference to the accompanying drawings.
Example (b):
the method for acquiring high-precision chrominance and spectral images in the embodiment is shown in fig. 1 and comprises the following steps:
s1: in this embodiment, a multispectral color filter array including 8 channels is taken as an example, the spectral sensitivity of the multispectral color filter array for 8 channels is gaussian, the full width at half maximum of the peak value is about 20nm, and the peak response wavelengths are respectively 420nm, 460nm, 500nm, 540nm, 580nm, 620nm, 660nm and 700 nm. 96 color blocks in an X-Rite color checker Digital SG color card are used as training samples, and black, white and gray blocks which are repeated at four sides in the color card are abandoned. Based on multispectral color filter array, multispectral image I of scene under unknown light source is instantly collectedx∈R512×512×1512 and 512 are the height and width of the image, respectively, and the number of color filters in the multispectral color filter array is 8, then a multispectral image of 8 channels is formed, multispectral image IxDimension of 512X 512 x 1. At this time, each pixel point only contains the sensor response value of one channel, and the whole color filter array forms a 8 × 1-dimensional response value vector.
S2: recovering a full-channel response value by using a multispectral demosaicing algorithm; taking the nearest neighbor interpolation algorithm as an example, the response values of the sensor response values of the other 7 channels of each pixel point select the response value of the corresponding channel of the pixel point nearest to the sensor response value, as shown in fig. 2, at this time, each pixel point forms a 8 × 1-dimensional response value vector, the whole color filter array forms an 8 × 8-dimensional response value matrix, and the multispectral image I is obtainedxHas dimensions of 512X 8, i.e. Ix∈R512×512×8。
S3: multispectral image I under unknown light source by using spectrum adaptation algorithmxSwitching to a reference light source; the specific steps are shown in fig. 3, and comprise the following steps:
s31: calculating illuminant color filter mapping and conversion vectors under unknown light source and reference light source
Wherein, Fx∈R1×8And Fc∈R1×8Respectively representing the illuminant color filter mapping under an unknown light source and a reference light source, T is a transposed symbol,/represents the division of corresponding elements of a vector, R is belonged to R1×8Representing a translation vector, Lx∈R31×1And Lc∈R31×1Spectral power distribution of an unknown light source and a reference light source respectively, 31 is the sampling number of the spectral power distribution on the wavelength, and S belongs to R31×8Is the spectral sensitivity of the multi-spectral color filter array;
s32: multispectral image IxConversion into a multispectral matrix Ix1In particular, the multispectral image IxIs converted from 512 × 512 × 8 to 262144 × 8, and 262144 is 512 × 512, so as to obtain the multispectral matrix Ix1∈R262144×8. At this time, the multispectral image matrix IxEach row of (a) represents a pixel, and each column represents a response value of a channel;
S33: combining a multi-spectral matrix Ix1Conversion from unknown source to reference source
Wherein r isd∈R8×8Representing a diagonal matrix of the converted vector r, diag () representing the conversion of the vector into a diagonal matrix, Ic∈R262144×8Is a multispectral matrix under the reference light source after conversion.
S4: respectively establishing a chrominance characterization model and a spectral characterization model based on a multispectral color filter array under a reference light source; based on the multispectral color filter array, obtaining a colorimetric value U epsilon R of a training sample under a reference light source96×3The chromaticity values are CIE XYZ tristimulus values, 96 is the number of training samples, and 3 is the dimension of the chromaticity values. Obtaining the spectral reflectance V epsilon R of the training sample under a reference light source96×31Where 31 is the number of samples of spectral reflectance over wavelength, as well as the number of samples of spectral power distribution over wavelength. Taking the pseudo-inverse method as an example, a chromaticity characterization and spectrum characterization model is established by using the pseudo-inverse method,
wherein Q ∈ R96×8For training the multi-channel response values of the samples, each row represents a sample, each column represents the response value of one channel, superscript + represents the pseudo-inverse of the matrix, Mu∈R8×3For the chrominance transformation matrix, Mv∈R8×31Is a spectral transformation matrix.
S5: the chromaticity and spectral images were calculated under a reference light source. The method specifically comprises the following steps:
characterizing model M with chrominanceuAnd spectral characterization model MvAnd converting into multispectral matrix I under reference light sourcecObtaining a chromaticity and spectral image matrix
Wherein, Iu∈R262144×3For a chrominance image matrix of a scene, each line representing a pixel and each column representing one dimension of the chrominance value, Iv∈R262144×31A spectral image matrix for the scene, each row representing a pixel, each column representing a spectral reflectance at a respective sampling wavelength;
s52: respectively mixing IuAnd IvIs transformed into 512 × 512 × 3 and 512 × 512 × 31, i.e., Iu∈R512×512×3,Iv∈R512×512×31And obtaining a final chrominance image and a final spectrum image of the scene.
After the spectrum adaptive transformation, the obtained image is closer to the original image. It was calculated that the average color difference (average of all pixel color differences) without spectral adaptation transform was 16.71 Δ E for the obtained chroma image00(CIEDE2000 units of chromatic aberration), and the average chromatic aberration after spectral adaptation transform is 2.41 Δ E00. For the obtained spectral image, the average spectral difference (average value of all pixel spectral differences) without spectral adaptation transformation is 0.1178RMSE (root Mean Square error), and the average spectral difference after spectral adaptation transformation is 0.0411RMSE, so that the errors of the obtained chrominance image and the spectral image are obviously reduced after spectral adaptation transformation, and the acquisition precision of the chrominance image and the spectral image is improved.
The specific embodiments described herein are merely illustrative of the spirit of the invention. Various modifications or additions may be made to the described embodiments or alternatives may be employed by those skilled in the art without departing from the spirit or ambit of the invention as defined in the appended claims.