CN109389646A - A method of color camera radiation calibration is carried out using multispectral image - Google Patents
A method of color camera radiation calibration is carried out using multispectral image Download PDFInfo
- Publication number
- CN109389646A CN109389646A CN201811018813.8A CN201811018813A CN109389646A CN 109389646 A CN109389646 A CN 109389646A CN 201811018813 A CN201811018813 A CN 201811018813A CN 109389646 A CN109389646 A CN 109389646A
- Authority
- CN
- China
- Prior art keywords
- multispectral image
- response function
- function
- spectrum
- inverse
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/80—Analysis of captured images to determine intrinsic or extrinsic camera parameters, i.e. camera calibration
Landscapes
- Engineering & Computer Science (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Theoretical Computer Science (AREA)
- Spectrometry And Color Measurement (AREA)
- Color Television Image Signal Generators (AREA)
Abstract
The invention discloses a kind of methods for carrying out color camera radiation calibration using multispectral image, that is, solve the inverse camera response function and spectrum sensitive function of color camera.Method includes the following steps: finding the inherently linear relationship lain between multispectral image pixel, the initial estimation of inverse camera response function is obtained by restoring the inherently linear relationship from colored (RGB) image, the spectrum sensitive function being coupled in RGB image and inverse camera response function are then solved by data with existing alternating iteration.The scene spectral information that the present invention uses multispectral image to provide reduces conventional method because of error caused by use information inaccuracy, improves the radiation calibration precision of camera, and can simultaneously obtain inverse two kinds of calibration informations of camera response function and spectrum sensitive function.
Description
Technical field
The present invention relates to the radiation calibrations of color camera, can be accurate under conditions of known scene multispectral image information
Obtain the inverse camera response function and spectrum sensitive function of color camera.
Background technique
The radiation calibration information for obtaining accurate color camera facilitates the implementation of many image processing algorithms, such as multispectral
Imaging, illumination estimation, color correction etc..Since camera manufacturer does not provide these information, camera reverse response letter usually
Several and spectrum sensitive function need to be their locations accurately assessed.
The irradiation level that camera response function obtains camera sensor is converted to the brightness of image.In order to enhance acquisition image
Visual effect, camera response function is nonlinear in most of cameras, and the effect of inverse camera response function is to eliminate
This nonlinear mapping, the original colouring information of reduction scene.S.Lin, et al. in document [" Radiometric
calibration from a single image,”in Proceedings of IEEE Conference on
Computer Vision and Pattern Recognition, 938-945, IEEE (2004)] in propose on RGB image side
At boundary color linear distribution it is assumed that and solving camera reverse response letter as the distribution of color of boundary by linearized graph
Number.Due to being influenced by factors such as image resolution ratio and scene complexities, which is not bonded reality, thus estimated result
It is often not accurate enough.B.Wilburn, et al. in document [" Radiometric calibration using temporal
irradiance mixtures,”in Proceedings of IEEE Conference on Computer Vision and
Pattern Recognition, 1-7, IEEE (2008)] in propose have motion blur object exist in image boundary
Light mixing on time dimension, anti-phase machine reverse response function can be by by the brightness linear alternations of motion blur boundary
It acquires.However its solving precision nevertheless suffers from the influence of the factors such as picture quality and motion blur form.
Spectrum sensitive function describes fusion of the camera sensor to scene in the Spectral Radiation Information of visible light wave range and moves back
Change process.Accurately obtaining spectrum sensitive function facilitates the colouring information for preferably studying scene.J. Jiang, et al. in text
Offer [" What is the space of spectral sensitivity functions for digital color
cameras,”in IEEE Workshop on Applications of Computer Vision,168–179, IEEE
(2012)] in propose to solve spectrum sensitive function by shooting standard color card, and using the mathematical method that line returns.So
And the application scenarios of this method are limited, the color image that can not be obtained to any scene capture calculates, and needs to know field
The spectral information of scape light source promotes solving precision.S.Han, et al. in document [" Camera spectral sensitivity
estimation from a single image under unknown illumination by using
fluorescence,”in Proceedings of IEEE Conference on Computer Vision and
Pattern Recognition, 805-812, IEEE (2012)] in the spectrum that is emitted under different light sources using fluorescent material
This characteristic of amplitude difference is only existed, fluorescent material is added in photographed scene, and estimate light from corresponding color image
Compose sensitivity function.This method is not necessarily to know the spectrum of photographed scene light source in advance, and but limiting scene will include fluorescent material, this
Largely reduce the practical application of this method.
Summary of the invention
It is an object of the invention to not high for existing color camera radiation calibration arithmetic accuracy, application scenarios are limited etc. is asked
Topic proposes a kind of method for carrying out color camera radiation calibration using multispectral image, it is inverse to can simultaneously obtain accurate camera
Receptance function and spectrum sensitive function.
This method extracts representative intrinsic pixel spectrum from multispectral image；Then by these intrinsic spectral lines
Indicate all pixel spectrum in multispectral image with obtain in image inherently linear relationship；By scheming from nonlinear RGB
Restore the inherently linear relationship as in obtain the initial estimation of inverse camera response function；It is then based on spectral domain degradation model,
Spectrum sensitive function is solved using the initial estimate of inverse camera response function and Smoothing Constraint；It is quick to reuse the spectrum acquired
Feel the solution that function updates inverse camera response function；By successive ignition, the spectrum sensitive letter being coupled in RGB image is finally acquired
Several and inverse camera response function.Specifically, method includes the following steps:
(1) quasi color image Z and multispectral image Y is acquired and matched, is moved back by the spectral domain that formula (1) constructs color image Z
Change observation model；
G (Z)=RY (1)
Wherein, g () and R is respectively inverse camera response function and spectrum sensitive function to be asked；
(2) K is chosen in multispectral image Y sampleΦA intrinsic spectrum.The selection of intrinsic spectrum carries out in an iterative manner, plan
The intrinsic spectrum slightly newly chosen needs as different from the intrinsic spectrum chosen as possible；
(3) using obtained intrinsic spectrum set Φ linear expression in each of multispectral image Y pixel spectrum yi, right
The linear coefficient answered is ξi, in order to be concise in expression, these linear coefficients can be combined into a sparse coefficient matrix N, thus
Multispectral image Y can be Y=YN by Self representation；
(4) by the effect of inverse camera response function g (), it can be seen that being present in inherently linear in multispectral image Y
Relationship is still set up in RGB image Z, i.e. g (Z)=g (Z) N, the initial solution g of inverse camera response function0() is according to the property
It solves, it is made to meet g0(Z)=g0(Z)N；
(5) the iterative solution frame based on spectral domain degradation model, the inverse camera response function obtained with step (4) are constructed
g0() is used as iteration initial value, and by formula (2), alternating iteration solves formula (3) in order, finally estimates the light of color camera
Compose sensitivity function R and inverse camera response function g ()；
Wherein, ψR(R) and ψg(g) it is respectively solution space about spectrum sensitive function R and inverse camera response function g ()
It is prior-constrained；
Further, the solution form of inverse camera response function is the discrete sampling point structure by g () curve in (0,1)
At vector, and the mapping function of g () function be with based between sampled point interpolation operation realize.
Further, in the step (3), linear coefficient ξ is solved according to the following formula using linear regression methodi:
Wherein, sparse constraint has been used to prevent from occurring in solution procedure Singular Value, and η is that the permission of error is joined
Number.
Further, it in the step (4), is solved according to the following formula initially using the Non-Linear Programming operator based on interior point method
Solve g0():
The solution space of initial solution is constrained in the linear space being made of the column vector of basic matrix Ω；Basic matrix Ω passes through
The sample set of inverse camera response function is obtained using Principal Component Analysis training.
Further, in the step (5), R and g () is solved using augmentation Lagrangian method according to the following formula respectively:
Wherein, second-order smooth constraint is added when solving spectrum sensitive function, is increased when solving inverse camera response function
The constraint of single order monotonic increase, λRFor smoothness admissible parameter.
The invention has the advantages that present invention proposition obtains color camera radiation calibration information using multispectral image
Method, the inverse camera response function and spectrum sensitive function of color camera can be obtained simultaneously.From the perspective of estimated accuracy,
The method reduce other estimation methods because assuming evaluated error caused by invalid or scene is not applicable.The present invention is to picture
Photographed scene require more loose, do not need stringent limitation, and do not need in calibration process the auxiliary such as light source light spectrum letter
Breath.
Detailed description of the invention
Fig. 1 is image capturing system figure used in the present invention；
Fig. 2 is the flow chart that the present invention carries out color camera Calibration Method using multispectral image；
Fig. 3 is the assessment result of inverse camera response function and spectrum sensitive Function Estimation of the invention；
Fig. 4 be by root-mean-square error respectively to using the method for the present invention and based on linearisation border color location mode into
The accuracy comparison result of row camera reverse response Function Estimation.
Specific embodiment
The specific embodiment of the invention is described further with reference to the accompanying drawing.
Image capturing system of the invention is as shown in Figure 1.Multispectral imaging device is by camera lens, tunable optic filter group and list
Color industrial camera composition, for acquiring multispectral image at equal intervals in limit of visible spectrum.Then with RGB camera replacement mostly light
Imaging device is composed, the color image under same scene is shot.If the multispectral image and RGB picture that collect be not completely right
Together, image registration algorithm can be used to be pre-processed；Preferably, the present invention use S.J.Chen, et al. in document
【“Normalized total gradient:a new measure for multispectral image
Registration, " IEEE Transactions on Image Processing, 1297-1310,2018)] in propose
Registration Algorithm.
Fig. 2 is the method flow that the present invention carries out color camera radiation calibration using multispectral image, comprising the following steps:
1, K is chosen in multispectral image Y sampleΦA intrinsic spectrum.The selection of intrinsic spectrum carries out in an iterative manner, strategy
Intrinsic spectrum newly to choose needs as different from the intrinsic spectrum chosen as possible；Preferably, the present invention takes KΦ=70, and select equal
As measurement intrinsic spectrum to be selected and, anthology levies the index of difference between spectrum to square error.
2, using obtained intrinsic spectrum set Φ linear expression in each of multispectral image Y pixel spectrum yi, corresponding
Linear coefficient be ξi, these final linear coefficients can be combined into a sparse coefficient matrix N, and multispectral image Y can
To be Y=YN by Self representation；Preferably, the present invention solves linear coefficient ξ using linear regression method according to the following formulai,
Wherein, sparse constraint has been used to prevent from occurring in solution procedure Singular Value, and η is that the permission of error is joined
Number, the present invention take η=10-5。
3, by the effect of inverse camera response function g (), the inherently linear relationship in multispectral image Y is in RGB image
It is still set up in Z, i.e. g (Z)=g (Z) N.The initial solution g of inverse camera response function0() solves according to the property, makes its satisfaction
g0(Z)=g0(Z)N.Preferably, the present invention solves initial solution using the Non-Linear Programming operator based on interior point method according to the following formula
g0(),
In order to increase robustness, solution space is constrained in the linear space being made of the column vector of basic matrix Ω；Base
Matrix Ω is obtained by the sample set to inverse camera response function using Principal Component Analysis training, is chosen corresponding to main energy
KΩ=9 feature vector bases.
4, the iterative solution frame based on spectral domain degradation model is constructed, with the inverse camera response function g of acquisition0(·)
As iteration initial value, by formula (2), formula (3), alternating iteration is solved in order, finally estimates the spectrum sensitive of color camera
Function R and inverse camera response function g ()；Preferably, the present invention solves R using augmentation Lagrangian method according to the following formula respectively
With g (),
Wherein, it is added to second-order smooth constraint when solving spectrum sensitive function, increases when solving inverse camera response function
Single order monotonic increase has been added to constrain, to promote solution robustness.λRFor smoothness admissible parameter, the present invention takes λR=0.01.
Embodiment 1
The radiation calibration of color camera is realized following with the method for the present invention.In order to compare stated accuracy, first using
The camera response function and spectrum sensitive function known are emulated from multispectral image generates RGB image, then uses present invention side
Method carries out radiation calibration, and calibration result is compared with known true value.From figure 3, it can be seen that two groups of data use
Different camera response function, therefore the brightness of corresponding RGB image is different.Wherein GT curve represents true value, and the representative of Est curve is adopted
The result obtained with present invention estimation.By comparing the estimation knot that can be seen that inverse camera response function and spectrum sensitive function
Fruit is and true value is close, and error is small.
Embodiment 2
Estimation below by root-mean-square error, from quantization angle to inverse camera response function is realized using the method for the present invention
As a result it is measured.It can be seen from figure 4 that the precision of two kinds of estimation methods has differences.Wherein Single curve represents base
In linearized graph as the method that border color is distributed obtain as a result, Ours curve is using the obtained result of the present invention.It can be with
Find out that the present invention is whole to the estimated accuracy of 201 samples higher.
The above is only the specific embodiment of the invention, cannot be limited the scope of the invention with this, in the art
Those skilled in the art change according to known to equivalent change made by this creation and those skilled in that art, all should still belong to
The range that the present invention covers.
Claims (5)
1. a kind of method for carrying out color camera radiation calibration using multispectral image, which is characterized in that this method includes following
Step:
(1) colour (RGB) image Z and multispectral image Y is acquired and be registrated, is moved back by the spectral domain that formula (1) constructs color image Z
Change observation model；
G (Z)=RY (1)
Wherein, g () and R is respectively inverse camera response function and spectrum sensitive function to be asked；
(2) K is chosen in multispectral image Y sampleΦA intrinsic spectrum obtains intrinsic spectrum set Φ；The selection of the intrinsic spectrum with
The mode of iteration carries out, and strategy need to be as different from the intrinsic spectrum chosen as possible for the intrinsic spectrum newly chosen；
(3) using obtained intrinsic spectrum set Φ linear expression in each of multispectral image Y pixel spectrum yi, corresponding line
Property coefficient is ξi；Linear coefficient is combined into a sparse coefficient matrix N, so that multispectral image Y can be by Self representation
For Y=YN；
(4) by the effect of inverse camera response function g (), it can be seen that being present in the inherently linear relationship in multispectral image Y
It is still set up in color image Z, i.e. g (Z)=g (Z) N, the initial solution g of inverse camera response function0() is asked according to the property
Solution, makes it meet g0(Z)=g0(Z)N；
(5) the iterative solution frame based on spectral domain degradation model, the inverse camera response function g obtained with step (4) are constructed0
() is used as iteration initial value, and by formula (2), formula (3), alternating iteration is solved in order, finally estimates the spectrum of color camera
Sensitivity function R and inverse camera response function g ()；
Wherein, ψR(R) and ψg(g) it is respectively elder generation about the solution space of spectrum sensitive function R and inverse camera response function g ()
Test constraint.
2. a kind of method for carrying out color camera radiation calibration using multispectral image, feature exist according to claim 1
In the solution form of inverse camera response function is the vector that the discrete sampling point by g () curve in (0,1) is constituted, and g
The mapping function of () function is realized with based on the interpolation operation between sampled point.
3. a kind of method for carrying out color camera radiation calibration using multispectral image, feature exist according to claim 1
In solving linear coefficient ξ according to the following formula using linear regression method in the step (3)i:
Wherein, sparse constraint has been used to prevent from occurring in solution procedure Singular Value, and η is the permission parameter of error.
4. a kind of method for carrying out color camera radiation calibration using multispectral image, feature exist according to claim 1
In solving initial solution g according to the following formula using the Non-Linear Programming operator based on interior point method in the step (4)0():
The solution space of initial solution is constrained in the linear space being made of the column vector of basic matrix Ω；Basic matrix Ω passes through to inverse
The sample set of camera response function is obtained using Principal Component Analysis training.
5. a kind of method for carrying out color camera radiation calibration using multispectral image, feature exist according to claim 1
In, in the step (5), using augmentation Lagrangian method respectively according to the following formula solve R and g ():
Wherein, second-order smooth constraint is added when solving spectrum sensitive function, increases single order when solving inverse camera response function
Monotonic increase constraint, λRFor smoothness admissible parameter.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201811018813.8A CN109389646B (en) | 2018-09-03 | 2018-09-03 | Method for carrying out radiometric calibration on color camera by utilizing multispectral image |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201811018813.8A CN109389646B (en) | 2018-09-03 | 2018-09-03 | Method for carrying out radiometric calibration on color camera by utilizing multispectral image |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN109389646A true CN109389646A (en) | 2019-02-26 |
| CN109389646B CN109389646B (en) | 2021-10-08 |
Family
ID=65418716
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201811018813.8A Active CN109389646B (en) | 2018-09-03 | 2018-09-03 | Method for carrying out radiometric calibration on color camera by utilizing multispectral image |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN109389646B (en) |
Cited By (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN113496518A (en) * | 2020-03-20 | 2021-10-12 | 华为技术有限公司 | Camera response curve calculation method and device |
| WO2022126438A1 (en) * | 2020-12-16 | 2022-06-23 | 华为技术有限公司 | Image generation method and apparatus, and electronic device |
| CN115855107A (en) * | 2022-11-29 | 2023-03-28 | 中国科学院长春光学精密机械与物理研究所 | Image-based linear array space remote sensing load radiometric calibration method |
Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN1641374A (en) * | 2004-12-25 | 2005-07-20 | 中国科学院安徽光学精密机械研究所 | Satellite full optical-path radiation beaconing method |
| US20100245634A1 (en) * | 2009-03-26 | 2010-09-30 | Ahdoot Ned M | Hardware implemented pixel level digital filter and processing of electromagnetic signals |
| US20120183213A1 (en) * | 2009-09-03 | 2012-07-19 | National Ict Australia Limited | Illumination spectrum recovery |
| CN102607592A (en) * | 2012-02-24 | 2012-07-25 | 北京大学 | Remote sensing calibration comprehensive method and calibration equipment vehicle |
| CN104482939A (en) * | 2014-11-06 | 2015-04-01 | 中国资源卫星应用中心 | Time sequence-based space-borne camera radiometric cross-calibration method |
-
2018
- 2018-09-03 CN CN201811018813.8A patent/CN109389646B/en active Active
Patent Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN1641374A (en) * | 2004-12-25 | 2005-07-20 | 中国科学院安徽光学精密机械研究所 | Satellite full optical-path radiation beaconing method |
| US20100245634A1 (en) * | 2009-03-26 | 2010-09-30 | Ahdoot Ned M | Hardware implemented pixel level digital filter and processing of electromagnetic signals |
| US20120183213A1 (en) * | 2009-09-03 | 2012-07-19 | National Ict Australia Limited | Illumination spectrum recovery |
| CN102607592A (en) * | 2012-02-24 | 2012-07-25 | 北京大学 | Remote sensing calibration comprehensive method and calibration equipment vehicle |
| CN104482939A (en) * | 2014-11-06 | 2015-04-01 | 中国资源卫星应用中心 | Time sequence-based space-borne camera radiometric cross-calibration method |
Non-Patent Citations (4)
| Title |
|---|
| WOJCIECHOWSKI K等: "RADIOMETRIC CALIBRATION OF A MULTISPECTRAL CAMERA", 《COMPUTATIONAL IMAGING AND VISION] COMPUTER VISION AND GRAPHICS》 * |
| 李鑫等: "基于光谱靶标的光学遥感器光谱响应特性在轨评估方法研究", 《光谱学与光谱分析》 * |
| 杜丽丽等: "基于ＬＣＴＦ的多光谱面阵ＣＣＤ相机的辐射定标", 《光谱学与光谱分析》 * |
| 潘之玮: "基于图像融合的多光谱图像超分辨率重建算法", 《中国博士学位论文全文数据库 (信息科技辑)》 * |
Cited By (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN113496518A (en) * | 2020-03-20 | 2021-10-12 | 华为技术有限公司 | Camera response curve calculation method and device |
| WO2022126438A1 (en) * | 2020-12-16 | 2022-06-23 | 华为技术有限公司 | Image generation method and apparatus, and electronic device |
| CN115855107A (en) * | 2022-11-29 | 2023-03-28 | 中国科学院长春光学精密机械与物理研究所 | Image-based linear array space remote sensing load radiometric calibration method |
| CN115855107B (en) * | 2022-11-29 | 2023-08-25 | 中国科学院长春光学精密机械与物理研究所 | Image-based linear array space remote sensing load radiometric calibration method |
Also Published As
| Publication number | Publication date |
|---|---|
| CN109389646B (en) | 2021-10-08 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| Brauers et al. | Multispectral filter-wheel cameras: Geometric distortion model and compensation algorithms | |
| US9332197B2 (en) | Infrared sensor control architecture | |
| Robertson et al. | Estimation-theoretic approach to dynamic range enhancement using multiple exposures | |
| Shimano | Recovery of spectral reflectances of objects being imaged without prior knowledge | |
| Kelcey et al. | Sensor correction and radiometric calibration of a 6-band multispectral imaging sensor for UAV remote sensing | |
| US8619153B2 (en) | Radiometric calibration using temporal irradiance mixtures | |
| CN108020519B (en) | Virtual multi-light-source spectrum reconstruction method based on color constancy | |
| Cheng et al. | Beyond white: Ground truth colors for color constancy correction | |
| Foi et al. | Noise measurement for raw-data of digital imaging sensors by automatic segmentation of nonuniform targets | |
| CN109389646A (en) | A method of color camera radiation calibration is carried out using multispectral image | |
| Farrell et al. | Sensor calibration and simulation | |
| Webb et al. | Spectral and 3D cultural heritage documentation using a modified camera | |
| KR20170100717A (en) | Apparatus and method for analyzing skin condition using spectral reflectance estimation | |
| CN108267426B (en) | Drawing pigment identification system and method based on multispectral imaging | |
| Wilburn et al. | Radiometric calibration using temporal irradiance mixtures | |
| CN107170013B (en) | Calibration method for spectral response curve of RGB camera | |
| Meißner et al. | Towards standardized evaluation of image quality for airborne camera systems | |
| Cerman et al. | Exposure time estimation for high dynamic range imaging with hand held camera | |
| JP4692190B2 (en) | Spectral reflectance estimation method, spectral reflectance estimation apparatus, and spectral reflectance estimation program | |
| Shrestha et al. | Assessment of two fast multispectral systems for imaging of a cultural heritage artifact-a Russian icon | |
| Haefner | Best practices for imaging system MTF measurement | |
| Manakov | Evaluation of computational radiometric and spectral sensor calibration techniques | |
| Hanji et al. | Robust estimation of exposure ratios in multi-exposure image stacks | |
| Abdelhamed | Noise Modelling for Smartphone Cameras | |
| Teaney et al. | Evaluating the performance of reflective band imaging systems: a tutorial |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |