CN102404581A - Color image processing method and device based on interpolation and near infrared - Google Patents

Abstract

The invention discloses a color image processing method based on interpolation and near infrared. The method comprises the following steps: inputting a sensor source image and a near infrared image based on the same scene, processing the input images through grads difference sparseness constraint to obtain pixel information after grads difference sparseness constraint is performed, and obtaining pixel information after detail constraint is performed according to detail constraint among a plurality of unicolor channels corresponding to a plurality of unicolor channel images; calculating corresponding mean square error of each color image after detail constraint is performed; comparing a predetermined threshold and the mean square error of pixel information of a plurality of channels after detail constraint of two adjacent color images are performed, particularly, when the mean square error of the two adjacent color images is smaller than the predetermined threshold, an interpolation amalgamation image corresponding to each color image is generated according to pixel information of a plurality of channels after detail constraint is performed. The invention further discloses a color image processing device based on interpolation and near infrared. The invention can effectively meet the requirement of combination application of near infrared and visible light.

Description

Method and apparatus based on interpolation and near infrared Color Image Processing

Technical field

The present invention relates to computer vision field, particularly a kind of method and apparatus based on interpolation and near infrared Color Image Processing.

Background technology

Common single-chip digital camera passes through at imaging sensor (CCD; Charge-Coupled Device) preceding a color filter array (CFA is set; Color filter array) to obtain the color panel image; Each pixel is only caught a various colors spectral information, respectively corresponding red (R), green (G) and blue (B).Then this color panel image is obtained coloured image through color interpolation method, this process also is called mosaic effect.Most silicon material CCD can respond to the spectral information of 200-1100nm.The general appreciable spectral wavelength scope of human eye is between 400 to 700nm, and the spectrum of 700-1100nm wave-length coverage belongs to near-infrared (NIR, Near Infrared) spectrum.More brightness and spatial information because near infrared spectrum near visible light, has different with visible light.Consider this characteristic, unite near-infrared and visible images in recent years and be utilized to improve some visions and photography task.

In general, these application concentrate on two aspects: the extraneous information that is based on near-infrared image is on the one hand extracted original scene information, like light source detection, and shadow Detection, materials classification; Be to combine colour and near-infrared image to strengthen image on the other hand, for example to improve HDR (HDR, High-Dynamic Range) colored rendering, near-infrared image dyeing, coloured image denoising, sharpening, mist elimination or the like.These are used all is to carry out the color interpolation processing earlier to obtain conventional coloured image, and then carries out visible/near-infrared image Combined Treatment.

To the associating near-infrared and the visual light imaging of Same Scene, accomplish through separating or leaching wherein certain part.Visible light/near-infrared joint imaging the system that occurs at present mainly concentrates on the double camera system of preparation heat mirror.Such as utilizing conversion that NIR is set or the visible light optical filtering is realized, people such as S ü sstrunk then directly utilize the multispectral camera of two CCD.

But; Use the major defect of above-mentioned conventional method to be that operand is huge, when replacing the constrained of iterated interpolation, need observe the constraint of constraint and details, the computing more complicated; Noise spot is more; The image information that obtains is second-rate, effective fast processing coloured image, and what Fig. 4 b represented promptly is the image effect after the above-mentioned conventional method of utilization is carried out interpolation processing.

Summary of the invention

The object of the invention is intended to solve at least one of above-mentioned technological deficiency.

For this reason, first purpose of the present invention is to propose a kind of method based on interpolation and near infrared Color Image Processing, and this method can effectively satisfy the needs of near-infrared and visible images Combined application.

Second purpose of the present invention is to propose a kind of device based on interpolation and near infrared Color Image Processing, and this device can effectively satisfy the needs of near-infrared and visible images Combined application.

For realizing above-mentioned purpose, the embodiment of first aspect present invention has proposed a kind of method based on interpolation and near infrared Color Image Processing, comprises the steps:

S1: input is carried out color interpolation obtaining several monochromatic channel image to said sensor source image, and is obtained the prior information of said near-infrared image based on the sensor source image and the near-infrared image of Same Scene in internal memory;

S2: to every width of cloth coloured image; Prior information based on said near-infrared image is carried out the sparse constraint of gradient difference to said several monochromatic channel image; Obtain the Pixel Information after the sparse constraint of gradient difference; Based on details constraint between the corresponding a plurality of monochromatic passage of said several monochromatic channel image, upgrade Pixel Information after the sparse constraint of said gradient difference to obtain the Pixel Information after the details constraint;

S3:, calculate the mean square error of the Pixel Information of a plurality of passages after the corresponding details of every width of cloth coloured image retrains according to the Pixel Information after the said details constraint;

S4: mean square error and predetermined threshold to the Pixel Information of a plurality of passages after the constraint of the details of said adjacent two width of cloth coloured images compare; If the mean square error of the Pixel Information of a plurality of passages after the said details constraint is more than or equal to said predetermined threshold; The Pixel Information of a plurality of passages after then said details being retrained is as the Pixel Information of said several the monochromatic channel image among the said step S2; Repeating said steps S2 to S4, until the mean square error of said adjacent two width of cloth coloured images less than said predetermined threshold;

S5:, generate the corresponding interpolation fusion image of said every width of cloth coloured image according to the Pixel Information of a plurality of passages after the said details constraint when the mean square error of said adjacent two width of cloth coloured images during less than said predetermined threshold.

Method based on interpolation and near infrared Color Image Processing according to an embodiment of the invention, said sensor source image are Bayer color mode image.

Method based on interpolation and near infrared Color Image Processing according to an embodiment of the invention, the prior information of said near-infrared image are the gradient of said near-infrared image.

Method based on interpolation and near infrared Color Image Processing according to an embodiment of the invention, the corresponding a plurality of monochromatic passage of said several monochromatic channel image comprises R passage, G passage and B passage.

Method based on interpolation and near infrared Color Image Processing according to an embodiment of the invention, the sparse constraint of said gradient difference adopts following formula to calculate:

${\arg }_{v_i}\min \underset{n}{\mathrm{\Σ}}\left|\right|\▿V_i\left(n\right)-\▿N\left(n\right)\left|\right|$

Wherein, V _iRepresent R, G, B monochrome image respectively, i=1,2,3, N representes the Pixel Information of said near-infrared image, n representes the index of each pixel position,

Expression is to V _iAsk gradient by a position;

Expression is asked gradient to the Pixel Information N of said near-infrared image by a position,

Expression is to V _iAsk the 1-norm with the gradient difference of N.

Method based on interpolation and near infrared Color Image Processing according to an embodiment of the invention, the details constraint comprises the steps: between a plurality of monochromatic passage of said several monochromatic channel image correspondences

Utilize the details between said R passage, G passage and the B passage to retrain the channel value that adopts following formula to upgrade said G passage,

$G_{k+1}\left(w\right)=L\left(w\right)G_k\left(w\right)+\frac{H\left(w\right)R_k\left(w\right)+H\left(w\right)B_k\left(w\right)}{2}$

Wherein, G _K+1(w) be the G channel value after upgrading, L (w) is the LPF parameter, G _k(w) be the G channel value before upgrading, H (w) is the high-pass filtering parameter, R _k(w) be the channel value of said R passage, R _k(w) be the channel value of said B passage;

Utilize the G channel value after the said renewal to adopt following formula to carry out the details constraint to said R passage and said B passage

R _k+1(w)＝L(w)R _k(w)+H(w)G _k+1(w)

B _k+1(w)＝L(w)B _k(w)+H(w)G _k+1(w)

Wherein, R _K+1(w) be the R channel value after upgrading, L (w) is the LPF parameter, R _k(w) be the R channel value before upgrading, H (w) high-pass filtering parameter, G _K+1(w) be G channel value after the said renewal, B _K+1(w) be the B channel value after upgrading, B _k(w) be the B channel value before upgrading.

According to an embodiment of the invention based on the method for interpolation and near infrared Color Image Processing; Can realize processing synchronously to coloured image interpolation and near-infrared image; Through the method for coloured image interpolation constraint is improved, make that calculating is more quick, program simply is easy to realize; Effectively handle the problem that near-infrared and visible light are united application, the coloured image that obtains is more clear and complete.

The embodiment of second aspect of the present invention has proposed a kind of device based on interpolation and near infrared Color Image Processing, comprising:

Input module, said input module are used for to sensor source image and the near-infrared image of internal memory input based on Same Scene;

Pretreatment module, said pretreatment module links to each other with said input module, is used for said sensor source image is carried out color interpolation obtaining several monochromatic channel image, and obtains the prior information of said near-infrared image;

Constraints module; Said constraints module links to each other with said pretreatment module; Prior information according to said near-infrared image is carried out the sparse constraint of gradient difference to said several monochromatic channel image; Obtain the Pixel Information after the sparse constraint of gradient difference,, upgrade Pixel Information after the sparse constraint of said gradient difference to obtain the Pixel Information after the details constraint according to details constraint between the corresponding a plurality of monochromatic passage of said several monochromatic channel image;

Computing module, said computing module links to each other with said constraints module, according to the Pixel Information after the said details constraint, calculates the mean square error of the Pixel Information of a plurality of passages after the corresponding details of every width of cloth coloured image retrains;

Comparison module; Said comparison module links to each other with said constraints module with said computing module respectively; Mean square error and predetermined threshold to the Pixel Information of a plurality of passages after the constraint of the details of said adjacent two width of cloth coloured images compare; If the mean square error of the Pixel Information of a plurality of passages after the said details constraint is more than or equal to predetermined threshold; The Pixel Information of a plurality of passages after then said details being retrained is as the Pixel Information of said several the monochromatic channel image in the said constraints module; Said constraints module is carried out sparse constraint of gradient difference and details constraint to the Pixel Information of said several the monochromatic channel image after upgrading; Pixel Information after the constraint of details after obtaining upgrading, the mean square error of the Pixel Information of a plurality of passages after the said computing module computational details constraint, said comparison module compares mean square error until said adjacent two width of cloth coloured images less than said predetermined threshold to the mean square error of the Pixel Information of the said a plurality of passages after upgrading and said predetermined threshold;

The fusion image generation module; Said fusion image generation module links to each other with said comparison module; When the mean square error of said adjacent two width of cloth coloured images during, generate the corresponding interpolation fusion image of said every width of cloth coloured image according to the Pixel Information of a plurality of passages after the said details constraint less than said predetermined threshold.

Device based on interpolation and near infrared Color Image Processing according to an embodiment of the invention, said sensor source image are Bayer color mode image.

Device based on interpolation and near infrared Color Image Processing according to an embodiment of the invention, the prior information of said near-infrared image are the gradient of said near-infrared image.

Device based on interpolation and near infrared Color Image Processing according to an embodiment of the invention, the corresponding a plurality of monochromatic passage of said several monochromatic channel image comprises R passage, G passage and B passage.

Device based on interpolation and near infrared Color Image Processing according to an embodiment of the invention, said constraints module adopt following formula to the sparse constraint of said gradient difference:

${\arg }_{v_i}\min \underset{n}{\mathrm{\Σ}}\left|\right|\▿V_i\left(n\right)-\▿N\left(n\right)\left|\right|$

Wherein, V _iRepresent R, G, B monochrome image respectively, i=1,2,3, N representes the Pixel Information of said near-infrared image, n representes the index of each pixel position,

Expression is to V _iAsk gradient by a position;

Expression is asked gradient to the Pixel Information N of said near-infrared image by a position,

Expression is to V _iAsk the 1-norm with the gradient difference of N.

Device based on interpolation and near infrared Color Image Processing according to an embodiment of the invention, said constraints module utilize the details between said R passage, G passage and the B passage to retrain the channel value that adopts following formula to upgrade said G passage,

$G_{k+1}\left(w\right)=L\left(w\right)G_k\left(w\right)+\frac{H\left(w\right)R_k\left(w\right)+H\left(w\right)B_k\left(w\right)}{2}$

Wherein, G _K+1(w) be the G channel value after upgrading, L (w) is the LPF parameter, G _k(w) be the G channel value before upgrading, H (w) is the high-pass filtering parameter, R _k(w) be the channel value of said R passage, R _k(w) be the channel value of said B passage;

Said constraints module utilizes the G channel value after the said renewal to adopt following formula to carry out the details constraint to said R passage and said B passage

R _k+1(w)＝L(w)R _k(w)+H(w)G _k+1(w)

B _k+1(w)＝L(w)B _k(w)+H(w)G _k+1(w)

Wherein, R _K+1(w) be the R channel value after upgrading, L (w) is the LPF parameter, R _k(w) be the R channel value before upgrading, H (w) high-pass filtering parameter, G _K+1(w) be G channel value after the said renewal, B _K+1(w) be the B channel value after upgrading, B _k(w) be the B channel value before upgrading.

According to an embodiment of the invention based on the device of interpolation and near infrared Color Image Processing; Can realize processing synchronously to coloured image interpolation and near-infrared image; Through the method for coloured image interpolation constraint is improved, make that calculating is more quick, program simply is easy to realize; Effectively handle the problem that near-infrared and visible light are united application, the coloured image that obtains is more clear and complete.

Aspect that the present invention adds and advantage part in the following description provide, and part will become obviously from the following description, or recognize through practice of the present invention.

Description of drawings

Above-mentioned and/or additional aspect of the present invention and advantage are from obviously with easily understanding becoming the description of embodiment below in conjunction with accompanying drawing, wherein:

Fig. 1 is the flow chart based on the method for interpolation and near infrared Color Image Processing according to the embodiment of the invention;

Fig. 2 is the system framework figure based on the method for interpolation and near infrared Color Image Processing according to the embodiment of the invention;

The sensor source image of Fig. 3 a for importing among Fig. 1;

The near-infrared image of Fig. 3 b for importing among Fig. 1;

Fig. 4 obtains several monochromatic channel image among Fig. 1 the sensor source image of being imported being carried out color interpolation;

The interpolation fusion image of Fig. 5 a for generating among Fig. 1;

Fig. 5 b is the color interpolation image after the conventional process; With

Fig. 6 is the structural representation based on the device of interpolation and near infrared Color Image Processing according to the embodiment of the invention.

Embodiment

Describe embodiments of the invention below in detail, the example of said embodiment is shown in the drawings, and wherein identical from start to finish or similar label is represented identical or similar elements or the element with identical or similar functions.Be exemplary through the embodiment that is described with reference to the drawings below, only be used to explain the present invention, and can not be interpreted as limitation of the present invention.

Referring to figs. 1 to Fig. 4 the method based on interpolation and near infrared Color Image Processing according to the embodiment of the invention is described below.

As shown in Figure 1, the method based on interpolation and near infrared Color Image Processing according to the embodiment of the invention comprises the steps:

S11: input pickup source images and near-infrared image, carry out preliminary treatment to sensor source image and near-infrared image.

At first input is carried out the image preliminary treatment to sensor source image and the near-infrared image of being imported then based on the sensor source image and the near-infrared image of Same Scene in internal memory.Carry out color interpolation at last obtaining several monochromatic channel image, and obtain the prior information of near-infrared image.

In one embodiment of the invention, the sensor source image of in internal memory, importing is a Bayer color mode image.

The prior information of the near-infrared image that need obtain in one embodiment of the invention, is the gradient of near-infrared image.

In one embodiment of the invention, as shown in Figure 2, a color filter array is set before the imaging sensor of camera, the image that visible images obtains after the color filter array sampling is the sensor source image; The image that obtains behind the near infrared imaging sensor sample is a near-infrared image.

In one embodiment of the invention, Fig. 3 a representes that the visible images imported, Fig. 3 b represent the near-infrared image of importing.

In one embodiment of the invention; Fig. 4 representes that the sensor source image of being imported is carried out color interpolation obtains several monochromatic channel image; It is understandable that Fig. 4 is just based on an example of the embodiment of the invention, rather than to the restriction of embodiments of the invention.

S12: the gradient difference constraint of image and details constraint

To every width of cloth coloured image; At first several monochromatic channel image are carried out the sparse constraint of gradient difference according to the prior information of near-infrared image; To obtain the Pixel Information after the sparse constraint of gradient difference; According to details constraint between the corresponding a plurality of monochromatic passage of several monochromatic channel image, the Pixel Information after the sparse constraint of final updating gradient difference is to obtain the Pixel Information after details retrains then;

In one embodiment of the invention, the corresponding a plurality of monochromatic passage of several monochromatic channel image comprises R passage, G passage and B passage.

In one embodiment of the invention, the sparse constraint of gradient difference adopts following formula to calculate:

${\arg }_{v_i}\min \underset{n}{\mathrm{\Σ}}\left|\right|\▿V_i\left(n\right)-\▿N\left(n\right)\left|\right|$

Wherein, V _iRepresent R, G, B monochrome image respectively, i=1,2,3, N representes the Pixel Information of near-infrared image, n representes the index of each pixel position,

Expression is to V _iAsk gradient by a position; Expression is asked gradient to the Pixel Information N of near-infrared image by a position, Expression is to V _iAsk the 1-norm with the gradient difference of N.

In one embodiment of the invention, V ₁Expression R passage monochrome image, V ₂Expression G passage monochrome image, V ₃Expression B passage monochrome image it is understandable that, the example that this expression is just enumerated for embodiments of the invention are described, rather than to a kind of restriction of the embodiment of the invention.

This step is according to the prior information of near-infrared image the color interpolation processing procedure to be increased a kind of spectrum constraint, coloured image is carried out preliminary treatment, for follow-up work of treatment is laid a good foundation.

In one embodiment of the invention, in conjunction with the high frequency correlation of R, B passage and G passage list color image, can obtain the constraint of correlation details.Because the G channel image, with R, the B channel image has very big similitude on high frequency; So the high frequency characteristics and the R of the value that can estimate to draw with the G passage; The high frequency characteristics of B passage is compared, thereby lets R, and the high-frequency information of the high frequency characteristics of B passage and G passage is close.

In one embodiment of the invention, the full resolution R channel image that satisfies the constraint of correlation details is " details constrain set ", and is specific as follows:

$C_d\stackrel{\mathrm{def}}{=}\{r(x,y):|R\⊗A_i(x,y)-G\⊗A_i(x,y)|\≤T,i=\mathrm{1,2,3}\}$

Wherein, threshold value T is used for weighing high frequency degree of correlation between R passage and the G passage, is traditionally arranged to be T=0, and A _i(i=1,2,3) are analysis filters, obtain the high-frequency information of image, are provided with as follows:

I=1: row adopts LPF, and row adopt high-pass filtering;

I=2: row adopts high-pass filtering, and row adopt LPF;

I=3: row adopts high-pass filtering, and row adopt high-pass filtering.

It is understandable that realize that the method for this step is not limited to the given data of present embodiment, the example that present embodiment just provides for technical scheme is more readily understood can not be interpreted as limitation of the present invention.

In one embodiment of the invention, the details constraint comprises the steps: between a plurality of monochromatic passage of several monochromatic channel image correspondences

Utilize the channel value of the following formula renewal of the details constraint employing G passage between R passage, G passage and the B passage,

$G_{k+1}\left(w\right)=L\left(w\right)G_k\left(w\right)+\frac{H\left(w\right)R_k\left(w\right)+H\left(w\right)B_k\left(w\right)}{2}$

Wherein, G _K+1(w) be the G channel value after upgrading, L (w) is the LPF parameter, G _k(w) be the G channel value before upgrading, H (w) is the high-pass filtering parameter, R _k(w) be the channel value of R passage, B _k(w) be the channel value of B passage;

G channel value after the renewal adopts following formula that R passage and B passage are carried out the details constraint:

R _k+1(w)＝L(w)R _k(w)+H(w)G _k+1(w)

B _k+1(w)＝L(w)B _k(w)+H(w)G _k+1(w)

Wherein, R _K+1(w) be the R channel value after upgrading, L (w) is the LPF parameter, R _k(w) be the R channel value before upgrading, H (w) high-pass filtering parameter, G _K+1(w) be the G channel value after upgrading, B _K+1(w) be the B channel value after upgrading, B _k(w) be the B channel value before upgrading.

Embodiments of the invention utilize the G passage to come modified R, B for reference no longer merely; But use R simultaneously, the revised value of B passage is carried out high frequency confinement to the G passage.R, the B passage can be revised based on near-infrared image, in replacing iterative process, revises the G passage again, does further renewal, more helps the figure image intensifying.

S13: calculate mean square error

Based on the Pixel Information after the details constraint, calculate the mean square error of the Pixel Information of a plurality of passages after the corresponding details of every width of cloth coloured image retrains.

S14: judge that whether mean square error is less than threshold value

Mean square error and predetermined threshold to the Pixel Information of a plurality of passages after the constraint of the details of adjacent two width of cloth coloured images compare; If the mean square error of the Pixel Information of a plurality of passages after the details constraint is more than or equal to predetermined threshold; The Pixel Information of a plurality of passages after then details being retrained is as the Pixel Information of said several the monochromatic channel image among the step S12; Repeating said steps S12 to S14, until the mean square error of said adjacent two width of cloth coloured images less than said predetermined threshold;

In one embodiment of the invention, threshold value is 0.001.It is understandable that threshold value is can be by user or default, and is not limited to the example that embodiments of the invention are given.

S15: generate the interpolation fusion image

When the mean square error of adjacent two width of cloth coloured images during, generate the corresponding interpolation fusion image of every width of cloth coloured image according to the Pixel Information of a plurality of passages after the details constraint less than said predetermined threshold.

In one embodiment of the invention, Fig. 5 a is the last interpolation fusion image that generates.

According to an embodiment of the invention based on the device of interpolation and near infrared Color Image Processing; Can realize processing synchronously to coloured image interpolation and near-infrared image; Through the method for coloured image interpolation constraint is improved, make that calculating is more quick, program simply is easy to realize; Effectively handle the problem that near-infrared and visible light are united application, the coloured image that obtains is more clear and complete.

Below with reference to Fig. 6 the device 20 based on interpolation and near infrared Color Image Processing according to the embodiment of the invention is described.

As shown in Figure 6, the embodiment of the invention comprise input module 21, pretreatment module 22, constraints module 23, computing module 24, comparison module 25 and fusion image generation module 26 based on interpolation and near infrared color image processing apparatus 20.Wherein, Input module 21 links to each other with pretreatment module 22, and constraints module 23 links to each other with pretreatment module 22, and computing module 24 links to each other with constraints module 22; Comparison module 25 links to each other with constraints module 23 and computing module 24 respectively, and fusion image generation module 26 links to each other with comparison module 25.

Input module 21 is used for to sensor source image and the near-infrared image of internal memory input based on Same Scene.

In one embodiment of the invention, the sensor source image of input module 21 inputs is a Bayer color mode image.

In one embodiment of the invention, as shown in Figure 2, a color filter array is set before the imaging sensor of camera, the image that visible images obtains after the color filter array sampling is the sensor source image; The image that obtains behind the near infrared imaging sensor sample is a near-infrared image.

In one embodiment of the invention, Fig. 3 a is the visible images of input, and Fig. 3 b is the near-infrared image of input.

22 pairs of sensor source images of being imported of pretreatment module carry out color interpolation obtaining several monochromatic channel image, and obtain the prior information of near-infrared image.

The prior information of the near-infrared image that need obtain in one embodiment of the invention, is the gradient of near-infrared image.

In one embodiment of the invention; Fig. 4 representes that the sensor source image of being imported is carried out color interpolation obtains several monochromatic channel image; It is understandable that Fig. 4 is just based on an example of the embodiment of the invention, rather than to the restriction of embodiments of the invention.

To every width of cloth coloured image; Constraints module 23 is at first carried out the sparse constraint of gradient difference according to the prior information of near-infrared image to several monochromatic channel image; To obtain the Pixel Information after the sparse constraint of gradient difference; According to details constraint between the corresponding a plurality of monochromatic passage of several monochromatic channel image, the Pixel Information after the sparse constraint of final updating gradient difference is to obtain the Pixel Information after details retrains then;

In one embodiment of the invention, the corresponding a plurality of monochromatic passage of several monochromatic channel image comprises R passage, G passage and B passage.

In one embodiment of the invention, the sparse constraint of gradient difference adopts following formula to calculate:

${\arg }_{v_i}\min \underset{n}{\mathrm{\Σ}}\left|\right|\▿V_i\left(n\right)-\▿N\left(n\right)\left|\right|$

Wherein, V _iRepresent R, G, B monochrome image respectively, i=1,2,3, N representes the Pixel Information of near-infrared image, n representes the index of each pixel position,

Expression is to V _iAsk gradient by a position;

Expression is asked gradient to the Pixel Information N of near-infrared image by a position,

Expression is to V _iAsk the 1-norm with the gradient difference of N.

In one embodiment of the invention, V ₁Expression R passage monochrome image, V ₂Expression G passage monochrome image, V ₃Expression B passage monochrome image it is understandable that, the example that this expression is just enumerated for embodiments of the invention are described, rather than to a kind of restriction of the embodiment of the invention.

This step is according to the prior information of near-infrared image the color interpolation processing procedure to be increased a kind of spectrum constraint, coloured image is carried out preliminary treatment, for follow-up work of treatment is laid a good foundation.

In conjunction with the high frequency correlation of R, B passage and G passage list color image, can obtain the constraint of correlation details.Because the G channel image, with R, the B channel image has very big similitude on high frequency; So the high frequency characteristics and the R of the value that can estimate to draw with the G passage; The high frequency characteristics of B passage is compared, thereby lets R, and the high-frequency information of the high frequency characteristics of B passage and G passage is close.

In one embodiment of the invention, the full resolution R channel image that satisfies the constraint of correlation details is " details constrain set ", and is specific as follows:

$C_d\stackrel{\mathrm{def}}{=}\{r(x,y):|R\⊗A_i(x,y)-G\⊗A_i(x,y)|\≤T,i=\mathrm{1,2,3}\}$

Wherein, threshold value T is used for weighing high frequency degree of correlation between R passage and the G passage, is traditionally arranged to be T=0, and A _i(i=1,2,3) are analysis filters, obtain the high-frequency information of image, are provided with as follows:

I=1: row adopts LPF, and row adopt high-pass filtering;

I=2: row adopts high-pass filtering, and row adopt LPF;

I=3: row adopts high-pass filtering, and row adopt high-pass filtering.

It is understandable that realize that this step is not limited to the given data of present embodiment, the example that present embodiment just provides for technical scheme is more readily understood can not be interpreted as limitation of the present invention.

In one embodiment of the invention, details constraint between a plurality of monochromatic passage of several monochromatic channel image correspondences comprises:

Utilize the channel value of the following formula renewal of the details constraint employing G passage between R passage, G passage and the B passage,

$G_{k+1}\left(w\right)=L\left(w\right)G_k\left(w\right)+\frac{H\left(w\right)R_k\left(w\right)+H\left(w\right)B_k\left(w\right)}{2}$

Wherein, G _K+1(w) be the G channel value after upgrading, L (w) is the LPF parameter, G _k(w) be the G channel value before upgrading, H (w) is the high-pass filtering parameter, R _k(w) be the channel value of R passage, B _k(w) be the channel value of B passage;

G channel value after the renewal adopts following formula that R passage and B passage are carried out the details constraint:

R _k+1(w)＝L(w)R _k(w)+H(w)G _k+1(w)

B _k+1(w)＝L(w)B _k(w)+H(w)G _k+1(w)

Wherein, R _K+1(w) be the R channel value after upgrading, L (w) is the LPF parameter, R _k(w) be the R channel value before upgrading, H (w) high-pass filtering parameter, G _K+1(w) be the G channel value after upgrading, B _K+1(w) be the B channel value after upgrading, B _k(w) be the B channel value before upgrading.

Embodiments of the invention utilize the G passage to come modified R, B for reference no longer merely; But use R simultaneously, the revised value of B passage is carried out high frequency confinement to the G passage.R, the B passage can be revised based on near-infrared image, in replacing iterative process, revises the G passage again, does further renewal, more helps the figure image intensifying.

According to the Pixel Information after the details constraint, computing module 24 calculates the mean square error of the Pixel Information of a plurality of passages after the corresponding details constraint of every width of cloth coloured image.

Comparison module 25 is used for the mean square error and the predetermined threshold of the Pixel Information of a plurality of passages after the details constraint of adjacent two width of cloth coloured images are compared; If the mean square error of the Pixel Information of a plurality of passages after the details constraint is more than or equal to predetermined threshold; The Pixel Information of a plurality of passages after then details being retrained is as the Pixel Information of several monochromatic channel image; Return constraints module 23; Then the Pixel Information of a plurality of passages after the details constraint is carried out computing again as the Pixel Information of several the monochromatic channel image in the constraints module 23, until the mean square error of adjacent two width of cloth coloured images less than predetermined threshold;

In one embodiment of the invention, threshold value is 0.001.It is understandable that threshold value is can be by user or default, and is not limited to the example that embodiments of the invention are given.

When the mean square error of adjacent two width of cloth coloured images during less than said predetermined threshold, the Pixel Information of a plurality of passages of fusion image generation module 26 after according to the details constraint generates the corresponding interpolation fusion image of every width of cloth coloured image.

In one embodiment of the invention, Fig. 5 a is the last interpolation fusion image that generates.

According to an embodiment of the invention based on the device of interpolation and near infrared Color Image Processing; Can realize processing synchronously to coloured image interpolation and near-infrared image; Through the method for coloured image interpolation constraint is improved, make that calculating is more quick, program simply is easy to realize; Effectively handle the problem that near-infrared and visible light are united application, the coloured image that obtains is more clear and complete.

In the description of this specification, the description of reference term " embodiment ", " some embodiment ", " example ", " concrete example " or " some examples " etc. means the concrete characteristic, structure, material or the characteristics that combine this embodiment or example to describe and is contained at least one embodiment of the present invention or the example.In this manual, the schematic statement to above-mentioned term not necessarily refers to identical embodiment or example.And concrete characteristic, structure, material or the characteristics of description can combine with suitable manner in any one or more embodiment or example.

Although illustrated and described embodiments of the invention; For those of ordinary skill in the art; Be appreciated that under the situation that does not break away from principle of the present invention and spirit and can carry out multiple variation, modification, replacement and modification that scope of the present invention is accompanying claims and be equal to and limit to these embodiment.

Claims (12)

1. the method based on interpolation and near infrared Color Image Processing is characterized in that, comprises the steps:

S1: input is carried out color interpolation obtaining several monochromatic channel image to said sensor source image, and is obtained the prior information of said near-infrared image based on the sensor source image and the near-infrared image of Same Scene in internal memory;

S2: to every width of cloth coloured image; Prior information based on said near-infrared image is carried out the sparse constraint of gradient difference to said several monochromatic channel image; Obtain the Pixel Information after the sparse constraint of gradient difference; Based on details constraint between the corresponding a plurality of monochromatic passage of said several monochromatic channel image, upgrade Pixel Information after the sparse constraint of said gradient difference to obtain the Pixel Information after the details constraint;

S3: according to the Pixel Information after the said details constraint, calculate the mean square error of the Pixel Information of a plurality of passages after the corresponding details of every width of cloth coloured image retrains,

S4: mean square error and predetermined threshold to the Pixel Information of a plurality of passages after the constraint of the details of said adjacent two width of cloth coloured images compare; If the mean square error of the Pixel Information of a plurality of passages after the said details constraint is more than or equal to said predetermined threshold; The Pixel Information of a plurality of passages after then said details being retrained is as the Pixel Information of said several the monochromatic channel image among the said step S2; Repeating said steps S2 to S4, until the mean square error of said adjacent two width of cloth coloured images less than said predetermined threshold;

S5:, generate the corresponding interpolation fusion image of said every width of cloth coloured image according to the Pixel Information of a plurality of passages after the said details constraint when the mean square error of said adjacent two width of cloth coloured images during less than said predetermined threshold.

2. the method based on interpolation and near infrared Color Image Processing as claimed in claim 1 is characterized in that, said sensor source image is a Bayer color mode image.

3. the method based on interpolation and near infrared Color Image Processing as claimed in claim 1 is characterized in that, the prior information of said near-infrared image is the gradient of said near-infrared image.

4. the method based on interpolation and near infrared Color Image Processing as claimed in claim 1 is characterized in that, the corresponding a plurality of monochromatic passage of said several monochromatic channel image comprises R passage, G passage and B passage.

5. the method based on interpolation and near infrared Color Image Processing as claimed in claim 4 is characterized in that, the sparse constraint of said gradient difference adopts following formula to calculate:

${\arg }_{v_i}\min \underset{n}{\mathrm{\Σ}}\left|\right|\▿V_i\left(n\right)-\▿N\left(n\right)\left|\right|$

Wherein, V _iRepresent R, G, B monochrome image respectively, i=1,2,3, N representes the Pixel Information of said near-infrared image, n representes the index of each pixel position,

Expression is to V _iAsk gradient by a position; Expression is asked gradient to the Pixel Information N of said near-infrared image by a position,

Expression is to V _iAsk the 1-norm with the gradient difference of N.

6. the method based on interpolation and near infrared Color Image Processing as claimed in claim 5 is characterized in that, according to details constraint between the corresponding a plurality of monochromatic passage of said several monochromatic channel image, comprises the steps:

Utilize the details between said R passage, G passage and the B passage to retrain the channel value that adopts following formula to upgrade said G passage,

$G_{k+1}\left(w\right)=L\left(w\right)G_k\left(w\right)+\frac{H\left(w\right)R_k\left(w\right)+H\left(w\right)B_k\left(w\right)}{2}$

Wherein, G _K+1(w) be the G channel value after upgrading, L (w) is the LPF parameter, G _k(w) be the G channel value before upgrading, H (w) is the high-pass filtering parameter, R _k(w) be the channel value of said R passage, B _k(w) be the channel value of said B passage;

Utilize the G channel value after the said renewal that said R passage and said B passage are adopted following formula, carry out the details constraint

R _k+1(w)＝L(w)R _k(w)+H(w)G _k+1(w)

B _k+1(w)＝L(w)B _k(w)+H(w)G _k+1(w)

Wherein, R _K+1(w) be the R channel value after upgrading, L (w) is the LPF parameter, R _k(w) be the R channel value before upgrading, H (w) high-pass filtering parameter, G _K+1(w) be G channel value after the said renewal, B _K+1(w) be the B channel value after upgrading, B _k(w) be the B channel value before upgrading.

7. the device based on interpolation and near infrared Color Image Processing is characterized in that, comprising:

Input module, said input module are used for to sensor source image and the near-infrared image of internal memory input based on Same Scene;

Pretreatment module, said pretreatment module links to each other with said input module, is used for said sensor source image is carried out color interpolation obtaining several monochromatic channel image, and obtains the prior information of said near-infrared image;

Constraints module; Said constraints module links to each other with said pretreatment module; Prior information according to said near-infrared image is carried out the sparse constraint of gradient difference to said several monochromatic channel image; Obtain the Pixel Information after the sparse constraint of gradient difference,, upgrade Pixel Information after the sparse constraint of said gradient difference to obtain the Pixel Information after the details constraint according to details constraint between the corresponding a plurality of monochromatic passage of said several monochromatic channel image;

Computing module, said computing module links to each other with said constraints module, according to the Pixel Information after the said details constraint, calculates the mean square error of the Pixel Information of a plurality of passages after the corresponding details of every width of cloth coloured image retrains;

Comparison module; Said comparison module links to each other with said constraints module with said computing module respectively; Mean square error and predetermined threshold to the Pixel Information of a plurality of passages after the constraint of the details of said adjacent two width of cloth coloured images compare; If the mean square error of the Pixel Information of a plurality of passages after the said details constraint is more than or equal to said predetermined threshold; The Pixel Information of a plurality of passages after then said details being retrained is as the Pixel Information of said several the monochromatic channel image in the said constraints module; Said constraints module is carried out sparse constraint of gradient difference and details constraint to the Pixel Information of said several the monochromatic channel image after upgrading; Pixel Information after the constraint of details after obtaining upgrading, the mean square error of the Pixel Information of a plurality of passages after the said computing module computational details constraint, said comparison module compares mean square error until said adjacent two width of cloth coloured images less than said predetermined threshold to the mean square error of the Pixel Information of the said a plurality of passages after upgrading and said predetermined threshold;

The fusion image generation module; Said fusion image generation module links to each other with said comparison module; When the mean square error of said adjacent two width of cloth coloured images during, generate the corresponding interpolation fusion image of said every width of cloth coloured image according to the Pixel Information of a plurality of passages after the said details constraint less than said predetermined threshold.

8. the device based on interpolation and near infrared Color Image Processing as claimed in claim 7 is characterized in that, said sensor source image is a Bayer color mode image.

9. the device based on interpolation and near infrared Color Image Processing as claimed in claim 7 is characterized in that, the prior information of said near-infrared image is the gradient of said near-infrared image.

10. the device based on interpolation and near infrared Color Image Processing as claimed in claim 7 is characterized in that, the corresponding a plurality of monochromatic passage of said several monochromatic channel image comprises R passage, G passage and B passage.

11. the device based on interpolation and near infrared Color Image Processing as claimed in claim 10 is characterized in that, said constraints module adopts following formula that said several monochromatic channel image are carried out the sparse constraint of gradient difference:

${\arg }_{v_i}\min \underset{n}{\mathrm{\Σ}}\left|\right|\▿V_i\left(n\right)-\▿N\left(n\right)\left|\right|$

Wherein, V _iRepresent R, G, B monochrome image respectively, i=1,2,3, N representes the Pixel Information of said near-infrared image, n representes the index of each pixel position, Expression is to V _iAsk gradient by a position;

Expression is asked gradient to the Pixel Information N of said near-infrared image by a position,

Expression is to V _iAsk the 1-norm with the gradient difference of N.

12. the device based on interpolation and near infrared Color Image Processing as claimed in claim 11 is characterized in that, said constraints module utilizes the details between said R passage, G passage and the B passage to retrain the channel value that adopts following formula to upgrade said G passage,

$G_{k+1}\left(w\right)=L\left(w\right)G_k\left(w\right)+\frac{H\left(w\right)R_k\left(w\right)+H\left(w\right)B_k\left(w\right)}{2}$

Wherein, G _K+1(w) be the G channel value after upgrading, L (w) is the LPF parameter, G _k(w) be the G channel value before upgrading, H (w) is the high-pass filtering parameter, R _k(w) be the channel value of said R passage, B _k(w) be the channel value of said B passage;

Said constraints module utilizes the G channel value after the said renewal to adopt following formula to carry out the details constraint to said R passage and said B passage

R _k+1(w)＝L(w)R _k(w)+H(w)G _k+1(w)

B _k+1(w)＝L(w)B _k(w)+H(w)G _k+1(w)

Wherein, R _K+1(w) be the R channel value after upgrading, L (w) is the LPF parameter, R _k(w) be the R channel value before upgrading, H (w) high-pass filtering parameter, G _K+1(w) be G channel value after the said renewal, B _K+1(w) be the B channel value after upgrading, B _k(w) be the B channel value before upgrading.

Images