CN105869114B - Multispectral image and panchromatic image fusion method based on multilayer interband structural model - Google Patents
Multispectral image and panchromatic image fusion method based on multilayer interband structural model Download PDFInfo
- Publication number
- CN105869114B CN105869114B CN201610179673.7A CN201610179673A CN105869114B CN 105869114 B CN105869114 B CN 105869114B CN 201610179673 A CN201610179673 A CN 201610179673A CN 105869114 B CN105869114 B CN 105869114B
- Authority
- CN
- China
- Prior art keywords
- image
- resolution
- layer
- multispectral
- mean value
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
- 238000007500 overflow downdraw method Methods 0.000 title claims abstract description 23
- 238000000034 method Methods 0.000 claims abstract description 48
- 230000009466 transformation Effects 0.000 claims abstract description 34
- 238000012952 Resampling Methods 0.000 claims abstract description 6
- 230000004927 fusion Effects 0.000 claims description 19
- 238000000354 decomposition reaction Methods 0.000 claims description 14
- 230000015572 biosynthetic process Effects 0.000 claims description 4
- 238000002156 mixing Methods 0.000 claims description 4
- 238000003786 synthesis reaction Methods 0.000 claims description 4
- TVEXGJYMHHTVKP-UHFFFAOYSA-N 6-oxabicyclo[3.2.1]oct-3-en-7-one Chemical compound C1C2C(=O)OC1C=CC2 TVEXGJYMHHTVKP-UHFFFAOYSA-N 0.000 claims description 2
- 230000033228 biological regulation Effects 0.000 claims description 2
- 239000011159 matrix material Substances 0.000 claims description 2
- 238000011426 transformation method Methods 0.000 claims description 2
- 239000002356 single layer Substances 0.000 abstract description 5
- 239000010410 layer Substances 0.000 description 50
- 238000001228 spectrum Methods 0.000 description 27
- 238000013461 design Methods 0.000 description 8
- 238000002474 experimental method Methods 0.000 description 8
- 230000008901 benefit Effects 0.000 description 7
- 230000003595 spectral effect Effects 0.000 description 7
- 230000000694 effects Effects 0.000 description 4
- 238000004321 preservation Methods 0.000 description 3
- 238000005070 sampling Methods 0.000 description 3
- 238000012360 testing method Methods 0.000 description 3
- 238000004458 analytical method Methods 0.000 description 2
- 238000010586 diagram Methods 0.000 description 2
- 230000001737 promoting effect Effects 0.000 description 2
- TUSUWHFYKZZRIG-JQWMYKLHSA-N C([C@@H](NC(=O)[C@@H](C(C)C)NC(=O)[C@@H](CC(C)C)NC)C(=O)N[C@H](CC=1C=CC=CC=1)C(=O)N[C@H](CC(C)C)C(N)=O)C1=CC=CC=C1 Chemical compound C([C@@H](NC(=O)[C@@H](C(C)C)NC(=O)[C@@H](CC(C)C)NC)C(=O)N[C@H](CC=1C=CC=CC=1)C(=O)N[C@H](CC(C)C)C(N)=O)C1=CC=CC=C1 TUSUWHFYKZZRIG-JQWMYKLHSA-N 0.000 description 1
- 208000036758 Postinfectious cerebellitis Diseases 0.000 description 1
- 238000004364 calculation method Methods 0.000 description 1
- 238000007796 conventional method Methods 0.000 description 1
- 230000007423 decrease Effects 0.000 description 1
- 230000002708 enhancing effect Effects 0.000 description 1
- ZZUFCTLCJUWOSV-UHFFFAOYSA-N furosemide Chemical group C1=C(Cl)C(S(=O)(=O)N)=CC(C(O)=O)=C1NCC1=CC=CO1 ZZUFCTLCJUWOSV-UHFFFAOYSA-N 0.000 description 1
- 238000003384 imaging method Methods 0.000 description 1
- 230000014759 maintenance of location Effects 0.000 description 1
- 238000013178 mathematical model Methods 0.000 description 1
- VMXUWOKSQNHOCA-UKTHLTGXSA-N ranitidine Chemical compound [O-][N+](=O)\C=C(/NC)NCCSCC1=CC=C(CN(C)C)O1 VMXUWOKSQNHOCA-UKTHLTGXSA-N 0.000 description 1
- 238000011160 research Methods 0.000 description 1
- 238000004611 spectroscopical analysis Methods 0.000 description 1
- 230000002194 synthesizing effect Effects 0.000 description 1
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T3/00—Geometric image transformations in the plane of the image
- G06T3/40—Scaling of whole images or parts thereof, e.g. expanding or contracting
- G06T3/4053—Scaling of whole images or parts thereof, e.g. expanding or contracting based on super-resolution, i.e. the output image resolution being higher than the sensor resolution
- G06T3/4061—Scaling of whole images or parts thereof, e.g. expanding or contracting based on super-resolution, i.e. the output image resolution being higher than the sensor resolution by injecting details from different spectral ranges
Landscapes
- Physics & Mathematics (AREA)
- Spectroscopy & Molecular Physics (AREA)
- General Physics & Mathematics (AREA)
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Compression Or Coding Systems Of Tv Signals (AREA)
- Image Processing (AREA)
Abstract
Based on the multispectral image and panchromatic image fusion method of multilayer interband structural model, belong to image interfusion method field.In order to solve during establishing ARSIS model, limitation of the single layer detail pictures when describing interband relationship.The present invention is by low resolution multispectral image MSLRResampling is to PAN image size, using its mean value as z1LRImage；To z1LRAnd MSLRGS orthogonal transformation is carried out, by z1LRThe transformed image of image is denoted asMultiple dimensioned model M SM is constructed using SWT；Approximate image each in MSM and detail pictures are divided into the image block of 64 × 64 sizes；It is rightWithGlobal structure similarity SSIM is calculated, partial structurtes similarity is calculated to each image block, and the model of application definition acquiresIt is obtained using SWT inverse transformationObtain z1HRGS inverse transformation is carried out afterwards；Export high-resolution multi-spectral image MSHR.The spatial resolution of multispectral image is effectively promoted in the present invention, realizes the purpose of the panchromatic sharpening of multispectral image.
Description
Technical field
The present invention relates to a kind of multispectral images and panchromatic image fusion method based on multilayer interband structural model.
Background technique
ARSIS model refers to through injecting structure information the fusion method for promoting image spatial resolution.Due to mostly light
Spectrogram as spectral information indicated by its low frequency part, spatial information is then indicated by high frequency section.Therefore, it is keeping
There is significant advantage in terms of spectral characteristic.The core of ARSIS model is multiple dimensioned model (MSM) and interband structural model
(IBSM).By establishing the IBSM of detail pictures under multispectral image low resolution, accurately describing it in multiple dimensioned model
Detailed information under the high-resolution lacked.Existing interband structural model is usually with the details subgraph of single layer under low resolution
As removing building IBSM, but the detailed information of other levels is not effectively utilised in multiple dimensioned model, causes information
It loses seriously, model describes not accurate enough phenomenon.Even more important is a bit, the differences in resolution of remote sensing images and uses mathematics
Model not is completely the same when going to describe.That is, the multiresolution analysis method generally used, as pyramid becomes
It changes, wavelet transformation etc., is different with the differences in resolution in real image.This is because influencing the factor of actual imaging process
It is increasingly complex when often than going to describe with simple mathematical model.For example, the multispectral figure of existing some commercial satellites
The resolution ratio of picture and full-colour image is usually 4, such as QuickBird, IKONOS, however is carried out to full-colour image 2 layers small
Image (theoretically with multispectral image resolution ratio having the same) after Wave Decomposition, between true multispectral image still
There is the differences in resolution ratio, therefore, go to describe this interband relationship using single layer detail pictures to there is biggish mistake
Difference.
Summary of the invention
The purpose of the present invention is to solve during establishing ARSIS model, single layer detail pictures are in description interband
The problem of limitation when relationship, and propose a kind of multispectral image based on multilayer interband structural model and full-colour image fusion
Method.
A kind of multispectral image and panchromatic image fusion method based on multilayer interband structural model, the method by with
Lower step is realized:
For the image MS to be fused of inputLRWith High-resolution Panchromatic Images PAN；
Step 1: by low resolution multispectral image MSLRUtilize bilinear interpolation resampling to full-colour image PAN image
Size, and by image MS to be fusedLRMean value image as mean value image z1LR；
Step 2: to mean value image z1LRWith image MS to be fusedLRGS orthogonal transformation is carried out, by mean value image z1LRAfter transformation
Image be denoted as scale be 2 low resolution multispectral imageI.e. the 2nd layer approximate low resolution multispectral image
Step 3: using stationary wavelet transform SWT to the 2nd layer of approximate low resolution multispectral imageCarry out 2 layers of small echo
Transformation carries out 4 layers of wavelet transformation to High-resolution Panchromatic Images PAN, constructs multiple dimensioned model M SM；
Wherein, stationary wavelet transform refers to, by the 2nd layer of approximate low resolution multispectral imageIt is panchromatic with high-resolution
Image PAN carries out 2 layers and four layers decomposition respectively；Every one layer of decomposition can generate an approximate image and horizontal, vertical, diagonal three
The detail pictures in direction, the more high then resolution ratio of the number of plies is lower, and High-resolution Panchromatic Images PAN is identical as the resolution ratio of z1HR, andWithAnd MSLRWith z1LRAll there is equal resolution；And the 1st, 2 layer be known as low layer, and the 3rd, 4 layer and referred above to
It is high-rise；
Step 4: by each approximate image in multiple dimensioned model M SM to AMSWith APANAnd detail pictures are to DMSWith DPAN, all
It is divided into the image block of 64 × 64 pixel sizes；
Step 5: the approximate image pair for being 2 for scale: the 2nd layer of approximate low resolution multispectral imageWith the 2nd layer
Low resolution full-colour imageGlobal structure similarity SSIM is calculated, the partial structurtes similarity of each image block is calculated
LSSIM, and finally acquire the Lower-level details coefficient image of each image block
Step 6: the 2nd layer of detail coefficients image that step 5 acquiresLow resolution multispectral image approximate with the 2nd layerStationary wavelet inverse transformation is carried out, synthesis obtains the approximate low resolution multispectral image that scale is 1I.e. the 1st layer low point
Resolution multispectral image
Step 7: the approximate image pair for being 1 for scale: the 1st layer of low resolution multispectral imageWith the 1st layer low point
Resolution full-colour imageThe process for repeating step 4 to step 6 obtains the mean value image z of high-resolution multi-spectral image1HR,
That is the mean value image z of high-resolution multi-spectral image1HR；
Step 8: to the mean value image z of high-resolution multi-spectral image1HRWith low resolution multispectral image MSLRCarry out GS
Inverse transformation；
Final output high-resolution multi-spectral image MSHR。
The invention has the benefit that
To solve during establishing ARSIS model, limitation of the single layer detail pictures when describing interband relationship, this
Invention devises a kind of multispectral figure based on multilayer interband structural model on the basis of analyzing the interband relationship of different levels
As fusion method, the mutual of the Bu Tong high-rise detail pictures under the multispectral multi-resolution models with full-colour image is efficiently utilized
Relationship, more efficiently improves the spatial resolution of multispectral image, realizes multispectral the shortcomings that overcoming conventional method
The purpose of the panchromatic sharpening of image.
The fusion method that the present invention designs is shown to the experiment of the panchromatic sharpening of multispectral image, is not only had highest flat
Equal gradient, i.e., spatial information abundant, and there is the smallest error, i.e. spectral preservation information.Sharpening to high spectrum image
Experiment shows that the fusion method that the present invention designs has the smallest fusion error and highest signal-to-noise ratio.With the general side of fusion
Method is compared, and this method shows significant advantage.
Detailed description of the invention
Fig. 1 is the panchromatic sharpening scheme signal of the multispectral image based on multilayer interband structural model that the method for the present invention is related to
Figure；
Fig. 2 is that the present invention tests multispectral and full-colour image used, respectively Fig. 2 a- Fig. 2 f are as follows: QuickBird, IKONOS
With WorldView-II multispectral image and QuickBird, IKONOS and WorldView-II full-colour image；
Fig. 3 a and Fig. 3 b are that the present invention tests two panel height spectrum pictures used；Fig. 3 a are as follows: Santiago naval base image；
Fig. 3 b are as follows: university of Pavia image；
Fig. 4 is 6 width partial fusion result effect diagrams: where Fig. 4 a is the original multispectral image of Hobart, and Fig. 4 b is
The fusion results of CBD model, Fig. 4 c are the fusion results of proposition method, and Fig. 4 d is the original multispectral figure of Rio de Janeiro
Picture, Fig. 4 e are the fusion results of RWM model, and Fig. 4 f is the fusion results of proposition method；
Fig. 5 a-e is that 5 panel height spectrum picture fusion results compare: where Fig. 5 a and Fig. 5 d are that original high spectrum image is false color
Image after color synthesis, Fig. 5 b are the curve of spectrum of Aircraft Targets in red circle in Fig. 5 a, and Fig. 5 c stops for what is marked in Fig. 5 a
The curve of spectrum on machine level ground, Fig. 5 e are the curve of spectrum of the trees in Fig. 5 d in red circle；
Fig. 6 a, Fig. 6 b are that the Y-PSNR of two panel height spectrum picture fusion results compares: where Fig. 6 a: Santiago
Naval base image；Fig. 6 b: university of Pavia image；
Fig. 7 a and Fig. 7 b are respectively blending image average gradient AG and root-mean-square error RMSE and the Stationary Wavelet Decomposition number of plies
Relation schematic diagram, Fig. 7 a be Boulder data, Fig. 7 b be Hobart data；
Fig. 8 is the method for the present invention flow chart.
Specific embodiment
Specific embodiment 1:
The multispectral image and panchromatic image fusion method based on multilayer interband structural model of present embodiment, in conjunction with figure
Method flow shown in 1, the method are realized by following steps:
For the image MS to be fused of inputLRWith High-resolution Panchromatic Images PAN；
Step 1: by low resolution multispectral image MSLRUtilize bilinear interpolation resampling to full-colour image PAN image
Size, and by image MS to be fusedLRMean value image as mean value image z1LR；
Step 2: to mean value image z1LRWith image MS to be fusedLRGS orthogonal transformation is carried out, by mean value image z1LRAfter transformation
Image be denoted as scale be 2 low resolution multispectral imageI.e. the 2nd layer approximate low resolution multispectral image
Step 3: using stationary wavelet transform SWT to the 2nd layer of approximate low resolution multispectral imageCarry out 2 layers of small echo
Transformation carries out 4 layers of wavelet transformation to High-resolution Panchromatic Images PAN, constructs multiple dimensioned model M SM；
Wherein, stationary wavelet transform refers to, by the 2nd layer of approximate low resolution multispectral imageIt is panchromatic with high-resolution
Image PAN carries out 2 layers and four layers decomposition respectively；Every one layer of decomposition can generate an approximate image and horizontal, vertical, diagonal three
The detail pictures in direction, the more high then resolution ratio of the number of plies is lower, so High-resolution Panchromatic Images PAN and z in Fig. 11HRResolution
Rate is identical, andWithAnd MSLRWith z1LRAll there is equal resolution；And the 1st, 2 layer be known as low layer, and the 3rd, 4 layer and
Referred above to high level；
Step 4: by each approximate image in multiple dimensioned model M SM to AMSWith APANAnd detail pictures are to DMSWith DPAN, all
It is divided into the image block of 64 × 64 pixel sizes；
Step 5: the approximate image pair for being 2 for scale: the 2nd layer of approximate low resolution multispectral imageWith the 2nd layer
Low resolution full-colour imageGlobal structure similarity SSIM is calculated, the partial structurtes similarity of each image block is calculated
LSSIM, and finally acquire the Lower-level details coefficient image of each image block
Step 6: the 2nd layer of detail coefficients image that step 5 acquiresLow resolution multispectral image approximate with the 2nd layerStationary wavelet inverse transformation is carried out, synthesis obtains the approximate low resolution multispectral image that scale is 1I.e. the 1st layer low point
Resolution multispectral image
Step 7: the approximate image pair for being 1 for scale: the 1st layer of low resolution multispectral imageWith the 1st layer low point
Resolution full-colour imageThe process for repeating step 4 to step 6 obtains the mean value image z of high-resolution multi-spectral image1HR,
That is the mean value image z of high-resolution multi-spectral image1HR；
Step 8: to the mean value image z of high-resolution multi-spectral image1HRWith low resolution multispectral image MSLRCarry out GS
Inverse transformation；
Final output high-resolution multi-spectral image MSHR。
Specific embodiment 2:
Unlike specific embodiment one, the multispectral image based on multilayer interband structural model of present embodiment
And panchromatic image fusion method, by low resolution multispectral image MS described in step 1LRIt is arrived using bilinear interpolation resampling
High-resolution Panchromatic Images PAN image size, and by image MS to be fusedLRMean value image as mean value image z1LRProcess
For to low resolution multispectral image MSLRThe summation of each band image mean value image is generated, as z divided by band image number1
Image
Specific embodiment 3:
Unlike specific embodiment one or two, present embodiment based on the multispectral of multilayer interband structural model
Image and panchromatic image fusion method, to mean value image z described in step 21LRWith image MS to be fusedLRGS orthogonal transformation is carried out,
By mean value image z1LRTransformed image is denoted as the low resolution multispectral image that scale is 2That is the 2nd layer of low resolution of approximation
Rate multispectral imageProcess be that general ARSIS model is typically directly applied on each wave band of multispectral image,
Also have and principal component transform (PCA) first is carried out to multispectral image, and ARSIS model is applied to the research on first principal component.
Studies have shown that Gram-Schmidt wave spectrum, which sharpens (GS) method, can generate preferable effect to Multispectral Image Fusion.Therefore,
Step 2 one, by low resolution multispectral image MSLRCarry out GS orthogonal transformation:
If Z=XR-1Indicate improved GS orthogonal transformation method, in formula, X indicates low resolution multispectral image MSLRRespectively
Wave band, Z indicate transformed each component, and R is transformation matrix, and
Element in R calculates as follows:
zk=xk/‖xk‖, subscript k indicate the component currently calculated, k=1,2 ..., s；Subscript j table
Show each component after present component, j=k+1 ..., s, calculates a r every timekjAfterwards, xjIt is updated to xj=xj-zk·rkj；
Step 2 two, the mean value image z obtained using step 11LRIt calculates GS and converts other component zs, then low resolution is more
Spectrum picture MSLRIt is established respectively with the multiple dimensioned model of High-resolution Panchromatic Images PAN in mean value image z1LRIt is complete with high-resolution
Chromatic graph is as on PAN, as shown in Figure 1, mean value image z1LRFor the mean value image synthesized above, correspond in MSMz1HRFor
The fused image of ARSIS, with other component zsGS inverse transformation is carried out, fused high-resolution multi-spectral image MS is obtainedHR。
Specific embodiment 4:
Unlike specific embodiment three, the multispectral image based on multilayer interband structural model of present embodiment
And panchromatic image fusion method, using stationary wavelet transform SWT to the 2nd layer of approximate low resolution multispectral image described in step 3 2 layers of wavelet transformation are carried out, after carrying out 4 layers of wavelet transformation to High-resolution Panchromatic Images PAN, are obtained: an approximate imageOrAnd the detail pictures D=D respectively on horizontal, vertical, diagonal three directionsH, DV, DD, due to three directions
Independently, it is independent of each other, and method used in back is all identical, so being indicated with a D.
Specific embodiment 5:
Unlike specific embodiment one, two or four, present embodiment based on the more of multilayer interband structural model
Spectrum picture and panchromatic image fusion method, by each approximate image in multiple dimensioned model M SM to A described in step 4MSWith APAN, with
And detail pictures are to DMSWith DPAN, the process for being all divided into the image block of 64 × 64 pixel sizes is, without loss of generality,
Step 4 one indicates each coefficient layer in MSM using symbol defined above:WithIndicate stationary wavelet
Scale is horizontal, vertical and diagonal direction the levels of detail on i after transformation；
Step 4 two, the first step of multilayer interband structural model MLIBSM are all to divide each approximate image and detail pictures
For the image block of 64 × 64 sizes, each layer coefficients are calculated in each image block respectively later.
Specific embodiment 6:
Unlike specific embodiment five, the multispectral image based on multilayer interband structural model of present embodiment
And panchromatic image fusion method, the approximate image pair for being 2 for scale described in step 5: the 2nd layer of approximate low resolution is multispectral
ImageWith the 2nd layer of low resolution full-colour imageGlobal structure similarity SSIM is calculated, the office of each image block is calculated
Portion structural similarity LSSIM, and finally acquire the Lower-level details coefficient image of each image blockProcess be,
Step 5 one, the approximate image pair for being 2 for the scale in Fig. 1: the 2nd layer of approximate low resolution multispectral image With the 2nd layer of low resolution full-colour imageCalculate whole graph structure similarity:
In formula, μ and σ are the respective mean value of image and standard deviation respectively；Cov isWithCovariance；c1And c2It is
The constant for being much smaller than 1 in order to guarantee denominator being not 0 and being arranged；
Step 5 two, the partial structurtes similarity LSSIM for calculating each image block；
Step 5 three calculates each image block
If
Then
Otherwise
In formula, the value of scale i is 2；α, β indicate regulation coefficient, for by the Lower-level details coefficient of full-colour imageIt adjusts
The whole Lower-level details coefficient for multispectral image
The optimal solution of calculating parameter α and β later；
Step 5 four, the definition according to ARSIS, the pass that the relationship of high-rise detail coefficients will as far as possible with low-level details coefficient
System is consistent, thenIt is equally applicable to scale i=3,4 ...；Due to
Then α and β are respectively indicated are as follows:
It is defined as follows objective function, the α found out and β is made preferably to be fitted the detail coefficients of different scale:
In formula, N is Decomposition order；
It is calculate by the following formula partial derivative of the f (α) about α:
Then formula (7) indicates are as follows:
To which the optimal solution of α in formula (5) must be found out are as follows:
Similarly, it is calculate by the following formula the partial derivative of g (β):
The optimal solution of β in formula (6) must be found out are as follows:
The optimal solution of the α acquired and β is substituted into formula (2), each image block is obtainedAt this point, scale i=2, j=
The high level of 3,4 ..., N, j expression MSM.
Specific embodiment 7:
Unlike specific embodiment one, two, four or six, present embodiment based on multilayer interband structural model
Multispectral image and panchromatic image fusion method, the 1st layer of low resolution multispectral image for being 1 for scale described in step 7 With the 1st layer of low resolution full-colour imageThe process for repeating step 4 to step 6 obtains high-resolution multi-spectral image
Mean value image z1HRProcess be,
Step 7 one, the model for indicating formula (2) are applied in the plane that scale is 1, and are asked using formula (9) and formula (11)
The optimal solution of α and β out, obtainsWherein, i=1, j=2,3,4 ..., N；
Step 7 two, progress wavelet inverse transformation obtain the mean value image z of high-resolution multispectral image1HRImage.
Embodiment 1:
Experiment of the invention is for three groups of satellite-borne multispectral/full-colour images and two groups of high spectrum image expansion.Light more than three groups
Spectrum and full-colour image are as shown in Fig. 2, parameter is as shown in the table.4 times of sub-samplings are carried out to all images in experiment, and original more
Spectrum picture as reference picture, image fused so just have with original multispectral image resolution ratio having the same,
So as to more effectively be evaluated.
Table 1 tests multispectral and full-colour image detail parameters
Meanwhile in order to preferably show advantage of the method for the invention designed in terms of spectrum retention performance, two are used
Group high spectrum image carries out emulation experiment, as shown in Figure 3.Wherein, the first panel height spectrum picture is a width low orbit AVIRIS high
Spectroscopic data is collected in California, USA Santiago naval base, it includes 126 wave bands, and size is 400 × 400,
Resolution ratio is 3.5 meters.Here the wave band of serial number 6-36 is synthesized into a width full-colour image, and original high spectrum image is carried out
4 times of down-samplings carry out emulation experiment.Second width image is ROSIS high-spectral data, is collected in Italy Pavia university, is wrapped
Containing 103 wave bands, size is that 610 × 340 resolution ratio are 1.3 meters.Similarly, the wave band of 1-65 is synthesized into a width full-colour picture
Picture, and 4 times of down-samplings are carried out to original high spectrum image, carry out emulation experiment.
Experimental result and analysis:
Experimental result is measured by some common fusion rules indexs, respectively average gradient AG, root-mean-square error
RMSE, opposite dimensionless overall situation composition error ERGAS.Table 2 show each wave band mean value of blending image.
Table 2 is multispectral and full-colour image fusion results
The method (i.e. Prop.) proposed as can be seen from the table has in terms of enhancing spatial information and spectral preservation characteristic
Significant advantage, as all having maximum average gradient, the smallest error in most situations.APCA-CT is in spectral preservation
There is certain advantage in terms of characteristic, especially in the second set of experiments.Fig. 4 shows partial fusion result figure.It can be with from figure
Find out, the method for proposition can enhance image definition, such as Hobart image, can be more clearly in red circle
See the profile of building.For Rio de Janeiro image, the method for proposition is it is observed that in red circle
The center line of runway, and in middle graph, then do not observe.Therefore, from the results of view, the method that the present invention designs is sharp in image
Changing aspect has significant advantage.
Table 3 lists the fusion results of high spectrum image.As can be seen from the table, in addition to excellent in terms of spatial resolution
Except gesture, method of the invention has the smallest fusion error, therefore fusion results are closest to ideal situation.
3 high spectrum image fusion results of table
Fig. 5 shows the curve of spectrum of partial target or atural object.It can be seen from the figure that in the first panel height spectrum picture
In, method that the present invention designs keeps best for the spectral information of Aircraft Targets, and the curve of spectrum is closest and reference picture
(REF).And for airplane parking area, method of the invention and M2 are closer to reference picture.For the second panel height spectrum
For image, the curve of spectrum of these three methods is all in close proximity to reference picture, but (the green at two reflection peaks of vegetation
At wave band and near infrared band), method of the invention is slightly better than other two kinds.In order to show several method more on a macro scale
Several method fusion results are drawn Y-PSNR (PSNR) by wave band, as shown in Figure 6 by superiority and inferiority.It can be seen from the figure that phase
Than under, the method that the present invention designs has highest Y-PSNR, i.e., closest to original high spectrum image.In addition,
It can be seen that, respective Y-PSNR is relatively high in preceding tens wave bands, and then compares in the wave band below in two width figures
It is low, this is because only having used preceding tens wave bands when synthesizing full-colour image.Such as Santiago naval base image
For, the wave-length coverage of full-colour image covers wave band 6-36, therefore the Y-PSNR of preceding 36 wave bands in the figure is higher,
And value later is then lower.
From the point of view of final experimental result, the method that the present invention designs is promoting image spatial resolution and is keeping image light
Other several methods are superior in terms of spectral property.
In addition, influence of the wavelet decomposition number of plies for image syncretizing effect in the method for this patent design is provided by Fig. 7,
It can be seen from the figure that a certain extent, it is more quasi- for the description of detailed information with the increase of the wavelet decomposition number of plies
Really, therefore, the average gradient of blending image increases therewith, and root-mean-square error then decreases, and illustrates that image syncretizing effect exists
It gradually increases.But the increase of the wavelet decomposition number of plies, calculation amount can be made and calculate the time in the trend significantly increased.Therefore, exist
In the method that the present invention designs, the number of plies of Stationary Wavelet Decomposition is generally set 4-5 layers.
Claims (7)
1. multispectral image and panchromatic image fusion method based on multilayer interband structural model, it is characterised in that: the method
It is realized by following steps:
For the image MS to be fused of inputLRWith High-resolution Panchromatic Images PAN；
Step 1: by image MS to be fusedLRIt is big to High-resolution Panchromatic Images PAN image using bilinear interpolation resampling
It is small, and by image MS to be fusedLRMean value image as mean value image z1LR；
Step 2: to mean value image z1LRWith image MS to be fusedLRGS orthogonal transformation is carried out, by mean value image z1LRTransformed figure
The low resolution multispectral image for being 2 as being denoted as scaleI.e. the 2nd layer approximate low resolution multispectral image
Step 3: using stationary wavelet transform SWT to the 2nd layer of approximate low resolution multispectral image2 layers of small echo are carried out to become
It changes, 4 layers of wavelet transformation is carried out to High-resolution Panchromatic Images PAN, construct multiple dimensioned model M SM；
Wherein, stationary wavelet transform refers to, by the 2nd layer of approximate low resolution multispectral imageAnd High-resolution Panchromatic Images
PAN carries out 2 layers and four layers decomposition respectively；Every one layer of decomposition can generate an approximate image and horizontal, vertical, diagonal three directions
Detail pictures, the more high then resolution ratio of the number of plies is lower, High-resolution Panchromatic Images PAN and z1HRResolution ratio it is identical, z1HRFor height
The mean value image of resolution multi-spectral image, andWithAnd MSLRWith z1LRAll there is equal resolution,It is the 2nd
Layer low resolution full-colour image；And the 1st, 2 layer is known as low layer, and the 3rd, 4 layer and referred above to high level；
Step 4: by each approximate image in multiple dimensioned model M SM to AMSWith APANAnd detail pictures are to DMSWith DPAN, all divide
For the image block of 64 × 64 pixel sizes；
Step 5: the approximate image pair for being 2 for scale: the 2nd layer of approximate low resolution multispectral imageWith the 2nd layer low point
Resolution full-colour imageGlobal structure similarity SSIM is calculated, the partial structurtes similarity LSSIM of each image block is calculated,
And finally acquire the 2nd layer of detail coefficients image of each image block
Step 6: the 2nd layer of detail coefficients image that step 5 acquiresLow resolution multispectral image approximate with the 2nd layer
Stationary wavelet inverse transformation is carried out, synthesis obtains the approximate low resolution multispectral image that scale is 1That is the 1st layer of low resolution
Multispectral image
Step 7: the approximate image pair for being 1 for scale: the 1st layer of low resolution multispectral imageWith the 1st layer of low resolution
Full-colour imageThe process for repeating step 4 to step 6 obtains the mean value image z of high-resolution multi-spectral image1HR, i.e., high
The mean value image z of resolution multi-spectral image1HR；
Step 8: to the mean value image z of high-resolution multi-spectral image1HRWith image MS to be fusedLRCarry out GS inverse transformation；
Final output high-resolution multi-spectral image MSHR。
2. multispectral image and panchromatic image fusion method according to claim 1 based on multilayer interband structural model,
It is characterized in that: by image MS to be fused described in step 1LRUtilize bilinear interpolation resampling to High-resolution Panchromatic Images PAN
Image size, and by image MS to be fusedLRMean value image as mean value image z1LRProcess be to treat blending image MSLR
The summation of each band image mean value image is generated, as z divided by band image number1LRImage.
3. the multispectral image and panchromatic image fusion method according to claim 1 or claim 2 based on multilayer interband structural model,
It is characterized by: to mean value image z described in step 21LRWith image MS to be fusedLRGS orthogonal transformation is carried out, by mean value image z1LR
Transformed image is denoted as the low resolution multispectral image that scale is 2I.e. the 2nd layer approximate low resolution multispectral imageProcess be,
Step 2 one, by image MS to be fusedLRCarry out GS orthogonal transformation:
If Z=XR-1Indicate improved GS orthogonal transformation method, in formula, X indicates image MS to be fusedLREach wave band, Z indicate to become
Each component after changing, R are transformation matrix, and
Element in R calculates as follows:
zk=xk/||xk| |, subscript k indicates the component currently calculated, k=1,2 ..., s；Subscript j is indicated
Each component after present component, j=k+1 ..., s calculate a r every timekjAfterwards, xjIt is updated to xj=xj-zk·rkj；
Step 2 two, the mean value image z obtained using step 11LRIt calculates GS and converts other component zs, then image MS to be fusedLR
It is established respectively with the multiple dimensioned model of High-resolution Panchromatic Images PAN in mean value image z1LRWith High-resolution Panchromatic Images PAN
On, mean value image z1LR is the mean value image synthesized above, in corresponding MSMZ1HR is the fused image of ARSIS, with
Other component zsGS inverse transformation is carried out, fused high-resolution multi-spectral image MS is obtainedHR。
4. multispectral image and panchromatic image fusion method according to claim 3 based on multilayer interband structural model,
It is characterized in that: using stationary wavelet transform SWT to the 2nd layer of approximate low resolution multispectral image described in step 3Carry out 2 layers
Wavelet transformation obtains: an approximate image after carrying out 4 layers of wavelet transformation to High-resolution Panchromatic Images PANOr
And the detail pictures D=D respectively on horizontal, vertical, diagonal three directionsH, DV, DD。
5. according to claim 1,2 or 4 multispectral image and full-colour image fusion side based on multilayer interband structural model
Method, it is characterised in that: by each approximate image in multiple dimensioned model M SM to A described in step 4MSWith APANAnd detail pictures pair
DMSWith DPAN, the process for being all divided into the image block of 64 × 64 pixel sizes is,
Step 4 one indicates each coefficient layer in MSM using symbol:WithScale is i after indicating stationary wavelet transform
On horizontal, vertical and diagonal direction levels of detail；
Step 4 two, the first step of multilayer interband structural model MLIBSM are that each approximate image and detail pictures are all divided into 64
The image block of × 64 sizes calculates each layer coefficients in each image block respectively later.
6. multispectral image and panchromatic image fusion method according to claim 5 based on multilayer interband structural model,
It is characterized in that: the approximate image pair for being 2 for scale described in step 5: the 2nd layer of approximate low resolution multispectral imageWith
2 layers of low resolution full-colour imageGlobal structure similarity SSIM is calculated, the partial structurtes similarity of each image block is calculated
LSSIM, and finally acquire the Lower-level details coefficient image of each image blockProcess be,
Step 5 one, the approximate image pair for being 2 for scale: the 2nd layer of approximate low resolution multispectral imageIt is low with the 2nd layer
Resolution panchromatic imageCalculate whole graph structure similarity:
In formula, μ and σ are the respective mean value of image and standard deviation respectively；Cov isWithCovariance；c1With c2 be in order to
Guarantee that denominator is not the constant for being much smaller than 1 of 0 and setting；
Step 5 two, the partial structurtes similarity LSSIM for calculating each image block；
Step 5 three calculates each image block
If
ThenOtherwise
In formula, the value of scale i is 2；α, β indicate regulation coefficient, for by the Lower-level details coefficient of full-colour imageIt is adjusted to
The Lower-level details coefficient of multispectral image
The relationship of step 5 four, the definition according to ARSIS, high-rise detail coefficients will be protected with the relationship of low-level details coefficient as far as possible
It holds unanimously, thenIt is equally applicable to scale i=3,4 ...；Due to
Then α and β are respectively indicated are as follows:
It is defined as follows objective function, the α found out and β is made preferably to be fitted the detail coefficients of different scale:
In formula, N is Decomposition order；
It is calculate by the following formula partial derivative of the f (α) about α:
Then formula (7) indicates are as follows:
To which the optimal solution of α in formula (5) must be found out are as follows:
It is calculate by the following formula the partial derivative of g (β):
The optimal solution of β in formula (6) must be found out are as follows:
The optimal solution of the α acquired and β is substituted into formula (2), each image block is obtainedAt this point, scale i=2, j=3,
The high level of 4 ..., N, j expression MSM.
7. according to claim 1,2,4 or 6 multispectral image and the full-colour image fusion based on multilayer interband structural model
Method, it is characterised in that: the 1st layer of low resolution multispectral image for being 1 for scale described in step 7With the 1st layer low point
Resolution full-colour imageThe process for repeating step 4 to step 6 obtains the mean value image z of high-resolution multi-spectral image1HR
Process be,
Step 7 one, the model for indicating formula (2) are applied in the plane that scale is 1, and find out α using formula (9) and formula (11)
With the optimal solution of β, obtainWherein, i=1, j=2,3,4 ..., N；
Step 7 two, progress wavelet inverse transformation obtain the mean value image z of high-resolution multispectral image1HRImage.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201610179673.7A CN105869114B (en) | 2016-03-25 | 2016-03-25 | Multispectral image and panchromatic image fusion method based on multilayer interband structural model |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201610179673.7A CN105869114B (en) | 2016-03-25 | 2016-03-25 | Multispectral image and panchromatic image fusion method based on multilayer interband structural model |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN105869114A CN105869114A (en) | 2016-08-17 |
| CN105869114B true CN105869114B (en) | 2018-12-11 |
Family
ID=56626018
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201610179673.7A Active CN105869114B (en) | 2016-03-25 | 2016-03-25 | Multispectral image and panchromatic image fusion method based on multilayer interband structural model |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN105869114B (en) |
Families Citing this family (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN106875370B (en) * | 2017-01-24 | 2020-11-06 | 中国科学院空间应用工程与技术中心 | Fusion method and device for full-color image and multispectral image |
| CN107622479B (en) * | 2017-09-04 | 2020-11-27 | 南京理工大学 | Contour wave sub-band self-adaptive detail injection method for multi-spectral-band image panchromatic sharpening |
| WO2020051897A1 (en) * | 2018-09-14 | 2020-03-19 | 浙江宇视科技有限公司 | Image fusion method and system, electronic device, and computer readable storage medium |
| CN109859143B (en) * | 2019-02-22 | 2020-09-29 | 中煤航测遥感集团有限公司 | Hyperspectral image panchromatic sharpening method and device and electronic equipment |
| CN111681194B (en) * | 2020-06-04 | 2023-03-14 | 西安电子科技大学 | Multi-source image fusion method based on wavelet transformation and Schmidt spectrum sharpening |
Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101216557A (en) * | 2007-12-27 | 2008-07-09 | 复旦大学 | Residual hypercomplex number dual decomposition multi-light spectrum and full-color image fusion method |
| CN101930604A (en) * | 2010-09-08 | 2010-12-29 | 中国科学院自动化研究所 | Infusion method of full-color image and multi-spectral image based on low-frequency correlation analysis |
| CN102446351A (en) * | 2010-10-15 | 2012-05-09 | 江南大学 | Multispectral and high-resolution full-color image fusion method study |
-
2016
- 2016-03-25 CN CN201610179673.7A patent/CN105869114B/en active Active
Patent Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101216557A (en) * | 2007-12-27 | 2008-07-09 | 复旦大学 | Residual hypercomplex number dual decomposition multi-light spectrum and full-color image fusion method |
| CN101930604A (en) * | 2010-09-08 | 2010-12-29 | 中国科学院自动化研究所 | Infusion method of full-color image and multi-spectral image based on low-frequency correlation analysis |
| CN102446351A (en) * | 2010-10-15 | 2012-05-09 | 江南大学 | Multispectral and high-resolution full-color image fusion method study |
Non-Patent Citations (2)
| Title |
|---|
| PANCHROMATIC AND MULTISPECTRAL IMAGES FUSION BASED ON MODIFIED GS-SWT;Xiaochen Lu等;《Geoscience and Remote Sensing Symposium（IGARSS），2014 IEEE International》;20141231;第2530-2533页 * |
| Synergetic Classification of Long-Wave Infrared Hyperspectral and Visible Images;Xiaochen Lu等;《IEEE JOURNAL OF SELECTED TOPICS IN APPLIED EARTH OBSERVATIONS AND REMOTE SENSING》;20150731;第8卷(第7期);第3546-3557页 * |
Also Published As
| Publication number | Publication date |
|---|---|
| CN105869114A (en) | 2016-08-17 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN105869114B (en) | Multispectral image and panchromatic image fusion method based on multilayer interband structural model | |
| Zhou et al. | Pyramid fully convolutional network for hyperspectral and multispectral image fusion | |
| Liebel et al. | Single-image super resolution for multispectral remote sensing data using convolutional neural networks | |
| Loncan et al. | Hyperspectral pansharpening: A review | |
| El-Mezouar et al. | A pan-sharpening based on the non-subsampled contourlet transform: Application to worldview-2 imagery | |
| Li et al. | Hyperspectral pansharpening via improved PCA approach and optimal weighted fusion strategy | |
| CN110544212B (en) | Convolutional neural network hyperspectral image sharpening method based on hierarchical feature fusion | |
| Dian et al. | Hyperspectral image super-resolution via local low-rank and sparse representations | |
| CN109993717A (en) | A kind of remote sensing image fusion method of combination guiding filtering and IHS transformation | |
| CN107220957B (en) | It is a kind of to utilize the remote sensing image fusion method for rolling Steerable filter | |
| CN108765359A (en) | A kind of fusion method of target in hyperspectral remotely sensed image and full-colour image based on JSKF models and NSCT technologies | |
| JP7152131B2 (en) | Image processing device, image processing method and program | |
| Paris et al. | A novel sharpening approach for superresolving multiresolution optical images | |
| Kotwal et al. | An optimization-based approach to fusion of hyperspectral images | |
| CN107169946A (en) | Image interfusion method based on non-negative sparse matrix Yu hypersphere color transformation | |
| CN110940638B (en) | Hyperspectral image sub-pixel level water body boundary detection method and detection system | |
| CN109859153B (en) | Multispectral image fusion method based on adaptive spectrum-spatial gradient sparse regularization | |
| Fryskowska et al. | Some aspects of satellite imagery integration from Eros B and Landsat 8 | |
| CN102156872B (en) | Multispectral-data-based object identification method and device | |
| CN110163830B (en) | Image fusion method based on Riesz-Lap transformation and PCNN | |
| Ma et al. | Hyperspectral image denoising based on low-rank representation and superpixel segmentation | |
| CN111383203A (en) | Panchromatic and multispectral remote sensing image fusion method based on regional fitting | |
| CN109544495B (en) | SoC chip image processing fusion method based on Gaussian filtering and ratio transformation | |
| Al-Mokhtar et al. | A review of digital image fusion and its application | |
| CN110097501B (en) | NDVI image fusion method based on non-local mean gradient sparse regularization |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| C06 | Publication | ||
| PB01 | Publication | ||
| C10 | Entry into substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant | ||
| TR01 | Transfer of patent right | ||
| TR01 | Transfer of patent right |
Effective date of registration: 20201012 Address after: 150001 No. 434, postal street, Nangang District, Heilongjiang, Harbin Patentee after: Harbin Institute of Technology National University Science Park Development Co.,Ltd. Address before: 150001 Harbin, Nangang, West District, large straight street, No. 92 Patentee before: HARBIN INSTITUTE OF TECHNOLOGY |
|
| TR01 | Transfer of patent right | ||
| TR01 | Transfer of patent right |
Effective date of registration: 20201203 Address after: Room A101, building 1, Yinxing Zhijie phase II, No. 1301-76, sightseeing Road, Xinlan community, Guanlan street, Longhua District, Shenzhen City, Guangdong Province Patentee after: Shenzhen National Research Institute of high performance Medical Devices Co.,Ltd. Address before: 150001 No. 434, postal street, Nangang District, Heilongjiang, Harbin Patentee before: Harbin Institute of Technology National University Science Park Development Co.,Ltd. |