The content of the invention
The technical problems to be solved by the invention are, a kind of method for registering images based on improved structural similarity is provided, by being improved to existing structural similarity function defined formula, and the function after improvement is used for image registration, no matter single mode or multi-modal, sub-pixel registration is attained by.
The technical scheme that present invention solution above-mentioned technical problem is used is as follows:
A kind of method for registering images based on improved structural similarity, it is characterised in that follow the steps below image registration:
1), input two images subject to registration, reference picture and floating image are designated as respectively;
2), using principal axes and centroid based method rough registration:Image centroid is tried to achieve by the first moment of image, then the angle of main shaft and coordinate system is tried to achieve by second-order moment around mean, four parameter values of rough registration are obtained with this:X translations, Y translations, anglec of rotation R, zoom factor S;
3), carry out space geometry conversion to floating image with four parameter values of rough registration, obtained image and merges with reference picture display as the floating image of smart registration;
4), using Powell optimized algorithms essence registration:With reference picture and the 3rd)Obtained floating image is walked as two image subject to registration, measure function is used as using improved SSIM, it is [X translation Y translation anglec of rotation R zoom factors S]=[0 00 1] to optimize starting point, using bicubic interpolation method, wherein one-dimensional optimized algorithm uses Brunt method, the step-size in search of four parameters is corresponded to [1 11 0.05], dynamic range len=20 of search, iteration precision
;
Wherein improved SSIM measure functions are obtained by following formula:
Wherein, X, Y represent reference image block and floating image block respectively,
,
,
X and Y brightness correlation function, contrast correlation function and structure correlation function is represented respectively,
、
、
For adjusting brightness, contrast and the weight of structural information,
>0、
>0、
>0,
、
、
、
、
X, Y local luminance average, standard deviation and covariance are represented respectively;
、
、
,
、
<<1, L is the dynamic range of pixel;If taken
, then it is reduced to:
。
K1、K1Preferred value be:K1=0.01, K2=0.03。
Research discovery, C1、C2Value is relevant with the dynamic range and minimal gray of gradation of image,,, C1、C2Image minimal gray and variance can not be noticeably greater than, the registering curve peak shape acutancees of SSIM otherwise can be influenceed, be easily caused registration and be absorbed in local extremum.Assuming that image is 8 gray level images, its dynamic range 0~255, as L=255 that Zhou Wang et al. are proposed, then K1=0.01, K2=0.03;If image binaryzation, the dynamic range 0~1 of gradation of image, L is 1, and minimal gray is 0, then requires K1≤ 0.000001, K2≤0.000003。
5), with smart registration parameter to the 3rd)Walk obtained floating image and carry out space geometry conversion, obtained image merges with reference picture shown as final registration result again.
The positive effect of the present invention is:
Firstth, the present invention is improved existing structural similarity function defined formula, and the function after improvement is used for into image registration first, and there is provided more common, accurate, robustness a algorithm.
Especially, in being defined to structural similarity
When taking absolute value, structural similarity has good Convex Functions feature, no matter single mode or multi-modality image registration, sub-pixel registration can be reached using structural similarity as measure function, it is experimentally confirmed that the registration accuracy and robustness of the present invention are better than the normalized mutual information image registration algorithm of classics.
Secondth, K1≤ 0.000001, K2When≤0.000003, the pixel level registration of binary image can be realized using the SSIM functions of the present invention.
Brief description of the drawings
Fig. 1 is the SSIM drawn and x directions translation curve using one-parameter space geometry changing image as floating image.
Fig. 2 is the SSIM drawn and y directions translation curve using one-parameter space geometry changing image as floating image.
Fig. 3 is the SSIM drawn and anglec of rotation curve using one-parameter space geometry changing image as floating image.
Fig. 4 is the SSIM drawn and consistent zoom factor curve using one-parameter space geometry changing image as floating image.
Fig. 5~Fig. 8 is not plus noise, SSIM and the graph of a relation of two-parameter change:Fig. 5 X translations, Y translation curves, Fig. 6 Y translations, rotating curve, Fig. 7 rotations, scaling curve, Fig. 8 X translations, scaling curve.
Fig. 9 left figures are original image, and middle figure is that with the addition of white Gaussian noise(Gaussian white noise,
,
), right figure is that with the addition of salt-pepper noise(" salt and pepper " noise, noise density D=0.05).
Figure 10 images containing white Gaussian noise match somebody with somebody directrix curve as floating image:1st behavior SSIM's matches somebody with somebody directrix curve, respectively~x ,~y translation curves ,~r rotating curves ,~s scaling curves;2nd behavior NMI match somebody with somebody directrix curve, it is identical successively.
Figure 11 matches somebody with somebody directrix curve containing the images with salt and pepper noise as floating image:1st behavior SSIM's matches somebody with somebody directrix curve, respectively~x ,~y translation curves ,~r rotating curves ,~s scaling curves;2nd behavior NMI match somebody with somebody directrix curve, it is identical successively.
Figure 12 left figures are original image, and right figure is the floating image for having 60*50 size masked areas.
Figure 13 has matches somebody with somebody directrix curve during covering:1st behavior SSIM matches somebody with somebody directrix curve, and the 2nd behavior NMI matches somebody with somebody directrix curve.
Figure 14 is binary image:Left figure is reference picture, and right figure is floating image.
Figure 15 be image binaryzation after match somebody with somebody directrix curve:1st behavior SSIM matches somebody with somebody directrix curve, and the 2nd behavior NMI matches somebody with somebody directrix curve.
Figure 16 is that the SSIM for reducing binary image after C1, C2 matches somebody with somebody directrix curve.
Figure 17, Figure 18 are that single mode image is based on improved SSIM registration results figure:(a) reference picture, (b) floating image, (c) rough registration image, the registering image of (d) essence.
Figure 19 is multi-modality images.
Figure 20 is multi-modality image registration curve:1st behavior SSIM curves, the 2nd behavior NMI curves.
Multi-modality images SSIM matches somebody with somebody directrix curve when Figure 21 SSIM are not improved.
Figure 22 is multi-modality image registration result:(a) reference picture, (b) floating image, (c) rough registration, (d) SSIM essence registrations, (e) NMI essence registrations.
Figure 23 is the registration result of multi-mode image binaryzation pretreatment:(a) the registering image of reference picture (b) floating image (c) binaryzation reference picture (d) binaryzation floating image (e) SSIM rough registration image (f) SSIM essences registration image (g) NMI rough registration image (h) NMI essences.
Figure 24 bianry image registration results.
Embodiment
The present invention is further illustrated with reference to the accompanying drawings and examples(Following structural similarity SSIM refers both to improved structural similarity).
The method for registering images step of the present invention is as follows:
1. input two images subject to registration, are designated as reference picture and floating image respectively;
2. using principal axes and centroid based method rough registration (Coarse registration):Image centroid is tried to achieve by the first moment of image, then the angle of main shaft and coordinate system is tried to achieve by second-order moment around mean, 4 parameter values [x y r 1] of rough registration are obtained with this;
3. carry out space geometry conversion to floating image with 4 parameter values of rough registration, obtained image and merges with reference picture display as the floating image of smart registration;
4. it is registering (fine registration) using Powell optimized algorithms essence:The new floating image obtained using reference picture and the 3rd step is as two image subject to registration, using SSIM as measure function, and optimization starting point is [X translation Y translation anglec of rotation R zoom factors S]=[0 00 1], using bicubic interpolation method(bicubic), wherein one-dimensional optimized algorithm uses Brunt(Brent)Method, the step-size in search of 4 parameters is corresponded to [1 11 0.05], dynamic range len=20 of search, and actual is X [- 20 20], Y [- 20 20], R [- 20 20], S [- 1 1], iteration precision
If, actually
;
5. carrying out space geometry conversion to new floating image with smart registration parameter, obtained image merges with reference picture shown as final registration result again.
The following is single mode image registration algorithm.
The Rigid Registration of this experimental study single mode 2-D gray image, space geometry conversion is from 4 parameters:X directions are translated(It is set to positive direction downwards), y directions translation(Positive direction is set to the right), around vertical axis anglec of rotation r(It is set to positive direction counterclockwise)And consistent scaling s, using bicubic interpolation(Bicubic Interpolation), structural similarity parameter takes L=255, K
1=0.01, K
2=0.03, then
,
, translation is with pixel (pixel) for unit, and the anglec of rotation is with " degree(degree)" it is unit.
Single mode image structure similarity and the relation of space geometry transformation parameter:
Using medical image BrainP.bmp as reference picture, floating image space geometry variable dynamic range:Translate X [- 30,30], Y [- 30,30](pixel)Rotate R [- 30,30] (degree), scale S [0.1,2], analysis is free of structural similarity and the variation relation of space geometry 4 parameters of conversion between noise situations hypograph, if being rendered as strict convex or concave function feature, and peak is preferable, then illustrate that structural similarity can be as registering similarity measure function.
(1)Not plus noise, the SSIM for drawing the change of one-parameter space geometry matches somebody with somebody directrix curve, as shown in Figure 1 to 4.
Translational movement, rotation amount are 0 and zoom factor is 1 to represent image alignment, preferable peak value SSIM=1 now occur, and smooth near peak value, are occurred without obvious local extremum.
(2)With x, y double flat move, translation put english turn, rotation plus scaling, translation plus scaling carry out two-parameter space geometry conversion respectively, registering surface chart is drawn, as shown in Fig. 5~Fig. 8.
No matter one-parameter changes or two-parameter change simultaneously, the sharp Convex Functions feature of smoother is rendered as in wider excursion, if this explanation is appropriate to choose registering initial point and step-size in search, local extremum will not be absorbed in, and extreme point is in ideal position, meet and match somebody with somebody alignment request, so structural similarity can be estimated as image registration.
Robust analysis:
Respectively from noise immunity, covering and binary conversion treatment tripartite's surface analysis, while being compared with based on normalized mutual information is registering.Classical normalized mutual information is defined as similarity measure:
Wherein
With
For reference picture and the edge entropy of floating image,
For the combination entropy of the two, it is respectively defined as:
、
The respectively probability density function of reference picture and floating image,
For the joint probability density function between two images.
For normalized mutual information,
, ideal value is 2.
(1)Respectively white Gaussian noise is added to original image(Gaussianwhite noise of mean
andvariance
to the image), salt-pepper noise("salt and pepper" noise to the image , where D is the noise density D=0.05), using noisy image as floating image, as shown in Fig. 9, Figure 10, Figure 11.
SSIM peak value relatively sharp but both sides are upwarped slightly, and peak-fall is than more serious(
), rotating curve peak value both sides are convex, and scaling curve deformation is serious;NMI changes are shallower, but rotating curve peak value both sides are also convex, there is local extremum appearance near scaling curve peak value.
Salt-pepper noise curve characteristic is similar to white noise, and only because noise pollution is lighter, registering curve deformation is not obvious.Experiment also found, when noise pollution is lighter, such as white Gaussian noise variance is smaller(
), preferably, as variance increases, peak value is begun to decline, and local extremum occurs in scaling curve first for curve characteristic holding, rotates with curve and local extremum occurs, last translation curve starts to be deteriorated;When variance is sufficiently large, so that noise has flooded picture material, curve deformation is serious, and registering condition is unsatisfactory for.So SSIM and NMI is insensitive to noise, but when noise is larger, needed before registration to image denoising.
(2)Give reference picture to cover 60*50 size areas, cause missing, using having missing image as floating image, as shown in Figure 12 and Figure 13.
From Figure 12, Figure 13, SSIM is that peak value slightly declines with directrix curve(
), and the decline of NMI peak of curves is more apparent(
), and there is local extremum near scaling curve peak value." covering " is that the SSIM for making covered parts of images block diminishes, will not be to the SSIM after ensemble average if masked area is not very big(That is MSSIM)There is big influence, so the anti-missing performance of the anti-coverings of SSIM is stronger.
(3)Binary image can simplify data, improve calculating speed.Tonal range [171,255] is determined by original image histogram, image is split, obtained binary image splits obtained binary image as floating image, using arest neighbors interpolation method as reference picture with [181,255](nearest), such as Figure 14, it is as shown in figure 15 with directrix curve.
SSIM numerical value is between 0.95~0.99, and still in smooth Convex Functions feature, but peak is not sharp;NIM numerical value is between 1~1.35, and peak-fall is larger, but peak shape is preferably, and the local extremum of scaling curve disappears.Binary image pixel grey scale is that covariance is respectively less than 1 between 0 or 1, therefore average, variance and image, if the C in SSIM1、C2Value reduces, and it is much smaller than 1, we make K1=0.000001, K2=0.000003, then it is as shown in figure 16 with directrix curve.
Now SSIM is between 0.35~0.65, and peak shape is sharpened, experiment it has also been found that, when two binary images are identical, registering peak of curve be 1, therefore image binaryzation after still can using structural similarity registration;Similarly, NMI peak-falls between two binary images also in that had differences, and experiment is found, when two images are identical, is 2 with quasi-peak value, because binaryzation makes image data reduction, and it is more sharp that NMI curves become more smooth, peak value.
Single mode image registration is tested:
Reference picture is used as using the medical image BrainP.bmp of symmetrical structure, image size is 221*257, by its X-direction downwards translation 30 pixels, Y-direction to the pixel of right translation 20, turn clockwise 10 degree, unanimously scale 0.5, obtained image is as floating image, and spatial alternation function uses bilinear interpolation(bilinear).Specific steps and result are as follows:
Step 1 reads in image as reference picture, and it then is done into space geometry conversion with [30 20-10 0.5] and obtains floating image;
Step 2 uses principal axes and centroid based method rough registration:Image centroid is tried to achieve by the first moment of image, then the angle of main shaft and coordinate system is tried to achieve by second-order moment around mean, 4 parameter values of rough registration are obtained with this;
Step 3 carries out geometric transformation with rough registration parameter to floating image, and gained image is used as new floating image;
Step 4 is using Powell optimized algorithms essence registration:Using
step 3 gained image as new floating image, using improved structural similarity as water rogulator function, optimization starting point is [X translation Y translation anglec of rotation R zoom factors S]=[0 00 1], wherein one-dimensional optimized algorithm uses Brunt(Brent)Method, the step-size in search of 4 parameters is corresponded to [1 11 0.05], dynamic range len=20 of search, and actual is X [- 20 20], Y [- 20 20], R [- 20 20], S [- 1 1], iteration precision
If, actually
.
The new floating image that step 5 is obtained with 4 parameters of smart registration gained to step 3 carries out space geometry conversion, and display registration result such as Figure 17 is then merged with reference picture(d).
Iteration 3 times terminates, and takes 152 seconds, is as a result mainly for the less reasons of SSIM=0.6666, SSIM after floating image amplification, because interpolation error cause it is image blurring caused by;Registration accuracy reaches 2~3 pixels.Registration result is as shown in figure 17.
Rough registration and the registering image of essence are substantially the fused images of floating image and reference picture after registration.
Precision analysis:
Water rogulator is used as using structural similarity and normalized mutual information respectively, reference picture is used as using the Lena.bmp of asymmetric texture-rich, image size is 256*256, by its X-direction downwards translation 10 pixels, Y-direction to the pixel of right translation 8, turn clockwise 5 degree, unanimously scale 0.8, obtained image is as floating image, the interpolation influence registration accuracy of spatial alternation, we use bicubic interpolation method(bicubic).The registration result of two measures is approached, and Figure 18 shows the registering images of SSIM.
Precision analytical method:Appoint on a reference and take a bit, for example point P0(80,50), respective coordinates vector [80 50 1], with the transformation matrix of 4 parameters [10 × 0.8 8 × 0.8-5 0.8] by this point be mapped to Q0, then with Q0For starting point, to calculate obtained rough registration parameter transformation matrix by Q0It is mapped to Q1, finally with the transformation matrix of smart registering parameters obtained again by Q1It is mapped to Q2, compare Q2And P0.Concrete outcome is as follows:
SSIM:5 time-consuming 351 seconds, SSIM=0.8001 of iteration, rough registration parameter is [- 5.0000 0 0.3950 1.000], and final [80 50 1] are mapped to [80.4346 50.1932 1.0000], it is seen that reach sub-pixel registration;
NMI:5 time-consuming 336 seconds, NMI=1.4100 of iteration, rough registration parameter is [- 5.0000 0 0.3950 1.000], and final [80 50 1] are mapped to [80.4346 50.1932 1.0000], it is seen that also reach sub-pixel registration.
Because rough registration parameter is inaccurate, cause the algorithmic statement time longer, but SSIM and NMI are attained by sub-pixel registration.
Many experiments are proved, even if mismatch is heavier, are attained by pixel level registration, but NMI algorithms are easier to be absorbed in local extremum;If registering again after reference picture and the amplification of floating image equal proportion, registration accuracy can be improved, so that reaching sub-pixel registration.
The following is multi-modality image registration algorithm
Using multi-modality medical image MR-T1.jpg and MR-T2.jpg as reference picture and floating image, as shown in figure 19, image size is 256*256, and the parameter of selection is identical with single mode registration, the registering performance under com-parison and analysis structural similarity and normalized mutual information two measures.
With the relation curve of space geometry transformation parameter:
Dynamic range:Translate X [- 50,50], Y [- 50,50](pixel), rotation R [- 50,50] (degree), scaling S [0.1,3] draw the relation curve of structural similarity and normalized mutual information and 4 parameters, as shown in figure 20 respectively
Good Convex Functions feature is presented in SSIM curves, and SSIM numerical value is between 0.1~0.8, and peak shape is sharp, smooth, and peak point is located at ideal position, meets and matches somebody with somebody alignment request;NMI numerical value is between 1~1.03, and peak-fall is serious, and there is stronger local extremum, is unsatisfactory for registering condition.SSIM is to be based on human visual system's feature, and from the brightness between the angle changing rate image of statistics, contrast and structural information correlation, the influence of content deltas is weaker.
If not in SSIM defined formulas
Take absolute value, then SSIM with directrix curve as shown in figure 21, SSIM coordinates take [0,0.6], and translation and rotating curve sink between 0.35~0.55 and at peak value, and scaling curve is between 0~0.6, and now SSIM is unsatisfactory for matching somebody with somebody alignment request.
Robust analysis:
Anti-interference is uncorrelated to image own content, therefore the robustness of multimode image registering is consistent with single mode graphical analysis.
Multi-modality image registration is tested:
Reference picture is used as using medical image MR-T1.jpg, by MR-T2.jpg images X-direction downwards translation 10 pixels, Y-direction to the pixel of right translation 8, turn clockwise 5 degree, unanimously scale 0.8, obtained image is as floating image, and spatial alternation function uses bicubic interpolation method(bicubic).Method for registering and Optimal Parameters and single mode image BrainP.bmp are registering completely the same, compare SSIM and NMI as the registering performance of measure function.Registration result is as shown in figure 22.
SSIM=0.7649,3 time-consuming 201 seconds, Figure 22 of iteration(d)Show that degree of registration is good;NMI=1.0507, iteration 5 times terminates, and takes 208 seconds, Figure 22(e)Display registration does not succeed, it is clear that be absorbed in local extremum.
Precision analysis:
Analysis method is consistent with single mode, as a result for:
SSIM:[80 50 1] are mapped to [79.3739 50.8609 1.0000], reach sub-pixel registration;
NMI:[80 50 1] are mapped to [101.8300 24.2465 1.0000], and error is larger, registration failure.
The multi-modality image registration as rough registration is directly registrable after image binaryzation:
Image binaryzation can simplify data, calculating speed is improved, the characteristic curve for knowing binary image by analysis is still met with alignment request, we inquire into directly carries out registration using binary image, as rough registration, essence registration is then carried out to original image using rough registration parameter.The SSIM parameters K of binary image rough registration1=0.000001, K2=0.000003, spatial alternation uses arest neighbors interpolation(nearest), the SSIM parameters K of original image essence registration1=0.01, K2=0.03, using bicubic interpolation method(bicubic).Measure function is used as using SSIM and NMI respectively, still reference picture is used as using medical image MR-T1.jpg, by MR-T2.jpg images X-direction downwards translation 16 pixels, Y-direction to the pixel of right translation 12, turn clockwise 10 degree, unanimously scale 0.5, obtained image is as floating image, the Optimal Parameters and single mode of rough registration and essence registration are completely the same, registration Algorithm:
Step 1 image binaryzation:Using gray level image threshold function table threshold value, then corrected, reference picture and floating image are converted into bianry image respectively(Figure 23 (b) (c));
Step 2 rough registration:Structural similarity and normalized mutual information is respectively adopted as measure function, using Powell optimized algorithms directly to bianry image registration;
Step 3 spatial alternation:Space geometry conversion is carried out to former floating image using rough registration parameter, the image after conversion is used as smart registering floating image;
Step 4 essence registration:Structural similarity and normalized mutual information is respectively adopted as measure function, essence registration is carried out to the former floating image after former reference picture and spatial alternation using Powell optimized algorithms.As a result it is as follows:
SSIM=0.7650, total time-consuming 392 seconds.The rough registration result of bianry image is close, but undesirable, and reason is that registration is directly optimized, and rough registration starting point mismatch is larger, may be absorbed in local extremum;The smart registration result difference of two measures function is larger, is consistent with analysis, even if rough registration is undesirable, the smart registration result that SSIM estimates is still preferable, Figure 23(f)It is shown.
SSIM precision analysis:[80 50 1] are mapped to [71.9242 55.1510 1.0000], because rough registration error is larger, precise decreasing, but still reach pixel level registration.
Bianry image matches somebody with somebody quasi-experiment
Using medical image MR-T1.jpg as reference picture, by MR-T2.jpg images X-direction downwards translation 16 pixels, Y-direction to the pixel of right translation 12, turn clockwise 10 degree, unanimously scale 0.8, obtained image takes K as floating image1=0.000001, K2=0.000003, first by image binaryzation, other method for registering and Optimal Parameters are identical with single mode, as a result as shown in figure 24.
Registration result:SSIM=0.5807, rough registration parameter is [- 10.0000-10.0000 8.9429 1.0000], smart registration parameter is [- 6.0000 5.0000 1.0000 1.1500], [80 50 1] are mapped to [69.7807 51.7752 1.0000], it is seen that reach pixel level registration.Bianry image gray scale only has 0 and 1, therefore registration error is bigger than ordinary gamma image, if consistent scaling takes 0.5, difficult registration is easily absorbed in local extremum.Experiment is it has also been found that K1>0.000001, K2>Local extremum easily is absorbed in when 0.000003, makes registration failure;K1<0.000001, K2<Registration result and K after 0.0000031=0.000001, K2=0.000003 is identical, and simply SSIM values have minor variations, and this is related to SSIM defined formulas.
Above comparative analysis shows, for single mode image registration, if rough registration algorithm picks are appropriate, then two measures function can reach preferable sub-pixel registration, and speed, and robustness is preferable;For multi-modality image registration, SSIM measure functions registration performance is significantly better than NMI measure functions, sub-pixel registration can be reached, in the case of optimization starting point mismatch is larger, still pixel level registration can be reached, and NMI measure functions are unsatisfactory for registering condition in itself, it is impossible to be directly used in multi-modality image registration;SSIM and NMI before registration, are required to denoising to insensitive for noise, but when noise is larger;SSIM anti-covering performance is significantly better than NMI, and insensitive to image modalities;If K1≤0.000001, K2≤0.000003, bianry image can reach pixel level registration.