CN103985133A - Search method and system for optimal splicing lines among images based on graph-cut energy optimization - Google Patents

Abstract

The invention discloses a search method and system for optimal splicing lines among images based on graph-cut energy optimization. According to the method, data preparation of the images to be spliced is performed, and all overlap regions and corresponding overlap degrees among all the images to be spliced are obtained; the images to be spliced are preprocessed, color, gradient and texture information is comprehensively considered to determine a graph-cut global energy function which includes a data item and a smooth item, weighting setting is performed on energy of the smooth item, the graph-cut energy optimization method is adopted for optimizing total energy, and therefore the optimal splicing lines are obtained. The obtained splicing lines are prevented from passing through regions which are overlarge in color difference and complex in texture and passing through edges obvious in ground feature as much as possible, so that optimization of the splicing lines is ensured to the maximum degree, the problem of optimizing multi-image multi-degree overlap region splicing lines in a combined mode is solved, and the application range is wide.

Description

Cut optimum splicing line finding method and system between energy-optimised image based on figure

Technical field

The invention belongs to digitized video processing technology field, particularly relate to one and cut optimum splicing line searching technical scheme and system between (Graph-Cuts) energy-optimised image based on figure.

Background technology

In digitized video splicing, owing to being subject to the impact of the factor such as Image registration geometric error and image aberration, causing the region (as the obviously atural object such as buildings, vehicle and pedestrian) differing greatly between image in shooting visual angle or target travel etc., often there is comparatively significantly inconsistent phenomenon in spliced image.Therefore, how to find quickly and efficiently an optimum splicing line (avoiding through the larger region of image difference), it is technology the most key in image joint, also be one of hot subject of computer vision and photogrammetric area research, in full-view image splicing and aviation image splicing (especially low latitude inclination image joint) etc., be most widely used.

The method of splicing line Automatic-searching mainly contains three kinds of technology paths at present: the one, and based on the method for auxiliary data, the 2nd, object-based splicing line finding method, the 3rd, based on the method for overlapping region image difference.

Splicing line finding method based on auxiliary data mainly utilizes auxiliary data (road vectors data, LiDAR cloud data etc.) to retrain the trend of splicing line, thereby obtains good splicing result.The major defect of the method is the dependence to auxiliary data, and the quality of splicing result depends on the precision of auxiliary data to a great extent.

Object-based splicing line finding method, in conjunction with Image Segmentation algorithm, carries out Image Segmentation to overlapping region, obtains the distributed areas of obvious atural object, thereby for instructing the searching of optimum splicing line.The limitation of the method is mainly that the accuracy of Target Segmentation can not be guaranteed, the effect of impact splicing.

Splicing line finding method based on overlapping region image difference utilizes the difference of overlapping region respective pixel, builds rational energy function, and adopts suitable energy optimizing method to be optimized.The method can obtain good splicing effect, but consuming time longer, computation complexity is higher, and efficiency is lower.

Summary of the invention

In order to solve the problems of the technologies described above, can be fast, stable, between image, overlapping region, search out optimum splicing line exactly, the invention provides and a kind ofly cut optimum splicing line searching technology between (Graph-Cuts) energy-optimised image based on figure.

The technical solution adopted in the present invention provides a kind of and has cut optimum splicing line finding method between energy-optimised image based on figure, comprises the following steps:

Step 1, carries out image data preparation to be spliced, obtains all overlapping regions and corresponding degree of overlapping between each image to be spliced;

Step 2, carries out pre-service to image to be spliced;

Step 3, determines that figure cuts global energy function as follows,

Open if current image to be spliced is N, be respectively image I ₁, I ₂, I ₃..., I _n, the image set of formation is figure cuts global energy function and comprises E _dataand E _smoothtwo,

Wherein, E _dataand E _smoothrepresentative data energy term and level and smooth energy term respectively, it is specifically calculated as follows,

Described data energy term be calculated as follows,

Wherein, E _data(I _k) represent that each pixel comes from image I in the spliced image J of optimum splicing line _kenergy punishment, be defined as follows,

E _data＝(I _k)Σ _p∈JR _p(Ik)

Wherein, p is certain pixel on image J after splicing, R _p(I _k) represent pixel p comes from image I _kenergy punishment, be defined as follows,

$R_p\left(I_k\right)=\left\{\begin{array}{cc}0& p\∈I_k\\ \∞& p\∉I_k\end{array}\right.$

Described level and smooth energy term be defined as follows,

Wherein, color space energy term and gradient space energy term computing formula as follows,

Wherein, p and q are two adjacent pixels, for the neighbours territory of pixel p, with the numbering of pixel p and the affiliated image of q in the rear image J of representative splicing respectively, function be defined as follows,

with represent respectively the level and smooth energy of color space and the level and smooth energy of gradient space that neighbor p and q are corresponding in overlapping region, be defined as follows,

Wherein, I _aand I _bfor image set in any two images that have an overlapping region, establish image I _a, I _bform image pair to be spliced with be respectively neighbor p and q at I _aand I _bthe level and smooth energy of color space that overlapping region is corresponding and the level and smooth energy of gradient space;

Step 4, level and smooth energy weight setting, comprises the color space energy term of single pixel and gradient space energy term redefine as follows,

Wherein, w _d(p) be the weight of pixel p near edge, overlapping region, be defined as follows,

$w_d\left(p\right)=\left\{\begin{array}{cc}1& \mathrm{ifd}\left(p\right)>d_{\mathrm{th}}\\ \frac{d_{\mathrm{th}}}{d\left(p\right)}& \mathrm{otherwise}\end{array}\right.$

Wherein, d (p) represents the bee-line of pixel p to edge, overlapping region, d _thfor predetermined threshold value;

Step 5, employing figure cuts energy optimizing method, to redefine color space energy term through step 4 and gradient space energy term after gross energy be optimized, obtain optimum splicing line.

And, there is local optimum poor effect after being optimized according to the optimum splicing line of step 5 gained time, each overlapping region in local optimum poor effect region is added respectively to intervention, optimize splicing line, step is as follows,

Steps A 1 is chosen point set in overlapping region, and gives corresponding image ownership numbering, and the sets definition of establishing M the point of choosing is the corresponding image ownership number table of giving is shown

Steps A 2, the point set of choosing for steps A 1 corresponding level and smooth energy term is not changed, and corresponding data item is amended as follows,

Wherein, x _ia pixel of choosing for steps A 1, for steps A 1 is given to pixel x _iimage ownership numbering, after this amendment energy term, employing figure cuts energy optimizing method and obtains optimum splicing line again.

And described neighbor p and q are at I _aand I _bthe level and smooth energy of color space that overlapping region is corresponding and the level and smooth energy of gradient space with obtain manner as follows,

Carry out self imaging pair for each pixel p in image J after splicing texture space energy be defined as follows,

Wherein, Γ _a(p) and Γ b (p) respectively represent pixel p at image I _aand I _bon Texture complication;

Carry out self imaging pair for each pixel p in image J after splicing color space energy be defined as follows,

Wherein, H (p) and S (p) represent respectively the value of pixel p in image H passage and S passage, w _hand w _sbe respectively corresponding power;

Carry out self imaging pair for each pixel p in image J after splicing gradient space energy definition as follows,

Wherein, λ is balance parameters, with be respectively pixel p at image I _aon horizontal direction gradient magnitude, at image I _aon vertical gradient size, at image I _bon horizontal direction gradient magnitude and at image I _bon vertical gradient size;

The level and smooth energy of color space and the level and smooth energy of gradient space with be defined as follows,

And, image to be spliced is carried out to pre-service described in step 2, comprise that image is down-sampled, the conversion of image color space and gray processing, image gradient calculation and Texture complication calculate.

And, in the time that image to be spliced is panoramic picture, image to be spliced is carried out to pre-service described in step 2 and comprise and carry out image expansion.

The present invention is also corresponding to be provided a kind of and has cut optimum splicing line searching system between energy-optimised image based on figure, comprises with lower module:

Data preparation module, for carrying out image data preparation to be spliced, obtains all overlapping regions and corresponding degree of overlapping between each image to be spliced;

Pretreatment module, for carrying out pre-service to image to be spliced;

Energy function determination module, as follows for determining that figure cuts global energy function,

Open if current image to be spliced is N, be respectively image I ₁, I ₂, I ₃..., I _n, the image set of formation is figure cuts global energy function and comprises E _dataand E _smoothtwo,

Wherein, E _dataand E _smoothrepresentative data energy term and level and smooth energy term respectively, embodiment specifically comprises following sub-step:

Described data energy term be calculated as follows,

Wherein, E _data(Ik) representative each pixel in the spliced image J of optimum splicing line comes from image I _kenergy punishment, be defined as follows,

E _data(I _k)＝Σ _p∈JR _p(I _k)

Wherein, p is certain pixel on image J after splicing, R _p(I _k) represent pixel p comes from image I _kenergy punishment, be defined as follows,

$R_p\left(I_k\right)=\left\{\begin{array}{cc}0& p\∈I_k\\ \∞& p\∉I_k\end{array}\right.$

Described level and smooth energy term be defined as follows:

Wherein, color space energy term and gradient space energy term computing formula as follows,

Wherein, p and q are two adjacent pixels, for the neighbours territory of pixel p, with the numbering of pixel p and the affiliated image of q in the rear image J of representative splicing respectively, function be defined as follows,

with represent respectively the level and smooth energy of color space and the level and smooth energy of gradient space that neighbor p and q are corresponding in overlapping region, be defined as follows,

Wherein, I _aand I _bfor image set in any two images that have an overlapping region, establish image I _a, I _bform image pair to be spliced with be respectively neighbor p and q at I _aand I _bthe level and smooth energy of color space that overlapping region is corresponding and the level and smooth energy of gradient space;

Weight setting module, for level and smooth energy weight setting, comprises the color space energy term of single pixel and gradient space energy term redefine as follows,

Wherein, w _d(p) be the weight of pixel p near edge, overlapping region, be defined as follows:,

$w_d\left(p\right)=\left\{\begin{array}{cc}1& \mathrm{ifd}\left(p\right)>d_{\mathrm{th}}\\ \frac{d_{\mathrm{th}}}{d\left(p\right)}& \mathrm{otherwise}\end{array}\right.$

Wherein, d (p) represents the bee-line of pixel p to edge, overlapping region, d _thfor predetermined threshold value;

Optimize module, for adopting figure to cut energy optimizing method, to redefine color space energy term through weight setting module and gradient space energy term after gross energy be optimized, obtain optimum splicing line.

And, intervention module is set, for when according to optimizing while there is local optimum poor effect after the optimum splicing line of module gained is optimized, each overlapping region in local optimum poor effect region is added respectively to intervention, optimize splicing line, step is as follows,

Steps A 1 is chosen point set in overlapping region, and gives corresponding image ownership numbering, and the sets definition of establishing M the point of choosing is the corresponding image ownership number table of giving is shown

Steps A 2, the point set of choosing for steps A 1 corresponding level and smooth energy term is not changed, and corresponding data item is amended as follows,

Wherein, x _ia pixel of choosing for steps A 1, for steps A 1 is given to pixel x _iimage ownership numbering, after this amendment energy term, employing figure cuts energy optimizing method and obtains optimum splicing line again.

And described neighbor p and q are at I _aand I _bthe level and smooth energy of color space that overlapping region is corresponding and the level and smooth energy of gradient space with obtain manner as follows,

Carry out self imaging pair for each pixel p in image J after splicing texture space energy be defined as follows,

Wherein, Γ _aand Γ (p) _b(p) difference represent pixel p is at image I _aand I _bon Texture complication;

Carry out self imaging pair for each pixel p in image J after splicing color space energy be defined as follows,

Wherein, H (p) and S (p) represent respectively the value of pixel p in image H passage and S passage, w _hand w _sbe respectively corresponding power;

Carry out self imaging pair for each pixel p in image J after splicing gradient space energy definition as follows,

Wherein, λ is balance parameters, with be respectively pixel p at image I _aon horizontal direction gradient magnitude, at image I _aon vertical gradient size, at image I _bon horizontal direction gradient magnitude and at image I _bon vertical gradient size;

The level and smooth energy of color space and the level and smooth energy of gradient space with be defined as follows,

And the pre-service that pretreatment module is carried out image to be spliced, comprises that image is down-sampled, the conversion of image color space and gray processing, image gradient calculation and Texture complication calculate.

And in the time that image to be spliced is panoramic picture, the pre-service that pretreatment module image to be spliced carries out comprises carries out image expansion.

With respect to prior art, the present invention can quick and precisely automatically find and obtain optimum splicing line between image.In splicing line Automatic-searching process, the present invention has considered color, gradient and texture information, make splicing line avoid, texture complex region excessive by aberration and the obvious edge of atural object as far as possible, thereby ensured to the full extent the optimization of splicing line.Meanwhile, the present invention has creatively used figure to cut energy optimizing method, is compared to the optimization methods such as traditional ant group, dynamic programming (DP, Dynamic Programming) and Dijkstra, and this optimization method effect of optimization is better, and efficiency is higher.Moreover, the present invention has also solved many images and has spent the problem of overlapping region splicing line combined optimization more, is compared to traditional method being optimized between image between two, and many image associations optimization efficiency is higher, and effect is also more reasonable.In addition, applicability of the present invention is wide, can solve preferably the Bonding Problem of the data such as full-view image, orthography and low latitude inclination image.

Brief description of the drawings

Fig. 1 is the overall flow figure of the embodiment of the present invention.

Fig. 2 is that the figure of the embodiment of the present invention cuts (Graph-Cuts) schematic diagram.

Fig. 3 is that two degree superimposed images of the embodiment of the present invention add the splicing line after intervention to find schematic diagram.

Fig. 4 is that three degree superimposed images of the embodiment of the present invention add the splicing line after intervention to find schematic diagram.

Embodiment

When concrete enforcement, flow process that technical solution of the present invention provides can adopt computer software technology to realize operation automatically by those skilled in the art.In order to understand better technical scheme of the present invention, below in conjunction with drawings and Examples, the present invention is described in further detail.

Referring to accompanying drawing 1, embodiment of the present invention institute supplying method comprises the following steps:

Step 1, image data to be spliced is prepared.

Because this algorithm compatibility is stronger, can process various images, therefore for different images, data set-up procedure is slightly different.Generally first input image is identified, if full-view image (having proofreaied and correct to sphere), without carrying out directional process; If input image is aviation image, can adopt homography matrix to carry out relative geometric orientation, by all image unifications under the same coordinate system.On this basis, the image good to all geometry corrections carries out Overlap Analysis, builds the topological relation of image, and obtains all overlapping regions (comprise twice overlapping region and spend overlapping region) more.Embodiment specifically comprises following sub-step:

Step 1.1, image data type is judged, if image data to be spliced is full-view image, judge whether to proofread and correct, be directly to enter step 1.3, otherwise first adopt prior art to proofread and correct, the full-view image that obtains having proofreaied and correct, go to step 1.3, if image data to be spliced is aviation image, perform step 1.2.

Step 1.2, carries out Image Matching, by matching result, calculate the transformation matrix H (homography matrix) between image, and utilize this matrix to carry out relative geometric orientation, by all image unifications under the same coordinate system.

Step 1.3, carries out Overlap Analysis to the image of handling well, according to the topological relation between image, obtains overlapping region image data, meanwhile, calculates the degree of overlapping of each overlapping region, as overlapping even four degree of overlapping, three degree are overlapping etc. twice.According to overlapping region with and degree of overlapping, build image joint collection.

When concrete enforcement, the topological relation of Image Matching, computational transformation matrix H, relative orientation, overlay analysis, structure image, calculating degree of overlapping etc. can adopt prior art to carry out, for example, take manual reconnaissance mode to complete Image Matching.

Step 2, carries out pre-service to image to be spliced.Image pre-service mainly comprises four parts, and image is down-sampled, the conversion of image color space and gray processing, and image gradient calculation and Texture complication calculate, and embodiment specifically comprises following sub-step:

Step 2.1, image is down-sampled.In order to save the algorithm process time, the S% (the present embodiment is 20%) by original splicing image size reduction to raw video size.Meanwhile, in order to avoid the distortion of image data as far as possible, adopt bicubic interpolation (BicubicInterpolation) method to carry out interpolation.

Step 2.2, image expansion.This step is only for 360 ° of full-view image data, and other type image data, without carrying out this step, directly goes to step 2.3.Because full-view image is 360 ° of omnibearing visual angle photographies, after splicing, the rightmost side and the leftmost side of full-view image has corresponding continuity.In order to ensure this continuity, image is carried out to left and right edges expansion, extension width is K pixel (the present embodiment extension width is 10 pixels), by leftmost side width be the strip region duplication of K pixel to the rightmost side, be that the strip region duplication of K pixel is to the leftmost side by rightmost side width.

Step 2.3, the conversion of image color space and gray processing.For the top layer pyramid image after expansion, carry out the conversion of RGB color space to hsv color space, meanwhile, adopt image greyscale algorithm, RGB video conversion is arrived to gray space.Specifically be converted to prior art, it will not go into details in the present invention.

Step 2.4, image gradient calculation.For grayscale image data, adopt Sobel operator of the prior art calculate respectively each pixel p in the horizontal direction with the gradient magnitude G of vertical direction ^xand G (p) ^y(p), obtain gradient intensity figure.

Step 2.5, Texture complication calculates.Adopt gray processing image, calculate the Texture complication of each pixel.According to the gradient intensity figure of the horizontal direction calculating in step 2.4 and vertical direction, calculate the gradient direction size O (p) of each pixel p, computing formula is as follows:

$O\left(p\right)={\tan }^{-1}\frac{G^y\left(p\right)}{G^x\left(p\right)},$

And, its value is transformed in [0,2 π] scope.

According to gradient direction, each pixel is weighted to projection in histogram, weights size is horizontal direction and vertical gradient size absolute value sum, thereby obtain the gradient orientation histogram H of whole image, histogram dimension is h (this embodiment histogram dimension is 12).To each pixel p, centered by it, set up the window of a k × k (this embodiment window size is 11), for this window, in conjunction with gradient orientation histogram, can calculate its Texture complication, and formula is as follows:

Wherein, for the value of current interest window i dimension on gradient orientation histogram, the present embodiment adopts integrogram to calculate fast while specifically enforcement for the mean value in all dimensions, computing formula is as follows:

Step 3, determines that figure cuts global energy function.Suppose that current image to be spliced is that N opens, and is respectively image I ₁, I ₂, I ₃..., I _n, the image set of formation is the figure that the inventive method adopts cuts global energy function and comprises E _dataand E _smoothtwo, that is:

Wherein, E _dataand E _smoothrepresentative data energy term and level and smooth energy term respectively, embodiment specifically comprises following sub-step:

Step 3.1, data item energy design.Data energy term only whether overlapping relevant to image, be specifically calculated as follows:

Wherein, E _data(I _k) represent that each pixel comes from image I in the spliced image J of optimum splicing line _kenergy punishment, about image I _kdata item energy, it is defined as follows:

E _data(I _k)＝Σ _p∈JR _p(I _k)，

Wherein, p is certain pixel on image J after splicing, R _p(I _k) represent pixel p comes from image I _kenergy punishment, it is defined as follows:

$R_p\left(I_k\right)=\left\{\begin{array}{cc}0& p\∈I_k,\\ \∞& p\∉I_k.\end{array}\right.$

Step 3.2, a level and smooth energy design.Level and smooth energy term formed by three parts, i.e. texture space energy term E _texture, color space energy term and gradient space energy term

In order to describe better level and smooth energy term circular, first just there is the image I of overlapping region _a, I _bform image pair to be spliced calculation procedure be described in detail, and then be expanded to multiple image set

Carry out self imaging pair for each pixel p in image J after splicing texture space energy be defined as follows:

Wherein, Γ _aand Γ (p) _b(p) difference represent pixel p is at image I _aand I _bon Texture complication, adopt step 2.5 calculated results.

Carry out self imaging pair for each pixel p in image J after splicing color space energy be defined as follows:

Wherein, H and S represent that respectively raw video is converted to form and aspect (H) and saturation degree (S) the passage image in HSV space by rgb space, H (p) and S (p) represent respectively the value of pixel p in image H passage and S passage, w _hand w _s(those skilled in the art can according to circumstances preset value voluntarily, and in the present embodiment, weights are made as respectively w to be respectively corresponding power _h=1 and ws=0.1).

Carry out self imaging pair for each pixel p in image J after splicing gradient space energy definition as follows:

Wherein, λ is balance parameters (those skilled in the art can according to circumstances preset value voluntarily, are made as λ=0.25 in the present embodiment), with be respectively image I _ain the horizontal direction with the gradient magnitude of vertical direction, with be respectively image I _bin the horizontal direction with the gradient magnitude of vertical direction. with be respectively pixel p at image I _aon horizontal direction gradient magnitude, at image I _aon vertical gradient size, at image I _bon horizontal direction gradient magnitude, with at image I _bon vertical gradient size.

Level and smooth energy consideration be two energy punishment between neighbor, above-mentioned definition for be all single pixel, level and smooth energy punishment is defined as follows:

Wherein, p and q are two adjacent pixels (neighbours territories), for the neighbours territory of pixel p, with represent that respectively neighbor p and q are at I _aand I _bthe level and smooth energy of color space that overlapping region is corresponding and the level and smooth energy of gradient space.

Given image pair to be spliced its level and smooth energy definition is as follows:

Level and smooth energy theorem defined above is all for image pair between overlapping region twice, further can extend to many images between many degree overlapping regions, its level and smooth energy punishment is defined as follows:

with represent respectively the level and smooth energy of color space and the level and smooth energy of gradient space that neighbor p and q are corresponding in overlapping region.

Color space energy term and gradient space energy term specific formula for calculation as follows:

Wherein, with the numbering of pixel p and the affiliated image of q in the rear image J of representative splicing respectively, function be defined as follows:

Texture, color and the gradient space energy definition of comprehensive foregoing description, E _smoothbe defined as follows:

For texture space energy term in color space energy term with gradient space energy term calculating in existing consider, therefore also can select not consider according to actual conditions.

Step 4, level and smooth energy weight setting.Appear near edge zone, overlapping region for fear of splicing line, ensure the rationality of splicing line searching result, the closer to edge, weight is larger, less the closer to zone line weight, and the color of single pixel and gradient space energy redefine as follows:

Wherein, w _d(p) be the weight of pixel p near edge, overlapping region, it is defined as follows:

$w_d\left(p\right)=\left\{\begin{array}{cc}1& \mathrm{ifd}\left(p\right)>d_{\mathrm{th}}\\ \frac{d_{\mathrm{th}}}{d\left(p\right)}& \mathrm{otherwise}\end{array}\right.$

Wherein, d (p) represents the bee-line of pixel p to edge, overlapping region, d _thfor predetermined threshold value, (while specifically enforcement, those skilled in the art can according to circumstances preset value voluntarily, and in the present embodiment, weights are made as d _th=10).

Step 5, employing figure cuts (Graph-Cuts) energy optimizing method, to redefine color space energy term through step 4 and gradient space energy term after gross energy be optimized, obtain optimum splicing line.This energy-optimised mode is divided into α-expansion and alpha-beta-swap, and the present embodiment adopts alpha-beta-swap to gross energy be optimized, specific implementation is prior art.In order to ensure the optimality of optimum results, iterations of the present invention is set to 10 times.In iterative process, when gross energy hour, the splicing effect obtaining is best.Accompanying drawing 2 is for cutting the simple examples that (Graph-Cuts) splicing line is found based on figure, between image A, image B, there is overlapping region, the size of the level and smooth energy of the thickness of line representative between pixel, thicker corresponding energy is larger, " cut " is final optimum results, i.e. best splicing line.

Consider after being optimized according to the optimum splicing line of step 5 gained and may occur that local optimum effect does not reach user's request, for local optimum poor effect region, the further proposition of the present invention can belong to numbering by choosing point set and giving its image in each overlapping region in local optimum poor effect region, thereby walk around obvious atural object, to reach best splicing effect.Embodiment concrete steps are as follows:

Steps A 1 is chosen point set in image overlap area, can draw straight line, curve, rectangle frame etc. by user and choose point set, and give its image ownership numbering while specifically enforcement, and the sets definition of establishing M the point of choosing is the image ownership number table that it is given is shown

Steps A 2, the point set of choosing for steps A 1 the level and smooth energy term of its correspondence is not changed, and corresponding data item is amended as follows:

Wherein, x _ia pixel of choosing for steps A 1, for steps A 1 is given to pixel x _iimage ownership numbering.After this amendment energy term, employing figure cuts (Graph-Cuts) energy optimizing method and obtains optimum splicing line, i.e. step 4 gained gross energy again middle E _data(I _k) make into after re-execute step 5 and obtain new optimum splicing line.

Accompanying drawing 3 is that two degree superimposed images add the splicing line after intervening to find schematic diagram, there are image A (video number is 1) and image B (video number is 2), their overlapping region is O (representing with dash area in figure), the M that belongs to image A choosing in O _athe point set that point forms is (the short solid line with black in figure represents), the image ownership number table that it is given is shown the M that belongs to image B choosing _bthe point set that point forms is (in figure, showing with black rectangle frame table), the image ownership number table that it is given is shown m (the M=M choosing _a+ M _b) set of individual point is Υ=Υ ₁∪ Υ ₂do not change for the level and smooth energy term that the point set of choosing is corresponding, after corresponding data item amendment, re-execute step 5, obtain increasing the splicing line (representing with solid line in figure) after intervening, than original splicing line (dotting in figure), splicing line makes to choose point set and belongs to respectively image A and B, thereby walks around obvious atural object, can reach best splicing effect.Accompanying drawing 4 is same principles, has image A, image B and image C, therefrom chooses respectively point set and intervenes optimization.Can expand in addition many degree superimposed images adds connecing of intervention to spell line searching.

When concrete enforcement, also can adopt corresponding system to realize, provided by the inventionly a kind ofly cut optimum splicing line searching system between energy-optimised image based on figure, comprise with lower module:

Data preparation module, for carrying out image data preparation to be spliced, obtains all overlapping regions and corresponding degree of overlapping between each image to be spliced;

Pretreatment module, for carrying out pre-service to image to be spliced;

Energy function determination module, for determining that figure cuts global energy function;

Weight setting module, for level and smooth energy weight setting;

Optimize module, for adopting figure to cut energy optimizing method, the gross energy after weight setting module redefines is optimized, obtain optimum splicing line.

Each module specific implementation can be participated in corresponding steps and be illustrated.

Instantiation described herein is only to the explanation for example of the present invention's spirit.Those skilled in the art can make various amendments or supplement or adopt similar mode to substitute described instantiation, but can't depart from spirit of the present invention or surmount the defined scope of appended claims.

Claims (10)

1. cut an optimum splicing line finding method between energy-optimised image based on figure, comprise the following steps:

Step 1, carries out image data preparation to be spliced, obtains all overlapping regions and corresponding degree of overlapping between each image to be spliced;

Step 2, carries out pre-service to image to be spliced;

Step 3, determines that figure cuts global energy function as follows,

Open if current image to be spliced is N, be respectively image I ₁, I ₂, I ₃..., I _n, the image set of formation is figure cuts global energy function and comprises E _dataand E _smoothtwo,

Wherein, E _dataand E _smoothrepresentative data energy term and level and smooth energy term respectively, it is specifically calculated as follows,

Described data energy term be calculated as follows,

Wherein, E _data(I _k) represent that each pixel comes from image I in the spliced image J of optimum splicing line _kenergy punishment, be defined as follows,

Edata(I _k)＝Σ _p∈JR _p(I _k)

Wherein, p is certain pixel on image J after splicing, R _p(I _k) represent pixel p comes from image I _kenergy punishment, be defined as follows,

$R_p\left(I_k\right)=\left\{\begin{array}{cc}0& p\∈I_k\\ \∞& p\∉I_k\end{array}\right.$

Described level and smooth energy term be defined as follows,

Wherein, color space energy term and gradient space energy term computing formula as follows,

Wherein, p and q are two adjacent pixels, for the neighbours territory of pixel p, with the numbering of pixel p and the affiliated image of q in the rear image J of representative splicing respectively, function be defined as follows,

with represent respectively the level and smooth energy of color space and the level and smooth energy of gradient space that neighbor p and q are corresponding in overlapping region, be defined as follows,

Wherein, I _aand I _bfor image set in any two images that have an overlapping region, establish image I _a, I _bform image pair to be spliced with be respectively neighbor p and q at I _aand I _bthe level and smooth energy of color space that overlapping region is corresponding and the level and smooth energy of gradient space;

Step 4, level and smooth energy weight setting, comprises the color space energy term of single pixel and gradient space energy term redefine as follows,

Wherein, w _d(p) be the weight of pixel p near edge, overlapping region, be defined as follows:,

$w_d\left(p\right)=\left\{\begin{array}{cc}1& \mathrm{ifd}\left(p\right)>d_{\mathrm{th}}\\ \frac{d_{\mathrm{th}}}{d\left(p\right)}& \mathrm{otherwise}\end{array}\right.$

Wherein, d (p) represents the bee-line of pixel p to edge, overlapping region, d _thfor predetermined threshold value;

Step 5, employing figure cuts energy optimizing method, to redefine color space energy term through step 4 and gradient space energy term after gross energy be optimized, obtain optimum splicing line.

2. cut optimum splicing line finding method between energy-optimised image based on figure according to claim 1, it is characterized in that: there is local optimum poor effect after being optimized according to the optimum splicing line of step 5 gained time, each overlapping region to local optimum poor effect region adds respectively intervention, optimize splicing line, step is as follows

Steps A 1 is chosen point set in overlapping region, and gives corresponding image ownership numbering, and the sets definition of establishing M the point of choosing is the corresponding image ownership number table of giving is shown

Steps A 2, the point set of choosing for steps A 1 corresponding level and smooth energy term is not changed, and corresponding data item is amended as follows,

Wherein, x _ia pixel of choosing for steps A 1, for steps A 1 is given to pixel x _iimage ownership numbering, after this amendment energy term, employing figure cuts energy optimizing method and obtains optimum splicing line again.

3. according to cutting optimum splicing line finding method between energy-optimised image based on figure described in claim 1 or 2, it is characterized in that: described neighbor p and q are at I _aand I _bthe level and smooth energy of color space that overlapping region is corresponding and the level and smooth energy of gradient space with obtain manner as follows,

Carry out self imaging pair for each pixel p in image J after splicing texture space energy be defined as follows,

Wherein, Γ _aand Γ (p) _b(p) difference represent pixel p is at image I _aand I _bon Texture complication;

Carry out self imaging pair for each pixel p in image J after splicing color space energy be defined as follows,

Wherein, H (p) and S (p) represent respectively the value of pixel p in image H passage and S passage, w _hand w _sbe respectively corresponding power;

Carry out self imaging pair for each pixel p in image J after splicing gradient space energy definition as follows,

wherein, λ is balance parameters, with be respectively pixel p at image I _aon horizontal direction gradient magnitude, at image I _aon vertical gradient size, at image I _bon horizontal direction gradient magnitude and at image I _bon vertical gradient size;

The level and smooth energy of color space and the level and smooth energy of gradient space with be defined as follows,

4. according to cutting optimum splicing line finding method between energy-optimised image based on figure described in claim 1 or 2, it is characterized in that: described in step 2, image to be spliced is carried out to pre-service, comprise that image is down-sampled, the conversion of image color space and gray processing, image gradient calculation and Texture complication calculate.

5. cut optimum splicing line finding method between energy-optimised image based on figure according to claim 4, it is characterized in that: in the time that image to be spliced is panoramic picture, image to be spliced is carried out to pre-service described in step 2 and comprise and carry out image expansion.

6. cut an optimum splicing line searching system between energy-optimised image based on figure, comprise with lower module:

Data preparation module, for carrying out image data preparation to be spliced, obtains all overlapping regions and corresponding degree of overlapping between each image to be spliced;

Pretreatment module, for carrying out pre-service to image to be spliced;

Energy function determination module, as follows for determining that figure cuts global energy function,

Open if current image to be spliced is N, be respectively image I ₁, I ₂, I ₃..., I _n, the image set of formation is figure cuts global energy function and comprises E _dataand E _smoothtwo,

Wherein, E _dataand E _smoothrepresentative data energy term and level and smooth energy term respectively, it is specifically calculated as follows,

Described data energy term be calculated as follows,

Wherein, E _data(I _k) represent that each pixel comes from image I in the spliced image J of optimum splicing line _kenergy punishment, be defined as follows,

Edata(I _k)＝Σ _p∈JR _p(I _k)

Wherein, p is certain pixel on image J after splicing, R _p(I _k) represent pixel p comes from image I _kenergy punishment, be defined as follows,

$R_p\left(I_k\right)=\left\{\begin{array}{cc}0& p\∈I_k\\ \∞& p\∉I_k\end{array}\right.$

Described level and smooth energy term be defined as follows:

Wherein, color space energy term and gradient space energy term computing formula as follows,

Wherein, p and q are two adjacent pixels, for the neighbours territory of pixel p, with the numbering of pixel p and the affiliated image of q in the rear image J of representative splicing respectively, function be defined as follows,

with represent respectively the level and smooth energy of color space and the level and smooth energy of gradient space that neighbor p and q are corresponding in overlapping region, be defined as follows,

Wherein, I _aand I _bfor image set in any two images that have an overlapping region, establish image I _a, I _bform image pair to be spliced with be respectively neighbor p and q at I _aand I _bthe level and smooth energy of color space that overlapping region is corresponding and the level and smooth energy of gradient space;

Weight setting module, for level and smooth energy weight setting, comprises the color space energy term of single pixel and gradient space energy term redefine as follows,

Wherein, w _d(p) be the weight of pixel p near edge, overlapping region, be defined as follows,

$w_d\left(p\right)=\left\{\begin{array}{cc}1& \mathrm{ifd}\left(p\right)>d_{\mathrm{th}}\\ \frac{d_{\mathrm{th}}}{d\left(p\right)}& \mathrm{otherwise}\end{array}\right.$

Wherein, d (p) represents the bee-line of pixel p to edge, overlapping region, d _thfor predetermined threshold value;

Optimize module, for adopting figure to cut energy optimizing method, to redefine color space energy term through weight setting module and gradient space energy term after gross energy be optimized, obtain optimum splicing line.

7. cut optimum splicing line searching system between energy-optimised image based on figure according to claim 6, it is characterized in that: intervention module is set, while being used for occurring local optimum poor effect after being optimized according to the optimum splicing line of optimization module gained, each overlapping region to local optimum poor effect region adds respectively intervention, optimize splicing line, step is as follows

Steps A 1 is chosen point set in overlapping region, and gives corresponding image ownership numbering, and the sets definition of establishing M the point of choosing is the corresponding image ownership number table of giving is shown

Steps A 2, the point set of choosing for steps A 1 corresponding level and smooth energy term is not changed, and corresponding data item is amended as follows,

Wherein, x _ia pixel of choosing for steps A 1, for steps A 1 is given to pixel x _iimage ownership numbering, after this amendment energy term, employing figure cuts energy optimizing method and obtains optimum splicing line again.

8. according to cutting optimum splicing line searching system between energy-optimised image based on figure described in claim 6 or 7, it is characterized in that: described neighbor p and q are at I _aand I _bthe level and smooth energy of color space that overlapping region is corresponding and the level and smooth energy of gradient space with obtain manner as follows,

Carry out self imaging pair for each pixel p in image J after splicing texture space energy be defined as follows,

Wherein, Γ _aand Γ (p) _b(p) difference represent pixel p is at image I _aand i _bon Texture complication;

Carry out self imaging pair for each pixel p in image J after splicing color space energy be defined as follows,

Wherein, H (p) and S (p) represent respectively the value of pixel p in image H passage and S passage, w _hand w _sbe respectively corresponding power;

Carry out self imaging pair for each pixel p in image J after splicing gradient space energy definition as follows,

wherein, λ is balance parameters, with be respectively pixel p at image I _aon horizontal direction gradient magnitude, at image I _aon vertical gradient size, at image I _bon horizontal direction gradient magnitude and at image I _bon vertical gradient size;

The level and smooth energy of color space and the level and smooth energy of gradient space with be defined as follows,

9. according to cutting optimum splicing line searching system between energy-optimised image based on figure described in claim 6 or 7, it is characterized in that: the pre-service that pretreatment module is carried out image to be spliced, comprises that image is down-sampled, the conversion of image color space and gray processing, image gradient calculation and Texture complication calculate.

10. cut optimum splicing line searching system between energy-optimised image based on figure according to claim 9, it is characterized in that: in the time that image to be spliced is panoramic picture, the pre-service that pretreatment module image to be spliced carries out comprises carries out image expansion.