CN103985133B - Search method and system for optimal splicing lines among images based on graph-cut energy optimization - Google Patents

Abstract

The invention discloses a search method and system for optimal splicing lines among images based on graph-cut energy optimization. According to the method, data preparation of the images to be spliced is performed, and all overlap regions and corresponding overlap degrees among all the images to be spliced are obtained; the images to be spliced are preprocessed, color, gradient and texture information is comprehensively considered to determine a graph-cut global energy function which includes a data item and a smooth item, weighting setting is performed on energy of the smooth item, the graph-cut energy optimization method is adopted for optimizing total energy, and therefore the optimal splicing lines are obtained. The obtained splicing lines are prevented from passing through regions which are overlarge in color difference and complex in texture and passing through edges obvious in ground feature as much as possible, so that optimization of the splicing lines is ensured to the maximum degree, the problem of optimizing multi-image multi-degree overlap region splicing lines in a combined mode is solved, and the application range is wide.

Description

Optimum splicing line finding method and system between energy-optimised image are cut based on figure

Technical field

The invention belongs to digitized video processing technology field, more particularly to one kind cut (graph-cuts) energy based on figure Between the image optimizing, optimum splicing line finds technical scheme and system.

Background technology

In digitized video splicing, due to the shadow by the factor such as Image registration geometric error and image aberration Ring, cause the region differing greatly between image (as building, vehicle and pedestrian etc. substantially in shooting visual angle or target motion etc. Atural object), spliced image often exists and is more substantially misaligned phenomenon.Therefore, how quickly and efficiently to find an optimum Splicing line (avoiding through image difference large area), be the most key technology in image joint, be also computer vision One of with the hot subject of photogrammetric area research, in full-view image splicing and aviation image splicing, (especially low latitude tilts Image joint) etc. in be most widely used.

The method of splicing line Automatic-searching mainly has three kinds of technology paths at present: one is the method based on assistance data, and two It is object-based splicing line finding method, three is the method based on overlapping region image difference.

Splicing line finding method based on assistance data mainly utilizes assistance data (road vectors data, lidar point cloud number According to etc.) row constraint is entered to the trend of splicing line, thus obtaining preferable splicing result.The major defect of the method is to auxiliary The dependence of data, the quality of splicing result is heavily dependent on the precision of assistance data.

Object-based splicing line finding method combines image segmentation algorithm, carries out Image Segmentation to overlapping region, obtains The substantially distributed areas of atural object, thus for the searching instructing optimum splicing line.The limitation of the method essentially consists in target and divides The accuracy cut cannot be guaranteed, the effect of impact splicing.

Splicing line finding method based on overlapping region image difference utilizes the difference of overlapping region respective pixel, builds and closes The energy function of reason, and be optimized using suitable energy optimizing method.The method can obtain preferable splicing effect, but Time-consuming longer, computation complexity is higher, less efficient.

Content of the invention

In order to solve above-mentioned technical problem, energy is quick, stable, search out optimum exactly from overlapping region between image Splicing line, the invention provides a kind of cut optimum splicing line searching skill between (graph-cuts) energy-optimised image based on figure Art.

The technical solution adopted in the present invention provides a kind of cut between energy-optimised image optimum splicing line based on figure and seeks Look for method, comprise the following steps:

Step 1, carries out image data preparation to be spliced, obtains between each image to be spliced all overlapping regions and accordingly weighs Folded degree；

Step 2, pre-processes to image to be spliced；

Step 3, determines that figure cuts global energy function as follows,

If currently image to be spliced is opened for n, respectively image i₁, i₂, i₃..., i_n, the image set of composition isFigure cuts global energy function and includes e_dataAnd e_smoothTwo,

Wherein, e_dataAnd e_smoothRepresent data capacity item and smooth energy term respectively, it is specifically calculated as follows,

Described data capacity itemBe calculated as follows,

Wherein, e_data(i_k) represent come from image i through each pixel in optimum splicing line spliced image j_kEnergy Punishment, is defined as follows,

e_data=(i_k)σ_p∈jr_p(ik)

Wherein, p is certain pixel after splicing on image j, r_p(i_k) represent pixel p and come from image i_kEnergy punishment, It is defined as follows,

$r_p\left(i_k\right)=\left\{\begin{array}{cc}0& p\&element;i_k\\ \∞& p\&notelement;i_k\end{array}\right.$

Described smoothed energy itemIt is defined as follows,

Wherein, color space energy termAnd gradient space energy termComputing formula such as Under,

Wherein, p and q is two adjacent pixels,For four neighborhoods of pixel p,With Represent the numbering of pixel p and the affiliated image of q in image j after splicing, function respectivelyIt is defined as follows,

WithRepresent neighbor p and q respectively in the corresponding color in overlapping region Space smoothing item energy and gradient space smooth item energy, are defined as follows,

Wherein, i_aAnd i_bFor image setThere is the image of overlapping region in middle any two, if image i_a, i_b Constitute image pair to be splicedWithIt is respectively neighbor p and q In i_aAnd i_bThe corresponding color space in overlapping region smooths item energy and gradient space smooths item energy；

Step 4, the setting of smooth item energy weight, including by the color space energy term of single pixelAnd ladder Degree dimensional energy itemRedefine as follows,

Wherein, w_dP (), is defined as follows near the weight at overlapping region edge for pixel p,

$w_d\left(p\right)=\left\{\begin{array}{cc}1& \mathrm{ifd}\left(p\right)>d_{\mathrm{th}}\\ \frac{d_{\mathrm{th}}}{d\left(p\right)}& \mathrm{otherwise}\end{array}\right.$

Wherein, d (p) represent pixel p to overlapping region edge beeline, d_thFor predetermined threshold value；

Step 5, cuts energy optimizing method using figure, redefines color space energy term to through step 4With And gradient space energy termGross energy afterwardsIt is optimized, obtain optimum splicing line.

And, when local optimum effect on driving birds is not good occurring after being optimized according to step 5 gained optimum splicing line, to local Each overlapping region in the not good region of effect of optimization is separately added into intervention, optimizes splicing line, and step is as follows,

Step a1, chooses point set in overlapping region, and gives corresponding image ownership numbering, if the collection of the m point chosen Conjunction is defined asThe corresponding image ownership numbering giving is expressed as

Step a2, the point set chosen for step a1Corresponding smoothed energy item is without modification, corresponding Data item is amended as follows,

Wherein, x_iThe pixel chosen for step a1,It is given to pixel x for step a1_iImage ownership compile Number, again energy optimizing method is cut using figure after this modification energy term and obtain optimum splicing line.

And, described neighbor p and q is in i_aAnd i_bThe corresponding color space in overlapping region smooths item energy and gradient is empty Between smooth item energyWithAcquisition modes as follows,

Carry out self imaging pair for each pixel p in image j after splicingTexture space energyIt is defined as follows,

Wherein, γ_aP () and γ b (p) represent pixel p respectively in image i_aAnd i_bOn Texture complication；

Carry out self imaging pair for each pixel p in image j after splicingColor space energyIt is defined as follows,

Wherein, h (p) and s (p) represents value in image h passage and s passage for the pixel p, w respectively_hAnd w_sIt is respectively and correspond to Power；

Carry out self imaging pair for each pixel p in image j after splicingGradient space energy definition such as Under,

Wherein, λ is balance parameters,WithIt is respectively pixel p in image i_aOn water Square to gradient magnitude, in image i_aOn vertical gradient size, in image i_bOn horizontal direction gradient magnitude and in shadow As i_bOn vertical gradient size；

Color space smooths item energy and gradient space smooths item energyWith It is defined as follows,

And, described in step 2, image to be spliced is pre-processed, including image is down-sampled, the conversion of image color space And gray processing, image gradient calculate and Texture complication calculates.

And, when image to be spliced is panoramic picture, image to be spliced is pre-processed described in step 2 include into Row image extends.

The present invention further correspondingly provide a kind of optimum splicing line searching system is cut between energy-optimised image based on figure, including With lower module:

Data preparation module, is used for carrying out image data preparation to be spliced, obtains all overlaps between each image to be spliced Region and corresponding degree of overlapping；

Pretreatment module, for pre-processing to image to be spliced；

Energy function determining module, as follows for determining that figure cuts global energy function,

If currently image to be spliced is opened for n, respectively image i₁, i₂, i₃..., i_n, the image set of composition isFigure cuts global energy function and includes e_dataAnd e_smoothTwo,

Wherein, e_dataAnd e_smoothRepresent data capacity item and smooth energy term respectively, embodiment specifically includes following sub-step Rapid:

Described data capacity itemBe calculated as follows,

Wherein, e_data(ik) represent and come from image i through each pixel in optimum splicing line spliced image j_kEnergy Punishment, is defined as follows,

e_data(i_k)=σ_p∈jr_p(i_k)

Wherein, p is certain pixel after splicing on image j, r_p(i_k) represent pixel p and come from image i_kEnergy punishment, It is defined as follows,

$r_p\left(i_k\right)=\left\{\begin{array}{cc}0& p\&element;i_k\\ \∞& p\&notelement;i_k\end{array}\right.$

Described smoothed energy itemIt is defined as follows:

Wherein, color space energy termAnd gradient space energy termComputing formula such as Under,

Wherein, p and q is two adjacent pixels,For four neighborhoods of pixel p,With Represent the numbering of pixel p and the affiliated image of q in image j after splicing, function respectivelyIt is defined as follows,

WithRepresent neighbor p and q respectively in the corresponding color in overlapping region Space smoothing item energy and gradient space smooth item energy, are defined as follows,

Wherein, i_aAnd i_bFor image setThere is the image of overlapping region in middle any two, if image i_a, i_bStructure Become image pair to be splicedWithIt is respectively neighbor p and q to exist i_aAnd i_bThe corresponding color space in overlapping region smooths item energy and gradient space smooths item energy；

Weight setting module, for smoothing the setting of item energy weight, including by the color space energy term of single pixelAnd gradient space energy termRedefine as follows,

Wherein, w_dP (), is defined as follows near the weight at overlapping region edge for pixel p:,

$w_d\left(p\right)=\left\{\begin{array}{cc}1& \mathrm{ifd}\left(p\right)>d_{\mathrm{th}}\\ \frac{d_{\mathrm{th}}}{d\left(p\right)}& \mathrm{otherwise}\end{array}\right.$

Wherein, d (p) represent pixel p to overlapping region edge beeline, d_thFor predetermined threshold value；

Optimization module, for cutting energy optimizing method using figure, redefines color space energy to through weight setting module QuantifierAnd gradient space energy termGross energy afterwardsIt is optimized, obtain optimum splicing Line.

And, intervention module is set, for local after being optimized according to optimization module gained optimum splicing line When effect of optimization is good, intervention is separately added into each overlapping region in the not good region of local effect of optimization, optimizes splicing line, step It is suddenly as follows,

Step a1, chooses point set in overlapping region, and gives corresponding image ownership numbering, if the collection of the m point chosen Conjunction is defined asThe corresponding image ownership numbering giving is expressed as

Step a2, the point set chosen for step a1Corresponding smoothed energy item is without modification, corresponding Data item is amended as follows,

Wherein, x_iThe pixel chosen for step a1,It is given to pixel x for step a1_iImage ownership numbering, Again energy optimizing method is cut using figure after this modification energy term and obtain optimum splicing line.

And, described neighbor p and q is in i_aAnd i_bThe corresponding color space in overlapping region smooths item energy and gradient is empty Between smooth item energyWithAcquisition modes as follows,

Carry out self imaging pair for each pixel p in image j after splicingTexture space energyIt is defined as follows,

Wherein, γ_a(p) and γ_bP () represents pixel p respectively in image i_aAnd i_bOn Texture complication；

Carry out self imaging pair for each pixel p in image j after splicingColor space energyIt is defined as follows,

Wherein, h (p) and s (p) represents value in image h passage and s passage for the pixel p, w respectively_hAnd w_sIt is respectively and correspond to Power；

Carry out self imaging pair for each pixel p in image j after splicingGradient space energy definition such as Under,

Wherein, λ is balance parameters,WithIt is respectively pixel p in image i_aOn water Square to gradient magnitude, in image i_aOn vertical gradient size, in image i_bOn horizontal direction gradient magnitude and in shadow As i_bOn vertical gradient size；

Color space smooths item energy and gradient space smooths item energyWith It is defined as follows,

And, the pretreatment that pretreatment module is carried out to image to be spliced, including image is down-sampled, image color space turns Change and gray processing, image gradient calculate and Texture complication calculates.

And, when image to be spliced is panoramic picture, the pretreatment that pretreatment module image to be spliced is carried out includes Carry out image extension.

With respect to prior art, the present invention can quick and precisely automatically look for obtaining optimum splicing line between image.Spelling During wiring Automatic-searching, the present invention has considered color, gradient and texture information so that splicing line avoids leading to as far as possible Cross that aberration is excessive, the edge of texture complex region and obvious atural object, thus ensure that the optimization of splicing line to the full extent. Meanwhile, the present invention creatively employs figure and cuts energy optimizing method, be compared to traditional ant colony, Dynamic Programming (dp, Dynamic programming) and the optimization method such as dijkstra, this optimization method effect of optimization is more preferably, in hgher efficiency.No Only such, this invention also solves the problem of the overlapping region of degree more than many images splicing line combined optimization, it is compared to traditional two The method being optimized between two images, many image associations optimization efficiency is higher, and effect is also more reasonable.In addition, the present invention fits Wide with property, can preferably solve the Bonding Problem that full-view image, orthography and low latitude tilt the data such as image.

Brief description

Fig. 1 is the overall flow figure of the embodiment of the present invention.

Fig. 2 is that the figure of the embodiment of the present invention cuts (graph-cuts) schematic diagram.

Fig. 3 is that two degree of superimposed images of the embodiment of the present invention add the splicing line after intervening to find schematic diagram.

Fig. 4 is that three degree of superimposed images of the embodiment of the present invention add the splicing line after intervening to find schematic diagram.

Specific embodiment

When being embodied as, the provided flow process of technical solution of the present invention can adopt computer software skill by those skilled in the art Art realizes automatic running.Technical scheme for a better understanding of the present invention, does to the present invention with reference to the accompanying drawings and examples Further detailed description.

Referring to accompanying drawing 1, the provided method of the embodiment of the present invention comprises the following steps:

Step 1, image data to be spliced prepares.

Because this algorithm compatibility is stronger, various images can be processed, therefore be directed to different images, Data Preparation Process It is slightly different.Typically first input image is identified, if full-view image (having corrected to sphere), then need not be oriented Process；If input image is aviation image, geometric orientation relatively can be carried out using homography matrix, the unification of all images is arrived Under the same coordinate system.On this basis, the image good to all geometric corrections is laid out analyzing, and builds the topology pass of image System, and obtain all overlapping regions (include twice overlapping region and spend overlapping region) more.Embodiment specifically includes following Sub-step:

Step 1.1, judges to image data type, if image data to be spliced is full-view image, judges whether Correct, be, be directly entered step 1.3, be otherwise corrected initially with prior art, obtain the full-view image having corrected, Go to step 1.3, if image data to be spliced is aviation image, execution step 1.2.

Step 1.2, carries out Image Matching, by matching result, calculates transformation matrix h (the homography square between image Battle array), and carry out geometric orientation relatively using this matrix, by all image unifications under the same coordinate system.

Step 1.3, is laid out to the image handled well analyzing, according to the topological relation between image, obtains overlay region Domain image data, calculates the degree of overlapping of each overlapping region meanwhile, such as overlapping twice, three degree of overlapping even four degree of overlaps etc..Root According to overlapping region and its degree of overlapping, build image joint collection.

When being embodied as, Image Matching, calculating transformation matrix h, relative orientation, overlay analysis, the topology pass of structure image System, calculating degree of overlapping etc. can be carried out using prior art, for example, take manual reconnaissance mode to complete Image Matching.

Step 2, pre-processes to image to be spliced.Yunnan snub-nosed monkey mainly includes four parts, and that is, image is down-sampled, shadow As color space conversion and gray processing, image gradient calculates and Texture complication calculates, and embodiment specifically includes following sub-step Rapid:

Step 2.1, image is down-sampled.For saving algrithm process time, will be extremely original for original splicing image size reduction The s% (the present embodiment is 20%) of image size.Meanwhile, in order to avoid the distortion of image data as far as possible, using bicubic interpolation (bicubicinterpolation) method enters row interpolation.

Step 2.2, image extends.This step need not be entered just for 360 ° of full-view image data, other type image datas This step of row, directly goes to step 2.3.Because full-view image is 360 ° of omnibearing visual angle photographies, that is, full-view image after splicing The rightmost side and the leftmost side have corresponding continuity.In order to ensure this continuity, image is carried out with left and right edges extension, extension is wide Spend for k pixel (the present embodiment extension width be 10 pixels), will leftmost side width be that the strip region of k pixel is answered Make the rightmost side, by rightmost side width be k pixel strip region duplication to the leftmost side.

Step 2.3, the conversion of image color space and gray processing.For the top pyramid image after extension, carry out rgb Color space to the conversion of hsv color space, meanwhile, using image greyscale algorithm, by rgb video conversion to gray space. Specifically be converted to prior art, it will not go into details for the present invention.

Step 2.4, image gradient calculates.For grayscale image data, counted respectively using sobel operator of the prior art Calculate each pixel p gradient magnitude g both horizontally and vertically^x(p) and g^yP (), obtains gradient intensity figure.

Step 2.5, Texture complication calculates.Using gray processing image, calculate the Texture complication of each pixel.According to step Calculated gradient intensity figure horizontally and vertically in rapid 2.4, the gradient direction being calculated each pixel p is big Little o (p), computing formula is as follows:

$o\left(p\right)={\tan }^{-1}\frac{g^y\left(p\right)}{g^x\left(p\right)},$

And, its value is transformed in the range of [0,2 π].

Each pixel is weighted in histogram project according to gradient direction, weights size is horizontal direction and vertical Direction gradient size absolute value sum, thus obtaining the gradient orientation histogram h of whole image, histogram dimension is h (this enforcement Example histogram dimension is 12).To each pixel p, centered on it, set up the window of a k × k(this enforcement Example window size is 11), for this window, in conjunction with gradient orientation histogram, you can be calculated its Texture complication, formula As follows:

Wherein,For the value of current interest window i-th dimension on gradient orientation histogram, the present embodiment tool Body is quickly calculated using integrogram when implementingFor the mean value in all dimensions, computing formula As follows:

Step 3, determines that figure cuts global energy function.Assume that currently image to be spliced is opened for n, respectively image i₁, i₂, i₃..., i_n, the image set of composition isThe figure that the inventive method is adopted cuts global energy function and includes e_dataWith e_smoothTwo it may be assumed that

Wherein, e_dataAnd e_smoothRepresent data capacity item and smooth energy term respectively, embodiment specifically includes following sub-step Rapid:

Step 3.1, data item energy design.Data capacity itemOnly whether overlapping to image related, concrete calculating As follows:

Wherein, e_data(i_k) represent come from image i through each pixel in optimum splicing line spliced image j_kEnergy Punishment, that is, with regard to image i_kData item energy, it is defined as follows:

e_data(i_k)=σ_p∈jr_p(i_k),

Wherein, p is certain pixel after splicing on image j, r_p(i_k) represent pixel p and come from image i_kEnergy punishment, It is defined as follows:

$r_p\left(i_k\right)=\left\{\begin{array}{cc}0& p\&element;i_k,\\ \∞& p\&notelement;i_k.\end{array}\right.$

Step 3.2, smooth item energy design.Smoothed energy itemIt is made up of three parts, i.e. texture space energy Item e_texture, color space energy termAnd gradient space energy term

In order to preferably describe smoothed energy itemCircular, first there is the shadow of overlapping region As i_a, i_bConstituted image pair to be splicedCalculation procedure be described in detail, then further expansion is to multiple shadows Image set

Carry out self imaging pair for each pixel p in image j after splicingTexture space energyIt is defined as follows:

Wherein, γ_a(p) and γ_bP () represents pixel p respectively in image i_aAnd i_bOn Texture complication, using step 2.5 Calculated results.

Carry out self imaging pair for each pixel p in image j after splicingColor space energyIt is defined as follows:

Wherein, h and s represents that raw video is converted into the form and aspect (h) in hsv space by rgb space and saturation degree (s) is led to respectively Road image, h (p) and s (p) represent value in image h passage and s passage for the pixel p, w respectively_hAnd w_sIt is respectively corresponding power (this Skilled person voluntarily can according to circumstances preset value, and in the present embodiment, weights are set to w_h=1 and ws=0.1).

Carry out self imaging pair for each pixel p in image j after splicingGradient space energy definition such as Under:

Wherein, λ is that (those skilled in the art voluntarily can according to circumstances preset value to balance parameters, in the present embodiment It is set to λ=0.25),WithIt is respectively image i_aGradient magnitude both horizontally and vertically,WithIt is respectively Image i_bGradient magnitude both horizontally and vertically.WithIt is respectively pixel p In image i_aOn horizontal direction gradient magnitude, in image i_aOn vertical gradient size, in image i_bOn horizontal direction Gradient magnitude, and in image i_bOn vertical gradient size.

Smooth item energy consideration between two neighbors energy punishment, what above-mentioned definition was directed to is all single Pixel, smooth item energy punishment is defined as follows:

Wherein, p and q is two adjacent pixels (four neighborhoods), that is,For four neighborhoods of pixel p,WithRepresent neighbor p and q respectively in i_aAnd i_bThe corresponding color in overlapping region Space smoothing item energy and gradient space smooth item energy.

Give image pair to be splicedIts smooth item energy definition is as follows:

Smooth item energy theorem defined above is both for image pairBetween overlapping region twice, enter One step can be extended to many imagesBetween many degree overlapping regions, its smooth item energy punishment is defined as follows:

WithRepresent neighbor p and q respectively in the corresponding color in overlapping region Space smoothing item energy and gradient space smooth item energy.

Color space energy termAnd gradient space energy termSpecific formula for calculation as follows:

Wherein,WithRepresent the numbering of pixel p and the affiliated image of q in image j after splicing, function respectivelyIt is defined as follows:

The texture of summary description, color and gradient space energy definition, e_smoothIt is defined as follows:

For texture space energy termIn color space energy termWith gradient space energy termCalculating in considered, therefore can also select not consider according to actual conditions.

Step 4, smooth item energy weight setting.In order to avoid splicing line occurs near overlapping region edge zone, protect Card splicing line finds the reasonability of result, and the closer to edge, then weight is bigger, less the closer to zone line weight, single pixel Color and gradient space energy redefine as follows:

Wherein, w_dP () is the weight near overlapping region edge for the pixel p, it is defined as follows:

$w_d\left(p\right)=\left\{\begin{array}{cc}1& \mathrm{ifd}\left(p\right)>d_{\mathrm{th}}\\ \frac{d_{\mathrm{th}}}{d\left(p\right)}& \mathrm{otherwise}\end{array}\right.$

Wherein, d (p) represent pixel p to overlapping region edge beeline, d_thFor predetermined threshold value (when being embodied as, originally Skilled person voluntarily can according to circumstances preset value, and in the present embodiment, weights are set to d_th=10).

Step 5, cuts (graph-cuts) energy optimizing method using figure, redefines color space energy to through step 4 ?And gradient space energy termGross energy afterwardsIt is optimized, obtain optimum splicing line. This energy-optimised mode is divided into α-expansion and alpha-beta-swap, and the present embodiment adopts alpha-beta-swap to gross energyCarry out Optimize, be implemented as prior art.In order to ensure the optimality of optimum results, iterations of the present invention is set to 10 times.Repeatedly During generation, when gross energy is minimum, the splicing effect obtaining is optimal.Accompanying drawing 2 is to cut (graph-cuts) based on figure , there is overlapping region in the simple examples that splicing line is found, between pixel, the thickness of line represents and puts down between image a, image b The size of sliding item energy, more thick corresponding energy is bigger, and " cut " is final optimum results, i.e. optimal splicing line.

Do not reach user in view of being likely to occur local optimum effect after being optimized according to step 5 gained optimum splicing line Demand, for local optimum effect on driving birds is not good region, present invention further propose that can in local optimum effect on driving birds is not good region each Overlapping region is passed through to choose point set and give its image ownership numbering, thus bypassing obvious atural object, to reach optimal splicing effect. Embodiment specifically comprises the following steps that

Step a1, chooses point set in image overlap area, can draw straight line, curve, rectangle by user when being embodied as Frame etc. is choosing point set, and gives its image ownership numbering, if the set of the m point chosen is defined asIt gives Image ownership numbering be expressed as

Step a2, the point set chosen for step a1Its corresponding smoothed energy item is without modification, corresponding Data item be amended as follows:

Wherein, x_iThe pixel chosen for step a1,It is given to pixel x for step a1_iImage ownership numbering. Again (graph-cuts) energy optimizing method is cut using figure after this modification energy term and obtain optimum splicing line, i.e. step 4 institute Obtain gross energyMiddle e_data(i_k) make intoRe-execute step 5 afterwards and obtain new optimum splicing line.

Accompanying drawing 3 is that two degree of superimposed images add the splicing line after intervening to find schematic diagram, that is, have image a (video number For 1) and image b (video number is 2), their overlapping region is o (in figure dash area represents), the genus chosen in o M in image a_aPutting the point set constituting is(the short solid line of in figure black represents), the image ownership that it gives Numbering is expressed asThe m belonging to image b choosing_bPutting the point set constituting is(in figure Represented with black rectangle frame), the image ownership numbering that it gives is expressed asM (the m=m then choosing_a +m_b) collection of individual point is combined into υ=υ₁∪υ₂, for the point set corresponding smoothed energy item chosen without modification, corresponding data Re-execute step 5 after item modification, obtain increasing the splicing line (in figure is indicated by the solid line) after intervening, compared to original spelling Wiring (in figure is represented by dashed line), splicing line makes selection point set be belonging respectively to image a and b, thus bypassing obvious atural object, can reach To optimal splicing effect.Accompanying drawing 4 is same principle, has image a, image b and image c, therefrom chooses point set respectively and is done Pre-optimized.In addition can be extended to superimposed images of spending adds the spelling line that connects intervened to find more.

When being embodied as, also can be realized using corresponding system, a kind of of present invention offer cuts energy-optimised shadow based on figure Optimum splicing line searching system between picture, including with lower module:

Data preparation module, is used for carrying out image data preparation to be spliced, obtains all overlaps between each image to be spliced Region and corresponding degree of overlapping；

Pretreatment module, for pre-processing to image to be spliced；

Energy function determining module, for determining that figure cuts global energy function；

Weight setting module, for smoothing the setting of item energy weight；

Optimization module, for cutting energy optimizing method using figure, to the gross energy after redefining through weight setting module It is optimized, obtain optimum splicing line.

Each module implements can participate in corresponding steps and illustrate.

Instantiation described herein is only explanation for example to present invention spirit.The technical field of the invention Technical staff described instantiation can be made various modification supplement or using similar mode substitute, but Spirit without departing from the present invention or surmount scope defined in appended claims.

Claims (10)

1. a kind of optimum splicing line finding method is cut between energy-optimised image based on figure, comprise the following steps:

Step 1, carries out image data preparation to be spliced, obtains between each image to be spliced all overlapping regions and accordingly overlapping Degree；

Step 2, pre-processes to image to be spliced；

Step 3, determines that figure cuts global energy function as follows,

If currently image to be spliced is opened for n, respectively image i₁,i₂,i₃,…,i_n, the image set of composition isFigure Cut global energy function and include e_dataAnd e_smoothTwo,

Wherein, e_dataAnd e_smoothRepresent data capacity item and smooth energy term respectively, it is specifically calculated as follows,

Described data capacity itemBe calculated as follows,

Wherein, e_data(i_k) represent come from image i through each pixel in optimum splicing line spliced image j_kEnergy punish Penalize, be defined as follows,

e_data(1_k)=∑_p∈jr_p(i_k)

Wherein, p is certain pixel after splicing on image j, r_p(i_k) represent pixel p and come from image i_kEnergy punishment, definition It is as follows,

$r_p\left(i_k\right)=\left\{\begin{array}{cc}0& p\&element;i_k\\ \mathrm{\∞}& p\&notelement;i_k\end{array}\right.$

Described smoothed energy itemIt is defined as follows,

Wherein, color space energy termAnd gradient space energy termComputing formula as follows,

Wherein, p and q is two adjacent pixels, For four neighborhoods of pixel p,WithGeneration respectively The numbering of pixel p and the affiliated image of q, function in image j after table splicingIt is defined as follows,

WithRepresent neighbor p and q respectively to put down in the corresponding color space in overlapping region Sliding item energy and gradient space smooth item energy, are defined as follows,

Wherein, i_aAnd i_bFor image setThere is the image of overlapping region in middle any two, if image i_a,i_bComposition is treated Splicing image pairWithIt is respectively neighbor p and q in i_aAnd i_b The corresponding color space in overlapping region smooths item energy and gradient space smooths item energy；

Step 4, smooth item energy weight setting, including based on step 3 acquired results, by the color space energy term of single pixelAnd gradient space energy termRedefine as follows,

Wherein, w_dP (), is defined as follows near the weight at overlapping region edge for pixel p:,

$w_d\left(p\right)=\left\{\begin{array}{cc}1& ifd\left(p\right)>d_{th}\\ \frac{d_{th}}{d\left(p\right)}& otherwise\end{array}\right.$

Wherein, d (p) represent pixel p to overlapping region edge beeline, d_thFor predetermined threshold value；

Step 5, cuts energy optimizing method using figure, redefines color space energy term to through step 4And gradient Dimensional energy itemGross energy afterwardsIt is optimized, obtain optimum splicing line.

2. according to claim 1 optimum splicing line finding method is cut between energy-optimised image based on figure it is characterised in that: When local optimum effect on driving birds is not good occurring after being optimized according to step 5 gained optimum splicing line, not good to local effect of optimization Each overlapping region in region is separately added into intervention, optimizes splicing line, and step is as follows,

Step a1, chooses point set in overlapping region, and gives corresponding image ownership numbering, if the set of the m point chosen is fixed Justice isThe corresponding image ownership numbering giving is expressed as

Step a2, the point set chosen for step a1Corresponding smoothed energy item without modification, corresponding data The energy punishment of energy term is amended as follows,

Wherein, x_iThe pixel chosen for step a1,It is given to pixel x for step a1_iImage ownership numbering, through this Again energy optimizing method is cut using figure after modification energy term and obtain optimum splicing line.

3. according to claim 1 or claim 2 optimum splicing line finding method is cut between energy-optimised image based on figure, its feature exists In: described neighbor p and q is in i_aAnd i_bThe corresponding color space in overlapping region smooths item energy and gradient space smooths item energy AmountWithAcquisition modes as follows,

Carry out self imaging pair for each pixel p in image j after splicingTexture space energy It is defined as follows,

Wherein, γ_a(p) and γ_bP () represents pixel p respectively in image i_aAnd i_bOn Texture complication；

Carry out self imaging pair for each pixel p in image j after splicingColor space energyFixed Justice is as follows,

Wherein, h (p) and s (p) represents value in image h passage and s passage for the pixel p, w respectively_hAnd w_sIt is respectively corresponding power；

Carry out self imaging pair for each pixel p in image j after splicingGradient space energy definition as follows,

Wherein, λ is balance parameters,WithIt is respectively pixel p in image i_aOn level side To gradient magnitude, in image i_aOn vertical gradient size, in image i_bOn horizontal direction gradient magnitude and in image i_b On vertical gradient size；

Color space smooths item energy and gradient space smooths item energyWithDefinition is such as Under,

4. according to claim 1 or claim 2 optimum splicing line finding method is cut between energy-optimised image based on figure, its feature exists In: described in step 2, image to be spliced is pre-processed, including image is down-sampled, the conversion of image color space and gray processing, shadow As gradient calculation and Texture complication calculate.

5. according to claim 4 optimum splicing line finding method is cut between energy-optimised image based on figure it is characterised in that: When image to be spliced is panoramic picture, described in step 2, image to be spliced is pre-processed and include carrying out image extension.

6. a kind of optimum splicing line searching system is cut between energy-optimised image based on figure, including with lower module:

Data preparation module, is used for carrying out image data preparation to be spliced, obtains all overlapping regions between each image to be spliced And corresponding degree of overlapping；

Pretreatment module, for pre-processing to image to be spliced；

Energy function determining module, as follows for determining that figure cuts global energy function,

If currently image to be spliced is opened for n, respectively image i₁,i₂,i₃,…i_n, the image set of composition isFigure cuts Global energy function includes e_dataAnd e_smoothTwo,

Wherein, e_dataAnd e_smoothRepresent data capacity item and smooth energy term respectively, it is specifically calculated as follows,

Described data capacity itemBe calculated as follows,

Wherein, e_data(i_k) represent come from image i through each pixel in optimum splicing line spliced image j_kEnergy punish Penalize, be defined as follows,

$e_{data}\left(i_k\right)={\σ}_{p\&element;j}{r}_p\left(i_k\right)$

Wherein, p is certain pixel after splicing on image j, r_p(i_k) represent pixel p and come from image i_kEnergy punishment, definition It is as follows,

$r_p\left(i_k\right)=\left\{\begin{array}{cc}0& p\&element;i_k\\ \mathrm{\∞}& p\&notelement;i_k\end{array}\right.$

Described smoothed energy itemIt is defined as follows:

Wherein, color space energy termAnd gradient space energy termComputing formula as follows,

Wherein, p and q is two adjacent pixels, For four neighborhoods of pixel p,WithGeneration respectively The numbering of pixel p and the affiliated image of q, function in image j after table splicingIt is defined as follows,

WithRepresent neighbor p and q respectively to put down in the corresponding color space in overlapping region Sliding item energy and gradient space smooth item energy, are defined as follows,

Wherein, i_aAnd i_bFor image setThere is the image of overlapping region in middle any two, if image i_a,i_bComposition is treated Splicing image pairWithIt is respectively neighbor p and q in i_aAnd i_b The corresponding color space in overlapping region smooths item energy and gradient space smooths item energy；

Weight setting module, for smoothing the setting of item energy weight, including based on energy function determining module acquired results, by list The color space energy term of individual pixelAnd gradient space energy termRedefine as follows,

Wherein, w_dP (), is defined as follows near the weight at overlapping region edge for pixel p,

$w_d\left(p\right)=\left\{\begin{array}{cc}1& ifd\left(p\right)>d_{th}\\ \frac{d_{th}}{d\left(p\right)}& otherwise\end{array}\right.$

Wherein, d (p) represent pixel p to overlapping region edge beeline, d_thFor predetermined threshold value；

Optimization module, for cutting energy optimizing method using figure, redefines color space energy term to through weight setting moduleAnd gradient space energy termGross energy afterwardsIt is optimized, obtain optimum splicing line.

7. according to claim 6 optimum splicing line searching system is cut between energy-optimised image based on figure it is characterised in that: , for local optimum effect on driving birds is not good after being optimized according to optimization module gained optimum splicing line in setting intervention module When, intervention is separately added into each overlapping region in the not good region of local effect of optimization, optimizes splicing line, step is as follows,

Step a1, chooses point set in overlapping region, and gives corresponding image ownership numbering, if the set of the m point chosen is fixed Justice isThe corresponding image ownership numbering giving is expressed as

Step a2, the point set chosen for step a1Corresponding smoothed energy item without modification, corresponding data The energy punishment of energy term is amended as follows,

Wherein, x_iThe pixel chosen for step a1,It is given to pixel x for step a1_iImage ownership numbering, through this Again energy optimizing method is cut using figure after modification energy term and obtain optimum splicing line.

8. according to claim 6 or 7, optimum splicing line searching system between energy-optimised image is cut based on figure, its feature exists In: described neighbor p and q is in i_aAnd i_bThe corresponding color space in overlapping region smooths item energy and gradient space smooths item energy AmountWithAcquisition modes as follows,

Carry out self imaging pair for each pixel p in image j after splicingTexture space energy It is defined as follows,

Wherein, γ_a(p) and γ_bP () represents pixel p respectively in image i_aAnd i_bOn Texture complication；

Carry out self imaging pair for each pixel p in image j after splicingColor space energyFixed Justice is as follows,

Wherein, h (p) and s (p) represents value in image h passage and s passage for the pixel p, w respectively_hAnd w_sIt is respectively corresponding power；

Carry out self imaging pair for each pixel p in image j after splicingGradient space energy definition as follows,

Wherein, λ is balance parameters,WithIt is respectively pixel p in image i_aOn level side To gradient magnitude, in image i_aOn vertical gradient size, in image i_bOn horizontal direction gradient magnitude and in image i_b On vertical gradient size；

Color space smooths item energy and gradient space smooths item energyWithDefinition is such as Under,

9. according to claim 6 or 7, optimum splicing line searching system between energy-optimised image is cut based on figure, its feature exists In the pretreatment that pretreatment module is carried out to image to be spliced, including image is down-sampled, the conversion of image color space and gray scale Change, image gradient calculates and Texture complication calculates.

10. according to claim 9 optimum splicing line searching system between energy-optimised image is cut based on figure, its feature exists In: when image to be spliced is panoramic picture, the pretreatment that pretreatment module image to be spliced is carried out includes carrying out image expansion Exhibition.