CN107067370A - A kind of image split-joint method based on distortion of the mesh - Google Patents
A kind of image split-joint method based on distortion of the mesh Download PDFInfo
- Publication number
- CN107067370A CN107067370A CN201710236731.XA CN201710236731A CN107067370A CN 107067370 A CN107067370 A CN 107067370A CN 201710236731 A CN201710236731 A CN 201710236731A CN 107067370 A CN107067370 A CN 107067370A
- Authority
- CN
- China
- Prior art keywords
- image
- mesh
- point
- msub
- distortion
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000000034 method Methods 0.000 title claims abstract description 31
- 230000009466 transformation Effects 0.000 claims abstract description 40
- 239000011159 matrix material Substances 0.000 claims abstract description 24
- 238000005457 optimization Methods 0.000 claims abstract description 7
- 239000000284 extract Substances 0.000 claims abstract description 5
- 238000006243 chemical reaction Methods 0.000 claims description 11
- PXFBZOLANLWPMH-UHFFFAOYSA-N 16-Epiaffinine Natural products C1C(C2=CC=CC=C2N2)=C2C(=O)CC2C(=CC)CN(C)C1C2CO PXFBZOLANLWPMH-UHFFFAOYSA-N 0.000 claims description 10
- 239000013598 vector Substances 0.000 claims description 9
- 230000008859 change Effects 0.000 claims description 8
- 238000009499 grossing Methods 0.000 claims description 6
- 230000003044 adaptive effect Effects 0.000 claims description 5
- 238000005259 measurement Methods 0.000 claims description 4
- 238000012216 screening Methods 0.000 claims description 2
- 238000005516 engineering process Methods 0.000 abstract description 5
- 238000010276 construction Methods 0.000 abstract description 4
- RZVAJINKPMORJF-UHFFFAOYSA-N Acetaminophen Chemical compound CC(=O)NC1=CC=C(O)C=C1 RZVAJINKPMORJF-UHFFFAOYSA-N 0.000 abstract 1
- 208000029618 autoimmune pulmonary alveolar proteinosis Diseases 0.000 abstract 1
- 230000006870 function Effects 0.000 description 16
- 238000010586 diagram Methods 0.000 description 2
- 230000008901 benefit Effects 0.000 description 1
- 238000000605 extraction Methods 0.000 description 1
- 210000003746 feather Anatomy 0.000 description 1
- 230000004927 fusion Effects 0.000 description 1
- 238000002156 mixing Methods 0.000 description 1
- 238000007500 overflow downdraw method Methods 0.000 description 1
- 230000008447 perception Effects 0.000 description 1
- 238000007781 pre-processing Methods 0.000 description 1
- 238000002360 preparation method Methods 0.000 description 1
- 230000008569 process Effects 0.000 description 1
- 230000020509 sex determination Effects 0.000 description 1
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T3/00—Geometric image transformations in the plane of the image
- G06T3/40—Scaling of whole images or parts thereof, e.g. expanding or contracting
- G06T3/4038—Image mosaicing, e.g. composing plane images from plane sub-images
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T3/00—Geometric image transformations in the plane of the image
- G06T3/14—Transformations for image registration, e.g. adjusting or mapping for alignment of images
- G06T3/147—Transformations for image registration, e.g. adjusting or mapping for alignment of images using affine transformations
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T3/00—Geometric image transformations in the plane of the image
- G06T3/18—Image warping, e.g. rearranging pixels individually
Landscapes
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Image Processing (AREA)
Abstract
The invention discloses a kind of image split-joint method based on distortion of the mesh, it is related to image mosaic technology field, comprises the following steps：S1：Two images adjacent to space, extract SIFT feature, matching characteristic respectively；S2：Calculated using RANSAC algorithm iterations, screen characteristic point pair；S3：The global projection matrix of estimation simultaneously carries out global projective transformation；S4：With Delaunay Triangulation, to the dense triangle gridding of overlapping regional structure, Non-overlapping Domain construction sparse grid；S5：Set up distortion of the mesh energy function, the global projection registration of optimization；S6：Image co-registration generates wide field-of-view image.The present invention characterizes overlapping region using dense triangle gridding, and the Non-overlapping Domain of image is represented using sparse triangle gridding, so relative APAP methods entire image uses intensive equal matrix grid, substantially increases efficiency of algorithm.Distortion of the mesh simultaneously can make full use of GPU parallel, high-speed computation performance, can greatly improve splicing speed.
Description
Technical field
The invention belongs to technical field of computer vision, it is related to image mosaic technology field, more particularly to it is a kind of based on net
Lattice optimize the real-time joining method of image.
Background technology
Image mosaic is the image using the different angles of multiple video cameras acquisition and progress is spliced to form wide field-of-view image
Technology, image mosaic is broadly divided into three key steps：Image preprocessing, figure registration, image co-registration.
In order to reduce splicing ghost image, fuzzy problem caused by parallax in splicing, current existing related algorithm can be concluded
It is to become in the registering stage using the replacement of local parameter transformation model is global for Spatially-Varying Warping methods
Scene of the mold changing type to differ suitable for depth size.
Zaragoza et al. proposes As-Projective-As-Possible (APAP) method, divides an image into multiple
Intensive rectangular mesh, to each rectangular mesh, using the characteristic point pair of matching, the local projective transform matrix of estimation, and introduce
The distance weighting factor, characteristic point weight closely is big, and remote characteristic point weight is small, adjusts distance weighting factor adaptive
Whole projection model parameter, to adapt to the scene of different depth size.APAP methods introduce the concept of grid, and registration accuracy is obtained
Large increase, but shape keeps not good enough, and also computation complexity is higher, and real-time is bad.
Shape-Preserving-Half-Projective (SPHP) methods are from the angle of shape correction, by image
Three regions are divided into, from overlapping region to Non-overlapping Domain, similarity transformation is gradually transitioned into by projective transformation, shape is reduced
Distortion, but registration accuracy is not high, there is obvious ghost image overlapping region and fuzzy.
Natural Image Stitching with the Global Similarity Prior methods, using straight line
Alignment constraint determines the angle Selection of global similar matrix, and grid is initialized with APAP methods, while using local similar constraint
Constraint similar with the overall situation, many figure splicing performances and the lifting of perception naturalness.
The content of the invention
The present invention is on the premise of high registration accuracy and shape holding, in order to improve the real-time of splicing, it is proposed that a kind of
Realtime graphic joining method based on distortion of the mesh.
A kind of image split-joint method based on distortion of the mesh, comprises the following steps：
S1：Two images adjacent to space, extract SIFT feature, matching characteristic respectively；
S2：Calculated using RANSAC algorithm iterations, screen characteristic point pair；
S3：The global projection matrix of estimation simultaneously carries out global projective transformation；
S4：With Delaunay Triangulation, to the dense triangle gridding of overlapping regional structure, Non-overlapping Domain construction is sparse
Grid；
The triangulation is specifically defined：If v is the summit of triangle gridding, V is all grid vertexes on the plane of delineation
Set, e is the side of triangle gridding, and E is e set, and t is some triangular mesh, and T is the set of triangular mesh, then
A point set V triangulation T=(V, E) is a plan, and the plan need to meet following condition：1st, except end points,
Side in plan does not include any point that point is concentrated；2nd, without intersection edges；3rd, face all in plan is all triangular facet, and
The intersection of all triangular facets is scatterplot collection V convex closure；
Delaunay sides are specifically defined：Assuming that a line e (two end points are a, b) in E, if e meets following condition,
Then it is referred to as Delaunay sides：Pass through in the presence of a circle in a, 2 points of b, circle and be free of any other point, this characteristic in point set V
Also known as empty circle characteristic；
The Delaunay Triangulation is specifically defined：If a point set V triangulation only includes Delaunay
Side, then the triangulation is referred to as Delaunay Triangulation；
Adaptive triangle gridding is used dilute in grid of the overlapping region of image using encryption in image Non-overlapping Domain
Thin grid, sets dense meshes length of side spacing as lmin, sparse grid length of side spacing is lmax, lmax=3lmin；To be obtained after screening
The SIFT feature and sharp point arrived is prioritizing selection, with lminThe summit of overlapping region triangle gridding is gathered for radius, with
lmaxRadius gathers the summit of Non-overlapping Domain triangle gridding；After grid vertex is obtained, constructed using Delaunay Triangulation
Triangle gridding；
S5：Set up distortion of the mesh energy function, the global projection registration of optimization；
S6：Image co-registration generates wide field-of-view image.
Further, the S1 steps are specially：First to input picture to (I, I '), SIFT (Scale are respectively adopted
Invariant Feature Transform) algorithm extraction SIFT feature, obtain SIFT feature point set F={ X=[xi yi]T|i
=1,2 ..., m }, F '={ X '=[x 'i y′i]T| i=1,2 ..., n }, X and X ' they are spy of the image to (I, I ') overlapping region
Levy a little pair, 128 dimensional feature vectors are generated to each SIFT feature, two width figures are used as using the Euclidean distance of key point characteristic vector
The similarity determination measurement of key point as in, obtains SIFT feature to collection F 'match。
Further, the S2 steps are specially：RANSAC algorithms are by being chosen over characteristic point to collection F 'matchOne group
Random subset, the subset being selected is assumed to be intra-office point, and is verified with following methods：
S201：Selection projection model is adapted to the intra-office point of hypothesis, and subset estimation is penetrated using randomly selected characteristic point
The parameter of shadow model；
S202：With all further feature points of the model measurement obtained in S201, if some point is suitable for the mould of estimation
Type, then it is assumed that it is also intra-office point；If enough points are classified as the intra-office point of hypothesis, the model just conjunction enough of estimation
Reason；
S203：Go to reevaluate model with the intra-office point of all hypothesis, because it is only estimated by initial hypothesis intra-office point
Counted；
S204：Finally, by estimating the error rate of intra-office point and model come assessment models；
Above procedure is repeatedly executed, and is finally selected all intra-office points and is designated as Fmatch。
Further, it is characterised in that the S5 steps are specially：Estimate target gridding summit so that energy function is most
Smallization, local similarity transformation model is estimated using former target gridding summit and target gridding summit, and for optimizing global projection
Matrix parameter；
Set the apex coordinate of original image grid asImage after the conversion to be calculated
Grid vertex isSpecifically include following steps：
S501：Tectonic scale scaled error
Change of scale error term builds as follows：
Wherein, AtIt is triangle gridding tpArea, | | | |FRepresent Frobenius norms, Jt(q) it is one 2 × 2
Jacobian matrixes, triangle tpTransform to tqGeneration affine transformation, Jt(q) be affine transformation rotation and scaling part；
Minimize scaling error EsSo that the linear segment of affine transformation is as close possible to change of scale Gt；
S502：Construct smoothing error
Introduce smoothing error：
Wherein Att'=(At+At′)/2；
S503：The similar item of structure form
Assuming that three continuity points on original meshThe relational expression on two sides can be expressed asWherein riIt is sideWithLength ratio,
It is angle, θiSpin matrix, θiIt is the angle on two sides；
Length is introduced than the angle with side to characterize the similar item of shape
S504：Build energy function and solve, obtain target gridding summit
Combining step S501, S502 and S503, introduce weight factor ε and μ, obtain energy function as follows：
E=Es+εEm+μEf
Above distortion of the mesh energy function is on MqSummit and triangle scale factor quadratic equation；Grid becomes
The minimum of shape energy function can be realized by solving sparse vectors, be minimized by energy function, can be obtained
To the scale factor of target gridding apex coordinate He each triangle gridding；
S505：Estimate local rigid transformation model parameter
The target gridding apex coordinate obtained according to S504 steps, with reference to former mesh vertex coordinates, calculates each three respectively
The rigid transformation matrix S of angle grid；
S506：The global projection model parameter of optimization
Using local rigid transformation matrix, make further conversion to the image after global projective transformation.
The present invention characterizes overlapping region using dense triangle gridding, and the non-heavy of image is represented using sparse triangle gridding
Folded region, so relative APAP methods entire image uses intensive equal matrix grid, substantially increases efficiency of algorithm.Together
When distortion of the mesh can make full use of GPU parallel, high-speed computation performance, splicing speed can be greatly improved.The present invention will figure
As being divided into multiple triangle griddings, the local rigid transformation parameter of each grid is individually calculated, spelling caused by parallax can be reduced
Connect ghost image and fuzzy；The continuity item similar with shape of triangle gridding deformation is considered, so the target image after conversion compares
Naturally, the shape facility of image can preferably be kept.
The additional aspect and advantage of the present invention will be set forth in part in the description, and will partly become from the following description
Obtain substantially, or recognized by the practice of the present invention.
Brief description of the drawings
In order to illustrate more clearly about the embodiment of the present invention or technical scheme of the prior art, below will be to embodiment or existing
There is the accompanying drawing used required in technology description to be briefly described, it should be apparent that, drawings in the following description are only this
Some embodiments of invention, for those of ordinary skill in the art, without having to pay creative labor, may be used also
To obtain other accompanying drawings according to these accompanying drawings.
Fig. 1 is the system flow chart of the real-time joining method of panoramic picture of the embodiment of the present invention；
Fig. 2 is the schematic diagram of intermediate cam grid of the embodiment of the present invention；
Fig. 3 is the flow chart of distortion of the mesh algorithm in the embodiment of the present invention；
Fig. 4 is geometrical relationship schematic diagram of the image after converting twice in the embodiment of the present invention.
Embodiment
Below in conjunction with the accompanying drawing in the embodiment of the present invention, the technical scheme in the embodiment of the present invention is carried out clear, complete
Site preparation is described, it is clear that described embodiment is only a part of embodiment of the invention, rather than whole embodiments.It is based on
Embodiment in the present invention, it is every other that those of ordinary skill in the art are obtained under the premise of creative work is not made
Embodiment, belongs to the scope of protection of the invention.
The invention provides a kind of real-time joining method of the image based on distortion of the mesh, represent the image as special based on image
The adaptive triangle gridding levied, sets up distortion of the mesh energy function, and adds bound term, is thus converted to image conversion in phase
Close the distortion of the mesh problem under constraint.Fig. 1 is the system flow chart of the real-time joining method of panoramic picture of the embodiment of the present invention.As schemed
Shown in 1, a kind of realtime graphic joining method based on distortion of the mesh of the present invention comprises the following steps：
S1：Two images adjacent to space, extract SIFT feature, matching characteristic respectively
First to input picture to (I, I '), SIFT (Scale Invariant Feature are respectively adopted
Transform) algorithm extracts SIFT feature, obtains SIFT feature point set F={ X=[xi yi]T| i=1,2 ..., m }, F '=
{ X '=[x 'i y′i]T| i=1,2 ..., n }, X and X ' they are characteristic point pair of the image to (I, I ') overlapping region.To each SIFT
Feature generates 128 dimensional feature vectors.Using key point characteristic vector Euclidean distance as in two images key point it is similar
Sex determination is measured, and obtains SIFT feature to collection F 'match。
S2：Calculated using RANSAC algorithm iterations, screen characteristic point pair
RANSAC is by being chosen over characteristic point to collection F 'matchOne group of random subset, the subset being selected is assumed to be
Intra-office point, and verified with following methods：
S201：Selection projection model is adapted to the intra-office point of hypothesis, and subset estimation is penetrated using randomly selected characteristic point
The parameter of shadow model.
S202：With all further feature points of the model measurement obtained in S201, if some point is suitable for the mould of estimation
Type, then it is assumed that it is also intra-office point.If enough points are classified as the intra-office point of hypothesis, the model just conjunction enough of estimation
Reason.
S203：Go to reevaluate model with the intra-office point of all hypothesis, because it is only estimated by initial hypothesis intra-office point
Counted.
S204：Finally, by estimating the error rate of intra-office point and model come assessment models.
Above procedure is repeatedly executed, and is finally selected all intra-office points and is designated as Fmatch。
S3：The global projection matrix of estimation simultaneously carries out global projective transformation
Using all intra-office points selected in S2 steps, for reevaluating projection model parameter, then to original image
Projective transformation is carried out, if using left image as reference picture, right image carries out projective transformation according to H.
S4：With Delaunay Triangulation, to the dense triangle gridding of overlapping regional structure, Non-overlapping Domain construction is sparse
Grid
The definition of triangulation：If v is the summit of triangle gridding, V is the set of all grid vertexes on the plane of delineation, e
It is the side of triangle gridding, E is e set, and t is some triangular mesh, and T is the set of triangular mesh, then point set V's
One triangulation T=(V, E) is a plan, and the plan meets condition：1) except end points, the side in plan is not wrapped
Containing an any point concentrated.2) without intersection edges.3) face all in plan is all triangular facet, and the intersection of all triangular facets
It is scatterplot collection V convex closure.
In practice with most triangulations be Delaunay Triangulation, it is that a kind of special triangle is cutd open
Point.First talked about from Delaunay sides：
Delaunay sides：Assuming that a line e (two end points are a, b) in E, if e meets following condition, is referred to as
Delaunay sides：Pass through in the presence of a circle in a, 2 points of b, circle and be free of any other point in point set V, this characteristic is also known as empty circle
Characteristic.
Delaunay Triangulation：If a point set V triangulation only includes Delaunay sides, then the triangle is cutd open
Divide and be referred to as Delaunay Triangulation.
Adaptive triangle gridding is used dilute in grid of the overlapping region of image using encryption in image Non-overlapping Domain
Thin grid, sets dense meshes length of side spacing as lmin, sparse grid length of side spacing is lmax, usual lmax=3lmin.To screen
The SIFT feature and sharp point obtained afterwards is prioritizing selection, with lminThe top of overlapping region triangle gridding is gathered for radius
Point, with lmaxRadius gathers the summit of Non-overlapping Domain triangle gridding.After grid vertex is obtained, cutd open using Delaunay triangles
Divide construction triangle gridding.As shown in Fig. 2 (a) is original image, (b) is the grid image of original image.If the shell of image is image
SIFT feature distributed areas, then the shell region builds intensive triangle gridding, and remaining region builds sparse triangle
Grid.
Overlapping region is characterized using dense triangle gridding in the step, the non-of image is represented using sparse triangle gridding
Overlapping region, so relative APAP methods entire image uses intensive equal matrix grid, substantially increases efficiency of algorithm.
Distortion of the mesh simultaneously can make full use of GPU parallel, high-speed computation performance, can greatly improve splicing speed.
S5：Set up distortion of the mesh energy function, the global projection registration of optimization
Rigid transformation only occurs for feature triangle gridding, and adjacent mesh deformation is as smooth as possible, thus sets up distortion of the mesh energy
Flow function.Estimate target gridding summit so that energy function is minimized, and is estimated using former target gridding summit and target gridding summit
The local similarity transformation model of meter, and for optimizing global projection matrix parameter.
Set the apex coordinate of original image grid asImage after the conversion to be calculated
Grid vertex isThe flow chart of the distortion of the mesh algorithm of the present invention is as shown in Figure 3：
S501：Tectonic scale scaled error
Context of methods wishes feature triangle gridding does not rotate during distortion of the mesh, only occurs change of scaleAnd other triangle griddings are allowed with occur affine transformation.Change of scale error term builds as follows：
Wherein, AtIt is triangle gridding tpArea, | | | |FRepresent Frobenius norms, Jt(q) it is one 2 × 2
Jacobian matrixes, triangle tpTransform to tqGeneration affine transformation, Jt(q) be affine transformation rotation and scaling part.
Minimize scaling error EsSo that the linear segment of affine transformation owns as close possible to change of scale Gt
Triangle tends to equably scale with not rotation mode.
S502：Construct smoothing error
In order to obtain more natural target image, the continuity of target image is kept, it is desirable to which adjacent mesh is in deformation process
The change of scale of middle generation should be similar as much as possible.Introduce smoothing error：
Wherein Att'=(At+At′)/2。
S503：The similar item of structure form
Assuming that three continuity points on original meshThe relational expression on two sides can be expressed asWherein riIt is sideWithLength ratio,
It is angle, θiSpin matrix, θiIt is the angle on two sides.
In order to keep the shape similarity of triangle gridding, it is more similar to characterize shape than the angle with side that we introduce length
：
S504：Build energy function and solve, obtain target gridding summit
Combining step S501~S503, introduces weight factor ε and μ, obtains energy function as follows：
E=Es+εEm+μEf
Above distortion of the mesh energy function is on MqSummit and triangle scale factor quadratic equation.Grid becomes
The minimum of shape energy function can be realized by solving sparse vectors, be minimized by energy function, can be obtained
To the scale factor of target gridding apex coordinate He each triangle gridding.
S505：Estimate local rigid transformation model parameter
The target gridding apex coordinate obtained according to S504 steps, with reference to former mesh vertex coordinates, calculates each three respectively
The rigid transformation matrix S of angle grid.
S506：The global projection model parameter of optimization
Using local rigid transformation matrix, make further conversion to the image after global projective transformation.In order to preferably
Show the relation of image conversion twice, geometrical relationship of the image after converting twice such as Fig. 4 is represented.Original image passes through H first
Global projective transformation obtains I, then carries out quadratic transformation WH to image-1, wherein W is the rigid transformation square of local triangle grid
Battle array, H-1For the inverse matrix that H is global projection matrix.
In S5 steps, multiple triangle griddings are divided an image into, the local rigid transformation parameter of each grid is individually calculated,
Splicing ghost image caused by parallax can be reduced and fuzzy；The continuity item similar with shape of triangle gridding deformation is considered, so
Target image after conversion compares naturally, the shape facility of image can preferably be kept.
S6：Image co-registration generates wide field-of-view image
Original image after converting twice, using Feather blending fusion methods, to the left images after conversion
Merged, the last panoramic picture of fusion generation.
Above disclosed is only a kind of preferred embodiment of the invention, can not limit the power of the present invention with this certainly
Sharp scope, therefore the equivalent variations made according to the claims in the present invention, still belong to the scope that the present invention is covered.
Claims (4)
1. a kind of image split-joint method based on distortion of the mesh, it is characterised in that comprise the following steps：
S1：Two images adjacent to space, extract SIFT feature, matching characteristic respectively；
S2：Calculated using RANSAC algorithm iterations, screen characteristic point pair；
S3：The global projection matrix of estimation simultaneously carries out global projective transformation；
S4：With Delaunay Triangulation, to the dense triangle gridding of overlapping regional structure, Non-overlapping Domain constructs sparse net
Lattice；
The triangulation is specifically defined：If v is the summit of triangle gridding, V is the collection of all grid vertexes on the plane of delineation
Close, e is the side of triangle gridding, E is e set, and t is some triangular mesh, and T is the set of triangular mesh, then the point
It is a plan to collect a V triangulation T=(V, E), and the plan need to meet following condition：1st, except end points, plane
Side in figure does not include any point that point is concentrated；2nd, without intersection edges；3rd, face all in plan is all triangular facet, and all
The intersection of triangular facet is scatterplot collection V convex closure；
Delaunay sides are specifically defined：Assuming that a line e (two end points are a, b) in E, if e meets following condition, claims
Be Delaunay sides：Pass through in the presence of a circle in a, 2 points of b, circle and be free of any other point in point set V, this characteristic is also known as
Sky circle characteristic；
The Delaunay Triangulation is specifically defined：If a point set V triangulation only includes Delaunay sides, that
The triangulation is referred to as Delaunay Triangulation；
Adaptive triangle gridding is used sparse in grid of the overlapping region of image using encryption in image Non-overlapping Domain
Grid, sets dense meshes length of side spacing as lmin, sparse grid length of side spacing is lmax, lmax=3lmin；With what is obtained after screening
SIFT feature and sharp point are prioritizing selection, with lminThe summit of overlapping region triangle gridding is gathered for radius, with lmax
Radius gathers the summit of Non-overlapping Domain triangle gridding；After grid vertex is obtained, three are constructed using Delaunay Triangulation
Angle grid；
S5：Set up distortion of the mesh energy function, the global projection registration of optimization；
S6：Image co-registration generates wide field-of-view image.
2. the image split-joint method according to claim 1 based on distortion of the mesh, it is characterised in that the S1 steps are specific
For：First to input picture to (I, I '), SIFT (Scale Invariant Feature Transform) algorithm is respectively adopted
SIFT feature is extracted, SIFT feature point set is obtained
F={ X=[xi yi]T| i=1,2 ..., m }, F '=[X '=[x 'i y′i]T| i=1,2 ..., n }, X and X ' they are images
To the characteristic point pair of (I, I ') overlapping region, 128 dimensional feature vectors are generated to each SIFT feature, using key point characteristic vector
Euclidean distance as in two images key point similarity determination measure, obtain SIFT feature to collection F 'match。
3. the image split-joint method according to claim 1 based on distortion of the mesh, it is characterised in that the S2 steps are specific
For：RANSAC algorithms are by being chosen over characteristic point to collection F 'matchOne group of random subset, the subset being selected is assumed to be office
It is interior, and verified with following methods：
S201：Selection projection model is adapted to the intra-office point of hypothesis, and projective module is estimated to subset using randomly selected characteristic point
The parameter of type；
S202：With all further feature points of the model measurement obtained in S201, if some point is suitable for the model of estimation,
It is also intra-office point to think it；If enough points are classified as the intra-office point of hypothesis, the model of estimation is just reasonable enough；
S203：Go to reevaluate model with the intra-office point of all hypothesis, because it is only by initial hypothesis intra-office point estimation；
S204：Finally, by estimating the error rate of intra-office point and model come assessment models；
Above procedure is repeatedly executed, and is finally selected all intra-office points and is designated as Fmatch。
4. the image split-joint method according to claim 1 based on distortion of the mesh, it is characterised in that the S5 steps are specific
For：Estimate target gridding summit so that energy function is minimized, utilize former target gridding summit and target gridding summit estimation office
Portion's similarity transformation model, and for optimizing global projection matrix parameter；
Set the apex coordinate of original image grid asThe grid of image after the conversion to be calculated
Summit isSpecifically include following steps：
S501：Tectonic scale scaled error
Change of scale error term builds as follows：
<mrow>
<msub>
<mi>E</mi>
<mi>s</mi>
</msub>
<mo>=</mo>
<munder>
<mi>&Sigma;</mi>
<mrow>
<mi>t</mi>
<mo>&Element;</mo>
<mi>T</mi>
</mrow>
</munder>
<msub>
<mi>A</mi>
<mi>t</mi>
</msub>
<mo>|</mo>
<mo>|</mo>
<msub>
<mi>J</mi>
<mi>t</mi>
</msub>
<mrow>
<mo>(</mo>
<mi>q</mi>
<mo>)</mo>
</mrow>
<mo>-</mo>
<msub>
<mi>G</mi>
<mi>t</mi>
</msub>
<mo>|</mo>
<msubsup>
<mo>|</mo>
<mi>F</mi>
<mn>2</mn>
</msubsup>
</mrow>
Wherein, AtIt is triangle gridding tpArea, | | | |FRepresent Frobenius norms, Jt(q) Jacobian for one 2 × 2
Matrix, triangle tpTransform to tqGeneration affine transformation, Jt(q) be affine transformation rotation and scaling part；
Minimize scaling error EsSo that the linear segment of affine transformation is as close possible to change of scale Gt；
S502：Construct smoothing error
Introduce smoothing error：
<mrow>
<msub>
<mi>E</mi>
<mi>m</mi>
</msub>
<mo>=</mo>
<munder>
<mi>&Sigma;</mi>
<mrow>
<mi>t</mi>
<mo>,</mo>
<msup>
<mi>t</mi>
<mo>&prime;</mo>
</msup>
<mo>&Element;</mo>
<mi>T</mi>
<mi> </mi>
<mi>a</mi>
<mi>n</mi>
<mi>d</mi>
<mi> </mi>
<mi>t</mi>
<mo>,</mo>
<msup>
<mi>t</mi>
<mo>&prime;</mo>
</msup>
<mi>a</mi>
<mi>r</mi>
<mi>e</mi>
<mi> </mi>
<mi>a</mi>
<mi>d</mi>
<mi>j</mi>
<mi>a</mi>
<mi>c</mi>
<mi>e</mi>
<mi>n</mi>
<mi>t</mi>
</mrow>
</munder>
<msub>
<mi>A</mi>
<mrow>
<msup>
<mi>tt</mi>
<mo>&prime;</mo>
</msup>
</mrow>
</msub>
<mo>|</mo>
<mo>|</mo>
<msub>
<mi>G</mi>
<mi>t</mi>
</msub>
<mo>-</mo>
<msub>
<mi>G</mi>
<msup>
<mi>t</mi>
<mo>&prime;</mo>
</msup>
</msub>
<mo>|</mo>
<msubsup>
<mo>|</mo>
<mi>F</mi>
<mn>2</mn>
</msubsup>
</mrow>
Wherein Att′=(At+At′)/2；
S503：The similar item of structure form
Assuming that three continuity points on original meshThe relational expression on two sides can be expressed asWherein riIt is sideWithLength ratio,
It is angle, θiSpin matrix, θiIt is the angle on two sides；
Length is introduced than the angle with side to characterize the similar item of shape：
S504：Build energy function and solve, obtain target gridding summit
Combining step S501, S502 and S503, introduce weight factor ε and μ, obtain energy function as follows：
E=Es+εEm+μEf
Above distortion of the mesh energy function is on MqSummit and triangle scale factor quadratic equation；Distortion of the mesh energy
The minimum of flow function can be realized by solving sparse vectors, be minimized by energy function, can obtain mesh
Mark mesh vertex coordinates and the scale factor of each triangle gridding；
S505：Estimate local rigid transformation model parameter
The target gridding apex coordinate obtained according to S504 steps, with reference to former mesh vertex coordinates, calculates each triangulation network respectively
The rigid transformation matrix S of lattice；
S506：The global projection model parameter of optimization
Using local rigid transformation matrix, make further conversion to the image after global projective transformation.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201710236731.XA CN107067370A (en) | 2017-04-12 | 2017-04-12 | A kind of image split-joint method based on distortion of the mesh |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201710236731.XA CN107067370A (en) | 2017-04-12 | 2017-04-12 | A kind of image split-joint method based on distortion of the mesh |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN107067370A true CN107067370A (en) | 2017-08-18 |
Family
ID=59603040
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201710236731.XA Pending CN107067370A (en) | 2017-04-12 | 2017-04-12 | A kind of image split-joint method based on distortion of the mesh |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN107067370A (en) |
Cited By (20)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN108307200A (en) * | 2018-01-31 | 2018-07-20 | 深圳积木易搭科技技术有限公司 | A kind of Online Video joining method system |
| CN108470324A (en) * | 2018-03-21 | 2018-08-31 | 深圳市未来媒体技术研究院 | A kind of binocular stereo image joining method of robust |
| CN108765292A (en) * | 2018-05-30 | 2018-11-06 | 中国人民解放军军事科学院国防科技创新研究院 | Image split-joint method based on the fitting of space triangular dough sheet |
| CN108898550A (en) * | 2018-05-30 | 2018-11-27 | 中国人民解放军军事科学院国防科技创新研究院 | Image split-joint method based on the fitting of space triangular dough sheet |
| CN109598674A (en) * | 2017-09-30 | 2019-04-09 | 杭州海康威视数字技术股份有限公司 | A kind of image split-joint method and device |
| CN109816706A (en) * | 2019-02-01 | 2019-05-28 | 辽宁工程技术大学 | A kind of smoothness constraint and triangulation network equal proportion subdivision picture are to dense matching method |
| CN109961398A (en) * | 2019-02-18 | 2019-07-02 | 鲁能新能源(集团)有限公司 | Fan blade image segmentation and grid optimization joining method |
| CN110111375A (en) * | 2019-03-29 | 2019-08-09 | 中国地质大学（武汉） | A kind of Image Matching elimination of rough difference method and device under Delaunay triangulation network constraint |
| CN110211043A (en) * | 2019-05-11 | 2019-09-06 | 复旦大学 | A kind of method for registering based on grid optimization for Panorama Mosaic |
| CN110246161A (en) * | 2019-06-04 | 2019-09-17 | 哈尔滨工程大学 | A kind of method that 360 degree of panoramic pictures are seamless spliced |
| CN110399931A (en) * | 2019-07-30 | 2019-11-01 | 燕山大学 | A kind of fish eye images matching process and system |
| CN111080525A (en) * | 2019-12-19 | 2020-04-28 | 成都海擎科技有限公司 | Distributed image and primitive splicing method based on SIFT (Scale invariant feature transform) features |
| CN111161143A (en) * | 2019-12-16 | 2020-05-15 | 首都医科大学 | Optical positioning technology-assisted operation visual field panoramic stitching method |
| CN111768338A (en) * | 2020-06-29 | 2020-10-13 | 广东小天才科技有限公司 | Method and device for splicing test question images, electronic equipment and storage medium |
| CN112084938A (en) * | 2020-09-08 | 2020-12-15 | 哈尔滨工业大学（深圳） | Method and device for improving representation stability of plane target based on graph structure |
| CN112465704A (en) * | 2020-12-07 | 2021-03-09 | 清华大学深圳国际研究生院 | Global-local adaptive optimization panoramic light field splicing method |
| CN112508773A (en) * | 2020-11-20 | 2021-03-16 | 小米科技(武汉)有限公司 | Image processing method and device, electronic device and storage medium |
| CN112837353A (en) * | 2020-12-29 | 2021-05-25 | 北京市遥感信息研究所 | Heterogeneous image matching method based on multi-order characteristic point-line matching |
| CN112967217A (en) * | 2021-03-11 | 2021-06-15 | 大连理工大学 | Image splicing method based on linear feature matching and constraint |
| CN113658041A (en) * | 2021-07-23 | 2021-11-16 | 华南理工大学 | Image fast splicing method based on multi-image feature joint matching |
Citations (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN103886569A (en) * | 2014-04-03 | 2014-06-25 | 北京航空航天大学 | Parallel and matching precision constrained splicing method for consecutive frames of multi-feature-point unmanned aerial vehicle reconnaissance images |
| CN104915921A (en) * | 2015-05-25 | 2015-09-16 | 浙江大学 | Triangular-mesh-deformation-based geometrical boundary image mapping method with kept content |
-
2017
- 2017-04-12 CN CN201710236731.XA patent/CN107067370A/en active Pending
Patent Citations (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN103886569A (en) * | 2014-04-03 | 2014-06-25 | 北京航空航天大学 | Parallel and matching precision constrained splicing method for consecutive frames of multi-feature-point unmanned aerial vehicle reconnaissance images |
| CN104915921A (en) * | 2015-05-25 | 2015-09-16 | 浙江大学 | Triangular-mesh-deformation-based geometrical boundary image mapping method with kept content |
Non-Patent Citations (2)
| Title |
|---|
| YONG JIN等: "Nonhomogeneous scaling optimization for realtime image resizing", 《THE VISUAL COMPUTER》 * |
| 金勇等: "基于自适应网格变形的图像编辑算法", 《软件学报》 * |
Cited By (33)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN109598674A (en) * | 2017-09-30 | 2019-04-09 | 杭州海康威视数字技术股份有限公司 | A kind of image split-joint method and device |
| CN108307200A (en) * | 2018-01-31 | 2018-07-20 | 深圳积木易搭科技技术有限公司 | A kind of Online Video joining method system |
| CN108307200B (en) * | 2018-01-31 | 2020-06-09 | 深圳积木易搭科技技术有限公司 | Online video splicing method and system |
| CN108470324A (en) * | 2018-03-21 | 2018-08-31 | 深圳市未来媒体技术研究院 | A kind of binocular stereo image joining method of robust |
| CN108470324B (en) * | 2018-03-21 | 2022-02-25 | 深圳市未来媒体技术研究院 | Robust binocular stereo image splicing method |
| CN108898550B (en) * | 2018-05-30 | 2022-05-17 | 中国人民解放军军事科学院国防科技创新研究院 | Image splicing method based on space triangular patch fitting |
| CN108898550A (en) * | 2018-05-30 | 2018-11-27 | 中国人民解放军军事科学院国防科技创新研究院 | Image split-joint method based on the fitting of space triangular dough sheet |
| CN108765292B (en) * | 2018-05-30 | 2022-04-29 | 中国人民解放军军事科学院国防科技创新研究院 | Image splicing method based on space triangular patch fitting |
| CN108765292A (en) * | 2018-05-30 | 2018-11-06 | 中国人民解放军军事科学院国防科技创新研究院 | Image split-joint method based on the fitting of space triangular dough sheet |
| CN109816706A (en) * | 2019-02-01 | 2019-05-28 | 辽宁工程技术大学 | A kind of smoothness constraint and triangulation network equal proportion subdivision picture are to dense matching method |
| CN109961398A (en) * | 2019-02-18 | 2019-07-02 | 鲁能新能源(集团)有限公司 | Fan blade image segmentation and grid optimization joining method |
| CN110111375A (en) * | 2019-03-29 | 2019-08-09 | 中国地质大学（武汉） | A kind of Image Matching elimination of rough difference method and device under Delaunay triangulation network constraint |
| CN110111375B (en) * | 2019-03-29 | 2021-05-04 | 中国地质大学（武汉） | Image matching gross error elimination method and device under Delaunay triangulation network constraint |
| CN110211043A (en) * | 2019-05-11 | 2019-09-06 | 复旦大学 | A kind of method for registering based on grid optimization for Panorama Mosaic |
| CN110211043B (en) * | 2019-05-11 | 2023-06-27 | 复旦大学 | Registration method based on grid optimization for panoramic image stitching |
| CN110246161A (en) * | 2019-06-04 | 2019-09-17 | 哈尔滨工程大学 | A kind of method that 360 degree of panoramic pictures are seamless spliced |
| CN110399931B (en) * | 2019-07-30 | 2021-07-06 | 燕山大学 | Fisheye image matching method and system |
| CN110399931A (en) * | 2019-07-30 | 2019-11-01 | 燕山大学 | A kind of fish eye images matching process and system |
| CN111161143A (en) * | 2019-12-16 | 2020-05-15 | 首都医科大学 | Optical positioning technology-assisted operation visual field panoramic stitching method |
| CN111080525A (en) * | 2019-12-19 | 2020-04-28 | 成都海擎科技有限公司 | Distributed image and primitive splicing method based on SIFT (Scale invariant feature transform) features |
| CN111768338A (en) * | 2020-06-29 | 2020-10-13 | 广东小天才科技有限公司 | Method and device for splicing test question images, electronic equipment and storage medium |
| CN111768338B (en) * | 2020-06-29 | 2024-02-06 | 广东小天才科技有限公司 | Method and device for splicing test question images, electronic equipment and storage medium |
| CN112084938A (en) * | 2020-09-08 | 2020-12-15 | 哈尔滨工业大学（深圳） | Method and device for improving representation stability of plane target based on graph structure |
| CN112084938B (en) * | 2020-09-08 | 2023-07-28 | 哈尔滨工业大学（深圳） | Method and device for improving stability of plane target representation based on graph structure |
| CN112508773B (en) * | 2020-11-20 | 2024-02-09 | 小米科技(武汉)有限公司 | Image processing method and device, electronic equipment and storage medium |
| CN112508773A (en) * | 2020-11-20 | 2021-03-16 | 小米科技(武汉)有限公司 | Image processing method and device, electronic device and storage medium |
| CN112465704A (en) * | 2020-12-07 | 2021-03-09 | 清华大学深圳国际研究生院 | Global-local adaptive optimization panoramic light field splicing method |
| CN112465704B (en) * | 2020-12-07 | 2024-02-06 | 清华大学深圳国际研究生院 | Global-local self-adaptive optimized panoramic light field splicing method |
| CN112837353A (en) * | 2020-12-29 | 2021-05-25 | 北京市遥感信息研究所 | Heterogeneous image matching method based on multi-order characteristic point-line matching |
| CN112967217B (en) * | 2021-03-11 | 2024-02-06 | 大连理工大学 | Image stitching method based on straight line feature matching and constraint |
| CN112967217A (en) * | 2021-03-11 | 2021-06-15 | 大连理工大学 | Image splicing method based on linear feature matching and constraint |
| CN113658041A (en) * | 2021-07-23 | 2021-11-16 | 华南理工大学 | Image fast splicing method based on multi-image feature joint matching |
| CN113658041B (en) * | 2021-07-23 | 2024-04-02 | 华南理工大学 | Image rapid splicing method based on multi-image feature joint matching |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN107067370A (en) | A kind of image split-joint method based on distortion of the mesh | |
| US8830236B2 (en) | Method for estimating a pose of an articulated object model | |
| CN101866482B (en) | Panorama splicing method based on camera self-calibration technology, and device thereof | |
| CN103226830B (en) | The Auto-matching bearing calibration of video texture projection in three-dimensional virtual reality fusion environment | |
| US8743119B2 (en) | Model-based face image super-resolution | |
| CN106101535B (en) | A kind of video stabilizing method based on part and mass motion disparity compensation | |
| CN103106688A (en) | Indoor three-dimensional scene rebuilding method based on double-layer rectification method | |
| CN107734268A (en) | A kind of structure-preserved wide baseline video joining method | |
| CN105389787A (en) | Panorama image stitching method and device | |
| CN106709948A (en) | Quick binocular stereo matching method based on superpixel segmentation | |
| CN107679537A (en) | A kind of texture-free spatial target posture algorithm for estimating based on profile point ORB characteristic matchings | |
| CN105678687A (en) | Stereo image stitching method based on content of images | |
| CN103559737A (en) | Object panorama modeling method | |
| CN106534670B (en) | It is a kind of based on the panoramic video generation method for connecting firmly fish eye lens video camera group | |
| CN110246161B (en) | Method for seamless splicing of 360-degree panoramic images | |
| CN106997579A (en) | The method and apparatus of image mosaic | |
| CN108537865A (en) | A kind of the pseudo-classic architecture model generation method and device of view-based access control model three-dimensional reconstruction | |
| CN112862683B (en) | Adjacent image splicing method based on elastic registration and grid optimization | |
| CN109816706A (en) | A kind of smoothness constraint and triangulation network equal proportion subdivision picture are to dense matching method | |
| CN103955960B (en) | Image viewpoint transformation method based on single input image | |
| CN104751508B (en) | The full-automatic of new view is quickly generated and complementing method in the making of 3D three-dimensional films | |
| CN112288637A (en) | Unmanned aerial vehicle aerial image rapid splicing device and rapid splicing method | |
| CN110267101A (en) | A kind of unmanned plane video based on quick three-dimensional picture mosaic takes out frame method automatically | |
| CN114202632A (en) | Grid linear structure recovery method and device, electronic equipment and storage medium | |
| Patil et al. | A comparative evaluation of SGM variants (including a new variant, tMGM) for dense stereo matching |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| RJ01 | Rejection of invention patent application after publication | ||
| RJ01 | Rejection of invention patent application after publication |
Application publication date: 20170818 |