CN112991176B - Panoramic image splicing method based on optimal suture line - Google Patents
Panoramic image splicing method based on optimal suture line Download PDFInfo
- Publication number
- CN112991176B CN112991176B CN202110300585.9A CN202110300585A CN112991176B CN 112991176 B CN112991176 B CN 112991176B CN 202110300585 A CN202110300585 A CN 202110300585A CN 112991176 B CN112991176 B CN 112991176B
- Authority
- CN
- China
- Prior art keywords
- image
- pixel
- optimal
- point
- line
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
- 238000000034 method Methods 0.000 title claims abstract description 32
- 230000004927 fusion Effects 0.000 claims abstract description 50
- 239000011159 matrix material Substances 0.000 claims abstract description 18
- 230000009466 transformation Effects 0.000 claims abstract description 4
- 238000013507 mapping Methods 0.000 claims description 7
- 230000008569 process Effects 0.000 description 10
- 230000007704 transition Effects 0.000 description 4
- 238000007781 pre-processing Methods 0.000 description 3
- 238000012937 correction Methods 0.000 description 2
- 238000010586 diagram Methods 0.000 description 2
- 230000000007 visual effect Effects 0.000 description 2
- 230000006978 adaptation Effects 0.000 description 1
- 230000004075 alteration Effects 0.000 description 1
- 230000001174 ascending effect Effects 0.000 description 1
- 230000009286 beneficial effect Effects 0.000 description 1
- 230000008859 change Effects 0.000 description 1
- 238000005336 cracking Methods 0.000 description 1
- 230000000694 effects Effects 0.000 description 1
- 238000007499 fusion processing Methods 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 230000000630 rising effect Effects 0.000 description 1
- 239000004576 sand Substances 0.000 description 1
- 230000011218 segmentation Effects 0.000 description 1
- 238000009958 sewing Methods 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T3/00—Geometric image transformations in the plane of the image
- G06T3/40—Scaling of whole images or parts thereof, e.g. expanding or contracting
- G06T3/4038—Image mosaicing, e.g. composing plane images from plane sub-images
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T3/00—Geometric image transformations in the plane of the image
- G06T3/14—Transformations for image registration, e.g. adjusting or mapping for alignment of images
Landscapes
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Image Processing (AREA)
Abstract
The invention discloses a panoramic image splicing method based on an optimal suture line, which belongs to the field of computer vision and comprises the following steps: matching the images to be spliced with the space overlapping area by using a characteristic matching algorithm; calculating a homography matrix, carrying out global homography transformation on the images to be spliced, projecting the images to the same plane, and aligning the overlapping areas; defining an image energy function based on color, gradient and similarity, and searching an optimal suture line by using a minimized image energy function based on dynamic programming in an overlapping area; splicing the images to be spliced into a panoramic image according to the optimal suture line, expanding the panoramic image from the optimal suture line to two sides, and selecting an optimal fusion area according to a gray characteristic function; and merging the optimal fusion area by using an image fusion algorithm to obtain a final panoramic image. The method effectively improves the quality of the panoramic image, and is suitable for the practical application fields of medical images, remote sensing images, virtual reality and the like.
Description
Technical Field
The invention belongs to the field of computer vision, and particularly relates to a panoramic image splicing method based on an optimal suture line.
Background
And (3) splicing the panoramic images, and performing dislocation correction and chromatic aberration fusion on an image sequence with a space coincidence region by using a computer vision technology to finally obtain the panoramic images containing the information of each image in the sequence. At present, hardware equipment for acquiring panoramic images is few, and wide-view-angle images can be acquired only by a machine equipped with a fisheye lens. Due to the limitation of hardware equipment, when a large-view-angle image is required, an image sequence with a space overlapping area is generally obtained by shooting the same scene image with different view angles, and then a panoramic image is generated by using an image stitching method. The image splicing method mainly comprises the steps of image preprocessing, image registration, image fusion and the like, wherein the image registration and the image fusion are the most critical steps. The image registration is to find a mapping relation between two images to be spliced, so that the two images are projected to the same plane, and the overlapping areas are aligned. The result of the image registration determines the accuracy of the alignment of the overlapping regions and indirectly also the final stitching quality. The image fusion directly determines the splicing quality, and the main methods are gradient template weighted fusion, optimal suture line fusion, multiband fusion and the like. In practical use, due to the interference of the environment and the change of the view angle in the shooting process, the problems that the overlapping areas cannot be aligned, the fusion transition zone is obvious, the moving object has double images and the like may exist in the image splicing process, and in order to solve the problems as much as possible, a proper image splicing and fusion strategy needs to be established.
Disclosure of Invention
The purpose of the invention is as follows: in order to overcome the problems in the prior art, the invention discloses a panoramic image splicing method based on an optimal suture line, which can effectively solve the problems of double images, errors, visible splicing positions and the like in image splicing, improve the image splicing quality and enable the spliced panoramic image to have a good visual effect.
The technical scheme is as follows: in order to achieve the purpose, the invention adopts the following technical scheme: a panoramic image stitching method based on an optimal stitching line is characterized in that an image sequence with a space overlapping area is input, each image in the image sequence is preprocessed to obtain images to be stitched, and then the following operations are performed on every two images to be stitched with the space overlapping area:
step S1: image registration: matching the images to be spliced with the space overlapping area by using a feature matching algorithm to obtain a mapping relation between feature points;
step S2: alignment of the overlapping areas: calculating a homography matrix by using the mapping relation between the matched characteristic points, carrying out global homography transformation on the images to be spliced, projecting the images to the same plane, and aligning an overlapping area;
step S3: optimal suture line search: defining an image energy function based on color, gradient and similarity, and searching an optimal suture line by using a minimized image energy function based on dynamic programming in an overlapping area;
step S4: selecting an optimal fusion area: folding the images to be folded into panoramic images according to the optimal suture lines, expanding the panoramic images from the optimal suture lines to two sides, and selecting an optimal fusion area according to a gray characteristic function;
step S5: image fusion: and merging the optimal fusion area by using an image fusion algorithm to obtain a final panoramic image.
Preferably, in step S3, the two images to be stitched are respectively set as a reference image and a target image, and the overlapping areas of the reference image and the target image are respectively IrcAnd ItcFor a certain pixel point (x, y) in the overlap region, the image energy function EsComprises the following steps:
Es=Ecolor+Egeometry+Ecos
intensity of color difference E at the pixel pointcolorComprises the following steps:
Ecolor(x,y)=Irc(x,y)-Itc(x,y)
wherein, Irc(x, y) and Itc(x, y) are the pixel intensity values of the reference image and the target image at the pixel points (x, y) in the overlapping area respectively;
structural difference intensity E at the pixel pointgeometryComprises the following steps:
Egeometry(x,y)=[Sx·(Irc(x,y)-Itc(x,y))]2+[Sy·(Irc(x,y)-Itc(x,y))]2
when the image to be spliced on the left side is a reference image and the image to be spliced on the right side is a target image, the cosine similarity E of the pixel points on the two sides of the pixel pointcosComprises the following steps:
wherein, Irc(x-k, y) is the pixel intensity value of the pixel at the left side of the same line in the reference image and k pixel units away from the pixel, Itc(x + k, y) is the pixel intensity value of the pixel point at the right side of the same line in the target image and k pixel units away from the pixel point;
when the image to be spliced on the right side is a reference image and the image to be spliced on the left side is a target image, the cosine similarity E of the pixel points on the two sides of the pixel pointcosThe definition is as follows:
wherein, Irc(x + k, y) is the pixel intensity value of the pixel at the right side of the same line in the reference image and k pixel units away from the pixel, Itc(x-k, y) is the pixel intensity value of the pixel point at the left side of the same line in the target image and k pixel units away from the pixel point.
Preferably, the searching for the optimal suture line in step S3 includes:
calculating an image energy function value of each pixel point in the overlapping area;
in the overlapping area, each pixel point in the first row is used as a starting point of a suture line, and a connecting point of the suture line is obtained by extending from each starting point to the next row until the last row obtains an end point of the suture line;
traversing all the suture lines, and selecting the image energy functions of all the pixel points on the suture lines and the minimum suture line as the optimal suture line;
when the suture line extends to the next line, comparing the image energy function values of the pixels adjacent to the current pixel point in the next line, selecting the pixel point with the minimum energy function value in the adjacent pixel points as the connection point of the suture line in the next line, and updating the connection point as the current point.
Preferably, in step S4, the optimal suture line is expanded to the left and right, and the gray scale feature function of the expanded region is:
CS(m,n)=ω1fC(m,n)-ω2fIDM(m,n)
wherein M and n are the top-bottom width and the left-right length of the expanded region expanded from the optimal suture line to both sides, respectively, n is a variable, M is a fixed value, i.e., the top-bottom width of the overlapped region, ω1And ω2Is a constant coefficient weight set by experience;
contrast f of gray level co-occurrence matrixCComprises the following steps:
inverse difference moment f of gray level co-occurrence matrixIDMComprises the following steps:
wherein, L is the gray scale of the image;the gray value of the pixel point is in a value range of { i, j |0 is not less than i and not more than L-1, j is not less than 0 and not more than L-1}, and k is | i-j |;
gray level co-occurrence matrix Pθ,δ(i, j) is:
Pθ,δ(i,j)={[(x,y),(x+dx,y+dy)]|f(x,y)=i,f(x+dx,y+dy)=j}
wherein, (x, y) is the coordinate of a certain pixel point, and (x + dx, y + dy) is the coordinate of another pixel point; theta is an included angle between a connecting line of the two pixel points and the positive direction of the abscissa, and the value of theta is 0 degree, 45 degrees, 90 degrees or 135 degrees;is the Euclidean distance between two pixel points; pθ,δ(i, j) denotes a slave extensionStarting a certain pixel point with the gray value i in the region, and counting the probability of existence of the pixel point with the distance delta from the pixel point and the gray value j.
Preferably, in step S4, the selecting the optimal fusion region includes:
and after a gray characteristic function curve related to the left and right lengths of the extension area is obtained, comparing the function values of the current point and the previous point from the second point of the curve, and if the function value of the current point is larger than the function value of the previous point, taking the n value corresponding to the current point as the left and right lengths of the optimal fusion area.
Has the advantages that: the invention has the following remarkable beneficial effects:
according to the method, cosine similarity is introduced to select an optimal suture line, so that the condition that the difference of pixels on two sides in the spliced image is too large, and further foreground cracking or texture fracture is formed is avoided; the optimal fusion area is selected through the gray characteristic function, so that the transition zone can be optimized, the possibility that the fusion area damages texture consistency is reduced, and the phenomenon that ghost images are generated due to the fact that moving objects exist in the selected fusion area is avoided; therefore, the method and the device can effectively solve the problems of double images, errors, visible splicing positions and the like in image splicing, improve the image splicing quality, enable the spliced panoramic image to have a good visual effect, and are suitable for the practical application fields of medical images, remote sensing images, virtual reality and the like.
Drawings
FIG. 1 is a flow chart of a method of the present invention;
FIG. 2 is an image registration result;
FIG. 3 is a schematic diagram of a suture search process;
FIG. 4 is a schematic view of an optimal suture position;
FIG. 5 is a schematic diagram of an optimal fusion region selection process;
FIG. 6 is a graph illustrating a gray scale characteristic function;
FIG. 7 is a schematic view of an optimal fusion zone;
fig. 8 is a fused panoramic image.
Detailed Description
The present invention will be further described with reference to the accompanying drawings.
The invention discloses a panoramic image splicing method based on an optimal suture line, as shown in figure 1, firstly inputting an image sequence with a space overlapping area, preprocessing each image in the image sequence to obtain an image to be spliced, wherein the preprocessing mainly comprises denoising, white balance, brightness correction and the like, and then executing the following operations on every two images to be spliced with the space overlapping area:
step S1, image registration: and matching the two images to be spliced with the space overlapping area, and obtaining the mapping relation between the matched feature points by using a feature matching algorithm.
The feature matching algorithm may adopt an ORB algorithm, a SIFT algorithm, a SURF algorithm, and an LBP algorithm.
The two images to be stitched after the image registration is completed are shown in fig. 2.
Step S2, overlapping area alignment: and calculating a homography matrix by using the mapping relation between the matched characteristic points in the two images to be spliced, and carrying out global homography transformation on the images to be spliced so as to project the images to the same plane and align the overlapped areas.
In one embodiment of the invention, two images to be spliced are respectively set as a reference image and a target image, the target image is transformed to a plane where the reference image is located, and then an overlapping area is aligned.
The process of computing the homography matrix can be done with the RANSAC algorithm.
Step S3, optimal suture line search: and defining an image energy function based on the color, gradient and similarity of the images to be spliced, and searching an optimal suture line by using the image energy function in an overlapping area based on a dynamic programming method.
The search process for the optimal stitch line is performed in the overlapping area of the images to be stitched. Let I be the region of the reference image and the target image which coincide with each otherrcAnd Itc。
Image energy function E defined based on color, gradient and similarity of images to be stitchedsThe following were used:
Es=Ecolor+Egeometry+Ecos (1)
wherein, for a certain pixel point in the overlapping region, EcolorIs the intensity of the color difference at that pixel point, i.e. IrcAnd ItcThe color difference intensity of the same pixel point in the image; ggeometryIs the structural difference intensity at that pixel point; ecosThe cosine similarity of the pixel points on the two sides of the pixel point represents the similarity of the pixel points on the two sides of the suture line, so that the situation that the difference of the pixel points on the two sides in the spliced image is too large, and further the foreground is cut or the texture is broken is avoided.
For a certain pixel point (x, y) in the overlapping region, the color difference intensity E at the pixel pointcolorThe definition is as follows:
Ecolor(x,y)=Irc(x,y)-Itc(x,y) (2)
wherein, Irc(x, y) and Itc(x, y) are the pixel intensity values of the reference image and the target image at pixel point (x, y) in the overlap region, respectively.
Structural difference intensity E at the pixel pointgeometryThe definition is as follows:
Egeometry(x, less) ═ Sx·(Irc(x,y)-Itc(x,y))]2+[Sy·(Irc(x,y)-Itc(x,y))]2 (3)
In the two images to be spliced, the image to be spliced on the left side is taken as a reference image, the image to be spliced on the right side is taken as a target image, and the cosine similarity E of pixel points on two sides of the pixel point is obtainedcosThe definition is as follows:
wherein, Irc(x-k, y) is the pixel intensity value of the pixel at the left side of the same line in the reference image and k pixel units away from the pixel, Itc(x + k, y) is the pixel intensity value of the pixel at the right side of the same line in the target image and k pixel units away from the pixel.
Similarly, in the two images to be stitched, the image to be stitched on the right side is set as a reference image, the image to be stitched on the left side is set as a target image, and the cosine similarity E of the pixel points on the two sides of the pixel point is set ascosThe definition is as follows:
wherein, Irc(x + k, y) is the pixel intensity value of the pixel at the right side of the same line in the reference image and k pixel units away from the pixel, Itc(x-k, y) is the pixel intensity value of the pixel point at the left side of the same line in the target image and k pixel units away from the pixel point.
The process of searching for the best suture line using the dynamic planning concept is to find the segmentation line with the smallest image energy function value in the overlapping region. Setting the size of an overlapping area of the images to be spliced as NxM, wherein N is the left and right length of the overlapping area, and M is the upper and lower width of the overlapping area, and the specific search process is as follows:
calculating image energy function value E of each pixel point in the overlapping region by using image energy criterionsAnd obtaining an N multiplied by M energy matrix D.
In the energy matrix D or in the overlapping area, each pixel point in the first row is used as a starting point of a suture line, and the connecting point of the suture line is obtained by extending from each starting point to the next row until the end point of the suture line is obtained in the last row. The extension rule is to compare the current pixel point with three pixel points adjacent to the pixel point in the next row: under the present pixel, 45 degrees below the left and 45 degrees below the right, it is special, if the present pixel is located the left edge department of this row, then compare two adjacent pixels with this pixel in the next row: and (3) directly below the current pixel point, 45 degrees below the left side or 45 degrees below the right side, wherein the pixel point with the minimum image energy function value is the connection point of the suture line in the next line, the suture line is updated to be the current point, and the process is repeated until the last line is taken as the end point.
The suture line searching process is as shown in fig. 3, and it is assumed that the q (q e [1, N ]) th pixel point of the p (p e [1, M ]) th row in the overlapping region is a (p, q), wherein the suture line is searched by using the first pixel point a (1, 1) in the first row as a starting point, and the image energy function values of the pixel points a (2, 1) and a (2, 2) adjacent to the pixel point a (1, 1) in the second row are 5 and 4, respectively, so that the connection point of the suture line in the second row is the pixel point a (2, 2); and image energy function values of pixel points a (3, 1), a (3, 2) and a (3, 3) adjacent to the pixel point a (2, 2) in the third row are respectively 6, 4 and 7, so that the connection point of the suture line in the third row is the pixel point a (3, 2), and the like, searching to the last row, determining a suture line, and the image energy function of the suture line is the sum of the image energy function values of all the pixel points on the suture line.
Obtaining N sewing lines after traversing each initial point of the first row, and setting the ith (i belongs to [1, N ]]) Strip of suture liThe sum of the image energy functions of all the pixels is Es(li) And selecting the minimum image energy function and the minimum suture line as the optimal suture line:
lbestthe optimal suture line is obtained.
The best suture position found is shown in fig. 4.
Step S4, selecting the optimal fusion area: as shown in fig. 5, the images to be stitched are stitched into a panoramic image according to the optimal stitching line, the panoramic image is expanded from the optimal stitching line to the left and right sides, and the optimal fusion area is selected according to the gray characteristic function.
The optimal fusion area is selected by utilizing the gray characteristic function and then spliced, and compared with direct splicing, the transition zone can be optimized; the possibility that the fused area destroys the texture consistency can be reduced; the phenomenon that ghost images are generated due to the fact that moving objects exist in the selected fusion area can be avoided.
The gray scale feature function is defined as follows:
CS(m,n)=ω1fC(m,n)-ω2fIDM(m,n) (7)
wherein M and n are the upper and lower width and the left and right length of the extended region respectively, n is a variable, and M is a fixed value, i.e. the upper and lower width of the overlapped region; omega1And ω2For empirically set constant coefficient weights, ω is typically taken1=1,ω2=1;fCIs the contrast of the gray level co-occurrence matrix, fIDMIs the inverse difference moment of the gray level co-occurrence matrix. So that the characteristic function of the gray scale CS(m, n) is a function of the left and right length n of the extended region.
The contrast of the gray level co-occurrence matrix is defined as follows:
the inverse difference moment of the gray level co-occurrence matrix is defined as follows:
wherein L represents the gray scale of the image, typically L ═ 16;the gray value of the hook pixel point is in a value range { i, j |0 is more than or equal to i and less than or equal to L-1, and j is more than or equal to 0 and less than or equal to L-1 }; k ═ i-j |; pθ,δ(i, j) is a gray level co-occurrence matrix defined as follows:
Pθ,δ(i,j)={[(x,y),(x+dx,y+dy)]|f(x,y)=i,f(x+dx,y+dy)=j} (10)
wherein, (x, y) is the coordinate of a certain pixel point, and (x + dx, y + dy) is the coordinate of another pixel point; theta is an included angle between a connecting line of the two pixel points and the positive direction of the abscissa, and is generally 0 degree, 45 degrees, 90 degrees or 135 degrees;the Euclidean distance between two pixel points is hooked; pθ,δAnd (i, j) calculating the probability of existence of a pixel point which is delta away from a certain gray value i in the expansion area and has a gray value j from the certain gray value i.
And after a gray characteristic function curve about the left and right length n of the extension area is obtained, starting from a second point of the curve, comparing the function values of the current point and the previous point, and if the function value of the current point is greater than the function value of the previous point, taking the n value corresponding to the current point as the left and right length of the optimal fusion area.
When foreground targets or moving objects exist on two sides of the optimal suture line, the obtained gray characteristic function curve quickly descends at the beginning and then fluctuates, and then a first ascending point is selected, so that a smaller length value can be obtained, the transition zone is optimized, and obvious foreground targets or moving objects are avoided being touched; under very special conditions, namely when no foreground object or moving object exists on two sides of the optimal suture line, a rising trend is likely to appear at the beginning of the gray characteristic function curve, and by utilizing the selection rule, n is 2, namely the suture line can be spliced according to the optimal suture line.
As shown in fig. 6, the gray scale feature function curve is in a descending state at the beginning, and the function value is raised when n is 12, so that n is 12, which is the left and right length of the selected optimal fusion region. The selected optimal fusion region is schematically shown in fig. 7.
Step S5, image fusion: and merging the optimal fusion areas, and obtaining a final panoramic image by using an image fusion algorithm.
The image fusion algorithm may employ linear fusion, laplacian pyramid fusion, poisson fusion, or the like. As shown in fig. 6.
The panoramic image after completion of the image fusion is shown in fig. 8.
The image to be spliced is respectively spliced directly, the whole overlapped area is fused gradually and gradually, the method of the invention carries out the fusion processing of the optimal fusion area gradually and gradually, and the fusion indexes of the optimal fusion area are respectively calculated as shown in tables 1, 2 and 3:
TABLE 1
TABLE 2
TABLE 3
As can be seen from tables 1, 2 and 3, the information entropy and the average gradient of the method are optimal, and the mutual information and the cross entropy are optimal or suboptimal, so that the four fusion indexes are integrated, and the method has the best splicing effect.
The above description is only of the preferred embodiments of the present invention, and it should be noted that: it will be apparent to those skilled in the art that various modifications and adaptations can be made without departing from the principles of the invention and these are intended to be within the scope of the invention.
Claims (4)
1. A panoramic image stitching method based on an optimal stitching line is characterized in that an image sequence with a space overlapping area is input, each image in the image sequence is preprocessed to obtain images to be stitched, and then the following operations are performed on every two images to be stitched with the space overlapping area:
step S1: image registration: matching the images to be spliced with the space overlapping area by using a feature matching algorithm to obtain a mapping relation between feature points;
step S2: alignment of the overlapping areas: calculating a homography matrix by using the mapping relation between the matched characteristic points, carrying out global homography transformation on the images to be spliced, projecting the images to the same plane, and aligning an overlapping area;
step S3: optimal suture line search: defining an image energy function based on color, gradient and similarity, and searching an optimal suture line by using a minimized image energy function based on dynamic programming in an overlapping area;
step S4: selecting an optimal fusion area: splicing the images to be spliced into a panoramic image according to the optimal suture line, expanding the panoramic image from the optimal suture line to two sides, and selecting an optimal fusion area according to a gray characteristic function;
step S5: image fusion: merging the optimal fusion area by using an image fusion algorithm to obtain a final panoramic image;
in step S3, the two images to be stitched are respectively set as a reference image and a target image, and the overlapping areas of the reference image and the target image are respectively IrcAnd ItcFor a certain pixel point (x, y) in the overlap region, the image energy function EsComprises the following steps:
Es=Ecolor+Egeometry+Ecos
intensity of color difference E at the pixel pointcolorComprises the following steps:
Ecolor(x,y)=Irc(x,y)-Itc(x,y)
wherein, Irc(x, y) and Itc(x, y) are the pixel intensity values of the reference image and the target image at the pixel points (x, y) in the overlapping area respectively;
structural difference intensity E at the pixel pointgeometryComprises the following steps:
Egeometry(x,y)=[Sx·(Irc(x,y)-Itc(x,y))]2+[Sy·(Irc(x,y)-Itc(x,y))]2
when the image to be spliced on the left side is a reference image and the image to be spliced on the right side is a target image, the cosine similarity E of the pixel points on the two sides of the pixel pointcosComprises the following steps:
wherein, Irc(x-k, y) is the pixel intensity value of the pixel at the left side of the same line in the reference image and k pixel units away from the pixel, Itc(x + k, y) is the pixel intensity value of the pixel point at the right side of the same line in the target image and k pixel units away from the pixel point;
when the image to be spliced on the right side is a reference image and the image to be spliced on the left side is a target image, the cosine similarity E of the pixel points on the two sides of the pixel pointcosThe definition is as follows:
wherein, Irc(x + k, y) is the pixel intensity value of the pixel at the right side of the same line in the reference image and k pixel units away from the pixel, Itc(x-k, y) is the pixel intensity value of the pixel point at the left side of the same line in the target image and k pixel units away from the pixel point.
2. The panoramic image stitching method based on the optimal stitching line of claim 1, wherein the step S3 of searching for the optimal stitching line comprises:
calculating an image energy function value of each pixel point in the overlapping area;
in the overlapping area, each pixel point in the first row is used as a starting point of a suture line, and a connecting point of the suture line is obtained by extending from each starting point to the next row until the last row obtains an end point of the suture line;
traversing all the suture lines, and selecting the image energy functions of all the pixel points on the suture lines and the minimum suture line as the optimal suture line;
when the suture line extends to the next line, comparing the image energy function values of the pixels adjacent to the current pixel point in the next line, selecting the pixel point with the minimum energy function value in the adjacent pixel points as the connection point of the suture line in the next line, and updating the connection point as the current point.
3. The panoramic image stitching method based on the optimal stitching line as claimed in claim 1, wherein in step S4, the panoramic image is expanded from the optimal stitching line to the left and right, and the gray scale feature function of the expanded region is as follows:
CS(m,n)=ω1fC(m,n)-ω2fIDM(m,n)
wherein M and n are the top-bottom width and the left-right length of the expanded region expanded from the optimal suture line to both sides, respectively, n is a variable, M is a fixed value, i.e., the top-bottom width of the overlapped region, ω1And ω2Is a constant coefficient weight set by experience;
contrast f of gray level co-occurrence matrixCComprises the following steps:
inverse difference moment f of gray level co-occurrence matrixIDMComprises the following steps:
wherein L is the gray scale of the imageGrade;the gray value of the pixel point is in a value range of { i, j |0 is not less than i and not more than L-1, j is not less than 0 and not more than L-1}, and k is | i-j |;
gray level co-occurrence matrix Pθ,δ(i, j) is:
Pθ,δ(i,j)={[(x,y),(x+dx,y+dy)]|f(x,y)=i,f(x+dx,y+dy)=j}
wherein, (x, y) is the coordinate of a certain pixel point, and (x + dx, y + dy) is the coordinate of another pixel point; theta is an included angle between a connecting line of the two pixel points and the positive direction of the abscissa, and the value of theta is 0 degree, 45 degrees, 90 degrees or 135 degrees;is the Euclidean distance between two pixel points; pθ,δAnd (i, j) calculating the probability of existence of a pixel point which is delta away from a certain gray value i in the expansion area and has a gray value j from the certain gray value i.
4. The method for stitching panoramic images based on the optimal stitching line according to claim 3, wherein the step S4 of selecting the optimal fusion area comprises:
and after a gray characteristic function curve related to the left and right lengths of the extension area is obtained, comparing the function values of the current point and the previous point from the second point of the curve, and if the function value of the current point is larger than the function value of the previous point, taking the n value corresponding to the current point as the left and right lengths of the optimal fusion area.
Priority Applications (1)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
CN202110300585.9A CN112991176B (en) | 2021-03-19 | 2021-03-19 | Panoramic image splicing method based on optimal suture line |
Applications Claiming Priority (1)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
CN202110300585.9A CN112991176B (en) | 2021-03-19 | 2021-03-19 | Panoramic image splicing method based on optimal suture line |
Publications (2)
Publication Number | Publication Date |
---|---|
CN112991176A CN112991176A (en) | 2021-06-18 |
CN112991176B true CN112991176B (en) | 2022-03-01 |
Family
ID=76332730
Family Applications (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
CN202110300585.9A Active CN112991176B (en) | 2021-03-19 | 2021-03-19 | Panoramic image splicing method based on optimal suture line |
Country Status (1)
Country | Link |
---|---|
CN (1) | CN112991176B (en) |
Families Citing this family (11)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN113160048A (en) * | 2021-02-02 | 2021-07-23 | 重庆高新区飞马创新研究院 | Suture line guided image splicing method |
CN113506216B (en) * | 2021-06-24 | 2024-03-12 | 煤炭科学研究总院 | Rapid suture line optimizing method for panoramic image stitching |
CN113962911B (en) * | 2021-08-08 | 2024-08-30 | 南京理工大学 | Visual field blind-supplement video rapid fusion method for special vehicle |
CN114549301A (en) * | 2021-12-29 | 2022-05-27 | 浙江大华技术股份有限公司 | Image splicing method and device |
CN114842095A (en) * | 2022-03-28 | 2022-08-02 | 南京邮电大学 | Optimal suture image fusion method considering space-time relation and oriented to virtual reality |
CN114943823B (en) * | 2022-07-26 | 2022-10-28 | 武汉图科智能科技有限公司 | Unmanned aerial vehicle image splicing method and system based on deep learning semantic perception |
CN117114997B (en) * | 2023-10-23 | 2024-01-30 | 四川新视创伟超高清科技有限公司 | Image stitching method and device based on suture line search algorithm |
CN117670667B (en) * | 2023-11-08 | 2024-05-28 | 广州成至智能机器科技有限公司 | Unmanned aerial vehicle real-time infrared image panorama stitching method |
CN118015237B (en) * | 2024-04-09 | 2024-06-21 | 松立控股集团股份有限公司 | Multi-view image stitching method and system based on global similarity optimal seam |
CN118096523B (en) * | 2024-04-25 | 2024-07-12 | 陕西旭腾光讯科技有限公司 | Image stitching method based on computer vision |
CN118314309B (en) * | 2024-06-11 | 2024-08-13 | 光原科技(杭州)有限公司 | 3D suture splicing and fusion method and system based on structural content perception |
Citations (3)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN104835117A (en) * | 2015-05-11 | 2015-08-12 | 合肥工业大学 | Spherical panorama generating method based on overlapping way |
CN108961162A (en) * | 2018-03-12 | 2018-12-07 | 北京林业大学 | A kind of unmanned plane forest zone Aerial Images joining method and system |
CN110175011A (en) * | 2019-05-06 | 2019-08-27 | 长春理工大学 | A kind of full-view image seamless joint method |
Family Cites Families (1)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
WO2010025309A1 (en) * | 2008-08-28 | 2010-03-04 | Zoran Corporation | Robust fast panorama stitching in mobile phones or cameras |
-
2021
- 2021-03-19 CN CN202110300585.9A patent/CN112991176B/en active Active
Patent Citations (3)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN104835117A (en) * | 2015-05-11 | 2015-08-12 | 合肥工业大学 | Spherical panorama generating method based on overlapping way |
CN108961162A (en) * | 2018-03-12 | 2018-12-07 | 北京林业大学 | A kind of unmanned plane forest zone Aerial Images joining method and system |
CN110175011A (en) * | 2019-05-06 | 2019-08-27 | 长春理工大学 | A kind of full-view image seamless joint method |
Also Published As
Publication number | Publication date |
---|---|
CN112991176A (en) | 2021-06-18 |
Similar Documents
Publication | Publication Date | Title |
---|---|---|
CN112991176B (en) | Panoramic image splicing method based on optimal suture line | |
CN111080529A (en) | Unmanned aerial vehicle aerial image splicing method for enhancing robustness | |
CN105957015B (en) | A kind of 360 degree of panorama mosaic methods of threaded barrel inner wall image and system | |
Mistry et al. | Image stitching using Harris feature detection | |
CN111553939B (en) | Image registration algorithm of multi-view camera | |
CN102834845A (en) | Method and arrangement for multi-camera calibration | |
CN109886124B (en) | Non-texture metal part grabbing method based on wire harness description subimage matching | |
JP6612822B2 (en) | System and method for modifying a model | |
CN103258321A (en) | Image stitching method | |
JP5468824B2 (en) | Method and apparatus for determining shape match in three dimensions | |
CN111192194B (en) | Panoramic image stitching method for curtain wall building facade | |
CN111242848B (en) | Binocular camera image suture line splicing method and system based on regional feature registration | |
CN102004911B (en) | Method for improving accuracy of face identification | |
CN110288511A (en) | Minimum error joining method, device, electronic equipment based on double camera image | |
CN112365518A (en) | Image splicing method based on optimal suture line self-selection area gradual-in and gradual-out algorithm | |
CN113793266A (en) | Multi-view machine vision image splicing method, system and storage medium | |
CN103793891A (en) | Low-complexity panorama image joint method | |
CN107067368B (en) | Streetscape image splicing method and system based on deformation of image | |
CN112258561A (en) | Matching point acquisition method for image stitching | |
CN115393187A (en) | High-freedom-degree microscopic image splicing and fusing method and system | |
Shaikh et al. | Multiple feature extraction techniques in image stitching | |
Jagadeeswari et al. | A comparative study based on video stitching methods | |
JP6278757B2 (en) | Feature value generation device, feature value generation method, and program | |
Xi et al. | Multi-retinal images stitching based on the maximum fusion and correction ratio of gray average | |
Jain et al. | Panorama construction from multi-view cameras in outdoor scenes |
Legal Events
Date | Code | Title | Description |
---|---|---|---|
PB01 | Publication | ||
PB01 | Publication | ||
SE01 | Entry into force of request for substantive examination | ||
SE01 | Entry into force of request for substantive examination | ||
GR01 | Patent grant | ||
GR01 | Patent grant | ||
TR01 | Transfer of patent right |
Effective date of registration: 20230627 Address after: Room 317-319, Building 43, Headquarters Base, No. 70 Phoenix Road, Jiangning District, Nanjing, Jiangsu 211106 (Jiangning Development Zone) Patentee after: Nanjing Qianlian Technology Co.,Ltd. Address before: 1 No. 211167 Jiangsu city of Nanjing province Jiangning Science Park Hongjing Road Patentee before: NANJING INSTITUTE OF TECHNOLOGY |
|
TR01 | Transfer of patent right |