CN114862672B - A fast image stitching method based on vector shape preserving transformation - Google Patents

Abstract

The invention discloses a fast image splicing method based on vector shape-preserving transformation, which includes: reading original images to be spliced, performing denoising preprocessing on the images; extracting SIFT features of all images; and purifying and obtaining interior points between matching images. ;Judge the image matching relationship through the quantitative relationship between interior points and original feature points; construct feature vectors through image interior points to calculate two types of parameters in the image transformation matrix step by step; use matching interior points to calculate the initial values of the two types of parameters; use The constructed feature vector is iteratively optimized to calculate the rotation parameters of the transformation matrix; the translation parameters of the transformation matrix are iteratively calculated based on the matching interior points and the optimized rotation parameters; the transformation matrix of each image is calculated through the rotation parameters and translation parameters obtained through step-by-step optimization; Get the final splicing effect picture. The invention can significantly improve the splicing quality and effectively reduce the time required for splicing multiple pictures, so that it can meet the industrial real-time splicing requirements.

Description

A fast image stitching method based on vector shape preserving transformation

Technical Field

The invention relates to the field of image processing technology, and in particular to a fast image splicing method based on vector shape-preserving transformation.

Background technique

Image stitching is a technology that fuses two or more local area observation pictures with certain overlapping areas to form a wide-angle, high-resolution picture that includes the entire observation area. In actual application scenarios, such as actual battlefield surveillance tasks, image stitching technology mainly has two specific requirements. One is speed, that is, it can quickly complete the stitching after capturing a large number of pictures, and complete the panoramic view of the target area in real time; the other The second is accuracy, that is, the spliced image has no imaging quality problems such as ghosting and ghosting, and the image is natural and beautiful, truly reflecting the actual imaging area. Therefore, fast and accurate image stitching is an extremely important requirement for practical application scenarios. Image stitching technology is mainly divided into four steps, namely image acquisition, image preprocessing, image registration and image fusion. The most critical step is image registration. In this step, the transformation matrix parameters between matching images, namely rotation parameters and translation parameters, are calculated based on the extracted image feature point position information, and then the Bundle Adjustment method is used to iteratively optimize All parameters of all images, and then achieve alignment of the geometric positions of the images according to the required parameters. Using iteratively optimized parameters can improve the image splicing effect. However, due to the limitation of using feature point position information, all parameters need to be calculated at the same time. The matrix size during the optimization process is too large and time-consuming. Moreover, due to the two parameters in the transformation matrix If the difference is too large, they will affect each other during optimization, and the stitched images will still have poor registration results.

Therefore, it is necessary to design a multi-image stitching method that is computationally fast and accurate. This invention is based on the vector shape-preserving transformation method and uses two types of parameters in the vector separation transformation matrix to reduce the calculation scale of the optimization matrix and eliminate the phenomenon of mutual interference of parameters during the optimization process, so that it can obtain excellent results on the basis of fast splicing. Splicing effect.

Contents of the invention

In order to overcome the shortcomings and deficiencies of the existing technology, the present invention provides a fast image splicing method based on vector shape-preserving transformation.

The invention can speed up the splicing of multiple images on the premise of obtaining better splicing effect, so as to meet the needs of actual industrial applications.

In order to achieve the above purpose, the present invention adopts the following technical solutions:

A fast image splicing method based on vector shape-preserving transformation, including:

Read multiple images to be stitched and preprocess them;

Extract the SIFT feature points of each image separately and save them;

Purify the SIFT feature points extracted between any two images, obtain the feature interior points between the image pairs, and calculate the matching relationship between the image pairs;

Construct feature vectors based on feature interior points;

According to the matching relationship of the matching images and the purified feature internal points, the transformation matrix between the two matching images is calculated, and the initial values of the iterative optimization of the rotation parameters and translation parameters of each image are further obtained;

Iteratively optimize the rotation transformation parameters of each image according to the feature vector;

Iteratively optimize the translation transformation parameters of each image based on the optimized rotation transformation parameters and feature interior point matching relationship;

Calculate the final transformation matrix based on the optimized rotation transformation parameters and translation transformation parameters;

According to the calculated transformation matrix of each image, the relative positions of all images are obtained, and the final spliced image is obtained through the image fusion step.

Further, the preprocessing step is denoising.

Furthermore, the SIFT feature points extracted between any two images are purified using a RANSAC algorithm to remove unmatched points.

Furthermore, the feature inliers between the image pairs are obtained, and the matching relationship between the image pairs is calculated, specifically:

Assume that the total number of SIFT feature matching pairs extracted is n _f , and the number of feature interior point pairs purified by the RANSAC algorithm is n _i . If n _i >8+0.3·n _f , the two images match.

Further, the feature vector is constructed according to the feature interior points, specifically:

In a single image, according to the order in which the feature interior points are saved, with the k-th point as the starting point and the k+1 point as the end point, vectors are constructed sequentially:

in, are respectively the k-th and k+1-th interior points among the saved feature interior points,/> is the kth vector obtained by construction;

Furthermore, the initial values of the rotation parameters and translation parameters of each image are obtained by iterative optimization, specifically:

Assume that a pair of feature inliers in any two matching images are and/> The specific calculation steps of the transformation matrix rotation parameter θ and translation parameter T = [t _x t _y ] ^T initial value are as follows:

Among them, A∈R ^2N×4 , B∈R ^2N×1 , N is the number of extracted image interior point features. Matrix A and matrix B are the combination of a _k and b _k calculated based on all feature interior points. .

Furthermore, the rotation transformation parameters of each image are optimized iteratively according to the feature vector, specifically:

The error between two matching images i and j is defined as the sum of the modulus of the vector difference after rotation transformation of the internal feature vectors of the image. The calculation method is as follows:

in, and/> are the k-th matching feature vector in images i, j, respectively, /> Represents all feature vectors constructed in pictures i,, j, R _ij represents the rotation transformation matrix between pictures i,, j,

The cumulative error of the entire image is the sum of the distances between the corresponding feature vectors of all images and their matching images after passing through the rotation transformation matrix. The calculation method is as follows:

Where n represents the number of images to be spliced, I(i) represents all images matching image i, and then iteratively optimizes and calculates all rotation transformation matrices R _ij to obtain the rotation parameter θ _ij .

Furthermore, the final transformation matrix is calculated based on the optimized rotation transformation parameters and translation transformation parameters, specifically as follows:

The error between two matching images i and j is defined as the sum of the distances of all feature internal points after the optimized rotation transformation and the translation transformation, and is calculated as follows:

in, and/> are the kth matching inliers in images i, j respectively,/> Represents all feature points in image i, j,/> represents the optimized rotation transformation matrix between images i, j, _Tij represents the translation transformation parameter between images i, j, and the cumulative error of the entire image is the sum of the distances between the feature inliers of all images and their matching images after the translation transformation, which is calculated as follows:

Where n represents the number of images to be spliced, I(i) represents all images matching image i, and then iterative optimization is performed to calculate all translation transformation parameters T _ij .

Further, the final transformation matrix is expressed as follows:

Furthermore, matrix A and matrix B are the combination of a _k and b _k calculated based on all feature interior points, specifically:

Beneficial effects of the present invention:

(1) The method proposed by the present invention to use vectors to replace traditional feature points calculates the transformation matrix of each image in the registration process. It can calculate the two parameters in the transformation matrix step by step, and then realize the step-by-step optimization of the two parameters. Compared with the traditional iterative optimization matrix that optimizes all parameters at the same time, the size of the two iterative optimization matrices at the end of the step is significantly reduced, which greatly reduces the amount of calculation and significantly improves the splicing speed;

(2) The method proposed by the present invention to use vectors to replace traditional feature points calculates the transformation matrix of each image in the registration process, which can realize the separate calculation of the two parameters in the transformation matrix and eliminate the excessive difference between the two parameters in the optimization process. The phenomenon of mutual interference improves the accuracy of optimizing the calculation matrix transformation parameters and significantly improves the splicing effect.

Description of drawings

Figure 1 is a schematic diagram of the splicing method based on vector shape-preserving transformation according to the present invention;

Figure 2(a) is the original image to be spliced, Figure 2(b) is the original unoptimized splicing rendering, Figure 2(c) is the splicing rendering using traditional point feature optimization and calculation transformation matrix, and Figure 2(d) is the splicing rendering using this method. The splicing effect diagram of the method described in the embodiment;

Figure 3 is a work flow chart of the present invention.

Detailed ways

The present invention will be further described in detail below in conjunction with the embodiments and drawings, but the embodiments of the present invention are not limited thereto.

Example

This embodiment provides a fast image splicing method based on vector shape-preserving transformation, extracts all feature points in the image to be spliced, constructs vector features based on the internal feature points of the image, and separately calculates two parameters in the image transformation matrix, so that During the iterative optimization process, there will be no mutual influence due to excessive parameter differences, making the matrix calculation results more accurate and greatly improving the image stitching effect. At the same time, due to the separation of parameter calculations, the iterative optimization process can also be separated. After separation, the size of the two optimization matrices is greatly reduced compared with the original optimization matrix containing all parameters, and the splicing speed is significantly accelerated.

As shown in Figures 1 and 3, the process includes the following steps:

S1 reads all original images to be spliced, preprocesses the original images and removes noise;

S2 directly extracts the SIFT feature points of each image and saves them;

S3 uses the RANSAC (Random sample consensus) algorithm to purify the directly extracted SIFT feature points, remove mismatched feature points, retain and save the correctly matched feature points, and then determine the image matching relationship. Assume that the total number of SIFT feature matching pairs extracted directly is n _f , and the number of interior points obtained after purification through the RANSAC algorithm is n _i . If n _i >8+0.3·n _f , it can be judged that the two images match;

S4 constructs a feature vector through the purified feature interior points. The construction method is to construct the vector in sequence according to the order of the saved interior points in a single image, with the previous point as the starting point and the latter point as the end point:

in, are respectively the k-th and k+1-th interior points among the saved feature interior points,/> is the kth vector obtained by construction;

S5 calculates the transformation matrix between the two matched images based on the matching relationship of the matched images and the purified feature interior point information, providing initial values for the iterative optimization of Bundle Adjustment. Assume that one pair of interior points in the two matched images are respectively and/> The specific calculation steps for the initial value of the transformation matrix rotation parameter θ and translation parameter T = [t _{x ty} ] ^T are as follows:

Among them, A∈R ^2N×4 , B∈R ^2N×1 , N is the number of extracted image interior point features, and the matrices A and B are the combination of a _k and b _k calculated based on all feature interior points.

S6 iteratively optimizes the rotation parameters based on the constructed feature vector matching relationship. The specific calculation steps are as follows:

The error between two matching images i and j is defined as the sum of the modulus of the vector difference after rotation transformation of the internal feature vectors of the image. The calculation method is as follows:

in, and/> are the k-th matching feature vector in images i, j, respectively, /> Represents all feature vectors constructed in pictures i,,j, R _ij represents the rotation transformation matrix between pictures i,,j.

The cumulative error of the entire image is the sum of the distances between the corresponding feature vectors of all images and their matching images after passing through the rotation transformation matrix. The calculation is as follows:

Where n represents the number of images to be spliced, and I(i) represents all images matching image i. Then it is iteratively optimized according to the Levenberg-Marquardt algorithm to calculate all rotation transformation matrices R _ij , and the rotation parameter θ _ij is obtained;

S7 combines the interior points between all matching images to iteratively optimize the translation parameters of each image based on the optimized rotation transformation parameters and the interior point matching relationship between the images. The specific calculation steps are as follows:

The error between two correctly matched images i and j is defined as the sum of the distances of all feature internal points after the optimized rotation transformation and the translation transformation, which is calculated as follows:

in, and/> are the kth matching inliers in images i, j respectively,/> Represents all feature points in image i, j,/> represents the optimized rotation transformation matrix between images i, j, and _Tij represents the translation transformation parameter between images i, j. The cumulative error of the entire image is the sum of the distances between the feature inliers of all images and their matching images after translation transformation. The calculation method is as follows:

Where n represents the number of images to be spliced, I(i(represents all images matching image i. Then it is iteratively optimized according to the Levenberg-Marquardt algorithm to calculate all translation transformation parameters T _ij ;

S8 constructs the final transformation matrix based on the rotation parameters and translation parameters obtained from optimization. Its specific form is as follows:

S9 calculates the relative positions of all images based on the calculated transformation matrix, implements image fusion based on the average fusion algorithm, and obtains the final spliced image.

Figure 2(a) is the original image to be spliced, Figure 2(b) is the original unoptimized stitching rendering, Figure 2(c) is the stitching rendering using traditional feature point optimization calculation transformation matrix, Figure 2(d) is the stitching rendering using this implementation The splicing effect diagram of the method described in the example. The method of using traditional feature points to iteratively calculate the transformation matrix has a large registration error, obvious ghost blur phenomenon, poor splicing quality, and a long running time, which cannot meet the needs of actual industrial applications; and the method proposed in this embodiment The method of step-by-step iterative optimization of parameters based on feature vectors achieves good splicing results, small image registration errors, and the time required is significantly reduced compared to the method of simultaneously iteratively optimizing all parameters, indicating that this embodiment has better performance than existing algorithms. More in line with actual application needs.

The above embodiments are preferred embodiments of the present invention, but the embodiments of the present invention are not limited to the above embodiments. Any other changes, modifications, substitutions, and combinations may be made without departing from the spirit and principles of the present invention. , simplification, should all be equivalent replacement methods, and are all included in the protection scope of the present invention.

Claims (7)

1. A method for fast image splicing based on vector shape preserving transformation, characterized by comprising: Read multiple images to be stitched and preprocess them; Extract the SIFT feature points of each image separately and save them; Purify the SIFT feature points extracted between any two images, obtain the feature interior points between the image pairs, and calculate the matching relationship between the image pairs; Construct feature vectors based on feature interior points; According to the matching relationship of the matched images and the purified feature interior points, calculate the transformation matrix between the two matched images, and further obtain the iterative optimization initial values of the rotation parameters and translation parameters of each image; Iteratively optimize the rotation transformation parameters of each image according to the feature vector; Iteratively optimize the translation transformation parameters of each image based on the optimized rotation transformation parameters and feature interior point matching relationships; Calculate the final transformation matrix based on the optimized rotation transformation parameters and translation transformation parameters; According to the calculated transformation matrix of each image, the relative positions of all images are obtained, and the final spliced image is obtained through the image fusion step; The feature vector is constructed according to the feature interior points, specifically: In a single image, according to the order in which the feature interior points are saved, with the k-th point as the starting point and the k+1 point as the end point, vectors are constructed sequentially: in, They are the kth and k+1th inliers in the saved feature inliers, respectively./> is the kth vector obtained by construction; The further steps are to obtain the iterative optimization initial values of the rotation parameters and translation parameters of each image, specifically as follows: Assume that a pair of feature interior points in any two matching images are respectively and/> The specific calculation steps for the initial value of the transformation matrix rotation parameter θ and translation parameter T = [t _{x ty} ] ^T are as follows: Where A∈R ^2N×4 , B∈R ^2N×1 , N is the number of extracted image internal point features, and matrix A and matrix B are the combination of a _k and b _k calculated based on all feature internal points; The rotation transformation parameters of each image are iteratively optimized according to the feature vector, specifically: The error between two matching images i and j is defined as the sum of the modulus of the vector difference after rotation transformation of the internal feature vectors of the image. The calculation method is as follows: in, and/> are the k-th matching feature vector in images i, j, respectively, /> Represents all feature vectors constructed in pictures i,, j, R _ij represents the rotation transformation matrix between pictures i,, j, The cumulative error of the entire image is the sum of the distances between the corresponding feature vectors of all images and their matching images after passing through the rotation transformation matrix. The calculation method is as follows: Where n represents the number of images to be spliced, I(i) represents all images matching image i, and then iteratively optimizes and calculates all rotation transformation matrices R _ij to obtain the rotation parameter θ _ij . 2. The image fast splicing method according to claim 1, characterized in that the preprocessing step is denoising. 3. The image fast splicing method according to claim 1, characterized in that the RANSAC algorithm is used to purify the SIFT feature points extracted between any two images to remove unmatched points. 4. The image fast splicing method according to claim 3, characterized in that, the characteristic interior points between the image pairs are obtained, and the matching relationship between the image pairs is calculated, specifically: Assume that the total number of SIFT feature matching pairs extracted is n _f , and the number of feature point pairs purified by the RANSAC algorithm is n _i . If n _i >8+0.3·n _f , the two images match. 5. The image fast splicing method according to claim 1, characterized in that the final transformation matrix is calculated according to the optimized rotation transformation parameters and translation transformation parameters, specifically: The error between two matched images i and j is defined as the sum of distances between all feature interior points after optimization of rotation transformation and translation transformation. The calculation method is as follows: in, and/> are the kth matching inliers in images i, j respectively,/> Represents all feature points in image i, j,/> represents the optimized rotation transformation matrix between images i, j, _Tij represents the translation transformation parameter between images i, j, and the cumulative error of the entire image is the sum of the distances between the feature inliers of all images and their matching images after the translation transformation, which is calculated as follows: Where n represents the number of images to be spliced, I(i) represents all images matching image i, and then iterative optimization is performed to calculate all translation transformation parameters T _ij . 6. The image fast splicing method according to claim 5, characterized in that the final transformation matrix is expressed as follows: 7. The image fast splicing method according to claim 1, characterized in that matrix A and matrix B are a combination of a _k and b _k calculated based on all feature interior points, specifically: