CN116645405A

CN116645405A - ORB-GMS-based image matching method, device, computer equipment and storage medium

Info

Publication number: CN116645405A
Application number: CN202310722912.9A
Authority: CN
Inventors: 王文辉; 李贻凯; 彭煜民; 马一鸣; 张豪; 李尧; 黄凡旗
Original assignee: Energy Storage Research Institute Of China Southern Power Grid Peak Regulation And Frequency Regulation Power Generation Co ltd
Current assignee: Energy Storage Research Institute Of China Southern Power Grid Peak Regulation And Frequency Regulation Power Generation Co ltd
Priority date: 2023-06-16
Filing date: 2023-06-16
Publication date: 2023-08-25

Abstract

The embodiment of the application discloses an ORB-GMS-based image matching method, an ORB-GMS-based image matching device, a computer device and a storage medium; the method comprises the following steps: acquiring a plurality of images to be spliced; performing feature matching on the image based on an ORB algorithm to obtain a feature descriptor; carrying out fast robust matching on the feature descriptors based on an ORB-GMS algorithm to obtain homography matrixes; inputting the expected distorted image, the homography matrix and the shape of the output image according to perspective transformation, and splicing the images; the embodiment of the application performs image matching based on the ORB-GMS algorithm, and has high performance, small calculated amount and strong practicability.

Description

ORB-GMS-based image matching method, device, computer equipment and storage medium

Technical Field

The embodiment of the application relates to the technical field of computer vision, in particular to an ORB-GMS-based image matching method, an ORB-GMS-based image matching device, computer equipment and a storage medium.

Background

In the process of stitching images, the problem of image fusion of a plurality of cameras is required. Before fusion begins, corrected undistorted images need to be obtained by de-distortion correction, and then the images are aligned and fused. For the alignment and fusion problem of the undistorted image, one general method is as follows: and performing feature point matching on the images to be spliced, and determining global deformation of the whole image by fitting the rotation, translation and scaling relation of the feature points. That is, image stitching is accomplished by determining parameters for rotation, translation, and scaling between multiple images through common features between the images. Therefore, whether the characteristic points are matched well or not directly influences the subsequent splicing work.

Although the existing characteristic point algorithm can generate dense characteristic points in a short time, the description performance of a part of characteristic points is reduced; the better feature points can be achieved by a complex algorithm, but the calculation load is too large and the practicability is poor.

Disclosure of Invention

The embodiment of the application provides an ORB-GMS-based image matching method, an ORB-GMS-based image matching device, computer equipment and a storage medium, which are used for solving the problems of overlarge calculation load and poor practicability of the conventional image feature point algorithm.

In a first aspect, an embodiment of the present application provides an ORB-GMS based image matching method, comprising the steps of:

acquiring a plurality of images to be spliced;

performing feature matching on the image based on an ORB algorithm to obtain a feature descriptor;

carrying out fast robust matching on the feature descriptors based on an ORB-GMS algorithm to obtain homography matrixes;

the images are stitched according to the perspective transformation, inputting the expected warped image, homography matrix, and the shape of the output image.

Further, the feature matching is performed on the image based on the ORB algorithm to obtain a feature descriptor, including:

detecting the image by using a FAST algorithm to obtain feature points;

and describing the feature points through a BRIEF algorithm to obtain feature descriptors.

Further, the feature point detecting the image by using the FAST algorithm includes:

selecting a pixel p, taking the pixel as a circular template to obtain a circular upper pixel, and comparing the difference value of the circular upper pixel and the pixel p;

calculating the difference value of four pixels, namely an upper pixel, a lower pixel, a left pixel and a right pixel, of the pixel p, if three pixels exceed a set threshold value, setting the pixel p as a candidate point p, otherwise, discarding the pixel p;

acquiring a circular clockwise outer pixel where the candidate point p is located, comparing the difference value between the clockwise outer pixel and the candidate point p, and if nine pixels exceed a set threshold value, setting the candidate point p as a characteristic point p;

suppressing non-maxima of the image:

calculating FAST score values at the feature points, judging that the difference value sum of clockwise outer pixels and central pixel points of each feature point is in a neighborhood taking the feature point p as the center, and judging that the difference value sum of clockwise outer pixels and central pixel points of each feature point is in the presence of a plurality of feature points after calculation;

if the characteristic point p is the largest response value in all the characteristic points in the neighborhood, reserving the value of the point, otherwise, discarding the value of the changed point; if only one characteristic point exists in the neighborhood, reserving the value of the point;

the score calculation formula is as follows:

where V denotes a score and t denotes a set threshold.

Further, the describing the feature points by the BRIEF algorithm to obtain feature descriptors includes:

n point pairs are selected in a set sampling mode in the key point Kp field through a BRIEF algorithm,

defining an operator T:

wherein P is a point pair, and V is pixel brightness;

combining the comparison results of the N point pairs to serve as a feature descriptor;

is described by T as an N-dimensional feature vector (T1, T2, T3, T4).

Further, before describing the feature points by the BRIEF algorithm to obtain the feature descriptors, the method further includes:

gradient direction is calculated by centroid algorithm:

first calculate movement (M ₀₀ M ₁₀ M ₀₁ )：

Obtaining a Q vector:

the rotation direction is regulated through the Q vector, the Q vector direction is considered when the BRIEF algorithm is calculated, and the direction is normalized when the BRIEF algorithm is calculated; when the image is rotated, the original BRIEF information is restored by the Q direction between the two images.

Further, the fast robust matching of the feature descriptors based on the ORB-GMS algorithm to obtain a homography matrix includes:

matching the feature descriptors through a GMS algorithm according to the obtained feature descriptors;

probability distribution for correct matches and errors in the correct matching neighborhood:

mathematical angle-inference is then expressed as a probability distribution over correct matches and errors in the correct match neighborhood:

m _true ＝knp _ture ,

m _false ＝knp _false ,

B(kn,p _true )x _i is a correct match; b (kn, p) _fales )x _i Is a false match;

the maximum probability P1 is determined:

the feature descriptors that are not matched are filtered out, and the feature descriptors that are not matched are reserved.

Further, after the obtaining the plurality of images to be spliced, the method further includes:

performing de-distortion correction on the image to obtain a corrected undistorted image;

and aligning and fusing a plurality of the undistorted images.

In a second aspect, an embodiment of the present application further provides an image matching apparatus based on ORB-GMS, including:

the image acquisition module is used for acquiring a plurality of images to be spliced;

the feature acquisition module is used for carrying out feature matching on the image based on an ORB algorithm to obtain a feature descriptor;

the feature matching module is used for carrying out fast robust matching on the feature descriptors based on an ORB-GMS algorithm to obtain homography matrixes;

and the image stitching module is used for stitching the images according to the perspective transformation, the expected distorted images, the homography matrix and the shape of the output image.

In a third aspect, an embodiment of the present application further provides a computer apparatus, including: a memory and one or more processors;

the memory is used for storing one or more programs;

the one or more programs, when executed by the one or more processors, cause the one or more processors to implement an ORB-GMS based image matching method as described above.

In a fourth aspect, embodiments of the present application also provide a storage medium containing computer-executable instructions, which when executed by a computer processor, are adapted to carry out an ORB-GMS based image matching method as described above.

According to the embodiment of the application, a plurality of images to be spliced are acquired; performing feature matching on the image based on an ORB algorithm to obtain a feature descriptor; carrying out fast robust matching on the feature descriptors based on an ORB-GMS algorithm to obtain homography matrixes; inputting the expected distorted image, the homography matrix and the shape of the output image according to perspective transformation, and splicing the images; and image matching is performed based on an ORB-GMS algorithm, so that the performance is high, the calculated amount is small, and the practicability is high.

Drawings

Fig. 1 is a flowchart of an image matching method based on ORB-GMS according to an embodiment of the present application;

FIG. 2 is a perspective transformation schematic diagram of an ORB-GMS-based image matching method according to an embodiment of the present application;

fig. 3 is a schematic structural diagram of an image matching device based on ORB-GMS according to an embodiment of the present application;

FIG. 4 is a schematic diagram of a computer device according to an embodiment of the present application;

fig. 5 is a diagram of a feature point matching algorithm in the present stage according to an embodiment of the present application.

Detailed Description

In order to make the objects, technical solutions and advantages of the present application more apparent, the following detailed description of specific embodiments of the present application is given with reference to the accompanying drawings. It is to be understood that the specific embodiments described herein are merely illustrative of the application and are not limiting thereof. It should be further noted that, for convenience of description, only some, but not all of the matters related to the present application are shown in the accompanying drawings. Before discussing exemplary embodiments in more detail, it should be mentioned that some exemplary embodiments are described as processes or methods depicted as flowcharts. Although a flowchart depicts operations (or steps) as a sequential process, many of the operations can be performed in parallel, concurrently, or at the same time. Furthermore, the order of the operations may be rearranged. The process may be terminated when its operations are completed, but may have additional steps not included in the figures. The processes may correspond to methods, functions, procedures, subroutines, and the like.

In an image stitching system, there is a need to face the problem of image fusion of multiple cameras. Before fusion begins, corrected undistorted images need to be obtained by de-distortion correction, and then the images are aligned and fused.

There is a need to focus on the alignment and fusion of non-distorted images. A general approach is: and performing feature point matching on the images to be spliced, and determining global deformation of the whole image by fitting the rotation, translation and scaling relation of the feature points. That is, image stitching is accomplished by determining parameters for rotation, translation, and scaling between multiple images through common features between the images. Therefore, whether the characteristic points are matched well or not directly influences the subsequent splicing work.

The feature Point is data obtained by sparsely representing texture content in an image, and comprises two parts, namely Key Point (Key Point) position detection and feature description.

Therefore, the necessary information of the feature point can be obtained by performing key point detection by taking a certain feature (such as a corner point) of the image as a criterion and then describing the vicinity of the coordinate designated by the point. In the feature point detection and description method, the most representative is a scale invariant feature transform (Scale Invariant Feature Transform), abbreviated as SIFT. Although references and studies to SIFT in the academy have reached a certain limit, the necessary references to SIFT are still needed here to visually compare other methods.

Firstly, the SIFT algorithm is a feature extraction algorithm capable of overcoming rotation, scaling and illumination (brightness) change of local features, and the algorithm can still ensure correct matching under the influence of affine transformation or visual angle change of an image and even other noise, namely, the SIFT algorithm has certain robustness. And secondly, the key point detection algorithm of SIFT is completed based on the extremum detection of the DoG space, a large number of candidate key point positions can be obtained, the scale of the key point is determined through the stability degree, and the direction histogram is used for obtaining the key point direction. It is to be noted that the SIFT algorithm satisfies most of the requirements for feature point detection based on the above operations:

1. has multiple characteristics, and can obtain richer key points in most cases.

2. Local feature descriptors can overcome most of the effects of rotation, scaling, illumination variation, and other noise.

The two characteristics make the SIFT algorithm appear earlier, but the SIFT algorithm is still not decayed in most fields (three-dimensional reconstruction, visual SLAM and image stitching) and is new for a long time.

Indeed, the SIFT algorithm is in a state of relatively balanced in terms of robustness and real-time performance, but with the continuous expansion of the application of the present vision field, more SIFT algorithm replacement has appeared, and the SIFT algorithm is emphasized from the balance point of the SIFT, or is completely surpassed by the SIFT algorithm, as shown in fig. 5.

The SURF, AKAZE, ORB algorithm is smaller in calculation amount than the SIFT algorithm, which means that the resolving speed is faster, but the robustness is reduced, and the algorithm is suitable for being implemented in real-time application or mobile terminal application, because the algorithm can obtain more characteristic points for matching and filtering in a shorter time. To a certain extent, more feature points means better robustness, the robustness requirement on each link is reduced in the whole flow, and further means that the algorithm can provide tolerable calculation time when being implemented on mobile equipment. But on the other side is a more robust algorithm, and due to some application limitations (such as SFM: three-dimensional scene reconstruction, etc.) no particularly many image samples can be obtained, or there is a requirement for precision, no mismatching of correlation is allowed to occur, so that more correct matching needs to be obtained, so that less loss is caused in subsequent filtering.

On the other hand, the A-SIFT and PCA-SIFT, CODE, LIFT algorithms all want to achieve an enhanced effect on algorithm performance by means of more complex and more robust means. The A-SIFT is a simple writing of the Affine-SIFT, and is a characteristic point matching algorithm designed through a refined thought, the core idea of the A-SIFT is an algorithm for obtaining robust characteristic points through multiple Affine simulation of images, the algorithm is a framework, and the algorithm can be realized through SURF, SIFT and other algorithms, but basically, the algorithm uses a very time-consuming thought. The algorithm can obtain the characteristic points with relatively high robustness, but is rough in thought, an ideal balance is not obtained in the expenditure of the calculation amount of a processor and the benefit of the practical robustness, and unfortunately, the algorithm needs a very large continuous memory section, namely the memory utilization rate of equipment is very high, the use efficiency is low, and the algorithm is not realistic to be transplanted to mobile equipment. The PCA-SIFT algorithm is used for Principal Component Analysis (PCA) algorithms in the fields like machine learning and the like, and has the effect of using the PCA to be more robust and representative in describing the features, and the essential idea is to improve the performance of the algorithm by improving the feature description quality. The algorithm uses a relatively linear idea in the process of describing the features, namely the features with higher quality are obtained by increasing dimensions, the SIFT algorithm uses 4 x 8 descriptors, PCA-SIFT directly obtains 3024-dimensional data through 39 x 2 gradient derivatives, and 36-dimensional descriptors are obtained through PCA. This approach allows feature points with less distortion to be obtained, but the time taken for principal component analysis is still considerable, and this calculation makes the actual time consumption of the algorithm more exaggerated as feature points increase.

The CODE is a comparatively front-edge algorithm, and is more effective especially for a scene with a large base line, that is, in panoramic stitching with a larger rotation angle, the result obtained by using the CODE algorithm is comparatively ideal. The CODE needs to be emphasized here, since the overall model of the algorithm can be redesigned to a computationally low-cost and robust algorithm, GMS, by a subjective, perceptual, a priori criterion.

It is first to be determined that the computational burden of such an algorithm is still significant: the main idea of the CODE is multiple filtering, and the standard and means of filtering are different. First, the CODE takes a common feature point matching result as an initial input, and then selects some better matching results. The better result is that fitting (regression) is carried out through Likelihood standard, then a classifier is designed through SVM thought, the incorrect point matching relation is filtered, and finally an affine transformation regression model is used to obtain the result.

The regression model of CODE can be understood simply as a statistic, and first, if the input result is three-dimensional, it can be analogized by plane fitting: the discrete points are used for fitting the plane, a plane is finally obtained, the data points belonging to the plane are the accepted correct points, and the points which are not on the plane are regarded as error points which need to be removed. The CODE Likelihood regression uses 8-dimensional data as an input point, which can be described by the following expression:

D(x,y,dx,dy,T1,T2,T3,T4)

wherein, (x, y) is the key point position of the reference frame, (dx, dy) is the difference of the (x, y) positions in the two images, and (T1, T2, T3, T4) is four transfer matrices specified in the algorithm, and finally, the result is scored through a Huber cost function, so that a correct result is obtained. The Affine regression, that is, the Affine transformation regression model, is performed later, and will not be described in detail herein.

By now it is apparent that the CODE model is a method that exceeds the computational complexity of a-SIFT while being more practical than PCA-SIFT and that can solve the problem of large baselines leading to erroneous estimates of both images.

For LIFT, the fashion method of deep learning the popular vocabulary is to take off the fetus. The core idea of the algorithm is that the feature point description of the feature point detection algorithm such as SIFT, ORB, SURF is based on manual design, that is, under the assumption that all feature point description means are traversed, a more efficient feature point description operator exists finally. The LIFT algorithm is totally called Learned Invariant Feature Points, and as the name implies, is a feature point detection and description method based on machine learning (or called deep learning), and is also a complete deep learning attempt: the algorithm uses three Convolutional Neural Networks (CNNs) in total, and the three steps of key point detection, gradient direction estimation and feature sub-description are respectively applied. However, the algorithm is trained without considering the benefit brought by diversity to the performance of the deep neural network, unfortunately, SIFT output data is used as a training sample, and although three-dimensional reconstruction results are used to ensure that input data is correct, input diversity is limited, and the sample obtained by filtering SIFT algorithm output is necessarily one-sided, so that the effect of the algorithm is not breakthrough high performance. In addition, the algorithm has the more serious defect that three convolutional neural networks are used simultaneously, so that the calculation amount is difficult to control from the detection of key points, and the calculation burden of the algorithm is frosted on snow by the network of the two remaining feature sub-descriptions. The method has the effective significance of obtaining the characteristic points with the highest quality as possible, reducing the difficulty of filtering after matching, but if more (for example, 1000) key points are generated by key point detection, the calculated amount of the algorithm is directly caused to explode, and the significance of three-dimensional reconstruction with high quality is lost. Furthermore, the structure of using CNNs in series also results in the algorithm failing to develop its high performance characteristics in the field of image stitching.

The left side of fig. 5 is a lightweight algorithm with lower calculation amount and more mismatching rate, and the right side is a robust algorithm with high calculation amount and low mismatching rate realized by means of multiple iterations, high-dimensional feature description and the like. By researching the fitting thought of the CODE and lightweight algorithms such as ORB, AKAZE and the like, the feature point matching can be found to obtain a more effective feature point matching scheme by adjusting the quantity and optimizing the filtering method.

Obviously, faster algorithms can generate denser feature points in a shorter time, but some feature point description performance is reduced; the better feature points can be achieved by a complex algorithm, but the calculation load is too large, and the practicability is greatly reduced.

The embodiment of the application establishes a set of image matching method based on ORB-GMS, and solves the problems of overlarge calculation load and poor practicability of the existing image feature point algorithm.

The ORB-GMS based image matching method provided in the embodiments may be performed by an ORB-GMS based image matching device, which may be implemented in software and/or hardware and integrated in an ORB-GMS based image matching apparatus. The image matching device based on ORB-GMS may be a computer or the like.

Fig. 1 is a flowchart of an image matching method based on ORB-GMS according to an embodiment of the present application. Referring to fig. 1, the method comprises the steps of:

step 110, acquiring a plurality of images to be spliced.

Optionally, after a plurality of images to be spliced are acquired, performing de-distortion correction on the images to obtain corrected undistorted images; and aligning and fusing a plurality of the undistorted images.

And 120, performing feature matching on the image based on an ORB algorithm to obtain a feature descriptor.

Specifically, feature points are obtained by detecting the image through a FAST algorithm; calculating the gradient direction through a centroid algorithm; and describing the feature points through a BRIEF algorithm to obtain feature descriptors.

Optionally, the detecting the image by using a FAST algorithm to obtain the feature point includes:

suppressing non-maxima of the image:

calculating FAST score values at the feature points, judging that if a plurality of feature points exist in a neighborhood (such as 3x3 or 5x 5) taking the feature point p as the center, and judging the sum of absolute values of differences between clockwise outer pixels and central pixels of each feature point;

the score calculation formula is as follows:

where V denotes a score and t denotes a set threshold.

Optionally, the describing the feature points by the BRIEF algorithm to obtain feature descriptors includes:

defining an operator T:

wherein P is a point pair, and V is pixel brightness;

is described by T as an N-dimensional feature vector (T1, T2, T3, T4).

Illustratively, the actual BRIEF uses 512-dimensional feature vectors, described by T as n=4-dimensional feature vectors.

Optionally, the calculating the gradient direction by the centroid algorithm includes:

first calculate movement (M ₀₀ M ₁₀ M ₀₁ )：

Obtaining a Q vector:

It can be understood that the ORB obtains the key point location through the FAST algorithm and obtains the feature descriptors through the modified BRIEF feature description algorithm. Rotation invariance is primarily dependent on gradient direction, which the ORB calculates before BRIEF description.

And 130, carrying out fast robust matching on the feature descriptors based on an ORB-GMS algorithm to obtain a homography matrix.

Specifically, the GMS is known as Grid Motion Statistics and can be directly translated into a grid-based motion statistics method. The grid-based motion statistical algorithm can obtain a large baseline effect of a comparable CODE and has low calculation amount, thereby providing a key point with higher quality for searching the estimation of the homography matrix. Through GMS, a combination algorithm which is denser than SIFT and shorter in CPU calculation time can be obtained under the condition of ensuring high speed: i.e. feature point matching based on ORB-GMS.

The subjective idea of GMS is to summarize the multiple regression of CODE: i.e. the CODE considers that a seemingly irregular match is a false match. That is, the matching result should appear continuous in motion for the motion relationship of the two images. The GMS uses this idea to consider these successive matches as close to correct matches.

Motion estimation matching method based on GMS priori:

the subjective idea of GMS is that the smooth (clean) degree of Motion, the Motion estimation around a point is all counted, and thus by means of statistics it is determined whether the match is smooth: the Motion that appears smooth is the correct match, and the Motion that is abrupt is the wrong match. The calculation principle of the GMS is that matching statistics is carried out on the vicinity (neighborhood) of two points in a single matching pair, and the statistical significance is that the number of matching points in the technical field is used for judging whether matching is correct or not.

The GMS algorithm is based on two assumptions:

motion smoothing allows surrounding neighbors to be co-located, and as such, mismatching with non-identical 3D locations; assuming a match is wrong, its nearest match may be at any of the M possible locations.

m _true ＝knp _ture ,

m _false ＝knp _false ,

B(kn,p _true )x _i is a correct match; b (kn, p) _fales )x _i Is a false match;

the maximum probability P1 is determined:

And 140, inputting the expected distorted image, the homography matrix and the shape of the output image according to perspective transformation, and splicing the images.

Alternatively, by obtaining the homography matrix H as described above, two images need to be stitched together; the image to be warped, the homography matrix H, and also the shape of the output image are input using perspective transformation, and the derived shape of the output image is determined by taking the sum of the widths of the two images and then using the height of the second image. The perspective transformation (Perspective Transformation) is to project the picture to a new Viewing Plane, also called projection map (Projective Mapping). The perspective transformation is as in fig. 2.

On the basis of the above embodiment, please refer to fig. 3, an image matching device based on ORB-GMS according to an embodiment of the present application specifically includes: an image acquisition module 201, a feature acquisition module 202, a feature matching module 203, and an image stitching module 204.

The image acquisition module 201 is used for acquiring a plurality of images to be spliced; the feature acquisition module 202 is configured to perform feature matching on the image based on an ORB algorithm to obtain a feature descriptor; the feature matching module 203 is configured to perform fast robust matching on the feature descriptors based on an ORB-GMS algorithm to obtain a homography matrix; the image stitching module 204 is configured to stitch the images according to the perspective transformation, the expected distorted image, the homography matrix, and the shape of the output image.

The plurality of images to be spliced are acquired; performing feature matching on the image based on an ORB algorithm to obtain a feature descriptor; carrying out fast robust matching on the feature descriptors based on an ORB-GMS algorithm to obtain homography matrixes; inputting the expected distorted image, the homography matrix and the shape of the output image according to perspective transformation, and splicing the images; and image matching is performed based on an ORB-GMS algorithm, so that the performance is high, the calculated amount is small, and the practicability is high.

The image matching device based on ORB-GMS provided by the embodiment of the application can be used for executing the image matching method based on ORB-GMS provided by the embodiment, and has corresponding functions and beneficial effects.

The embodiment of the application also provides computer equipment which can integrate the ORB-GMS-based image matching device provided by the embodiment of the application. Fig. 4 is a schematic structural diagram of a computer device according to an embodiment of the present application. Referring to fig. 4, the computer apparatus includes: an input device 53, an output device 54, a memory 52, and one or more processors 51; the memory 52 is configured to store one or more programs; the one or more programs, when executed by the one or more processors 51, cause the one or more processors 51 to implement an ORB-GMS based image matching method as provided by the above embodiments. Wherein the input device 53, the output device 54, the memory 52 and the processor 51 may be connected by a bus or otherwise, for example in fig. 4 by a bus connection.

The processor 51 performs various functional applications of the device and data processing, i.e., implements the ORB-GMS based image matching method described above, by running software programs, instructions and modules stored in the memory 52.

The computer equipment provided by the embodiment can be used for executing the image matching method based on ORB-GMS, and has corresponding functions and beneficial effects.

The embodiments of the present application also provide a storage medium containing computer-executable instructions, which when executed by a computer processor, are for performing an ORB-GMS based image matching method comprising: acquiring a plurality of images to be spliced; performing feature matching on the image based on an ORB algorithm to obtain a feature descriptor; carrying out fast robust matching on the feature descriptors based on an ORB-GMS algorithm to obtain homography matrixes; the images are stitched according to the perspective transformation, inputting the expected warped image, homography matrix, and the shape of the output image.

Storage media-any of various types of memory devices or storage devices. The term "storage medium" is intended to include: mounting media such as CD-ROM, floppy disk or tape devices; computer device memory or random access memory such as DRAM, DDR RAM, SRAM, EDO RAM, lanbas (Rambus) RAM, etc.; nonvolatile memory such as flash memory, magnetic media (e.g., hard disk or optical storage); registers or other similar types of memory elements, etc. The storage medium may also include other types of memory or combinations thereof. In addition, the storage medium may be located in a first computer device in which the program is executed, or may be located in a different second computer device connected to the first computer device through a network (such as the internet). The second computer means may provide program instructions to the first computer for execution. The term "storage medium" may include two or more storage media that may reside in different locations (e.g., in different computer devices connected by a network). The storage medium may store program instructions (e.g., embodied as a computer program) executable by one or more processors.

Of course, the storage medium containing the computer executable instructions provided in the embodiments of the present application is not limited to the image matching method based on ORB-GMS as described above, and may also perform the related operations in the image matching method based on ORB-GMS provided in any embodiment of the present application.

The image matching device, the storage medium and the computer device based on ORB-GMS provided in the foregoing embodiments may perform the image matching method based on ORB-GMS provided in any embodiment of the present application, and technical details not described in detail in the foregoing embodiments may be referred to the image matching method based on ORB-GMS provided in any embodiment of the present application.

The foregoing description is only of the preferred embodiments of the application and the technical principles employed. The present application is not limited to the specific embodiments described herein, but is capable of numerous modifications, rearrangements and substitutions as will now become apparent to those skilled in the art without departing from the scope of the application. Therefore, while the application has been described in connection with the above embodiments, the application is not limited to the embodiments, but may be embodied in many other equivalent forms without departing from the spirit of the application, the scope of which is set forth in the following claims.

Claims

1. An ORB-GMS based image matching method, comprising the steps of:

acquiring a plurality of images to be spliced;

2. The ORB-GMS based image matching method according to claim 1, wherein the performing feature matching on the image based on the ORB algorithm to obtain feature descriptors comprises:

detecting the image by using a FAST algorithm to obtain feature points;

3. The ORB-GMS based image matching method according to claim 2, wherein the feature points detected from the image by FAST algorithm comprises:

suppressing non-maxima of the image:

the score calculation formula is as follows:

where V denotes a score and t denotes a set threshold.

4. The ORB-GMS based image matching method according to claim 2, wherein the describing the feature points by the BRIEF algorithm to obtain feature descriptors includes:

defining an operator T:

wherein P is a point pair, and V is pixel brightness;

is described by T as an N-dimensional feature vector (T1, T2, T3, T4).

5. The ORB-GMS based image matching method according to claim 4 wherein the describing the feature points by the BRIEF algorithm further comprises, before obtaining the feature descriptors:

gradient direction is calculated by centroid algorithm:

first calculate movement (M ₀₀ M ₁₀ M ₀₁ )：

Obtaining a Q vector:

6. The ORB-GMS based image matching method according to claim 1, wherein the ORB-GMS based algorithm performs fast robust matching on the feature descriptors to obtain a homography matrix, comprising:

B(kn,p _true )x _i is a correct match; b (kn, p) _fales )x _i Is a false match;

the maximum probability P1 is determined:

7. The ORB-GMS based image matching method according to claim 1, wherein after the capturing the plurality of images to be stitched, further comprising:

and aligning and fusing a plurality of the undistorted images.

8. An ORB-GMS based image matching device comprising:

9. A computer device, comprising: a memory and one or more processors;

the memory is used for storing one or more programs;

when the one or more programs are executed by the one or more processors, the one or more processors are caused to implement an ORB-GMS based image matching method according to any of claims 1 to 8.

10. A storage medium containing computer executable instructions which, when executed by a computer processor, are for performing an ORB-GMS based image matching method according to any of claims 1 to 8.