CN106683046A - Real-time image splicing method for police unmanned aerial vehicle investigation and evidence obtaining - Google Patents

Abstract

The invention discloses a real-time image splicing method for police unmanned aerial vehicle investigation and evidence obtaining. The method includes the three steps of improved ORB algorithm operation, image registration and image fusion. According to the method, a multi-scale space is constructed; a significance analysis model is adopted to obtain an optimal angle point detection threshold value, and feature points are extracted; an ORB descriptor is adopted to describe the feature points; and a Hamming distance and RANSAC-combined method is adopted to achieve fast matching. As indicated by results, an improved ORB algorithm maintains the superiority of speed, and the matching rate of the improved ORB algorithm is improved as for images on which scale, angle of view, rotation and illumination change occur.

Description

Image real-time splicing method for police unmanned aerial vehicle reconnaissance and evidence obtaining

Technical Field

The invention belongs to the technical field of police unmanned aerial vehicle shooting and evidence obtaining, and particularly relates to a real-time image splicing method for reconnaissance and evidence obtaining of a police unmanned aerial vehicle.

Background

In recent years, the increasing rate of computer crime at home and abroad poses serious threats to national security and social security, serious losses to legal public and private property, and new challenges and requirements on computer evidence-taking technology. Computer evidence collection is used as a new research field, and has great significance for crime fighting and social stability maintenance. The rotary wing type unmanned aerial vehicle is applied to reconnaissance and evidence obtaining, has the advantages of fast response, high real-time performance, real and reliable images and the like, and can effectively solve the problems of insufficient reconnaissance and evidence obtaining means and low efficiency. However, as the aerial image of the unmanned aerial vehicle has a large amount of information and a plurality of visual angles, certain challenges are brought to subsequent information analysis.

The unmanned aerial vehicle is used for reconnaissance and evidence obtaining, and in order to timely and accurately reflect the field situation, the obtained images need to be spliced in real time. The accuracy and efficiency of image feature point matching affect the quality of image stitching. At present, many algorithms are applied to matching of image feature points. The SIFT algorithm is used as a classic feature point matching algorithm, although the matching accuracy is high, the calculation complexity is high, and the real-time requirement cannot be met. Then Bay et al improve it and propose an algorithm for SURF feature point extraction. In recent years, many new feature point matching algorithms, such as BRIEF, ORB, BRISK, FREAK, etc., have emerged.

ORB is an algorithm based on FAST feature extraction and BRIEF feature description, has the advantage of high speed, but does not have scale invariance, and in the feature extraction stage, the algorithm threshold is selected fixedly, and the difference of significant features between images is not considered. Therefore, the method has extremely strong application value to the improvement of the ORB algorithm.

Disclosure of Invention

The invention aims at the problems and provides a real-time image splicing method for police unmanned aerial vehicle reconnaissance and evidence collection.

In order to achieve the technical purpose, the invention adopts a real-time image splicing method for police unmanned aerial vehicle reconnaissance and evidence obtaining, which comprises three steps of improved ORB algorithm, image registration and image splicing;

the improved ORB algorithm includes: the ORB is improved, a significant model based on space-frequency domain analysis is applied to the selection of the optimal threshold value in the characteristic extraction stage by combining with a KSW entropy method, and a multi-scale space is constructed by utilizing a Gaussian pyramid;

an image I is input, first transformed from the RGB color space to the CIE Lab color space, where the image contains three channels: a brightness channel (L channel) and two color channels (a channel and b channel), for the color channels, eliminating fine texture details by adopting Gaussian blur in a time domain to obtain a characteristic diagram of the corresponding channel; for the brightness channel, calculating by adopting a high-frequency enhanced Butterworth high-pass filter to obtain a feature map of the L channel, and finally combining the feature maps of all the channels together to form a saliency map of an original image;

any input image can be represented by a magnitude spectrum and a phase spectrum, wherein the phase spectrum contains texture detail information of the image, the magnitude spectrum contains light and shade contrast information, if only the phase spectrum is reserved, the obtained image significant features contain partial background information and interfere the detection of image feature points to cause a mismatching phenomenon, a high-pass filter can realize the purpose of sharpening the edge of a target and simultaneously reserve the edge information to the maximum extent by attenuating and inhibiting low-frequency components, and in order to enhance the image detail information, sharpen the edge of the target and reduce noise interference, a high-frequency enhanced Butterworth high-pass filter is adopted;

cut-off frequency of D₀The nth order butterworth high pass filter is defined as:

wherein,representing the frequency midpoint (u, v) and the frequency rectangle centerThe distance of (d);

inputting an image I, combining feature maps of three channels L, a and b together to obtain a final significant feature SM which is:

SM(x,y)＝||I_L-I_L(x,y)||+||I_a-I_a(x,y)||+||I_b-I_b(x,y)||

wherein, I_L(x, y) is the corresponding characteristic image pixel value of the brightness channel of the original image after passing through a high-frequency enhanced Butterworth high-pass filter, I_a(x,y),I_b(x, y) are respectively the corresponding characteristic map pixel values of the color channels, I_L,I_a,I_bRespectively, the average feature vectors of the corresponding channel images,is a two-norm euclidean distance;

the selection of the threshold value should reasonably change along with the change of the image gray scale characteristics, and the selection of the optimal threshold value is determined by combining a KSW entropy method according to the image significant characteristics, and the specific steps are as follows:

let the image gray scale range be [0, L-1]The threshold t divides the data into A, B categories, and the corresponding probability distributions are { p } respectively₀,p₁,p₂,...,p_t},{p_t+1,p_t+2,p₂,...,p_L-1In which p is_iFor the frequency of occurrence of the corresponding gray level, orderThen the entropy for A, B classes would be:

the total entropy of the image is H ═ H_A+H_B. The optimal threshold value T is

Where k is a scaling factor, since the feature point threshold is related to the pixel contrast of the image, it is not necessary to determine the scale factor

Determining an image gray level difference, namely an optimal threshold value, through image salient features and an entropy method;

extracting feature points by using a best threshold obtained by combining a Fast corner detection algorithm, and describing the feature points by using an rBRIEF descriptor for subsequent image registration;

the image registration comprises the steps of utilizing feature point matching and utilizing RANSAC algorithm to screen matching points;

the characteristic point matching is used for searching two characteristic points with the shortest distance in two groups of characteristic point sets by using a distance function, and the distance between two binary descriptors can be represented by using a Hamming distance, wherein the Hamming distance refers to the number of characters with different corresponding positions between two character strings with the same length. The smaller the Hamming distance is, the more similar the two binary descriptors are;

calculating the shortest Hamming distance and the next shortest Hamming distance of each feature point to obtain a group of feature point matching pairs, and considering that the two feature points are matched when the ratio of the shortest distance to the next shortest distance is less than a threshold value;

the method comprises the steps of selecting a certain number of samples at random to estimate model parameters by screening matching points by using a RANSAC algorithm, classifying the rest data according to the estimated parameters, determining that one part of data is within an allowable error range, namely an inner point, and removing an error matching point pair through multiple hypothesis verification if the part of data is an outer point;

the RANSAC algorithm needs to use a homography matrix H which describes the transformation relation between the coordinates of two image points, including translation, rotation, scaling and the like, and can find the position of a point in one image in the other image through the matrix H, and assume a pair of matching points p in an image 1 and an image 2₁(x,y)，p₂The transformation relationship between (x ', y') is:

8 parameters of the matrix H can be calculated by 4 pairs of matching points, and the RANSAC algorithm comprises the following steps:

(1) setting an initial value of iteration times as 0, a maximum iteration time N, an internal point number threshold T1 and an error threshold T2;

(2) randomly selecting 4 pairs from the n pairs of points to be matched, and calculating parameters of a transformation matrix H between the two images;

(3) calculating the distance between the coordinate values of the other feature points after H transformation and the matching points of the feature points, if the distance is smaller than an error threshold value T2, considering the matching point pair as an inner point, and if the distance is not smaller than the error threshold value T2, calculating the number of the inner points;

(4) if the number of inliers is greater than the threshold number of inliers T1, the current model is saved as the optimal model. Otherwise, adding one to the iteration times, and turning to the step (2) to continue the next iteration;

(5) if the maximum iteration number N is reached, returning a group of interior points with the maximum number of corresponding interior points, and obtaining a transformation matrix H;

the image stitching means: after the images are registered, the images are spliced through image fusion, wherein the image fusion is the last step of image splicing and mainly comprises two parts: merging the images and eliminating image splicing lines, wherein the merging of the images is to eliminate redundant pixel information in the overlapped area and align the images to be spliced according to the image registration result; eliminating image splicing lines and performing weighted average fusion processing near the splicing lines; the weighted average weighting function can use a gradual-in gradual-out method, the complexity is low, the speed is high, and smooth transition of images can be realized in the overlapping area. The gradual-in gradual-out method is to calculate a weight according to the distance from a pixel to be fused to the boundary of a coincidence region, the weight is changed linearly, and a fusion formula is as follows:

in the formula (d)₁、d₂Representing the weight of the pixel point (x, y) on the image corresponding to the overlapped part, and meeting the following conditions: d₁+d₂＝1,0＜d₁,d₂＜1；

d₁、d₂The calculation formula of (2) is as follows:

wherein x_iIs the abscissa, x, of the pixel point to be fused_i、x_rRespectively are the horizontal coordinates of the left and right borders of the image overlapping area;

the method comprises the steps of firstly constructing a multi-scale space, obtaining an optimal corner detection threshold value by utilizing a significance analysis model, extracting feature points, describing the feature points by utilizing an ORB descriptor, and finally realizing rapid matching by utilizing a Hamming distance and RANSAC method.

Drawings

FIG. 1 shows an algorithm flow diagram of the present invention;

FIG. 2 is a graph showing the gradual-in and gradual-out weight variation;

Detailed Description

The invention is further illustrated with reference to the following figures and detailed description.

With reference to fig. 1, first, it is understood what is called the ORB algorithm, which combines the FAST feature point detection method with the BRIEF feature descriptor, and performs improvement and optimization, and the present invention mainly introduces two parts, feature point detection and feature point description, of the algorithm.

1) Feature point detection

The ORB algorithm uses a gaussian pyramid structure and calculates its principal direction for each feature point, so that the detected feature points have scale invariance and rotation invariance.

(1) Firstly, establishing a scale space, constructing an image pyramid, and only one image is arranged on each layer, which is different from SIFT.

(2) Calculating the number n of feature points to be extracted of each layer according to a formula, detecting the feature points on images with different scales by using a FAST algorithm, sequencing according to FAST corner response values, reserving the first 2n points, then calculating Harris corner response values of the feature points, sequencing, and reserving the first n points as the feature points of the layer.

(3) The principal direction of the feature points is calculated. ORB proposes a gray centroid method, i.e. there is an offset between the gray of a corner point and the centroid in its neighborhood, and this vector is taken as the direction of the feature point.

Defining the moment of the neighborhood S of any one feature point p as:

where I (x, y) is the gray value at point (x, y).

The centroid of the neighborhood S is:

the included angle between the characteristic point and the mass center is defined as the main direction of the characteristic point: θ ═ arctan (M)_0,1/M_1,0)

2) Description of characteristic points

The ORB algorithm improves the BRIEF descriptor, namely, the rBRIEF description method enables the descriptor to have rotation invariance. The BRIEF descriptor is essentially a binary code string with the length of m, m point pairs are selected around the characteristic points, the gray value of each point pair is compared, and the descriptor is coded into a binary form.

A binary comparison criterion function τ is defined as:

where p (x) is the gray value at x in the neighborhood. In order to remove noise interference, the ORB algorithm selects a 5 × 5 image block at a feature point, and replaces the gray value of the feature point with the average gray value of the image block after smoothing.

Selecting m point pairs near the feature points, and comparing to obtain a binary string with the length of m as a feature descriptor:

the ORB algorithm uses the above calculated principal directions of feature points to determine the direction of a feature descriptor in order to make the descriptor rotationally invariant. The m point pairs around the feature point are combined into a matrix S:

defining a rotation matrix corresponding to the characteristic point direction theta as R_θCharacteristic point pair matrix S corresponding to direction theta_θ＝R_θAnd S. Wherein,θ is the principal direction of the feature point.

The feature descriptors after determining the direction are: g_m(p,θ)＝f_m(p)|(x_i,y_i)∈S_θ

In order to improve the discrimination performance of the descriptors, the ORB uses greedy search to select 256 test point pairs with the largest variance and the lowest correlation from all possible binary tests to form the required feature descriptors.

On the basis, the invention discloses a real-time image splicing method for reconnaissance and evidence obtaining of an unmanned aerial vehicle for police, which comprises three steps of improved ORB algorithm, image registration and image fusion;

the improved ORB algorithm includes: the ORB is improved, a significant model based on space-frequency domain analysis is applied to the selection of the optimal threshold value in the characteristic extraction stage by combining with a KSW entropy method, and a multi-scale space is constructed by utilizing a Gaussian pyramid;

an image I is input, first transformed from the RGB color space to the CIE Lab color space, where the image contains three channels: a brightness channel (L channel) and two color channels (a channel and b channel), for the color channels, eliminating fine texture details by adopting Gaussian blur in a time domain to obtain a characteristic diagram of the corresponding channel; for the brightness channel, calculating by adopting a high-frequency enhanced Butterworth high-pass filter to obtain a feature map of the L channel, and finally combining the feature maps of all the channels together to form a saliency map of an original image;

any input image can be represented by a magnitude spectrum and a phase spectrum, wherein the phase spectrum contains texture detail information of the image, the magnitude spectrum contains light and shade contrast information, if only the phase spectrum is reserved, the obtained image significant features contain partial background information and interfere the detection of image feature points to cause a mismatching phenomenon, a high-pass filter can realize the purpose of sharpening the edge of a target and simultaneously reserve the edge information to the maximum extent by attenuating and inhibiting low-frequency components, and in order to enhance the image detail information, sharpen the edge of the target and reduce noise interference, a high-frequency enhanced Butterworth high-pass filter is adopted;

cut-off frequency of D₀The nth order butterworth high pass filter is defined as:

wherein,representing the frequency midpoint (u, v) and the frequency rectangle centerThe distance of (d);

inputting an image I, combining feature maps of three channels L, a and b together to obtain a final significant feature SM which is:

SM(x,y)＝||I_L-I_L(x,y)||+||I_a-I_a(x,y)||+||I_b-I_b(x,y)||

wherein, I_L(x, y) is the corresponding characteristic image pixel value of the brightness channel of the original image after passing through a high-frequency enhanced Butterworth high-pass filter, I_a(x,y),I_b(x, y) are respectively the corresponding characteristic map pixel values of the color channels, I_L,I_a,I_bRespectively, the average feature vectors of the corresponding channel images,is a two-norm euclidean distance;

the selection of the threshold value should reasonably change along with the change of the image gray scale characteristics, and the selection of the optimal threshold value is determined by combining a KSW entropy method according to the image significant characteristics, and the specific steps are as follows:

let the image gray scale range be [0, L-1]The threshold t divides the data into A, B categories, and the corresponding probability distributions are { p } respectively₀,p₁,p₂,...,p_t},{p_t+1,p_t+2,p₂,...,p_L-1In which p is_iFor the frequency of occurrence of the corresponding gray level, orderThen the entropy for A, B classes would be:

the total entropy of the image is H ═ H_A+H_B. The optimal threshold value T is

K is a proportionality coefficient, and since the threshold of the feature point is related to the pixel contrast of the image, the gray level difference of the image, namely the optimal threshold, is determined by the significant feature of the image and an entropy method;

extracting feature points by using a best threshold obtained by combining a Fast corner detection algorithm, and describing the feature points by using an rBRIEF descriptor for subsequent image registration;

the image registration comprises the steps of utilizing feature point matching and utilizing RANSAC algorithm to screen matching points;

the characteristic point matching is used for searching two characteristic points with the shortest distance in two groups of characteristic point sets by using a distance function, and the distance between two binary descriptors can be represented by using a Hamming distance, wherein the Hamming distance refers to the number of characters with different corresponding positions between two character strings with the same length. The smaller the Hamming distance is, the more similar the two binary descriptors are;

calculating the shortest Hamming distance and the next shortest Hamming distance of each feature point to obtain a group of feature point matching pairs, and considering that the two feature points are matched when the ratio of the shortest distance to the next shortest distance is less than a threshold value;

the method comprises the steps of selecting a certain number of samples at random to estimate model parameters by screening matching points by using a RANSAC algorithm, classifying the rest data according to the estimated parameters, determining that one part of data is within an allowable error range, namely an inner point, and removing an error matching point pair through multiple hypothesis verification if the part of data is an outer point;

the RANSAC algorithm needs to use a homography matrix H which describes the transformation relation between the coordinates of two image points, including translation, rotation, scaling and the like, and can find the position of a point in one image in the other image through the matrix H, and assume a pair of matching points p in an image 1 and an image 2₁(x,y)，p₂The transformation relationship between (x ', y') is:

8 parameters of the matrix H can be calculated by 4 pairs of matching points, and the RANSAC algorithm comprises the following steps:

(1) setting an initial value of iteration times as 0, a maximum iteration time N, an internal point number threshold T1 and an error threshold T2;

(2) randomly selecting 4 pairs from the n pairs of points to be matched, and calculating parameters of a transformation matrix H between the two images;

(3) calculating the distance between the coordinate values of the other feature points after H transformation and the matching points of the feature points, if the distance is smaller than an error threshold value T2, considering the matching point pair as an inner point, and if the distance is not smaller than the error threshold value T2, calculating the number of the inner points;

(4) if the number of inliers is greater than the threshold number of inliers T1, the current model is saved as the optimal model. Otherwise, adding one to the iteration times, and turning to the step (2) to continue the next iteration;

(5) if the maximum iteration number N is reached, returning a group of interior points with the maximum number of corresponding interior points, and obtaining a transformation matrix H;

the image stitching means: after the images are registered, the images are spliced through image fusion, wherein the image fusion is the last step of image splicing and mainly comprises two parts: merging the images and eliminating image splicing lines, wherein the merging of the images is to eliminate redundant pixel information in the overlapped area and align the images to be spliced according to the image registration result; eliminating image splicing lines and performing weighted average fusion processing near the splicing lines; the weighted average weighting function can use a gradual-in gradual-out method, the complexity is low, the speed is high, and smooth transition of images can be realized in the overlapping area. The gradual-in gradual-out method is to calculate a weight according to the distance from a pixel to be fused to the boundary of a coincidence region, the weight is changed linearly, and a fusion formula is as follows:

in the formula (d)₁、d₂Representing the weight of the pixel point (x, y) on the image corresponding to the overlapped part, and meeting the following conditions: d₁+d₂＝1,0＜d₁,d₂＜1；

d₁、d₂The calculation formula of (2) is as follows:

wherein x_iIs the abscissa, x, of the pixel point to be fused_i、x_rRespectively, the abscissa of the left and right borders of the image overlapping area.

As shown in fig. 2, d1 changes gradually from 1 to 0, and the corresponding d2 changes from 0 to 1, so that a smooth transition is achieved in the overlapping region of the images.

The method comprises the steps of firstly constructing a multi-scale space, obtaining an optimal corner detection threshold value by utilizing a significance analysis model, extracting feature points, describing the feature points by utilizing an ORB descriptor, and finally realizing rapid matching by utilizing a Hamming distance and RANSAC method.

Table 1 and table 2 show the comparison results of the registration rate and registration time of the improved ORB algorithm herein with BRISK algorithm, FAST-ORB algorithm, respectively. The FAST-ORB algorithm refers to an image feature point matching algorithm which is subjected to feature extraction by the FAST algorithm and is described by an ORB descriptor. The improved ORB algorithm herein has the advantage of high registration rate compared to other algorithms. The comparison result of the time used by each algorithm is shown, and it can be seen that the registration time is slightly higher than that of the other two algorithms due to the addition of the step of significance analysis in the algorithm, but is almost the same, and the advantage of high speed is maintained. The experimental result shows that the improved ORB algorithm keeps the superiority of speed, and the matching rate of the image with changes of scale, rotation, visual angle, illumination and the like is improved.

TABLE 1 image registration ratio comparison results

TABLE 2 image registration time comparison results

Claims (1)

1. The image real-time splicing method for police unmanned aerial vehicle reconnaissance and evidence collection is characterized by comprising three steps of improved ORB algorithm, image registration and image fusion;

the improved ORB algorithm includes: the ORB is improved, a significant model based on space-frequency domain analysis is applied to the selection of the optimal threshold value in the characteristic extraction stage by combining with a KSW entropy method, and a multi-scale space is constructed by utilizing a Gaussian pyramid;

an image I is input, first transformed from RGB color space to CIELab color space, the image containing three channels in Lab color space: a brightness channel (L channel) and two color channels (a channel and b channel), for the color channels, eliminating fine texture details by adopting Gaussian blur in a time domain to obtain a characteristic diagram of the corresponding channel; for the brightness channel, calculating by adopting a high-frequency enhanced Butterworth high-pass filter to obtain a feature map of the L channel, and finally combining the feature maps of all the channels together to form a saliency map of an original image;

any input image can be represented by a magnitude spectrum and a phase spectrum, wherein the phase spectrum contains texture detail information of the image, the magnitude spectrum contains light and shade contrast information, if only the phase spectrum is reserved, the obtained image significant features contain partial background information and interfere the detection of image feature points to cause a mismatching phenomenon, a high-pass filter can realize the purpose of sharpening the edge of a target and maximally reserve edge information by attenuating and inhibiting low-frequency components, and in order to enhance the image detail information, sharpen the edge of the target and reduce noise interference, a high-frequency enhanced Butterworth high-pass filter is adopted;

cut-off frequency of D₀The nth order butterworth high pass filter is defined as:

$H(u,v)=\frac{1}{1+{\[D_0/D(u,v)\]}^{2n}}$

wherein,representing the frequency midpoint (u, v) and the frequency rectangle centerThe distance of (d);

inputting an image I, combining feature maps of three channels L, a and b together to obtain a final significant feature SM which is:

SM(x,y)＝||I_L-I_L(x,y)||+||I_a-I_a(x,y)||+||I_b-I_b(x,y)||

wherein, I_L(x, y) is the corresponding characteristic image pixel value of the brightness channel of the original image after passing through a high-frequency enhanced Butterworth high-pass filter, I_a(x,y),I_b(x, y) are respectively the corresponding characteristic map pixel values of the color channels, I_L,I_a,I_bRespectively, the average feature vectors of the corresponding channel images,is a two-norm euclidean distance;

the selection of the threshold value should reasonably change along with the change of the image gray scale characteristics, and the selection of the optimal threshold value is determined by combining a KSW entropy method according to the image significant characteristics, and the specific steps are as follows:

let the image gray scale range be [0, L-1]The threshold t divides the data into A, B categories, and the corresponding probability distributions are { p } respectively₀,p₁,p₂,...,p_t},{p_t+1,p_t+2,p₂,...,p_L-1In which p is_iFor the frequency of occurrence of the corresponding gray level, orderThen the entropy for A, B classes would be:

$H_A=-\underset{i=0}{\overset{t}{\Σ}}\frac{p_i}{P_i}ln\frac{p_i}{P_i}$

$H_B=-\underset{i=t+1}{\overset{L-1}{\Σ}}\frac{p_i}{1-P_i}ln\frac{p_i}{1-P_i}$

the total entropy of the image is H ═ H_A+H_B. The optimal threshold value T is

$T=\frac{1}{k}*|Arg\underset{0\≤t\≤L-1}{max}H-Arg\underset{0\≤t\≤L-1}{\min }H|$

K is a proportionality coefficient, and since the threshold of the feature point is related to the pixel contrast of the image, the gray level difference of the image, namely the optimal threshold, is determined by the significant feature of the image and an entropy method;

extracting feature points by using a best threshold obtained by combining a Fast corner detection algorithm, and describing the feature points by using an rBRIEF descriptor for subsequent image registration;

the image registration comprises the steps of utilizing feature point matching and utilizing RANSAC algorithm to screen matching points;

the characteristic point matching is used for searching two characteristic points with the shortest distance in two groups of characteristic point sets by using a distance function, and the distance between two binary descriptors can be represented by using a Hamming distance, wherein the Hamming distance refers to the number of characters with different corresponding positions between two character strings with the same length. The smaller the Hamming distance is, the more similar the two binary descriptors are;

calculating the shortest Hamming distance and the next shortest Hamming distance of each feature point to obtain a group of feature point matching pairs, and considering that the two feature points are matched when the ratio of the shortest distance to the next shortest distance is less than a threshold value;

the method comprises the steps of selecting a certain number of samples at random to estimate model parameters by screening matching points by using a RANSAC algorithm, classifying the rest data according to the estimated parameters, determining that one part of data is within an allowable error range, namely an inner point, and removing an error matching point pair through multiple hypothesis verification if the part of data is an outer point;

the RANSAC algorithm needs to use a homography matrix H which describes the transformation relation between the coordinates of two image points, including the relation of translation, rotation, scaling and the like, and the position of a point in one image in the other image can be found through the matrix HSuppose a pair of matching points p in image 1 and image 2₁(x,y)，p₂The transformation relationship between (x ', y') is:

$\left[\begin{array}{c}{x}^{\′}\\ y^{\′}\\ 1\end{array}\right]=H\left[\begin{array}{c}x\\ y\\ 1\end{array}\right]=\left[\begin{array}{ccc}{a}_0& a_1& a_2\\ a_3& a_4& a_5\\ a_6& a_7& 1\end{array}\right]\left[\begin{array}{c}x\\ y\\ 1\end{array}\right]$

8 parameters of the matrix H can be calculated by 4 pairs of matching points, and the RANSAC algorithm comprises the following steps:

(1) setting an initial value of iteration times as 0, a maximum iteration time N, an internal point number threshold T1 and an error threshold T2;

(2) randomly selecting 4 pairs from the n pairs of points to be matched, and calculating parameters of a transformation matrix H between the two images;

(3) calculating the distance between the coordinate values of the other feature points after H transformation and the matching points of the feature points, if the distance is smaller than an error threshold value T2, considering the matching point pair as an inner point, and if the distance is not smaller than the error threshold value T2, calculating the number of the inner points;

(4) if the number of inliers is greater than the threshold number of inliers T1, the current model is saved as the optimal model. Otherwise, adding one to the iteration times, and turning to the step (2) to continue the next iteration;

(5) if the maximum iteration number N is reached, returning a group of interior points with the maximum number of corresponding interior points, and obtaining a transformation matrix H;

the image stitching means: after the images are registered, the images are spliced through image fusion, wherein the image fusion is the last step of image splicing and mainly comprises two parts: merging the images and eliminating image splicing lines, wherein the merging of the images is to eliminate redundant pixel information in the overlapped area and align the images to be spliced according to the image registration result; eliminating image splicing lines and performing weighted average fusion processing near the splicing lines; the weighted average weighting function can use a gradual-in gradual-out method, the complexity is low, the speed is high, and smooth transition of images can be realized in the overlapping area. The gradual-in gradual-out method is to calculate a weight according to the distance from a pixel to be fused to the boundary of a coincidence region, the weight is changed linearly, and a fusion formula is as follows:

$f(x,y)=\left\{\begin{array}{cc}{f}_1(x,y)& (x,y)\∈f_1\\ d_1(x,y)f_1(x,y)+d_2(x,y)f_2(x,y)& (x,y)\∈(f_1\∩f_2)\\ f_2(x,y)& (x,y)\∈f_2\end{array}\right.$

in the formula (d)₁、d₂Representing the weight of the pixel point (x, y) on the image corresponding to the overlapped part, and meeting the following conditions: d₁+d₂＝1,0＜d₁,d₂＜1；

d₁、d₂The calculation formula of (2) is as follows:

wherein x_iIs the abscissa, x, of the pixel point to be fused_i、x_rRespectively, the abscissa of the left and right borders of the image overlap region.