CN109508674A - Airborne lower view isomery image matching method based on region division - Google Patents
Airborne lower view isomery image matching method based on region division Download PDFInfo
- Publication number
- CN109508674A CN109508674A CN201811348826.1A CN201811348826A CN109508674A CN 109508674 A CN109508674 A CN 109508674A CN 201811348826 A CN201811348826 A CN 201811348826A CN 109508674 A CN109508674 A CN 109508674A
- Authority
- CN
- China
- Prior art keywords
- image
- matching
- region
- target image
- airborne
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
- 238000000034 method Methods 0.000 title claims abstract description 76
- 238000004422 calculation algorithm Methods 0.000 claims abstract description 26
- 230000008569 process Effects 0.000 claims abstract description 14
- 238000011156 evaluation Methods 0.000 claims abstract description 11
- 238000003709 image segmentation Methods 0.000 claims abstract description 11
- 238000001514 detection method Methods 0.000 claims description 8
- 238000005259 measurement Methods 0.000 claims description 3
- 235000007926 Craterellus fallax Nutrition 0.000 abstract 1
- 240000007175 Datura inoxia Species 0.000 abstract 1
- 230000000694 effects Effects 0.000 description 8
- 238000010586 diagram Methods 0.000 description 7
- 239000011159 matrix material Substances 0.000 description 7
- 238000005516 engineering process Methods 0.000 description 6
- 230000009466 transformation Effects 0.000 description 6
- 230000011218 segmentation Effects 0.000 description 5
- 238000005070 sampling Methods 0.000 description 4
- 230000008859 change Effects 0.000 description 3
- 238000005316 response function Methods 0.000 description 3
- 238000004458 analytical method Methods 0.000 description 2
- 230000009286 beneficial effect Effects 0.000 description 2
- 238000004364 calculation method Methods 0.000 description 2
- 238000010606 normalization Methods 0.000 description 2
- 238000012545 processing Methods 0.000 description 2
- 238000012360 testing method Methods 0.000 description 2
- 238000012549 training Methods 0.000 description 2
- 241000282320 Panthera leo Species 0.000 description 1
- OAICVXFJPJFONN-UHFFFAOYSA-N Phosphorus Chemical compound [P] OAICVXFJPJFONN-UHFFFAOYSA-N 0.000 description 1
- 230000003044 adaptive effect Effects 0.000 description 1
- 238000013398 bayesian method Methods 0.000 description 1
- 238000006243 chemical reaction Methods 0.000 description 1
- 238000000605 extraction Methods 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 238000003909 pattern recognition Methods 0.000 description 1
- 238000007781 pre-processing Methods 0.000 description 1
- 230000001932 seasonal effect Effects 0.000 description 1
- 238000004088 simulation Methods 0.000 description 1
- 238000000638 solvent extraction Methods 0.000 description 1
- 230000002123 temporal effect Effects 0.000 description 1
- 238000012546 transfer Methods 0.000 description 1
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V20/00—Scenes; Scene-specific elements
- G06V20/10—Terrestrial scenes
- G06V20/13—Satellite images
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/20—Image preprocessing
- G06V10/26—Segmentation of patterns in the image field; Cutting or merging of image elements to establish the pattern region, e.g. clustering-based techniques; Detection of occlusion
- G06V10/267—Segmentation of patterns in the image field; Cutting or merging of image elements to establish the pattern region, e.g. clustering-based techniques; Detection of occlusion by performing operations on regions, e.g. growing, shrinking or watersheds
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/40—Extraction of image or video features
- G06V10/46—Descriptors for shape, contour or point-related descriptors, e.g. scale invariant feature transform [SIFT] or bags of words [BoW]; Salient regional features
- G06V10/462—Salient features, e.g. scale invariant feature transforms [SIFT]
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/40—Extraction of image or video features
- G06V10/50—Extraction of image or video features by performing operations within image blocks; by using histograms, e.g. histogram of oriented gradients [HoG]; by summing image-intensity values; Projection analysis
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
- G06V10/74—Image or video pattern matching; Proximity measures in feature spaces
- G06V10/75—Organisation of the matching processes, e.g. simultaneous or sequential comparisons of image or video features; Coarse-fine approaches, e.g. multi-scale approaches; using context analysis; Selection of dictionaries
- G06V10/757—Matching configurations of points or features
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Multimedia (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Health & Medical Sciences (AREA)
- Remote Sensing (AREA)
- Astronomy & Astrophysics (AREA)
- Artificial Intelligence (AREA)
- Computing Systems (AREA)
- Databases & Information Systems (AREA)
- Evolutionary Computation (AREA)
- General Health & Medical Sciences (AREA)
- Medical Informatics (AREA)
- Software Systems (AREA)
- Image Analysis (AREA)
- Studio Devices (AREA)
Abstract
The present invention provides the airborne lower view isomery image matching method based on region division, belongs to technical field of image matching.The standard deviation STD of present invention use direction histogram first determines the textural characteristics of target image；If horn of plenty texture image, target image and realtime graphic are split using Meanshift image segmentation algorithm respectively, are divided into several regions, and generate corresponding mask images；If non-abundant texture image, realtime graphic is divided, forms several region units；Then SIFT feature matching process is used, each realtime graphic region and all object regions are subjected to consistency matching respectively；Finally each matched result is evaluated using the evaluation function based on direction histogram, chooses optimal matching area as matching result.The present invention solves the problems, such as existing not high for complicated and isomery onboard image matching accuracy rate.The present invention can be used for the isomery images match of unmanned plane.
Description
Technical Field
The invention relates to an airborne downward-looking heterogeneous image matching method, and belongs to the technical field of image matching.
Background
With the promotion of unmanned aerial vehicle technology, unmanned aerial vehicle technology has attracted a large amount of producers and users, and this makes unmanned aerial vehicle correlation technique have extensive market. Heterogeneous image matching based on the unmanned aerial vehicle is a process of aligning a real-time image from the unmanned aerial vehicle and a target image from a satellite, and the technology has wide application in the aspects of unmanned aerial vehicle navigation, unmanned aerial vehicle landing, unmanned aerial vehicle attack, space vehicle navigation and the like. However, airborne downward-looking images have complex structural features, and accurate matching of airborne downward-looking heterogeneous images becomes a key technology in unmanned aerial vehicle application and has important significance.
In unmanned aerial vehicle target location, the target image is non-real-time, and the location image is real-time, which makes the target image and the location image have different image structures. Real-time images will be very complex, typically including scale-varying images, rotation-varying images, seasonal images, blurred images, occlusion images, and SAR (Synthetic Aperture Radar) images. The airborne downward-looking heterogeneous image matching is a challenging problem, and the application requirement of unmanned aerial vehicle heterogeneous image matching is difficult to meet through a classical image matching method.
The most classical image matching method at present is a Scale-invariant feature transform (SIFT) -based feature image matching method. Wu-just et al invented a method for matching and fusing building images based on contour extraction, which comprises the steps of firstly extracting object contours, extracting straight lines on contour maps, matching two images according to straight line characteristics by using a straight line matching algorithm, calculating the included angles of the straight lines in an optimal matching pair set to obtain an included angle matrix, and calculating the similarity of the included angle matrix. Zhaoheng et al invented a spatial target image matching method, which utilizes GMS (Grid-based Motion estimation) matching algorithm to perform rough matching on three-view images of a spatial target, introduces error threshold algorithm NFA to eliminate mismatching point pairs, and makes the algorithm more adaptive. The method comprises the steps of firstly making a data set of a heterogeneous image block, carrying out image preprocessing, obtaining a characteristic diagram of the image block, obtaining a characteristic vector through the characteristic diagram, fusing and normalizing the characteristic diagram, training an image matching network, predicting the matching probability and effectively solving the problem of over-fitting of image matching. The above mentioned image matching methods are all mature, and can achieve a relatively high image matching accuracy in general situations, but in actual situations, airborne downward-looking images are often relatively complex, as shown in fig. 1, fig. 2, fig. 3, fig. 4, fig. 5, fig. 6, and fig. 7; for complex and heterogeneous airborne images, these methods still do not achieve good effects, and a technology capable of achieving higher matching accuracy needs to be sought.
Disclosure of Invention
The invention provides an airborne downward-looking heterogeneous image matching method based on region division, aiming at solving the problem that the matching accuracy of complex and heterogeneous airborne images is low in the prior art.
The invention relates to a region division-based airborne downward-looking heterogeneous image matching method, which is realized by the following technical scheme:
(1) determining the texture features of the target image by using the standard deviation STD of the direction histogram as a parameter: if the standard deviation STD is larger than the threshold value S, judging that the target image is a rich texture image; if the standard deviation STD is less than or equal to the threshold value S, judging that the target image is a non-rich texture image;
(2) if the target image is a rich texture image, the target image and the real-time image are respectively segmented by using a Meanshift mean shift image segmentation algorithm to be segmented into a plurality of regions, and the segmented target image region and the real-time image region are respectively layered to generate corresponding mask images;
if the target image is a non-rich texture image, dividing the real-time image to form a plurality of region blocks, and regarding the whole target image as a region;
(3) respectively carrying out consistency matching on each real-time image area and all target image areas by using an SIFT feature matching method;
(4) and evaluating each matching result by utilizing an evaluation function based on the direction histogram, and selecting an optimal matching area as a matching result.
The most prominent characteristics and remarkable beneficial effects of the invention are as follows:
1. classifying the target images, and performing different processing on different types of images to improve the matching accuracy of the images; in a simulation experiment, compared with the traditional SIFT algorithm, the image matching accuracy of the method (the airborne downward-looking heterogeneous image matching method based on region division) is improved by about 15 percent;
2. various image processing technologies are fused, and the image matching process can be robustly and accurately finished in airborne downward-looking images with a large amount of noise and interference by using information of different types of real-time images;
3. the method has important significance for the target positioning system of the airborne downward-looking image, and the application range of the unmanned aerial vehicle target positioning system based on image matching is greatly expanded;
4. the method uses the standard deviation of the direction histogram as an evaluation parameter, determines the texture characteristics of the target image, classifies the target, comprises rich texture images and non-rich texture images, and respectively processes the two images, thereby improving the robustness of the system;
5. aiming at abundant texture images, mask images of different segmentation areas are obtained by using a Meanshift image segmentation method, different areas are respectively matched by introducing the mask images, and the result of optimal matching area matching is taken as a final matching result, so that the intra-class area similarity is stronger, the inter-class area similarity is weaker, and the target matching accuracy is higher;
6. the method comprises the steps of partitioning a real-time wide image aiming at a non-rich texture image, matching the region blocks with a target image respectively, and taking the result of optimal matching region matching as a final matching result. For the non-rich texture image, after the segmentation method is used for segmentation, the segmentation areas are dispersed, accurate matching cannot be completed in the segmentation areas, the areas are partitioned to a certain degree, so that the intra-class area similarity is stronger, the inter-class area similarity is weaker, and the accuracy of matching the non-rich texture target image is improved;
7. the invention provides a method for obtaining a matching result of an optimal matching area by using an evaluation function based on a direction histogram. In order to ensure the accuracy of the matching result, the similarity of two matching images is compared by using the Bhattacharyya Distance (BD) (Bhattacharyya distance), and if the BD value is larger, the matching is considered to be more accurate, so that the final matching result is more accurate.
Drawings
FIG. 1 is a real-time image of a scale change;
FIG. 2 is a real-time image of a rotational change;
FIG. 3 is a real-time image of spring;
FIG. 4 is a real-time image of winter;
FIG. 5 is a blurred real-time image;
FIG. 6 is a real-time image of an occlusion;
FIG. 7 is a real-time image of a synthetic aperture radar SAR;
FIG. 8 is a schematic flow chart of the method of the present invention;
FIG. 9 is a map image (which is also the target image in the example) of a;
FIG. 10 is a directional histogram at location a; pixel number indicates the number of pixels, Angle indicates the Angle;
FIG. 11 is a b-map image;
FIG. 12 is a histogram of the orientation of position b;
FIG. 13 is an image of c;
FIG. 14 is a direction histogram at position c;
FIG. 15 is an image of d;
FIG. 16 is a d-position orientation histogram;
FIG. 17 is a diagram illustrating execution time and matching rate corresponding to different h in the SIFT algorithm;
FIG. 18 is a schematic diagram of maximum and minimum extreme point detection in a DOG image;
FIG. 19 is a diagram illustrating the calculation of an operator;
fig. 20 is a real-time image for matching the c-map image in the embodiment.
Detailed Description
The first embodiment is as follows: the embodiment is described with reference to fig. 8, and the onboard downward-looking heterogeneous image matching method based on region division provided by the embodiment is used for the target image and the complex and heterogeneous onboard real-time image shown in fig. 1, and the method firstly uses the standard deviation STD of the direction histogram as a parameter to determine the texture feature of the target image; if the target image is a rich texture image, completing an airborne downward-looking image positioning process by using an image matching method based on image segmentation, wherein the process uses image segmentation to generate mask images of different regions of the image, in the regions, matching each region by using an improved SIFT image matching method, and obtaining an optimal matching region as a matching result by using an evaluation function based on a direction histogram; the image positioning process is completed on the basis of an image matching method based on region block division for a non-rich texture image, the process divides the region of a real-time image, an improved SIFT image matching method is used for matching each block region with a target image, and an optimal matching block region is obtained by using an evaluation function based on a direction histogram as a matching result. The method specifically comprises the following steps:
(1) determining the texture features of the target image by using the standard deviation STD of the direction histogram as a parameter: if the standard deviation STD is larger than the threshold value S, judging that the target image is a rich texture image; if the standard deviation STD is less than or equal to the threshold value S, judging that the target image is a non-rich texture image; as shown in fig. 9 to 16, the images a, b, c, and d have different textures and their direction histograms, where the image STD of a is 0.152, the image STD of b is 0.157, the image STD of c is 0.129, and the image STD of d is 0.0747.
(2) If the target image is a rich texture image, the target image and the real-time image are respectively segmented by using a Meanshift mean shift image segmentation algorithm to be segmented into a plurality of regions, and the segmented target image region and the real-time image region are respectively layered to generate corresponding mask images;
if the target image is a non-rich texture image, the features detected by the traditional feature detection algorithm are not obviously distinctive, and matching failure is caused when image matching is performed on a large-amplitude real-time image.
MeanShift is a feature space analysis method, which maps problems to color feature space for image segmentation. The image segmentation problem is that for each pixel point, the class center problem of the pixel point is found, and MeanShift considers that the center is the maximum value of probability densityAnd (4) point. Kernel probability density estimation (which can be viewed as Parzen windowing in pattern recognition) is the most popular method of probability density estimation。Point set x for n dataiI 1.. n, in d-dimensional space RdIn point x, given a symmetric positive definite matrix H of kernel k (x) and d × d sizes, the multivariate kernel density estimate is defined as:
here, the condition is to be satisfied:
KH(x)＝|H|-1/2K(H-1/2x)
and K (x) satisfies:
where c isKIs a constant, xTRepresenting the transpose of x and I the gray value of the image. In practical applications, to reduce the complexity of the algorithm, H uses a diagonal matrixThe best known expression is obtained:
here, ,is a diagonal element of the diagonal matrix H; h is the width of the kernel K (x), where K (x) usually takes a special class of symmetric kernels:
K(x)＝cK,dk(||x||2)
wherein, cK,dIs a normalization coefficient; k (| | x | | non-conducting phosphor)2) Representing a projection function;
the final probability density estimate can be written as:
meanshift is the determination(probability Density estimation)Let g (x) be-k '(x), k' (x) being the derivative of k (x); definition of core G (x) ═ cg,dg(||x||2)，cg,dIs a normalization constant. The MeanShift vector can be derived from the linear expression of the above equation:
thus, the MeanShift algorithm execution includes two parts: 1) calculate MeanShift vector mh,G(x) (ii) a 2) Through mh,G(x) The kernel window g (x) is convolved. The method can ensure the final probability density estimation expressionConverge near the point where the gradient is 0 and find the cluster center.
The target image segmentation effect is good; however, the real-time image is usually large, and fig. 17 is a schematic diagram of the execution time and the matching rate corresponding to different h in the SIFT algorithm; in order to meet the real-time requirement, the target can be segmented after the resolution of the image is reduced.
(3) And respectively carrying out consistency matching on each real-time image region and all target image regions by using an SIFT feature matching method.
(4) And evaluating each matching result by utilizing an evaluation function based on the direction histogram, and selecting an optimal matching area as a matching result.
The second embodiment is as follows: the difference between this embodiment and the specific embodiment is that, in step (3), a SIFT feature matching method is used, and in the process of respectively performing consistency matching on each real-time image region and all target image regions, a Corner (Corner) is added as a feature key point.
The traditional SIFT algorithm comprises four parts of extreme value detection of a scale space, key point positioning, direction setting and key point description operator determination. The improved SIFT image matching method combines the corner key points and the SIFT key points to form a key point set of an algorithm, the corner regions are used as masks to generate key points of the corner regions, and the SIFT image matching algorithm with the corner added as feature key points is executed in the segmented image regions, so that the image matching performance is improved.
The SIFT image matching method with the added corner points as the feature key points specifically comprises the following steps:
(3.1) selection of peaks in the Scale space
The selection of the peak in scale space is to find the peak on the convolution image D (x, y, σ) of DOG (difference-of-Gaussian) function of different scales determined by a constant factor k, D (x, y, σ) being defined as:
D(x,y,σ)＝(G(x,y,kσ)-G(x,y,σ))*I(x,y)＝L(x,y,kσ)-L(x,y,σ)
g (x, y, σ) represents a gaussian filter, L (x, y, σ) represents a gaussian smoothed image, and the maximum value and the minimum value in the scale space become maximum or minimum candidate pixels if the current value is maximum or minimum by comparing the current pixel in the different scale spaces with 26 neighboring pixels in a 3 × 3 × 3 neighborhood, as shown in fig. 18.
(3.2) Key Point determination
In 2002, Lowe et al proposed a method for determining sampling points by taylor expansion of D (x, y, σ) as follows:
d denotes the function value at point D (x) 0 or D denotes the function value at point D (x) at the sampling point x (0,0,0), the extreme value of the sampling point being calculated by calculating the gradient of the function as zeroObtaining, then:
selectingThe position where the peak point is less than 0.03 is taken as a key point while taking into account the edge effect.
(3.3) Direction setting
The amplitude m and θ of each point on the image are calculated from the pixel differential, then:
by the above two formulas, a histogram of the direction of all the sampling points is calculated, and the peak of the histogram can be regarded as the main direction of the local gradient, as shown in the right side of fig. 19.
(3.4) local descriptor determination and keypoint matching
Calculation of SIFT description operator as shown in fig. 19, at each keypoint, the image gradient and direction are calculated, direction histograms are created in 4 × 4 regions where the keypoint is located, each histogram contains 8 directions, each arrow represents each direction in the histogram, and the length represents the magnitude of each direction. A gradient sample may select four 4 x 4 regions, each of which may compute a histogram of directions, arranged as descriptors of SIFT in the middle graph of fig. 19. In the present embodiment, 4 × 4 small regions are selected, and a vector of 128 dimensions, i.e., 4 × 4 × 8, is obtained, and the 128-dimensional vector is used for similarity measurement in image matching.
After the keypoint description of the image is generated, for each feature point in the target, a corresponding consistent keypoint is found in the real-time image, and the process is called keypoint matching. This problem can be described as: given the set T and the unknown sample x, the most likely sample c corresponding to x is found. Using Bayesian methods, one can selectHere P (c)j| x, T) denotes c where x is locatedjPosterior probability of class, cjBelonging to one element of the training data set T. The k-nearest neighbor algorithm used in the present embodiment is as follows:
step 1: finding k x nearest neighbor elements in the set T, i.e. finding a setAt the same time satisfy
Step 2: in Y, the frequently occurring key point pairs are put into the T set, breaking the randomly generated associations.
(3.5) determination of the transfer function
In the present embodiment, the transformation function is perspective transformation, which essentially projects an image onto a new viewing plane, and the general transformation formula is:
where (u, v) is the original image position and (x, y) is the position of the transformed image. Thus, x ═ x*/w*,y＝x*/w*The transformation matrix can be divided into four parts:representing a linear transformation, [ a ]31a32]Is an offset amount, [ a ]13a23]TA perspective transformation is generated. Thus can obtain
The RANSAC method is a method for performing model fitting on data, and can be used for matching a target image with an airborne downward-looking real-time image. The RANSAC method has the inherent property of detecting and rejecting erroneous pixels, as compared to the classical least squares fit determination of the projection function. RANSAC performs the process as follows:
given known target image and realityThe corresponding key point pair set P of the temporal image, here (| P>n), n is the minimum number of key point pairs required to instantiate the projection parameters. Randomly selecting a subset of n data points in the set P S1, instantiating the model, and determining the subset of the set P S1 using the instantiated model M1*The data in the set has the smallest error for M1. If S1*Is greater than a certain threshold value, a new model M1 is calculated by using a least square fitting method*If S1*If the size of the data is smaller than the threshold t, a new subset is randomly selected S1, the data is instantiated again, if the cycle number is larger than the threshold ln, the consistency point set cannot be found, and the algorithm is terminated. And obtaining a projection function corresponding to matching through a RANSAC algorithm, and simultaneously marking a target in a real-time image.
Other steps and parameters are the same as those in the first embodiment.
The third concrete implementation mode: the embodiment is different from the specific embodiment in that the specific process of evaluating the matching result by using the evaluation function based on the histogram of directions and selecting the optimal matching area as the matching result in the step (4) includes:
(4.1) using the babbitt distance BD as a matching similarity measurement coefficient for the two region histograms being matched;
(4.2) if the BD is greater than the threshold T, judging that the matching is successful; otherwise, if the BD is less than or equal to the threshold T, the matching fails;
and (4.3) selecting the area with the maximum corresponding BD value as the optimal matching area in the areas successfully matched.
Other steps and parameters are the same as those in the first or second embodiment.
The fourth concrete implementation mode: the third difference between this embodiment and this embodiment is that, in step (4.1), the babbitt distance BD is specifically:
wherein th (j) represents the direction histogram of the target image, and rh (j) represents the direction histogram of the real-time image; j is 1, …, N; n represents the number of classes of image grey levels.
Other steps and parameters are the same as those in the first, second or third embodiment.
The fifth concrete implementation mode: the third difference between this embodiment and the third embodiment is that the standard deviation STD in step (1) is specifically defined as:
wherein μ represents a mean value of the direction histogram; h (j) a direction histogram representing an image; j is 1, …, N; n represents the number of classes of image grey levels.
Other steps and parameters are the same as those in the first, second, third or fourth embodiments.
The sixth specific implementation mode: the second difference between this embodiment and the second embodiment is that the corner is obtained by using a Harris corner detection method.
The Harris corner detection method determines the corners in the image by determining the size of the corner response function R. I represents the gray scale value of the image, and (x, y) represents the pixel coordinate position in the image, and the variation E of the region is relative to the small region of the (x, y) positionx,yThe definition is as follows:
Ex,y＝∑u,vwu,v|Ix+u,y+v-Iu,v|2
where w isu,vRepresenting an image window, (u, v) is the position of the coordinates in the window, (x, y) can be translated by 4 directions { (-1,1), (1,1), (-1, -1), (1, -1) }. E can be obtained through analysis and expansionx,yAnother expression form of (a):
Ex,y＝(x,y)M(x,y)T
m is a 2 × 2 symmetric matrix:
wherein,
the following formula is defined:
Tr(M)＝A+B
Det(M)＝AB-C2
then the corner response function R can be derived:
R＝Det(M)-qTr2(M)
where q is an empirical value, corner detection is to find a point where the response function is greater than a certain threshold as the corner.
Other steps and parameters are the same as those of the first, second, third, fourth, fifth, and sixth embodiments.
The seventh embodiment: the difference between this embodiment and the first to sixth embodiments is that the value of the threshold S is 0.13 to 0.15. For images with less textures, the features detected by the traditional feature detection algorithm are not obviously distinctive, and the matching is failed when the images are matched on the large-scale real-time images. Calculating STD statistics through a large number of images to obtain a non-rich texture image which can be judged to be in a common meaning when the STD is less than 0.13; when the STD is larger than 0.15, the texture image can be judged to be a rich texture image in the general sense, so that when the threshold S is 0.13-0.15, the texture image is used for distinguishing rich texture from non-rich texture, and the method is reasonable and can achieve a good effect.
Other steps and parameters are the same as those in the first, second, third, fourth, fifth or sixth embodiment.
Examples
The following examples were used to demonstrate the beneficial effects of the present invention:
selecting the image of a as a target image (as shown in fig. 9), and selecting the real-time image as fig. 1, fig. 2 and fig. 6; the c-place image (as shown in fig. 13) is selected as the target image, and the real-time image is shown in fig. 20.
The airborne downward-looking heterogeneous image matching method based on region division comprises the following specific steps:
1. setting the value of the threshold S to be 0.14 by using a formulaCalculating to obtain a local orientation histogram standardDetermining the difference STD to be 0.152 (as shown in fig. 10) as a rich texture image; the location c histogram standard deviation STD is 0.129 (see fig. 14), and the texture image is determined to be non-rich.
2. As shown in fig. 8, the target image and the real-time image are segmented by using a Meanshift mean shift image segmentation algorithm, and are segmented into a plurality of regions, and the segmented target image region and the real-time image region are layered to generate corresponding mask images. The method comprises the steps of detecting key points in different regions, finding consistent key points in different region pairs, then generating a marked image by using a conversion function, and finally selecting an optimal matching region by using an evaluation function based on a direction histogram.
In order to test the effect of the method, the target image and the real-time image are subjected to a matching test by using other existing algorithms, and the comparison condition of the effect of the method of the invention and the effect of other algorithms is obtained as follows:
TABLE 1 comparison of the matching effects of different methods
As can be seen from the above table, the method of the present invention can achieve higher image matching accuracy. The combination mode of MeanShift, SIFT and angular points used in the invention can greatly improve the accuracy of image matching of rotation change, and when other methods are ineffective for non-rich texture images, the image matching algorithm based on block division used in the invention can effectively match images. Compared with the traditional SIFT algorithm, the image matching accuracy of the method (the airborne downward-looking heterogeneous image matching method based on region division) is improved by about 15 percent;
the present invention is capable of other embodiments and its several details are capable of modifications in various obvious respects, all without departing from the spirit and scope of the present invention.
Claims (7)
1. The airborne downward-looking heterogeneous image matching method based on region division is characterized by comprising the following steps:
(1) determining the texture features of the target image by using the standard deviation STD of the direction histogram as a parameter: if the standard deviation STD is larger than the threshold value S, judging that the target image is a rich texture image; if the standard deviation STD is less than or equal to the threshold value S, judging that the target image is a non-rich texture image;
(2) if the target image is a rich texture image, the target image and the real-time image are respectively segmented by using a Meanshift mean shift image segmentation algorithm to be segmented into a plurality of regions, and the segmented target image region and the real-time image region are respectively layered to generate corresponding mask images;
if the target image is a non-rich texture image, dividing the real-time image to form a plurality of region blocks, and regarding the whole target image as a region;
(3) respectively carrying out consistency matching on each real-time image area and all target image areas by using an SIFT feature matching method;
(4) and evaluating each matching result by utilizing an evaluation function based on the direction histogram, and selecting an optimal matching area as a matching result.
2. The region division-based airborne downward-looking heterogeneous image matching method according to claim 1, wherein in the step (3), an SIFT feature matching method is used, and in the process of respectively performing consistency matching on each real-time image region and all target image regions, a corner point is added as a feature key point.
3. The method for matching airborne downward-looking heterogeneous images based on region division according to claim 1, wherein the specific process of evaluating the matching result by using the evaluation function based on the histogram of directions in the step (4) and selecting the optimal matching region as the matching result comprises:
(4.1) using the babbitt distance BD as a matching similarity measurement coefficient for the two region histograms being matched;
(4.2) if the BD is greater than the threshold T, judging that the matching is successful; otherwise, if the BD is less than or equal to the threshold T, the matching fails;
and (4.3) selecting the area with the maximum corresponding BD value as the optimal matching area in the areas successfully matched.
4. The method for matching airborne downward-looking heterogeneous images based on regional division according to claim 3, wherein the Bhattacharyya distance BD in the step (4.1) is specifically:
wherein th (j) represents the direction histogram of the target image, and rh (j) represents the direction histogram of the real-time image; j is 1, …, N; n represents the number of classes of image grey levels.
5. The method for matching airborne downward-looking heterogeneous images based on regional division according to claim 1, wherein the standard deviation STD in step (1) is specifically defined as:
wherein μ represents a mean value of the direction histogram; h (j) a direction histogram representing an image; j is 1, …, N; n represents the number of classes of image grey levels.
6. The region-division-based airborne downward-looking heterogeneous image matching method according to claim 2, wherein the corner points are obtained by a Harris corner point detection method.
7. The matching method of the airborne downward-looking heterogeneous images based on the regional division according to any one of claims 1-6, wherein the value of the threshold S is 0.13-0.15.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201811348826.1A CN109508674B (en) | 2018-11-13 | 2018-11-13 | Airborne downward-looking heterogeneous image matching method based on region division |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201811348826.1A CN109508674B (en) | 2018-11-13 | 2018-11-13 | Airborne downward-looking heterogeneous image matching method based on region division |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN109508674A true CN109508674A (en) | 2019-03-22 |
| CN109508674B CN109508674B (en) | 2021-08-13 |
Family
ID=65748318
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201811348826.1A Expired - Fee Related CN109508674B (en) | 2018-11-13 | 2018-11-13 | Airborne downward-looking heterogeneous image matching method based on region division |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN109508674B (en) |
Cited By (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN110523657A (en) * | 2019-09-03 | 2019-12-03 | 广州能源检测研究院 | A kind of instrument appearance delection device and its working method based on 3D scanning method |
| CN114565861A (en) * | 2022-03-02 | 2022-05-31 | 佳木斯大学 | Airborne downward-looking target image positioning method based on probability statistic differential homoembryo set matching |
| CN116778513A (en) * | 2023-08-24 | 2023-09-19 | 国网山西省电力公司太原供电公司 | Intelligent archiving control method for bills in power industry |
Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN103914847A (en) * | 2014-04-10 | 2014-07-09 | 西安电子科技大学 | SAR image registration method based on phase congruency and SIFT |
| CN104933434A (en) * | 2015-06-16 | 2015-09-23 | 同济大学 | Image matching method combining length between perpendiculars (LBP) feature extraction method and surf feature extraction method |
| CN105160684A (en) * | 2015-09-30 | 2015-12-16 | 中国科学院遥感与数字地球研究所 | Online automatic matching method for geometric correction of remote sensing image |
| CN106355577A (en) * | 2016-09-08 | 2017-01-25 | 武汉科技大学 | Method and system for quickly matching images on basis of feature states and global consistency |
| US20170308673A1 (en) * | 2011-08-12 | 2017-10-26 | Help Lightning, Inc. | System and method for image registration of multiple video streams |
| CN107480727A (en) * | 2017-08-28 | 2017-12-15 | 荆门程远电子科技有限公司 | The unmanned plane image fast matching method that a kind of SIFT and ORB are combined |
-
2018
- 2018-11-13 CN CN201811348826.1A patent/CN109508674B/en not_active Expired - Fee Related
Patent Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20170308673A1 (en) * | 2011-08-12 | 2017-10-26 | Help Lightning, Inc. | System and method for image registration of multiple video streams |
| CN103914847A (en) * | 2014-04-10 | 2014-07-09 | 西安电子科技大学 | SAR image registration method based on phase congruency and SIFT |
| CN104933434A (en) * | 2015-06-16 | 2015-09-23 | 同济大学 | Image matching method combining length between perpendiculars (LBP) feature extraction method and surf feature extraction method |
| CN105160684A (en) * | 2015-09-30 | 2015-12-16 | 中国科学院遥感与数字地球研究所 | Online automatic matching method for geometric correction of remote sensing image |
| CN106355577A (en) * | 2016-09-08 | 2017-01-25 | 武汉科技大学 | Method and system for quickly matching images on basis of feature states and global consistency |
| CN107480727A (en) * | 2017-08-28 | 2017-12-15 | 荆门程远电子科技有限公司 | The unmanned plane image fast matching method that a kind of SIFT and ORB are combined |
Non-Patent Citations (3)
| Title |
|---|
| JIANG YANG 等: "A Remote Sensing Imagery Automatic Feature Registration Method Bases on Mean-Shift", 《2012 IEEE INTERNATIONALGEOSCIENCE AND REMOTE SENSING SYMPOSIUM》 * |
| WENPING MA 等: "Remote Sensing Image Registration Based on Multifeature and Region Division", 《IEEE GEOSCIENCE AND REMOTE SENSING LETTERS》 * |
| 薛武 等: "沙漠地区无人机影像连接点提取", 《测绘科学技术学报》 * |
Cited By (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN110523657A (en) * | 2019-09-03 | 2019-12-03 | 广州能源检测研究院 | A kind of instrument appearance delection device and its working method based on 3D scanning method |
| CN114565861A (en) * | 2022-03-02 | 2022-05-31 | 佳木斯大学 | Airborne downward-looking target image positioning method based on probability statistic differential homoembryo set matching |
| CN114565861B (en) * | 2022-03-02 | 2024-04-30 | 佳木斯大学 | Airborne downward-looking target image positioning method based on probability statistical differential homoembryo set matching |
| CN116778513A (en) * | 2023-08-24 | 2023-09-19 | 国网山西省电力公司太原供电公司 | Intelligent archiving control method for bills in power industry |
| CN116778513B (en) * | 2023-08-24 | 2023-10-27 | 国网山西省电力公司太原供电公司 | Intelligent archiving control method for bills in power industry |
Also Published As
| Publication number | Publication date |
|---|---|
| CN109508674B (en) | 2021-08-13 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN108319964B (en) | Fire image recognition method based on mixed features and manifold learning | |
| CN107301661B (en) | High-resolution remote sensing image registration method based on edge point features | |
| CN103325112B (en) | Moving target method for quick in dynamic scene | |
| CN111145228B (en) | Heterologous image registration method based on fusion of local contour points and shape features | |
| CN108446634B (en) | Aircraft continuous tracking method based on combination of video analysis and positioning information | |
| CN109785366B (en) | Related filtering target tracking method for shielding | |
| CN109508674B (en) | Airborne downward-looking heterogeneous image matching method based on region division | |
| CN109446894B (en) | Multispectral image change detection method based on probability segmentation and Gaussian mixture clustering | |
| CN111914832B (en) | SLAM method of RGB-D camera under dynamic scene | |
| CN104036523A (en) | Improved mean shift target tracking method based on surf features | |
| CN102360503B (en) | SAR (Specific Absorption Rate) image change detection method based on space approach degree and pixel similarity | |
| CN103927511A (en) | Image identification method based on difference feature description | |
| CN109635789B (en) | High-resolution SAR image classification method based on intensity ratio and spatial structure feature extraction | |
| CN106570887A (en) | Adaptive Mean Shift target tracking method based on LBP features | |
| CN109472770B (en) | Method for quickly matching image characteristic points in printed circuit board detection | |
| CN107862708A (en) | A kind of SAR and visible light image registration method | |
| CN108401565B (en) | Remote sensing image registration method based on improved KAZE features and Pseudo-RANSAC algorithms | |
| CN110222661B (en) | Feature extraction method for moving target identification and tracking | |
| CN105894037A (en) | Whole supervision and classification method of remote sensing images extracted based on SIFT training samples | |
| CN103353941A (en) | Natural marker registration method based on viewpoint classification | |
| CN105447488B (en) | SAR image target detection method based on sketch line segment topological structure | |
| CN114358166B (en) | Multi-target positioning method based on self-adaptive k-means clustering | |
| CN108549905A (en) | A kind of accurate method for tracking target under serious circumstance of occlusion | |
| CN105787505A (en) | Infrared image clustering segmentation method combining sparse coding and spatial constraints | |
| CN103336964B (en) | SIFT image matching method based on module value difference mirror image invariant property |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant | ||
| CF01 | Termination of patent right due to non-payment of annual fee |
Granted publication date: 20210813 |
|
| CF01 | Termination of patent right due to non-payment of annual fee |