CN111461196B - Rapid robust image identification tracking method and device based on structural features - Google Patents
Rapid robust image identification tracking method and device based on structural features Download PDFInfo
- Publication number
- CN111461196B CN111461196B CN202010229998.8A CN202010229998A CN111461196B CN 111461196 B CN111461196 B CN 111461196B CN 202010229998 A CN202010229998 A CN 202010229998A CN 111461196 B CN111461196 B CN 111461196B
- Authority
- CN
- China
- Prior art keywords
- matching
- graph
- image
- feature
- point
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
- G06V10/74—Image or video pattern matching; Proximity measures in feature spaces
- G06V10/75—Organisation of the matching processes, e.g. simultaneous or sequential comparisons of image or video features; Coarse-fine approaches, e.g. multi-scale approaches; using context analysis; Selection of dictionaries
- G06V10/751—Comparing pixel values or logical combinations thereof, or feature values having positional relevance, e.g. template matching
-
- Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
- Y02D—CLIMATE CHANGE MITIGATION TECHNOLOGIES IN INFORMATION AND COMMUNICATION TECHNOLOGIES [ICT], I.E. INFORMATION AND COMMUNICATION TECHNOLOGIES AIMING AT THE REDUCTION OF THEIR OWN ENERGY USE
- Y02D10/00—Energy efficient computing, e.g. low power processors, power management or thermal management
Abstract
The invention provides a rapid robust image recognition tracking method and device based on structural features, wherein the method adopts a GMS feature matching algorithm to perform feature point matching pair screening on a query image and a training matching image; if the correct feature point matching pair exists, equally dividing the interested region in the query image and the training matching image into small grids, and determining a key point for each grid; modeling key points as nodes of the graph, constructing a graph model, and fusing feature matching with weight parameters matched with the graph; and (5) performing approximate matching on the graph by using a random walk algorithm to finish image identification and tracking. The invention can effectively accelerate the matching recognition method, and provides more accurate recognition tracking performance under the condition of fewer feature point matching pairs.
Description
Technical Field
The invention relates to the field of image matching in computer vision, in particular to a rapid robust image identification tracking method and device based on structural features.
Background
The task of image matching is to find the correspondence between pixels in two or more images of the same scene. The method is a very important hot spot problem in the research fields of computer vision, information processing and the like, and is also the basis of a plurality of computer vision theories and applications. The image matching technology mainly comprises a gray matching-based method and a feature matching-based method. The gray matching-based method can obtain better effect in general cases, but has poor performance in areas with smaller gray information. The feature matching-based method can obtain feature points in the original image according to certain feature extraction operators, the feature points can better represent the image, and the image is matched through the mapping relation among the feature points.
The main stream image matching method still has some defects, the gray matching method has high requirements on image sources, the gray values of the same object shot by different cameras have large differences, the method needs to circularly compare the template images in the test images, the calculation complexity is high, and the practical application is difficult. The feature matching-based method is small in calculated amount and suitable for the conditions of deformation, external points and shielding in the image, but high-quality feature points are needed, and the main stream feature matching method only focuses on the relation among the feature points, and the structural features of the image are not considered, so that false matching is caused. In summary, how to balance the efficiency and accuracy of image matching becomes a problem to be solved.
Disclosure of Invention
Aiming at the defects of the prior art, the invention aims to provide a rapid robust image identification tracking method and device based on structural features, which can provide more accurate identification tracking performance under the condition of fewer feature point matching pairs by using the structural features among key points to assist in matching.
According to a first aspect of the present invention, there is provided a fast robust image recognition tracking method based on structural features, which is characterized by comprising:
adopting a GMS feature matching algorithm to perform feature point matching pair screening on the query image and the training matching image;
if the correct feature point matching pair exists, equally dividing the interested region in the query image and the training matching image into small grids, and determining a key point for each grid;
modeling key points as nodes of the graph, generating edges, constructing a graph model, and fusing feature matching with weight parameters matched with the graph;
and (5) performing approximate matching on the graph by using a random walk algorithm to finish image identification and tracking.
Optionally, the filtering the feature point matching pair of the query image and the training matching image by adopting a GMS feature matching algorithm includes:
matching the characteristic points through a BF violent matching algorithm;
dividing the query image and the training matching image into G grids respectively, and calculating feature point pairs F with good BF matching p And F is equal to q Number of correct matches S in the vicinity pq ，F p And F is equal to q Characteristic points in the query image and the training matching image are respectively screened as matching pairs by a BF violent matching algorithm;
by characteristic point F p And F q The grid is taken as the center, 9 grids around the grid are selected as regions to calculate the matching number, wherein K=3x3 is taken as the region around the gridIs a grid { p } k ,q k The matching pair number between the two is between 1 and 9;
setting a threshold value for true-false matchingEta takes the empirical value of 6, n i Is the total feature number of the grid; comparing the number S of correct matches pq And threshold t p To determine if the point is correctly matched:
wherein p and q are respectively expressed as characteristic points F in the query image p Matching feature points F in the image with training q 。
Optionally, the matching the feature points by BF violent matching algorithm includes:
firstly, selecting a characteristic point from a query image;
then sequentially carrying out BRIEF description Fu Hanming distance test with the feature points in the matched training images;
and finally, returning the nearest characteristic points to form a characteristic point matching set from the query image to the matched training image.
Optionally, the BRIEF description Fu Hanming distance test, wherein BRIEF descriptors are obtained by:
extracting FAST feature points from each picture by using an ORB algorithm;
and taking each FAST characteristic point as a center, taking an S multiplied by S neighborhood large window, randomly selecting point pairs in the large window, carrying out binary assignment, and calculating BRIEF descriptors.
Optionally, the equally dividing the query image and the region of interest in the training matching image into small grids, determining the keypoints for each grid includes:
(1) Drawing a region of interest R in a query image, equally dividing the region R into N grids, querying whether correct matched pairs which are reserved by screening exist in each grid, and if so, taking the point which is closest to a BRIEF description Fu Hanming as a key point of the grid; if the correct feature point matching pair does not exist in the grid, taking the largest harris response corner point in the FAST feature point in the grid as a key point of the grid;
(2) Selecting 4 pairs of correct matching pairs which are reserved in screening from the query image, and solving perspective transformation tau generated between the query image and the training matching image:
let z=a 33 ＝1/>
Obtaining 8 equations from the 4 points, solving 8 unknowns, and solving a perspective transformation matrix; and obtaining an area R of interest in the query image to correspond to an area in the matching training image through the matrix, and dividing grids for the area in the same way to select representative key points.
Optionally, modeling the key points as nodes of the graph, generating edges of the graph, and constructing the graph model includes:
modeling the selected key points as nodes of the graph, then calculating descriptors of the key points as attributes of the nodes, constructing a triangular network through coordinates of the key points by using Dirony triangulation, and forming edges of the graph with translational invariance, scaling invariance and rotational invariance; the interested region in the query image and the feature point information and the structure information corresponding to the region in the matched training image are converted into a graph model G α ＝(V α ,E α ) And G β ＝(V β ,E β ) Where G represents the graph model, α represents the query image, and β represents the matching training image.
Optionally, the performing the approximate matching of the graphs by using the random walk algorithm means: and searching a plurality of nodes with the maximum weight by using a PageRank graph matching algorithm to finish matching.
According to a second aspect of the present invention, there is provided a fast robust image recognition tracking apparatus based on structural features, comprising a memory, a processor and a computer program stored on the memory and executable on the processor, the processor being operable to perform the fast robust image recognition tracking method based on structural features as described above when executing the program.
Compared with the prior art, the embodiment of the invention has at least one of the following beneficial effects:
(1) The invention uses GMS algorithm to carry out quick and extremely robust image characteristic preliminary matching, and the screened correct matching pair can be simultaneously used for key point selection, two-image mapping transformation matrix calculation and similarity matrix K optimization, thereby optimizing the integral matching structure and accelerating the matching efficiency.
(2) The invention combines the graph model and the graph matching mechanism with the image matching, applies the relation and the structural characteristics among the characteristic points to the matching, optimizes the selection of the characteristic points and fuses the selection to the problem of searching the optimal solution. The object recognition can be completed under the condition of fewer improved feature points, and the accuracy is improved.
Drawings
Other features, objects and advantages of the present invention will become more apparent upon reading of the detailed description of non-limiting embodiments, given with reference to the accompanying drawings in which:
FIG. 1 is a flowchart of an image recognition tracking method according to a preferred embodiment of the present invention;
fig. 2 is a schematic diagram of detecting FAST corner points according to an embodiment of the present invention;
FIG. 3 is a schematic diagram of extracting BRIEF feature operators according to an embodiment of the invention;
fig. 4 is a schematic diagram of a GMS matching model according to an embodiment of the present invention;
fig. 5 is a diagram showing the effect of a GMS matching algorithm according to an embodiment of the present invention;
FIG. 6 is a schematic view of a given planar object of interest according to an embodiment of the present invention;
FIG. 7 is a diagram illustrating meshing according to an embodiment of the present invention;
FIG. 8 is a perspective transformation solution schematic diagram of an embodiment of the present invention;
fig. 9 is a diagram illustrating dironi triangulation according to an embodiment of the present invention;
FIG. 10 is a two-diagram fusion schematic diagram of an embodiment of the present invention;
FIG. 11 is a diagram illustrating the PageRank algorithm according to an embodiment of the present invention.
Detailed Description
The following describes embodiments of the present invention in detail: the embodiment is implemented on the premise of the technical scheme of the invention, and detailed implementation modes and specific operation processes are given. It should be noted that variations and modifications can be made by those skilled in the art without departing from the spirit of the invention, which falls within the scope of the invention.
Fig. 1 is a flowchart of an image recognition tracking method according to an embodiment of the invention. Referring to fig. 1, a fast robust image recognition tracking method based on structural features includes:
s11, adopting a GMS feature matching algorithm to perform feature point matching pair screening on the query image and the training matching image;
s12, if a correct feature point matching pair exists, equally dividing the interested region in the query image and the training matching image into small grids, and determining key points for each grid;
s13, modeling the key points as nodes of the graph, generating edges, constructing a graph model, and fusing feature matching with weight parameters matched with the graph;
s14, performing approximate matching on the image by using a random walk algorithm, and completing image identification tracking.
Preferably, a GMS (Grid-based Motion Statistics for Fast) feature matching algorithm is used for screening correct feature point matching pairs through a Grid division and motion statistics characteristic method. Equally dividing a given interested plane object R area into N grids, and taking the point closest to the matching distance as the key point of the grid if the screened correct characteristic point matching pair exists in the grid; if the correct feature point matching pair does not exist in the grid, the largest harris response corner in the grid is taken as the key point of the grid. Modeling the selected key points as nodes of the graph, and constructing edges of the graph by using Dirony triangulation. And then calculating their descriptors as attributes of the node, and fusing the feature matching with the weight parameters of the graph matching. And finally, searching a plurality of nodes with the maximum weight by using a PageRank random walk graph matching algorithm to finish matching.
The embodiment of the invention uses the structural features among the key points to assist matching, and can provide more accurate identification tracking performance under the condition of fewer feature point matching pairs.
Fig. 2 is a schematic diagram of detecting FAST corner points according to an embodiment of the present invention; FIG. 3 is a schematic diagram of an extracted feature operator according to an embodiment of the present invention. Conventionally, a SIFT (Scale-Invariant Feature Transform) algorithm is mostly used to extract features, and key points (feature points) are searched on different Scale spaces, wherein the points are very prominent and cannot be changed due to factors such as illumination, affine transformation and noise. The method has the defects that the key point searching speed is low, and the feature point descriptors are 128-dimensional vector characterization, so that the method is more time-consuming in feature matching. The ORB algorithm combines a detection method of FAST feature points with BRIEF feature descriptors, improves and optimizes the detection method based on the original detection method and the BRIEF feature descriptors, has invariance to noise and perspective transformation thereof, and also extracts and describes features of images with different scales, so that the scale problem is tried to be solved to a certain extent. The ORB algorithm has the speed 100 times that of SIFT, can ensure the calculation instantaneity, is more convenient to apply to engineering, and can well replace SIFT. Referring to fig. 2, in another preferred embodiment, the FAST corner is first extracted for each picture using the ORB algorithm (Oriented FAST and Rotated BRIEF), and the operator is calculated BRIEF (Binary Robust Indenpendent Elementary Features) for feature point matching pairs.
In one embodiment, reference may be made to the following specific steps:
(1) The FAST corner is extracted for each frame of image,
in the above formula, t is a threshold (default value is 10, different scene values are different), I p Representing the pixel value of the center pixel, I p→x Representing the pixel values in a circular template. Referring to fig. 2, the pixel value I of the center pixel p Less than the pixel value I at x in the surrounding pixel circular template p→x When the pixel belongs to the dark er, S p→x =d, the other two cases represent bright and similar, respectively. Such a block (circular) region can be classified into three types d, s and b. At this time, as long as the number of times d or b in the circular area is counted, as long as d or b occurs more than n (n=12 is generally set), indicating that the point is darker or brighter than the surrounding, the point is considered as a candidate corner point. If X feature points are to be extracted from the image, reducing a threshold t to enable the feature points detected by a FAST algorithm to be larger than X, calculating Harris response values R of the feature points at the position of the feature points, and taking the points with the first X response values as the FAST feature points;
(2) Referring to fig. 3, for each FAST feature point, taking a neighborhood large window of s×s as a center, randomly selecting point pairs (generally 256 pairs) in the large window, performing binary assignment, and calculating a BRIEF operator.
Fig. 4 is a schematic diagram of a GMS matching model according to an embodiment of the present invention. Referring to fig. 4, in another preferred embodiment, the GMS feature matching algorithm is used to screen the correct feature point matching pairs, which may be as follows:
(1) Matching the characteristic points through a BF (Brute Force) violent matching algorithm;
firstly, selecting a feature point from a query image, then sequentially carrying out BRIEF description Fu Hanming distance test on the feature point and the feature point in a matched training image, and finally returning the feature point closest to the feature point to form a feature point matching set from the query image to the matched training image.
(2) Referring to FIG. 4, an image is divided into G grids (typically G is 20×20), and feature point pairs F with well-matched BF are calculated p And F is equal to q Number of correct matches S in the vicinity pq ，
Wherein the method comprises the steps ofIs a grid { p } k ,q k Matches between.
Threshold of true-false matchingEta is larger to take empirical value of 6, n p Is the total feature number of 3 x 3 grids. Comparing the number S of correct matches pq And threshold t p To determine if the point is correctly matched:
the GMS feature matching algorithm screens the effect of correct matching pairs referring to fig. 5, the connection is the correct matching pair after deletion, and it can be seen that the matching condition is almost correct, and there are few incorrect matching connection lines.
FIG. 6 is a schematic view of a given planar object of interest according to an embodiment of the present invention; FIG. 7 is a diagram illustrating meshing according to an embodiment of the present invention; FIG. 8 is a perspective transformation solution schematic of an embodiment of the present invention.
In another preferred embodiment, the given planar object of interest R region is equally divided into N grids, and a representative keypoint is selected for each grid based on an empirical value N, typically 6×10. In one embodiment, the process may be performed as follows:
(1) Referring to fig. 6 and 7, a region of interest R is drawn in a query image, the region R is equally divided into N grids, whether a correct matching pair is found by screening is queried in each grid, and if so, a point closest to the BRIEF description Fu Hanming by the matching pair is taken as a key point of the grid. If the correct feature point matching pair does not exist in the grid, the largest harris response corner in the FAST feature point in the grid is taken as the key point of the grid.
(2) Selecting 4 pairs of correct matching pairs (grid key points are selected preferentially) which are reserved in screening in the query image, and referring to fig. 8, obtaining perspective transformation tau generated by two graphs:
let z=a 33 ＝1/>
8 equations can be obtained for the 4 points, and 8 unknowns can be solved, so that the perspective transformation matrix can be solved. Through the matrix, the region R of interest in the query image can be obtained to correspond to the region in the matching training image, and the region is divided into grids in the same way to select representative key points.
Fig. 9 is a diagram illustrating dironi triangulation according to an embodiment of the present invention. Referring to fig. 9, in another preferred embodiment, the key point feature descriptors and the structural feature data determined in the mesh are converted into a data structure of a graph, each graph g= (V, E) is represented as a set of nodes V and edges E, the key points are modeled as nodes of the graph, and a graph model is constructed, specifically, may be adopted: the selected keypoints are modeled as nodes of the graph, and their descriptors are then computed as attributes of this node. Triangle mesh is constructed by coordinates of key points using dironi triangulation, forming edges of the graph with translational, scaling and rotational invariance.
FIG. 10 is a two-diagram fusion schematic diagram of an embodiment of the present invention. Referring to fig. 10, in another preferred embodiment, the fusion of feature matches with weight parameters of graph matches may be implemented as follows:
(1) Referring to fig. 10, the graph matching problem is translated, the result of which is represented by an assignment matrix (assignment matrix) X. Since both drawing grids are N, the assignment matrix is an N x N {0,1} matrix with each row, each column of the assignment matrix and only one element being 1. In the graph matching problem, the first-order similarity between nodes and the second-order similarity between edges between graph structures are considered at the same time. The similarity matrix K simultaneously comprises first-order node similarity information and second-order side similarity information. The matching problem is converted into the following mathematical form:
x * ＝argmaxε(x)＝x T Kx
s.t.X1 n ≤1 m ,X T 1 m ≤1 n
wherein c i,a Representative graph G α Intermediate node i to graph G β Consistency of middle node a, graph G α Graph G β Representing the graph models generated from the query image and the matching training image, d i,j,a,b Representative graph G α Middle line segment ij to graph G β The consistency of the middle line segment ab, wherein X is a corresponding matrix to represent a matching result; i. j represents diagram G α The nodes a and b in the graph G β Is a node in (a); graph G α Graph G β Respectively by point set V α Edge set E α Sum point set V β Edge set E β Composition, X i,a Representation of diagram G α Node i and graph G β The correspondence of node b in (i.e. and only when node i e V) α Corresponding to node a epsilon V β Time X i,a ＝1，X j,b And the same is done; epsilon (X) is an evaluation function and represents the graph G under the corresponding assignment matrix X α And graph G β The higher the evaluation quality, the more similar the two graphs are;vectorization of matrix X; x is x T Transpose of vector x; x is x * Vectorizing the solved optimal corresponding matrix to represent the optimal matching relation of the two graphs; k is a similarity matrix, and comprises first-order node similarity information and second-order side similarity information; 1 n A column vector representing N vectors, the constraint ensuring that each part matches at most once, since both graphs consist of N nodes, where n=m=n;
(2) Assigning a similarity matrix K:
wherein A is α And A β Is G α And G β Is a contiguous matrix of (a) a plurality of (b) a plurality of (c). Combining feature points and geometric constraints to make And->BRIEF descriptor for Key points, < ->Is a graph G α Midpoint v i And v j Coordinate positions of (2); />、/>Is a graph G β Midpoint v a And v b Coordinate positions of (2); τ is perspective transformation of the query image and the training matching image; d, d i,j,a,b And (τ) represents the consistency of the edges (i, j) and edges (a, b) under the perspective transformation τ, ω being chosen large enough to ensure that the pair of edges are similar to each other by more than 0. The diagonal elements of the similarity matrix K contain node-to-node similarity information, and the non-diagonal elements contain side-to-side similarity information.
(3) Filtering candidate matching, preferentially selecting key points reserved by GMS screening, sorting according to matching distance, and reserving at most n c Set of key points C t And the similarity is highest, deletion is not C t The rows and columns of key points are used to concentrate the similarity matrix K to reduce the size to n c 2 N 2 . Take 5 as n c Is a desired value for (a). Of course, in other embodiments, n c Other values may also be used.
FIG. 11 is a diagram illustrating the PageRank algorithm according to an embodiment of the present invention. In another preferred embodiment, described with reference to FIG. 11. The existing graph matching methods are mainly divided into an accurate graph matching method and an approximate graph matching method. The exact-graph matching method is more complex and time consuming than the approximate-graph matching method. Moreover, in an actual graph, the exact graph matching method is not robust to external points and various changes. Approximate graph matching has low complexity, high efficiency and wider application than exact graph matching. Therefore, although the present problem is close to exact graph matching of N to N nodes, it is more appropriate to convert it to approximate graph matching in view of efficiency and robustness. And expanding the PageRank algorithm to the problem of graph matching, and finding out a plurality of nodes with the maximum weight to complete matching. The principle is that through random walk of random walker, the whole graph reaches stable state, each node on the associated graph corresponds to a similar probability value, namely the probability that the candidate matching pair is reliable matching, the larger the similar probability value of the node, the more correct the matching pair corresponding to the node. Specifically, the following specific steps may be adopted:
(1) And initializing the access probability of each point for the assigned similarity matrix K, enabling the similarity matrix K to start jumping and random walk to the matching constraint, and finally converging to the quasi-stable distribution. For each candidate match (i, a) ∈C t Initializing the corresponding probability as:
(2) In order to solve the end point problem and the trap problem, the transition probability is set to be eta, and the random jump probability is set to be (1-eta). The random walk iterative formula becomes X' =ηkx+ (1- η) e, e is an equally divided random jump probability matrix. And iterating until convergence, selecting a plurality of nodes with the maximum weights, and completing matching.
The preferred features of the above embodiments may be used alone in any of the embodiments, or in any combination without interfering with each other. In addition, portions of the embodiments not described in detail may be implemented using prior art techniques.
Based on the foregoing embodiments, in another embodiment, the present invention further provides a fast robust image recognition tracking apparatus based on structural features, including a memory, a processor, and a computer program stored on the memory and executable on the processor, where the processor executes the program and is configured to perform the method for fast robust image recognition tracking based on structural features in the foregoing optional embodiment.
Optionally, a memory for storing a program; memory, which may include volatile memory (english) such as random-access memory (RAM), such as static random-access memory (SRAM), double data rate synchronous dynamic random-access memory (Double Data Rate Synchronous Dynamic Random Access Memory, DDR SDRAM), and the like; the memory may also include a non-volatile memory (English) such as a flash memory (English). The memory 62 is used to store computer programs (e.g., application programs, functional modules, etc. that implement the methods described above), computer instructions, etc., which may be stored in one or more memories in a partitioned manner. And the above-described computer programs, computer instructions, data, etc. may be invoked by a processor.
The computer programs, computer instructions, etc. described above may be stored in one or more memories in partitions. And the above-described computer programs, computer instructions, data, etc. may be invoked by a processor.
A processor for executing the computer program stored in the memory to implement the steps in the method according to the above embodiment. Reference may be made in particular to the description of the embodiments of the method described above.
The processor and the memory may be separate structures or may be integrated structures that are integrated together. When the processor and the memory are separate structures, the memory and the processor may be connected by a bus coupling.
The embodiment of the invention aims at the problem of image area identification and tracking, starts from a GMS (Grid-based Motion Statistics for Fast) quick search technology, adds a graph matching algorithm based on geometric constraint, provides a quick and robust image identification and tracking method and device, can effectively accelerate the matching identification method, and provides more accurate identification and tracking performance under the condition of fewer characteristic point matching pairs.
The foregoing describes specific embodiments of the present invention. It is to be understood that the invention is not limited to the particular embodiments described above, and that various changes or modifications may be made by those skilled in the art within the scope of the appended claims without affecting the spirit of the invention.
Claims (8)
1. The rapid robust image identification tracking method based on the structural features is characterized by comprising the following steps of:
adopting a GMS feature matching algorithm to perform feature point matching pair screening on the query image and the training matching image;
if the correct feature point matching pair exists, equally dividing the interested region in the query image and the training matching image into small grids, and determining a key point for each grid;
modeling key points as nodes of the graph, generating edges, constructing a graph model, and fusing feature matching with weight parameters matched with the graph;
the approximate matching of the images is completed by using a random walk algorithm, and the image recognition and tracking are completed;
modeling the key points as nodes of the graph, generating edges of the graph, and constructing a graph model comprises:
modeling the selected key points as nodes of the graph, then calculating descriptors of the key points as attributes of the nodes, constructing a triangular network through coordinates of the key points by using Dirony triangulation, and forming edges of the graph with translational invariance, scaling invariance and rotational invariance; the interested region in the query image and the feature point information and the structure information corresponding to the region in the matched training image are converted into a graph model G α ＝(V α ,E α ) And G β ＝(V β ,E β ) Wherein G represents a graph model, α represents a query image, and β represents a matching training image;
the fusing of the feature matching and the weight parameters of the graph matching comprises the following steps:
(1) Converting the graph matching problem, wherein the result is represented by an assignment matrix X; the grids of the query image and the training matching image are N, the assignment matrix is an N multiplied by N {0,1} matrix, wherein each row, each column and only one element of the assignment matrix is 1;
in the graph matching problem, the first-order similarity between nodes and the second-order similarity between edges of the graph structure are considered, and the similarity matrix K simultaneously comprises first-order node similarity information and second-order edge similarity information; the matching problem is converted into the following mathematical form:
x * ＝argmaxε(x)＝x T Kx
s.t.X1 n ≤1 m ,X T 1 m ≤1 n
wherein c i,a Representative graph G α Intermediate node i to graph G β Consistency of middle node a, graph G α Graph G β Representing the graph models generated from the query image and the matching training image, d i,j,a,b Representative graph G α Middle line segment ij to graph G β The consistency of the middle line segment ab, wherein X is a corresponding matrix to represent a matching result; i. j represents diagram G α The nodes a and b in the graph G β Is a node in (a); graph G α Graph G β Respectively by point set V α Edge set E α Sum point set V β Edge set E β Composition, X i,a Representation of diagram G α Node i and graph G β The correspondence of node b in (i.e. and only when node i e V) α Corresponding to node a epsilon V β Time X i,a ＝1，X j,b And the same is done; epsilon (X) is an evaluation function and represents the graph G under the corresponding assignment matrix X α And graph G β The higher the evaluation quality, the more similar the two graphs are;vectorization of matrix X; x is x T Transpose of vector x; x is x * Vectorizing the solved optimal corresponding matrix to represent the optimal matching relation of the two graphs; k is a similarity matrix, and comprises first-order node similarity information and second-order side similarity information; 1 n A column vector representing N vectors, the constraint ensuring that each part matches at most once, since both graphs consist of N nodes, where n=m=n;
(2) Assigning a similarity matrix K:
wherein A is α And A β Is G α And G β Is a contiguous matrix of (a);representation of diagram G α Midpoint v i And point v j When v is i And v j When there are connected edges->And the same is done; c i,a Is a graph G α Midpoint v i And graph G β Midpoint v a Is the consistency of (3); d, d i,j,a,b (τ) is the graph G α Middle edge (i, j) and graph G β Consistency of the middle edges (a, b);
combining feature points and geometric constraints to make f i α 、/>BRIEF descriptors that are key points; />Is a graph G α Midpoint v i And v j Coordinate positions of (2); />Is a graph G β Midpoint v a And v b Coordinate positions of (2); τ is perspective transformation of the query image and the training matching image;
d i,j,a,b (τ) represents the perspective transformation τ lower edge (i, j) and edgeThe consistency of (a, b), ω is chosen large enough to ensure that the pair of edges is more than 0, the diagonal elements of the similarity matrix K contain node-to-node similarity information, and the non-diagonal elements contain edge-to-edge similarity information;
(3) Filtering candidate matching, selecting key points reserved by GMS screening, sorting according to matching distance, and reserving at most n c Set of key points C t And the similarity is highest, deletion is not C t The rows and columns of key points are used to concentrate the similarity matrix K to reduce the size to n c 2 N 2 。
2. The method for identifying and tracking the rapid robust image based on the structural features according to claim 1, wherein the step of performing feature point matching pair screening on the query image and the training matching image by using a GMS feature matching algorithm comprises the steps of:
matching the characteristic points through a BF violent matching algorithm;
dividing the query image and the training matching image into G grids respectively, and calculating feature point pairs F with good BF matching p And F is equal to q Number of correct matches S in the vicinity pq ，F p And F is equal to q Characteristic points in the query image and the training matching image are respectively screened as matching pairs by a BF violent matching algorithm;
by characteristic point F p And F q The grid is taken as the center, 9 grids around the grid are selected as regions to calculate the matching number, wherein K=3x3 is taken as the region around the gridIs a grid { p } k ,q k The matching pair number between the two is between 1 and 9;
setting a threshold value for true-false matchingEta takes the empirical value of 6, n p Is the total feature number of the grid; comparing the number S of correct matches pq And threshold t p To determine if the point is correctly matched:
wherein p and q are respectively expressed as characteristic points F in the query image p Matching feature points F in the image with training q 。
3. The method for identifying and tracking the rapid robust image based on the structural features according to claim 2, wherein the matching of the feature points by the BF violent matching algorithm comprises:
firstly, selecting a characteristic point from a query image;
then sequentially carrying out BRIEF description Fu Hanming distance test with the feature points in the matched training images;
and finally, returning the nearest characteristic points to form a characteristic point matching set from the query image to the matched training image.
4. The rapid robust image recognition tracking method based on structural features of claim 3, wherein the BRIEF description Fu Hanming distance test, wherein BRIEF descriptors are obtained by:
extracting FAST feature points from each picture by using an ORB algorithm;
and taking each FAST characteristic point as a center, taking an S multiplied by S neighborhood large window, randomly selecting point pairs in the large window, carrying out binary assignment, and calculating BRIEF descriptors.
5. The method for quickly identifying and tracking robust images based on structural features according to claim 1, wherein the equally dividing the query image and the region of interest in the training matching image into small grids, determining key points for each grid, comprises:
(1) Drawing a region of interest R in a query image, equally dividing the region R into N grids, querying whether correct matched pairs which are reserved by screening exist in each grid, and if so, taking the point which is closest to a BRIEF description Fu Hanming as a key point of the grid; if the correct feature point matching pair does not exist in the grid, taking the largest harris response corner point in the FAST feature point in the grid as a key point of the grid;
(2) Selecting 4 pairs of correct matching pairs which are reserved in screening from the query image, and solving perspective transformation tau generated between the query image and the training matching image:
let->
Obtaining 8 equations from the 4 points, solving 8 unknowns, and solving a perspective transformation matrix; and obtaining an area R of interest in the query image to correspond to an area in the matching training image through the matrix, and dividing grids for the area in the same way to select representative key points.
6. The method for identifying and tracking the rapid robust image based on the structural features according to claim 1, wherein the approximate matching of the image is achieved by using a random walk algorithm, which is that: and searching a plurality of nodes with the maximum weight by using a PageRank graph matching algorithm to finish matching.
7. The method for identifying and tracking the rapid robust image based on the structural features according to claim 6, wherein the searching the plurality of nodes with the maximum weight by using the PageRank graph matching algorithm comprises the following steps:
(1) Initializing access probability of each point for the assigned similarity matrix K, enabling the similarity matrix K to start jumping random walk to a matching constraint, and finally converging to quasi-stable distribution; for each candidate match (i, a) ∈C t Initializing the corresponding probability as:
(2) In order to solve the problem of termination points and trap points, setting the transition probability as eta, setting the random jump probability as (1-eta), changing the random walk iteration formula into X' =eta KX+ (1-eta) e, wherein e is an equally divided random jump probability matrix; and iterating until convergence, selecting a plurality of nodes with the maximum weights, and completing matching.
8. A fast robust image recognition tracking apparatus based on structural features, comprising a memory, a processor and a computer program stored on the memory and executable on the processor, wherein the processor is operable to perform the method of any of claims 1-7 when the program is executed by the processor.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010229998.8A CN111461196B (en) | 2020-03-27 | 2020-03-27 | Rapid robust image identification tracking method and device based on structural features |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010229998.8A CN111461196B (en) | 2020-03-27 | 2020-03-27 | Rapid robust image identification tracking method and device based on structural features |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN111461196A CN111461196A (en) | 2020-07-28 |
| CN111461196B true CN111461196B (en) | 2023-07-21 |
Family
ID=71683310
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202010229998.8A Active CN111461196B (en) | 2020-03-27 | 2020-03-27 | Rapid robust image identification tracking method and device based on structural features |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN111461196B (en) |
Families Citing this family (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN115601576B (en) * | 2022-12-12 | 2023-04-07 | 云南览易网络科技有限责任公司 | Image feature matching method, device, equipment and storage medium |
| CN116778141B (en) * | 2023-08-28 | 2023-12-22 | 深圳联友科技有限公司 | ORB algorithm-based method for rapidly identifying and positioning picture |
Citations (10)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102708370A (en) * | 2012-05-17 | 2012-10-03 | 北京交通大学 | Method and device for extracting multi-view angle image foreground target |
| CN105513094A (en) * | 2015-12-17 | 2016-04-20 | 上海交通大学 | Stereo vision tracking method and stereo vision tracking system based on 3D Delaunay triangulation |
| CN106682700A (en) * | 2017-01-05 | 2017-05-17 | 北京细推科技有限公司 | Block quick matching algorithm based on key point description operator |
| CN106780579A (en) * | 2017-01-17 | 2017-05-31 | 华中科技大学 | A kind of ultra-large image characteristic point matching method and system |
| CN108537287A (en) * | 2018-04-18 | 2018-09-14 | 北京航空航天大学 | Image closed loop detection method and device based on graph model |
| CN109598750A (en) * | 2018-12-07 | 2019-04-09 | 中国地质大学（武汉） | One kind being based on the pyramidal large scale differential image characteristic point matching method of deformation space |
| CN109697692A (en) * | 2018-12-29 | 2019-04-30 | 安徽大学 | One kind being based on the similar feature matching method of partial structurtes |
| CN109784353A (en) * | 2017-11-14 | 2019-05-21 | 北京三星通信技术研究有限公司 | A kind of matched method of non-directed graph, equipment and storage medium |
| CN109816706A (en) * | 2019-02-01 | 2019-05-28 | 辽宁工程技术大学 | A kind of smoothness constraint and triangulation network equal proportion subdivision picture are to dense matching method |
| CN110032983A (en) * | 2019-04-22 | 2019-07-19 | 扬州哈工科创机器人研究院有限公司 | A kind of track recognizing method based on ORB feature extraction and FLANN Rapid matching |
-
2020
- 2020-03-27 CN CN202010229998.8A patent/CN111461196B/en active Active
Patent Citations (10)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102708370A (en) * | 2012-05-17 | 2012-10-03 | 北京交通大学 | Method and device for extracting multi-view angle image foreground target |
| CN105513094A (en) * | 2015-12-17 | 2016-04-20 | 上海交通大学 | Stereo vision tracking method and stereo vision tracking system based on 3D Delaunay triangulation |
| CN106682700A (en) * | 2017-01-05 | 2017-05-17 | 北京细推科技有限公司 | Block quick matching algorithm based on key point description operator |
| CN106780579A (en) * | 2017-01-17 | 2017-05-31 | 华中科技大学 | A kind of ultra-large image characteristic point matching method and system |
| CN109784353A (en) * | 2017-11-14 | 2019-05-21 | 北京三星通信技术研究有限公司 | A kind of matched method of non-directed graph, equipment and storage medium |
| CN108537287A (en) * | 2018-04-18 | 2018-09-14 | 北京航空航天大学 | Image closed loop detection method and device based on graph model |
| CN109598750A (en) * | 2018-12-07 | 2019-04-09 | 中国地质大学（武汉） | One kind being based on the pyramidal large scale differential image characteristic point matching method of deformation space |
| CN109697692A (en) * | 2018-12-29 | 2019-04-30 | 安徽大学 | One kind being based on the similar feature matching method of partial structurtes |
| CN109816706A (en) * | 2019-02-01 | 2019-05-28 | 辽宁工程技术大学 | A kind of smoothness constraint and triangulation network equal proportion subdivision picture are to dense matching method |
| CN110032983A (en) * | 2019-04-22 | 2019-07-19 | 扬州哈工科创机器人研究院有限公司 | A kind of track recognizing method based on ORB feature extraction and FLANN Rapid matching |
Non-Patent Citations (1)
| Title |
|---|
| GMS: Grid-based Motion Statistics for Fast, Ultra-robust Feature Correspondence;JiaWang Bian等;《CVPR》;20171109;全文 * |
Also Published As
| Publication number | Publication date |
|---|---|
| CN111461196A (en) | 2020-07-28 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| Zhao et al. | Deep hough transform for semantic line detection | |
| WO2022002150A1 (en) | Method and device for constructing visual point cloud map | |
| US11341738B2 (en) | Using a probabtilistic model for detecting an object in visual data | |
| CN111780764B (en) | Visual positioning method and device based on visual map | |
| CN107301402B (en) | Method, device, medium and equipment for determining key frame of real scene | |
| CN112967341B (en) | Indoor visual positioning method, system, equipment and storage medium based on live-action image | |
| CN111179419A (en) | Three-dimensional key point prediction and deep learning model training method, device and equipment | |
| WO2020020047A1 (en) | Key point matching method and device, terminal device and storage medium | |
| CN111461196B (en) | Rapid robust image identification tracking method and device based on structural features | |
| CN110033514B (en) | Reconstruction method based on point-line characteristic rapid fusion | |
| WO2019041660A1 (en) | Face deblurring method and device | |
| CN112613553B (en) | Picture sample set generation method and device, computer equipment and storage medium | |
| CN111783753A (en) | Pedestrian re-identification method based on semantic consistency horizontal bar and foreground correction | |
| CN113095158A (en) | Handwriting generation method and device based on countermeasure generation network | |
| CN111414958B (en) | Multi-feature image classification method and system for visual word bag pyramid | |
| CN108304838B (en) | Picture information identification method and terminal | |
| JP6778625B2 (en) | Image search system, image search method and image search program | |
| CN113378864B (en) | Method, device and equipment for determining anchor frame parameters and readable storage medium | |
| CN112632315B (en) | Method and device for retrieving remote sensing image | |
| CN113947154A (en) | Target detection method, system, electronic equipment and storage medium | |
| CN113849679A (en) | Image retrieval method, image retrieval device, electronic equipment and storage medium | |
| TWI776668B (en) | Image processing method and image processing system | |
| CN116721301B (en) | Training method, classifying method, device and storage medium for target scene classifying model | |
| Sebe et al. | Which ranking metric is optimal? with applications in image retrieval and stereo matching | |
| CN116863085B (en) | Three-dimensional reconstruction system, three-dimensional reconstruction method, electronic equipment and storage medium |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |