CN111461196A - Method and device for identifying and tracking fast robust image based on structural features - Google Patents

Method and device for identifying and tracking fast robust image based on structural features Download PDF

Info

Publication number
CN111461196A
CN111461196A CN202010229998.8A CN202010229998A CN111461196A CN 111461196 A CN111461196 A CN 111461196A CN 202010229998 A CN202010229998 A CN 202010229998A CN 111461196 A CN111461196 A CN 111461196A
Authority
CN
China
Prior art keywords
matching
image
graph
feature
training
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Granted
Application number
CN202010229998.8A
Other languages
Chinese (zh)
Other versions
CN111461196B (en
Inventor
安平
孙源航
尤志翔
高伟
王嶺
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
SHANGHAI MEDIA & ENTERTAINMENT TECHNOLOGY GROUP
University of Shanghai for Science and Technology
Original Assignee
SHANGHAI MEDIA & ENTERTAINMENT TECHNOLOGY GROUP
University of Shanghai for Science and Technology
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by SHANGHAI MEDIA & ENTERTAINMENT TECHNOLOGY GROUP, University of Shanghai for Science and Technology filed Critical SHANGHAI MEDIA & ENTERTAINMENT TECHNOLOGY GROUP
Priority to CN202010229998.8A priority Critical patent/CN111461196B/en
Publication of CN111461196A publication Critical patent/CN111461196A/en
Application granted granted Critical
Publication of CN111461196B publication Critical patent/CN111461196B/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
    • G06V10/74—Image or video pattern matching; Proximity measures in feature spaces
    • G06V10/75—Organisation of the matching processes, e.g. simultaneous or sequential comparisons of image or video features; Coarse-fine approaches, e.g. multi-scale approaches; using context analysis; Selection of dictionaries
    • G06V10/751—Comparing pixel values or logical combinations thereof, or feature values having positional relevance, e.g. template matching
    • Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
    • Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
    • Y02D—CLIMATE CHANGE MITIGATION TECHNOLOGIES IN INFORMATION AND COMMUNICATION TECHNOLOGIES [ICT], I.E. INFORMATION AND COMMUNICATION TECHNOLOGIES AIMING AT THE REDUCTION OF THEIR OWN ENERGY USE
    • Y02D10/00—Energy efficient computing, e.g. low power processors, power management or thermal management

Landscapes

  • Engineering & Computer Science (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Theoretical Computer Science (AREA)
  • Evolutionary Computation (AREA)
  • Computing Systems (AREA)
  • Databases & Information Systems (AREA)
  • Artificial Intelligence (AREA)
  • General Health & Medical Sciences (AREA)
  • Medical Informatics (AREA)
  • Software Systems (AREA)
  • Health & Medical Sciences (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Multimedia (AREA)
  • Image Analysis (AREA)

Abstract

The invention provides a fast robust image recognition tracking method and a device based on structural features, wherein the method adopts GMS feature matching algorithm to carry out feature point matching pair screening on a query image and a training matching image; if a correct feature point matching pair exists, equally dividing interest areas in the query image and the training matching image into small grids, and determining key points for each grid; modeling key points as graph nodes, constructing a graph model, and fusing feature matching and the weight parameters of graph matching; and (4) completing approximate matching of the image by using a random walk algorithm, and completing image identification and tracking. The invention can effectively accelerate the matching identification method and provide more accurate identification tracking performance under the condition of less characteristic point matching pairs.

Description

Method and device for identifying and tracking fast robust image based on structural features
Technical Field
The invention relates to the field of image matching in computer vision, in particular to a method and a device for identifying and tracking a fast robust image based on structural features.
Background
The task of image matching is to find the corresponding relationship between pixel points in two or more images of the same scene. The method is a very important hotspot problem in the research fields of computer vision, information processing and the like, and is also the basis of many computer vision theories and applications. The image matching technology mainly comprises a method based on gray scale matching and a method based on feature matching. The method based on the gradation matching can obtain a good effect in general, but has poor performance in a region where gradation information is relatively small. The method based on feature matching can obtain feature points in an original image according to certain feature extraction operators, the feature points can represent the image better, and the image matching is completed through the mapping relation among the feature points.
The mainstream image matching method still has some disadvantages, the requirement of the method based on gray scale matching on an image source is very high, the gray scale values of the same object shot by different cameras have larger difference, and the method needs to use a template graph to circularly compare in a test graph, has higher calculation complexity and is difficult to practically apply. The feature matching-based method is small in calculation amount and is also suitable for the situation that deformation, external points and occlusion exist in an image, but high-quality feature points are needed, and the mainstream feature matching method only focuses on the relationship among the feature points and does not consider the structural features of the image, so that the generation of wrong matching can be caused. In summary, how to balance the efficiency and accuracy of image matching becomes an urgent problem to be solved.
Disclosure of Invention
Aiming at the defects of the prior art, the invention aims to provide a method and a device for quickly identifying and tracking robust images based on structural features, which are used for assisting matching by using the structural features among key points and can provide more accurate identification and tracking performance under the condition of fewer matched feature point pairs.
According to a first aspect of the present invention, there is provided a fast robust image recognition and tracking method based on structural features, comprising:
adopting GMS feature matching algorithm to carry out feature point matching pair screening on the query image and the training matching image;
if a correct feature point matching pair exists, equally dividing interest areas in the query image and the training matching image into small grids, and determining key points for each grid;
modeling key points as graph nodes, generating edges, constructing a graph model, and fusing feature matching and graph matching weight parameters;
and (4) completing approximate matching of the image by using a random walk algorithm, and completing image identification and tracking.
Optionally, the selecting the feature point matching pairs of the query image and the training matching image by using a GMS feature matching algorithm includes:
matching the feature points by a BF violence matching algorithm;
dividing the query image and the training matching image into G grids respectively, and calculating the characteristic point pairs F matched with BFpAnd FqNumber of correct matches S in the vicinitypq，FpAnd FqRespectively, the characteristic points in the query image and the training matching image are screened as matching pairs by a BF violence matching algorithm;
Figure BDA0002428988120000021
by characteristic point FpAnd FqTaking the located grid as the center, respectively selecting 9 grids around the located grid, K being 3 × 3, as the area to calculate the matching number, wherein
Figure BDA0002428988120000024
Is a grid { pk,qkThe number of matching pairs between every two adjacent pairs is 1-9;
threshold for true and false matching
Figure BDA0002428988120000022
η taking an empirical value of 6, niIs the total number of features of the grid; comparing the correct matching number SpqAnd a threshold value tpTo determine whether it shouldThe points are correctly matched:
Figure BDA0002428988120000023
wherein p and q are respectively expressed as characteristic points F in the query imagepMatching feature points F in images with trainingq。
Optionally, the matching the feature points by a BF violence matching algorithm includes:
firstly, selecting a characteristic point in a query image;
then carrying out BRIEF descriptor Hamming distance test on the feature points in the matched training images in sequence;
and finally, returning the feature points with the closest distance to form a feature point matching set from the query image to the matching training image.
Optionally, the BRIEF descriptor hamming distance test, wherein BRIEF descriptor is obtained by:
extracting FAST characteristic points from each picture by using an ORB algorithm;
and taking each FAST characteristic point as a center, taking a neighborhood large window of S × S, randomly selecting point pairs in the large window, carrying out binary assignment, and calculating a BRIEF descriptor.
Optionally, the equally dividing the region of interest in the query image and the training matching image into small grids, and determining a keypoint for each grid includes:
(1) drawing an interested region R in a query image, equally dividing the R region into N grids, querying whether a correct matching pair reserved by screening exists in each grid, and if so, taking a point with the closest Hamming distance of a BRIEF descriptor of the matching pair as a key point of the grid; if no correct feature point matching pair exists in the grid, taking the maximum harris response corner point in the FAST feature points in the grid as the key point of the grid;
(2) selecting 4 pairs of correct matching pairs reserved by screening from the query image, and solving the perspective transformation tau generated between the query image and the training matching image:
Figure BDA0002428988120000031
let Z be a33＝1
Figure BDA0002428988120000032
Obtaining 8 equations from 4 points, and solving 8 unknowns to obtain a perspective transformation matrix; and obtaining the region of interest R in the query image corresponding to the region in the matching training image through the matrix, and dividing the grid of the region in the same way to select representative key points.
Optionally, the modeling the key points as nodes of a graph, generating edges of the graph, and constructing the graph model includes:
modeling the selected key points as the nodes of the graph, then calculating descriptors of the key points as the attributes of the nodes, using Dirony triangulation, constructing a triangular net through the coordinates of the key points to form the edges of the graph, and having the invariance of translation, scaling and rotation; so far, the interest area in the query image and the feature point information and the structure information corresponding to the area in the matching training image are converted into a graph model Gα＝(Vα,Eα) And Gβ＝(Vβ,Eβ) Where G represents a graph model, α represents a query image, and β represents a matching training image.
Optionally, the performing of approximate matching on the graph by using a random walk algorithm is to: and searching a plurality of nodes with the maximum weight by using a PageRank graph matching algorithm to complete matching.
According to a second aspect of the present invention, there is provided a fast robust image recognition and tracking apparatus based on structural features, comprising a memory, a processor and a computer program stored in the memory and executable on the processor, wherein the processor is operable to execute the fast robust image recognition and tracking method based on structural features.
Compared with the prior art, the embodiment of the invention has at least one of the following beneficial effects:
(1) the invention uses GMS algorithm to carry out fast and extremely robust image characteristic preliminary matching, and the screened correct matching pair can be simultaneously used for key point selection, two-image mapping transformation matrix solving and similarity matrix K optimization, thereby optimizing the whole matching structure and accelerating the matching efficiency.
(2) The invention combines the graph model and graph matching mechanism with the image matching, applies the relation and the structural characteristics between the characteristic points to the matching, optimizes the selection of the characteristic points and integrates the selection into the problem of finding the optimal solution. The object recognition can be completed under the condition of fewer characteristic points, and the accuracy is improved.
Drawings
Other features, objects and advantages of the invention will become more apparent upon reading of the detailed description of non-limiting embodiments with reference to the following drawings:
FIG. 1 is a flow chart of an image recognition and tracking method according to a preferred embodiment of the present invention;
fig. 2 is a schematic diagram of detecting a FAST corner according to an embodiment of the present invention;
FIG. 3 is a schematic diagram of an operator for extracting BRIEF features according to an embodiment of the present invention;
FIG. 4 is a diagram of a GMS matching model according to an embodiment of the present invention;
fig. 5 is a diagram illustrating the effect of the GMS matching algorithm according to an embodiment of the present invention;
FIG. 6 is a schematic view of a given planar object of interest according to an embodiment of the present invention;
FIG. 7 is a schematic diagram of meshing according to an embodiment of the present invention;
FIG. 8 is a schematic diagram of a perspective transformation solution according to an embodiment of the present invention;
FIG. 9 is a schematic diagram of Dirony triangulation according to an embodiment of the present invention;
FIG. 10 is a schematic diagram of a fusion of two views according to an embodiment of the present invention;
FIG. 11 is a diagram of the PageRank algorithm according to an embodiment of the invention.
Detailed Description
The following examples illustrate the invention in detail: the embodiment is implemented on the premise of the technical scheme of the invention, and a detailed implementation mode and a specific operation process are given. It should be noted that, for a person skilled in the art, several variations and modifications can be made without departing from the inventive concept, which falls within the scope of the present invention.
Fig. 1 is a flowchart of an image recognition and tracking method according to an embodiment of the present invention. Referring to fig. 1, a fast robust image recognition and tracking method based on structural features includes:
s11, adopting GMS feature matching algorithm to screen the query image and the training matching image by feature point matching pair;
s12, if a correct feature point matching pair exists, equally dividing the interest areas in the query image and the training matching image into small grids, and determining key points for each grid;
s13, using the key point modeling as the node of the graph, generating edges, constructing a graph model, and fusing the feature matching and the weight parameters of the graph matching;
and S14, completing approximate matching of the graph by using a random walk algorithm, and completing image recognition and tracking.
Preferably, a GMS (Grid-based Motion Statistics for fast) feature matching algorithm is used for screening correct feature point matching pairs by a method of Grid division and Motion statistical characteristics. Equally dividing a given region of the object R of the plane of interest into N grids, and if a correctly screened feature point matching pair exists in the grid, taking a point with the closest matching distance as a key point of the grid; if no correct feature point matching pair exists in the grid, the maximum harris response corner point in the grid is taken as the key point of the grid. And modeling the selected key points as nodes of the graph, and constructing edges of the graph by using Dirony triangulation. Then calculating their descriptors as the attribute of the node, and fusing the feature matching with the weight parameter of the graph matching. And finally, searching a plurality of nodes with the maximum weight by using a graph matching algorithm of random walk of PageRank to finish matching.
The embodiment of the invention uses the structural characteristics among the key points to assist the matching, and can provide more accurate identification and tracking performance under the condition of less characteristic point matching pairs.
Fig. 2 is a schematic diagram of detecting a FAST corner according to an embodiment of the present invention; fig. 3 is a schematic diagram of an operator for extracting features according to an embodiment of the present invention. In the traditional method, SIFT (Scale-inverse Feature Transform) algorithm is mostly used for extracting features, and key points (Feature points) are searched in different Scale spaces, wherein the points are prominent and cannot be changed due to factors such as illumination, affine transformation and noise. The defects are that the speed of searching key points is slow, and the characteristic point descriptor is characterized by a 128-dimensional vector, so that the time is more consumed in characteristic matching. The ORB algorithm combines a detection method of FAST feature points with BRIEF feature descriptors, improves and optimizes the FAST feature points on the original basis, has invariance to noise and perspective transformation thereof, and extracts and describes features of images with different scales, thereby trying to solve the scale problem to a certain extent. The speed of the ORB algorithm is 100 times of that of SIFT, so that the calculation instantaneity can be ensured, the method is more convenient to apply to engineering, and SIFT can be well replaced. Referring to fig. 2, in another preferred embodiment, first, an ORB algorithm (organized FAST and Rotated BRIEF) is used to extract FAST corner points for each picture, and BRIEF (binary Robust independant Elementary features) operators are calculated for feature point matching pairs.
In one embodiment, reference may be made to the following specific steps:
(1) extracting a FAST corner point for each frame image,
Figure BDA0002428988120000061
in the above formula, t is a threshold (default value is 10, and values of different scenes are different), IpRepresenting the pixel value of the central pixel, Ip→xPixel values in a circular template are shown. Referring to FIG. 2, the pixel value I of the center pixelpSmaller than the pixel value I at x in the surrounding pixel circular templatep→xThen the pixel belongs to dark, Sp→xTwo other cases represent bright and similar, respectively. Such a block (circular) area can be divided into three types d, s and b. At this time, only the number of d or b in the circular area is countedIf d or b occurs more than n (n is generally set to 12), which means that the point is darker or lighter than the surroundings, the point is considered as a candidate corner point. Assuming that X characteristic points are to be extracted from the image, reducing the threshold value t, enabling the characteristic points detected by the FAST algorithm to be larger than X, then calculating the Harris response value R of the characteristic points at the position of the characteristic points, and taking the previous X points with large response values as FAST characteristic points;
(2) referring to fig. 3, for each FAST feature point, taking the FAST feature point as a center, taking a large window of the neighborhood of S × S, randomly selecting point pairs (generally 256 pairs) in the large window, performing binary assignment, and calculating a BRIEF operator.
Fig. 4 is a diagram of a GMS matching model according to an embodiment of the present invention. Referring to fig. 4, in another preferred embodiment, the GMS feature matching algorithm is used to filter correct feature point matching pairs, which may be as follows:
(1) matching the characteristic points by a BF (Brute force) violence matching algorithm;
firstly, selecting a feature point from a query image, then sequentially carrying out BRIEF descriptor Hamming distance test on the feature points in the matching training image, and finally returning the feature point with the closest distance to form a feature point matching set from the query image to the matching training image.
(2) Referring to fig. 4, the image is divided into G meshes (generally, G is 20 × 20), and BF-matched pairs of feature points F are calculatedpAnd FqNumber of correct matches S in the vicinitypq，
Figure BDA0002428988120000071
Wherein
Figure BDA0002428988120000076
Is a grid { pk,qkA match between.
Threshold for true and false matching
Figure BDA0002428988120000072
η is taken as a large empirical value of 6, npOf 3 × 3 meshesTotal number of features. Comparing the correct matching number SpqAnd a threshold value tpTo determine if the point is correctly matched:
Figure BDA0002428988120000073
the effect of the GMS feature matching algorithm on screening the correct matching pairs is shown in figure 5, the connecting lines are the correct matching pairs after deletion, and the matching conditions are almost correct, and few wrong matching connecting lines exist.
FIG. 6 is a schematic view of a given planar object of interest according to an embodiment of the present invention; FIG. 7 is a schematic diagram of meshing according to an embodiment of the present invention; FIG. 8 is a schematic diagram of solving for perspective transformation according to an embodiment of the present invention.
In another preferred embodiment, the above-described method of equally dividing a given planar object of interest R region into N grids, often 6 × 10 based on an empirical value of N, and selecting representative keypoints for each grid, may be performed according to the following steps:
(1) referring to fig. 6 and 7, an interested region R is marked out in a query image, the region R is equally divided into N grids, whether a correct matching pair retained by screening exists in each grid is queried, and if so, a point with the closest hamming distance of a BRIEF descriptor of the matching pair is taken as a key point of the grid. And if no correct feature point matching pair exists in the grid, taking the maximum harris response corner point in the FAST feature points in the grid as the key point of the grid.
(2) Selecting 4 pairs of correct matching pairs (preferably selecting grid key points) under screening reservation in the query image, and solving perspective transformation tau generated by the two images as shown in fig. 8:
Figure BDA0002428988120000074
let Z be a33＝1
Figure BDA0002428988120000075
8 equations can be obtained from 4 points, and 8 unknowns are solved, so that the perspective transformation matrix can be solved. Through the matrix, the region R of interest in the query image corresponding to the region in the matching training image can be obtained, and the region is divided into grids in the same way to select representative key points.
Fig. 9 is a schematic diagram of a dironi triangulation according to an embodiment of the present invention. Referring to fig. 9, in another preferred embodiment, the keypoint feature descriptors and structural feature data determined in the mesh are converted into a data structure of a graph, where each graph G ═ (V, E) is represented as a set of nodes V and edges E, and the keypoints are modeled as nodes of the graph to construct a graph model, which may specifically adopt: the selected keypoints are modeled as a node of the graph, and then their descriptors are computed as the attributes of this node. And constructing a triangular net by using Dirony triangulation and coordinates of key points to form edges of the graph, wherein the edges have translation, scaling and rotation invariance.
FIG. 10 is a schematic diagram of a fusion of two images according to an embodiment of the present invention. Referring to fig. 10, in another preferred embodiment, the feature matching is fused with the weight parameter of the graph matching, which can be implemented as follows:
(1) referring to FIG. 10, the graph matching problem is transformed, the result of which is represented by an assignment matrix (assignment matrix) X, since both graph grids are N, the assignment matrix is a {0,1} matrix of N × N, where each row, column and only one element of the assignment matrix is 1.
Figure BDA0002428988120000081
x*＝argmax(x)＝xTKx
s.t.X1n≤1m,X T1m≤1n
Wherein c isi,aRepresentative graph GαMiddle node i to graph GβConsistency of middle node a, graph GαFIG. GβRepresenting the graph model generated from the query image and the matching training image, respectively, di,j,a,bRepresentative graph GαMiddle line ij to graph GβThe consistency of the middle line segment ab, X represents the matching result for the corresponding matrix; i. j represents graph GαIn (1), a and b represent graph GβA node in (1); drawing GαFIG. GβRespectively by point set VαEdge set EαAnd set of points VβEdge set EβComposition Xi,aShows diagram GαNode i in (1) and graph GβOf node b, i.e. and only if node i ∈ VαCorresponding to node a ∈ VβWhen Xi,a＝1，Xj,bThe same process is carried out; (X) is an evaluation function, which represents the value under the corresponding assignment matrix X, graph GαAnd graph GβThe higher the evaluation quality is, the more similar the two images are;
Figure BDA0002428988120000082
vectorizing the matrix X; x is the number ofTIs the transpose of vector x; x is the number of*Vectorizing the obtained optimal corresponding matrix to show the optimal matching relation of the two graphs; k is a similarity matrix and simultaneously comprises first-order node similarity information and second-order edge similarity information; 1nRepresenting column vectors of N vectors, the constraint ensuring that each part matches at most once, since both graphs consist of N nodes, where N-m-N;
(2) and (3) assigning the similarity matrix K:
Figure BDA0002428988120000091
wherein A isαAnd AβIs GαAnd GβOf the adjacent matrix. Merging the feature points with the geometric constraints, order
Figure BDA0002428988120000092
Figure BDA0002428988120000093
Figure BDA0002428988120000094
And
Figure BDA0002428988120000095
being a BRIEF descriptor of the key point,
Figure BDA0002428988120000097
is shown as a drawing GαMidpoint viAnd vjThe coordinate position of (a);
Figure BDA0002428988120000098
、
Figure BDA0002428988120000096
is shown as a drawing GβMidpoint vaAnd vbThe coordinate position of (a); tau is the perspective transformation of the query image and the training matching image; di,j,a,b(τ) denotes the correspondence of the edge (i, j) with the edge (a, b) under the perspective transformation τ, ω being chosen large enough to ensure that the edge similarity is greater than 0. The diagonal elements of the similarity matrix K include node-to-node similarity information, and the off-diagonal elements include edge-to-edge similarity information.
(3) Filtering candidate matches, preferentially selecting GMS to screen reserved key points, sorting according to matching distances, and reserving at most ncSet of key points CtAnd the similarity is highest, and deletion does not belong to CtThe rows and columns of the key points are used to condense the similarity matrix K to a size reduced to nc 2N2. Taking 5 as ncThe ideal value of (c). Of course, in other embodiments, ncOther values may also be used.
FIG. 11 is a diagram of the PageRank algorithm according to an embodiment of the invention. In another preferred embodiment, as described with reference to FIG. 11. The existing graph matching methods are mainly divided into exact graph matching and approximate graph matching methods. The exact graph matching method is more complex and time consuming than the approximate graph matching method. Also, in an actual graph, the exact graph matching method is not robust against external points and various variations. Compared with exact graph matching, approximate graph matching has low complexity, high efficiency and wider application. Therefore, although the present problem is close to the exact graph matching of N to N nodes, it is more appropriate to convert it into approximate graph matching in view of efficiency and robustness. And expanding the graph matching problem by adopting a PageRank algorithm, and searching a plurality of nodes with the maximum weight to complete matching. The principle is that the whole graph reaches a stable state through random walk of a random walker, each node on the association graph corresponds to a similar probability value, namely the probability that a candidate matching pair is reliable matching, and the larger the similar probability value of the node is, the more correct the matching pair corresponding to the node is. Specifically, the following specific steps may be adopted:
(1) for the assigned similarity matrix K, the access probability of each point is initialized, so that the point starts to jump and randomly walks to the matching constraint and finally converges to the quasi-stationary distribution thereof ∈ C for each candidate match (i, a)tThe corresponding probability is initialized to:
Figure BDA0002428988120000101
(2) in order to solve the problem of termination points and traps, the transition probability is set to be η, the random jump probability is (1- η), the iterative formula of random walk is changed into X' ═ η KX + (1- η) e, e is an equally-divided random jump probability matrix, iteration is carried out until convergence, and a plurality of nodes with the maximum weight are selected to finish matching.
The above preferred features of the embodiments can be used alone in any embodiment, or in any combination thereof without conflict. In addition, portions which are not described in detail in the embodiments may be implemented by using the prior art.
Based on the foregoing embodiments, in another embodiment, the present invention further provides a device for fast and robust image recognition and tracking based on structural features, which includes a memory, a processor, and a computer program stored in the memory and executable on the processor, where the processor is operable to execute the method for fast and robust image recognition and tracking based on structural features according to any one of the foregoing embodiments when executing the program.
Optionally, a memory for storing a program; a Memory, which may include a volatile Memory (abbreviated RAM), such as a Random-Access Memory (RAM), a static Random-Access Memory (SRAM), a Double Data Rate Synchronous Dynamic Random Access Memory (DDR SDRAM), and the like; the memory may also comprise a non-volatile memory, such as a flash memory. The memory 62 is used to store computer programs (e.g., applications, functional modules, etc. that implement the above-described methods), computer instructions, etc., which may be stored in one or more memories in a partitioned manner. And the computer programs, computer instructions, data, etc. described above may be invoked by a processor.
The computer programs, computer instructions, etc. described above may be stored in one or more memories in a partitioned manner. And the computer programs, computer instructions, data, etc. described above may be invoked by a processor.
A processor for executing the computer program stored in the memory to implement the steps of the method according to the above embodiments. Reference may be made in particular to the description relating to the preceding method embodiment.
The processor and the memory may be separate structures or may be an integrated structure integrated together. When the processor and the memory are separate structures, the memory, the processor may be coupled by a bus.
The embodiment of the invention provides a Fast and robust image recognition and tracking method and device by starting with a GMS (Grid-based motion statistics for Fast) Fast search technology and adding a graph matching algorithm based on geometric constraint, can effectively accelerate the matching and recognition method, and provides more accurate recognition and tracking performance under the condition of less feature point matching pairs.
The foregoing description of specific embodiments of the present invention has been presented. It is to be understood that the present invention is not limited to the specific embodiments described above, and that various changes or modifications may be made by one skilled in the art within the scope of the appended claims without departing from the spirit of the invention.

Claims (10)

1. A fast robust image identification tracking method based on structural features is characterized by comprising the following steps:
adopting GMS feature matching algorithm to carry out feature point matching pair screening on the query image and the training matching image;
if a correct feature point matching pair exists, equally dividing interest areas in the query image and the training matching image into small grids, and determining key points for each grid;
modeling key points as graph nodes, generating edges, constructing a graph model, and fusing feature matching and graph matching weight parameters;
and (4) completing approximate matching of the image by using a random walk algorithm, and completing image identification and tracking.
2. The method for fast and robustly recognizing and tracking the image based on the structural feature of claim 1, wherein the step of performing feature point matching pair screening on the query image and the training matching image by using a GMS feature matching algorithm comprises the steps of:
matching the feature points by a BF violence matching algorithm;
dividing the query image and the training matching image into G grids respectively, and calculating the characteristic point pairs F matched with BFpAnd FqNumber of correct matches S in the vicinitypq，FpAnd FqRespectively, the characteristic points in the query image and the training matching image are screened as matching pairs by a BF violence matching algorithm;
Figure FDA0002428988110000011
by characteristic point FpAnd FqThe located grid is taken as the center, and 9 grids with K being 3 × 3 are respectively selected as the area to calculate the matching pair numberWherein
Figure FDA0002428988110000012
Is a grid { pk,qkThe number of matching pairs between every two adjacent pairs is 1-9;
threshold for true and false matching
Figure FDA0002428988110000013
η taking an empirical value of 6, npIs the total number of features of the grid; comparing the correct matching number SpqAnd a threshold value tpTo determine if the point is correctly matched:
Figure FDA0002428988110000014
wherein p and q are respectively expressed as characteristic points F in the query imagepMatching feature points F in images with trainingq。
3. The method for fast and robust image recognition and tracking based on structural features as claimed in claim 1, wherein the matching of feature points by BF brute force matching algorithm comprises:
firstly, selecting a characteristic point in a query image;
then carrying out BRIEF descriptor Hamming distance test on the feature points in the matched training images in sequence;
and finally, returning the feature points with the closest distance to form a feature point matching set from the query image to the matching training image.
4. The method for fast and robust image recognition and tracking based on structural features of claim 3, wherein the BRIEF descriptor is Hamming distance test, wherein the BRIEF descriptor is obtained by:
extracting FAST characteristic points from each picture by using an ORB algorithm;
and taking each FAST characteristic point as a center, taking a neighborhood large window of S × S, randomly selecting point pairs in the large window, carrying out binary assignment, and calculating a BRIEF descriptor.
5. The method for fast and robust image recognition and tracking based on structural features as claimed in claim 1, wherein the step of equally dividing the region of interest in the query image and the training matching image into small grids and determining the key points for each grid comprises:
(1) drawing an interested region R in a query image, equally dividing the R region into N grids, querying whether a correct matching pair reserved by screening exists in each grid, and if so, taking a point with the closest Hamming distance of a BRIEF descriptor of the matching pair as a key point of the grid; if no correct feature point matching pair exists in the grid, taking the maximum harris response corner point in the FAST feature points in the grid as the key point of the grid;
(2) selecting 4 pairs of correct matching pairs reserved by screening from the query image, and solving the perspective transformation tau generated between the query image and the training matching image:
Figure FDA0002428988110000021
order to
Figure FDA0002428988110000022
Obtaining 8 equations from 4 points, and solving 8 unknowns to obtain a perspective transformation matrix; and obtaining the region of interest R in the query image corresponding to the region in the matching training image through the matrix, and dividing the grid of the region in the same way to select representative key points.
6. The method for fast and robust image recognition and tracking based on structural features as claimed in claim 1, wherein the key points are modeled as nodes of a graph, edges of the graph are generated, and the constructing of the graph model comprises:
modeling the selected key points as the nodes of the graph, then calculating descriptors of the key points as the attributes of the nodes, constructing a triangular net through the coordinates of the key points by using Dirony triangulation, forming the edges of the graph, and enabling the edges to be flatShift, zoom, and rotation invariance; so far, the interest area in the query image and the feature point information and the structure information corresponding to the area in the matching training image are converted into a graph model Gα＝(Vα,Eα) And Gβ＝(Vβ,Eβ) Where G represents a graph model, α represents a query image, and β represents a matching training image.
7. The method for fast and robust image recognition and tracking based on structural features as claimed in claim 6, wherein the fusing the feature matching with the weight parameters of the graph matching comprises:
(1) the grids of the query image and the training matching image are both N, and the assignment matrix is a {0,1} matrix of N × N, wherein each row and each column of the assignment matrix have and only one element of 1;
in the graph matching problem, the first-order similarity between nodes and the second-order similarity between edges and the first-order similarity between the nodes in the graph structure are considered at the same time, and the similarity matrix K simultaneously contains first-order node similarity information and second-order edge similarity information; the matching problem is converted into the following mathematical form:
Figure FDA0002428988110000031
x*＝argmax(x)＝xTKx
s.t.X1n≤1m,XT1m≤1n
wherein c isi,aRepresentative graph GαMiddle node i to graph GβConsistency of middle node a, graph GαFIG. GβRepresenting the graph model generated from the query image and the matching training image, respectively, di,j,a,bRepresentative graph GαMiddle line ij to graph GβThe consistency of the middle line segment ab, X represents the matching result for the corresponding matrix; i. j represents graph GαIn (1), a and b represent graph GβA node in (1); drawing GαFIG. GβRespectively by point set VαEdge set EαAnd set of points VβEdge set EβComposition Xi,aShows diagram GαNode i in (1) and graph GβOf node b, i.e. and only if node i ∈ VαCorresponding to node a ∈ VβWhen Xi,a＝1，Xj,bThe same process is carried out; (X) is an evaluation function, which represents the value under the corresponding assignment matrix X, graph GαAnd graph GβThe higher the evaluation quality is, the more similar the two images are;
Figure FDA0002428988110000032
vectorizing the matrix X; x is the number ofTIs the transpose of vector x; x is the number of*Vectorizing the obtained optimal corresponding matrix to show the optimal matching relation of the two graphs; k is a similarity matrix and simultaneously comprises first-order node similarity information and second-order edge similarity information; 1nRepresenting column vectors of N vectors, the constraint ensuring that each part matches at most once, since both graphs consist of N nodes, where N-m-N;
(2) and (3) assigning the similarity matrix K:
Figure FDA0002428988110000033
wherein A isαAnd AβIs GαAnd GβThe adjacency matrix of (a);
Figure FDA0002428988110000041
shows diagram GαMidpoint viAnd point vjWhen v is an adjacent relation ofiAnd vjWhen there is a connected edge
Figure FDA0002428988110000042
Figure FDA0002428988110000043
The same process is carried out; c. Ci,aIs shown as a drawing GαMidpoint viAnd graph GβMidpoint vaUniformity of；di,j,a,b(τ) is graph GαMiddle edge (i, j) and graph GβThe consistency of the middle edges (a, b);
merging the feature points with the geometric constraints, order
Figure FDA0002428988110000044
Figure FDA0002428988110000045
fi、
Figure FDA0002428988110000046
A BRIEF descriptor that is a key point;
Figure FDA0002428988110000047
is shown as a drawing GαMidpoint viAnd vjThe coordinate position of (a);
Figure FDA0002428988110000048
is shown as a drawing GβMidpoint vaAnd vbThe coordinate position of (a); tau is the perspective transformation of the query image and the training matching image;
di,j,a,b(τ) represents the consistency of the lower edge (i, j) and the edge (a, b) of the perspective transformation τ, ω is selected to be large enough to ensure that the similarity of the pair of edges is greater than 0, the diagonal elements of the similarity matrix K contain the similarity information of the nodes and the nodes, and the non-diagonal elements contain the similarity information of the edges and the edges;
(3) filtering candidate matches, selecting GMS to screen retained key points, sorting according to matching distance, and retaining at most ncSet of key points CtAnd the similarity is highest, and deletion does not belong to CtThe rows and columns of the key points are used to condense the similarity matrix K to a size reduced to nc 2N2。
8. The method for fast and robust image recognition and tracking based on structural features of claim 7, wherein the approximate matching of the graph is completed by a random walk algorithm, which is characterized in that: and searching a plurality of nodes with the maximum weight by using a PageRank graph matching algorithm to complete matching.
9. The method for fast and robust image recognition and tracking based on structural features according to claim 8, wherein the finding of the nodes with the maximum weight by using the PageRank graph matching algorithm comprises:
(1) initializing the access probability of each point for the assigned similarity matrix K, making it start to jump and randomly walk to the matching constraint and finally converge on the quasi-stationary distribution, and (i, a) ∈ C for each candidate matchtThe corresponding probability is initialized to:
Figure FDA0002428988110000049
(2) in order to solve the problem of termination points and traps, the transition probability is set to be η, the random jump probability is set to be (1- η), the iterative formula of random walk is changed into X' ═ η KX + (1- η) e, e is an equally-divided random jump probability matrix, iteration is carried out until convergence, and a plurality of nodes with the maximum weight are selected to finish matching.
10. A fast robust image recognition tracking apparatus based on structural features, comprising a memory, a processor and a computer program stored on the memory and executable on the processor, wherein the processor when executing the program is operable to perform the method of any one of claims 1 to 9.
CN202010229998.8A 2020-03-27 2020-03-27 Rapid robust image identification tracking method and device based on structural features Active CN111461196B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202010229998.8A CN111461196B (en) 2020-03-27 2020-03-27 Rapid robust image identification tracking method and device based on structural features

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202010229998.8A CN111461196B (en) 2020-03-27 2020-03-27 Rapid robust image identification tracking method and device based on structural features

Publications (2)

Publication Number Publication Date
CN111461196A true CN111461196A (en) 2020-07-28
CN111461196B CN111461196B (en) 2023-07-21

Family

ID=71683310

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202010229998.8A Active CN111461196B (en) 2020-03-27 2020-03-27 Rapid robust image identification tracking method and device based on structural features

Country Status (1)

Country Link
CN (1) CN111461196B (en)

Cited By (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN114048240A (en) * 2021-11-18 2022-02-15 长春理工大学 Data integration method and system based on approximate graph matching algorithm
CN115601576A (en) * 2022-12-12 2023-01-13 云南览易网络科技有限责任公司（Cn） Image feature matching method, device, equipment and storage medium
CN116778141A (en) * 2023-08-28 2023-09-19 深圳联友科技有限公司 ORB algorithm-based method for rapidly identifying and positioning picture

Citations (10)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN102708370A (en) * 2012-05-17 2012-10-03 北京交通大学 Method and device for extracting multi-view angle image foreground target
CN105513094A (en) * 2015-12-17 2016-04-20 上海交通大学 Stereo vision tracking method and stereo vision tracking system based on 3D Delaunay triangulation
CN106682700A (en) * 2017-01-05 2017-05-17 北京细推科技有限公司 Block quick matching algorithm based on key point description operator
CN106780579A (en) * 2017-01-17 2017-05-31 华中科技大学 A kind of ultra-large image characteristic point matching method and system
CN108537287A (en) * 2018-04-18 2018-09-14 北京航空航天大学 Image closed loop detection method and device based on graph model
CN109598750A (en) * 2018-12-07 2019-04-09 中国地质大学（武汉） One kind being based on the pyramidal large scale differential image characteristic point matching method of deformation space
CN109697692A (en) * 2018-12-29 2019-04-30 安徽大学 One kind being based on the similar feature matching method of partial structurtes
CN109784353A (en) * 2017-11-14 2019-05-21 北京三星通信技术研究有限公司 A kind of matched method of non-directed graph, equipment and storage medium
CN109816706A (en) * 2019-02-01 2019-05-28 辽宁工程技术大学 A kind of smoothness constraint and triangulation network equal proportion subdivision picture are to dense matching method
CN110032983A (en) * 2019-04-22 2019-07-19 扬州哈工科创机器人研究院有限公司 A kind of track recognizing method based on ORB feature extraction and FLANN Rapid matching

Patent Citations (10)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN102708370A (en) * 2012-05-17 2012-10-03 北京交通大学 Method and device for extracting multi-view angle image foreground target
CN105513094A (en) * 2015-12-17 2016-04-20 上海交通大学 Stereo vision tracking method and stereo vision tracking system based on 3D Delaunay triangulation
CN106682700A (en) * 2017-01-05 2017-05-17 北京细推科技有限公司 Block quick matching algorithm based on key point description operator
CN106780579A (en) * 2017-01-17 2017-05-31 华中科技大学 A kind of ultra-large image characteristic point matching method and system
CN109784353A (en) * 2017-11-14 2019-05-21 北京三星通信技术研究有限公司 A kind of matched method of non-directed graph, equipment and storage medium
CN108537287A (en) * 2018-04-18 2018-09-14 北京航空航天大学 Image closed loop detection method and device based on graph model
CN109598750A (en) * 2018-12-07 2019-04-09 中国地质大学（武汉） One kind being based on the pyramidal large scale differential image characteristic point matching method of deformation space
CN109697692A (en) * 2018-12-29 2019-04-30 安徽大学 One kind being based on the similar feature matching method of partial structurtes
CN109816706A (en) * 2019-02-01 2019-05-28 辽宁工程技术大学 A kind of smoothness constraint and triangulation network equal proportion subdivision picture are to dense matching method
CN110032983A (en) * 2019-04-22 2019-07-19 扬州哈工科创机器人研究院有限公司 A kind of track recognizing method based on ORB feature extraction and FLANN Rapid matching

Non-Patent Citations (1)

* Cited by examiner, † Cited by third party
Title
JIAWANG BIAN等: "GMS: Grid-based Motion Statistics for Fast, Ultra-robust Feature Correspondence", 《CVPR》, 9 November 2017 (2017-11-09) *

Cited By (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN114048240A (en) * 2021-11-18 2022-02-15 长春理工大学 Data integration method and system based on approximate graph matching algorithm
CN115601576A (en) * 2022-12-12 2023-01-13 云南览易网络科技有限责任公司（Cn） Image feature matching method, device, equipment and storage medium
CN116778141A (en) * 2023-08-28 2023-09-19 深圳联友科技有限公司 ORB algorithm-based method for rapidly identifying and positioning picture
CN116778141B (en) * 2023-08-28 2023-12-22 深圳联友科技有限公司 ORB algorithm-based method for rapidly identifying and positioning picture

Also Published As

Publication number Publication date
CN111461196B (en) 2023-07-21

Similar Documents

Publication Publication Date Title
CN111795704B (en) Method and device for constructing visual point cloud map
US11341738B2 (en) Using a probabtilistic model for detecting an object in visual data
Zhao et al. Deep hough transform for semantic line detection
CN111780764B (en) Visual positioning method and device based on visual map
CN109815843B (en) Image processing method and related product
CN111461196B (en) Rapid robust image identification tracking method and device based on structural features
CN107133913B (en) Automatic-straightening image splicing method
CN108986152B (en) Foreign matter detection method and device based on difference image
CN112967341B (en) Indoor visual positioning method, system, equipment and storage medium based on live-action image
CN111179419A (en) Three-dimensional key point prediction and deep learning model training method, device and equipment
WO2020020047A1 (en) Key point matching method and device, terminal device and storage medium
WO2019041660A1 (en) Face deblurring method and device
CN116721301B (en) Training method, classifying method, device and storage medium for target scene classifying model
CN111783753A (en) Pedestrian re-identification method based on semantic consistency horizontal bar and foreground correction
CN111709317B (en) Pedestrian re-identification method based on multi-scale features under saliency model
CN111125397A (en) Cloth image retrieval method based on convolutional neural network
Bhattacharjee et al. Query adaptive multiview object instance search and localization using sketches
CN113642571A (en) Fine-grained image identification method based on saliency attention mechanism
CN111414958A (en) Multi-feature image classification method and system for visual word bag pyramid
Kristollari et al. Change detection in VHR imagery with severe co-registration errors using deep learning: A comparative study
CN108492256B (en) Unmanned aerial vehicle video fast splicing method
CN108304838B (en) Picture information identification method and terminal
JP6778625B2 (en) Image search system, image search method and image search program
CN112632315B (en) Method and device for retrieving remote sensing image
Luanyuan et al. MGNet: Learning Correspondences via Multiple Graphs

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
GR01 Patent grant
GR01 Patent grant