CN109766943B - Template matching method and system based on global perception diversity measurement - Google Patents
Template matching method and system based on global perception diversity measurement Download PDFInfo
- Publication number
- CN109766943B CN109766943B CN201910022545.5A CN201910022545A CN109766943B CN 109766943 B CN109766943 B CN 109766943B CN 201910022545 A CN201910022545 A CN 201910022545A CN 109766943 B CN109766943 B CN 109766943B
- Authority
- CN
- China
- Prior art keywords
- point set
- global
- diversity
- nearest neighbor
- dimensional point
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Images
Landscapes
- Image Analysis (AREA)
Abstract
The invention discloses a template matching method and a template matching system based on global perception diversity measurement. The template matching method comprises the following steps: acquiring an input image and a template image; performing feature extraction on the input image and the template image to determine a first high-dimensional point set and a second high-dimensional point set; determining a nearest neighbor field by utilizing nearest neighbor search according to the first high-dimensional point set and the second high-dimensional point set; determining global information and local diversity information of the nearest neighbor field by using a global perception diversity measurement algorithm and a local diversity algorithm; guiding the local diversity information to determine a heat map by using the global information; and determining a matching image according to the heat map. By adopting the template matching method and the template matching system provided by the invention, the complexity of the template matching method can be reduced, and the calculation efficiency can be improved.
Description
Technical Field
The invention relates to the field of template matching, in particular to a template matching method and a template matching system based on global perception diversity measurement.
Background
The current mainstream template matching algorithms are divided into two major categories according to the space of template matching input, namely a traditional template matching method based on a feature space and a template matching method based on a nearest neighbor field space.
In the template matching, different feature spaces, such as a color space, a geometric feature space, a depth feature space based on pre-training, and the like, may be used according to different application scenarios and requirements. Conventional template matching algorithms include Sum of Squared error (SSD), Sum of Absolute error (SAD), Normalized Correlation Coefficient (NCC), and the like. The traditional methods have simple measurement functions and high calculation efficiency, but have low accuracy. Further work has been done to construct similarity functions using more robust error functions, such as M-estimators or Hamming-based distance. Although these algorithms are generally less susceptible to additive and salt-and-pepper noise than the normalized cross-correlation method, they still assume that there is a rigid transformation (translation transformation) between the target image and the template image and penalize the template and all the pixels in the exact corresponding positions of the sliding self-window. The above algorithm basically fails once the object in the template has undergone rotation or even non-rigid transformation. Some work has been subsequently proposed to address the problem of rotational transform invariance. More recently, Korman et al have proposed a template matching algorithm that can remain invariant in a two-dimensional affine transformation, which ensures a result that is close to a global optimal solution. Similarly, Tian et al explores an estimation solution that can be used to find a global optimum for image deformation (non-rigid transformations). Nevertheless, these methods still try to find a one-to-one mapping among the implicit transformations. Therefore, they often show unsatisfactory performance and are prone to errors when a large number of outliers occur, such as occlusion and a complex background.
Based on the limitations of the conventional template matching algorithm and the shortcomings of its Similarity measure, Oron proposed a novel Best-beat Similarity measure algorithm (BBS) based on Nearest Neighbor Fields (NNF) in 2015. The BBS has a certain creativity, and is relatively robust to external points (noise, background variation, etc.) and various rotations and deformations for two reasons: on one hand, the feature space is mapped to the NNF space, and the similarity measure used in the algorithm only depends on the key point set matched by the nearest neighbor; on the other hand, it can distinguish the inner point and the outer point well without any prior knowledge or hypothesis. However, the BBS algorithm has the disadvantages that the algorithm complexity is very high, bidirectional brute force nearest neighbor matching is required for each sub-window and template, and a target image with high resolution (1280x720) generally requires tens of minutes. In 2017, Talmi et al inspired by the texture synthesis field propose Deformable Diversity of Similarity (DDIS), the algorithm is comprehensively superior to the previous BBS in both qualitative and quantitative results, and theoretical derivation about Diversity is given in the text, which proves that Diversity Similarity (DIS) based on one-way nearest neighbor matching can approximate BBS of two-way nearest neighbor matching. Furthermore, the DIS provides the DDIS according to the priori knowledge of the data, and explicitly adds the idea of deformation loss, so that the accuracy is greatly improved. From the aspect of computational efficiency, the method only uses one-way matching in similarity function calculation to further reduce the operation amount.
The traditional template matching algorithm has a plurality of defects, generally only has rotation or affine invariance, and still has limitation in the face of complex scenes and transformation. In addition, since an algorithm with specific invariance usually has very strong assumptions on data and builds a very complex model according to the assumptions, data of a real scene often cannot meet the assumptions, and the complex model has high calculation cost; although the nearest neighbor field based approach achieves superior performance over conventional algorithms, there are still some problems and challenges that are not solved. On the one hand, the nearest neighbor field based BBS and DDIS methods are very sensitive to large rotations and large deformations due to the very strong assumption for data; on the other hand, the calculation complexity of the method based on the nearest neighbor BBS and the DDIS is still very high, and the requirement of the application of the real scene is far from being met at present. Therefore, the above two aspects result in that the current algorithm has low robustness and still high algorithm complexity, and cannot meet the requirements of practical scenes.
Disclosure of Invention
The invention aims to provide a template matching method and a template matching system based on global perception diversity measurement, and aims to solve the problems that the traditional template matching method is high in complexity and cannot meet the requirements of actual scenes.
In order to achieve the purpose, the invention provides the following scheme:
a template matching method based on global perceptual diversity measurement comprises the following steps:
acquiring an input image and a template image;
performing feature extraction on the input image and the template image to determine a first high-dimensional point set and a second high-dimensional point set; points in the second high-dimensional point set correspond to points in the first high-dimensional point set one by one;
determining a nearest neighbor field by utilizing nearest neighbor search according to the first high-dimensional point set and the second high-dimensional point set;
determining global information and local diversity information of the nearest neighbor field by using a global perception diversity measurement algorithm and a local diversity algorithm;
guiding the local diversity information to determine a heat map by using the global information;
and determining a matching image according to the heat map.
Optionally, the performing feature extraction on the input image and the template image to determine a first high-dimensional point set and a second high-dimensional point set specifically includes:
converting the input image and the template image into two high-dimensional images based on image blocks;
and arranging the image blocks in the high-dimensional image into one-dimensional vectors according to a set image block area threshold, and determining a first high-dimensional point set and a second high-dimensional point set.
Optionally, the determining a nearest neighbor field by using nearest neighbor search according to the first high-dimensional point set and the second high-dimensional point set specifically includes:
for each point in the second high-dimensional point set, determining a nearest neighbor index of the point in the first high-dimensional point set by nearest neighbor search, and determining a nearest neighbor field.
Optionally, the determining global information and local diversity information of the nearest neighbor field by using a global perceptual diversity metric algorithm and a local diversity algorithm specifically includes:
according to the formula
Determining a similarity measure function; wherein NN (q)iP) is the point Q in the second high-dimensional point set QiNearest neighbors in said first set of high-dimensional points P; k (p)j) Counting the nearest neighbor indexes; k (p)j)＝|{qi∈Q:NN(qi,P)＝pj}|；QωA target point set in the range of the current sliding window omega is obtained;
decomposing the similarity measurement function into a global score and a local score corresponding to each sliding window;
determining global information of the nearest neighbor field by using a global perception diversity measurement algorithm according to the global score;
and determining the local diversity information of the nearest neighbor field by utilizing the local diversity algorithm according to the local score.
A template matching system based on a global perceptual diversity metric, comprising:
the image acquisition module is used for acquiring an input image and a template image;
the high-dimensional point set determining module is used for extracting the characteristics of the input image and the template image and determining a first high-dimensional point set and a second high-dimensional point set; points in the second high-dimensional point set correspond to points in the first high-dimensional point set one by one;
a nearest neighbor field determining module, configured to determine a nearest neighbor field by using nearest neighbor search according to the first high-dimensional point set and the second high-dimensional point set;
the global information and local diversity information determining module is used for determining global information and local diversity information of the nearest neighbor field by utilizing a global perception diversity measurement algorithm and a local diversity algorithm;
the heat map determining module is used for guiding the local diversity information to determine a heat map by utilizing the global information;
and the matching image determining module is used for determining a matching image according to the heat map.
Optionally, the high-dimensional point set determining module specifically includes:
an image conversion unit for converting the input image and the template image into two image block-based high-dimensional images;
and the high-dimensional point set determining unit is used for arranging the image blocks in the high-dimensional image into one-dimensional vectors according to a set image block area threshold value and determining a first high-dimensional point set and a second high-dimensional point set.
Optionally, the nearest neighbor field determining module specifically includes:
a nearest neighbor field determining unit, configured to determine, for each point in the second high-dimensional point set, a nearest neighbor index of the point in the first high-dimensional point set through nearest neighbor search, and determine a nearest neighbor field.
Optionally, the global information and local diversity information determining module specifically includes:
a similarity metric function determination unit for determining a similarity metric according to a formula
Determining a similarity measure function; wherein NN (q)iP) is the point Q in the second high-dimensional point set QiNearest neighbors in said first set of high-dimensional points P; k (p)j) Counting the nearest neighbor indexes; k (p)j)＝|{qi∈Q:NN(qi,P)＝pj}|；QωA target point set in the range of the current sliding window omega is obtained;
the decomposition unit is used for decomposing the similarity measurement function into a global score and a local score corresponding to each sliding window;
a global information determining unit, configured to determine global information of the nearest neighbor field by using a global perceptual diversity metric algorithm according to the global score;
a local diversity information determination unit, configured to determine, according to the local score, local diversity information of the nearest neighbor field by using the local diversity algorithm.
According to the specific embodiment provided by the invention, the invention discloses the following technical effects: the invention provides a template matching method and a template matching system based on global perception diversity measurement, which disclose the importance of global semantic information, and a global perception diversity measurement algorithm effectively utilizes global context information to supervise local diversity information, can effectively filter irrelevant areas such as background or local outliers, and has stronger robustness; according to the GAD similarity measurement function, the GAD similarity measurement function is divided into two parts of mutually independent operation, namely global perception information calculation and local diversity information calculation, so that a large amount of repeated calculation is avoided. The method and the system provided by the invention can greatly reduce the calculation complexity, and the running time can be shortened by 3 to 4 orders of magnitude.
Drawings
In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings needed to be used in the embodiments will be briefly described below, and it is obvious that the drawings in the following description are only some embodiments of the present invention, and it is obvious for those skilled in the art to obtain other drawings without inventive exercise.
FIG. 1 is a flowchart of a template matching method based on global perceptual diversity measurement according to the present invention;
FIG. 2 is a schematic diagram of feature extraction provided by the present invention;
FIG. 3 is a transformation diagram for converting a high dimensional point cloud to NNFs as provided by the present invention;
FIG. 4 is a global profile provided by the present invention;
FIG. 5 is a flow chart of a global perceptual diversity metric algorithm provided by the present invention;
FIG. 6 is a flowchart of the actual operation of the template matching method based on global perceptual diversity measurement according to the present invention;
FIG. 7 is a block diagram of a template matching system based on global perceptual diversity metrics as provided by the present invention;
FIG. 8 is a comparison graph of qualitative results of three sub-evaluation sets of BBS for different algorithms provided by the present invention;
FIG. 9 is a comparison of qualitative results of the TinyTLP subset evaluation set by different algorithms provided by the present invention.
Detailed Description
The technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the drawings in the embodiments of the present invention, and it is obvious that the described embodiments are only a part of the embodiments of the present invention, and not all of the embodiments. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present invention.
The invention aims to provide a template matching method and a template matching system based on global perception diversity measurement, which can reduce the complexity of the template matching method and meet the requirements of actual scenes.
In order to make the aforementioned objects, features and advantages of the present invention comprehensible, embodiments accompanied with figures are described in further detail below.
Fig. 1 is a flowchart of a template matching method based on global perceptual diversity measurement according to the present invention, and as shown in fig. 1, a template matching method based on global perceptual diversity measurement includes:
step 101: an input image and a template image are acquired.
Step 102: performing feature extraction on the input image and the template image to determine a first high-dimensional point set and a second high-dimensional point set; points in the second high-dimensional point set correspond to points in the first high-dimensional point set one to one.
Two images are converted into a first high-dimensional point set P and a second high-dimensional point set Q based on image blocks, for each position of the images, the image blocks with a certain size are selected and arranged into one-dimensional vectors, and then a high-dimensional point set can be obtained. The size of the point set is related to the overlapping flag bit, namely whether the image blocks are overlapped or not overlapped; the dimension of the point set is related to the size of the image block, the larger the image block is, the higher the dimension of the point set is, as shown in fig. 2, where K is the size of the slice block, in the present invention, K takes 5 pixels, and an overlapped portion slice extraction method is used.
Step 103: and determining a nearest neighbor field by utilizing nearest neighbor search according to the first high-dimensional point set and the second high-dimensional point set.
As shown in fig. 3, for each point in Q, by the Nearest Neighbor technique, the Nearest Neighbor index of the point in P is obtained, and finally, the whole Nearest Neighbor Field (NNF) is obtained.
Step 104: and determining global information and local diversity information of the nearest neighbor field by utilizing a global perception diversity measurement algorithm and a local diversity algorithm.
The invention provides a Global Diversity (GAD) method, which can organically utilize Global context to guide local Diversity information. NNF is converted into global perception information and local perception information according to perception capability, a heat map is obtained through post-processing to obtain a final matching result, and a GAD specific formula is as follows:
k(pj)＝|{qi∈Q:NN(qi,P)＝pj}|
wherein NN (q)iP) is qiNearest neighbor in P, k (P)j) The nearest neighbor index is counted.
As can be seen from the definition of the GAD method, GAD can be understood as the re-weighting of the local diversity information, the cumulative sum of the first terms being the weighting factor, if QωCorresponds to the background or irrelevant area, then QωK (NN (q))iρ)) will generally be large, resulting in the sum of their inverses being generally small close to 0, and therefore the similarity score of ω should be suppressed; if Q isωMiddle pairShould the region of interest, then QωK in (1) (NN (q)iρ)) is typically within a few tens of meters, resulting in a summation of their inverses that is typically large, and therefore the similarity fraction of ω should be amplified. Through the mechanism, the GAD can better inhibit the background and highlight the region of interest to be matched.
In addition, the invention verifies the validity of the global semantic information. The method comprises the following steps: first go through all piCalculate nearest neighbor index count k (p)j) That is, a histogram of NNFs is calculated, and then k (p) with the nearest neighbor index value of top-20 is taken out from Qj) And setting the pixel values of all the positions to be 1 and setting the pixel values of other positions to be 0, thereby obtaining a Mask of global semantic information. The formula is as follows:
since the Mask has many holes, the Mask is subjected to morphological open operation processing under the condition of no distortion, and the specific operation is shown in the following formula:
in the formula
Representing morphological dilation operations
Representing a morphological etching operation.
As shown in fig. 4, the mark frame of the real target is a green boundary frame, the first line is t (template), the second line is i (input), the third line is a binarized Mask, and the fourth line is a visualized image obtained by fusing the Mask to the input. From the last line of visualization results, it can be seen that the Mask 1 area, i.e. the blue area, occupies almost most of the complex background or outlier, while most of the area where the object to be matched is located is preserved. Therefore we conclude that: the global context information utilized by Mask can be used to eliminate irrelevant areas such as complex background or local points to a great extent and to reserve the area of interest we need.
The GAD method provided by the invention can effectively combine context information and local diverse information, effectively filter irrelevant areas or local points and obtain more accurate results.
(1) An efficient global perceptual diversity metric algorithm:
the invention analyzes the importance of global context information in detail in the third section and in this way proposes a GAD similarity measure. However, in order to make the GAD similarity metric truly practical, the algorithm implementation of the metric must have low time complexity and very fast run time. The invention provides a set of efficient algorithm realization based on GAD similarity measurement, so that the time complexity of the algorithm is reduced from original O (| T |. I |) to O (| I |). As can be seen from (3) below, the similarity can be measured as a function
Is divided into two parts
And
fig. 5 is a flowchart of the global sensing algorithm provided by the present invention, as shown in fig. 5:
obtaining a global probability image theta according to formula (3):
the integral image Φ is calculated, defined and calculated as follows:
and traversing the integral image phi to calculate a global perception image:
SG[x,y]＝θ[x+h,y+w]-Φ[x,x+h]-Φ[x+w,y]-Φ[x,y](6)
(2) efficient local diversity algorithm:
when NNF performs sliding window calculation of local diversity, the former omegat-1And current ωtThe presence of a large number of caches, i.e. common elements, means that there may be a large number of duplicate calculations when calculating local diversity. In the process of sliding the window, the cache can be fully utilized for quick updating, and a large amount of repeated calculation is avoided. It should be noted that in the upper left-most corner
In the calculation of (2), the whole is required to be calculated
The traversal is performed to obtain the nearest neighbor index set omega, and the cached information cannot be utilized. All others
Can be quickly computed using a cache to delete a single row or column from the nearest neighbor index set omega and to add a single row or column. The design of the invention has the efficient steps as follows:
step 2, moving omega downwards by one step to the bottom, and removing NNF omega from the nearest neighbor index set omega in the process of moving downwardst-1Top row and added to NNF ωtThe lowest row;
and step 3: after the bottom is reached, moving omega to the right by one step length, and then, similarly doing the first step;
and 4, step 4: move ω up one step to the top, in the up-shift process remove NNF ω from nearest neighbor index set Ωt-1Bottom row and added to NNF ωtThe uppermost row;
and 5: and (4) continuously repeating the steps 1 to 4.
(3) Time complexity analysis:
based on the high-efficiency global perception algorithm and the high-efficiency local diversity algorithm provided by the invention, the complexity is analyzed as follows. The time of the algorithm is mainly divided into the time O spent on global perceptionGTime spent in local perception OL. And (3) performing four-time traversal on the NNF through the algorithm flow chart in the step (1), and assuming that the number contained in the NNF set is | I |, performing
OG＝2O(|I|)+O(|I|)+O(|I|)＝O(|I|)
Second for local perception algorithms, upper and left most
The NNF needs to be traversed once, the time complexity is O (| I |), then other omega are quickly updated by utilizing cache, the width of the template is assumed to be w, the number of the used threads is v, and the reason is that the width of the template is w
Is constant, the complexity of the whole local sensing algorithm is
Due to the fact that
And
the calculation of (A) is independent and does not influence each other, and then the calculation of the (A) and the (B) can be parallelized and put into different threads. Therefore, based on GAD similarityEfficient algorithmic realization of quantities corresponding to a final time complexity of OGADComprises the following steps:
OGAD＝max(OG,OL)＝O(|I|)
therefore, the template matching method provided by the invention can greatly reduce the complexity O (| I |. T |) to O (| T |), the running time can be shortened by 3 to 4 orders of magnitude, and the calculation efficiency is improved.
Step 105: and guiding the local diversity information to determine a heat map by using the global information.
Step 106: and determining a matching image according to the heat map.
In order to obtain a final result, the invention needs to further post-process the heat map to finally obtain a matching image:
(1) gaussian filtering: the resulting heat map is gaussian filtered using a filter sized to the template 1/3 in order to remove isolated outliers and achieve more robust performance.
(2) And projecting and dividing the binarized character area into single characters according to the vertical direction.
(3) Filling: since the generated heat map is smaller than the original target image in size, the boundary needs to be filled in.
(4) Finding the position of the maximum: and traversing the filtered heat map, finding out the maximum value of the heat map, and recording the position of the heat map in the two-dimensional image.
(5) Obtaining a boundary frame: and finding out a final boundary box, namely a detection result, according to the position of the maximum value and the size of the template, and determining a final matching image.
Given an input image and a template image, extracting slice-based features from the image yields two sets of high-dimensional points P and Q, each high-dimensional point being a local slice feature at a location on the original image, as shown in fig. 6. In order to ensure the local feature invariance, the slice size is not too large, and 3-7 pixels are generally taken.
By nearest neighbor search, Q for each point in QiBy means of a specified distance function (employed in the invention)Euclidean distance) finds the corresponding nearest neighbor in the whole P and records the nearest neighbor index to obtain the whole NNF.
After NNFs are obtained, a global score is calculated for each window w through an efficient global perceptibility algorithm
Obtaining global information, and calculating local scores for each window w through an efficient local perceptibility algorithm
Local diversity information is obtained.
The global information guides local diversity, namely two matrixes are multiplied to obtain an overall heat map (heatmap). The location of the maximum of the heat map is the center point of the bounding box of the final result.
Fig. 7 is a structural diagram of a template matching system based on global perceptual diversity measurement provided in the present invention, and as shown in fig. 7, a template matching system based on global perceptual diversity measurement includes:
an image obtaining module 701, configured to obtain an input image and a template image.
A high-dimensional point set determining module 702, configured to perform feature extraction on the input image and the template image, and determine a first high-dimensional point set and a second high-dimensional point set; points in the second high-dimensional point set correspond to points in the first high-dimensional point set one to one.
The high-dimensional point set determining module 702 specifically includes: an image conversion unit for converting the input image and the template image into two image block-based high-dimensional images; and the high-dimensional point set determining unit is used for arranging the image blocks in the high-dimensional image into one-dimensional vectors according to a set image block area threshold value and determining a first high-dimensional point set and a second high-dimensional point set.
A nearest neighbor field determining module 703, configured to determine a nearest neighbor field by using nearest neighbor search according to the first high-dimensional point set and the second high-dimensional point set.
The nearest neighbor field determining module 703 specifically includes: a nearest neighbor field determining unit, configured to determine, for each point in the second high-dimensional point set, a nearest neighbor index of the point in the first high-dimensional point set through nearest neighbor search, and determine a nearest neighbor field.
A global information and local diversity information determining module 704, configured to determine global information and local diversity information of the nearest neighbor field by using a global perceptual diversity metric algorithm and a local diversity algorithm.
The global information and local diversity information determining module 704 specifically includes: a similarity metric function determination unit for determining a similarity metric according to a formula
Determining a similarity measure function; wherein NN (q)iP) is the point Q in the second high-dimensional point set QiNearest neighbors in said first set of high-dimensional points P; k (p)j) Counting the nearest neighbor indexes; k (p)j)＝|{qi∈Q:NN(qi,P)＝pj}|；QωA target point set in the range of the current sliding window omega is obtained; the decomposition unit is used for decomposing the similarity measurement function into a global score and a local score corresponding to each sliding window; a global information determining unit, configured to determine global information of the nearest neighbor field by using a global perceptual diversity metric algorithm according to the global score; a local diversity information determination unit, configured to determine, according to the local score, local diversity information of the nearest neighbor field by using the local diversity algorithm.
A heat map determining module 705, configured to utilize the global information to guide the local diversity information to determine a heat map.
And a matching image determining module 706, configured to determine a matching image according to the heat map.
The invention verifies the invention through the practical application data:
(1) data set
The data set of the invention is mainly a BBS (Best-budclips Similarity) evaluation set, and the BBS evaluation set is composed of BBS25/BBS50/BBS100 and three subdata sets respectively. Numeral 25/50/100 denotes the sampling interval between T (template) and I (input) obtained from the video sequence. In addition, the invention also uses a fourth data set TinyTLP evaluation set which is higher than the previous BBS evaluation set in quantity and difficulty.
(2) Evaluation index
In order to verify the robustness and the high efficiency of the GAD algorithm provided by the invention, indexes mainly used by the invention are Success Rate (SR) and average intersection-over-Union (MIoU). The present invention will use these two indicators to measure the accuracy of the algorithm. To measure the efficiency and complexity of the algorithm, the third measure given in the experimental comparison is the running time Runtime of the algorithm, usually milliseconds (ms). Since SR and MIoU are calculated from IoU, first the formula for calculation IoU is given:
further, the SR and MIoU metrics may be calculated by the following equations:
where sgn is a sign function, the function value is 1 when the condition indicating the function in parentheses is true, and 0 otherwise.
(3) Robustness:
table 1 is a result comparison table of different algorithms on three sub-evaluation sets of BBS provided by the invention, the quantitative results of the BBS evaluation set are shown in Table 1, and in the three sub-evaluation sets of BBS25/BBS50/BBS100, the accuracy evaluation indexes SR and MIoU of GAD are superior to DIS and IWU.
TABLE 1
Table 2 is a result comparison table of TinyTLP evaluation sets of different algorithms provided by the present invention, and the quantitative results of the TinyTLP evaluation sets are shown in table 2. It can be seen that the GAD proposed herein has a great advantage over DIS and IWU in both accuracy measures (SR and MIoU) and algorithm efficiency.
TABLE 2
To better visualize the advantages of GAD, a specific qualitative comparison of results is given in fig. 8 and 9. As can be seen from fig. 8-9, when both DIS and IWU algorithms obtain the result of the wrong bounding box, GAD can accurately find the correct bounding box and the corresponding heat map can clearly reflect the maximum value.
(4) Rotational insensitivity:
although DIS/IWU corresponding algorithm with deformation loss DDIS/DIWU has very good results on the data set, since both algorithms have a strong assumption: the object in the template image t (template) and the corresponding object in the best matching window in the target image i (input) have the same orientation and do not undergo large rotation and deformation. However, in real-world scenarios, large rotations and large deformations often occur and are a difficult problem to avoid. The invention provides that the four evaluation sets are changed again, so that the evaluation sets can have large rotation. Specifically, the template image T is rotated clockwise by 180 ° (the template is rotated by 180 °), and the object image I is kept unchanged. Since the template images in the original four evaluation sets have a certain degree of rotation and large deformation in the target image, the evaluation set rotated by 180 ° can be considered to have the characteristics of large rotation and large deformation.
Table 3 is a comparison table of quantitative results of the sensitivity of different algorithms to rotation provided by the present invention, as shown in table 3, it can be observed that DDIS and DIWU all obtain the best effect on the original four evaluation sets, but the success rate SR or the average cross-over ratio MIoU is sharply decreased on the new four evaluation data sets "rotation". And the GAD does not have the phenomenon of sharp decline of SR and MIoU on the 'rotation' of the new four evaluation data sets, almost simultaneously shows the best on the four data sets, and the best precision at present is obtained.
TABLE 3
The invention provides an efficient and robust similarity measurement aiming at template matching in a natural scene, has a plurality of invariances and shows excellent performance in a complex natural scene, and has the following characteristics:
(1) robust reliable-a novel global perceptual diversity (GAD) similarity metric is proposed. And reveals the importance of global semantic information. The GAD effectively utilizes global context information to supervise local diversity information, can effectively filter irrelevant areas such as background or local points, and has stronger robustness.
(2) High efficiency-a set of efficient algorithms for GAD similarity measurement is elaborated. According to the GAD similarity measurement function, the GAD similarity measurement method is divided into two parts which are mutually independent to operate, namely global perception information calculation and diversity information calculation, so that a large amount of repeated calculation is fully avoided, and efficient algorithm implementation of each part is elaborately designed. The efficient algorithm implementation can greatly reduce the complexity O (| I |. |) to O (| T |), which means that the operation time can be shortened by 3 to 4 orders of magnitude theoretically.
(3) According to the GAD similarity measurement function and the corresponding efficient algorithm implementation, the GAD algorithm is fully compared with the most advanced algorithms in the academia and the challenging evaluation set. It can be seen from the experiments that the GAD algorithm achieves the best results both qualitatively and quantitatively compared to the algorithm without deformation loss; in addition to this, the present invention is,
compared with an algorithm with deformation loss, the GAD algorithm is more robust under the condition of large rotation or large deformation; from the aspect of running time, the GAD algorithm is the most efficient template matching method based on the nearest neighbor field at present, and can be controlled to be tens of milliseconds under the condition of high resolution.
The embodiments in the present description are described in a progressive manner, each embodiment focuses on differences from other embodiments, and the same and similar parts among the embodiments are referred to each other. For the system disclosed by the embodiment, the description is relatively simple because the system corresponds to the method disclosed by the embodiment, and the relevant points can be referred to the method part for description.
The principles and embodiments of the present invention have been described herein using specific examples, which are provided only to help understand the method and the core concept of the present invention; meanwhile, for a person skilled in the art, according to the idea of the present invention, the specific embodiments and the application range may be changed. In view of the above, the present disclosure should not be construed as limiting the invention.
Claims (6)
1. A template matching method based on global perceptual diversity measurement is characterized by comprising the following steps:
acquiring an input image and a template image;
performing feature extraction on the input image and the template image to determine a first high-dimensional point set and a second high-dimensional point set; points in the second high-dimensional point set correspond to points in the first high-dimensional point set one by one;
determining a nearest neighbor field by utilizing nearest neighbor search according to the first high-dimensional point set and the second high-dimensional point set;
determining global information and local diversity information of the nearest neighbor field by using a global perception diversity measurement algorithm and a local diversity algorithm; the determining global information and local diversity information of the nearest neighbor field by using a global perceptual diversity metric algorithm and a local diversity algorithm specifically includes: according to the formula
Determining a similarity measure function; wherein NN (q)iP) is the point Q in the second high-dimensional point set QiNearest neighbors in said first set of high-dimensional points P; k (p)j) Counting the nearest neighbor indexes; k (p)j)＝|{qi∈Q:NN(qi,P)＝pj}|；QωA target point set in the range of the current sliding window omega is obtained;
decomposing the similarity measurement function into a global score and a local score corresponding to each sliding window;
determining global information of the nearest neighbor field by using a global perception diversity measurement algorithm according to the global score;
determining local diversity information of the nearest neighbor field by using the local diversity algorithm according to the local score;
guiding the local diversity information to determine a heat map by using the global information; local diversity is guided through global information, namely two matrixes are multiplied to obtain an integral heat map;
and determining a matching image according to the heat map.
2. The template matching method based on global perceptual diversity measure according to claim 1, wherein the performing feature extraction on the input image and the template image to determine a first high-dimensional point set and a second high-dimensional point set specifically comprises:
converting the input image and the template image into two high-dimensional images based on image blocks;
and arranging the image blocks in the high-dimensional image into one-dimensional vectors according to a set image block area threshold, and determining a first high-dimensional point set and a second high-dimensional point set.
3. The template matching method based on the global perceptual diversity metric according to claim 1, wherein the determining a nearest neighbor field by using a nearest neighbor search according to the first high-dimensional point set and the second high-dimensional point set specifically comprises:
for each point in the second high-dimensional point set, determining a nearest neighbor index of the point in the first high-dimensional point set by nearest neighbor search, and determining a nearest neighbor field.
4. A template matching system based on a global perceptual diversity metric, comprising:
the image acquisition module is used for acquiring an input image and a template image;
the high-dimensional point set determining module is used for extracting the characteristics of the input image and the template image and determining a first high-dimensional point set and a second high-dimensional point set; points in the second high-dimensional point set correspond to points in the first high-dimensional point set one by one;
a nearest neighbor field determining module, configured to determine a nearest neighbor field by using nearest neighbor search according to the first high-dimensional point set and the second high-dimensional point set;
the global information and local diversity information determining module is used for determining global information and local diversity information of the nearest neighbor field by utilizing a global perception diversity measurement algorithm and a local diversity algorithm; the global information and local diversity information determining module specifically includes: a similarity metric function determination unit for determining a similarity metric according to a formula
Determining a similarity measure function; wherein NN (q)iP) is the point Q in the second high-dimensional point set QiNearest neighbors in said first set of high-dimensional points P; k (p)j) Counting the nearest neighbor indexes; k (p)j)＝|{qi∈Q:NN(qi,P)＝pj}|；QωA target point set in the range of the current sliding window omega is obtained;
the decomposition unit is used for decomposing the similarity measurement function into a global score and a local score corresponding to each sliding window;
a global information determining unit, configured to determine global information of the nearest neighbor field by using a global perceptual diversity metric algorithm according to the global score;
a local diversity information determination unit configured to determine local diversity information of the nearest neighbor field by using the local diversity algorithm according to the local score;
the heat map determining module is used for guiding the local diversity information to determine a heat map by utilizing the global information; local diversity is guided through global information, namely two matrixes are multiplied to obtain an integral heat map;
and the matching image determining module is used for determining a matching image according to the heat map.
5. The template matching system based on global perceptual diversity measure of claim 4, wherein the high-dimensional point set determination module specifically comprises:
an image conversion unit for converting the input image and the template image into two image block-based high-dimensional images;
and the high-dimensional point set determining unit is used for arranging the image blocks in the high-dimensional image into one-dimensional vectors according to a set image block area threshold value and determining a first high-dimensional point set and a second high-dimensional point set.
6. The template matching system based on global perceptual diversity metric of claim 4, wherein the nearest neighbor field determination module specifically comprises:
a nearest neighbor field determining unit, configured to determine, for each point in the second high-dimensional point set, a nearest neighbor index of the point in the first high-dimensional point set through nearest neighbor search, and determine a nearest neighbor field.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910022545.5A CN109766943B (en) | 2019-01-10 | 2019-01-10 | Template matching method and system based on global perception diversity measurement |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910022545.5A CN109766943B (en) | 2019-01-10 | 2019-01-10 | Template matching method and system based on global perception diversity measurement |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN109766943A CN109766943A (en) | 2019-05-17 |
| CN109766943B true CN109766943B (en) | 2020-08-21 |
Family
ID=66453689
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201910022545.5A Active CN109766943B (en) | 2019-01-10 | 2019-01-10 | Template matching method and system based on global perception diversity measurement |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN109766943B (en) |
Families Citing this family (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN110738268A (en) * | 2019-10-18 | 2020-01-31 | 广东华南半导体光电研究院有限公司 | intelligent stereoscopic warehouse goods automatic identification method based on SIFT and DDIS |
| CN111950584B (en) * | 2020-06-16 | 2024-05-14 | 江西中科九峰智慧医疗科技有限公司 | Intelligent identification method and system for position integrity in X-ray chest radiography |
| CN111709434B (en) * | 2020-06-28 | 2022-10-04 | 哈尔滨工业大学 | Robust multi-scale template matching method based on nearest neighbor feature point matching |
Citations (9)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101147159A (en) * | 2005-02-21 | 2008-03-19 | 三菱电机株式会社 | Fast method of object detection by statistical template matching |
| US8811724B2 (en) * | 2010-05-11 | 2014-08-19 | The University Of Copenhagen | Classification of medical diagnostic images |
| CN104036528A (en) * | 2014-06-26 | 2014-09-10 | 西北农林科技大学 | Real-time distribution field target tracking method based on global search |
| CN104517101A (en) * | 2014-12-19 | 2015-04-15 | 中南大学 | Game poker card recognition method based on pixel square difference matching |
| CN104915949A (en) * | 2015-04-08 | 2015-09-16 | 华中科技大学 | Image matching algorithm of bonding point characteristic and line characteristic |
| WO2016010724A1 (en) * | 2014-07-15 | 2016-01-21 | Qualcomm Incorporated | Multispectral eye analysis for identity authentication |
| CN105513038A (en) * | 2014-10-20 | 2016-04-20 | 网易（杭州）网络有限公司 | Image matching method and mobile phone application test platform |
| CN105740899A (en) * | 2016-01-29 | 2016-07-06 | 长安大学 | Machine vision image characteristic point detection and matching combination optimization method |
| CN108846390A (en) * | 2013-09-16 | 2018-11-20 | 眼验股份有限公司 | Feature extraction and matching and template renewal for biological identification |
-
2019
- 2019-01-10 CN CN201910022545.5A patent/CN109766943B/en active Active
Patent Citations (9)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101147159A (en) * | 2005-02-21 | 2008-03-19 | 三菱电机株式会社 | Fast method of object detection by statistical template matching |
| US8811724B2 (en) * | 2010-05-11 | 2014-08-19 | The University Of Copenhagen | Classification of medical diagnostic images |
| CN108846390A (en) * | 2013-09-16 | 2018-11-20 | 眼验股份有限公司 | Feature extraction and matching and template renewal for biological identification |
| CN104036528A (en) * | 2014-06-26 | 2014-09-10 | 西北农林科技大学 | Real-time distribution field target tracking method based on global search |
| WO2016010724A1 (en) * | 2014-07-15 | 2016-01-21 | Qualcomm Incorporated | Multispectral eye analysis for identity authentication |
| CN105513038A (en) * | 2014-10-20 | 2016-04-20 | 网易（杭州）网络有限公司 | Image matching method and mobile phone application test platform |
| CN104517101A (en) * | 2014-12-19 | 2015-04-15 | 中南大学 | Game poker card recognition method based on pixel square difference matching |
| CN104915949A (en) * | 2015-04-08 | 2015-09-16 | 华中科技大学 | Image matching algorithm of bonding point characteristic and line characteristic |
| CN105740899A (en) * | 2016-01-29 | 2016-07-06 | 长安大学 | Machine vision image characteristic point detection and matching combination optimization method |
Non-Patent Citations (6)
| Title |
|---|
| Globally Optimal Estimation of Nonrigid Image Distortion;Yuandong Tian et al;《International Journal of Computer Vision》;20111221;第279-302页 * |
| OATM: Occlusion Aware Template Matching by Consensus Set Maximization;Simon Korman et al;《arXiv:1804.02638v1 [cs.CV]》;20180408;第1-9页 * |
| Template matching with deformable diversity similarity;Itamar Talmi et al;《arXiv:1612.02190v2 [cs.CV]》;20170418;第1-9页 * |
| 基于形变多样相似性的空间正则化相关滤波跟踪;毛宁 等;《光学学报》;20181217;第39卷(第4期);第1-11页 * |
| 基于边缘几何特征的高性能模板匹配算法;吴晓军 等;《仪器仪表学报》;20130715;第34卷(第7期);第1462-1469页 * |
| 多尺度显著性区域提取的模板匹配;逯睿琦 等;《光学精密工程》;20181115;第26卷(第11期);第2776-2784页 * |
Also Published As
| Publication number | Publication date |
|---|---|
| CN109766943A (en) | 2019-05-17 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN109766943B (en) | Template matching method and system based on global perception diversity measurement | |
| CN103310453B (en) | A kind of fast image registration method based on subimage Corner Feature | |
| Venkatesh et al. | Efficient object-based video inpainting | |
| CN105009170A (en) | Object identification device, method, and storage medium | |
| Kat et al. | Matching pixels using co-occurrence statistics | |
| CN105976399A (en) | Moving object detection method based on SIFT (Scale Invariant Feature Transform) feature matching | |
| Montesinos et al. | Matching color uncalibrated images using differential invariants | |
| JP5754055B2 (en) | Information representation method of object or shape | |
| CN105654493B (en) | A kind of affine constant binocular solid Matching power flow of improved optics and parallax optimization method | |
| US11030478B1 (en) | System and method for correspondence map determination | |
| CN108960280B (en) | Picture similarity detection method and system | |
| Li et al. | Coarse-to-fine PatchMatch for dense correspondence | |
| CN110598795A (en) | Image difference detection method and device, storage medium and terminal | |
| Yung et al. | Efficient feature-based image registration by mapping sparsified surfaces | |
| US11256949B2 (en) | Guided sparse feature matching via coarsely defined dense matches | |
| Donoser et al. | Robust planar target tracking and pose estimation from a single concavity | |
| Cui et al. | A new image registration scheme based on curvature scale space curve matching | |
| CN117078726A (en) | Different spectrum image registration method based on edge extraction | |
| JP2005354201A (en) | Apparatus and method for image processing | |
| CN110503110A (en) | Feature matching method and device | |
| CN116188535A (en) | Video tracking method, device, equipment and storage medium based on optical flow estimation | |
| Phogat et al. | Different image registration methods—an overview | |
| US11232323B2 (en) | Method of merging images and data processing device | |
| CN107451574B (en) | Motion estimation method based on Haar-like visual feature perception | |
| Hu et al. | Digital video stabilization based on multilayer gray projection |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |