WO2021017361A1

WO2021017361A1 - Template matching algorithm based on edge and gradient feature

Info

Publication number: WO2021017361A1
Application number: PCT/CN2019/123994
Authority: WO
Inventors: 邢述达; 郭晓锋; 余章卫
Original assignee: 苏州中科全象智能科技有限公司
Priority date: 2019-07-31
Filing date: 2019-12-09
Publication date: 2021-02-04
Also published as: CN110472674A; CN110472674B

Abstract

A template matching algorithm based on edge and gradient features. A matching process comprises an offline stage and an online stage. The offline stage corresponds to processing, comprising at least edge extraction and gradient computation, of a template image, and generation of an R-table of generalized Hough transform; and the online stage corresponds to edge extraction, gradient computation and coarse-to-fine matching processing of an input target image, acquisition, according to an angle and a position and by means of the R-table, of a value in a voting space of generalized Hough transform, and acquisition of a target position. The template matching algorithm combines a pyramid model with an optimized search strategy, has an anti-interference capability, can achieve rotational invariance, and has a certain capability to resist scaling and occlusion; moreover, the computation amount of generalized Hough transform is greatly reduced, and the matching rate is increased manyfold.

Description

A template matching algorithm based on edge and gradient features

This disclosure claims the priority of a Chinese patent application filed with the Chinese Patent Office with an application number of 201910699607.6 on July 31, 2019, and the entire content of the above application is incorporated into this disclosure by reference.

Technical field

This application relates to an industrial machine vision image processing technology, for example, to a template matching algorithm based on edge and gradient features.

Background technique

Template matching algorithm is one of the difficulties and cores of machine vision, and it is widely used in the fields of target recognition, workpiece positioning and target extraction in videos. The purpose of template matching is to find an object similar to the target in the image and obtain its position. This method includes many image processing techniques, such as image preprocessing, image segmentation, and similarity evaluation.

In related technologies, the methods used in the industry mainly include two aspects, feature-based and grayscale-based matching algorithms. Feature-based algorithms mainly include keypoint-based, edge-based, and line-based and arc-based primitive models. .

Template matching based on grayscale is time-consuming, and it is difficult to resist rotation and scaling. Therefore, this method is generally not used in situations where time requirements are relatively strict. Template matching based on key points is very sensitive to occlusion, and strongly depends on key point search, and has poor anti-interference. Although model matching algorithms based on primitives such as lines and arcs have certain advantages, primitive modeling and Difficulty in parameter adjustment. Therefore, it is necessary to develop a matching algorithm with low development complexity, reliable matching and strong real-time performance.

Summary of the invention

This application provides a template matching algorithm based on edge and gradient features, which is based on the generalized Hough transform and combined with a pyramid model to reduce computational complexity, and endow it with strong anti-interference, rotation invariance, resistance to partial occlusion and scaling, etc. performance.

This application provides a template matching algorithm based on edge and gradient features. The matching process includes two stages, offline and online. The offline stage corresponds to the template image and performs at least edge extraction and gradient calculation processing to generate R of the generalized Hough transform. -table; the online stage corresponds to the input target image for edge extraction, gradient calculation, and coarse-to-fine matching processing. According to the angle and position, the R-table is used to obtain the value in the voting space of the generalized Hough transform to obtain the target position.

Optionally, the processing process in the offline phase includes:

Image pyramid generation, using the pyramid model to layer the template image to get each layer of images from high to low;

Sub-pixel edge extraction, using the local area effect to obtain the single boundary of the sub-pixel edge of different pyramid layer images, and obtain the boundary direction at the same time;

Construct the R-table of the generalized Hough transform, rotate and scale the obtained features, in which the topmost image directly establishes a 360°rotated R-table, and the non-rotated R-table is established for the images except the topmost layer.

Optionally, before constructing the R-table of the generalized Hough transform, edge filtering is also included. The edge is chain-coded and the first threshold is preset, and the eight-neighborhood search of the edge angle is performed one by one: the angle of the current point is located at the front and rear edges The retention within the range of the angle, the retention where the angle deviation between the current point angle and the previous edge or the back edge is less than the first threshold value, the retention where the angle deviation from the front and rear edges is greater than the removal of the first threshold value.

Optionally, the processing process in the online phase includes:

Image pyramid generation, using the pyramid model to layer the input target image to get each layer of images from high to low;

Layered coarse matching, the highest level of the image pyramid adopts the generalized Hough transform, and the 360° full template matching with the default zoom factor of 1 is performed to obtain the rough position and angle (x, y, theta) of the elements in the target image at the highest level; The matching results of one layer are matched layer by layer down to the bottom layer, and the search is performed in the surrounding range of the matching results of the upper layer to obtain the position and angle of the elements in the target image at each non-highest layer;

Fine matching, according to the position and angle obtained by rough matching, obtain the Hough space result conforming to the Gaussian distribution law, fit the surface and obtain the global maximum value to the position and angle obtained in each layer, and obtain the final element in the target image Output the result.

Optionally, the pyramid model used for the layering of the target image is a combined model of Gaussian pyramid and mean value pyramid, and the Gaussian convolution and mean value filtering both use 3*3 convolution kernels.

Optionally, the optimal T-point automatic threshold calculation based on the gradient histogram is used in the edge extraction process, and the threshold is given to the extracted boundary.

Optionally, the fitting surface adopts a quadratic spherical fitting equation, and the local extreme point is obtained within the range of the selected point to obtain the coordinate point position of the sub-pixel.

Compared with related technologies, the template matching algorithm method of this application is applied and implemented. The algorithm combines the pyramid model and optimized search strategy, has anti-interference, can achieve rotation invariance, and has a certain ability to resist scaling and occlusion; moreover, the generalized Huo The calculation amount of the husband transform is greatly reduced, and the matching rate is doubled.

Description of the drawings

Figure 1 is a schematic diagram of the generalized Hough transform.

Figure 2 is a schematic flow chart of the template matching algorithm of this application.

Figure 3 is a schematic diagram of edge screening in the offline phase.

Figure 4 is a schematic diagram of the range of coarse matching in the online phase.

Fig. 5 is a template image and a corner frame as an element.

Figure 6 shows the target image and its elements of occlusion, scaling, and rotation.

Detailed ways

The innovative features and implementation manners of the technical solution of the present application will be further described in detail below in conjunction with the accompanying drawings.

In view of the various defects of various template matching-based algorithms in related technologies, this application is committed to optimizing industrial machine vision image processing technology, combining its own experience and creative work, and researches and proposes a method based on edges and gradients. The characteristic template matching algorithm has the advantages of strong anti-interference, rotation invariance, resistance to partial occlusion and scaling.

This application discloses a template matching algorithm based on edge and gradient features. The main body algorithm of template matching uses generalized Hough transform. The generalized Hough transform describes a graph in the form of a table, as shown in Figure 1. The parameters mainly include the angle of the edge normal vector

The vector r where the edge point points to the center, the angle ψ of the vector r. Save the coordinates of the edge points of the graph in a table, then the graph is determined, whether it is a straight line (actually a line segment), a circle, an ellipse or other geometric shapes, it can be processed in the same way. The generalized Hough transform adds correlation to the parameters, so that only one parameter needs to be traversed in the case of translation and rotation (no scaling). The three parameters of the edge of the image are the center coordinate (horizontal and vertical) of the graphic, the rotation angle (relative to the reference graphic) and the vector from the edge to the center point. First, save the radial value of the reference graphic edge point to the center, use the gradient direction of the graphic edge point to be searched (represented by the angle relative to the coordinate axis) as an index to find the corresponding radial value, and add the vector to complete the projection. So the only parameter to be traversed is the rotation angle, as shown in the following table:

IndexIndex	ψψ	r _i r _i
00	00	{r _i\|ψ _i＝0} {r _i \|ψ _i = 0}
11	dψdψ	{r _i\|ψ _i＝dψ} {r _i \|ψ _i ＝dψ}
22	2dψ2dψ	{r _i\|ψ _i＝2dψ} {r _i \|ψ _i ＝2dψ}
....	....	....
....	....	....

To build an R-table, plus zoom, only need to traverse two parameters. The main thing in template matching is to obtain the angle and vector table in the offline stage. In the online detection stage, the position of the voting point is obtained by using the vector table according to the angle and position, and the point whose voting exceeds a certain threshold is the candidate point of the matching target.

In order to improve the matching speed, this application adopts the pyramid model to reduce the computational complexity. The matching process mainly includes two stages, offline and online, as shown in FIG. 2, which is the process of an embodiment of the template matching algorithm of the present application.

The main function of the offline stage is to process the template image and generate the R-table of the generalized Hough transform. The process can include image pyramid generation, stable sub-pixel edge extraction and construction of the R-table of generalized Hough transform.

In one embodiment, the image pyramid is generated by using a Gaussian pyramid model to layer the template image to obtain images of each layer from high to low; the Gaussian pyramid model has a good ability to preserve edges and uses a 3*3 convolution kernel , Can take into account the speed requirements.

Sub-pixel edge extraction uses the local area effect to obtain the single boundary of the sub-pixel edge of different pyramid layer images, while obtaining the boundary direction; this algorithm has the advantages of boundary positioning accuracy and angle calculation error.

Edge screening, because in the calculation process, the local area will have calculation errors due to the interference of noise. In order to avoid such problems, this application adopts the eight-neighborhood search of the edge angles one by one after edge extraction. The current point's angle is located The retention within the range of the front and rear edge angles, the retention where the angle deviation between the current point angle and the previous edge or the back edge is less than the first threshold, and the removal where the angle deviation from the front and rear edges is greater than the first threshold. As shown in Figure 3, the search process can be: the edge adopts chain coding and the first threshold is preset, and the current point calculation angle is within the range of the front and rear edge angles, that is, point 2 is reserved. Otherwise, perform threshold filtering: the angle deviation between the current angle and the previous edge is within the first threshold, then point 4 is retained, when the condition is not met, and then compared with the next edge, when both exceed the first threshold, point 5 is removed .

The online stage corresponds to the input target image for edge extraction, gradient calculation, and coarse-to-fine matching processing. According to the angle and position, the R-table is used to obtain the value in the voting space of the generalized Hough transform to obtain the target position. It mainly includes image pyramid generation, edge extraction and gradient calculation, violent matching at the highest level of the pyramid, downward layer-by-layer matching and fine matching algorithms. The overview process is shown in Figure 2. The image pyramid uses the Gaussian pyramid model, the mean pyramid model or a combination of the two, and the edge extraction algorithm can realize automatic threshold setting. After the highest level of the pyramid performs template matching, the object is obtained In the rough position of the highest layer, when entering the next layer for matching, the surrounding changes of the acquired parameters can be matched within a certain range without full matching, which can greatly reduce the computational complexity and reduce the running time. Fine matching is to perform surface fitting on the results (including position and angle) within the variation range of the result after coarse matching, and obtain the global maximum value, which is the calculation result of fine matching.

In some embodiments, the processing in the online stage includes: using a combination of a mean pyramid and a Gaussian pyramid to layer the input target image to obtain images of each layer from high to low. The Gaussian pyramid has a good ability to preserve edges. The mean pyramid has the characteristics of fast calculation speed and also has a good ability to suppress noise, but it will blur the edges. Therefore, in this application, the Gaussian pyramid is used when extracting edges, and the mean pyramid is used when matching from top to bottom. Among them, Gaussian convolution and mean filtering both use a 3*3 convolution kernel, which can achieve a good balance between speed and accuracy.

Sub-pixel edge gradient acquisition and automatic threshold algorithm. As in the offline phase, the single boundary of the sub-pixel edge is obtained based on the local area effect, and the direction of the boundary is obtained at the same time. The rough matching process of this application controls the angle at plus or minus 2 degrees. Therefore, if the angle is accurate to 1 degree, it is directly divided into 360 degrees; the specified angle is from 0 to 360 degrees, and the double type angle floor value is adopted. Because in the process of extracting edges, it is necessary to manually set thresholds for the extracted boundaries. In order to avoid this problem, this application adopts the optimal T-point automatic threshold calculation based on the gradient histogram, and sets the threshold for the extracted boundaries. value.

As shown in Figure 4, first perform layered rough matching. At the highest level of the image pyramid, generalized Hough transform is used to perform 360° full template matching with the default zoom factor of 1, to obtain the rough position and angle of the element in the target image at the highest level (x ,y,theta); use the matching result of the upper layer to match down to the bottom layer, and search in the surrounding range of the matching result of the upper layer (related to position and angle) to obtain the elements in the target image at each non-top layer Position and angle;

Then fine matching is performed. According to the position and angle obtained by rough matching, the Hough space result conforming to the Gaussian distribution law is obtained, and the position and angle obtained in each layer are fitted to the surface and the global maximum value is obtained to obtain the elements in the target image The final output result. The fitting surface adopts the quadratic spherical fitting equation to obtain the local extreme point within the range of the selected point, and obtain the coordinate point position of the sub-pixel.

In summary of the detailed description of the embodiments of the present application and the accompanying drawings, it can be understood that: the application and implementation of the template matching algorithm of the present application, combined with the pyramid model and optimized search strategy, has a small amount of calculation, anti-interference, can achieve rotation invariance, and It has a certain ability to resist scaling and occlusion. The angle irons in the template image and target image shown in Figure 5 and Figure 6 are visible. The matching target has a certain occlusion and screening at any angle. The tested image parameters and PC parameters are as follows The table shows:

图像参数Image parameters	电脑参数Computer parameters
类型：RGB图像Type: RGB image	处理器：Intel Core i7Processor: Intel Core i7
大小：1000600Size: 1000600	CPU频率：3.7GHzCPU frequency: 3.7GHz

The test result shows that the time required to use the generalized Hough transform in the related technology is 734 ms, while the template matching algorithm of the present application only requires 74 ms, and the matching rate is doubled.

Claims

A template matching algorithm based on edge and gradient features. The matching process includes two stages, offline and online. The offline stage corresponds to the template image for processing at least including edge extraction and gradient calculation to generate a generalized Hough transform R-table; In the stage, edge extraction, gradient calculation and matching processing from coarse to fine are performed on the input target image. According to the angle and position, the R-table is used to obtain the value in the voting space of the generalized Hough transform to obtain the target position.
The template matching algorithm based on edge and gradient features according to claim 1, wherein the processing process in the offline phase includes:

Image pyramid generation, using the pyramid model to layer the template image to get each layer of images from high to low;

Sub-pixel edge extraction, using the local area effect to obtain the single boundary of the sub-pixel edge of different pyramid layer images, and obtain the boundary direction at the same time;

Construct the R-table of the generalized Hough transform, rotate and scale the obtained features, in which the topmost image directly establishes a 360°rotated R-table, and the non-rotated R-table is established for the images except the topmost layer.
The template matching algorithm based on edge and gradient features according to claim 2, wherein before constructing the R-table of the generalized Hough transform, it also includes edge filtering, adopting chain coding for the edges and preset the first threshold value, and the edges are performed one by one. Eight-neighborhood search of angles: the retention of the current point's angle within the range of the front and rear edge angles, the retention of the angle of the current point and the previous or back edge is less than the first threshold, and the angle deviation from the front and back edges Both are greater than the removal of the first threshold.
The template matching algorithm based on edge and gradient features of claim 1, wherein the processing process in the online phase includes:

Image pyramid generation, using the pyramid model to layer the input target image to get each layer of images from high to low;

Sub-pixel edge extraction, using the local area effect to obtain a single boundary of the sub-pixel edges of different pyramid layer images, and obtain the boundary direction at the same time;

Layered coarse matching, the highest level of the image pyramid adopts the generalized Hough transform, and the 360° full template matching with the default zoom factor of 1 is performed to obtain the rough position and angle (x, y, theta) of the elements in the target image at the highest level; The matching results of one layer are matched layer by layer down to the bottom layer, and the search is performed in the surrounding range of the matching results of the upper layer to obtain the position and angle of the elements in the target image at each non-highest layer;

Fine matching, according to the position and angle obtained by rough matching, obtain the Hough space result conforming to the Gaussian distribution law, fit the surface and obtain the global maximum value to the position and angle obtained in each layer, and obtain the final element in the target image Output the result.
The template matching algorithm based on edge and gradient features according to claim 4, wherein the pyramid model used for the layering of the target image is a combined model of Gaussian pyramid and mean pyramid, and both Gaussian convolution and mean filtering are used It is a 3*3 convolution kernel.
The template matching algorithm based on edge and gradient features according to claim 4, wherein the optimal T-point automatic threshold calculation based on the gradient histogram is used in the process of edge extraction, and the threshold is given to the extracted boundary.
The template matching algorithm based on edge and gradient features according to claim 4, wherein the fitting surface adopts a quadratic spherical fitting equation to obtain local extreme points within the range of selected points to obtain sub-pixel coordinates Point location.