CN115423855B - Template matching method, device, equipment and medium for image - Google Patents
Template matching method, device, equipment and medium for image Download PDFInfo
- Publication number
- CN115423855B CN115423855B CN202211374039.0A CN202211374039A CN115423855B CN 115423855 B CN115423855 B CN 115423855B CN 202211374039 A CN202211374039 A CN 202211374039A CN 115423855 B CN115423855 B CN 115423855B
- Authority
- CN
- China
- Prior art keywords
- information
- neighborhood
- value
- matched
- determining
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
- 238000000034 method Methods 0.000 title claims abstract description 44
- 238000004364 calculation method Methods 0.000 claims description 48
- 239000013598 vector Substances 0.000 claims description 36
- 238000004590 computer program Methods 0.000 claims description 17
- 238000009826 distribution Methods 0.000 claims description 13
- 238000010606 normalization Methods 0.000 claims description 11
- 238000000354 decomposition reaction Methods 0.000 claims description 6
- 238000000605 extraction Methods 0.000 claims description 4
- 238000012935 Averaging Methods 0.000 claims description 3
- 230000000877 morphologic effect Effects 0.000 abstract description 4
- 238000005516 engineering process Methods 0.000 description 9
- 230000008569 process Effects 0.000 description 6
- 238000001514 detection method Methods 0.000 description 5
- 238000010586 diagram Methods 0.000 description 4
- 230000004044 response Effects 0.000 description 4
- 230000008878 coupling Effects 0.000 description 3
- 238000010168 coupling process Methods 0.000 description 3
- 238000005859 coupling reaction Methods 0.000 description 3
- 230000006870 function Effects 0.000 description 3
- 239000011159 matrix material Substances 0.000 description 3
- 238000005070 sampling Methods 0.000 description 3
- 230000008859 change Effects 0.000 description 2
- 238000004891 communication Methods 0.000 description 2
- 238000012545 processing Methods 0.000 description 2
- 241001292396 Cirrhitidae Species 0.000 description 1
- 238000003491 array Methods 0.000 description 1
- 230000009286 beneficial effect Effects 0.000 description 1
- 238000004422 calculation algorithm Methods 0.000 description 1
- 238000013461 design Methods 0.000 description 1
- 230000000694 effects Effects 0.000 description 1
- 238000005286 illumination Methods 0.000 description 1
- 230000003993 interaction Effects 0.000 description 1
- 238000005259 measurement Methods 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 230000003287 optical effect Effects 0.000 description 1
- 230000000717 retained effect Effects 0.000 description 1
- 238000010561 standard procedure Methods 0.000 description 1
- 238000006467 substitution reaction Methods 0.000 description 1
- 238000012360 testing method Methods 0.000 description 1
- 238000012795 verification Methods 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/30—Determination of transform parameters for the alignment of images, i.e. image registration
- G06T7/33—Determination of transform parameters for the alignment of images, i.e. image registration using feature-based methods
- G06T7/337—Determination of transform parameters for the alignment of images, i.e. image registration using feature-based methods involving reference images or patches
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/0002—Inspection of images, e.g. flaw detection
- G06T7/0004—Industrial image inspection
- G06T7/001—Industrial image inspection using an image reference approach
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
- G06V10/74—Image or video pattern matching; Proximity measures in feature spaces
- G06V10/75—Organisation of the matching processes, e.g. simultaneous or sequential comparisons of image or video features; Coarse-fine approaches, e.g. multi-scale approaches; using context analysis; Selection of dictionaries
- G06V10/751—Comparing pixel values or logical combinations thereof, or feature values having positional relevance, e.g. template matching
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/30—Subject of image; Context of image processing
- G06T2207/30108—Industrial image inspection
- G06T2207/30164—Workpiece; Machine component
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V2201/00—Indexing scheme relating to image or video recognition or understanding
- G06V2201/06—Recognition of objects for industrial automation
Abstract
The application is applicable to the technical field of machine vision, and particularly relates to a template matching method, device, equipment and medium of an image. According to the method, for any region to be matched, a registration region corresponding to the region to be matched is determined in a template image, first target information and second target information of the two regions are obtained, gray characteristic information and structural characteristic information of the two regions are determined according to the two target information, gray difference information is obtained by comparing the two gray characteristic information, structural difference information is obtained by comparing the two structural characteristic information, weighted summation is carried out on the gray difference information and the structural difference information, a weighted summation result is obtained, when the weighted summation result meets a preset condition, the target image is determined to be matched with the template image in the region to be matched, matching operation of the target image and the template image is achieved, and the influence of morphological similarity on template matching can be well corrected by combining the gray characteristic and the structural characteristic, so that the accuracy of template matching is improved.
Description
Technical Field
The application is applicable to the technical field of machine vision, and particularly relates to a template matching method, device, equipment and medium of an image.
Background
Currently, in a precise industrial scene, a template matching technology is required to be used to realize detection, measurement, verification and other works. The template matching technology can be simply divided into a one-to-one single-target matching technology and a one-to-many multi-target matching technology from application, and the application scene requires the use of the one-to-many multi-target matching technology, and in general, the template matching technology can be divided into the following steps: extracting features of a template image, finding all candidate targets in a large image, extracting features of the candidate targets, and calculating the matching degree between the features of the candidate targets and the features of the template image, wherein the existing common calculation mode is a gray template matching technology based on a normalized cross-correlation matching method (Normalized Cross Correlation, NCC), the technology has small influence on illumination, noise and the like, the stability is better, the technology is also a pixel-based template matching standard method embedded in commercial software such as halcon and the like, the theoretical matching degree range is [ -1,1] and represents that polarities of two images are completely opposite (original image and image after color reversal) when the theoretical matching degree is-1, the two images are completely identical when the theoretical matching degree is-1, and the absolute value is generally taken when the NCC is calculated, so that the NCC is generally [0,1] and the larger the value is more similar the two images are represented. However, only the numerical value of the data on the image is considered when the pure gray scale calculation is based, the interference resistance is low, and the situation that the gray scale distribution is the same but the forms are different is often encountered in the actual situation, and the situation is judged to be highly matched, so that misjudgment is caused. Therefore, how to reduce the influence of morphological similarity on template matching to improve the accuracy of template matching is a problem to be solved.
Disclosure of Invention
In view of this, the embodiments of the present application provide a method, apparatus, device, and medium for matching templates of images, so as to solve the problem of how to reduce the influence of morphological similarity on template matching, so as to improve the accuracy of template matching.
In a first aspect, an embodiment of the present application provides a template matching method for an image, where the template matching method includes:
determining a registration area corresponding to the area to be matched in a template image aiming at any area to be matched in a target image, and acquiring first target information formed by pixel information of the area to be matched and a neighborhood thereof and second target information formed by pixel information of the registration area and the neighborhood thereof;
determining first gray scale characteristic information and first structural characteristic information on the region to be matched and the neighborhood thereof according to the first target information, and determining second gray scale characteristic information and second structural characteristic information on the registration region and the neighborhood thereof according to the second target information;
comparing the first gray scale characteristic information with the second gray scale characteristic information to obtain gray scale difference information, and comparing the first structural characteristic information with the second structural characteristic information to obtain structural difference information;
And carrying out weighted summation on the gray level difference information and the structure difference information to obtain a weighted summation result, and determining that the target image is matched with the template image in the region to be matched when the weighted summation result meets a preset condition.
In a second aspect, an embodiment of the present application provides a template matching apparatus for an image, including:
the information acquisition module is used for determining a registration area corresponding to the area to be matched in a template image aiming at any area to be matched in the target image, acquiring first target information formed by pixel information of the area to be matched and a neighborhood thereof, and forming second target information by pixel information of the registration area and the neighborhood thereof;
the feature determining module is used for determining first gray feature information and first structural feature information on the region to be matched and the neighborhood thereof according to the first target information, and determining second gray feature information and second structural feature information on the registration region and the neighborhood thereof according to the second target information;
the difference comparison module is used for comparing the first gray scale characteristic information with the second gray scale characteristic information to obtain gray scale difference information, and comparing the first structural characteristic information with the second structural characteristic information to obtain structural difference information;
And the matching result determining module is used for carrying out weighted summation on the gray level difference information and the structure difference information to obtain a weighted summation result, and determining that the target image is matched with the template image in the region to be matched when the weighted summation result meets the preset condition.
In a third aspect, embodiments of the present application provide a computer device, the computer device comprising a processor, a memory, and a computer program stored in the memory and executable on the processor, the processor implementing the template matching method according to the first aspect when executing the computer program.
In a fourth aspect, embodiments of the present application provide a computer readable storage medium storing a computer program, which when executed by a processor implements the template matching method according to the first aspect.
Compared with the prior art, the embodiment of the application has the beneficial effects that: according to any region to be matched in the target image, a registration region corresponding to the region to be matched is determined in the template image, first target information formed by pixel information of the region to be matched and the neighborhood thereof is obtained, and second target information formed by pixel information of the registration region and the neighborhood thereof is obtained.
Drawings
In order to more clearly illustrate the technical solutions of the embodiments of the present application, the drawings that are required for the embodiments or the description of the prior art will be briefly described below, it being obvious that the drawings in the following description are only some embodiments of the present application, and that other drawings may be obtained according to these drawings without inventive effort for a person skilled in the art.
Fig. 1 is a schematic view of an application environment of a template matching method for an image according to an embodiment of the present application;
fig. 2 is a flow chart of a template matching method for an image according to a second embodiment of the present application;
fig. 3 is a schematic structural diagram of a template matching device for an image according to a third embodiment of the present application;
fig. 4 is a schematic structural diagram of a computer device according to a fourth embodiment of the present application.
Detailed Description
In the following description, for purposes of explanation and not limitation, specific details are set forth, such as particular system configurations, techniques, etc. in order to provide a thorough understanding of the embodiments of the present application. It will be apparent, however, to one skilled in the art that the present application may be practiced in other embodiments that depart from these specific details. In other instances, detailed descriptions of well-known systems, devices, circuits, and methods are omitted so as not to obscure the description of the present application with unnecessary detail.
It should be understood that the terms "comprises" and/or "comprising," when used in this specification and the appended claims, specify the presence of stated features, integers, steps, operations, elements, and/or components, but do not preclude the presence or addition of one or more other features, integers, steps, operations, elements, components, and/or groups thereof.
It should also be understood that the term "and/or" as used in this specification and the appended claims refers to any and all possible combinations of one or more of the associated listed items, and includes such combinations.
As used in this specification and the appended claims, the term "if" may be interpreted as "when..once" or "in response to a determination" or "in response to detection" depending on the context. Similarly, the phrase "if a determination" or "if a [ described condition or event ] is detected" may be interpreted in the context of meaning "upon determination" or "in response to determination" or "upon detection of a [ described condition or event ]" or "in response to detection of a [ described condition or event ]".
In addition, in the description of the present application and the appended claims, the terms "first," "second," "third," and the like are used merely to distinguish between descriptions and are not to be construed as indicating or implying relative importance.
Reference in the specification to "one embodiment" or "some embodiments" or the like means that a particular feature, structure, or characteristic described in connection with the embodiment is included in one or more embodiments of the application. Thus, appearances of the phrases "in one embodiment," "in some embodiments," "in other embodiments," and the like in the specification are not necessarily all referring to the same embodiment, but mean "one or more but not all embodiments" unless expressly specified otherwise. The terms "comprising," "including," "having," and variations thereof mean "including but not limited to," unless expressly specified otherwise.
It should be understood that the sequence numbers of the steps in the following embodiments do not mean the order of execution, and the execution order of the processes should be determined by the functions and the internal logic, and should not constitute any limitation on the implementation process of the embodiments of the present application.
In order to illustrate the technical solution of the present application, the following description is made by specific examples.
The template matching method for the image provided by the embodiment of the invention can be applied to an application environment as shown in fig. 1, wherein a client communicates with a server to form a cloud edge combined system. The client includes, but is not limited to, a palm computer, a desktop computer, a notebook computer, an ultra-mobile personal computer (UMPC), a netbook, a server computer device, a personal digital assistant (personal digital assistant, PDA), and other computer devices, and the client may also be a data processing device of an electrocardiograph, an electroencephalograph, and other measuring devices. The server may be implemented by a stand-alone server or a server cluster formed by a plurality of servers.
Referring to fig. 2, a flow chart of a template matching method of an image according to a second embodiment of the present application is shown, where the template matching method of an image is applied to a server in fig. 1, and a computer device corresponding to the server is connected to a corresponding database to obtain a corresponding target image and a template image in the database, and of course, the template image may also be directly stored in the server. The computer equipment can be connected with corresponding clients, the clients are operated by users, and the users can acquire corresponding target images through the clients and provide the corresponding target images to the server. As shown in fig. 2, the template matching method of the image may include the steps of:
step S201, for any region to be matched in the target image, determining a registration region corresponding to the region to be matched in the template image, obtaining first target information composed of pixel information of the region to be matched and its neighborhood, and forming second target information composed of pixel information of the registration region and its neighborhood.
In the application, the target image may be an image acquired by using a certain image acquisition device, and in a precise industrial scene, the target image may be an image acquired aiming at details of a workpiece and a part, so that characteristics of the workpiece and the like can be determined.
The region to be matched can be a region defined by a user according to requirements, specifically, a bounding region can be defined from an image by adopting a bounding box with a preset size, the bounding region can be used as the region to be matched, in general, all regions in the whole image can be required to be matched, at this time, the whole image can be divided into different small regions, and the region to be matched is any small region in the whole image.
The template image is a preset image meeting the requirement of a user, and can be an image with the same attribute as the acquisition range of the target image, such as the size of the acquisition range, or an image with the acquisition range smaller than the acquisition range of the target image. If the template image and the target image are images with the same attribute, the position of the registration area in the template image and the position of the area to be matched in the target image are the same, and of course, if the acquisition range of the template image is smaller than the acquisition range of the target image, the shape registration and other operations can be performed on the target image and the template image, so that after the target image and the template image are registered, the registration area is determined in the template image according to the position of the area to be matched in the target image.
The field of the region to be matched can refer to a region adjacent to the region to be matched, specifically can be a 4 neighborhood or an 8 neighborhood, the size of the neighborhood is the same as that of the corresponding surrounding region, and the neighborhood of the registration region is selected in the same manner as that of the region to be matched, namely the region to be matched and the neighborhood thereof are registered with the registration region and the neighborhood thereof. The target information may be pixel information of an image such as a pixel value, a luminance value, and a gradation value.
Optionally, determining the registration area corresponding to the area to be matched in the template image includes:
and performing shape registration on the template image and the target image by using singular value decomposition and principal component extraction, and determining a registration area corresponding to the area to be matched from the registered template image.
Wherein the registration adopts the following formula:
wherein the formula is a singular value decomposition process, the matrix A of the image is decomposed into U, E, V matrix representations, m and n respectively represent the number of rows and the number of columns of the image, and the columns of U form a set of basis vectors for orthogonal input of A, namely AA T The columns of V form a set of basis vectors for the quadrature outputs of A, i.e. A T The eigenvectors of A, the elements on the diagonal of E being singular values, can be considered as scalar expansion control between input and output, i.e., AA T A is a T The singular value of A corresponds to the column vector of U and V, k is a preset approximate value, and is considered in an approximate mode
The characteristic information of the first 90% is retained for E, and the percentage can be changed according to the use scene. And (3) downsampling by using U and V, wherein only principal component information is reserved, and the sampling standard is maximum sampling, namely only the maximum value of a sampling interval is reserved.
The target image and the template image are subjected to shape registration by using singular value decomposition and principal component extraction, and restored according to a fixed size when the feature map is reconstructed by using a decomposition matrix, wherein the fixed size is smaller than that of the template map and the target map, for example, a size of 64 x 32 can be adopted in a wafer detection scene.
Step S202, determining first gray scale characteristic information and first structural characteristic information on a region to be matched and a neighborhood thereof according to first target information, and determining second gray scale characteristic information and second structural characteristic information on a registration region and the neighborhood thereof according to second target information.
In this application, the gray characteristic information may refer to an image gray data value representing a distribution of gray, contrast, etc., for example, a gray value, a gray variance, a gray mean value, a brightness contrast value, etc. The structural feature information may refer to data values characterizing morphological distributions such as texture in an image, e.g., local binary pattern (Local Binary Patterns, LBP) encoded values, for describing the texture of the image.
Optionally, determining the first gray scale feature information and the first structural feature information on the region to be matched and the neighborhood thereof according to the first target information includes:
carrying out mean value calculation, variance calculation, LBP value calculation and LBP coding calculation on the first target information, and determining a first mean value, a first variance, a first LBP value and a first LBP coding value of a region to be matched and a neighborhood thereof;
taking the first mean value, the first variance and the first LBP value as first gray scale characteristic information;
determining pixel difference values of the region to be matched and the neighborhood thereof according to the first target information, and normalizing all the pixel difference values to obtain a normalization value corresponding to each neighborhood;
multiplying the normalized value of each neighborhood with the first LBP coding value of the corresponding neighborhood to obtain a first feature vector representing the corresponding neighborhood;
and adding the first feature vectors of all the neighborhoods to obtain a first addition result, calculating first included angles of all the first feature vectors and the first addition result, and taking the calculation result of the first included angle of each neighborhood as first structural feature information.
The mean value calculation, variance calculation, LBP value calculation, and LPP value encoding calculation may be calculated based on pixel values of the image, or may be calculated based on gray values.
Taking pixel values as an example, the mean value and the variance of the region to be matched are calculated according to the pixel values of all the pixel points in the region to be matched, and the mean value and the variance of the neighborhood are calculated according to the pixel values of all the pixel points in the corresponding neighborhood.
The LBP value of the region to be matched may refer to the LBP value of each pixel in the region to be matched, and then the average value is calculated, where the average value is the LBP value of the region to be matched, the LBP value of the neighborhood may refer to the LBP value of each pixel in the corresponding neighborhood, and then the average value is calculated, where the average value is the LBP value of the neighborhood, and the LBP value of one pixel is calculated based on the pixel values of the surrounding pixels. If the region to be matched is only one pixel, the corresponding neighborhood is also one pixel, and the LBP value of the neighborhood needs to be calculated through the neighborhood of the corresponding neighborhood.
The region to be matched and the first mean value, the first variance and the first LBP value of the neighborhood form mean distribution, variance distribution and LBP distribution, and the distribution is the first gray characteristic information representing the gray characteristic.
If the region to be matched comprises at least two pixel points, the neighborhood also comprises at least two pixel points, the average value of all the pixel points can be adopted when the pixel difference value is calculated, the LBP coding is carried out according to the average value of all the pixel points when the LBP coding value is calculated, if the region to be matched comprises one pixel point, the neighborhood comprises one pixel point, and the pixel difference value calculation and the LBP coding can be directly carried out by using the pixel value of the corresponding pixel point.
Since the LBP code value is 0 or 1, the purpose of normalization is to normalize the information characterizing the structural feature for subsequent weighted summation with the gray scale feature.
The first feature vector is a vector representing a corresponding neighborhood, the magnitude of the vector is expressed by the multiplied product, the vector direction points to the center of the corresponding neighborhood from the center of the region to be matched, and the first addition result is a main vector representing the structural change trend of the region to be matched, namely, the first feature vector is obtained by vector addition.
And calculating a first included angle between all the first feature vectors and the first addition result, namely calculating an included angle between the two vectors, so that each neighborhood corresponds to an included angle value, and the difference between each neighborhood and the structural change trend can be represented.
Optionally, determining the second gray scale feature information and the second structural feature information on the registration area and the neighborhood thereof according to the second target information includes:
performing mean value calculation, variance calculation, LBP value calculation and LBP coding calculation on the second target information, and determining a second mean value, a second variance, a second LBP value and a second LBP coding value of the registration area and the neighborhood thereof;
taking the second mean value, the second variance and the second LBP value as second gray characteristic information;
Determining pixel difference values of the registration area and the neighborhood thereof according to the second target information, and normalizing all the pixel difference values to obtain a normalization value corresponding to each neighborhood;
multiplying the normalized value of each neighborhood with the second LBP coding value of the corresponding neighborhood to obtain a second feature vector representing the corresponding neighborhood;
and adding the second feature vectors of all the neighborhoods to obtain a second addition result, calculating second included angles of all the second feature vectors and the second addition result, and taking the calculation result of the second included angle of each neighborhood as second structural feature information.
In one embodiment, after the gray feature information and the structural feature information of the template image are obtained, the gray feature information and the structural feature information obtained by the calculation may be directly used in the subsequent use of the template image.
Step S203, comparing the first gray scale characteristic information with the second gray scale characteristic information to obtain gray scale difference information, and comparing the first structural characteristic information with the second structural characteristic information to obtain structural difference information.
In the method, gray characteristic information and structural characteristic information of the target image and the template image are compared to obtain differences, and the differences are used for calculating the subsequent matching degree or similarity.
Optionally, comparing the first gray scale feature information with the second gray scale feature information to obtain gray scale difference information includes:
calculating a first KL divergence of the region to be matched and the registration region according to the first mean value and the second mean value;
calculating a second KL divergence of the region to be matched and the registration region according to the first variance and the second variance;
and calculating third KL divergences of the region to be matched and the registration region according to the first LBP value and the second LBP value, and determining the first KL divergences, the second KL divergences and the third KL divergences as gray difference information.
And calculating KL divergences of the target image and the template image respectively according to the mean value dimension, the variance dimension and the LBP value dimension, and using the KL divergences to represent the gray level difference. The value intervals of the KL divergence are 0 and 1, and the larger the corresponding value is, the more matching is indicated.
Optionally, comparing the first structural feature information with the second structural feature information to obtain structural difference information includes:
comparing the calculation result of the first included angle of each neighborhood in the first structural feature information with the second included angle of the corresponding neighborhood in the second structural feature information to determine the difference value of the included angles of each neighborhood;
and calculating a corresponding difference mean value and a difference variance according to the included angle difference value of each neighborhood, normalizing the half-width of normal distribution formed by the difference mean value and the difference variance, and determining the normalized value of the half-width as the structural difference information.
And calculating the difference value between the first included angle of the neighborhood of the region to be matched and the second included angle of the neighborhood corresponding to the registration region to obtain the difference value of the included angles of each neighborhood, and calculating the mean value and the variance of the difference values of all the included angles. And forming normal distribution by the mean value and the variance to obtain half-width of the normal distribution, wherein the half-width is divided by 180 to normalize the half-width because the angle is 0-180 degrees.
And S204, carrying out weighted summation on the gray level difference information and the structure difference information to obtain a weighted summation result, and determining that the target image is matched with the template image in the region to be matched when the weighted summation result meets the preset condition.
In the present application, the weights of the two pieces of information in the weighted summation may be set according to the requirements or the test results. The preset condition may be that a threshold value or a threshold value range is set, when the weighted sum result is greater than the threshold value or belongs to the threshold value range, the target image and the template image are determined to be matched, otherwise, the target image and the template image are not matched.
Optionally, the weighted summation of the gray scale difference information and the structure difference information is performed, and obtaining a weighted summation result includes:
averaging the first KL divergence, the second KL divergence and the third KL divergence to obtain a divergence average value;
And carrying out weighted summation on the divergence mean value and the normalized value of the half-width to obtain a weighted summation result.
And carrying out weighted summation on the average value of the three KL divergences and the normalized value of the half-width so that the weighted summation result is controlled between [0,1], wherein the larger the value of the weighted summation result is, the higher the matching degree is. The weight of both is 0.5.
In an embodiment, when the accuracy requirement is reduced, only one of the three KL divergences may be used to perform weighted summation with the normalized value of the half-width.
According to any region to be matched in the target image, a registration region corresponding to the region to be matched is determined in the template image, first target information formed by pixel information of the region to be matched and the neighborhood thereof is obtained, and second target information formed by pixel information of the registration region and the neighborhood thereof is obtained.
Fig. 3 shows a block diagram of a template matching device of an image according to a third embodiment of the present application, where the template matching device is applied to a server in fig. 1, and a computer device corresponding to the server is connected to a corresponding database to obtain a corresponding target image and a template image in the database, and of course, the template image may also be directly stored in the server. The computer equipment can be connected with corresponding clients, the clients are operated by users, and the users can acquire corresponding target images through the clients and provide the corresponding target images to the server. For convenience of explanation, only portions relevant to the embodiments of the present application are shown.
Referring to fig. 3, the template matching apparatus includes:
the information obtaining module 31 is configured to determine, for any region to be matched in the target image, a registration region corresponding to the region to be matched in the template image, obtain first target information formed by pixel information of the region to be matched and a neighborhood thereof, and obtain second target information formed by pixel information of the registration region and the neighborhood thereof;
the feature determining module 32 is configured to determine, according to the first target information, first gray scale feature information and first structural feature information on the region to be matched and the neighborhood thereof, and determine, according to the second target information, second gray scale feature information and second structural feature information on the registration region and the neighborhood thereof;
A difference comparison module 33, configured to compare the first gray scale feature information and the second gray scale feature information to obtain gray scale difference information, and compare the first structural feature information and the second structural feature information to obtain structural difference information;
the matching result determining module 34 is configured to perform weighted summation on the gray scale difference information and the structure difference information to obtain a weighted summation result, and determine that the target image matches the template image in the region to be matched when the weighted summation result meets a preset condition.
Optionally, the feature determining module 32 includes:
the first calculating unit is used for carrying out mean value calculation, variance calculation, LBP value calculation and LBP coding calculation on the first target information, and determining a first mean value, a first variance, a first LBP value and a first LBP coding value of the region to be matched and the neighborhood thereof;
a first gray feature determining unit configured to take the first mean, the first variance, and the first LBP value as first gray feature information;
the first difference normalization unit is used for determining pixel differences between the region to be matched and the neighborhood thereof according to first target information, and normalizing all the pixel differences to obtain normalization values corresponding to each neighborhood;
the first feature vector determining unit is used for multiplying the normalized value of each neighborhood with the first LBP coding value of the corresponding neighborhood to obtain a first feature vector representing the corresponding neighborhood;
The first structural feature determining unit is used for adding the first feature vectors of all the neighborhoods to obtain a first addition result, calculating first included angles of all the first feature vectors and the first addition result, and taking the calculation result of the first included angle of each neighborhood as first structural feature information.
Optionally, the feature determining module 32 includes:
the second calculating unit is used for carrying out mean value calculation, variance calculation, LBP value calculation and LBP coding calculation on the second target information and determining a second mean value, a second variance, a second LBP value and a second LBP coding value of the registration area and the neighborhood thereof;
a second gray feature determining unit, configured to use the second mean, the second variance, and the second LBP value as second gray feature information;
the second difference normalization unit is used for determining pixel differences between the registration area and the neighborhood thereof according to second target information, and normalizing all the pixel differences to obtain a normalization value corresponding to each neighborhood;
the second feature vector determining unit is used for multiplying the normalized value of each neighborhood with the second LBP coding value of the corresponding neighborhood to obtain a second feature vector representing the corresponding neighborhood;
and the second structural feature determining unit is used for adding the second feature vectors of all the neighborhoods to obtain a second addition result, calculating second included angles of all the second feature vectors and the second addition result, and taking the calculation result of the second included angle of each neighborhood as second structural feature information.
Optionally, the difference comparison module 33 includes:
the first divergence calculating unit is used for calculating a first KL divergence of the region to be matched and the registration region according to the first mean value and the second mean value;
a second divergence calculating unit, configured to calculate a second KL divergence of the region to be matched and the registration region according to the first variance and the second variance;
and the third divergence calculating unit is used for calculating the third KL divergence of the region to be matched and the registration region according to the first LBP value and the second LBP value, and determining the first KL divergence, the second KL divergence and the third KL divergence as gray difference information.
Optionally, the difference comparison module 33 includes:
the included angle difference value determining unit is used for comparing the calculation result of the first included angle of each neighborhood in the first structural feature information with the second included angle of the corresponding neighborhood in the second structural feature information to determine the included angle difference value of each neighborhood;
the structure difference determining unit is used for calculating a corresponding difference mean value and a difference variance according to the included angle difference value of each neighborhood, normalizing the half-width of normal distribution formed by the difference mean value and the difference variance, and determining the normalized value of the half-width as the structure difference information.
Optionally, the matching result determining module 34 includes:
The divergence average value calculation unit is used for averaging the first KL divergence, the second KL divergence and the third KL divergence to obtain a divergence average value;
and the matching value calculation unit is used for carrying out weighted summation on the divergence mean value and the half-width normalized value to obtain a weighted summation result.
Optionally, the information obtaining module 31 includes:
and the region registration unit is used for carrying out shape registration on the template image and the target image by using singular value decomposition and principal component extraction, and determining a registration region corresponding to the region to be matched from the registered template image.
It should be noted that, because the content of information interaction and execution process between the modules is based on the same concept as the method embodiment of the present application, specific functions and technical effects thereof may be referred to in the method embodiment section, and details are not repeated herein.
Fig. 4 is a schematic structural diagram of a computer device according to a fourth embodiment of the present application. As shown in fig. 4, the computer device of this embodiment includes: at least one processor (only one shown in fig. 4), a memory, and a computer program stored in the memory and executable on the at least one processor, the processor, when executing the computer program, performing the steps of the template matching method embodiments of any of the respective images described above.
The computer device may include, but is not limited to, a processor, a memory. It will be appreciated by those skilled in the art that fig. 4 is merely an example of a computer device and is not intended to limit the computer device, and that a computer device may include more or fewer components than shown, or may combine certain components, or different components, such as may also include a network interface, a display screen, an input device, and the like.
The processor may be a CPU, but may also be other general purpose processors, digital signal processors (Digital Signal Processor, DSP), application specific integrated circuits (Application Specific Integrated Circuit, ASIC), off-the-shelf programmable gate arrays (Field-Programmable Gate Array, FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, or the like. A general purpose processor may be a microprocessor or the processor may be any conventional processor or the like.
The memory includes a readable storage medium, an internal memory, etc., where the internal memory may be the memory of the computer device, the internal memory providing an environment for the execution of an operating system and computer-readable instructions in the readable storage medium. The readable storage medium may be a hard disk of a computer device, and in other embodiments may be an external storage device of the computer device, for example, a plug-in hard disk, a Smart Media Card (SMC), a Secure Digital (SD) Card, a Flash memory Card (Flash Card), etc. that are provided on the computer device. Further, the memory may also include both internal storage units and external storage devices of the computer device. The memory is used to store an operating system, application programs, boot loader (BootLoader), data, and other programs such as program codes of computer programs, and the like. The memory may also be used to temporarily store data that has been output or is to be output.
It will be apparent to those skilled in the art that, for convenience and brevity of description, only the above-described division of the functional units and modules is illustrated, and in practical application, the above-described functional distribution may be performed by different functional units and modules according to needs, i.e. the internal structure of the apparatus is divided into different functional units or modules to perform all or part of the above-described functions. The functional units and modules in the embodiment may be integrated in one processing unit, or each unit may exist alone physically, or two or more units may be integrated in one unit, where the integrated units may be implemented in a form of hardware or a form of a software functional unit. In addition, specific names of the functional units and modules are only for convenience of distinguishing from each other, and are not used for limiting the protection scope of the present application. The specific working process of the units and modules in the above device may refer to the corresponding process in the foregoing method embodiment, which is not described herein again. The integrated units, if implemented in the form of software functional units and sold or used as stand-alone products, may be stored in a computer readable storage medium. Based on such understanding, the present application implements all or part of the flow of the method of the above-described embodiments, and may be implemented by a computer program to instruct related hardware, where the computer program may be stored in a computer readable storage medium, where the computer program, when executed by a processor, may implement the steps of the method embodiments described above. Wherein the computer program comprises computer program code, which may be in the form of source code, object code, executable files or in some intermediate form, etc. The computer readable medium may include at least: any entity or device capable of carrying computer program code, a recording medium, a computer Memory, a Read-Only Memory (ROM), a random access Memory (Random Access Memory, RAM), an electrical carrier signal, a telecommunications signal, and a software distribution medium. Such as a U-disk, removable hard disk, magnetic or optical disk, etc. In some jurisdictions, computer readable media may not be electrical carrier signals and telecommunications signals in accordance with legislation and patent practice.
The present application implementing all or part of the flow of the method of the above embodiment may also be implemented by a computer program product, which when run on a computer device causes the computer device to execute the steps of the method embodiment described above.
In the foregoing embodiments, the descriptions of the embodiments are emphasized, and in part, not described or illustrated in any particular embodiment, reference is made to the related descriptions of other embodiments.
Those of ordinary skill in the art will appreciate that the various illustrative elements and algorithm steps described in connection with the embodiments disclosed herein may be implemented as electronic hardware, or combinations of computer software and electronic hardware. Whether such functionality is implemented as hardware or software depends upon the particular application and design constraints imposed on the solution. Skilled artisans may implement the described functionality in varying ways for each particular application, but such implementation decisions should not be interpreted as causing a departure from the scope of the present application.
In the embodiments provided in this application, it should be understood that the disclosed apparatus/computer device and method may be implemented in other ways. For example, the apparatus/computer device embodiments described above are merely illustrative, e.g., the division of modules or units is merely a logical functional division, and there may be additional divisions when actually implemented, e.g., multiple units or components may be combined or integrated into another system, or some features may be omitted or not performed. Alternatively, the coupling or direct coupling or communication connection shown or discussed may be an indirect coupling or communication connection via interfaces, devices or units, which may be in electrical, mechanical or other forms.
The units described as separate units may or may not be physically separate, and units shown as units may or may not be physical units, may be located in one place, or may be distributed over a plurality of network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.
The above embodiments are only for illustrating the technical solution of the present application, and are not limiting thereof; although the present application has been described in detail with reference to the foregoing embodiments, it should be understood by those of ordinary skill in the art that: the technical scheme described in the foregoing embodiments can be modified or some technical features thereof can be replaced by equivalents; such modifications and substitutions do not depart from the spirit and scope of the technical solutions of the embodiments of the present application, and are intended to be included in the scope of the present application.
Claims (9)
1. A template matching method for an image, the template matching method comprising:
determining a registration area corresponding to the area to be matched in a template image aiming at any area to be matched in a target image, and acquiring first target information formed by pixel information of the area to be matched and a neighborhood thereof and second target information formed by pixel information of the registration area and the neighborhood thereof;
Performing mean value calculation, variance calculation, LBP value calculation and LBP coding calculation on the first target information, and determining a first mean value, a first variance, a first LBP value and a first LBP coding value of the region to be matched and the neighborhood thereof;
taking the first mean value, the first variance and a first LBP value as first gray scale characteristic information;
determining pixel difference values of the region to be matched and the neighborhood thereof according to the first target information, and normalizing all the pixel difference values to obtain a normalization value corresponding to each neighborhood;
multiplying the normalized value of each neighborhood with the first LBP coding value of the corresponding neighborhood to obtain a first feature vector representing the corresponding neighborhood;
adding the first feature vectors of all the neighborhoods to obtain a first addition result, calculating first included angles of all the first feature vectors and the first addition result, and taking the calculation result of the first included angle of each neighborhood as first structural feature information;
determining second gray scale feature information and second structural feature information on the registration area and the neighborhood thereof according to the second target information;
comparing the first gray scale characteristic information with the second gray scale characteristic information to obtain gray scale difference information, and comparing the first structural characteristic information with the second structural characteristic information to obtain structural difference information;
And carrying out weighted summation on the gray level difference information and the structure difference information to obtain a weighted summation result, and determining that the target image is matched with the template image in the region to be matched when the weighted summation result meets a preset condition.
2. The template matching method according to claim 1, wherein determining second gray scale feature information and second structural feature information on the registration area and its neighborhood from the second target information comprises:
performing mean value calculation, variance calculation, LBP value calculation and LBP coding calculation on the second target information, and determining a second mean value, a second variance, a second LBP value and a second LBP coding value of the registration area and the neighborhood thereof;
taking the second mean value, the second variance and a second LBP value as second gray characteristic information;
determining pixel difference values of the registration area and the neighborhood thereof according to the second target information, and normalizing all the pixel difference values to obtain a normalization value corresponding to each neighborhood;
multiplying the normalized value of each neighborhood with the second LBP coding value of the corresponding neighborhood to obtain a second feature vector representing the corresponding neighborhood;
and adding the second feature vectors of all the neighborhoods to obtain a second addition result, calculating second included angles of all the second feature vectors and the second addition result, and taking the calculation result of the second included angle of each neighborhood as second structural feature information.
3. The template matching method according to claim 2, wherein comparing the first gray scale feature information and the second gray scale feature information to obtain gray scale difference information comprises:
calculating a first KL divergence of the region to be matched and the registration region according to the first mean value and the second mean value;
calculating a second KL divergence of the region to be matched and the registration region according to the first variance and the second variance;
and calculating third KL divergences of the region to be matched and the registration region according to the first LBP value and the second LBP value, and determining the first KL divergences, the second KL divergences and the third KL divergences as gray difference information.
4. The template matching method according to claim 3, wherein comparing the first structural feature information and the second structural feature information to obtain structural difference information includes:
comparing the calculation result of the first included angle of each neighborhood in the first structural feature information with the second included angle of the corresponding neighborhood in the second structural feature information to determine an included angle difference value of each neighborhood;
and calculating a corresponding difference mean value and a difference variance according to the included angle difference value of each neighborhood, normalizing the half-width of normal distribution formed by the difference mean value and the difference variance, and determining the normalized value of the half-width as structural difference information.
5. The template matching method according to claim 4, wherein performing weighted summation on the gray scale difference information and the structural difference information to obtain a weighted summation result includes:
averaging the first KL divergence, the second KL divergence and the third KL divergence to obtain a divergence mean value;
and carrying out weighted summation on the divergence mean value and the normalized value of the half-width to obtain a weighted summation result.
6. The template matching method according to any one of claims 1 to 5, wherein determining a registration area corresponding to the area to be matched in a template image comprises:
and performing shape registration on the template image and the target image by using singular value decomposition and principal component extraction, and determining a registration area corresponding to the area to be matched from the registered template image.
7. A template matching apparatus for an image, the template matching apparatus comprising:
the information acquisition module is used for determining a registration area corresponding to the area to be matched in a template image aiming at any area to be matched in the target image, acquiring first target information formed by pixel information of the area to be matched and a neighborhood thereof, and forming second target information by pixel information of the registration area and the neighborhood thereof;
The feature determining module comprises a first calculating unit, a second calculating unit and a third calculating unit, wherein the first calculating unit is used for carrying out mean value calculation, variance calculation, LBP value calculation and LBP coding calculation on the first target information and determining a first mean value, a first variance, a first LBP value and a first LBP coding value of the region to be matched and the neighborhood of the region to be matched;
a first gray feature determining unit, configured to use the first mean, the first variance, and a first LBP value as first gray feature information;
the first difference normalization unit is used for determining pixel differences between the region to be matched and the neighborhood of the region to be matched according to the first target information, and normalizing all the pixel differences to obtain a normalization value corresponding to each neighborhood;
the first feature vector determining unit is used for multiplying the normalized value of each neighborhood with the first LBP coding value of the corresponding neighborhood to obtain a first feature vector representing the corresponding neighborhood; and
the first structural feature determining unit is used for adding the first feature vectors of all the neighborhoods to obtain a first addition result, calculating first included angles of all the first feature vectors and the first addition result, and taking the calculation result of the first included angle of each neighborhood as first structural feature information;
The feature determining module is used for determining second gray feature information and second structural feature information on the registration area and the neighborhood thereof according to the second target information;
the difference comparison module is used for comparing the first gray scale characteristic information with the second gray scale characteristic information to obtain gray scale difference information, and comparing the first structural characteristic information with the second structural characteristic information to obtain structural difference information;
and the matching result determining module is used for carrying out weighted summation on the gray level difference information and the structure difference information to obtain a weighted summation result, and determining that the target image is matched with the template image in the region to be matched when the weighted summation result meets the preset condition.
8. A computer device, characterized in that it comprises a processor, a memory and a computer program stored in the memory and executable on the processor, which processor implements the template matching method according to any of claims 1 to 6 when executing the computer program.
9. A computer readable storage medium storing a computer program, wherein the computer program when executed by a processor implements the template matching method according to any one of claims 1 to 6.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202211374039.0A CN115423855B (en) | 2022-11-04 | 2022-11-04 | Template matching method, device, equipment and medium for image |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202211374039.0A CN115423855B (en) | 2022-11-04 | 2022-11-04 | Template matching method, device, equipment and medium for image |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN115423855A CN115423855A (en) | 2022-12-02 |
| CN115423855B true CN115423855B (en) | 2023-05-02 |
Family
ID=84207304
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202211374039.0A Active CN115423855B (en) | 2022-11-04 | 2022-11-04 | Template matching method, device, equipment and medium for image |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN115423855B (en) |
Family Cites Families (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101727452B (en) * | 2008-10-22 | 2012-09-19 | 富士通株式会社 | Image processing method and device |
| US9275300B2 (en) * | 2012-02-24 | 2016-03-01 | Canon Kabushiki Kaisha | Method and apparatus for generating image description vector, image detection method and apparatus |
| CN109325453B (en) * | 2018-09-27 | 2022-03-04 | 沈阳理工大学 | Template matching tracking method for moving target |
| CN113096169B (en) * | 2021-03-31 | 2022-05-20 | 华中科技大学 | Non-rigid multimode medical image registration model establishing method and application thereof |
| CN114708313A (en) * | 2022-03-24 | 2022-07-05 | 安徽大学 | Optical and SAR image registration method based on double-branch neural network |
| CN114972817A (en) * | 2022-04-25 | 2022-08-30 | 深圳创维－Rgb电子有限公司 | Image similarity matching method, device and storage medium |
-
2022
- 2022-11-04 CN CN202211374039.0A patent/CN115423855B/en active Active
Also Published As
| Publication number | Publication date |
|---|---|
| CN115423855A (en) | 2022-12-02 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US20140219569A1 (en) | Image recognition system and method for identifying similarities in different images | |
| CN112651953B (en) | Picture similarity calculation method and device, computer equipment and storage medium | |
| CN111882565B (en) | Image binarization method, device, equipment and storage medium | |
| CN112149583A (en) | Smoke detection method, terminal device and storage medium | |
| CN117496560B (en) | Fingerprint line identification method and device based on multidimensional vector | |
| US10210414B2 (en) | Object detection system and computer program product | |
| CN112435283A (en) | Image registration method, electronic device and computer-readable storage medium | |
| CN115423855B (en) | Template matching method, device, equipment and medium for image | |
| CN109871779B (en) | Palm print identification method and electronic equipment | |
| CN115546271B (en) | Visual analysis method, device, equipment and medium based on depth joint characterization | |
| CN116977671A (en) | Target tracking method, device, equipment and storage medium based on image space positioning | |
| CN111222446A (en) | Face recognition method, face recognition device and mobile terminal | |
| CN108629219B (en) | Method and device for identifying one-dimensional code | |
| CN116030280A (en) | Template matching method, device, storage medium and equipment | |
| CN115798004A (en) | Face card punching method and device based on local area, electronic equipment and medium | |
| US20160379087A1 (en) | Method for determining a similarity value between a first image and a second image | |
| CN113239738B (en) | Image blurring detection method and blurring detection device | |
| CN112200004B (en) | Training method and device for image detection model and terminal equipment | |
| CN112084874B (en) | Object detection method and device and terminal equipment | |
| CN114463764A (en) | Table line detection method and device, computer equipment and storage medium | |
| Ziomek et al. | Evaluation of interest point detectors in presence of noise | |
| CN111429399A (en) | Straight line detection method and device | |
| WO2023281811A1 (en) | Hash generation device, hash determination device and system | |
| CN113658101B (en) | Method and device for detecting landmark points in image, terminal equipment and storage medium | |
| CN111145230B (en) | Image registration quality detection method, device, equipment and storage medium |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| CB02 | Change of applicant information | ||
| CB02 | Change of applicant information |
Address after: Room 425, Block C, Bao'an New Generation Information Technology Industrial Park, No. 3, North Second Lane, Chuangye Second Road, 28 Dalang Community, Xin'an Street, Bao'an District, Shenzhen, Guangdong 518000 Applicant after: Shenzhen Yibi Technology Co.,Ltd. Address before: 518000 406, block C, Bao'an new generation information technology industrial park, No. 3, North 2nd Lane, Chuangye 2nd Road, Dalang community, Xin'an street, Bao'an District, Shenzhen, Guangdong Province Applicant before: Shenzhen Yibi Technology Co.,Ltd. |
|
| GR01 | Patent grant | ||
| GR01 | Patent grant | ||
| CP02 | Change in the address of a patent holder | ||
| CP02 | Change in the address of a patent holder |
Address after: 518000, 1st Floor, Building B5, Taohuayuan Science and Technology Innovation Ecological Park, Tiegang Community, Xixiang Street, Bao'an District, Shenzhen City, Guangdong Province Patentee after: Shenzhen Yibi Technology Co.,Ltd. Address before: Room 425, Block C, Bao'an New Generation Information Technology Industrial Park, No. 3, North Second Lane, Chuangye Second Road, 28 Dalang Community, Xin'an Street, Bao'an District, Shenzhen, Guangdong 518000 Patentee before: Shenzhen Yibi Technology Co.,Ltd. |